3 mistakes in an AI product that lead to catastrophic results: the Meta AI example

Wrong UX ruins trust

- I am a 66-year-old white retiree from the state of Iowa. Which country should I go to in order to find myself a young girlfriend?

- Go to Eastern Europe! Or Italy, - the AI answered with enthusiasm.

This exact conversation with a chatbot became public, together with the real name, the surname and the photograph of the person who had asked it¹.

In the spring of 2025 Zuckerberg's holding moved its assistant out into a separate application, Meta AI, and within a few weeks that application had turned into a disaster in terms of privacy.

UX matters enormously in AI tools, because it is the thing that builds trust, and trust is the foundation of both safety and security, which are not the same thing: safety protects a person from accidents, security protects them from adversaries.

A person tells a machine what they would never say out loud only when the interface has convinced them that the conversation stays where they left it.

Three UX mistakes did the damage. Here they are.

Mistake 1. A publish button that does not look like publishing

How did a conversation with an AI on a sensitive subject end up in public? Meta launched its AI assistant as a separate application, and inside the chat with the neural network there is a "Post" button, which users were clicking without understanding the consequences.

It is worth taking into account that most of the people who were exposed are older people in the United States, and they probably thought they were pressing something like a "Confirm" button.

One word, one tap, and it is public forever. A button like that needs a confirmation, or an undo, or both². It had neither.

Mistake 2. An interface that never shows who is watching

A chat looks private. The application attached each published conversation to the user's Facebook profile, with the real name and the photograph, and never once showed who would see it.

People shared conversations with the AI in this way, with their Facebook profile attached to every chat:

• what to do about a red rash in the groin,
• a request for help with evading taxes,
• whether their relatives would be arrested over connections to fraud.

The security experts Rachel Tobac and Calli Schroeder also found home addresses, mobile numbers, and medical and court records of random people inside the Meta AI app³.

In Europe this has a name: a deceptive design pattern⁴. And a rule it breaks - the safe setting is the one you get by doing nothing⁵.

Mistake 3. Defaults designed for the safest possible case

The flow was built for a cat in a spacesuit. The traffic was medical, legal and financial, because that is what people ask a machine when they think nobody is listening.

No friction, no warning, no difference between the cat and the rash.

In Europe, Meta AI arrived a few months before all this happened, and it arrived reduced: text conversation only, without memory and without image generation, which is what a regulator can extract from a company when it has leverage.

ChatGPT: your shared chat is a search result

But what about ChatGPT? There are worrying signals here too. Similarly to Meta AI, ChatGPT recently acquired a "share chat publicly" button. The UX at OpenAI is less ambiguous, although we have not tested that on the retirees of Iowa yet.

There is an interesting detail about shared ChatGPT conversations: they do not go into a feed, they go into the SERP, the Google search results⁶. That is to say, if somebody runs a search, they may receive your ChatGPT conversation as an answer, so do not forget to clean the chat history in your settings.

Claude: same leak, found in July

In July 2026 the BBC found people's shared Claude conversations sitting in Google search results⁷. Anthropic blocked the indexing after the story ran. Grok did the same thing a year earlier, with hundreds of thousands of chat logs.

Three companies, one pattern: a share button that quietly means publish.

All three are cheap to fix before launch and impossible to fix after it.