Why does AI give a different answer every time?

AI gives a different answer every time because language models build their text word by word using a degree of probability. On top of that, the model you use, whether search is on or off, your location, your language and the exact wording of your question all play a part. That is why a single question tells you very little: you only get a reliable picture by measuring across several questions and several weeks.

Built-in variation

A language model chooses from several possible words at each step. It usually picks a likely option, but not always the same one. As a result, the same question can produce slightly different sentences, a different order or a different list of businesses.

That is not a bug. It makes answers sound more natural. But it does mean that a bakery in Leiden may top the list one time and land in fourth place, or disappear entirely, the next.

Which model you use

ChatGPT, Gemini, Perplexity and Claude run on different models. Even within one assistant there are often several versions, such as a fast model and a more thorough one. They are trained on different data and make different choices. Providers also update their models regularly, so an answer today may differ from one a few months ago.

Search on or off

Some answers come only from what the model learned during training. Others draw on live web search results. Perplexity almost always searches. With ChatGPT and other assistants, it depends on the question and the settings.

With search on, a new page on your site or a recent review can make a difference fairly quickly. Without search, the model relies on older knowledge. That alone can make two answers very different.

Location and language

Ask for "the best plumber near me" and an assistant may use your location or IP address. Someone in Rotterdam will then see different names than someone in Groningen.

Language matters too. A question in Dutch often draws on different sources than the same question in English. If your business also serves expats or international customers, it is worth checking both languages.

How the question is worded

Small differences in wording can lead to different answers. Compare:

• "Which physiotherapist in Amersfoort is good?"
• "My knee hurts, where can I get help in Amersfoort?"
• "Physiotherapy Amersfoort sports injury"

All three are about the same thing, but an AI assistant may name different practices for each. Earlier messages in the same conversation and any personal settings can also colour the answer.

What this means for measuring

Because answers vary so much, a single screenshot is not worth much. A sound measurement accounts for the variation:

1. Measure several questions your customers really ask, not just one.
2. Use different phrasings for the same topic.
3. Check several AI assistants.
4. Repeat the measurement every week in a consistent way.
5. Look at the trend over weeks, not at a single outlier.

Our article on which questions customers ask AI helps you choose the right ones. For a fuller explanation of measuring, read how to measure whether AI mentions your business.

How we handle this

At HelloGeoLabs we ask the same set of questions to ChatGPT, Gemini, Perplexity and Claude in a consistent way. That lets you compare results from week to week. You see how often you are mentioned, in what position and which competitors appear. The details are on our page about how we measure.

Rather not track all that variation yourself? With GEO tracking you get a clear report in your inbox after every measurement, so you can see whether your visibility is really improving.