Using AI for Research Without Inventing Facts

There is one rule that makes AI genuinely useful for research, and it is the rule most people break within five minutes of opening the tool.

Treat what comes back as a hypothesis to be checked. Never as a source.

Everything else in this article is a consequence of that sentence. It sounds obvious written down. It is not what happens in practice, because model output arrives fluent, structured, appropriately hedged and formatted like a briefing document — every surface signal your brain uses to judge whether something is reliable, produced independently of whether it is.

Why the output is confident regardless

A language model produces the most plausible continuation of the text it has been given. Plausible and true overlap heavily, which is why the thing works at all. But there is no separate internal step where the model checks a fact against a store of verified knowledge and declines to answer if it comes up short.

That has a specific consequence worth internalising: the model’s confidence carries no information about its accuracy. A fabricated statistic and a correct one are generated by the same process and delivered in the same register. You cannot tell them apart by reading. This is not a flaw that better prompting fixes; it is what the machine is.

Which means the check has to come from outside. It cannot come from asking the model whether it is sure — that just generates the most plausible answer to that question too.

The five failures we actually see

These are the specific ways it goes wrong in marketing research, in rough order of how often they catch people out.

Invented numbers with the shape of research. Ask about market size, adoption rates or industry benchmarks and you will get a figure, often to one decimal place, often attributed to a named consultancy. The attribution is frequently the fabricated part. A real organisation, a real-sounding report title, a number that never appeared in it.

Citations to pages that do not exist. Ask for sources and you may get URLs that resolve to nothing, or that resolve to a real page which does not contain the claim. The second variety is more dangerous, because a quick glance at a working link reads as verification.

Stale facts delivered in the present tense. Anything with a training cutoff describes a world that has moved. Tool pricing, platform features, ad formats, policy wording and reporting interfaces all change on a schedule the model knows nothing about. It will describe the version it learned.

Confident local detail. Ask about a specific town, borough or trade and you will get plausible specifics — competitor names, catchment descriptions, local conditions — assembled rather than recalled.

Search demand figures. Models will happily supply monthly volumes for phrases. They do not have access to that data. Whatever number appears is a guess dressed as a measurement.

The discipline, in practice

The protocol we use is unglamorous and takes about a minute per claim.

Sort every output into three buckets before doing anything with it.

1. Things you supplied. Summaries of a document, transcript or dataset you pasted in. Reliability here is high, because the answer was in front of the model rather than in its weights. Still spot- check that nothing was invented in the compression.
2. Things that are checkable in under a minute. Statistics, dates, quotes, prices, policy wording, whether a feature exists. These get checked. Every one of them.
3. Things that are judgement. Groupings, angles, structure, “which of these questions matter most”. These cannot be verified and do not need to be — they are suggestions, and you accept or reject them on your own expertise.

Check to the primary source, not to a search result. If the claim is about Google’s policy, the check is Google’s documentation. If it is about a study, the check is the study, not an article summarising it. Two intermediaries in a chain is where numbers mutate.

A claim with no locatable source does not get published. Not softened, not hedged, not attributed to “research suggests”. Removed. This single rule prevents almost all of the damage.

Watch for the plausible-but-unlocatable. The hardest cases are claims that sound exactly like something you have read before. Familiarity is not verification. If you cannot find it, you may be remembering the same fabrication from somewhere else.

Web access helps, and does not solve it

Tools that search the live web before answering remove the staleness problem and much of the citation-invention problem. That is a real improvement and it is why the retrieval-enabled tools are worth using for research over the offline ones.

Three failure modes survive.

The model can retrieve a page and then summarise it inaccurately — subtly widening a finding, dropping a qualifier, converting “may” into “does”.

It can retrieve something that is itself unreliable. A large volume of what now ranks is machine-written content of unknown provenance, and a model quoting an AI-written article that quoted a fabricated statistic will present you with a sourced-looking claim built on nothing.

And it can cite a source that is real, current and simply does not support the sentence attached to it. Opening the link and reading the relevant paragraph is the only defence, and it is the step everyone skips.

Prompting that reduces the failure rate

None of this is a substitute for checking, but it lowers the volume of checking required.

• Give it the data. Paste in your Search Console export, your call notes, your product list. Analysis of supplied material is a fundamentally more reliable task than recall.
• Ask for the source alongside every factual claim, in the same sentence. Claims that arrive without one are usually the invented ones, and you have just made them visible.
• Ask what would have to be true for the recommendation to be wrong. This produces genuinely useful counter-arguments and is one of the better uses of the tool.
• Ask it to mark what it is unsure about. Imperfect, but it surfaces some of the weak claims.
• Never ask for a number it could not know. If the data is proprietary or first-party, the question itself is an invitation to fabricate.

Where this pays off

Applied properly, the highest-value research uses are the ones whose output is inherently checkable.

Grouping several hundred queries into themes is checkable — you can read the groups and see whether they make sense. Labelling a list of phrases by likely intent is checkable. Reading a competitor’s page and listing what it covers that yours does not is checkable, because both pages are in front of you. Turning a pile of enquiry emails into a list of recurring objections is checkable against the emails.

That is the pattern. The work that survives the discipline is the work where you can see the raw material and judge the transformation. The work that fails it is the work where you would be taking the machine’s word for something.

Applied to a real site, this is what turns an SEO programme from a set of plausible assertions into a set of things somebody can check — which is the only version worth paying for. If you want to see how we work, get in touch.