Cover graphic for 'AI Hallucinations: Why AI Confidently Makes Things Up': a speech bubble with a fabricated citation, 'Per Smith et al. (2019)...', tagged with a red 'no such paper' cross-badge, beside the headline Trained to guess.
AI Concepts

AI Hallucinations: Why AI Confidently Makes Things Up

In November 2022, Jake Moffatt’s grandmother died. He needed to fly from Vancouver to Toronto for the funeral, and while he was booking the ticket on Air Canada’s website, a chatbot popped up and told him something useful: he could book the flight now, at full fare, and apply for a bereavement discount afterward, within 90 days of travel.

That wasn’t true. Air Canada’s actual bereavement policy required the discount to be requested before the flight, not after, a detail sitting in plain text on a different page of the airline’s own site. When Moffatt applied for the refund, Air Canada pointed him to that real policy and declined. So he took the airline to British Columbia’s Civil Resolution Tribunal.

Air Canada’s defense is the part worth sitting with. The airline argued that the chatbot was, in effect, its own agent, a “separate legal entity” responsible for its own words, and that Moffatt should have trusted the correct static webpage instead of what the airline’s own AI assistant told him directly. The tribunal wasn’t persuaded: in February 2024, it ruled that the chatbot “is still just a part of Air Canada’s website,” and the airline was responsible for everything on it, including the parts that made things up. Air Canada was ordered to pay Moffatt CAD $650.88.

Read that defense again, because it’s a near-perfect crystallization of a much bigger problem. A company wired an AI model into a customer-facing product, the model stated a policy that didn’t exist, and the company’s first instinct wasn’t “we have a bug to fix.” It was “the AI said it, not us.” That instinct is understandable, because a hallucination doesn’t look like a bug in the traditional sense. Nothing crashed. No error was thrown. The chatbot answered fluently, helpfully, and in a completely reasonable tone. It just wasn’t right, and nothing in the system knew that at the moment it said it.

That’s what an AI hallucination is: a false statement generated with exactly the same confidence and fluency as a true one. It isn’t rare, it isn’t a glitch that better engineering quietly fixes, and, this is the part most explainers skip, it isn’t something anyone has actually solved, including the labs spending billions of dollars on the problem. What follows is the real mechanism behind it, what recent research says causes it, where it does the most damage, and the concrete habits that catch it before it costs you $650.88 or considerably more.

Quick answer: An AI hallucination is a false or fabricated statement that a language model generates with the same fluent, confident tone it uses for true statements. It happens because these models are trained to predict plausible-sounding text, not to verify facts, and, according to OpenAI’s own 2025 research, because the standard way the industry grades AI models rewards a confident wrong answer over an honest “I don’t know.” It isn’t a rare glitch. It’s a structural, predictable side effect of how these systems are built and tested.

Here’s what you’ll walk away knowing:

  • Hallucination isn’t one thing. A fabricated legal citation, a phantom software package, and a wrong number buried inside an otherwise accurate paragraph are all the same underlying failure wearing different clothes.
  • New OpenAI research argues hallucination persists mainly because of how AI models get graded, not because the models are unfinished. Guessing scores better than admitting uncertainty, so guessing is what gets reinforced.
  • Bigger and newer isn’t automatically better here. OpenAI’s own o3 and o4-mini models hallucinated more often than their predecessor on a key internal benchmark, not less.
  • Retrieval-augmented generation and web search meaningfully reduce hallucination. Neither eliminates it, and both can fail in ways that still sound completely confident.
  • You don’t need to be technical to catch most hallucinations. A handful of concrete checks catch the majority of what actually causes damage.

What “hallucination” actually means, and where the word came from

A hallucination, in the AI sense, is any output stated as fact that isn’t grounded in anything real: a citation to a court case that was never decided, a statistic from a report that doesn’t exist, a phone number invented because it looked plausible, a summary that includes a detail the source document never mentioned. The defining trait isn’t that it’s wrong. Models are wrong about genuinely uncertain things constantly, which is normal and forgivable. A hallucination is wrong while sounding exactly as certain as everything else the model says.

The term has a strange history worth two sentences. It didn’t originate to describe AI going wrong at all: computer vision researchers used it in the early 2000s in a positive sense, to describe a model plausibly filling in missing detail, like sharpening a blurry photo by inventing convincing extra pixels. The negative meaning came out of machine translation, where a “hallucinated” translation was one that had drifted entirely away from the source text. Google DeepMind researchers picked up the term for language models around 2018, and ChatGPT’s public launch in late 2022 took a piece of internal AI-research jargon and made it a word ordinary people needed to know. It’s also one of the twenty-four terms in our plain-English AI glossary, if jargon like this is still new territory for you.

That history is a useful clue to the mechanism. Hallucination isn’t a foreign contaminant that occasionally leaks into an otherwise fact-based system. It’s the model doing the exact same “fill in something plausible” trick it always does, the trick that makes it good at drafting emails and summarizing meetings, except this time firing on a gap it doesn’t actually have the information to fill.

Why it happens: the model was never trying to be right

To understand hallucination properly, it helps to know what a language model’s actual job is, because it’s narrower than most people assume. We’ve covered the full mechanism elsewhere, but the short version: a large language model doesn’t look facts up. It predicts, one small chunk of text at a time, whatever continuation is statistically most plausible given everything written so far and everything it absorbed during training. That’s the entire mandate. Not “say true things.” Predict likely text.

For most of what you ask an AI model to do, plausible and true point in the same direction, because the internet contains an enormous amount of correct information stated in recognizable, well-worn patterns. Ask about a well-documented historical event and the plausible continuation is also the true one, because true accounts of it appear constantly in the training data. The mechanism accidentally produces accuracy as a side effect of pattern strength.

Hallucination is what happens the moment that alignment breaks. Ask about something obscure, something that happened after training ended, something with only one or two examples buried in the training data, or something that simply never existed — a court case, a software package, a study — and the model doesn’t have a “no data available” state it defaults to. It has no separate module that checks whether an answer is grounded in something real before it’s spoken. It just keeps predicting the next plausible token, and a confident, complete, grammatically flawless paragraph is a very plausible thing for a language model’s output to look like, whether or not anything in it is true.

The real reason it doesn’t go away: how AI models are graded

For a while, the working assumption among a lot of AI researchers was some version of “hallucination is an unfinished-technology problem, and it’ll fade out as models improve.” A September 2025 paper from OpenAI researchers, titled plainly “Why Language Models Hallucinate,” makes a more uncomfortable argument: hallucination isn’t primarily a capability gap. It’s a scoring problem, baked in by how the entire industry evaluates AI models, and it will persist even as models get smarter, unless the way they’re graded changes.

The paper’s central analogy is a multiple-choice exam. Picture a student who genuinely doesn’t know the answer to a question. If leaving it blank scores zero and a wrong guess also scores zero, but a correct guess scores full marks, the rational strategy is obvious: always guess. You can never do worse than blank, and you might do better. Scale that logic up to nearly every benchmark used to compare AI models. Most are graded on simple accuracy, right or wrong, with no separate credit for correctly saying “I’m not sure.” A model that hedges honestly scores worse, on paper, than a model that confidently guesses and gets it wrong exactly as often as chance would predict, because “wrong” and “honestly uncertain” are graded identically: zero.

A comparison diagram showing that under typical AI benchmark scoring, a confident guess earns full credit when right and zero when wrong, while honestly saying "I don't know" always scores zero, so guessing is always the higher-scoring strategy

Under the scoring most AI benchmarks use, a wrong guess costs exactly as much as honesty. That’s the incentive researchers say keeps hallucination alive.

The researchers trace part of the problem to something even more basic, upstream of grading entirely: during the earliest stage of training, a model is just learning to predict the next word across an enormous pile of text, with no reliable way to tell which of the statistical patterns it absorbed were true and which were false. Both were simply “text that occurred.” If a false but confident-sounding pattern is statistically indistinguishable from a true one at that stage, some amount of hallucination is, in the researchers’ framing, a natural consequence of ordinary statistical error, not a defect specific to any one model or company.

Where this gets genuinely useful, rather than just bleak, is the paper’s proposed fix, because it isn’t “train harder” or “add more data.” The authors argue the fix has to happen at the evaluation layer: stop grading benchmarks in a way that rewards confident guessing over honest uncertainty, the way a well-designed exam gives real credit for “I don’t know” over a wrong guess. Until the leaderboards labs compete on actually reward that kind of honesty, there’s no competitive pressure pushing any model toward it. A model trained to say “I’m not sure” more often will simply look worse on the tests everyone is optimizing for, and lose to a model that bluffs.

Not all hallucinations look the same

Talking about “hallucination” as one category makes it sound like a single failure mode. In practice it shows up in a handful of recognizably different shapes, and knowing which one you’re dealing with changes how you’d catch it.

Four cards showing distinct types of AI hallucination: a fabricated fact or figure, an invented citation, a phantom code package, and a quiet error hidden inside otherwise accurate text

Four different failure shapes, one underlying cause: the model filling a gap with something plausible instead of something real.

Fabricated facts and figures. The model states a specific number, date, or claim that simply doesn’t exist anywhere, delivered with the same confidence as a verified statistic. This is the most talked-about kind and the easiest to demonstrate, because it’s often checkably false the moment you look.

Invented citations and sources. Ask a model for supporting sources and it will frequently generate ones that look exactly right — plausible author names, a real-sounding journal, a period-appropriate title — and don’t exist. This is the flavor behind the most infamous hallucination story of the ChatGPT era: two New York lawyers were sanctioned in 2023 for filing a legal brief full of fabricated court cases, complete with docket numbers and quoted judicial reasoning, that the model had generated for them.

Phantom software packages. Ask an AI coding assistant to solve a problem and it will sometimes recommend importing a library that sounds exactly like something that should exist, and doesn’t. Researchers studying this at scale generated more than 2.2 million code samples across sixteen popular AI coding models and found that nearly 1 in 5 contained at least one hallucinated package name: open-source models hallucinated packages at roughly 22% on average, commercial models at roughly 5%. This has turned into an active security threat called “slopsquatting,” where attackers register the exact fake package names AI models tend to invent, load them with malware, and wait for a coding assistant to recommend one to a developer who doesn’t check. One hallucinated package name, huggingface-cli, was downloaded more than 30,000 times in three months before anyone flagged it.

Quiet errors inside otherwise correct text. The most dangerous version, because it’s the hardest to catch: a paragraph that’s 90% accurate, with one wrong detail embedded in the middle — a misquoted figure, a misattributed quote, a summary that adds a claim the source document never actually made. Nothing about the surrounding accurate text flags the one sentence that isn’t.

There’s a fifth, subtler version worth naming separately: confabulated reasoning. Newer models that show their step-by-step thinking before answering don’t always arrive at their answer the way that visible reasoning suggests. Sometimes the written-out reasoning is constructed to justify a conclusion the model had already effectively landed on, rather than actually deriving it. It’s the most rigorous-looking version of the model’s output, careful, sequential, shown rather than just told, which is exactly why it’s worth remembering that a convincing chain of steps is still generated text, not a guarantee that those steps are the real reason the answer came out the way it did.

Where hallucination does real damage

None of this stays theoretical for long. A few fields make the stakes concrete.

Law. Stanford RegLab researchers tested three general-purpose AI models, GPT-3.5, Llama 2, and PaLM 2, against more than 200,000 legal queries in 2024, and found hallucination rates between 69% and 88% depending on the question type, meaning most specific legal questions produced at least one fabricated detail. Even the specialized commercial legal-AI products built specifically to reduce this — Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI — still hallucinated more than 17% to 34% of the time in Stanford’s follow-up testing, despite being built with legal-specific grounding baked in.

Customer service. The Air Canada case from the opening isn’t an outlier; it’s a preview. Any company that puts a customer support chatbot in front of customers and lets it answer policy questions freely is exposed to exactly this failure, and, per the tribunal’s ruling, is legally on the hook for it regardless of what the AI vendor’s terms of service say.

The legal system hasn’t settled on one rule, and won’t apply this evenly. Air Canada lost because a company is responsible for its own product, chatbot included. But when radio host Mark Walters sued OpenAI for defamation after ChatGPT falsely told a journalist he’d embezzled funds from a gun-rights nonprofit, a Georgia court dismissed the case in May 2025. The reasoning: ChatGPT carries visible disclaimers about possible inaccuracy, the journalist who received the false claim recognized it as likely wrong and never published it, and Walters couldn’t show he’d actually been harmed. Two hallucinations, two very different outcomes, decided largely by who acted on the false information and how clearly they’d been warned not to trust it blindly. Expect the legal rules here to keep getting worked out case by case for a while yet.

Software development. Slopsquatting turns a language model’s confident guess into a real supply-chain attack surface, not a hypothetical one. It’s already happening, at the scale of tens of thousands of downloads for a single fake package.

Research and anything you’d cite. Academic work, journalism, research reports, medical information, any context where a specific, checkable fact matters is a context where a hallucinated one does real damage, precisely because it reads exactly like a verified one. This is also why AI research tools that show their sources are worth actively seeking out over ones that just state an answer.

Do bigger, newer models hallucinate less?

You’d assume so, and for a long stretch of AI progress it was even mostly true, hallucination rates trended down release over release. Then, in April 2025, OpenAI published something that broke that pattern in public. According to the official system card for o3 and o4-mini, the company’s own newer, more capable reasoning models both hallucinated more often than their predecessor on a benchmark called PersonQA, which tests factual claims about people. o1 hallucinated on about 16% of questions. o3 hallucinated on 33%, roughly double. o4-mini hallucinated on 48%, essentially a coin flip.

OpenAI’s own explanation, stated plainly in the card, is that these newer models simply make more total claims per answer, more assertions, more specifics, more detail, which produces more correct claims and more incorrect ones side by side. The company was candid that it doesn’t yet fully understand why the effect is so much sharper on this particular benchmark, which is itself telling: this isn’t a solved problem being polished at the margins. It’s an open one, acknowledged by the lab best positioned to know.

The pattern generalizes beyond one company’s benchmark. Vectara’s hallucination leaderboard, an independent, continuously updated benchmark that tests how often different models introduce unsupported claims while summarizing a document, regularly shows a spread from under 2% to well over 20% across current models on the exact same task, with no clean story of “newer always wins.” Model size and release date turn out to be weak predictors of hallucination rate. What a model was specifically trained and evaluated to do is a much better one, which loops right back to the OpenAI paper’s argument: this is substantially a design and incentive problem, not a scale problem, and scale alone won’t fix it.

It’s worth connecting this back to the confabulated reasoning mentioned above, because “reasoning” models are exactly where that failure mode lives. These systems are specifically built to write out a longer chain of intermediate steps before answering, which is precisely how a model ends up making more total claims per response, and more total claims is, mechanically, more surface area for some of them to be wrong. A longer, more detailed chain of visible reasoning can look like more rigor. Per OpenAI’s own numbers above, it has sometimes meant the opposite.

Does RAG fix this?

If you’ve read about retrieval-augmented generation, you might reasonably assume it’s the answer here, and it does help, meaningfully. RAG gives a model real documents to read before it answers, instead of relying purely on what it memorized during training, and grounding an answer in an actual source measurably cuts hallucination rates.

It doesn’t eliminate them. A RAG system can retrieve the wrong document, and the model will still answer confidently from it. It can retrieve the right document, and the model can still misread or misquote a detail from it. It adds a search step, not a truth detector, and the model at the end of the pipeline is still the same next-token predictor described above, just given better material to predict from.

Even a system built specifically to ground its answers in retrieved documents can fail this way in public. In May 2024, Google’s AI Overviews, a retrieval-based search feature, told some users searching for tips on getting cheese to stick to pizza that they could add about an eighth of a cup of non-toxic glue to the sauce. The advice traced back to an eleven-year-old joke comment on Reddit that the retrieval step surfaced as a plausible source, which the model then summarized as straightforward, sarcasm and all. Google pulled the specific answer, and outside tracking firms reported the company quietly scaled back how often AI Overviews appeared in results in the weeks that followed. The lesson isn’t that Google’s engineers were careless. It’s that retrieval only grounds an answer in whatever it happens to retrieve, and the open web contains no shortage of confident, well-written text that isn’t true.

Treat “this tool cites its sources” as a real, meaningful signal of lower risk, not a reason to stop checking altogether.

How to actually catch a hallucination

This is the part that matters most day to day, and none of it requires a technical background.

Get specific before you get comfortable. Names, dates, statistics, quotes, citations, and URLs are the highest-risk category, because they’re precise enough to be checkably wrong and vague enough that a model can generate a plausible-sounding fake in a fraction of a second. A general summary of a well-known topic is comparatively low-risk. A specific citation backing it up is not.

Ask “does this actually exist,” not just “does this sound right.” A fabricated citation or software package name is, by the very mechanism that produced it, designed to sound exactly like a real one. Fluency and plausibility are not evidence. If a claim matters, look the source, the package, or the case up directly, independent of the AI that gave it to you.

Notice when a model won’t stop being specific. A model that answers “I’m not certain, but here’s what I’d check” about an obscure question is behaving unusually well, because, per the research above, that isn’t the behavior most models default to. A model that answers every question, including ones a genuine expert would hedge on, with the exact same confident tone is a model you should verify more, not less.

Watch for narrow, specialized, or recent territory. Hallucination rates climb sharply wherever training data thins out: small companies, local regulations, niche legal jurisdictions, anything that happened after the model’s training cutoff, and any field with a specialized, easily confused vocabulary, which is exactly why case law and software packages both show up so often in the record.

For code specifically, verify the package before you install it. Check that the library actually exists on the real package registry (PyPI, npm) before running an install command an AI assistant suggested, especially for a name you don’t already recognize. This single habit defeats slopsquatting entirely.

Cross-check anything consequential with a second, independent source. Not necessarily a second AI model, different models can share the exact same blind spots, since they’re often trained on overlapping data. An actual independent source: a search, a primary document, a person who knows.

Test an unfamiliar tool on something you already know the answer to. Before trusting a new AI product with a question that matters, ask it something you can independently verify first. How it handles a question you can already check tells you a lot about how much to trust it on one you can’t.

Watch for instant agreement when you push back. Many models will cheerfully reverse a correct answer the moment you express doubt, a trait researchers call sycophancy. It’s a second flavor of the same underlying issue: a model optimizing for a response that sounds good in the moment, not one that’s been re-verified against anything real.

A checklist of six practical checks for catching an AI hallucination before trusting it: verify specific facts independently, ask whether something actually exists, treat confident specificity with skepticism, watch niche or recent topics, check code packages on the real registry, and prefer tools that cite sources

None of these require technical skill. They just require treating fluency as style, not proof.

A note for anyone deploying AI, not just chatting with it

Most of this article is aimed at using AI tools as a reader or a customer. If you’re the one deciding how a model gets deployed, inside a product, a support workflow, an internal tool, the Air Canada case is the one to actually study, because its core finding generalizes: you’re responsible for what your AI says, the same way you’re responsible for what your website says, whether or not a human reviewed that specific sentence first.

A few habits meaningfully lower the risk, without requiring a research team:

  • Ground answers in real documents wherever the question allows it. Retrieval-augmented generation is worth the engineering effort specifically for anything resembling policy, pricing, or account-specific information, exactly the category that got Air Canada sued.
  • Ask the model to cite what it’s basing an answer on, and treat any answer with no citable source as lower-confidence by default, even internally.
  • Keep a human in the loop for anything with real consequences attached — refunds, medical information, legal claims, financial figures — rather than letting a model’s output ship directly to a customer unreviewed.
  • Test with questions you already know the answer to before trusting a system with questions you don’t, the same way you’d sanity-check a new hire on something verifiable before handing them something that isn’t.
  • Assume this is permanent, not temporary. Build verification into the workflow rather than waiting for a future model version to make it unnecessary. Per the research above, there’s no clear evidence that version is coming.

Six common misconceptions

“Hallucination means the AI is lying.” Lying requires knowing the truth and saying something else on purpose. A model has no concept of what’s true and nothing resembling intent. It’s closer to a very confident, very fluent guess than a lie, which doesn’t make it less dangerous, but does change how you should think about fixing it.

“It only happens with cheap or outdated models.” It happens with every model in production today, including the most capable ones from every major lab, and, per OpenAI’s own data on o3 and o4-mini, sometimes gets worse in newer models, not better. No current tier of AI model, free or expensive, is immune.

“If the answer sounds detailed and confident, it’s probably accurate.” This is close to the opposite of true. Length, fluency, and confidence are stylistic patterns a model learned from confident writing in its training data. None of them are a signal the model uses to represent how certain it actually is, because it doesn’t have a reliable internal measure of that to draw on in the first place.

“RAG or web search solves the problem.” It substantially reduces it and is genuinely worth seeking out in a tool. It doesn’t remove the underlying mechanism: the model can still misread a real source, or the retrieval step can quietly return the wrong one.

“This will get fixed as models get smarter.” Maybe eventually, but not automatically, and not from scale alone. OpenAI’s own researchers argue that hallucination is substantially caused by how models are scored, not by how capable they are, which means the fix, if it arrives industry-wide, looks more like a change to evaluation standards than a bigger model quietly making the problem disappear.

“This only happens with chatbots, not search or voice assistants.” Any product that generates a fluent sentence from a language model can hallucinate, including retrieval-based ones. The glue-on-pizza example above happened inside Google’s AI Overviews, a system built specifically to summarize retrieved web results rather than freewheel from memory.

What this means for how you actually use AI

None of this is a reason to distrust AI tools wholesale, any more than knowing a search engine can surface a bad result is a reason to stop using search engines. It’s a reason to calibrate your trust to the specific claim in front of you, not to the tool’s general reputation.

Treat any single, checkable fact — a number, a date, a quote, a citation, a package name, a legal claim — as unverified until you’ve confirmed it somewhere else, no matter how confidently or fluently it was delivered. Reserve your heaviest scrutiny for exactly the situations where hallucination is statistically most likely: niche topics, recent events, specialized fields, and anything with a name or number attached.

Notice which chat assistants and AI products show their sources and which don’t, and weight your trust accordingly, a cited answer is a checkable one. If you’re evaluating an AI product for your team or business, “does it hallucinate less, and how do you know” is a fair, answerable question to ask a vendor directly, and public benchmarks like Vectara’s leaderboard are a reasonable place to check a specific model’s track record before taking the vendor’s word for it.

And if you write code with AI assistance, make the package-verification habit automatic. It’s a thirty-second check standing between you and a supply-chain problem that’s already, demonstrably, cost other developers real security incidents.

A one-paragraph mental model to keep

If you remember nothing else: an AI model was built to predict plausible text, not to verify true text, and for most everyday questions those two things happen to line up closely enough that the distinction doesn’t matter. Hallucination is what surfaces the moment they come apart, when the model runs out of real pattern to draw on and fills the gap with something that merely resembles a correct answer, delivered in the exact same confident voice it uses for everything else. Recent research suggests this isn’t a bug waiting to be patched out; it’s a predictable consequence of how these systems are trained and, especially, how they’re graded, which is why scale alone hasn’t fixed it and likely won’t. Knowing that changes what you do next: you don’t stop using the tool, you stop trusting fluency as a proxy for truth, and you check the specific things that are cheap to check and expensive to get wrong.

The bottom line

An AI hallucination isn’t a rare malfunction you might get unlucky with. It’s a structural, well-documented, actively researched consequence of building a system optimized to sound right rather than be right, and then grading it in a way that keeps rewarding the sounding-right part. Jake Moffatt didn’t get unlucky with a buggy chatbot. He ran into the normal operating behavior of the system Air Canada deployed, doing exactly what it was built to do, in a spot where that happened to be the wrong thing to do.

That’s the useful takeaway, not a scary one: once you know specifically where and why this happens, you know exactly what to check, and you can keep getting real value from these tools without quietly absorbing their mistakes as your own. Now you know what’s actually happening when an AI sounds completely sure of something that isn’t true. That’s most of the battle.

Frequently asked questions

Is an AI hallucination the same thing as the AI lying?

No. Lying requires knowing the truth and deliberately saying something else. A language model has no concept of truth to begin with and no intent — it's predicting plausible text, and sometimes the most plausible-sounding continuation happens to be false. That doesn't make it less risky to rely on, but it does mean the fix is different from catching a liar: you can't out-interrogate a hallucination, you have to verify it.

Do bigger, newer AI models hallucinate less than older ones?

Not reliably. OpenAI's own system card for its o3 and o4-mini models showed both hallucinating more often than their predecessor, o1, on a benchmark testing factual claims about people, with o4-mini hallucinating on roughly 48% of questions. Independent benchmarks that track this over time show the same pattern: model size and release date are weak predictors of hallucination rate.

Does retrieval-augmented generation (RAG) or web search stop hallucinations completely?

It reduces them meaningfully but doesn't eliminate them. RAG gives a model real documents to read before answering instead of relying purely on memorized training data, which cuts hallucination rates. But the retrieval step can still return the wrong document, and the model can still misread or misquote the right one, so a cited answer is safer, not guaranteed.

Why does AI sometimes invent citations, court cases, or sources that don't exist?

Citations have a very recognizable format, author names, titles, years, docket numbers, that a model has seen thousands of times during training. When it doesn't have a real source to draw on, it will still generate text in that exact format, because producing plausible-looking text is its actual job. The result looks like a citation and isn't grounded in anything real.

How do I know if an AI coding assistant suggested a fake software package?

Check the exact package name on the real registry (PyPI for Python, npm for JavaScript) before you install anything an AI assistant recommends, especially a name you don't already recognize. Research on this problem, known as slopsquatting, found roughly 1 in 5 AI-generated code samples referenced at least one hallucinated package name, and attackers actively register those fake names with real malware behind them.

Which AI tools hallucinate the least?

It varies by task and changes often enough that any specific ranking goes stale fast, so treat a claimed number as a snapshot, not a permanent fact. Independent, continuously updated benchmarks like Vectara's hallucination leaderboard are a more reliable way to check a given model's current track record than relying on a single blog post's numbers, including this one.