OpenCase Logo

LEGAL CITATION BENCHMARK · NOVEMBER 2026

2026 Hallucinations in Legal AI Report

How often does AI cite fake case law? We asked eight AI models from OpenAI, Anthropic, Google and Meta 120 hard legal research questions and checked every citation they gave. Here is how often each one hallucinated a case or got a real one wrong, with and without web search.

Without tools · higher is better

  1. Claude Opus 5.5no tools
    98.9%
  2. Gemini 4 Carbonno tools
    97.1%
  3. Claude Haiku 5.5no tools
    96.5%
  4. GPT 5.6 Solno tools
    91.0%
  5. GPT 6 Lunano tools
    90.0%
  6. Gemini 4 Argonno tools
    88.6%
  7. Gemini 3.8 Flashno tools
    83.3%
  8. Muse Spark 1.3no tools
    74.6%

With web search · higher is better

  1. Muse Spark 1.3web search
    98.8%
  2. Claude Opus 5.5web search
    98.3%
  3. GPT 5.6 Solweb search
    96.9%
  4. Claude Haiku 5.5web search
    96.7%
  5. Gemini 4 Carbonweb search
    96.3%
  6. Gemini 4 Argonweb search
    94.8%
  7. GPT 6 Lunaweb search
    93.8%
  8. Gemini 3.8 Flashweb search
    93.7%

Key findings

  • Without tools, Muse Spark 1.3 got 25.4% of its citations wrong and Gemini 3.8 Flash 16.7%. Claude Opus 5.5 got 1.1% wrong, the fewest of any model.
  • Without tools, Muse invented at least one authority in 45% of its answers, and Gemini 3.8 Flash in 41%.
  • Web search helped the two weakest models most. Muse fell to 1.2% wrong and Gemini 3.8 Flash to 6.3%. Claude Opus moved from 1.1% to 1.7%.
  • With web search, neither GPT model invented an authority, but both still got details wrong. GPT 5.6 Sol miscited 3.1% of its citations and GPT 6 Luna 6.2%.
  • Google's Gemini 4 models cited more accurately than Gemini 3.8 Flash. Gemini 4 Carbon got 2.9% wrong without tools, second only to Claude Opus 5.5, and 3.7% with web search. Gemini 4 Argon got 11.4% wrong without tools and 5.2% with web search.
#SetupCorrectWrong
1Claude Opus 5.5 (no tools)98.9%1.1%
2Muse Spark 1.3 (web search)98.8%1.2%
3Claude Opus 5.5 (web search)98.3%1.7%
4Gemini 4 Carbon (no tools)97.1%2.9%
5GPT 5.6 Sol (web search)96.9%3.1%
6Claude Haiku 5.5 (web search)96.7%3.3%
7Claude Haiku 5.5 (no tools)96.5%3.5%
8Gemini 4 Carbon (web search)96.3%3.7%
9Gemini 4 Argon (web search)94.8%5.2%
10GPT 6 Luna (web search)93.8%6.2%
11Gemini 3.8 Flash (web search)93.7%6.3%
12GPT 5.6 Sol (no tools)91.0%9.0%
13GPT 6 Luna (no tools)90.0%10.0%
14Gemini 4 Argon (no tools)88.6%11.4%
15Gemini 3.8 Flash (no tools)83.3%16.7%
16Muse Spark 1.3 (no tools)74.6%25.4%

What a hallucination looks like

A hallucinated citation reads like a real one. A fake case has a plausible name, a real reporter and a page number. A fake statute has a section number that fits the code around it. The only way to catch either is to look it up.

Gemini 3.8 Flash, no tools · Ninth Circuit sentencingUnited States v. Peraza, 644 F. App'x 788 (9th Cir. 2016)That page is Henry v. Regents. No Ninth Circuit decision named Peraza matches.
Muse Spark 1.3, no tools · Kansas farm foreclosureK.S.A. 60-2422Kansas has no such section. The article of its code on executions and sales ends at 60-2420.
GPT 5.6 Sol, no tools · Michigan employment lawLaborde v City of Grosse Pointe Farms, 199 Mich. App. 590, 502 N.W.2d 336 (1993)Those pages hold Mills v. White Castle and Price v. Long Realty. No case by this name exists.
Gemini 3.8 Flash, web search · Fifth Circuit maintenance ratesBorders v. Abdon Callais Offshore, LLC, 831 F. Supp. 2d 946 (E.D. La. 2011)That page is Myers v. Simpson, a Virginia case. No suit named Borders against Abdon Callais turns up.

What we measured

Hallucinated
No case, statute or rule matching the citation exists.
Miscited
The authority is real, but a detail is wrong, such as the page, the year or the court.
Correct
Neither hallucinated nor miscited, as a share of the citations we could check. The few we could settle neither way are left out.

How we ran it

We wrote 120 questions of the kind a practicing attorney asks, each built around a real decision from a named American court. Every model answered every question twice: once with no tools, and once with its maker's own web search, at its maker's default reasoning level.

A checker with web access looked up each citation in court websites, official reporters, the Caselaw Access Project and other open sources. A second, separate pass re-checked every citation the first pass marked as hallucinated.

Limitations

  • OpenCase, which makes AI software for legal research, designed and ran this benchmark. We publish every question, citation and verdict so anyone can re-check our work.
  • A few questions got no answer, and each is left out of that setup's rate. Muse refused 4 questions in both modes under its content filter, and its web search failed on 2 more. Gemini 3.8 Flash with web search returned no answer to 9, even after repeated retries. Gemini 4 Carbon timed out or returned no answer to 1 of 120 questions without tools and 7 with web search.
  • Gemini 4 Argon often dropped or lowercased the first letters of a case name, writing “jasper v. State” for Jasper v. State. It did so in about one answer in five without tools. Gemini 4 Carbon did it in a handful of answers. Where the rest of the citation matched a real case, we counted the clipped name as a typo, not a hallucination. Where it matched no real case, we counted a hallucination. A reader could not always look these cases up as written.
  • Each answer could use up to 16,000 output tokens, including the model's reasoning. Gemini 4 Carbon used them all on 11 answers without tools, which stop mid-answer. We checked the citations they gave before the cut. No other model reached the limit.
  • Each model ran at its default reasoning level. A higher setting may cite more accurately, at more cost and delay.
  • The rate is per citation. It does not measure whether an answer cited enough authority, or whether a real case supports the point it was cited for. The few citations the checker could settle neither way are left out.