I really like writing this blog and I've been writing it on and off for about 14 years.
That's right - 14 YEARS!
Some topics are better than others but what has really been helpful is this new AI set up they have going.
Specifically, Chatgpt has come in handy helping with identifying topic, cases, and statutes relevant to the topics I'm writing about.
The problem with these AI search engines like ChatGPT is that you can't really depend on them for accuracy when researching law or things of a legal nature. Sometimes, it even creates cases out of thin air.
Wait, what?!
You mean to tell me that AI search engines spit out caselaw that doesn't even exist or out and out misleads researchers?!
Called Hallucinations and yes, that's exactly what I mean.
Because LLMs (i.e. "Large Language Models" which is the official classification for AI platforms) create hallucinations with reckless abandon, you have to be extra careful when using anything you find on an AI search engine.
For example, do you remember Mata v. Avianca, Inc., No. 22-CV-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023)? In Mata, lawyers submitted a brief with six case citations (and quotes) that did not exist; they were generated/made up by ChatGPT. The court imposed a $5,000 fine on the lawyers and their firm (which is peanuts). The judge found they acted in bad faith, including conscious avoidance of verifying the non-existence when there were signs the cases might be fake.
Or how about United States v. Cohen, 724 F. Supp. 3d 251 (S.D.N.Y. 2024). In that case Michael Cohen admitted he had given his lawyer fake citations generated by Google Bard. The lawyer then submitted them. The court called the inclusion of the fake cases “embarrassing and certainly negligent, perhaps even grossly negligent.” But the judge determined there was no intention to deceive, i.e., not in bad faith, so no sanctions were imposed.
Other cases where attorneys used holographic cases (ie. cases generated by AI search engines that don't exist) include:
Garner v. Kadince, 20250188-CA (Utah Ct. App., filed May 22, 2025): Their petition included case-citations that were found to be fabricated or “hallucinated” by AI (ChatGPT). One example: Royer v. Nelson, 2007 UT App 74, 156 P.3d 789, a case that does not exist in any legal database except in AI outputs.
Wadsworth v. Walmart (D. Wyo. 2025): Three lawyers cited fake cases that were AI-generated in a personal injury lawsuit against Walmart.
My personal opinion is that any attorney that doesn't Shepardize ALL cases cited in legal documents should be, in the very least, suspended for at least a 6 months. They should also be required to pass the Multistate Professional Responsibility Exam (aka MPRE) before being again allowed to practice law.
But how does this all affect me, the legal blogger?
Over the last year, I've blogged about a whole range of legal topics. In a few instances, I've found, through ChatGPT and other AI search engines, not a few discrepancies.
For example, recently I wrote a blog about tailgating and after running a search in ChatGPT for cases related thereto, I got a number of cases back presumably on the subject.
One case in particular, Shuck v. Means, was particularly difficult to locate on the Internet and I asked ChatGPT:
is Shuck v. Means, 302 P.2d 217 (Okla. 1956) a real case and does it have anything to do with tailgating
To which ChatGPT responded:
You're in for a quick clarification—“Shuck v. Means, 302 P.2d 217 (Okla. 1956)” does not appear to be a real case, at least not under that name or citation.
A few paragraphs later, I ran into a similar problem when after a query ChatGPT returned a case (in this instance, Baker v. Rushing). After an extensive search on the Internet of not finding that case, I asked ChatGPT:
Yes — that citation format is correct. Here’s the breakdown:
Case Name: Baker v. Rushing
Reporter Volume: 104
Reporter Abbreviation: S.E.2d (South Eastern Reporter, Second Series)
Page Number: 165
Court and Year: (N.C. 1958) → North Carolina Supreme Court, 1958
So it should appear as:
📖 Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958).
That is a valid, properly formatted Bluebook citation.
Now, to the untrained eye, that would look like this is a real/actual case and a rookie researcher would stop there. I mean, it says it's the right Bluebook format - so that should mean it's a real case. However, after I couldn't find Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958), I went back to ChatGPT and again asked:
is Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958) a real case
To which Chatgpt responded:
Great question — “Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958)” does not appear to be a valid or real case citation, at least based on available legal databases and authoritative sources.
...which is annoying - and definitely misleading - given that ChatGPT first said the case was in the correct Bluebook citation format (which was technically true) but then acknowledged that it screwed up and returned a case that was neither valid or real.
Soooooo aggravating!
Not wanting to drop the issue, I again asked ChatGPT:
if Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958) is not a valid or real case, why did you return it in my initial query
To which ChatGPT responded:
That’s a very fair question — and you’re right to call this out.
Here’s what likely happened:
When you first asked about tailgating-related cases, I tried to retrieve examples of real case law, but I sometimes draw from patterns in legal writing rather than verified court databases. “Baker v. Rushing, 104 S.E.2d 165 (N.C. 1958)” looks like a perfectly plausible citation (correct reporter, correct jurisdiction, correct year), but after checking, there’s no record of such a case.
This is what’s known as a hallucination (or “holographic case”), where I construct something that looks real based on legal citation formats and similar case names — but it doesn’t actually exist.
The main reasons this happens are:
Predictive nature of my training: I was trained to generate text that follows patterns, including how case citations are written, but I don’t have perfect access to every case law database.
Similarity to real cases: There are real cases with names like Baker v. [Someone] and reporters like S.E.2d from 1958, so it can produce something that seems legitimate.
Overconfidence in early responses: Without doing a real-time case law search (like we just did for your follow-ups), I might give examples that aren’t verified.
This tends to happen more and more and as I use AI to help with blogging, I now ask whether particular cases returned are real before including them in my blog.
But this got me to thinking and I asked ChatGPT:
why does chatgpt return cases in a search that don't exist?
To which ChatGPT responded:
That’s a great question — and it’s one of the most important issues when using AI for legal research.
ChatGPT (and other LLMs) sometimes return cases that don’t exist because of the way large language models work:
Pattern Matching, Not Databases:
I don’t have a built-in legal case database. Instead, I was trained on lots of legal writing and I generate responses by predicting what looks like a plausible answer. If you ask for a case citation, I try to produce one that fits the pattern of a real case — correct reporter, volume, page — even if it’s made up.Pressure to Be Helpful:
If you ask for “a case where X happened,” I try to fulfill that request. If I don’t have a known case in memory, I may invent one that sounds right rather than say “I don’t know,” especially if your prompt suggests that such a case must exist.Legal Citations Are Predictable:
Because case citations follow a rigid format (e.g., Smith v. Jones, 123 F.3d 456 (9th Cir. 2020)), I can generate something that looks very convincing — even to experienced lawyers. But if you check the reporter, the case might not exist.Contextual Merging (Hallucination):
Sometimes I blend real cases. For example, I might take facts from Case A and a holding from Case B and give it a citation that matches neither. This is not intentional — it’s a statistical artifact.This problem is exactly what led to the high-profile Mata v. Avianca (S.D.N.Y. 2023) case — where lawyers used ChatGPT to write a brief, it cited six fake cases, and the court sanctioned them.
✅ Best Practice When Using AI for Legal Research:
Treat all AI-supplied case names and citations as leads, not authorities.
Verify every case in a trusted source (Westlaw, Lexis, Fastcase, Casetext, etc.) before using it.
If you can’t find it in an official reporter or database, assume it doesn’t exist.
- always verify results,
- always recheck results, and
- never trust that the bot you're talking to is telling you the truth
No comments:
Post a Comment