The Question Every Lawyer Is Asking Right Now

The number of AI hallucination cases appearing in US courts has grown sharply since 2023, with legal researcher Damien Charlotin’s AI Hallucination Cases Database tracking a rapidly expanding body of judicial decisions and major outlets continuing to report new sanctions through 2025 and 2026. The problem is not whether lawyers use AI. It is whether they use it with a verification workflow.

Adoption is real and growing. The 2025 ABA Legal Industry Report found that 31% of legal professionals personally used generative AI at work, up from 27% the year before. The 2024 ABA TechReport found 30% of attorneys have offices currently using AI-based tools, with ChatGPT cited as the most widely used platform. Personal injury attorneys led all practice areas in individual AI adoption at 37%, according to the 2025 report.

The case that started the sanctions conversation was Mata v. Avianca, Inc. (S.D.N.Y. 2023). Attorney Steven Schwartz used ChatGPT to research an aviation personal injury claim. ChatGPT returned six case citations. All six were fabricated. Schwartz submitted them to federal court without verifying a single one against a primary legal database. When opposing counsel could not locate the cases, Schwartz doubled down and submitted what he believed were copies of the decisions. Judge Kevin Castel sanctioned Schwartz, his colleague Peter LoDuca, and their firm $5,000. In his testimony, Schwartz described ChatGPT as a “super search engine.” That phrase has since become the defining misunderstanding of the AI sanctions era.

That was 2023. Here is what has happened since. 

From One Case to Hundreds: How Fast This Has Accelerated

The Mata case was treated as an isolated incident. It was not.

For a running record of AI sanction cases and their outcomes, see the NexLaw 2026 AI sanctions tracker and the ByoPlanet case breakdown.

Can Lawyers Use ChatGPT for Legal Research?

This is the question at the center of every sanctions case documented so far. The answer requires understanding what ChatGPT actually is, not what it appears to be.

Schwartz’s description of ChatGPT as a super search engine captures the misunderstanding precisely. ChatGPT is not a search engine. It does not retrieve information from legal databases. It does not have access to Westlaw, LexisNexis, or any primary legal source. What it does is generate the next most probable word based on patterns learned from its training data. It is a prediction engine.

When you ask ChatGPT to find cases supporting a proposition about personal injury liability, it does not search for real cases. It generates text that looks like what a response to that query would look like. Case citations have a recognizable format: party names, reporter abbreviation, volume number, page, court, year. ChatGPT generates text in that format. The specific case it produces may or may not correspond to a case that actually exists. ChatGPT cannot tell you which.

Stanford HAI researchers tested leading legal AI tools and found they hallucinate in at least 1 in 6 queries. General-purpose chatbots like ChatGPT, by contrast, hallucinated between 58% and 82% of the time on legal queries in an earlier Stanford study. The Boston Bar Association addressed this directly in March 2026, noting that ChatGPT is designed to please the user, whether or not it can find real cases to support the argument it is asked to make.

There is one additional failure mode that appears consistently in sanctions cases. When Schwartz asked ChatGPT whether the Varghese case was real, ChatGPT confirmed that it was real and that it could be found on Westlaw and LexisNexis. It could not. Asking ChatGPT to verify its own citations does not work. It will generate a plausible-sounding confirmation because that is the most probable response to that prompt.

The real problem is not negligence. It is speed. Lawyers use ChatGPT because they are busy. The trouble starts when a time-saving drafting tool quietly becomes a research source, and nobody catches the shift before the filing goes out. For more context on how hallucinations reach court filings, see AI hallucinations hitting US courts: when AI generates fake cases and AI hallucination legal risk for US litigators.

What Does ABA Formal Opinion 512 Require?

The ABA issued Formal Opinion 512 in July 2024, the first formal ethics guidance specifically addressing generative AI. It does not prohibit using AI. It establishes that existing Model Rules apply to AI use and that ignorance of AI limitations is not a defense.

The opinion’s sharpest language is worth noting directly: “Lawyers’ uncritical reliance on content created by a GAI tool is risky and almost certainly malpractice.” Formal Opinion 512 was published in July 2024. Every lawyer practicing today is on notice that it exists.

What Tasks Can Lawyers Safely Use ChatGPT For?

Courts and the ABA have been consistent: using AI is not the problem. The problem is using AI for tasks where unverified output goes into a court filing.

The ByoPlanet sanctions made the paralegal problem explicit. Attorney Paul’s paralegal was drafting submissions and Paul was tweaking them without always reviewing the citations. Judge Leibowitz found this constituted the unauthorized practice of law. The signing attorney’s obligation to verify cannot be delegated.

A useful framing from the 12AM Agency guide to ChatGPT for lawyers: treat ChatGPT like a first-year associate. You would not file a brief written by a junior without reviewing it. Every citation, every proposition of law, every claim in a court filing is yours, regardless of who or what drafted the first version.

The Noland decision (California Court of Appeals, 2025) added a further dimension. The court declined to award attorneys’ fees to opposing counsel partly because opposing counsel had failed to detect and report the fabricated citations. Courts are beginning to signal that both sides have a duty to spot AI hallucinations, not only the party who filed the brief containing them.

Johnson v. Dunn (N.D. Ala., July 2025) stated explicitly that monetary sanctions are proving insufficient to change behavior. The trajectory from 2023 to 2026 is from modest fines toward career consequences.

The Safe Workflow: 3 Steps Every Sanctioned Attorney Skipped

Every documented sanction case shares a common failure pattern. Not one sanctioned attorney had all three of these steps in place before the sanctions occurred.

Step 1: Separate research from drafting in your AI workflow. Use ChatGPT or similar tools for drafting, structure, and idea generation. Use a source-linked legal research tool for any task that produces citations. The tools serve different purposes and should not be substituted for each other.

Step 2: Verify every citation against the primary source before it goes into a filing. Pull the case. Read the section your citation relies on. Confirm the holding says what your citation claims. Confirm the case has not been overruled. This step cannot be delegated to a paralegal without the supervising attorney also completing it. It cannot be completed by asking ChatGPT whether the case is real.

Step 3: Never ask ChatGPT to verify its own citations. Schwartz asked ChatGPT whether Varghese was real. It said yes. It was not. ChatGPT generates a plausible-sounding confirmation because that is the most probable response to that prompt. If you cannot locate a case that AI cited by searching Westlaw or LexisNexis directly, stop. Do not file it. The case almost certainly does not exist.

Legal AI tools built on retrieval-augmented generation change the structure of this problem. Rather than generating citations from pattern prediction and requiring you to verify from scratch, RAG-based tools query primary legal databases before generating output. Every result includes a source link. Verification becomes checking a source, not running an independent database search. NeXa legal research is built on this architecture, querying verified primary sources across all 50 states and federal circuits before returning any research output.

For a deeper look at how NexLaw approaches citation verification for litigators, see why lawyers use NexLaw and how NexLaw compares to other legal AI tools.