Advertisement
AI

Judges Are Fed up With Lawyers Using AI That Hallucinate Court Cases

Another lawyer was caught using AI and not checking the output for accuracy, while a previously-reported case just got hit with sanctions.
Judges Are Fed up With Lawyers Using AI That Hallucinate Court Cases
Photo by Conny Schneider / Unsplash

This article was produced in collaboration with Court Watch, an independent outlet that unearths overlooked court records. Subscribe to them here.

After a group of attorneys were caught using AI to cite cases that didn’t actually exist in court documents last month, another lawyer was told to pay $15,000 for his own AI hallucinations that showed up in several briefs. 

Attorney Rafael Ramirez, who represented a company called HoosierVac in an ongoing case where the Mid Central Operating Engineers Health and Welfare Fund claims the company is failing to allow the union a full audit of its books and records, filed a brief in October 2024 that cited a case the judge wasn’t able to locate. Ramirez "acknowledge[d] that the referenced citation was in error,” withdrew the citation, and “apologized to the court and opposing counsel for the confusion,” according to Judge Mark Dinsmore, U.S. Magistrate Judge for the Southern District of Indiana. But that wasn’t the end of it. An “exhaustive review” of Ramirez's other filings in the case showed that he’d included made-up cases in two other briefs, too. 

“Mr. Ramirez explained that he had used AI before to assist with legal matters, such as drafting agreements, and did not know that AI was capable of generating fictitious cases and citations,” Judge Dinsmore wrote in court documents filed last week. “These ‘hallucination cites,’ Mr. Ramirez asserted, included text excerpts which appeared to be credible. As such, Mr. Ramirez did not conduct any further research, nor did he make any attempt to verify the existence of the generated citations. Mr. Ramirez reported that he has since taken continuing legal education courses on the topic of AI use and continues to use AI products which he has been assured will not produce ‘hallucination cites.’” 

But the explanation and Ramirez’s promise to educate himself on the use of AI wasn’t enough, and the judge chided him for not doing his research before filing. “It is abundantly clear that Mr. Ramirez did not make the requisite reasonable inquiry into the law. Had he expended even minimal effort to do so, he would have discovered that the AI-generated cases do not exist. That the AI-generated excerpts appeared valid to Mr. Ramirez does not relieve him of his duty to conduct a reasonable inquiry,” Judge Dinsmore continued, before recommending that Ramirez be sanctioned for $15,000.

Lawyers Caught Citing AI-Hallucinated Cases Call It a ‘Cautionary Tale’
The attorneys filed court documents referencing eight non-existent cases, then admitted it was a “hallucination” by an AI tool.

The judge wrote that he “does not aim to suggest that AI is inherently bad or that its use by lawyers should be forbidden,” and noted that he’s a vocal advocate for the use of technology in the legal profession. “Nevertheless, much like a chain saw or other useful [but] potentially dangerous tools, one must understand the tools they are using and use those tools with caution,” he wrote. “It should go without saying that any use of artificial intelligence must be consistent with counsel's ethical and professional obligations. In other words, the use of artificial intelligence must be accompanied by the application of actual intelligence in its execution.” 

In January, as part of a separate case against a hoverboard manufacturer and Walmart seeking damages for an allegedly faulty lithium battery, attorneys filed court documents that cited a series of cases that don’t exist. In February, U.S. District Judge Kelly demanded they explain why they shouldn’t be sanctioned for referencing eight non-existent cases. The attorneys contritely admitted to using AI to generate the cases without catching the errors, and called it a “cautionary tale” for the rest of the legal world.  

Last week, Judge Rankin issued sanctions on those attorneys, according to new records, including revoking one of the attorneys’ pro hac vice admission (a legal term meaning a lawyer can temporarily practice in a jurisdiction where they're not licensed) and removed him from the case, and the three other attorneys on the case were fined between $1,000 and $3,000 each. 

Advertisement