Legal teams that are increasingly relying on AI tools to help generate some of their legal work are doing so at their peril. In a recent Mississippi case in federal court, lawyers on both sides of a dispute over a solar project's fees were disciplined for using AI software that hallucinated cases or included cases in filings that weren't part of state law.

The case, Withers v. City of Aberdeen, involved a dispute between a lawyer, Tom Withers III, and the city of Aberdeen, Miss. Withers claimed the city owed him fees involving a solar development project. He wasn't one of the lawyers reprimanded, but his legal team was, as were lawyers representing the city, when a judge determined that four of them -- two of them on each side -- had used AI in filings that weren't properly verified by human lawyers.

The outcome drew the attention of The New York Times, among other outlets, and could come to represent a cautionary tale for those in the legal profession. It was first spotted by marketing and commerce lawyer Rob Freund who posted about it on X, calling it a "comedy of AI errors."

The website 404 Media reported on Freund's post, drawing national attention to the cse.

US District Judge Sharion Aycock fined the four lawyers and shut down the case over the AI errors, among other legal sanctions.

"The Court finds that, through their own admissions, all four attorneys failed to verify the legal authorities cited in their respective filings in violation of Rule 11," she said.

Rule 11 of the Federal Rules of Civil Procedure states that a lawyer must sign a filed legal document, basically promising to the judge that what is being filed is truthful. 

"All attorneys are licensed and are presumably well trained in the law," Aycock wrote. "Their practice of blindly relying on technology resulted in the hallucinatory citations contained in their respective filings."

AI hallucinations are errors or falsehoods that large language models routinely serve up as they string together words into a plausible-sounding narrative.

AI's legal pitfalls

The Mississippi case isn't the only instance of lawyers getting into hot water over AI mistakes. Legal snafus recently led the state of New York to adopt new rules about generative AI -- as of June 1, for instance, results generated using tools like ChatGPT are not protected by attorney-client privilege.

The largest US association for lawyers, the American Bar Association, said in a statement to CNET that lawyers are facing new and complex challenges when it comes to using AI with their work.

"Lawyers understand that generative AI outputs require scrutiny and oversight," said ABA President Michelle A Behnke. "The ABA is here to support its members as they navigate new law, new technology and new practice tools."

The group has a task force on AI and in December published a report that covers AI adoption among lawyers, best practices and AI policies. It also has issued ethical guidance for lawyers using AI.

Among its advice: "lawyers should understand 'the benefits and risks associated' with the technologies used to deliver legal services to clients."