Utah case highlights a growing concern of lawyers over relying on AI

SALT LAKE CITY — Artificial intelligence is being trusted by a wide array of professionals. Software developers use it to generate code. Doctors are leaning on AI to craft insurance appeals, or even to help diagnose diseases. And lawyers can use AI to draft legal documents, summarize case law and perform legal research more efficiently. Now these important pros need to be careful, because AI can get stuff wrong. But sometimes it just isn’t wrong. Our investigation found AI can pull made-up information right out of its digital hat.

Citing fake cases

It happened recently in a Utah courtroom. Two attorneys filed a legal petition containing case citations. Here’s the thing — their cited cases don’t actually exist. You’ll never find them in any legal database. They were faked by the AI bot, ChatGPT — entirely fabricated. “Well, one of the things about AI is it doesn't care,” said attorney Stephen Howard, who was not involved in this recent Utah case. Howard has been outspoken about some of the risks of lawyers relying on AI. “It is a set of algorithms that really are designed to present something that looks good, that seems to make sense,” he said. “There's no truth checking, necessarily.”Truth checking is the responsibility of the lawyer using AI. If they don't, there can be consequences. In the Utah case, the state court of appeals ordered the attorney citing the false AI case-law to refund their client, pay the opposing party's attorney fees, and make a $1,000 donation to a legal aid fund. But that case is a drop in the bucket. A database that tracks legal decisions where AI-created or “hallucinated” information has logged more than 130 cases. It’s a number that keeps growing. “The law is a pretty powerful thing. How concerned should people be that lawyers are using AI,” I asked Howard. “It depends on who your lawyer is,” he answered.

AI in Utah courtrooms

Elizabeth Wright is the executive director of the Utah State Bar, which issues the licenses lawyers need to practice law in the state of Utah. She said that while AI encroaching into courtrooms might not be altogether new, today’s accelerated pace of adoption and expansion tells her that all lawyers need to be on guard. “It's moving very quickly,” Wright said. Lawyers saying things in court that are flat out wrong tend not to leave judges amused. “Lawyers have an obligation that they've always had — even before technology — to ensure that the documents they’re preparing are correct,” Wright said.While getting caught citing fake cases created by AI can leave attorneys feeling embarrassed, where does it leave their clients? Could you forfeit your case, face penalties or even get locked up because your attorney prepared documents peppered with AI hallucinations? Wright said, not likely. “Fortunately, we have checks,” she said. “Checks in our system.” One of the biggest checks is that lawyers are required to tattle on themselves if they learn they've got something wrong. Not doing so could cost them their law license.

Representing yourself using AI

AI hallucinations are also something you should consider if you ever need quick legal advice and hope to save a few billable hours. From small claims to debt collection and even criminal defense, many people choose to represent themselves in court. If you plan to rely on AI to do your paralegal work, don’t assume it’s giving you the straight truth. “Clients ultimately have the choice to decide if they want their lawyers to use AI,” Wright said.

And your lawyer using AI?

Despite the recent goofs surfacing in courtrooms, Wright remains bullish on lawyers using AI. It can speed up a lot of the tedious work, which in turn can lower bills for clients. “Just have a very open conversation about the risks, the benefits and hopefully, the cost savings,” Wright advised. The Utah State Bar requires mandatory training and continuing education for its practicing attorneys — even after they’ve received their law licenses. The risks and rewards of using AI is a big focus of that training these days.

Related stories: Utah lawmakers bolster AI consumer protection laws

Related topics