Courts and schools tighten rules after AI adds errors to legal work

In short: More courts, judges, and schools are pushing back on careless AI use after filings included made-up cases and quotes.

What's going on

Courts in the US and abroad are reacting to legal documents that appear to have been drafted with generative AI, the kind of AI that writes text on demand. In several situations, judges withdrew or questioned decisions after learning that filings contained fake case citations, meaning references to court cases that do not exist.

In one 2024 example, a federal judge in New Jersey withdrew an opinion in a shareholder lawsuit after discovering it relied on a filing with AI-generated citations. The Georgia Supreme Court also sent a murder case back to a lower court after finding that a prosecutor used AI to draft filings with fake citations, and the prosecutor faced professional penalties and required training.

Similar warnings are coming from outside the US. India’s Supreme Court said it was “alarmed” after seeing petitions with non-existent cases and fabricated quotations, and it emphasized that judges should use AI very cautiously.

These legal problems echo what schools are seeing with students and AI. Surveys show many young adults use tools like ChatGPT, but courts and educators keep stressing the same point, AI can be like a fast intern who sometimes invents facts (you still have to check the work).

What to watch

Expect more formal rules that require lawyers, and by extension law students, to disclose when they used AI and to confirm every quote and citation. Also watch how institutions handle AI “re-creations” of public events, since they can improve access but may confuse people about what is real and what is generated.

Source: NYTimes