Generative AI tools can produce fluent, confident legal text that is simply wrong. They may cite cases that do not exist, misstate holdings, or attribute quotations to the wrong source. These errors are often called hallucinations. Courts have publicly sanctioned lawyers who filed briefs containing AI-generated citations that were never verified, and the episodes made headlines precisely because the filings looked professional.
The lesson is not that AI has no place in legal work. It is that a firm needs a repeatable verification workflow so no unchecked output reaches a client or a court.
What the ABA Has Said
ABA Formal Opinion 512, issued in July 2024, applies existing professional duties to generative AI. It discusses competence under Model Rule 1.1, noting that lawyers should understand the benefits and risks of the tools they use, and says lawyers must independently review and verify AI output to a degree appropriate to the task. It also touches on supervision under Rules 5.1 and 5.3, since junior lawyers and staff may use these tools. Check your state's guidance and any local court rules, because some judges have issued standing orders about AI use in filings.
Why Hallucinations Happen
These tools predict plausible text based on patterns. They do not look up a verified database unless they are specifically connected to one. When the model does not know an answer, it may still produce one that sounds right. Legal citations are especially vulnerable because their format is easy to imitate.
Even tools built for legal research and linked to real databases can make mistakes, such as summarizing a case inaccurately. Treat all output as a draft.
A Verification Workflow
1. Decide whether AI is appropriate for the task
Good candidates include brainstorming issues, summarizing a document you provide, creating a first-draft outline, and rephrasing your own text. Higher-risk tasks include producing citations, stating the current law, and drafting final work product without review. Never enter confidential information into a tool the firm has not approved.
2. Record the use
Note in the file that AI was used, which tool, and for what purpose. This helps with supervision, billing, client communication and any court disclosure requirements.
3. Verify every authority
For each case, statute, regulation or quotation in AI-generated text:
- Confirm it exists by locating it in a trusted legal research database
- Read the actual source, not just a summary
- Check that it says what the draft claims and that quotations match exactly
- Confirm the citation details, including court, year and pinpoint pages
- Check subsequent history to confirm it is still good law
- Check jurisdiction and relevance to your matter
4. Verify facts
Check any factual statement against the file. AI may merge details from different documents or introduce facts that were never there.
5. Second-person review
Have another attorney review work product that relied on AI, particularly filings. The reviewer should have access to the verification record, not just the final text.
6. Final certification
Before filing or sending, the responsible attorney confirms in writing, even if only an internal checklist item, that all authorities were independently verified.
Training Attorneys and Staff
- Show real examples of plausible but false output
- Teach prompt practices that reduce risk, such as providing source text and asking the tool to quote from it
- Emphasize that a fluent answer is not evidence of accuracy
- Remind junior attorneys that using a tool does not shift responsibility from the signer
Policy Points
- Name approved tools and the tasks for which they may be used
- Require verification before any AI-assisted authority is cited
- Require documentation of use
- Set an escalation path when someone discovers an error
- Review court rules and standing orders on AI disclosure
What Clients Expect
Clients increasingly ask whether their outside counsel uses AI, and how. A written workflow lets you answer confidently, and it supports your response to client questionnaires and outside counsel guidelines.
Technical Safeguards
IT can support the workflow by controlling which tools are accessible from firm devices, configuring business-grade accounts with data protections, and logging access. These controls complement but do not replace attorney judgment.
Next Steps
Counsel Cyber helps firms select and secure AI tools, configure approved options, and write practical policies that include verification procedures. If your firm wants to use AI responsibly, we can help you build the guardrails first.