Generative AI tools can produce fluent, confident text that is simply wrong. In legal work the most notorious form is the fabricated citation: a case name, court and quote that look real and do not exist. Several courts have sanctioned or admonished lawyers who submitted filings containing unverified AI-generated authorities, and the problem is not limited to citations. Tools can also misstate facts, blend jurisdictions, misread documents and omit critical caveats.
ABA Formal Opinion 512 on generative AI emphasizes that lawyers must understand the limits of the tools they use and independently verify output, in light of their duty of competence under Rule 1.1. The responsibility stays with the lawyer. This checklist turns that principle into a routine a firm can adopt.
Why hallucinations happen
Language models predict plausible text. They are not databases of verified law, and without a connection to authoritative sources they may fill gaps with something that sounds right. Even tools designed for legal research can make errors, so do not assume that a legal-branded product eliminates the risk. Treat every output as a draft written by a fast, eager assistant who sometimes bluffs.
A verification checklist
For citations and authorities
- Locate every case, statute and regulation in a trusted primary source such as an official reporter, a court website or your licensed research service.
- Read the actual text, not just a summary. Confirm the case says what the draft claims.
- Check quotations word for word against the source.
- Confirm jurisdiction and court level, since a persuasive out-of-state case is not binding.
- Run a citator check to confirm the authority is still good law.
- Verify pin cites and parallel citations.
- Look for what is missing, such as adverse authority the tool did not mention.
For facts and record references
- Compare every factual assertion with the underlying documents or testimony.
- Check dates, amounts, names and procedural history.
- Confirm that quoted evidence actually appears where cited.
- Watch for details that appear in no document, which are a classic sign of invention.
For summaries and analysis
- Spot check a sample of the source material against the summary, and expand the sample if you find errors.
- Ask whether the tool had access to the complete document or only a portion.
- Consider whether subtle distinctions that matter legally were flattened.
- Be skeptical of summaries that sound unusually neat.
For drafts and contracts
- Compare AI-suggested language against your forms and the governing law.
- Check defined terms and cross references for consistency.
- Confirm that nothing was silently added or dropped.
Build verification into workflow
Assign a responsible reviewer
Require a named attorney to review and sign off on any work product that involved AI, consistent with supervision duties under Rules 5.1 and 5.3. Do not allow a junior employee to be the only check on an AI-assisted filing.
Keep a short log
Record which tool was used for which task, who verified and when. A simple note in the matter file is enough. It helps if a question arises later and supports internal quality control.
Check court rules
Some courts and judges have issued standing orders about AI use or disclosure, and rules vary widely. Assign someone to check the local rules and any judge-specific requirements before filing.
Train for healthy skepticism
Short training sessions work better with examples. Show staff what a hallucinated citation looks like, how to spot overconfident language and why they should never paste in AI output without checking it. Encourage a culture where it is normal to say, "I couldn't verify this."
Prompts that reduce risk
- Ask the tool to say when it is uncertain, and treat the answer as a hint, not a guarantee.
- Provide the source documents you want it to rely on when the tool supports that.
- Ask it to quote the passage that supports each claim, then verify the quotes.
None of these eliminate errors. They only make errors easier to catch.
What to do when a mistake is found
If a filed document contains an unverified or false authority, consult the supervising attorney and ethics counsel quickly. Courts have generally responded better to prompt, candid correction than to concealment. Review how the error occurred and adjust your process.
Document your standard
Put the checklist into your AI use policy, and revisit it as tools evolve. Make it short enough that people use it.
Working with Counsel Cyber
Counsel Cyber helps law firms set up approved AI tools, access controls and training. If you would like to add a verification workflow to your AI policy, we can help draft it.