Legal AI products are arriving faster than most firms can evaluate them. Every demo looks impressive, and every vendor says its tool is secure. The risk is not that all of them are bad. It is that firms buy on enthusiasm and discover the data-handling terms later.
ABA Formal Opinion 512 on generative AI discusses lawyers' duties of competence, confidentiality, communication, supervision, and reasonable fees. Those duties translate into practical vendor due diligence. These ten questions are a place to start. Write the answers down and keep them with the contract.
Data handling and confidentiality
1. Is our data used to train any model?
Get the answer in writing in the contract, not only on a website. Ask whether the answer is the same for every plan tier, since consumer and enterprise versions of a product often differ.
2. Where is our data stored and for how long?
Ask whether prompts, uploaded documents, and outputs are retained, where, and whether you can set retention to zero or delete on request. Ask which subprocessors, such as cloud and model providers, touch the data.
3. Who at the vendor can see our content?
Ask about employee access, support access, and whether human reviewers ever read prompts. Ask how access is logged and limited.
4. What security assurances can you show?
Request current independent audit reports, such as a SOC 2 Type II report, and ask about encryption, MFA, and incident notification timelines. Have someone technical review the documents rather than accepting a one-page summary.
Accuracy and responsibility
5. How does the tool reduce and expose errors?
Generative tools can produce plausible but wrong statements. Ask whether outputs link to source documents or authorities so a lawyer can check them. Ask what the vendor does not claim the tool can do.
6. What review does the tool expect from us?
A responsible vendor will say that attorney review is required. Be wary of any pitch implying the tool replaces it. Under the Model Rules, the lawyer remains responsible for the work.
Contract and commercial terms
7. What does the contract say about liability and indemnity?
Read the limitation of liability. Many agreements cap liability at fees paid in the last year, which may be tiny relative to a confidentiality breach. Ask about breach notification obligations and who pays for response costs.
8. How is pricing structured, and what happens at renewal?
Look for usage-based charges, per-seat minimums, add-on fees, and price increase clauses. Ask how billing to clients should work, and decide in advance how the firm will treat the tool's cost and the time it saves.
9. Can we leave and take our data?
Ask for export formats, deletion confirmation upon termination, and a timeline. A tool that becomes embedded in your workflow but cannot be exited cleanly is a long-term risk.
Fit with your firm
10. Does it work with how we already operate?
Check integration with Microsoft 365, your document management system, and single sign-on. Check whether you can limit use by role, apply matter-based restrictions, and see usage logs for supervision purposes.
Run a controlled pilot
Do not roll out to the whole firm on day one.
- Pick a few users and a narrow, low-risk use case.
- Use sample or public documents at first.
- Compare outputs to what an attorney would produce.
- Track errors and corrections.
- Decide go or no-go with written criteria.
Tell your clients when needed
Some clients restrict AI use in outside counsel guidelines. Check them before using any tool on their matters, and ask your state bar about any disclosure expectations.
After the purchase
- Add the tool to your AI use policy as an approved product.
- Train users before they get access.
- Review the vendor's security posture at renewal.
- Re-check terms when the vendor announces changes, because privacy terms sometimes shift.
A word from us
Counsel Cyber helps firms review AI vendor contracts and security documentation from the technical side, and configure approved tools with sensible access controls. If you are weighing a purchase, we are happy to review the paperwork with you. This is general information, not legal advice.