New AI tools appear weekly, and each promises faster drafting, better research, or a shortcut through discovery. Some of them may be worth using. The problem is that the buying decision often happens informally: one attorney tries a tool, likes it, and tells a colleague. By the time the firm's administrator hears about it, client information has already been entered.
A short, repeatable vetting process prevents that. The five questions below can be answered by most vendors in a single call or questionnaire. They also give you a record that the firm exercised reasonable diligence, which matters under the competence and confidentiality duties the ABA discussed in Formal Opinion 512 on generative AI.
Question 1: What happens to our data when we use it?
This is the question that matters most. Ask the vendor directly:
- Are prompts, uploaded documents, and outputs used to train or improve models?
- Is training use off by default, or do we need to opt out?
- Is there a contractual commitment, or just a marketing statement?
Free and consumer tiers often have broader data-use rights than business tiers. Never assume the terms of one plan carry over to another. Get the answer for the specific plan you intend to buy.
Red flag: the vendor cannot give a clear written answer, or the answer changes depending on who you ask.
Question 2: Who can see it?
Find out which employees and subcontractors of the vendor can access your content, under what conditions, and with what logging. Ask whether human reviewers ever read prompts, for example to monitor abuse, and whether that can be disabled for business customers.
Also ask which other companies are involved. Many AI products are built on models from other providers, so your data may pass through more than one set of hands. Ask for the list of subprocessors.
Red flag: no list of subprocessors, or no ability to restrict human review.
Question 3: Where is it stored, and for how long?
Ask where data is processed and stored, how long prompts and outputs are retained, and how you can delete them. Confirm what happens at contract termination, and whether deletion extends to backups and logs.
Some clients and protective orders restrict where data may reside or require deletion on a schedule. Having these answers in hand lets you respond when a client asks.
Red flag: indefinite retention with no deletion option.
Question 4: How is it secured, and how do we control access?
Ask for the vendor's security documentation. Look for independent attestations such as a SOC 2 report, and ask about encryption in transit and at rest. For your own configuration, check:
- Does it support single sign-on and multi-factor authentication?
- Can administrators manage users and remove access quickly?
- Does the tool respect existing document permissions, or does it index everything it can reach?
- Is there an audit log of who used it and what they did?
The third point deserves emphasis. An AI assistant connected to your document store can surface files to a user who technically has access but never knew the file existed. Clean up permissions before connecting anything.
Red flag: no administrator controls, or shared logins.
Question 5: What do the contract terms say?
Read the agreement or have counsel read it. Look for:
- Confidentiality commitments that cover your content.
- Indemnification and liability limits, and whether they are meaningful.
- Breach notification timing.
- Rights to change terms unilaterally.
- Ownership of outputs.
Vendors sometimes update terms with a notice email nobody reads. Assign someone to watch for changes.
Red flag: terms that allow the vendor to use your content for any purpose and to change terms without notice.
Turning answers into a decision
Record the answers in a one-page summary and sort the tool into one of three categories: approved for client information, approved for non-confidential work only, or not approved. Share the list with every attorney and update it when the vendor changes anything.
Pair this with a firm policy on AI use and with attorney verification of all output. The ABA has stressed that lawyers remain responsible for their work product regardless of the tool, and your state bar may have added its own guidance.
Getting help
Vendor questionnaires are tedious but do not need to be done alone. Counsel Cyber helps law firms evaluate AI products, configure them with sensible permissions, and keep an approved tool list current. If your attorneys are already asking for a particular tool, we can help you assess it before it is rolled out.