---
title: "Test a sample of chatbot transcripts for client information and wrong law"
url: "https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law/united-states"
type: "task_jurisdiction_version"
site: "AI Law Framework"
id: "A43-A20260923-3-US"
practice_area: "Client-facing & access-to-justice AI tools"
tier: "yellow"
lawyer_review: "required"
jurisdictions: ["US"]
last_reviewed: "2026-10-22"
---

# Test a sample of chatbot transcripts for client information and wrong law

> On a closed sample the lawyer selects, flag turns that look like legal advice, citations, personal data or a promise to file or contact a court.

Canonical page: https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law/united-states  
Last reviewed: 2026-10-22

Site index for agents: https://ailawframework.com/llms.txt

## Key facts

| Fact | Value |
| --- | --- |
| Practice area | [Client-facing & access-to-justice AI tools](https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools.md) |
| Human involvement | Yellow: Lawyer and AI draft together |
| How AI is used | On a closed sample the lawyer selects, flag turns that look like legal advice, citations, personal data or a promise to file or contact a court. |
| What the lawyer must review | You read every flagged turn and a random slice of the rest. Opinion 512: uncritical reliance on generated output can breach competence, and client information must not be put into a tool that discloses it. California's guidance: competence is more than deleting hallucinations, and candour to a tribunal cannot be delegated if a transcript is later used in court. NYC Bar Formal Opinion 2024-5 covers the same confidentiality, competence and supervision duties. |
| Written for | United States |
| Applies to | [Test a sample of chatbot transcripts for confidential data and wrong law](https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law.md) |

## Checking plan

- What the AI does: On a closed sample the lawyer selects, flag turns that look like legal advice, citations, personal data or a promise to file or contact a court.
- Basis: Checked against the linked sources on 22 October 2026. Confirm the current instrument before reliance.

## Notes

- Do not paste live transcripts into a public model. The sample is client information under Model Rule 1.6.
- A citation in a transcript is unchecked until you open it. The tool that flagged it has not verified it.

## Sources

- [ABA Formal Opinion 512, Generative Artificial Intelligence Tools (29 July 2024)](https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/ethics-opinions/aba-formal-opinion-512.pdf) (checked 2026-10-22)
- [State Bar of California, Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law (2026)](https://www.calbar.ca.gov/Portals/0/documents/ethics/Generative-AI-Practical-Guidance.pdf) (checked 2026-10-22)
- [New York City Bar Formal Opinion 2024-5: Generative AI in the Practice of Law](https://www.nycbar.org/reports/formal-opinion-2024-5-generative-ai-in-the-practice-of-law/) (checked 2026-10-22)

## Forms and precedents for this task

| Form or precedent | Kind | AI level | Summary |
| --- | --- | --- | --- |
| [Chatbot transcript sample review worksheet](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/chatbot-transcript-sample-review.md) | Form | Yellow | Working sheet for a lawyer reviewing a sample of client-facing chatbot transcripts for wrong law, advice-line breaches, missed escalations and personal information that should not have been… |
| [Client-facing AI incident and near-miss report](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/client-facing-ai-incident-report.md) | Form | Red | Internal report when a client-facing assistant gives tailored or wrong information, misses a safety or urgency cue, mishandles personal information, or is the subject of a complaint.… |
| [Note escalating a client-facing tool transcript to a lawyer (United States)](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/us-chatbot-escalation-note.md) | Memo | Yellow | An internal note that lifts a transcript out of a client-facing tool and records why a lawyer must take the next step. It does not answer the person. |

Signed-in accounts can read the full text of each one.

*Practice guidance for planning and review. Confirm current primary sources and the requirements of the actual matter. This is general information, not legal advice.*