---
title: "Test a sample of chatbot transcripts for confidential data and wrong law"
url: "https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law"
type: "task"
site: "AI Law Framework"
id: "A43-A20260923-3"
practice_area: "Client-facing & access-to-justice AI tools"
tier: "yellow"
lawyer_review: "required"
last_reviewed: "2026-10-22"
---

# Test a sample of chatbot transcripts for confidential data and wrong law

> From a permitted sample, list transcripts that include identifying facts, hallucinated rules or missed escalations.

Canonical page: https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law  
Last reviewed: 2026-10-22

Site index for agents: https://ailawframework.com/llms.txt

## Key facts

| Fact | Value |
| --- | --- |
| Practice area | [Client-facing & access-to-justice AI tools](https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools.md) |
| Human involvement | Yellow: Lawyer and AI draft together |
| How AI is used | From a permitted sample, list transcripts that include identifying facts, hallucinated rules or missed escalations. |
| What the lawyer must review | Spot-check every flagged transcript Keep supervising: a clean sample is not a licence to stop |
| Written for / versions | 1 jurisdiction versions below |

## Checking plan

Spot-check every flagged transcript. A clean sample is not a licence to stop supervising.

- What the AI does: From a permitted sample, list transcripts that include identifying facts, hallucinated rules or missed escalations.
- What the lawyer does: Set the scope, verify the evidence, resolve exceptions and approve the final output for its intended use.
- Basis: Suggested checking approach based on the task guidance. Complete the assessment for the actual tool, information and jurisdiction.

## Checks the lawyer makes

1. Confirm the output against the client’s instructions and the complete relevant source material, including omissions or contradictions.
2. Verify every relied-on citation, figure and name against the source document.
3. The responsible lawyer approves the exact version before use.

## Sources

- [CLINIC: How to Safely Incorporate AI Into Your Immigration Practice, 17 December 2025](https://www.cliniclegal.org/resources/how-safely-incorporate-ai-your-immigration-practice) (checked 2026-09-23)

## Forms and precedents for this task

| Form or precedent | Kind | AI level | Summary |
| --- | --- | --- | --- |
| [Chatbot transcript sample review worksheet](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/chatbot-transcript-sample-review.md) | Form | Yellow | Working sheet for a lawyer reviewing a sample of client-facing chatbot transcripts for wrong law, advice-line breaches, missed escalations and personal information that should not have been… |
| [Client-facing AI incident and near-miss report](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/client-facing-ai-incident-report.md) | Form | Red | Internal report when a client-facing assistant gives tailored or wrong information, misses a safety or urgency cue, mishandles personal information, or is the subject of a complaint.… |
| [Note escalating a client-facing tool transcript to a lawyer (United States)](https://ailawframework.com/precedents/client-facing-and-access-to-justice-ai-tools/us-chatbot-escalation-note.md) | Memo | Yellow | An internal note that lifts a transcript out of a client-facing tool and records why a lawyer must take the next step. It does not answer the person. |

Signed-in accounts can read the full text of each one.

## Jurisdiction versions

| Jurisdiction | Level |
| --- | --- |
| [United States](https://ailawframework.com/obligations/client-facing-and-access-to-justice-ai-tools/test-a-sample-of-chatbot-transcripts-for-confidential-data-and-wrong-law/united-states.md) | Yellow |

*Practice guidance for planning and review. Confirm current primary sources and the requirements of the actual matter. This is general information, not legal advice.*