Test a sample of chatbot transcripts for confidential data and wrong law
Key facts
- Human involvement
- YellowClose supervision
- How AI is used
- From a permitted sample, list transcripts that include identifying facts, hallucinated rules or missed escalations.
- What the lawyer must review
- Spot-check every flagged transcript
- Keep supervising: a clean sample is not a licence to stop
- Written for / version
- US
Open a jurisdiction version of this task to see which instruments apply: US
Who does what
| Role | What they do |
|---|---|
| Assistance | From a permitted sample, list transcripts that include identifying facts, hallucinated rules or missed escalations. |
| Lawyer | Set the scope, supervise the work closely, verify every part relied on, resolve omissions and exceptions, and approve the exact output. |
Checklist
- Confirm the output against the client’s instructions and the complete relevant source material, including omissions or contradictions.
- Verify every relied-on citation, figure and name against the source document.
- The responsible lawyer approves the exact version before use.
Forms and precedents for this task
| Form or precedent | Kind | AI use | Summary |
|---|---|---|---|
| Chatbot transcript sample review worksheet | Form | AI use Yellow | Working sheet for a lawyer reviewing a sample of client-facing chatbot transcripts for wrong law, advice-line breaches, missed escalations and personal information that should not have been collected or repeated. |
| Client-facing AI incident and near-miss report | Form | AI use Red | Internal report when a client-facing assistant gives tailored or wrong information, misses a safety or urgency cue, mishandles personal information, or is the subject of a complaint. |
Signed-in accounts can read the full text of each one. All precedents.
Sources
Confirm the required checking with the responsible lawyer before work starts. The checking approach does not establish permission to use AI.