Compare the system's outputs on a test set for disparate results
Key facts
- Human involvement
- YellowClose supervision
- How AI is used
- From a lawyer-approved test set, tabulate outcomes by the attributes the lawyer has listed and mark unexplained differences.
- What the lawyer must review
- Treat differences as a reason to enquire, not a discrimination finding
- Handle protected attributes lawfully
- Written for / version
- US
Open a jurisdiction version of this task to see which instruments apply: US
Who does what
| Role | What they do |
|---|---|
| Assistance | From a lawyer-approved test set, tabulate outcomes by the attributes the lawyer has listed and mark unexplained differences. |
| Lawyer | Set the scope, supervise the work closely, verify every part relied on, resolve omissions and exceptions, and approve the exact output. |
Checklist
- Confirm the output against the client’s instructions and the complete relevant source material, including omissions or contradictions.
- Verify every relied-on citation, figure and name against the source document.
- The responsible lawyer approves the exact version before use.
Forms and precedents for this task
| Form or precedent | Kind | AI use | Summary |
|---|---|---|---|
| Record of a test for different results across groups | Checklist | AI use Yellow | Record of a test, designed by the principal, comparing a system's outputs across groups the principal has chosen, with the outputs tabulated and the conclusion left to the principal. |
Signed-in accounts can read the full text of each one. All precedents.
Sources
Confirm the required checking with the responsible lawyer before work starts. The checking approach does not establish permission to use AI.