Decoding the Answer: Verification & the Duty to Check
Verification is not optional polish — it is a professional duty, and the line between using AI well and being sanctioned.
The hook
“A fluent AI answer arrives dressed as binding proof. Before you cite it, ask the question that saved no one in Mata v. Avianca: how do I actually know this is true?”
What you'll be able to do
- Build a working verification toolkit for legal AI output — the Ladder of Misinference and SIFT/lateral reading, adapted to legal sources.
- Map verification onto the professional-ethics duties: competence (including technological competence), candour to the tribunal, confidentiality, and supervision.
- Recognize the citator (Shepardizing / KeyCite / SCC 'Note Up' / Manupatra citation analysis) as the lawyer's existing verification analogue and apply it to AI-generated claims.
- Verify every citation and proposition in an AI-drafted memo to source, producing a documented audit trail.
In short
This is the module where "decode the answer" becomes a duty to verify. Students learn to climb the Ladder of Misinference on legal AI output (statement → fact → authority → binding authority → settled law) and to apply SIFT/lateral reading and traditional citator discipline as professional verification. Verification is tied directly to the duties of competence, candour, confidentiality, and supervision — and to the Supreme Court of India's White Paper restrictions on GenAI for case data. The lab is the course's signature graded assignment: a Hallucination Audit.
The AI bridge
Verification is not optional polish — it is a professional duty and the difference between using AI well and being sanctioned. Knowing how to climb the Ladder and run SIFT/citator checks on AI output is the skill that keeps you out of Mata.
In this module
- 01
Apply the Ladder of Misinference to legal AI output: a chatbot's answer is a statement; promote it only as far as the evidence allows — statement → fact → authority → BINDING authority → settled law. A hallucinated citation is a statement costumed as binding proof; verification is the act of refusing the costume.
- 02
Adapt Caulfield's SIFT and lateral reading to legal verification: Stop, Investigate the source, Find better/original coverage, Trace claims to the original — i.e., never accept the AI's summary of a case; open the case itself.
- 03
Treat the citator as the professional verification analogue students may already half-know: Shepardizing / KeyCite, SCC Online's 'Note Up', Manupatra citation analysis — tools that tell you whether an authority exists, is good law, and still stands. The same discipline must now be applied to anything an AI hands you.
- 04
Verification is a professional DUTY, not optional polish: competence (including technological competence — you must understand the tool's failure modes), candour to the tribunal (you certify what you file), and supervision (you own what your tools and juniors produce).
- 05
Confidentiality as a live verification constraint: never paste privileged or client data into public LLMs; the SC White Paper restricts cloud GenAI for case data and is especially protective of children and sexual-violence survivors — verify the tool before you verify the answer.
- 06
Use the Calling Bullshit triage on any confident claim: who is telling me this, how do they know, and what are they selling? — applied to a model that sounds authoritative but has no stake in being right.
- 07
The career stakes are concrete: Mata v. Avianca, the SC of India's warning (in a pending 2026 SLP) that a decision built on fake AI-generated judgments would be misconduct, the Bombay HC cost order, and the recalled Buckeye Trust ITAT order all trace back to one missing step — verification to source.
The interactive demos
Every idea is a Mirror Move
Run it on the room, show it inside the machine, prove it live on a real AI, then name the skill.
Climb the Ladder / run SIFT on a live AI answer
On us
Pose a real legal question to the room and surface their instinct to accept a confident, well-formatted answer — the same over-trust the Mata lawyer showed when he assumed ChatGPT could not be fabricating cases.
In the machine
An LLM delivers statements in the register of settled authority; fluency and formatting signal certainty the model has no basis for. The answer arrives pre-promoted to the top of the Ladder.
Live AI
Put a real legal question to a chatbot in front of the room, then climb the Ladder and run SIFT on its answer live: stop, investigate each cited source, find the original, trace every proposition — demoting each claim to the rung the evidence actually supports.
The skill
Never let a confident AI answer self-certify. Climb the Ladder and trace every citation and proposition to its original source before it enters your work — verification is the duty, not the model's confidence.
The lab
Hallucination Audit (the signature graded assignment)
Students receive an AI-drafted legal memo seeded with errors — fabricated citations, misstated holdings, and overpromoted propositions — and verify every citation and proposition to source, using citators and lateral reading. Each claim is placed on the correct rung of the Ladder and either confirmed against the original, corrected, or flagged as fabricated.
Deliverable
An audit report documenting every citation and proposition checked, the source consulted, the verdict (verified / corrected / fabricated), and the verification trail — graded (see assessment scheme, 20%).
Key sources & cases
Caulfield's SIFT (Stop, Investigate, Find, Trace)
CC BY 4.0; the lateral-reading method, adapted here to legal verification — never accept the AI's summary; trace every claim to the original source.
Edmans, May Contain Lies (the Ladder of Misinference)
The course-wide answer-decoder: statement → fact → data/authority → evidence → proof; here applied to legal AI output as statement → fact → authority → binding authority → settled law.
Bergstrom & West, Calling Bullshit
The triage questions for any confident source: who's telling me this? how do they know? what are they selling? — turned on a fluent but disinterested model.
Supreme Court of India White Paper on AI and the Judiciary (Nov 2025)
Warns of hallucinations, bias, and confidentiality (esp. children and sexual-violence survivors); restricts cloud GenAI for case data; permits only approved tools.
Advocates Act / BCI professional-conduct duties
The Indian source of the duties of competence, candour, and confidentiality that make verification mandatory rather than discretionary.
ABA Model Rules 1.1[8] (competence), 3.3 (candour), 1.6 (confidentiality)
Comparative framing: technological competence, candour to the tribunal, and confidentiality as the ethics scaffolding for AI verification.
The recalled ITAT order
An Income Tax Appellate Tribunal order recalled after reliance on fictitious case law — the verification failure made concrete in an Indian forum.
Readings
- Edmans, May Contain Lies (2024) — the Ladder of Misinference
- Bergstrom & West, Calling Bullshit (2020)
- Caulfield's SIFT method (lateral reading; CC BY 4.0)
- Supreme Court of India White Paper on AI and the Judiciary (Nov 2025)
- Advocates Act / BCI professional-conduct duties; comparative ABA Model Rules 1.1[8], 3.3, 1.6
- Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023); the recalled ITAT order
Next module
Module 05 / 08
Prompting Like a Lawyer: Computational Thinking as Prompt-Craft
promptBring a credit-bearing AI course to your students
A 16-hour course that treats using AI well as a professional duty — one a council can approve, and a graduate can defend in court.