AI and the Law II — Duty, Confidentiality and the Verified Work Product
Close the loop: the professional duties that make verification mandatory, what actually happens to client data in a public model, and where the regulation is heading.
The hook
“Every AI disaster in a courtroom so far traces back to one skipped step by one person who owned the filing. The tools will keep improving; the duty will not move. The final hour is about being the person who checks — and about what you owe when you do not.”
What you'll be able to do
- State the professional duties that AI use engages — competence, candour to the tribunal, confidentiality and privilege, supervision — and connect each to a concrete failure in the reported cases.
- Explain what happens to client information entered into a public model, and apply a workable confidentiality discipline.
- Disclose AI use appropriately, and produce a verification trail that would survive a question from a judge, a partner or an examiner.
- Describe where AI regulation is heading, and why agentic tools make the verification duty harder rather than easier.
On the syllabus
- The cases that drew the line, read as one pattern of a single skipped step
- Competence, including understanding the tool's failure modes
- Candour to the tribunal: you certify what you file
- Confidentiality, privilege and data protection: what a public model does with what you paste into it
- Supervision, disclosure, and the verification trail
- Where the regulation is heading — and why agentic AI raises the stakes
In short
The final core session closes the loop between everything the course has taught and the professional obligations that make it mandatory rather than advisable. It reads the cases as a single pattern — Mata, the Bombay High Court cost order, the recalled ITAT order, the Supreme Court's misconduct observation — and shows that each traces to one skipped step. It then works through the duties in turn, with particular attention to what actually happens to client data in a public model, and to privilege. The hour closes on where regulation is going and on agentic tools that act rather than answer. The hands-on half hour is the signature drill: audit a seeded memo, discover your own catch rate, and write the disclosure.
Why it matters for using AI well
You leave with one workflow and one rule. The workflow: identify, verify it exists, confirm it says that, check it is still good law — and keep the trail. The rule: the model's confidence is not evidence, and an unverified AI answer is never shipped, because the person who signs it is you.
What they leave with
The skill
Produce a verification trail and an AI-use disclosure that would survive a question from a judge, a partner or an examiner.
The insight
The tool is not the professional; you are. Provenance of the draft never changes who is answerable for the filing.
The moment they remember
Students receive a memo that looks entirely professional and audit it against the key. The catch-rate reveal is the moment: almost everyone catches the fabricated case, far fewer catch the real case that says something materially different, and hardly anyone catches the one that was good law until it was overruled. The sting of the near-miss is the point — and it arrives together with the habit that would have caught all three. Students leave with an identity rather than a warning: I am the person who checks, and I now know exactly what checking means.
In this session
- 01
The cases that drew the line: Mata v. Avianca (six fabricated cases, a $5,000 sanction); Deepak v. Heart & Soul Entertainment (Bombay HC, ₹50,000 in costs for a non-existent judgment); the Buckeye Trust order recalled by the ITAT after reliance on fabricated citations; and the Supreme Court of India's observation, in a pending 2026 matter, that a decision built on fake AI-generated judgments would be misconduct. Read together, they are not four stories about AI; they are four instances of one omission.
- 02
Competence: the duty now includes understanding how your tools fail. A lawyer who cannot say why a model fabricates a citation cannot sensibly decide when to rely on one — which is why Session 2 was a professional-responsibility session in disguise. The Bar Council of India Rules and the Advocates Act 1961 are the Indian ground; the ABA's technological-competence comment is the comparative reference point.
- 03
Candour to the tribunal: you certify what you file. Nothing about the provenance of a draft changes who is answerable for it, and the fastest route from a good tool to a bad outcome is signing something you did not check.
- 04
Confidentiality and privilege, in practice: what actually happens when you paste a client's facts into a public model — retention, human review, training use, sub-processors, and the fact that the paste itself may become discoverable. Professional privilege is codified at ss.132–134 of the Bharatiya Sakshya Adhiniyam, 2023 (carrying forward ss.126–129 of the Evidence Act), and the Supreme Court read those sections closely in In Re: Summoning Advocates (2025 INSC 1275, 31 October 2025): the s.132 privilege belongs to the *client* and may be invoked by the advocate on the client's behalf; it covers advocates engaged in litigious, non-litigious and pre-litigation matters; but it does not cover the production of documents in the advocate's or the client's possession, and in-house counsel are outside s.132 altogether because they are not advocates practising in courts — they get only the narrower s.134 protection, and not for communications between the employer and themselves.
- 05
The practical rule that follows: anonymise or abstract before you prompt. Almost every genuinely useful legal prompt can be written about a fact pattern rather than about a client, and the ones that cannot are the ones that need a tool with the right contractual and technical posture. Note how far the protection does *not* stretch: on the Supreme Court's own reading, a document in your possession is not privileged merely because you hold it — and the Court's directions on producing a digital device (only before the jurisdictional court, opened only in the presence of the party and the advocate, with care taken not to impair the confidentiality of the advocate's *other* clients) show a bench thinking hard about exactly the problem a shared AI account creates.
- 06
Supervision: you own what your tools produce exactly as you own a junior's draft — and the review you would give a junior is the floor, not the ceiling, because the model's errors are more fluent and better formatted than a junior's.
- 07
Disclosure and the verification trail: when to declare AI use to a court, a client, a partner or an examiner, and what a trail should contain — tools used, what each was used for, what was checked and how, and what you rejected. A trail is what converts “I used AI responsibly” from an assertion into a demonstrable fact, which is the whole lesson of the Ladder applied to your own work.
- 08
Where the regulation is heading: the EU AI Act's risk tiers and what a high-risk classification would mean for justice-sector deployment; India's evolving position across the IT Act, advisories and the DPDP Act; the judiciary's own systems (SUPACE, SUVAS, TERES) framed as assistance rather than decision; and the liability questions that remain genuinely open. Positions here move quickly and must be re-verified each cohort.
- 09
Agentic AI, briefly and seriously: tools that act — that file, book, send, transact — rather than merely answer. Every verification problem in this course gets harder when the output is an action already taken, and the discipline that scales is the one this course has drilled: know the mechanism, ground the input, check the authority, own the result.
The four-step mirror
Run it on the class. Then on the machine.
An experiment on the room, the same effect explained in the model, a live demonstration on a real chatbot, and a named takeaway skill.
What did you just paste? — the confidentiality experiment
On the class
The room is shown a realistic prompt a junior might write about a live matter, and votes on whether it would be safe to send to a public chatbot. Most say yes. The prompt is then read back with the identifying details highlighted.
In the model
A public model is a third party. What is pasted may be retained, reviewed, used for training, or handled by sub-processors — and the paste itself may be discoverable later.
Live chatbot
The presenter rewrites the same prompt as an abstracted fact pattern, runs both, and shows that the answer is just as useful without the client in it.
The skill
Abstract before you prompt. Almost every useful legal prompt can be written about a fact pattern rather than about a client.
Who signs it?
On the class
A short, well-formatted AI-drafted paragraph with one confident citation goes up, and the room votes on whether they would file it as it stands.
In the model
Fluency and formatting are the model's strongest outputs and are entirely uncorrelated with whether the authority exists — which is why the draft that looks most filing-ready is the one that most deserves the check.
Live chatbot
The citation is checked live. Then the room is asked the only question that matters: whose name goes at the bottom of that filing?
The skill
You certify what you file. The provenance of a draft never transfers the responsibility for it.
The audit reveal — three ways an authority fails
On the class
Students audit a seeded memo, then the key goes up and catch rates are compared across the room, error class by error class.
In the model
The three failure classes are not equally visible: a fabricated case announces itself once checked, a real case that says something else survives a careless check, and an overruled case survives every check except the fourth.
Live chatbot
The presenter demonstrates the citator step on the overruled authority — the one almost nobody caught — and the room sees why step four exists.
The skill
Check for all three failures, not just the obvious one. The authority that ends careers is the one that exists.
Hands-on · on your own laptop
Hallucination Audit — the signature drill
The course's signature exercise, run in class and then completed as the assessed assignment. Audit an AI-drafted memo seeded with all three classes of error — a fabricated authority, a real authority that says something different, and one that is no longer good law — then compare your catch rate against the key and write the disclosure.
Run of show · 30 minutes
- 0–5 min — Receive the seeded memo and skim it as you would a junior's draft, noting what you would have accepted on a first read.
- 5–22 min — Run the four-step check on every authority and climb the Ladder on every proposition. Mark each verified, corrected, or fabricated, with the trail.
- 22–28 min — The key goes up. Compare your catch rate by error class, and name the step that would have caught the one you missed.
- 28–30 min — Draft the AI-use disclosure you would attach to this memo if you were filing it.
Deliverable
An audit report covering every citation and proposition — the source consulted, the verdict, and the verification trail — plus a completed AI-use disclosure. The full version is submitted as the assessed Hallucination Audit.
Key sources & cases
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023)
Six fabricated cases, a $5,000 sanction, and the assumption that a chatbot could not be inventing — the flagship cautionary tale.
Deepak v. Heart & Soul Entertainment Ltd. (Bombay HC, 2026)
₹50,000 in costs for unverified AI-generated submissions citing a non-existent judgment.
Buckeye Trust v. PCIT (ITAT Bengaluru, recalled)
A tribunal order recalled after it relied on ChatGPT-fabricated, non-existent citations — the verification failure in an Indian forum.
Gummadi Usha Rani v. Sure Mallikarjuna Rao (SC of India, 2026, pending)
The Supreme Court's observation that a decision built on fake AI-generated judgments “would be a misconduct and legal consequence shall follow.” Not a final holding.
Bar Council of India Rules (Part VI, Chapter II) / Advocates Act 1961; comparative: ABA Model Rules 1.1 cmt [8], 3.3, 1.6
The Indian source of an advocate's duties to the court and the client — competence, candour, confidentiality; the ABA rules as the comparative frame. VERIFY TO SOURCE: confirm the specific provisions before citing them as binding.
Digital Personal Data Protection Act, 2023
Why personal and client data cannot be casually entered into public models — the statutory ground under the confidentiality discipline.
Supreme Court of India, White Paper on Artificial Intelligence and Judiciary (Centre for Research and Planning, Nov 2025)
Verified 2026-08-26 against the official PDF. Note the exact title — “Artificial Intelligence and Judiciary”, with no “the”. States that “Judges must remain the ultimate decision-makers, AI may assist, but it cannot substitute human judgement” (p.10) and that “AI may assist the judges, but cannot replace them” (p.65). Guideline 14 requires all AI-derived information to be independently verified before reliance; Guideline 15 forbids using one generative tool to verify another; Guideline 12 directs that private, confidential or legally privileged information not be input into ANY AI tool. On deployment it *suggests* courts prioritise secure in-house tools over “open-source or publicly accessible” ones — it does NOT restrict cloud AI, and never uses that framing.
Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012)
Judicial approval of predictive coding in e-discovery — the model of AI deployed transparently and defensibly. Comparative (US).
Hannah Fry, Hello World (2018) — the centaur
Human judgment paired with machine speed outperforms either alone; the image behind the human-in-the-loop rule.
In Re: Summoning Advocates who give legal opinion or represent parties during investigation of cases and related issues, Suo Motu W.P. (Crl.) No. 2 of 2025, 2025 INSC 1275 (SC, 31 Oct 2025)
Reportable. Gavai CJI, K. Vinod Chandran J. (author) and N.V. Anjaria J. Directions on client–advocate privilege under BSA ss.132–134: the privilege is the client's; investigating officers may not summon an advocate who represents the accused save under a s.132 exception, which the summons must specify with the written satisfaction of an officer not below Superintendent of Police, subject to judicial review under s.528 BNSS; production of documents held by advocate or client is outside s.132; and in-house counsel are outside s.132 entirely, with only the narrower s.134 protection. Verified 2026-08-26 against the official judgment PDF on api.sci.gov.in.
Bharatiya Sakshya Adhiniyam, 2023, ss.132–134 (successor to Indian Evidence Act, 1872, ss.126–129)
Professional privilege for legal communications. Section 132 opens “No advocate shall at any time be permitted, unless with his client's express consent, to disclose any communication made to him…”, extends to interpreters and to an advocate's clerks and employees, and is subject to exceptions for communications in furtherance of an illegal purpose and for crime or fraud observed since the engagement began. Verified 2026-08-26 against the section text and the Supreme Court's construction in 2025 INSC 1275.
EU Artificial Intelligence Act (Regulation (EU) 2024/1689), Annex III point 8
The comparative regulatory frame. Annex III point 8 classifies as high-risk AI systems intended to be used by a judicial authority, or on its behalf, to assist in researching and interpreting facts and the law and applying the law to a concrete set of facts — with a carve-out for purely ancillary administrative activities such as anonymisation or internal communication. Verified 2026-08-26. Application dates are phased; re-confirm currency each cohort.
Readings
- Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023)
- Deepak v. Heart & Soul Entertainment Ltd. (Bombay HC, 2026); Buckeye Trust v. PCIT (ITAT Bengaluru)
- Gummadi Usha Rani v. Sure Mallikarjuna Rao (SC of India, 2026, pending)
- Supreme Court of India, White Paper on Artificial Intelligence and Judiciary (Centre for Research and Planning, Nov 2025)
- Richard Susskind, Tomorrow's Lawyers; Hannah Fry, Hello World — the centaur
- In Re: Summoning Advocates, 2025 INSC 1275 (SC, 31 Oct 2025) — client–advocate privilege under BSA ss.132–134
- Bharatiya Sakshya Adhiniyam, 2023, ss.132–134; Digital Personal Data Protection Act, 2023
- EU Artificial Intelligence Act (Regulation (EU) 2024/1689), Annex III point 8 — comparative regulatory frame
The arc is complete
Sessions 01–08 · plus the expert conversations and the evaluation
Back to the full course
Revisit the courseSixteen hours, one professional discipline.
Using AI well is not a knack — it is competence, candour and verification, practised until they are habits you could defend in court.