Ad/Marketing Communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers. This legal article concerns AI Law.
This case is not new and was widely reported at the time: Harber v HMRC (First-tier Tribunal) [2023] UKFTT 1007 (TC).
I remember reading it in 2023 and found it astonishing then. I thought I would revisit it briefly as the Tribunal Judge made clear observations assisted by the US case of Mata v Avianca 22-cv-1461(PKC), a case probably deserving its own dedicated blog post.
What Happened?
The appellant disposed of a property but failed to inform HMRC of the capital gains tax owed, resulting in a penalty. In her defence, she presented nine previous tribunal cases, seemingly showing that similar cases had succeeded due to reasonable excuses like ignorance of the law or mental health difficulties. However, these decisions weren’t genuine. They’d been entirely fabricated by an AI tool, such as ChatGPT.
The tribunal quickly realised something was amiss. Neither HMRC nor the tribunal could verify the existence of these cases, and upon investigation, it became clear they were not real decisions but plausible fictions generated by AI. The appellant had been unaware of this deception, trusting information provided by an acquaintance.
Ultimately, the tribunal ruled against the appellant, finding she did not have a reasonable excuse for her initial error. Importantly, their decision was independent of the fictitious cases.
AI’s Role in This Case
AI, specifically large language models like ChatGPT, played a central role here. The purported “precedents” were entirely AI-generated, a phenomenon commonly discussed on this blog and known as “hallucinations” i.e. AI-created misinformation that appears credible yet is entirely false. The tribunal explicitly noted the convincingly plausible yet fictional nature of these cases, making them particularly concerning.
Comment
The Judge’s observations were extremely helpful in highlighting why AI hallucinations are troubling, providing practical guidance on identifying AI-generated documents:
(5) The Tribunal was also assisted by the US case of Mata v Avianca 22-cv-1461(PKC), in which two barristers sought to rely on fake cases generated by ChatGPT. Like Mrs Harber, they placed reliance on summaries of court decisions which had “some traits that are superficially consistent with actual judicial decisions”. When directed by Judge Kastel to provide the full judgments, the barristers went back to ChatGPT and asked “can you show me the whole opinion”, and ChatGPT complied by inventing a much longer text. The barristers filed those documents with the court on the basis that they were “copies…of the cases previously cited”. Judge Kastel reviewed the purported judgments and identified “stylistic and reasoning flaws that do not generally appear in decisions issued by United States Courts of Appeals”.
(6) Unlike the barristers, Mrs Harber did not take the further step of asking ChatGPT for full judgments, so we had only the less detailed summaries. These had fewer identifiable flaws than those which Judge Kastel had identified in the longer full decisions with which he was provided. However, we noted that all but one of the cases in the Response related to penalties for late filing, and not for failures to notify a liability, which was the issue in Mrs Harber’s case. There were also the following stylistic points:
(a) The American spelling of “favor” in the sentence “The First-tier Tribunal (Tax Chamber) found in their favor” which appears in six of the nine cited cases.
(b) The frequent repetition of identical phrases: three of the four ignorance of the law” cases say that “the taxpayer argued that they had not been aware of the requirement to file a tax return as they had not received any correspondence from HMRC”. Two of the “mental health” cases say that “the taxpayer argued that their mental health condition, combined with other factors, had made it impossible for them to submit the return on time” and the other two both say “the taxpayer argued that they had a reasonable excuse for the late filing due to their mental health condition, which had prevented them from being able to manage their affairs effectively”.
So, in summary, the following principles may help identify false AI-generated authorities:
- Check for unusual language or repeated phrases (e.g., incorrect regional spellings or unnatural repetition).
- Verify citations thoroughly (ensure accuracy of dates, references, and tribunal details).
- Ensure context matches precisely (AI often cites similar but irrelevant legal issues).
- Cross-reference official databases (such as BAILII or court websites).
- Review complete judgments, not just summaries (to identify inconsistencies or reasoning flaws).
The Tribunal then explained very clearly why relying on AI hallucinations is far from trivial and carries serious consequences:
“Although we have accepted that Mrs Harber did not know the AI cases were not genuine, we reject her submission that this did not matter because the Tribunal had decided other reasonable excuse cases on the basis of ignorance of the law and/or mental health issues. We instead agree with Judge Kastel, who said on the first page of his judgment (where the term “opinion” is synonymous with “judgment”) that:
“Many harms flow from the submission of fake opinions. The opposing party wastes time and money in exposing the deception. The Court’s time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents. There is potential harm to the reputation of judges and courts whose names are falsely invoked as authors of the bogus opinions and to the reputation of a party attributed with fictional conduct. It promotes cynicism about the legal profession and the…judicial system. And a future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity.”
We acknowledge that providing fictitious cases in reasonable excuse tax appeals is likely to have less impact on the outcome than in many other types of litigation, both because the law on reasonable excuse is well-settled, and because the task of a Tribunal is to consider how that law applies to the particular facts of each appellant’s case. But that does not mean that citing invented judgments is harmless. It causes the Tribunal and HMRC to waste time and public money, and this reduces the resources available to progress the cases of other court users who are waiting for their appeals to be determined. As Judge Kastel said, the practice also “promotes cynicism” about judicial precedents, and this is important, because the use of precedent is “a cornerstone of our legal system” and “an indispensable foundation upon which to decide what is the law and its application to individual cases”, as Lord Bingham’s said in Kay v LB of Lambeth [2006] UKHL 10 at [42]. Although FTT judgments are not binding on other Tribunals, they nevertheless “constitute persuasive authorities which would be expected to be followed” by later Tribunals considering similar fact patterns, see Ardmore Construction Limited v HMRC [2014]”
Promotion of cynicism about judicial precedents struck me most in this judgment. In recent conversations, it has been suggested there’s been a notable increase in AI-generated documents in court proceedings. Given the rising adoption and awareness of AI, this wouldn’t surprise me, nonetheless, that is not something that can be accurately checked so I approach it with some caution. I wonder if, given the stressful and often overloaded nature of some courts, AI-generated documents might slip through initial judicial scrutiny, passing the “smell test” simply because hallucinated cases can appear convincingly authentic. This may not only lead to unjust decisions but may also start to develop certain conventions, practice or “persuasive” decisions if they are not carefully monitored and spotted at an early stage.
In this context, a crucial legal skill for the future will be the ability to quickly identify AI-generated hallucinations. Judges and lawyers will need to clearly recognise and understand the risks posed by these artificial inaccuracies within legal documents. As AI becomes more integrated into the legal profession, increased vigilance and adaptability will be essential. The key question now is how swiftly courts will learn to reliably distinguish between genuine precedents and artificial fabrications or will AI get to the point where hallucinations no longer happen and such vigilance will becomes less necessary.
