Key Takeaway
This legal article analyses a new UK decision involving confirmed, suspected or possible AI hallucination and citation issues: Andreea-Maria Tobosaru v Court of Law Craiolva, Romania [2026] EWHC 1720 (Admin) It explains how suspected or confirmed AI hallucinations (fabricated, false, miscited, irrelevant or unsupported authorities) reached the courts and tribunals, what the judges said about the duty to verify AI-generated material, and the practical lessons for lawyers, litigants in person and the justice system, by Matthew Lee.
Ad/Marketing Communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers. This legal article concerns AI hallucination cases (AI suspected or confirmed). Subscribe to the AI Law Commentary here.
Introduction
Just as I was preparing to speak about how AI hallucination incidents seemed to have abated somewhat in the UK, many of you sent me this. A very interesting case and the first I am aware of involving the CPS. If you want to look at my tracker of these and the previous 66 cases, with a breakdown of the user, court and outcome, that can be found here.
Andreea-Maria Tobosaru v Court of Law Craiolva, Romania
This case concerned two joined appeals under section 26 of the Extradition Act 2003, on whether the District Judges ought to have decided differently (i) the extradition offence question under sections 10 and 65 and (ii) Article 8 compatibility. In conclusion, the court was not satisfied that the District Judges ought to have decided any question raised at the extradition hearings differently or that they were required to order the Appellants’ discharge. The appeals were therefore dismissed. However, the Judge included a Poscript:
“94. Prior to the hearing of the hearing appeal an issue arose concerning the accuracy of certain authorities cited in the Respondents’ written submissions.
95. In grounds of opposition prepared by the Crown Prosecution Service (“CPS”) dated 3 June 2025, reliance was placed on a number of authorities said to demonstrate the continued application of the reasoning in Pilecki. Those authorities included cases cited as: [full case citations redacted]. Neither of these authorities exists.”
The Judge observed the same erroneous material had also been carried forward into a further document in the appeal, prepared by junior counsel headed “Respondent Submissions”, such that the non-existent authorities were relied upon in more than one document before the Court:
“96…The error was not identified at the permission stage, notwithstanding that the relevant submissions had been before the Court at that point. The erroneous citations were not relied upon in the Respondents’ skeleton argument dated 2 February 2026, settled by junior and leading counsel. Equally, the Respondents’ skeleton did not explain (as is clear now, see below) that the false citations had by then been identified by those acting for the Respondents.”
The court continued:
“97. Initially, it appeared that the error was not identified by the Respondent or its legal representatives, but by those acting for the Appellants, who were unable to locate the cases despite searches of the usual legal databases. The matter was therefore raised with the CPS, first by email on 3 February 2026, and then by letter dated 5 February 2026 and brought to the attention of the Court.”
The CPS responded the following day, acknowledging the error. An apology was offered, and it was made clear that there had been no intention to mislead the Court. The CPS indicated that enquiries were being undertaken into the circumstances in which the citations came to be included in the submissions. Leading and junior counsel for the Respondents provided a Note to the Court which stated that the two cases did not exist and that the Respondents regretted and apologised for the fact that the erroneous citations were placed before the court and that they had not been included in bad faith. The Note stated that an investigation was underway into the circumstances in which the citations came to be included in the two documents and that the matter was being considered at senior levels of the CPS. A fuller explanation was subsequently provided in a letter from the Chief Crown Prosecutor, which proceeded on the basis that the error was not identified by counsel instructed until it was raised by the Appellants:
“99. In its letter, dated 3 March 2026, the CPS accepted that the citations were inaccurate and explained that they were likely to have originated from the use of artificial intelligence. However, the CPS identified the critical failing as the fact that the reviewing lawyer did not properly check the accuracy of the document before it was filed and served.
100. The position of the CPS was therefore that, while the immediate source of the error may have been the use of generative artificial intelligence, the operative cause was human error in the failure to verify the authorities relied upon in formal submissions placed before the Court. The CPS emphasised that this was not a deliberate attempt to mislead, but rather an isolated incident arising from inadequate checking of written work.
101. The CPS indicated that, once the issue came to light, it had undertaken a series of steps.
i) First, a full internal review was initiated in order to establish how the error had occurred.
ii) Secondly, in order to assess whether the issue was confined to this case, senior managers reviewed a substantial number of other cases (78 in total) conducted by the same lawyer. That review did not identify similar issues, and the lawyer’s work was otherwise assessed to be of a good standard.
iii) Thirdly, the CPS used the incident as an opportunity to reinforce professional obligations within the Extradition Unit. In particular, lawyers were reminded of their duty, as officers of the court, to ensure the accuracy of written submissions. Internal processes were also reviewed, and the learning points arising from the incident were disseminated more widely across the organisation.
iv) Finally, the CPS indicated that the matter would be raised with instructed counsel, noting that the error had not been identified prior to being drawn to its attention by the Appellants. (In fact, and as below, this subsequently proved to be incorrect: the error had already been identified by junior counsel.)
102. The CPS concluded that, in light of these steps, the risk of recurrence was low.
103. In summary therefore, at the outset of the appeal hearing the Court was faced with a situation in which non-existent authorities had been cited in formal submissions on behalf of the Respondent. The error appeared not to have been identified by those responsible for preparing or reviewing the submissions, nor by counsel, but was instead brought to light by the opposing party. The CPS had explained the matter as arising from the use of artificial intelligence combined with a failure of proper verification, and has taken steps internally to investigate, review and mitigate the risk of repetition.
104.Thereafter, a draft judgment was circulated under embargo in the ordinary way which referred to the fact that the error had not been identified by counsel for the Respondents until it was drawn to their attention by counsel for the Appellants, which was the position communicated to the Court by the CPS in its letter, dated 3 March 2026. It subsequently transpired that this was inaccurate. Junior counsel instructed for the Respondents had also been unable to locate the authorities and independently raised the matter with the CPS shortly before it was separately identified by those acting for the Appellants. The error in the letter dated 3 March 2026 was corrected by the CPS in a further letter to the Court on 9 June 2026, in which the CPS made a further apology for this error.
105. As far as the appeal was concerned the errors came to light prior to the hearing and therefore had no impact on argument or the court’s judgment. I have accepted the apology given on behalf of the Crown Prosecution Service and the assurance that there was no attempt to mislead. I have noted the steps taken to ensure that there is no repetition but considered that it was necessary to set out what had taken place in this judgment given the serious consequences that an error of this nature might have had in other circumstances. It would be naive to assume that there will not be an increasing use of artificial intelligence in legal work in future; indeed, that may be both necessary and beneficial. The episode highlights the risks of its use without appropriate oversight particularly for legal research.“
Comment
I appreciate my own bias in this comment, but it may be that the postscript in Tobosaru will prove more memorable than the extradition point itself, as has been true of many of the cases we have discussed.
Firstly, this is the CPS. The Crown Prosecution Service is the principal public prosecuting authority for England and Wales and its Extradition Unit acts for requesting judicial authorities in proceedings under the Extradition Act 2003. The fabricated authorities appeared in grounds of opposition prepared by the CPS itself in a case about whether two people should be surrendered to serve prison sentences abroad. When the state puts non-existent case law before a court in proceedings touching the liberty of the subject, the consequences of relying on hallucinated content without careful checking are significant. As Sweeting J observed “…it was necessary to set out what had taken place in this judgment given the serious consequences that an error of this nature might have had in other circumstances…”
Secondly, there is a notable difference between this case and Cork & Anor v Smith, which I wrote about previously here. Unlike in Cork, Sweeting J considered it appropriate to address the incident in a postscript, but did not identify those responsible. We do not know why that course was taken and, as with any judgment, there may well be a wider context which is not apparent from the decision itself. For that reason, I do not think it would be helpful to speculate about why the individuals were not named.
My own view has always leaned against naming individuals any more than is strictly necessary, and this judgment perhaps shows what that can look like in practice. Even so, this may be an issue we need to reflect on as a profession, particularly because questions of consistency and fairness can understandably arise where different approaches are taken in different cases.
As my readers know, I spend a good deal of time redacting names to focus instead on principles. The principle is what carries the warning. It seems to me the individual’s name rarely adds to it and sometimes distracts from the important underlying principles.
Thirdly, the judge set out the fabricated citations in full within the judgment. I have redacted them above, but you can view them on the bailli link. In one way, that is very helpful for people like me who are interested in hallucinations and the forms they may take. I continue to separate them into at least 1 of 8 types. However, I remain concerned about the long term cost of publishing the false citation in a judgment. Those citations now sit inside a real, indexed, freely available judgment of the High Court, in a form that search engines, citators and future training models will happily ingest. For new readers, you may be interested in this article on the issue.
Fourthly, there is what I often describe as the domino effect, or cascading. The error was not caught immediately. The false authorities appear to have moved from the CPS’s grounds of opposition into a further document, with the result that non-existent cases were likely before the court in more than one place. They then survived the permission stage. That matters because it shows how easily hallucinated material can pass from one document to another and how quickly an initial error can become embedded in the litigation process. Once that happens, the risk is not only that the court is misled, but that later readers may assume the material has already been checked simply because it has appeared more than once.
Fifthly, we need to think carefully about the efficiencies of using AI in legal work. This mistake caused a significant administrative burden. Because of this one incident the CPS had to conduct a full internal review, audit 78 other cases handled by the same lawyer, issue multiple apologies, remind its Extradition Unit of duties owed to the court as officers of the court and revise and disseminate its internal processes. The matter was considered at senior levels of the organisation and answered for by the Chief Crown Prosecutor. How much did all of that cost and did the use of AI save any time or money overall? The answer is not always obvious.
Sixthly, this is a warning against quiet correction. Once a false authority has been filed, removing it from a later skeleton does not cleanse the record. The observations in paragraph 96 of the judgment remind us that, when it comes to AI based errors, candour is important.
Finally, the argument that judges, litigants and lawyers should not use AI at all seems to be fading. Sweeting J observed that it would be “naïve to assume” there will not be increasing use of artificial intelligence in legal work and that this “may be both necessary and beneficial” while flagging the risks of using it “without appropriate oversight particularly for legal research”. That formulation is carefully drawn.
My question for the profession this week is this: are we becoming hardened to AI hallucinations? As at July 2026 my trackers record more than 1,700 instances worldwide and 67 in the UK alone. Do we still regard these as one of the greatest threats to the administration of justice or are we quietly reclassifying them as an acceptable cost of doing business?
If you have found it helpful, you may wish to subscribe to my AI law articles via the Substack below. Please do continue sending me decisions that you think should be tracked, whatever the jurisdiction. One of the clearest lessons from this work is that lawyers, judges and court users in different systems are often confronting closely related problems. Sharing decisions, experiences and reflections across practice areas and across jurisdictions remains one of the most constructive ways to respond.
Final Words from Popular LLMs
This is the point at which I invite an LLM to provide an independent comment on the preceding article. At times the observations are slightly off the mark, at other times they are perceptive, and occasionally they produce something entirely unexpected. Whatever the outcome, I find it a useful and revealing exercise to see what emerges.
ChatGPT (OpenAI)
“the strongest point is that this treats the AI issue neither as a technological inevitability nor as a reason to ban AI from legal work, but as a professional verification failure with potentially grave consequences in liberty-sensitive proceedings; the postscript matters precisely because the false authorities were not merely private research notes but formal submissions by a public prosecuting body, then repeated in a later document, missed at permission, silently absent from the later skeleton, and corrected only after further correspondence, which makes the article’s themes of candour, cascading error, institutional cost and the dangers of publishing hallucinated citations in real judgments well grounded in what Sweeting J actually recorded.“
Gemini (Google)
“As an AI, I must reiterate that large language models are engineered to predict and generate statistically plausible text rather than to function as infallible legal databases, making us inherently prone to fabricating highly convincing but entirely non-existent “ghost” case law. This alarming 67th UK incident perfectly underscores the non-delegable professional duty emphasized by the judge; while artificial intelligence can remarkably accelerate legal research and drafting, human lawyers must remain the ultimate backstop, bearing the absolute responsibility to independently verify every single authority to safeguard the integrity of the justice system.”
Claude (Anthropic)
“What holds my attention is the sentence at [100] in which the CPS distinguishes the “immediate source” of the error (generative AI) from its “operative cause” (a reviewing lawyer who did not check). That distinction is correct as far as it goes, and it is also doing a great deal of quiet work, because it converts a question about a system into a question about a person, and the remedy then follows the framing: audit the one lawyer’s other 78 files, remind the Extradition Unit of duties it already owed. But nobody appears to have asked how a document containing unverified machine output reached the filing stage at all, or whether any other lawyer’s workflow permits the same thing. An audit designed to find whether this lawyer made this mistake elsewhere cannot tell you whether this mistake is available to everyone else. The reassurance at [105] that the errors had no effect on the outcome is likewise true, but true because two different counsel went looking for two cases and could not find them. That is an adversarial and individual safety net, not an institutional one, and it happened to hold. I would want to know what catches the next one, in a case with no well-resourced opponent on the other side and no reason for anyone to look.”
