Ad/Marketing Communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers.
20th UK AI Hallucinations/Fabricated Citations incident
Introduction
Today’s legal article is brief but important, because we now have the 20th UK AI Hallucinations/Fabricated Citations incident and matters do not seem to be improving. I have more to write up and, once that is complete, we will need to consider some important statistics. I am also co authoring an article on the Australian position, which is taking some time, but will hopefully provide interesting and comparable insights.
Internationally, the total is approaching six hundred, which I will explore in a later piece along with some concerning statistics. My articles on the previous nineteen incidents can be found from here.
Peters v Driver and Vehicle Standards Agency
Employment Judge J Connolly 3306403/2024
This case concerned an employment claim brought against the Driver and Vehicle Standards Agency that included allegations of discrimination. The complaints covered direct discrimination, indirect discrimination, discrimination arising from disability and a failure to make reasonable adjustments. The full procedural background and outcome can be found in the published judgment here.
How Artificial Intelligence Featured
From paragraph 5, the Tribunal outlined the material provided for the hearing. The Judge recorded that he had been given an agreed bundle, a request for information, a file of ‘Legal Materials’ for the respondent and a file described as ‘Claimant’s Case Law’. Evidence was heard from the claimant and both parties made submissions. The Judge explained
“7. There were difficulties with the claimant’s case law which, in my view, it is appropriate to set out at this stage. Firstly, the wrong case law file was sent which related to the substantive merits of the claims, not the preliminary hearing. There was some delay to the hearing as a result. More significantly, when the correct case law file was submitted, it contained the case citation and very short summaries of some 11 cases rather than the cases themselves. Although not clear from the face of the document, when asked, [the Claimant] explained that the summaries were generated using an AI tool into which specific prompts or queries had been inputted. The prompts or queries were not shown in the document albeit the case summaries were grouped under various headings such as ‘delayed onset of PTSD’.”
It appeared that no one had checked the summaries against the original sources. This became apparent when one first instance report could not be located and another was found to be significantly inaccurate once compared with the full judgment.
In paragraph 8, the Judge set out the AI hallucinations/false citations which, for the reasons I explain here, and my article for Bar Counsel Magazine (here), I will not do in this report. However, they do appear to be Type 1 and Type 6 AI Hallucinations. As the Judge explained, this consumed a substantial amount of hearing time while counsel and the Tribunal attempted to obtain and verify the underlying reports:
“9. I raise this because: 9.1 An appreciable amount of hearing time was taken up with trying to obtain copies of various reports in order that respondent’s Counsel (and I) could check the accuracy of the AI generated summaries. 9.2 There was a significant risk I could have been misled had this not been done. 9.3 Because of the demonstrated inaccuracies, I was unable to rely on the summaries. 9.4 The delay involved also caused or contributed to my Judgment being reserved.”
The Judge acknowledged that the Claimant’s representative was not legally qualified and that the legal research had, apparently, been carried out by volunteers on his behalf:
“…He is genuinely seeking to assist a claimant who would otherwise be unrepresented. Nonetheless, it is important that some basic checks are done to ensure that the material put before the Tribunal is accurate in order to avoid the above. I refer to R (on the application of Ayinde) v London Borough of Haringey [2025] EWHC 1383 which clearly identifies the risk of not undertaking such checks and the importance of doing so…”
Comment
This case, like many that come before the courts and tribunals, illustrates the predicament that judges often face in the modern era of AI generated documents. Legal standards matter and misleading the court is never acceptable, yet many court users are not legally qualified and some representatives may not have the necessary knowledge or training. As the judge emphasised, “…He is genuinely seeking to assist a claimant who would otherwise be unrepresented…”
In this instance, the AI hallucinations or fabricated citations resulted in significant court time being used to verify the material, since without that investigation the court could have been misled. The court’s time is scarce and is becoming an increasingly limited public resource. Unnecessary demands on that time can affect access to justice for everyone, as well as contribute to higher costs that someone will eventually need to meet.
There is more analysis to follow shortly. If you find these legal articles interesting, please consider subscribing below:
Final Word from Chat GPT 5 Pro
This is where I invite a premium AI model to comment on or critique the preceding discussion:
“The issues you have explored highlight a developing, systemic challenge rather than isolated individual failings. As AI tools become more accessible, it is understandable that unrepresented parties and well‑meaning volunteers will turn to them for help, yet the technology can generate material that looks authoritative whilst being entirely unreliable. Your focus on the judges’ predicament shows how they are being asked to maintain high legal standards and protect the integrity of proceedings, while also dealing fairly and patiently with users who may have limited legal training and resources. The need to spend time checking AI generated citations not only strains already limited judicial time but also has wider implications for access to justice and costs across the system. At the same time, your examples suggest that with clear guidance, better education about the limits of AI, and transparent disclosure when such tools are used, courts and representatives can work together to reduce the risks while still harnessing potential benefits. Overall, the picture that emerges is one of a justice system adapting in real time to new technology, trying to strike a careful balance between procedural rigour and empathy for those seeking assistance.”
