Ad/Marketing communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers.
Introduction
I am making my way through the emails and messages about AI hallucinations/Fabricated Citations and hope to write them all up in due course. Please do keep sending them. They are invaluable to my research and legal analysis. I’m afraid the problem does not appear to be improving, but as the case below illustrates, it is encouraging to see judges really engage with Gen-AI and AI hallucinations/Fabricated Citations. It seems to me that Judges are not only becoming more aware of the problem, but are also learning the tools and techniques needed to identify and confront it.
This is the nineteenth incident in my series, and, regrettably, it is not the only example I will be covering from the First-tier Tribunal (Property Chamber) (Residential Property), so there is more to come.
By way of announcement, I will be arranging a further webinar shortly. The response to the last one was overwhelmingly positive and I now have two expert witnesses lined up who want to speak about chatbot harm and many of the issues raised here. I am currently trying to make the timing work for everyone involved (further details here).
Victoria Place Flats et al v Assethold Limited
This was a decision of the First-tier Tribunal (Property Chamber) (Residential Property) concerning a determination of whether the applicants were liable to pay service charges and administration charges that had been demanded by the respondent. The applicants also sought an order that none of the respondent’s costs of the tribunal proceedings could be passed on to the lessees through the service charge or any administration charge. The full facts and determination can be read here.
How did AI feature?
From paragraph 82 onwards, the Judge expressed concern about the landlord’s managing agent’s reliance on Qdime. The case cited in the response was a decision involving a tripartite management company rather than a right to manage company, which prompted the Judge to question why it had been cited at all. To investigate, the Judge entered the following wording, without a question mark, into Microsoft’s M365 Copilot on an Android device:
“Is there case law that a landlord’s right to insure is preserved after the right to manage has been exercised”
The Judge then set out the AI generated response from M365 Copilot. I will not set out the response in full here because, as the Judge explained, what was generated does not appear to be accurate.
At paragraph 84 the Judge noted that, to the best of his knowledge, the Upper Tribunal decision in Qdime had never been the subject of an appeal to the Court of Appeal. The case reference provided in the AI’s answer in fact concerned a Court of Appeal decision in long-running care proceedings and not the Qdime decision at all:
“84.The Upper Tribunal decision in Qdime was to the best of my knowledge never subject to appeal to the Court of Appeal. The case reference above concerned the Court of Appeal decision in long running care proceedings.
The Judge investigated further:
85. I then typed the same wording into M365 Copilot on an Android device but adding a question mark at the end which gave a similar response, although the phrasing was markedly different, and it referred to the Upper Tribunal decision cited by [landlord’s managing agent] rather than the ‘hallucinated’ Court of Appeal citation. Repeating the same question sometime later would not re-produce reference to the Upper Tribunal decision, showing that AI adapts and an earlier answer may no longer be returned as the algorithm learns, demonstrating the care that needs to be taking in using AI. The idiom ‘shifting sands’ comes to mind.”
The Judge then typed the same wording, again without a question mark, into Microsoft Copilot on a desktop PC. This generated a further reference to Qdime in the Upper Tribunal (paragraph 86). Typing the same wording later returned reference to another wholly ‘hallucinated’ Court of Appeal decision with a different citation. Inputting that citation into bailii.org did not return any judgment (paragraph 87).
The Judge remarked that it might be the case that using an iOS device, another operating system or different versions of Copilot would give similar answers but phrased differently, but that this did not really matter. The Judge accepted that the landlord’s managing agent had cited a legitimate case reference for Qdime in the Upper Tribunal. The problem was that M365 Copilot had then hallucinated “key points from the judgment” which were broadly similar to the submissions made. The Judge noted that the landlord’s managing agent had not included either of the hallucinated Court of Appeal case references in his submissions. The Judge went on to set out the legal errors in the false citation and explained that, had the landlord’s managing agent “…taken the time to read the decision, he would have realised it was of no assistance…”,
The Judge found that Qdime had only been cited“…as a result of using AI to assist his legal research which became the basis for the drafting of his submissions, which bear a strong resemblance to the answer that was initially returned by M365 Copilot which I have quoted verbatim above.”
The Judge also found that a further invalid case citation had been provided to the tribunal, which he was not satisfied the landlord’s managing agent had read. He concluded that this, too, was an AI hallucination. At paragraph 94 the Judge observed:
“I recognise that litigants in person (‘LiP’) are increasingly turning to AI for legal research. That does not, however, exempt LiPs in general from their obligation to take any AI output and check its accuracy before drafting submissions to the tribunal based upon it.”
The Judge then drew attention to the fact that the landlord’s managing agent included “LLM” in his email signature, which, as the Judge put it, would suggest that he intended the reader to know that he holds a Master of Laws degree.
(Pausing from the commentary for a moment to note that LLM, in this context, is a reference to a legal qualification not a large language model)
The Judge also checked the SRA’s website, which showed that the landlord’s managing agent had been admitted as a solicitor and issued with an annual practising certificate, although there was no indication that he was acting as a solicitor in this particular case. The Judge explained that both the respondent and the landlord’s managing agent appeared regularly in both tribunals and expressed concern about the conduct described, which had been brought to the attention of Regional Judge Whitney.
Comment
Engaging with AI at a granular and technical level is becoming increasingly important and this judgment reflects that shift. The Judge did not treat the AI generated material as something to be accepted or dismissed in the abstract. Instead, he replicated the queries, adjusted the inputs, changed devices, and compared outputs across time. In doing so, he conducted a small but meaningful case study into how large language model tools behave in real conditions, specifically within the sphere that now matters most to lawyers and tribunals, namely legal research and citation practice.
It also offers a quiet reminder that litigants in person are still bound by the rules, particularly where they have some legal training or experience. The expectations of care and diligence do not fall away simply because a party appears without representation.
I do still remain concerned about the growing trend of judges reproducing AI hallucinations or fabricated citations in full within their decisions. This carries a genuine risk of polluting the legal canon and may ultimately deepen the problem rather than contain it. At the same time, I did find value in the Judge’s approach here, especially in showing how the responses shifted over time. I hope others will benefit from that demonstration as well. It may serve as a helpful model for how judges and practitioners can test AI generated material in future, although we will need to be thoughtful about how to balance transparency with the need to protect the integrity of the legal record.
If you find these legal articles interesting, please consider subscribing below:
Final Comment from ChatGPT 5 Pro
This is where I invite a premium AI model to comment on or critique the preceding discussion:
“[The] article captures something genuinely new in this space: a judge who does not merely criticise hallucinated citations but reproduces the research journey, compares devices and prompts, and records how the answers shift over time, turning methodology into evidence. That quiet rigour reframes the debate, because it shows that the problem is not the presence of AI in legal work but the absence of verification and judgment when using it. By distinguishing between litigants in person and legally trained actors, the decision reinforces existing professional duties without theatrics and signals where future guidance may land. As the nineteenth UK incident in your series, it also marks a change of pace, with tribunals moving from anecdote to analysis. The result is both a caution and a template, inviting practitioners to adopt simple habits that prevent harm and to treat any AI output as a lead to be tested, not a conclusion to be filed.”
