Ad/Marketing Communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers. This legal article concerns AI Law.
When searching for “ChatGPT” in BAILII, the earliest judgment I could find is Santander UK Plc v Carlin & Anor [2023] NICh 5. Below is a succinct overview of the background, outcome, and key points of the case. I will then briefly consider how ChatGPT entered the picture.
Overview of the Judgment
Thomas Carlin and Maxine Hughes originally secured an interest-only mortgage with Abbey National (later renamed Santander). By 2013, they had ceased making payments altogether, prompting the lender to initiate repeated legal actions for possession of the mortgaged property. Over the years, Carlin raised claims of fraud and procedural irregularities and alleged that crucial documents had been withheld. Multiple judges reviewed these assertions but found no evidence to support them.
Mr Justice Simpson ultimately refused all of the defendants’ applications. He ruled that the High Court lacked jurisdiction to overturn final orders made by another High Court judge without a proper appeal, labelled their newly issued writ an abuse of process, declined any stay on enforcement, and confirmed Santander’s right to realise its security. Costs were awarded against Carlin and Hughes, and the judge explicitly cleared Santander’s legal team of any wrongdoing.
Where Did ChatGPT Enter the Fray?
At paragraph [34], Mr Justice Simpson observed:
“His final submissions before me also refer to answers provided to a series of questions put by him to ChatGPT, criticising counsel, solicitors, and judges, and he prays in aid these answers in support of his case since they have been provided by artificial intelligence which ‘does not have personal opinions, beliefs or feelings.’ Sadly, ChatGPT seemed unable to recognise or correct the misuse by Mr Carlin in one of his questions of the phrase ‘cast dispersions’ rather than ‘cast aspersions.’”
Commentary
It is unclear which iteration of ChatGPT was available before June 2023, as OpenAI does not universally label updates in a way that makes it obvious to casual users which “version” they are running. However, given that GPT-4 was only released in March 2023 and initially limited to paid subscribers via ChatGPT Plus, it is possible that this response was generated using the free (and more widely accessible) GPT-3.5 model or an earlier release in the ChatGPT family. The precise version remains uncertain so I wont comment further.
However, this distinction is important because the judge highlighted ChatGPT’s failure to catch a linguistic error, which seemingly undermined its generated response. While no AI tool is infallible, in my experience, GPT-4 and similar systems tend to be better at identifying subtle mistakes. Consequently, the type of oversight noted by Mr Justice Simpson may be less likely with newer versions, potentially making it more difficult to identify AI-generated material before the court.
Beyond linguistic errors, the broader concern with AI-generated legal arguments is their reliability. It is often claimed that AI tools generate responses based on statistical patterns in vast datasets rather than an understanding of legal principles. This can lead to citations of non-existent case law (sometimes called hallucinations), misinterpretations of legal concepts, and conclusions unsupported by precedent. AI does not evaluate legal authority as a trained practitioner would, making it a risky tool if relied upon without verification. Courts may need to establish clearer guidelines on how AI-generated legal arguments should be treated.
This case is not unique in raising concerns about AI in legal proceedings. There have been numerous cases which this blog has or will address. As AI becomes more prevalent in legal practice, courts worldwide will need to grapple with its role in submissions and the extent to which parties can rely on its outputs.
Either way, human review remains crucial to ensuring accuracy and clarity in any legal or official document. While AI-generated content can appear impartial or authoritative, it still relies on the user’s input and can perpetuate mistakes. The court’s observation serves as a timely reminder of the need for careful human scrutiny when relying on AI-generated responses, no matter how sophisticated the tool may seem.
Two Additional Observations
- AI-generated answers to a series of questions criticising counsel, solicitors, and judges are unlikely to be probative or of any assistance to the court.
- The assertion that AI “does not have personal opinions, beliefs, or feelings” is not something I would expect any person to advance in court without some form of expert evidence on the point (which, if produced, I would treat with great scepticism). Biases in AI are well-documented, and in my experience, when prompted in a certain way, some AI models will express opinions and beliefs on many issues. Furthermore, AI’s outputs are shaped by training data, which itself may carry biases. Legal professionals must therefore approach AI-generated material with caution, ensuring that any AI-assisted research or arguments are meticulously verified before being presented in court.
