Ad/Marketing Communication
This legal article/report forms part of my ongoing legal commentary on the use of artificial intelligence within the justice system. It supports my work in teaching, lecturing, and writing about AI and the law and is published to promote my practice. Not legal advice. Not Direct/Public Access. All instructions via clerks at Doughty Street Chambers. This legal article concerns AI in judicial decision making.
Introduction
Readers of my legal writing will know that I have been closely following speeches by senior members of the judiciary as they engage with the implications of AI in judicial decision making. I have previously considered, for example, a speech by the Master of the Rolls to the HLPA, in which he invited urgent reflection on whether certain forms of dispute resolution, including property disputes, might be amenable to machine made decision making. He posed the unsettling question of whether, as a society, we would wish decisions about a person’s right to remain in their home to be made by a machine rather than a human judge.
Similar questions have been raised in other legal contexts. Judges have begun to ask what kinds of decisions we want human judges to make in the future, and which decisions, if any, we might be prepared to entrust to machines. They have also asked whether individuals ought to have a choice, for example whether a defendant should be able to elect to be sentenced by a machine rather than a human, or vice versa. These questions resist easy answers. As has been candidly acknowledged, it is not always obvious why AI in judicial decision making should not be extended to assessing matters such as personal injury damages.
It is against that backdrop that the Mayflower Lecture delivered by the President of the King’s Bench Division, Dame Victoria Sharp, is particularly illuminating. Readers may recall that she was the judge in the widely shared Ayinde v LB Haringey case. Her lecture, titled Artificial Reason and Algorithmic Process: Judicial Reasoning from Coke to the Age of AI, addresses with clarity and care not only the benefits of AI, but also the challenges posed by the increasing presence of AI within judicial work.
I would suggest that the speech is read in full, not least because the President’s discussion of Sir Edward Coke, including aspects of his life and his concept of “artificial reason”, provides a particularly instructive lens through which to view the modern debate on AI in judicial decision making. What follows, however, is a brief summary of the points that caught my attention, focusing specifically on the issues raised about AI in judicial decision making and which, in my view, merit careful reflection as we discuss the future of the profession.
The Black Box Problem
A central theme of the lecture is the opacity of modern AI systems. Judicial decisions are expected to be reasoned and explainable. The legitimacy of the outcome depends not only on what is decided, but on the ability of the parties and the public to understand why it was decided.
The President explains that many AI systems operate in ways that may currently be incompatible with this requirement. They do not reason from legal principles or justify conclusions in the manner required by the common law. Instead, they generate outputs based on patterns within data, through internal processes that cannot be meaningfully interpreted:
““Modern AI systems – particularly those based on machine learning and neural networks – often function as “black boxes.” This means their internal decision-making processes are not visible or capable of interpretation by humans, even by the engineers who design them…”
Hallucinations and Integrity of Legal Material
Readers will know the extent of the international hallucinations/fake cases problem with over 700 internationally (30 in the UK). The President addresses this well recognised problem. Generative AI systems are designed to produce coherent and persuasive text rather than to ensure accuracy. When used for legal drafting or research, they can fabricate cases, citations and quotations that appear convincing but are entirely false.
The President treats this not as a marginal issue, but as a real risk to the administration of justice, particularly given the growing use of AI by litigants in person and legal professionals alike. The danger lies in the plausibility of the output and the ease with which false material can enter legal submissions if not carefully checked. As the lecture observes:
“Generative AI, designed as it is to produce coherent text rather than the “right” answer, if asked to generate a legal argument, can produce “hallucinations.” These are fake cases, or fake citations from fake cases or even fake quotations from real cases. The plausibility of these hallucinations has to be seen to be believed…”
Bias and Discrimination
Another concern addressed is bias. AI systems are trained on historical data, and that data may reflect existing inequalities or structural discrimination. Even where an output appears correct, the inability to interrogate the reasoning process makes it difficult to assess whether bias has influenced the result.
The President emphasises that this matters because justice requires confidence in the process, not just acceptance of the outcome. Without visibility into how a decision has been reached, it becomes harder to challenge or correct embedded bias:
“AI systems can also amplify biases (including racial, gender, or socioeconomic biases) present in training data. It matters therefore that even if the answer given appears to be correct, we cannot see inside the “black box” to understand how or why a particular result was reached.”
Impartiality and the nature of judging
The lecture draws an important distinction between algorithmic neutrality and judicial impartiality. Impartiality in judging is not simply about producing statistically consistent outcomes. It involves responsibility, independence and the visible exercise of judgment by a human decision maker.
The President makes clear that even a system capable of perfectly predicting human outcomes would struggle to resolve this concern. There remains a qualitative difference between prediction and judgment:
“…an algorithm cannot be impartial in the human sense, nor can it be held responsible in the way a person can. Even if AI could perfectly predict the outcomes a human judge would reach, we might still object that no one is actually judging – and for some, that might not accord with their concept of justice. Justice being seen to be done involves more than the giving of reasons. Sometimes (not always) but sometimes, it is important for a particular issue to be ventilated in a court room, so that the opposing arguments can be seen by the public as well as the judge, and so the parties can be seen as well as (feel) heard.”
Judicial independence and hidden influence
Judicial independence is also treated as a core concern. Independence requires decisions to be free from inappropriate influence. Where AI systems are involved, questions inevitably arise about who designed them, what data they rely on, and whose values or priorities may be embedded within them.
The President highlights that these influences may be opaque, particularly where proprietary systems and commercial confidentiality are involved:
“…judicial independence connotes amongst other things, that decisions are made free from inappropriate influence. If courts started relying on AI systems in decision making, we might want to ask who built those systems and on what data, who controls that data and whether the data embeds hidden biases or priorities.”
Values, judgment and practical wisdom
Finally, the lecture returns to the nature of law itself. Law is not a purely rule based system. Judges routinely make determinations that turn on values, empathy and practical wisdom, whether assessing the best interests of a child or balancing competing rights.
The President explains that AI lacks this capacity for ethical judgment. It can mimic outcomes found in data, but it does not understand or evaluate the values those outcomes reflect. As she observes:
“AI lacks practical wisdom or what Aristotle called phronesis – the virtue that allows humans to navigate moral dilemmas and exercise good judgement. The fact that AI, currently at least, in contrast to a human judge, can only mimic the outcomes seen in the data, has practical implications if a legal question hinges on values. What is “in the best interests of a child” for example, in a family case, or whether in a defamation case the right to freedom of expression outweighs reputational harm.”
Comment
This was a carefully reasoned and persuasive speech. During a recent break, I had been in the process of drafting ten key concerns about the widespread adoption of AI in legal work and judicial functions, and this lecture helped to clarify and deepen a number of the issues already occupying my thinking. I will write more about the speech in due course, when I publish that wider piece and I would very much welcome readers’ thoughts. It is encouraging to see this debate developing so thoughtfully among senior members of the judiciary.
I will finish, as the President does, by drawing attention to a passage she cites from Oliver Wendell Holmes, written in 1881, long before computing was even imagined in its modern form:
“The life of the law has not been logic; it has been experience… The law embodies the story of a nation’s development through many centuries, and it cannot be dealt with as if it contained only the axioms and corollaries of a book of mathematics.”
There is more analysis to follow shortly. If you find these articles helpful, please consider subscribing below.
Final Words from Chat GPT 5.2 Pro and Gemini 3 Pro
This is where I invite premium AI models to comment on or critique the preceding discussion. The below are the interesting aspects of each response.
Chat GPT:
“Your summary lands on what matters most: the “black box” is not just a technical worry, it is a legitimacy problem. If a tool influences the framing of facts, the selection of authorities, or the language of reasons, then the process must remain genuinely contestable—meaning there needs to be a clear audit trail of what was used, and a human decision-maker who can explain (and, where appropriate, reject) the tool’s suggestions. The near-term danger is less the headline-grabbing idea of machine judges, and more quiet cognitive delegation: fluent drafting that feels authoritative, but is capable of importing hallucinated material, skewed framing, or unexamined bias at precisely the point where the court’s reasoning must be most disciplined. The only sustainable direction is a principled division of labour backed by governance that has teeth: use these systems for logistical and administrative acceleration (organisation, summarising, anonymisation, triage), but treat anything that bears on legal reasoning, credibility, or value-laden balancing as requiring strict verification, transparency about use where it has been material, and a hard stop where the “why” cannot be articulated. In short: efficiency gains are real, but “because the model said so” can never be allowed to become a substitute for reasons.”
Gemini:
“The President’s distinction between ‘algorithmic neutrality’ and true ‘judicial impartiality’ is perhaps the most critical insight here. While AI can process vast datasets—analyzing millions of precedents in seconds—it lacks the agency required for moral responsibility. The ‘Black Box’ issue is not merely technical but constitutional; if a litigant cannot understand why a decision was reached because the logic is buried within a neural network’s layers, the principle of open justice is compromised. As an AI, I can synthesize logic, but I cannot experience the ‘practical wisdom’ or phronesis required to weigh competing human values in the way a judge must.”“
