‘The Senate confirms federal judges – not computer software – to perform the duties mandated under Article III [of the United States Constitution].’
‘Any use of AI requires caution and humility’, US Supreme Court Justice John Roberts warned in 2023. That advice has seemingly gone unheeded, as the US Senate Judiciary Committee recently upbraided two federal district court judges for issuing, then retracting, ‘factually inaccurate court orders’ and court opinions drafted by federal judges’ law clerks and interns using generative artificial intelligence.
The inappropriate use of AI for court documents and filings is a significant menace to the judiciary workforce, often resulting in disciplinary action and court sanctions of myriad attorneys. Senate Judiciary Committee Chair Chuck Grassley adjured: ‘Each federal judge, and the judiciary as an institution, has an obligation to ensure the use of generative AI does not violate litigants’ rights or prevent fair treatment under the law […] We can’t allow laziness, apathy or overreliance on artificial assistance to upend the judiciary’s commitment to integrity and factual accuracy.’ Senator Grassley entreated: ‘I call on every judge in America to take this issue seriously and formalize measures to prevent the misuse of artificial intelligence in their chambers.’
AI use in the judiciary holds significant promise, often resulting in speedier court decisions and reduced administrative burdens that streamline judges’ duties and judicial counsels’ work. However, the perils of AI are substantial without national regulation amidst crises in enforcement of the rule of law.
Wolters-Kluwer’s ‘2023 Future Ready Survey Report’ predicted that over 70% of attorneys would use generative AI for legal research and case analyses between 2023 and 2024, ideally to improve efficiency and productivity. Thomson Reuters’ 2024 ‘Future of Professionals Report’ outlines several ways AI technology optimises lawyers’ time and enhances law firms’ competitive legal advantage. These include better access to justice for clients who cannot afford adequate legal representation, streamlined attorney caseloads, and improved caseload management for auxiliary legal professionals.
Constitutional and statutory provisions, along with legal precedents, are integral to judicial decision-making. Generative AI ChatGPT programs draft contracts, generate case analyses, and prompt legal decision-making. Claude, which is owned by Anthropic, performs legal research and document review. Large language models (LLMs) train and analyse colossal datasets, including hundreds or thousands of pages of legal documents. AI algorithms propagate automated judicial decisions based upon these massive datasets.
Thus, independent judicial decision-making could be bolstered through the use of AI systems. Human judges could theoretically be replaced by AI systems which would issue (ideally) unbiased decisions, thus, reducing disparities among judges’ rulings in similar cases, opinions, and rulings, and have authorisation to make sentencing decisions. Recently, the 2026 science fiction film Mercy highlighted such a scenario in which a criminal defendant on trial for murder appeared before an AI court which rendered a verdict based upon evidence gleaned from AI datasets.
However, biased or flawed information and datasets entered into LLM training programs could lead to faulty outcomes, errantly altering judicial decisions in a potentially negative manner. In addition to problems with bias, some LLMs have violated copyright laws, with the original authors claiming that the use of their copyrighted materials for LLM training models occurred without their permission. Thus, transparency is key–including full candour concerning the use of AI decision-making software in the judiciary.
Generative AI such as ChatGPT can cite legal cases, but it should be used with the caveat that it sometimes generates non-existent cases. Stanford University Human-Centered Artificial Intelligence discusses this issue in ‘AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries.” According to the American Bar Association, the inclusion of ‘phantom’ and ‘hallucinated’ cases in court documents has resulted in attorney disciplinary action for legal practice violations of the Federal Rules of Civil Procedure. Actually, many courts prohibit generative AI use by attorneys and legal staff. Thus, the ideal of universally unbiased AI judicial decision-making is unrealistic in practice.
Emergent problems with AI judicial decision-making indicate that neither the judiciary nor society is prepared for the replacement of human judges with AI systems. According to the Harvard Kennedy School’s Mossavar-Rahmani Center for Business and Government, ‘[h]uman intuition, emotional intelligence, sheer common sense: these are all an essential part of the makeup of a good judge. In other words, judgement isn’t merely about the legal or factual judgement that judges must come to, but the myriad of small decisions that, taken together, make sure that the administration of justice runs well. One of the main criticisms of human judgement and its application to judicial decision-making, however, as referred to above, is the existence of bias, either overt or covert. This is a problem that has been intrinsic to judicial decision-making over the generations.’
The US government executive branch uses AI to engage in surveillance of immigrants, migrants, college student protestors, and citizens. The Biden administration’s Office of Science and Technology Policy promulgated a ‘Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People’ in October 2022. This ‘Blueprint’ endorsed safe and publicly accessible automated AI data, systems, and processes designed with government and community input.
The Trump administration revoked Biden’s initiatives via Executive Order 14179, ‘Removing Barriers to American Leadership in Artificial Intelligence,’citing a need to re-evaluate the nation’s AI policies and initiatives. The Executive Order entitled, ‘Ensuring a National Policy Framework for Artificial Intelligence,’ promises to establish an AI litigation task force.
The Secretary of Commerce and selected executive branch officials will review the AI laws of all 50 US states and territories with the option to federally pre-empt AI practices which imply deceptive conduct or a lack of transparency. Additionally, the Trump administration introduced Executive Order 14161, ‘Protecting the United States from Foreign Terrorists and Other National Security and Public Safety Threats.’ The goal of the program is to screen undocumented immigrants and foreign nationals for potential criminal activity, terrorist activity, or support for foreign terrorist groups. Screening methods include status tracking of immigrant and student visas, social media post monitoring, media report reviews, and AI-driven facial recognition.
Concerns regarding the significant increase of AI use in policing mirror many of the aforementioned concerns pertinent to the executive branch. According to the ACLU,geographic and socioeconomic barriers could be obstacles to equitable implementation of AI in policing. A 2016 ProPublica article highlights facial recognition technology as being associated with complaints of racial bias, wrongful police apprehension and prosecution, and denial of due process for crime suspects. The article reveals that machine-learning algorithms programmed to conduct risk assessments of criminal suspects systematically assigned higher risk assessment scores – which are linked to past criminal histories and the likelihood of suspects to re-offend – to black suspects with less extensive and less severe criminal histories than white suspects. These risk assessment scores greatly increase the likelihood of receiving longer and harsher prison sentences.
Misidentification is a significant problem. AI-enabled facial recognition has a limited ability to accurately discern law-abiding individuals from crime suspects, such as a grandmother recently arrested and mistakenly jailed for bank fraud. This issue is, according to a 2023 Scientific American article, evidently pronounced amongst populations of colour. It follows that AI utilisation also fails to protect individuals in the US from constitutional privacy and due process violations.
AI public surveillance systems flout principles of safe AI, fostering mistrust through disparate and inconsistent access to AI surveillance materials for criminal defendants. State of New Jersey v. Francisco Arteaga presented the case of a private citizen who was erroneously flagged as a criminal suspect by a police AI surveillance program. Here, the court determined that the defendant had the right to access AI facial recognition data collected by law enforcement.
The European Union has demonstrated global leadership in AI and data governance in its AI Act of 2024, whereas the United States has simply proposed a Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People, or, the AI Bill of Rights, which has not become law. The AI Bill of Rights purports to examine data algorithms which contain inherently biased information upon which AI systems make decisions.
However, the AI Bill of Rights,initially touted as an innovative device for introducing fairness and equity, boosting data privacy, increasing transparency, and improving risk management, is riddled with difficulties, prompting experts to question whether AI use in the judiciary is beneficial at all. The EU AI Act builds upon existing personal data privacy protections, which include the right to access, possess knowledge concerning, and control the distribution of personal data shared online conferred by the General Data Protection Regulation (GDPR).
Individual US state laws have made progress in forward-thinking AI policy and regulation. The California Privacy Rights Act confers data privacy protections in the spirit of the GDPR. However, there is no nationwide equivalent of the GDPR in the United States. Increased transparency in the use of AI data systems should encourage more responsible implementation and use of AI in the future.
Privacy and AI experts have proposed Constitutional AI, in which essential ethical principles and operational core values are embedded into AI technology. Non-compliance could result in AI malfunction, denial, or termination of access to the AI technology.
Notably, Anthropic has emerged as a leader in Constitutional AI by promulgating a Claude constitution which emphasises that ‘Claude models should be: Broadly safe…broadly ethical…compliant with Anthropic’s guidelines, and genuinely helpful.’ Moreover, the Anthropic Claude constitution prioritises ‘individual privacy and freedom from undue surveillance’ and respect for ‘the rule of law, justice systems, and legitimate authority.’
As an example, Anthropic recently refused to allow the US military to use its technology for unregulated mass public surveillance. Furthermore, Anthropic has developed the Claude Mythos Preview language model, which, as noted by technology consortium Project Glasswing, could have disastrous cybersecurity consequences (e.g., unpredictable exploitation of software vulnerabilities and hacking behaviour) with unregulated use. Nonetheless, the AI constitution is not perfect, as it is unclear concerning human rights and the AI constitution still susceptible to user input bias.
Although Anthropic has adopted a progressive stance pertinent to the rule of law by drafting its own constitution, society seems to be moving backward. The executive branch of the US government has repeatedly challenged the rule of law with excessive issuances of Executive Orders – many of which challenge the authority of both Congress and the courts – unequally applying and enforcing existing laws, and defying federal court orders.
Consequently, scores of lawsuits are pending against the current Presidential administration. According to the Brookings Institution, the Trump administration’s plans regarding current AI executive branch use and oversight have not been clearly implemented or evaluated for risk and government accountability.
What is problematic regarding implementation of AI in the rule of law is that the very ideals that AI endeavours to remedy and promote – access to fair treatment and full access to due process of law – are impossible without the elimination of faulty algorithms. Increased privacy, better safeguards and transparency surrounding surveillance, statutory civil rights protections, and constitutional Fifth Amendment and Fourteenth Amendment due process rights protections are often flouted amidst poor or absent regulation of AI facial recognition technologies.
Most importantly, meaningful change concerning AI and rule of law adherence and enforcement is unlikely in the absence of (1) a collaborative framework for the ethical use of AI among both government and government-contracted entities and (2) a presidential administration which holds the rule of law in high esteem.