Should the Use of Generative AI Be Permitted to Produce Court Documents?

Artificial Intelligence, or AI, has been all the rage in recent years – used in everything from photos, to videos, to internet and intra-website search, to document generation and even internet scams.

In fact, the leading supplier of AI software and hardware, Nvidia, is now the largest company in the world by market capitalisation, valued in November 2024 at a whopping US$3.652 trillion, ahead of Apple (US$3.610 trillion), Microsoft (US$3.275 trillion), Saudi Aramco (US$2.564 trillion), Alphabet (US$2.150 trillion), Amazon (US$2 trillion) and Meta (US$1.220 trillion).

But the proliferation of AI has also come with great controversy, with debates swirling about moral, ethical and legal considerations in the context of generative AI.

So, what is generative AI? And how are educational institutions and courts dealing with its use?

What is Generative AI?

Generative AI is a form of artificial intelligence that is able to produce a wide range of content with minimal input.

So, for example, a user may be able to produce detailed answers by asking a few simple questions or otherwise entering minimal text.

It can also generate audio, images and videos in the same way, saving the user an enormous amount of time and resources, but ultimately producing content determined as relevant by the specific language model program – whether it be Chat-GPT, Google Genesis or Bard, Claude, Grok, Llama or otherwise.

Sydney University to Permit the Use of Generative AI

Until recently, using generative AI to produce essays and answer assignment and take-home exam questions was strictly prohibited by tertiary institutions in Australia – who saw it as a form of cheating and took the view it discouraged original thought and educational development.

But now, the University of Sydney has decided to allow the use of generative AI for teaching purposes as well as to generate parts of essays and answers to assignments and take-home exam questions.

The University’s Executive Education Committee has issued guidelines on the use of generative AI which “apply University-wide effective immediately”.

These guidelines state that:

1. All use of AI by staff in teaching, assessments, and learning, must be clearly communicated to students,
2. Submission of student work to AI must:

• Use only third-party AI programs that have been approved,
• Be explicitly communicated to students in writing, and students must give informed consent,
• Comply with privacy and data protection policies, including security or sensitive work, and
• Not be retained by the AI tool provider.
• For assessments that count towards the final grade, and
• For assessments that do not count towards the final grade (formative assessments).

Criticism

Critics of such AI use assert it enables students to quickly produce documents with minimal input, thereby discouraging original thought and reducing authenticity, ultimately leading to a studentship that is lazy and less critical.

They argue that the process of conducting, critiquing and applying original research using is an essential part of student development, especially at the university levels in fields which require critical thought and complex decision-making in later years.

But the university seems to disagree, apparently adopting the view that generative AI will see broad adoption in the future and, to an extent, replace the need for independent critical thought and decisions.

Supreme Court Practice Note on Use of Generative AI

Meanwhile, the Supreme Court of New South Wales has taken a different approach to the use of generative AI, issuing a Practice Note which in part provides that any affidavit, witness statement, or character reference must “contain a disclosure that Gen AI was not used in generating its content (including by way of altering, embellishing, strengthening, diluting, or rephrasing a witness’s evidence).”

The direction is contained in Practice Note SC Gen 23 — Use of Generative Artificial Intelligence which was created in consultation with the Law Society of New South Wales.

The Practice Note also prohibits judges from using generative AI to produce to produce reasons for judgments, to assess evidence in preparation for giving those reasons, or to edit or proof draft judgements.

The Practice Note directs judges to be mindful of the shortcomings of generative AI and vigilant about its potential use by others in court proceedings.

It stresses the importance of:

1. Understanding the presence of inaccurate or non-existent citations and fabricated legislative references,
2. The dependence that Gen AI programs can have on the quality of underlying data sets, which can include misinformation,
3. The scope of inaccurate output due to the limitations of the data sets,
4. Search requests or prompts with Gen AI can be automatically added to the language model database,
5. Inability or lack of safeguards to preserve the privacy of information or sensitive material, and
6. The risk of providing information to a judge’s or staff member’s devices.

So, unlike Sydney University, the Supreme Court of New South Wales has, for all intents and purposes, banned the use of AI in court documents and encourages judicial officers to be vigilant in detecting its use by others and bringing such persons to account.

Should the Use of AI Be Permitted in Universities and Courtroom?

The answer to this question is perhaps more complex than a simple yes or no.

On the one hand, using generative AI to produce essays, answer exam and assignment questions, and produce court documents such as pleadings (statements of claim etc), affidavits (written witness statements) and judgments may save a lot of time, there is potential to ‘dull the minds’ of students by averting the need for original and critical thought, while the potential to generate factually and legally inaccurate materials in the courtroom has obvious dangers.

But we’re still in the early days of widespread generative AI adoption, and as language models develop, become more comprehensive and accurate and spill into the workplace, the technology could well mean there is both less of a need for critical thought and lower prospects of generating inaccurate information – factors which some may feel weigh in favour of gradually allowing its use.