Law Firms Share Results from Assessing and Testing AI

Lawyers Speak Out About AI in Recent Studies

Artificial intelligence (AI) is changing various industries, including the legal sector. Law firms and legal departments worldwide are increasingly integrating AI to streamline routine tasks, such as document review and contract analysis. This allows lawyers to focus on more complex and rewarding aspects of their work, improving the quality of legal services and enhancing job satisfaction among lawyers.

Assessments, Reports, and Surveys

Recent surveys and studies by Ironclad, Allens, and Ashurst indicate how this technological shift is being met with enthusiasm and skepticism, particularly among top law firms.

The Ironclad survey reveals a notable divide between in-house legal teams and law firms regarding AI adoption, emphasising differing attitudes and usage rates. Allens’ AI Australian Law Benchmark focuses on the reliability and performance of large language models in providing legal advice. Meanwhile, Ashurst’s global trials highlight substantial time savings and the complementary role AI can provide in legal workflows.

Ironclad: The 2024 AI Legal Survey

The 2024 AI Legal Survey, conducted by Ironclad and OnePoll, reveals a significant divide between in-house legal teams and law firms regarding adopting AI technologies. It surveyed 800 American lawyers and legal professionals, focusing on AI adoption, sentiment, and use cases. Key findings include:

Trust and usage - 71% of legal professionals trust AI, 74% use it, and 92% report improved work quality
Adoption rates - In-house teams are more advanced in AI adoption than law firms, with 87% of in-house lawyers using AI tools versus 60% at law firms
Job satisfaction - 57% believe AI could reduce workload stress
Concerns - Accuracy (40%) and security (37%) 

Allens AI Australian Law Benchmark

The Allens AI Australian Law Benchmark tested several leading large language models (LLMs) for their ability to provide legal advice on Australian law. Key findings include:

Reliability issues - None of the LLMs consistently provided reliable legal advice without expert supervision
Performance - GPT-4 was the best performer, followed by Perplexity, but all models struggled with tasks requiring critical reasoning
Citation problems - Poor citation was a significant issue
Recommendation - AI tools should augment, not replace, human lawyers with necessary safeguards in place

 Ashurst Assesses AI Tools

Ashurst LLP, a major UK law firm, conducted global trials on generative AI tools to assess their impact on legal work efficiency. The trials, involving 411 participants from 23 offices, revealed substantial time savings: 80% for UK corporate filings, 59% for industry research, and 45% for first-draft legal briefings.

AI-generated content was comparable to human output in quality and accuracy. The study emphasises AI's role in augmenting, not replacing, human work and the need for continuous digital literacy development.

The Best of Both Worlds

Integrating AI in legal practice is changing the industry by enhancing efficiency and accuracy. The surveys and studies from Ironclad, Allens, and Ashurst demonstrate AI's potential to support legal professionals.

However, a balanced approach that combines AI's strengths with human expertise is crucial to achieving the best outcomes. Continuous digital literacy and a cautious approach to AI adoption will be key to successfully navigating this transformation in the legal field.

Back to blog