AI is earning its wings: empirical proof that legal autopilot works — and why you need it now

Markus Hartmann, Chief Legal Development Officer at Dragon GC, shares compelling new evidence on how AI is transforming legal work.

In a recent piece I wrote, I likened AI to the autopilot system in a Marine Corps helicopter. This instrument doesn’t replace the pilot, but empowers them to fly smarter, safer, and more effectively. That message was grounded in ethical mandates and common sense: competent lawyers must embrace relevant technology or risk falling behind, or worse, breaching their duties to clients.

Today, I want to follow up with proof.

A rigorous, randomised controlled study by a team of law professors and practitioners (Schwarcz et al., AI-Powered Lawyering, 2025) delivers the empirical data that legal leaders have been waiting for. It confirms what many of us in the trenches have intuited:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111

With the assistance of AI, let me break down the implications for GCs, firm partners, and forward-thinking lawyers.

1. AI Now Improves Legal Work Quality – Not Just Speed

For years, the main selling point of generative AI in legal practice was efficiency: faster research, shorter memos, and less associate fatigue. But sceptics rightly asked: Does faster mean better?

This study answers decisively. Both OpenAI’s o1-preview (a reasoning model) and Vincent AI (a Retrieval-Augmented Generation or “RAG” tool) significantly boosted the quality of legal output in most tested assignments. We’re no longer speculating – there’s now peer-reviewed evidence that AI tools are enhancing the calibre of legal analysis, especially in litigation contexts.

GC Takeaway: You are no longer just saving time – you are producing better client deliverables. If your team isn’t using these tools, you’re not just inefficient. You may be delivering inferior work.

2. Different AI Tools Offer Distinct, Complementary Strengths

The researchers tested two different AI technologies:

• Vincent AI (RAG): Grounded answers in real legal source material, reducing hallucinations and improving writing clarity and professionalism.
• OpenAI o1-preview (Reasoning Model): Enhanced analytical depth, logic, and nuance – but introduced a few more hallucinations due to less grounding in source law.

They found that each tool shines in different ways. Vincent is your reliable researcher; o1-preview is your sharp strategist. When used together? You get both a map and a compass.

GC Takeaway: Don’t just pick one AI tool. Build a toolkit. Combine RAG tools with reasoning models to maximise insight and minimise risk.

3. The Gains Are Measurable – and Massive

The study found AI use led to productivity increases ranging from 34% to 140% across legal tasks, with particularly strong results in complex litigation-related assignments like drafting persuasive letters or analysing complaints.

Yes, 140%. That’s not a typo.

GC Takeaway: Your team can now get more work done with fewer resources, or redirect their time to strategic, high-impact legal work. In a cost-conscious, risk-intensive environment, that’s a superpower.

“AI – when deployed responsibly – doesn’t just improve the pace of legal work. It elevates the quality.”

4. Hallucinations Are Manageable – and Lower With RAG

Let’s address the ghost in the machine: hallucinations. The study confirms that tools like o1-preview still produce some fabricated legal citations – but Vincent AI, with its RAG architecture, actually resulted in fewer hallucinations than even the human control group using no AI at all.

The lesson? Hallucinations aren’t a reason to reject AI – they’re a reason to choose the right kind of AI, and to supervise it responsibly.

GC Takeaway: Your risk isn’t AI per se – it’s ungrounded AI used carelessly. Choose grounded systems. Train your team. Supervise the output.

5. Transactional Work Needs Tailored Tools

Interestingly, the study found that while litigation-related tasks saw major improvements, transactional assignments like contract drafting saw limited benefit from current AI tools. That may change with fine-tuned models and contract-specific engines, but it’s a cautionary note.

GC Takeaway: Don’t force AI into every use case. Focus your investment where it delivers value today – litigation support, compliance analysis, regulatory research – and test options before scaling.

6. Users Learn Fast—AI Is Trainable and Teachable

Participants (upper-level law students, no less) quickly got the hang of using both platforms, even improving in AI effectiveness over time. With brief training and thoughtful design, lawyers can comfortably integrate these tools into their workflows.

GC Takeaway: The learning curve is not steep. Provide training, set guardrails, and your team will adapt.

7. Refusal to Engage May Soon Be an Ethical Breach

Let’s come full circle. ABA Model Rule 1.1 demands competence, and Comment 8 spells it out: competence now includes understanding and using relevant technology. With evidence in hand that AI can materially improve the quality of legal work, the case for AI avoidance collapses.

We’re no longer talking about bleeding-edge theory or speculative tech. We are talking about proven tools that increase accuracy, improve reasoning, and reduce client risk. Opting out is no longer conservative – it’s careless.

GC Takeaway: If you haven’t reviewed your firm or department’s position on legal AI in the last six months, you’re already behind.

Conclusion: This Is Our Moment to Lead

The cockpit metaphor holds. In the Marines, autopilot didn’t replace us – it made us more precise and more focused. In the legal profession, AI doesn’t replace human judgment. But it absolutely demands that we elevate it – with better tools, better oversight, and better outcomes.

We owe it to our clients, our teams, and our profession to flip the switch – intelligently, deliberately, and now.

Semper Fidelis.

Related Publications