We Were Both Wrong About Legal AI. Here’s What Changed Our Minds.

Let’s start with a confession. Neither of us was an early believer.

One of us (Daniel) spent the better part of a decade running Lexoo, an alternative legal service provider built around high-volume contract review and negotiation. He tried the AI tools that existed at the time. He timed his lawyers with literal stopwatches. He watched the technology underdeliver, repeatedly, and eventually said so publicly, in posts that are still sitting on his LinkedIn profile. He’ll be the first to tell you.

The other (CJ) watched the same movie from the other side of the aisle, deploying machine-learning (ML) tools inside Axiom’s contract intelligence operation. His teams were reviewing tens of thousands of documents, trying to get efficiency gains from a technology that kept producing more complexity than it resolved. By the end, the best thing the ML could do was scroll a lawyer to the right page. That was real progress. It also barely moved the needle.

We are not, in other words, coming to you as AI evangelists who have always believed. We’re coming as two practitioners who resisted, tested, revised, and eventually changed our minds—and who think the reasons we changed our minds matter more than the fact that we did.

The Thing That Actually Changed

The old tools were mostly machine-learning-based, and as a result, they were mostly limited to identifying problems. You’d submit a contract, receive a twenty-page report of everything that was wrong, and then a lawyer still had to read that report, locate the relevant clause in the agreement, and draft the fix themselves. When Daniel timed that process, the total time spent was essentially unchanged from working without the tool. The AI had created a deliverable (a list) without changing the work.

What changed with large language models (LLMs) is that the tool can now do the drafting. It can go clause by clause, in track changes, in a way that mirrors how a trained lawyer would actually markup a document. That sounds like a modest improvement. It is not. What used to take two and a half hours as a redline might now take twenty minutes.

The key word is might. We’ll come back to that.

Three Things We Got Wrong—And Think Most People Still Are

First: Your historical contract archive may not be the asset you think it is.

One of the most counterintuitive things discovered in developing an AI-powered contract review tool is that training a model on past redlines often doesn’t improve quality, and can actually degrade it. Ten lawyers with ten different risk tolerances, working across years of varying market conditions, don’t create institutional knowledge when you aggregate their work. Rather, they create noise, and at times, lots of examples of poor and inconsistent lawyering.

This matters because a large portion of the legal technology sales pitch for the past decade has been built on the premise that your data is your moat. Curate your contracts. Build your clause library. Train your systems on your history. Some vendors are still selling this idea hard.

What we’ve found is that a clear, plain-language playbook grounded in where you actually want to end up, and ideally anchored to your own template, beats a sprawling corpus of past negotiations more often than not. The conventional model also has a practical weakness: Every time a better underlying language model drops, tools that are deeply trained on historical data can’t simply switch.

Second: You don’t have to fully automate to win.

There’s a persistent assumption in the market that AI value scales linearly with automation. The more the tool does on its own, the better. We don’t think that’s right; at least not yet, and perhaps not ever in legal.

For example, DraftPilot (the AI redline solution Daniel and his partner Chris O'Sullivan created) deliberately does not have an “apply all” button. That’s not an oversight. It’s a design decision grounded in what actually happens when you make it too easy for lawyers to accept machine edits without scrutinizing them. Legal documents are not interchangeable.

A liability cap that’s right for one counterparty may be wrong for another depending on deal size, urgency, and relationship. The AI can surface a redline; it cannot assess those variables. The moment you build a mechanism that invites the lawyer to stop assessing them, you’ve created a tool that makes mistakes quietly, which is more dangerous than one that makes mistakes visibly.

What attorneys have expressed repeatedly is that they trust technology when it doesn’t make decisions for them. It suggests. It highlights. It gets out of the way. Partial adoption done well produces outsized gains without the downside risk of full automation done carelessly.

Third: A probabilistic approach will beat a deterministic one.

This one is worth sitting with, because it cuts against a lot of legal instinct. Lawyers are trained to seek certainty. The right answer, the correct clause, the defensible position. It’s uncomfortable to build a workflow around a tool that occasionally produces output that is patently wrong.

Here’s why you should do it anyway. The alternative is not certainty. The alternative is a lawyer reviewing a document without a junior associate helping surface issues, without a playbook running in the background, without a tool that has read the entire agreement and flagged every clause that’s out of spec.

The probabilistic tool with human review is a better system than the human review alone, not because the AI is always right, but because the combination catches more than either does separately. The right frame isn’t “is there a guarantee I’ll never catch this tool making a mistake?” It’s “is this better than my current workflow?” In our experience, consistently, it is.

What This Means for You

A client we met recently in Houston put it in terms we haven’t been able to improve on: “The robots aren’t going to take the jobs of lawyers. The lawyers who are using AI are going to take the jobs of lawyers who aren’t.”

That’s a more precise and more urgent observation than the usual robots-versus-lawyers narrative. It’s not about replacement. It’s about a split that is already happening between legal teams compounding their judgment with technology, and teams still billing as though raw manual effort were the primary unit of value.

Another attorney had a version of this that we also think captures something true: “I’m not worried about the robots taking my job. I’m really happy they’re taking the part of my job I don’t really like.”

Those are not contradictory perspectives. They’re two angles on the same shift. The mechanical work, the clause-by-clause markup, the rule-based position-taking, the first pass on a third-party paper, is becoming something a well-designed tool can help with, significantly, right now.

What remains, and what will only become more valuable, is the judgment that can’t be captured in a rule: knowing when to bend on a liability cap, when urgency changes the calculus, when a counterparty’s ask is unreasonable in a way that will blow out the negotiation.

Lawyers who treat AI as a threat to that judgment are missing what it actually is: the thing that gives them more time and more mental space to exercise it.

We were skeptics. The technology earned our confidence by actually working. We think it will earn others’ confidence as well, but only those who approach it honestly, test it against real work, and resist the sales pitches that promise certainty this technology doesn’t yet deliver.

The honest version is more useful anyway.