This is the work AI is supposed to change.
In some matters, it does. In others, it complicates the work without improving the result. The difference between the two outcomes is not the tool. It is how the analytical question gets framed before the tool is asked to do anything.
The framework below is for senior counsel deciding whether AI-assisted pattern detection is the right approach for a specific matter, and if it is, what good execution looks like. The framework is generic to matters across the document-heavy spectrum — complex commercial, regulatory investigations, government inquiries, document-intensive M&A, mass torts. It is not a methodology for everything. It is a methodology for the matters where pattern detection across a large corpus is part of the analytical work.
1. Define the analytical question before touching the corpus
The most common failure mode in AI-assisted legal analysis is starting with the documents. Ingest the corpus, run a model, see what surfaces, decide what it means.
This produces a different output than starting with the question. What are we trying to learn from these documents? What pattern, if it exists, would change how this matter proceeds? What pattern, if it does not exist, would close a line of inquiry? The analytical question is the design constraint for everything downstream — the corpus you assemble, the model you choose, the validation you do, the documentation you produce.
A well-framed analytical question is specific. “Find anything relevant” is not an analytical question. “Identify communications between [role A] and [role B] between [date X] and [date Y] discussing [topic Z], and rank by likelihood that the communication concerns [specific subject]” is an analytical question. The first produces an output that has to be re-reviewed by humans before it is useful. The second produces an output that can be acted on, with appropriate validation.
The senior practitioner’s discipline here is to write the analytical question down, in plain language, before the model is touched. The question becomes part of the work product. It becomes the reference point for what the AI was asked to do and what the AI was not asked to do.
2. Prepare the corpus deliberately
AI pattern detection is only as good as the corpus it runs against. Sloppy corpus preparation is the second most common failure mode.
Corpus preparation covers several distinct steps. Scope: which documents enter the analytical corpus, and on what defensible basis. Provenance: where each document came from, what its chain of custody is, and what its evidentiary posture is. De-duplication and threading: email threads collapsed, exact duplicates removed, near-duplicates flagged. Normalization: format conversion, OCR for scanned material, metadata standardization. Privilege screening: documents flagged for review before they enter the analytical workflow, not after.
The work of corpus preparation is unglamorous and time-consuming. It is also where the integrity of everything downstream is established. A pattern detected in a poorly prepared corpus is not a pattern. It is a function of the corpus’s defects.
In practice, corpus preparation is often two-thirds of the engagement effort. Senior practitioners plan for this. Junior approaches treat it as overhead and shortcut it. The shortcuts surface later, in cross-examination or in a regulator’s follow-up question, and they surface badly.
3. Run detection with a model matched to the question
The model choice is downstream of the analytical question and the corpus, not upstream. Counsel who select a tool first and adapt the analysis to the tool’s capabilities have inverted the work.
The relevant model dimensions, in approximate order of importance:
Whether the model is appropriate to the analytical question. A model that excels at semantic similarity may be wrong for a question that requires structured extraction. A model that excels at extraction may be wrong for a question that requires reasoning across documents.
Whether the model’s training and inference are compatible with the matter’s privilege and confidentiality posture. Models that train on submitted data are typically incompatible with privileged material. Models that retain prompts and outputs in logs accessible to vendor staff are typically incompatible with work-product protection. These are not edge cases. They are the rule for legal work, and they constrain model choice substantially.
Whether the model produces outputs that can be validated. A model that returns a score with no rationale is harder to defend than a model that returns a score with the documents that drove it.
The practitioner running detection should be able to articulate, in writing, why this model for this question against this corpus.
4. Validate before relying
Pattern detection produces hypotheses, not findings. The validation step is what converts the former into the latter.
Validation has three components. Sampling: pull a representative sample of the model’s hits and review them by hand. Adversarial review: pull a representative sample of the model’s misses and review them by hand. Documentation: record the false-positive rate, the false-negative rate, the documents reviewed, and the basis for relying on the model’s output.
A senior practitioner spends real time on adversarial review specifically. The hits the model surfaces are interesting, but they tell you only what the model found. The misses are where the model’s blind spots live. A model that flags 200 documents and misses 15 critical ones is a tool that needs different design or different validation, regardless of how good the 200 hits look.
A note on privilege and work product
In legal matters, the AI question is inseparable from the privilege question. The model is a third party. The prompts are arguably attorney work product. The outputs may carry derivative privilege depending on jurisdiction and on how the engagement is structured.
This is not a settled area of law in any jurisdiction. It is moving fast. The current safe practice — and what counsel running document-heavy matters should require of any AI vendor or tool — covers four points.
Engagement structure. The AI work should be performed under counsel’s direction, documented in the engagement letter or a supplemental document, and treated as part of the attorney’s analytical process from the outset.
Vendor posture. The vendor’s contract should explicitly preserve attorney-client privilege and work-product protection over inputs, outputs, and any derived analytics. Vendors who cannot or will not commit to this in writing are not appropriate for legal work involving privileged material.
Data handling. The vendor should not train on, retain, or share submitted material beyond what is necessary to perform the analysis. The data processing agreement should specify retention periods, deletion procedures, and what happens to data on contract termination.
Documentation. Counsel should maintain a privileged work-product file documenting why AI was used, what analytical question it answered, what corpus it ran against, what model was selected, and what validation was performed. This file is part of the defense if the analytical approach is later challenged.
The work that AI does not change
AI is useful for the analytical work where pattern detection across a large corpus is the bottleneck. It is not useful for — and does not replace — the work of judgment about what the patterns mean, what they are worth, and how they fit into the theory of the case.
Senior counsel run document-heavy matters with AI exactly the way they have always run them with junior associates and contract reviewers: as a force multiplier for analytical work, not as a substitute for the analytical judgment that determines what the work is for.
The matters that go well with AI are the matters where the analytical question is clear before the work begins, the corpus is prepared with discipline, the model is matched to the question, and the validation is honest. The matters that go badly are the matters where AI is asked to do the work that should have happened in the first hour of the engagement: figuring out what we are actually trying to learn.
The technology has changed. The discipline has not.