Redgrave LLP Publishes Empirical Study on Generative AI and TAR in Document Review

Redgrave LLP has published an empirical study comparing generative AI (GenAI) document review and traditional technology-assisted review (TAR), titled “Generative AI for Complex Document Review: A Comparative Evaluation Benchmarked Against Active Learning and an Independent Expert Reviewer.”  The study presents empirical findings on the performance and defensibility of each approach.

The study is authored by Robert Keeling, Ray Mangum, Eli Nelson, and Kevin Reiss.  It evaluates Relativity aiR for Review against Relativity Active Learning (RAL) on a 45,004-document corpus from the Mallinckrodt collection in the UCSF Opioid Industry Documents Archive.  Ground truth is established through a blinded independent review by a single subject-matter expert.  The findings are situated within the reasonableness and proportionality standards established by case law and the Sedona Conference TAR 1 Reference Model.

The GenAI workflow achieved 88% recall compared to 64% for the active learning workflow, with a lower elusion rate of 1% versus 3%.  In a secondary analysis, the independent expert revised his conclusions on 10 documents after reviewing the AI’s reasoning, confirming them as responsive after initially classifying them as not responsive.  The AI had identified something a trained expert missed, and when the expert saw the reasoning, he agreed.

The authors also discuss the study’s findings and implications in “Better Than TAR.  Nearly Expert: What a Major Study Shows About GenAI and TAR in a Complex Document Review,” published concurrently in Legaltech News.  Read the article here.

This is a working paper made available for discussion and comment.  It has not been peer reviewed.  This working paper is provided for informational purposes only and does not constitute legal advice.  The views expressed are those of the authors and do not necessarily reflect the views of any client or other party.  The analysis reflects information available as of the publication date.

The full study is available below.