Redgrave LLP Study Evaluates GenAI Against Human Reviewers in Large-Scale Document Review
Robert Keeling, Ray Mangum, Amy Hanke, and Alyssa Ogden have published “Prompting Change with GenAI in Large-Scale Document Review: A Real-World Study” in a special eDiscovery-focused issue of the University of Florida’s Journal of Technology Law & Policy, Volume 30:2.
The study evaluates Relativity’s aiR for Review against human attorney reviewers on a 1,600-document sample drawn from an actual regulatory matter. Two attorneys who led the original review re-coded the full sample to establish the study’s gold-standard benchmark, against which GenAI’s predictions were measured.
GenAI achieved 83.9 percent recall and 84.7 percent precision on responsiveness review, exceeding the widely accepted 70 to 75 percent recall benchmark for technology-assisted review. Issue-coding results were more mixed, with GenAI showing limitations on industry-specific workpaper documents.
The full article is available below.
The views expressed in this article are those of the authors and do not necessarily represent the views of their law firm or any of its clients.
Related people
-
Robert D. Keeling
Co-Managing Partner; Member, Executive Committee -
Ray Mangum
Partner -
Amy Hanke
Counsel -
Alyssa M. Ogden
Counsel
