Seventh Circuit Report: Daubert by the Numbers
When experts qualified to perform statistical analysis purport to draw conclusions from data, courts applying Daubert distinguish between testimony describing the statistical evidence and testimony characterizing what it means. A recent decision from the U.S. District Court for the Northern District of Illinois adds to a line of authority in which courts have admitted the former while finding the latter inadmissible.
A statistician testifying in support of wrongful death plaintiffs’ municipal liability claims under 42 U.S.C. § 1983 could reliably identify statistically significant patterns in how a city handled incidents involving police officers suspected of driving under the influence of alcohol, the district court held in Cazares v. Frugoli, No. 1:13-cv-05626, 2017 U.S. Dist. LEXIS 49938, at *1–3, 13–26 (N.D. Ill. Mar. 31, 2017). But the same expert’s conclusions that a “code of silence” existed at the police department and that its officers believed that they could drive under the influence with impunity were held inadmissible and were excluded because they exceeded the scope of her statistical expertise and lacked sufficient foundation. Id. The court drew an admissibility distinction between quantitative analysis and qualitative explanation that is consistent with prior authority and that may prove useful in other contexts in confining a qualified statistician’s testimony to statistical findings.
The plaintiffs in Cazares, survivors of two men killed when an intoxicated, off-duty police officer employed by the City of Chicago rear-ended their car on the expressway, sought to hold the City responsible under a theory of municipal liability brought under Section 1983 and Monell v. Dep’t of Social Servs., 436 U.S. 658 (1978). Specifically, the plaintiffs alleged that the city violated the decedents’ substantive due process right to bodily integrity through de facto policies of failing to investigate or prosecute police officer misconduct and a police department “code of silence,” which led the officer to drive drunk without fear of punishment. In support of this theory, the plaintiffs retained a statistician to analyze data from alcohol-related incidents involving city police officers.
Applying her expertise in data management and analysis, and employing accepted tools and tests to extract and analyze data and then to determine whether certain patterns were statistically significant or due to chance, the plaintiff’s statistician arrived at a series of opinions that the court characterized as her “statistical findings.” These included comparisons of alcohol-related incidents involving city police officers that occurred within versus outside of the city, such as:
- When city police officers were arrested for DUI outside of the city, an intoxication test was administered, on average, within an hour. When city police officers were arrested for DUI in the city, an average of three and a half hours passed before administration of an intoxication test.
- Most DUI citations issued outside the city against city officers resulted from routine traffic stops. Within the city, most DUI citations issued against city officers followed a vehicular collision with injuries, and few followed routine traffic stops.
- When an arresting officer knew that the arrestee was a city police officer, it took more than twice as long to measure intoxication as it did when the arrestee’s status was unknown.
The plaintiff’s statistician also calculated the median number of days it took to complete investigation of DUI complaints against city police officers, quantified the percentage of DUI complaints lacking evidence that the officer arrested reported the incident, and analyzed all complaints against the officer involved in the collision with the decedents’ vehicle.
The expert’s statistical findings were admissible, the district court held in Cazares, because she was qualified to perform the analysis, employed reliable methodologies, and offered statistical evidence relevant to whether a code of silence existed in the police department. Criticisms of the expert’s decisions about how to categorize certain data, such as her classification of lack of certain documentation as indicating “failure to report,” were proper subjects for cross-examination or rebuttal expert testimony, but they did not render her methodology unreliable.
By contrast, conclusions by the same expert that a code of silence existed within the police department and that officers felt that they could drive with impunity did not pass the threshold test of reliability under Daubert and Rule 702. The district court characterized these as “qualitative conclusions,” as distinguished from statistical analysis. The qualitative conclusions fell outside the statistician’s expertise in data management and her analytic skills, and she lacked the requisite training, knowledge, education, or experience regarding police departments, the Chicago Police Department, law enforcement policy, or administrative investigations involving law enforcement to offer them. “[T]oo great an analytical gap between the data and opinion proffered” also required exclusion of the qualitative conclusions, under Gen. Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997).
The admissibility distinction drawn in Cazares between a statistician’s qualitative analysis and quantitative conclusions that do not represent application of data analysis expertise is consistent with and builds upon prior authority from within and outside the Seventh Circuit.
The Cazares decision echoes a prior decision from the Northern District of Illinois in a Monell claim case. In Obrycka v. City of Chicago, No. 1:07-cv-02372, 2011 U.S. Dist. LEXIS 70018, at *15–30 (N.D. Ill. June 29, 2011), the district court admitted quantitative analysis from a qualified statistician about excessive-force complaints against city police officers but excluded his qualitative opinions characterizing data as pointing toward a code a code of silence. While the expert’s quantitative background provided sufficient foundation for his data analysis, “nothing in his academic or professional pursuits provide[d] a basis for the conclusions he presume[d] to draw from the data.” Id. at *21. Rather, the expert’s conclusions were modeled off of questions presented by plaintiff’s counsel and amounted to inadmissible “subjective belief or speculation.” Id.
In a much different context—antitrust litigation—the U.S. Court of Appeals for the Eleventh Circuit imposed essentially the same limits on testimony from a statistical expert as the Northern District of Illinois did in Cazares and Obrycka. In City of Tuscaloosa v. Harcros Chems., Inc., 158 F.3d 548 (11th Cir. 1998), the plaintiffs alleged that chemical companies engaged in a price-fixing conspiracy. A statistician’s market-share analysis was admissible as within his competence as a statistician, helpful to the trier of fact, and the product of well-established and reliable methodologies. Id. at 562, 565–67. However, “[h]is characterizations of documentary evidence as reflective of collusion, and his characterizations of particular bids as ‘signals’” of a conspiracy, were inadmissible because they did not reflect application of his statistical expertise and would not assist the trier of fact, which was capable of determining whether or not to draw such conclusions without expert assistance. Id. at 565, 567.
Earlier this year, in a product liability action against the manufacturer of a pharmaceutical drug used to treat osteoporosis, the U.S. District Court for the Northern District of Alabama similarly limited a statistician’s testimony to “statistical methods or other matters within his expertise.” Jones v. Novartis Pharms. Corp., No. 2:13-cv-00624-VEH, 2017 U.S. Dist. LEXIS 10849, at *79 (N.D. Ala. Jan. 26, 2017). The plaintiff, who alleged that the defendant’s drug was responsible for a fracture requiring surgery, retained an expert statistician with experience designing statistical aspects of clinical trials for pharmaceutical drugs and medical devices. The expert offered opinions on general causation—whether the drug had the potential to cause the plaintiff’s injury—and on the adequacy of the manufacturer’s clinical trials and compliance with Food and Drug Administration requirements. The district court excluded these opinions on general causation, clinical trials, and regulatory compliance, finding the plaintiff’s statistician unqualified to offer opinions requiring medical expertise and further holding that his methodology was unreliable. Id. at *75–108.
Experts qualified to undertake statistical analysis are retained in a wide variety of cases for testimony bearing upon key liability questions. But their proposed testimony sometimes exceeds the bounds of statistical analysis and expertise and crosses into improper explanation of the cause or the meaning of the data. When a plaintiff intends to use a statistician to supply a bottom-line opinion that the jury could reach on its own or that would require expertise beyond a background in statistical analysis, a body of case law now joined by Cazares provides an important tool for limiting the expert’s testimony to statistical evidence.
“Seventh Circuit Report: Daubert by the Numbers,” by Elaine M. Stoll was published in Daubert Online (Volume 1, Issue 25), a DRI publication, on October 13, 2017. To view the article online, please click here. Reprinted with permission.