By Anthony Califano and Lynn Kappelman
On September 28, 2012, Judge Nicholas G. Garaufis of the U.S. District Court for the Eastern District of New York granted the City of New York’s (“City”) unopposed motion seeking approval of the City’s retooled entry-level firefighter exam (“Exam 2000”) in United States v. The City of New York, No. 07-CV-2067, 2012 U.S. Dist. LEXIS 140766 (E.D.N.Y. Sept. 28, 2012).
The United States originally filed this lawsuit against the City in 2007, alleging that the City’s entry-level firefighter exams and applicant ranking had an unlawful disparate impact on African-American and Hispanic applicants. Id. at *5-6. The Court agreed, finding that the City’s procedures for screening and selecting entry-level firefighters violated Title VII, the Equal Protection Clause, and the Civil Rights Act of 1866, along with New York state and local law. Id. at *6-7; see also Docket Nos. 294 and 385. Consequently, the Court issued an order requiring the City to develop a non-discriminatory test for entry-level firefighter applicants. Id. at *6. We have blogged about this litigation in the past, as it raises a myriad of interesting Rule 23 and subsidiary issues for employers.
To that end, the City retained an independent test development company and ultimately moved the Court to approve Exam 2000. Id. at *7-8. Exam 2000 is designed on a 100-point scale, and the passing grade is 70 or higher. Id. at *8. Candidates who achieve the cut-off score or better are then ranked on a list of potential candidates for further consideration. Id. at *8-9. The Court-appointed Special Master endorsed Exam 2000 as proposed. Id. at *9.
Although the City’s motion to approve Exam 2000 was unopposed, Judge Garaufis conducted his own disparate impact analysis on Exam 2000. He evaluated whether the test had an adverse impact on minority candidates, and if so, whether the test was nevertheless justifiable as job-related. Id. at *9-10. Judge Garaufis evaluated the impact of two aspects of Exam 2000 — ranking candidates and using a pass-fail cut-off score. Id. at *9-10. With respect to ranking candidates, Judge Garaufis observed that “[a]ll the parties agree that the use of Exam 2000 score results in rank-ordering of candidates . . . would produce little to no difference in hiring between minority and white candidates.” Id. at *10. Referencing the EEOC’s “Four-Fifths Rule,” Judge Garaufis noted that “[m]inority candidates from all but one minority group will be certified for further processing at a rate that is at least eighty percent of the rate of success of white candidates” over the likely four-year use of the exam. Id. at *10-11. The “Four-Fifths Rule” refers to the EEOC’s Regulation stating that “[a] selection rate for any race . . . which is less than four-fifths . . . of the rate for the group with the highest rate will generally be regarded by the federal enforcement agencies as evidence of adverse impact . . .” 29 C.F.R. § 1607.4(D). Judge Garaufis noted that Native Americans had a projected success rate of less than eighty percent of white candidates for two of the four years that the City will use the exam, but he remarked that “standard deviation analysis suggests no adverse impact, as there is less than two units of standard deviation for the disparity for these years.” Id. at *11. Accordingly, Judge Garaufis approved the ranking aspect of Exam 2000. Id. at *12.
When the Court reviewed the statistics concerning candidates’ pass-fail rates on Exam 2000, it found that the results were different depending on the analysis employed. Id. at *12-13. Although the pass rate of all minority candidates was close to the pass rate of white candidates (ninety-seven to ninety-nine percent), the difference was statistically significant (i.e., greater than two units of standard deviation) for all minority groups except for women. Id. The Court held that it is inappropriate to rely on the Four-Fifths Rule by itself, and that it is also important to consider whether the differences in selection rates among groups are statistically significant. Id. at *13. Here, after reviewing the statistical significance of the test’s impact, the Court was concerned that the differences in pass-fail rates between minority and Caucasian candidates, even if relatively minor, might not be the result of mere chance. Id. (citing FDNY Firefighter Test Development and Validation Report).
The City argued that even if Exam 2000’s pass-fail component has an adverse impact, it is defensible as job-related. Id. at *14-19. Judge Garaufis agreed. Id. at *17-19. The City’s independent testing consultant performed a “criterion validity study” — a study that Judge Garaufis recognized as “among the most valuable methods in showing a connection between the use of a particular employment selection device and success on the job . . .” Id. at *16-18. The study included “administering a draft version of Exam 2000 to a large number of FDNY firefighters and comparing the score of each incumbent firefighter on Exam 2000 to that firefighter’s scores in Fire Academy examinations.” The study also compared the Exam 2000 scores to the firefighter’s scores on specially-created evaluations from their own supervisors. Id. at *16. Empirical data from the study revealed a significant correlation between success on Exam 2000 and success in the academy and on supervisor evaluations. Id. at *16-17. Accordingly, Judge Garaufis found that the pass-fail cut-off score was defensible as job related and held that the City could use Exam 2000.
One lesson from Judge Garaufis’s decision is that employers should not rely solely on the Four-Fifths Rule in determining whether a practice has a disparate impact. Id. at *13. Here, despite statistical significance in the difference between minority and non-minority pass rates, the City successfully used expert evidence on the test’s job-relatedness to validate its employment selection tool.