Some 14,000 people have recently opted in to a case that is effectively putting AI hiring systems on trial. The participants are all at least 40 years old and claim they were unfairly denied jobs after being screened by Workday's recruiting systems that score, sort and rank applicants.

The sweep of the case, Mobley v. Workday Inc., is large. It considers how antidiscrimination laws apply to AI systems and who is liable, the vendor or the customer. Customers aren't being sued; Workday is. Its defense is that employers -- not Workday -- control the hiring decisions and outcomes.

If that wasn't enough for CIOs to consider, the case is also becoming a fight over the mathematics used to detect bias, with both sides arguing that the same data proves their case. And that raises questions about whether bias audits can be trusted.

The importance of the case was noted by the Equal Employment Opportunity Commission. In 2024, it filed an amicus brief in support of Mobley, though it did not address the merits of the case. The agency -- then under the Biden administration -- warned that "if Workday's algorithmic tools in fact make hiring decisions (and on the scale Mobley suggests), it would be all the more important to ensure that Workday complies with federal anti-discrimination law."

Related:Workday’s AI reset: Agents and the race to remake SaaS

To be clear, Workday claims its systems are not biased. It argues that humans have complete control and make all the critical decisions. The plaintiffs argue otherwise. The case is a long way from being decided.

Derek Mobley, a Black man over 40 and a Morehouse College graduate, filed the case in February 2023 after he was rejected from more than 100 jobs he applied to through Workday's platform.

Disparate impact and AI hiring liability 

At the center of the case is a key question: whether a protected group -- people over 40, women and racial minorities -- was harmed, even if there was no intentional discrimination. This is called disparate impact analysis.

U.S. District Judge Rita Lin of the Northern District of California, who is hearing the case, wrote in a court order that the "critical issue at the heart of Mobley's claim is whether that system has a disparate impact on applicants over forty." She allowed the opt-ins, or the applicants claiming they were harmed, after Mobley showed enough to suggest the harm might be systemic.

Workday bias audit: Four-fifths rule vs. standard deviation analysis 

The methodological dispute in Mobley turns on a mathematical problem: both sides have analyzed largely the same numbers and reached opposite conclusions.

Related:The hidden high cost of training AI on AI

In late 2024, Workday published the results of an external bias audit covering 10 of its largest enterprise customers, conducted using the methodology of New York City's Local Law 144. The NYC law requires independent bias audits of automated hiring tools. The conclusion: "no evidence of disparate impact" on race or gender.

Mobley's lawyers ran their own analysis on the same published numbers. In their second amended complaint filed in January, they concluded the data showed statistically significant disparities against both African American applicants and women -- disparities, plaintiffs alleged, with odds greater than one in a quadrillion that the system was race-neutral.

Workday used the "four-fifths rule" -- a test recommended by the U.S. Equal Employment Opportunity Commission that flags a system as potentially biased only when one group's selection rate falls below 80% of the highest-selected group's rate.

Mobley's lawyers used standard-deviation analysis. It signals potential bias when hiring-rate differences across groups exceed what chance alone would predict. 

But Mobley's attorneys removed that statistical argument from the third amended complaint, filed in March.

In an email, Mobley's attorney, Lee Winston, confirmed that "the statistic from the earlier complaint is no longer in the operative complaint." But he did add that "discovery remains ongoing."

Related:What AI must learn from Roosevelt, conservation and 1929

A new filing suggests that the plaintiffs want more data from Workday, which may enable them to run a new analysis.

In April, the plaintiffs asked the court to compel Workday to turn over its bias-testing data, the source code for the testing, and the testing results. In previous filings, Workday has opposed this, claiming attributes such as algorithmic logic, if exposed, could be used by competitors, according to court papers.

Why AI bias audits can produce conflicting results 

The plaintiffs' motion underscores a broader challenge with AI systems. Outputs can shift or "drift" from their original behavior as the system gathers new data.

Bias testing is "an ongoing research challenge," said Jason Hong, professor emeritus at Carnegie Mellon University, whose research has focused on AI bias and auditing. "Right now, it's very chaotic," he said. He wasn't commenting on Workday's lawsuit.

Hong said the trouble starts with the word fairness, which has more than one definition when it comes to assessing bias. One method minimizes errors across the whole data set. A different one focuses on error rates, trying to ensure that the system's mistakes -- wrongly rejecting a qualified person, wrongly advancing an unqualified one -- happen at the same rate across groups. A third tries to ensure the system makes correct decisions at the same rate across groups.

But those definitions of fairness are mathematically incompatible.

Hong pointed to a 2016 paper by Alexandra Chouldechova, then a professor of statistics and public policy at Carnegie Mellon, "Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments," which underscores the limits of statistical definitions of bias: "It is important to bear in mind that fairness itself -- along with the notion of disparate impact -- is a social and ethical concept, not a statistical one," the paper notes. The paper shows that different statistical tests can measure different aspects of outcomes, and reach conflicting conclusions on the same data.

A Workday spokesperson, in an email, dismissed the plaintiffs' approach: "Plaintiff is taking the same data and running different analysis that simply is not scientific in this application."

Workday's own filings have raised concerns about the state of AI bias auditing. In a January 2023 public comment to New York City regulators on Local Law 144, the company urged regulators to "recognize the immature state of the AI auditing field" and argued that third-party AI auditors lack "a respected independent professional body to establish baseline auditing criteria or police unethical practices." Workday argued instead for allowing internal auditors, saying employers had strong incentives to ensure their tools were not used discriminatorily, since misuse would carry legal, financial and reputational consequences.

"The claims in the suit are false," Workday said in a statement. "Workday's AI recruiting tools don't make hiring decisions and are designed with human oversight at their core. Our technology looks only at job qualifications, not protected traits like race, age, or disability. We rigorously test our products as part of our Responsible AI program to confirm our tools do not harm protected groups."

Mobley alleges in the complaint that "the rejections -- often within hours or minutes of submission -- are consistent with the operation of these automated screening tools identifying and acting upon such proxy indicators of disability and health status, rather than any individualized assessment of his qualifications.

The political environment hasn't reduced the legal risk. President Donald Trump rejects the disparate-impact theory; in an executive order last year, he barred federal agencies from using it, arguing it forces hiring on the basis of race instead of merit. But the order doesn't address AI in hiring, bias or the need for audits. And it doesn't affect private litigation like Mobley.

CIOs should not rely solely on vendor AI audits

The Mobley v. Workday case may go on for years, but CIOS need a strategy now for independently auditing and overseeing AI hiring systems. The advice from the experts interviewed for this story is consistent: don't rely on the vendor's audit. Build internal oversight with technical, legal and ethics staff members who can question what the AI is doing and can override it.

Andrew Pery, an AI ethics evangelist at Abbyy, an intelligent automation company, said there is a misconception that a vendor's attestations and certifications are sufficient to manage the risk. "Nothing could be further from the truth," he said. Pery was speaking generally, not about the Workday case.

Effective oversight needs data scientists, technical staff, ethics specialists and human reviewers with the authority to override an AI decision, Pery said. Oversight of AI is also a board-level concern, he said. AI bias in hiring carries real consequences. "It impacts brand equity. It impacts customer loyalty. It impacts valuation, so governance is becoming part of ensuring that there's proper board-level controls implemented."

Strong governance only works if it can see the technical problems.

How AI hiring systems can use proxy data to infer protected traits 

AI systems, even if they're barred from using protected attributes such as gender, race or age, may rely on proxies like graduation year or full address to infer them, said Rodica Neamtu, a computer science professor at Worcester Polytechnic Institute. The system uses those proxies to make inferences a human never explicitly asked it to make.

"That's how bias starts creeping in," she said.

"Companies do not disclose enough about the tools that they sell, which means that it is quintessential to keep the humans in the loop," Neamtu said. Humans bring their own cognitive biases, but well-trained people who understand bias and how it develops would improve the process, she said.

"AI is a risk like any other mission-critical risk," said Carl Hahn, a partner at Steptoe LLP and former chief ethics and compliance officer at Northrop Grumman. 

"Management needs to establish effective controls and practices that govern AI systems and then audit whether those controls operate as designed."

The company that uses the AI is "ultimately responsible for the output of the audit and for demonstrating effective, robust and disciplined compliance," Hahn said.

"The vendor is simply contributing to the process."

About the Author