Real-World Examples of AI Bias in Healthcare

The Setup

A widely deployed risk-prediction algorithm used healthcare cost as a surrogate for illness severity to identify patients for high-risk care management programs.

The Problem

Healthcare expenditure isn't a neutral variable; it reflects access disparities. A landmark study found Black patients with the same number of chronic conditions incurred $1,800 less in annual healthcare costs than white patients, so the algorithm systematically prioritized healthier white patients over sicker Black patients.

What Changed

Replacing cost with direct health indicators increased the share of Black patients flagged for extra care from 17.7% to 46.5%.

Key Insight

Bias can emerge even when sensitive attributes like race are excluded entirely, because proxy variables encode them implicitly.

The Setup

Diagnostic dermatology AI systems trained on image datasets used to detect skin cancer, including melanoma.

The Problem

A study analyzing 21 datasets found only 10 images of brown skin and 1 image of dark skin out of more than 100,000 total samples. Models trained on this data show lower sensitivity and specificity for darker skin tones, increasing both false negatives (missed cancers) and false positives (unnecessary interventions).

Clinical Impact

Delayed melanoma diagnosis and increased morbidity from late-stage detection in underrepresented populations.

Key Insight

This isn't a simple data-volume problem; it reflects structural bias in clinical image collection and underrepresentation in dermatological research pipelines.

The Setup

AI-driven dosing algorithms for warfarin, an anticoagulant whose correct dose depends heavily on genetic variants that influence drug metabolism.

The Problem

Algorithms were trained primarily on European genetic data, excluding key variants more prevalent in African populations.

Clinical Impact

African American patients experienced higher rates of overdosing (supratherapeutic INR levels) and increased risk of bleeding complications.

Key Insight

Model generalization fails when real biological variability is treated as statistical noise instead of a core signal the model needs to account for.

The Setup

Pulse oximeters, the devices that estimate blood oxygen saturation, feeding data into downstream clinical AI systems.

The Problem

Devices overestimate oxygen saturation in patients with darker skin, delaying detection of hypoxia (dangerously low oxygen).

Clinical Impact

Studies have linked this to delayed treatment escalation, worse organ function, and increased mortality in Black patients.

Key Insight

When AI systems rely on biased sensor inputs, the bias is amplified downstream. Even a well-designed model inherits flawed measurements it has no way to detect.

The Setup

Genomic AI models used for disease prediction and personalized treatment planning.

The Problem

Over 80% of genomic datasets come from individuals of European ancestry, reducing accuracy for non-European populations.

The Feedback Loop

Underrepresentation leads to inaccurate predictions, inaccurate predictions reduce trust and participation from affected communities, and reduced participation perpetuates the original underrepresentation.

The Pattern

Research has documented AI clinical decision support systems recommending more aggressive treatment for white patients and more conservative approaches for Black patients with clinically similar presentations, patterns that trace back to historical treatment data the models were trained on.

The Scale of the Disparity

Overall mortality for non-Hispanic Black patients runs roughly 30% higher than for non-Hispanic white patients, a disparity that AI models trained on historical outcomes data risk reinforcing rather than correcting.

A More Recent Example

A November 2025 Cedars-Sinai study of large language models generating psychiatric treatment recommendations found explicit and implicit racial bias: some models omitted medication recommendations for ADHD when a patient's race was stated, or suggested guardianship for depression specifically when racial identifiers were present.

Technical Root Cause

Models trained on historical treatment patterns reproduce the biases embedded in those patterns, effectively reinforcement-learning from biased past clinical decisions.

The Pattern

Not every case of AI bias originates in data. Automation bias describes clinicians over-trusting AI outputs and reducing their own critical evaluation of a recommendation simply because it came from a model.

The Implication

Even a genuinely unbiased model can cause harm if clinicians assume algorithmic infallibility and human oversight weakens as a result.