Surgeon Scorecard

About the Project

Picking a surgeon can be a life-or-death decision. Yet somehow, it remains a choice Americans make based mostly on subjective recommendations and word of mouth – and without any hard facts about outcomes.

Last year, ProPublica changed that with “Surgeon Scorecard,” an investigation that lifted the curtain on the complication rates of more than 16,000 surgeons nationwide and enabled Americans to compare their results for the first time.

The stories and database, by team members Olga Pierce, Sisi Wei and Marshall Allen, revealed vast differences in performance between doctors who perform eight common elective surgeries. Using five years of Medicare data, they developed a methodology for calculating each surgeon’s complication rate, adjusted for differences in their patient mix.

Deep reporting showed how the medical system fails to police itself, a culture that allows patient harm to proliferate. A visual database let users find potential surgeons and see, for the first time, rigorous data about their surgical outcomes. The data is complex but the visualizations met the challenge with innovative displays that are easily understood but never dumbed-down for the user.

Our approach to explaining our work set a new standard for transparency for very complex data stories, including:

• A methodology post aimed at regular readers.
• A 21-page technical whitepaper aimed at researchers, detailing all of our assumptions and the guts of our math.
• An exhaustive debate held in the open – published on our site and in public events – with researchers kicking the tires on the new analysis.
• A page tracking what people were saying about Surgeon Scorecard – both the supporters and the most vocal critics.

The project was met with controversy, as new concepts usually are. Some praised it as a breakthrough in transparency that would prod underperformers to shape up or be shamed. Critics raised questions about whether it was sufficiently precise.

But its impact is impossible to ignore:

• Several hospital administrators said the release of Surgeon Scorecard prompted them to confront doctors they had previously identified as poor performers. One began sharing individual surgeons’ infection rates with the staff – a step he had been reluctant to take – after ProPublica released its numbers.
• Teams at Johns Hopkins University and Cleveland Clinic are using the data for studies that they hope will lead to safer elective surgery.
• For patients, the Scorecard has clearly become a critical resource. The database has been viewed millions of times. Ultimately, critics and fans alike have acknowledged that by daring to name names the project has forever changed American medicine, empowering and enlightening patients as never before.

“The old way – where all the information was privileged and known only among physicians – is gone. And it is not coming back,” wrote Dr. Ashish Jha, a leading patient safety researcher at the Harvard School of Public Health, with whom ProPublica consulted.