Q & A: Surgeon scorecard offers transparency, creates a path for improved care

Patient Safety Monitor Insider

October 7, 2015

The nonprofit news organization ProPublica has never been one to shy away from controversy, particularly when it comes to healthcare quality. In the past, it has published databases in which patients can see whether their doctor is taking money from pharmaceutical companies, and what medicine he or she is prescribing as a result.


In July, the news organization took another shot at tearing down the barrier between patients and healthcare organizations. This time, it targeted surgeons. With help from two dozen physicians, along with Medicare billing records from 2009 to 2013, ProPublica created the "Surgeon Scorecard" (https://projects.propublica.org/surgeons), which exposes complication rates for more than 16,000 individual surgeons operating at more than 3,500 hospitals.

Although many within the medical field celebrated this new step in transparency, the Scorecard has become relatively contentious, particularly because it unabashedly calls out surgeons based on their performance, both good and bad. ProPublica restricted its analysis to eight low-risk elective surgeries performed on healthy patients, but the organization was criticized for relying largely on unreliable readmission data to determine complication rates. Critics also argued that Medicare data represented a limited number of surgeries.

In particular, Peter Pronovost, MD, senior vice president for patient safety and quality, director of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine in Baltimore and a leader in the field of patient safety, told ProPublica the measures were not valid since the Scorecard fails to account for the variation within patient readmissions.

Patient Safety Monitor Journal spoke with three of the country's leading patient safety experts to get their reactions to the Surgeon Scorecard and how the data could impact transparency and quality initiatives at the provider level.

(Editor's note: Some responses have been lightly edited for space and clarity.)

Q: Do you believe the Surgeon Scorecard is an effective and fair way to measure quality care?

Joe Kiani, founder of the Patient Safety Movement Foundation (http://patientsafetymovement.org): I believe it's a great step forward. ProPublica seems to have done a thoughtful job on how to rate surgeons of elderly patients, and it certainly should help detect surgeons that are outliers, good and bad. That alone is a huge service to patients.

Ultimately, hospitals should be volunteering this information and more to create transparency and healthy competition for the betterment of patients.

Tejal K. Gandhi, MD, MPH, CPPS, president and CEO of the National Patient Safety Foundation (NPSF) in Boston: I believe it is a step in the right direction. The Surgeon Scorecard is not perfect, but I believe the reporters, by soliciting advice and input from researchers in the field, really did the best they could to risk-adjust and show the most accurate picture possible. More transparency is better than none. They also acknowledge the limits of the data that they used, so it is important to note that as well.

(Editor's note: Gandhi wrote about the Surgeon Scorecard in a blog post: http://npsf.site-ym.com/blogpost/1198150/224318/When-We-Share-Data-Patients-Win.)

Robert M. Wachter, MD, professor and interim chairman of the department of medicine, and chief of the division of hospital medicine at the University of California, San Francisco (UCSF): The devil's in the details, but I think it's an important concept, and I think we have to do this kind of measurement. The science is still a little bit rudimentary, so there are some flaws, but it's about as good as we can do today, and I think by virtue of doing this kind of thing, it will drive this process and get better over time.

Q: From a public health/patient safety perspective, how important is it to be transparent with surgical outcome information?

Kiani: It is critical. Full transparency is the only way we will eradicate preventable deaths in hospitals.

Gandhi: From a patient safety perspective, we at the National Patient Safety Foundation feel this is extremely important, but not so we can point fingers or punish people. What needs to come out of this is learning is: Why do the high performers have such low complication rates? What do they do that others don't? And how can we standardize the practices of the high performers to raise everyone's outcomes to better levels, improving care and outcomes? That's how I'd like to see these data used.

Wachter: I think the experience so far has been that putting out information about quality or safety or patient experience at the level of hospitals, health systems, or doctors has worked better than people thought, has motivated more change than people thought, and it's probably inexpensive when you think about other potential ways of doing it, like changing the payment system.

What is interesting is there is not a whole lot of evidence so far that a huge number of patients are going onto these websites, whether it's hospitals or doctors, and really basing their decisions on where they go on what they see. And yet it seems like probably some combination of shame and pride among doctors and healthcare administrators does seem to be enough to move the needle.

I can tell you, at the medical center level, our data that is reported on Hospital Compare, it's not clear that data really influences whether a patient in Marin County will come to UCSF or a different hospital. But I can tell you, we look at it a lot, and we pay a lot of attention to it. When we're not doing well, it motivates us to improve and develop a program in ways that are greater than I expected when all of this started 10 or 15 years ago.

What we've learned is: Transparency is both the right thing to do for patients to help them make decisions, but it's also very powerful and a relatively inexpensive tool for getting systems to get better at what they do.

Continue reading this article in the October issue of Patient Safety Monitor Journal. Visit here to login or subscribe.