Survival As a Quality Metric of Cancer Care: Use of the National Cancer Data Base to Assess Hospital Performance

Dr. Pennell discusses survival as an important indicator of the quality of cancer care with author Lawrence Shulman.
Related Article: Survival As a Quality Metric of Cancer Care: Use of the National Cancer Data Base to Assess Hospital Performance
Support for JCO Oncology Practice podcast is provided in part by AstraZeneca, dedicated to advancing options and providing hope for people living with cancer. More information at AstraZeneca-us.com.
Hello, and welcome back to the ASCO Journal of Oncology Practice podcast. This is Dr. Nate Pennell, Medical Oncologist at the Cleveland Clinic and Consultant Editor for the journal. Over the last decade, there's been an important movement to try and improve the quality of medical care in the United States. But to do that, of course, we have to have reliable measures of quality.
But how do you really do that? Is it enough to measure compliance with guidelines or expert recommendations for high quality care? Ultimately, you might think that high quality care should lead to improvements in survival for cancer patients. And naturally, that leads to the question of whether survival could be used to compare the quality of care between different practices or different hospitals.
Joining me today to talk about this topic is Dr. Larry Shulman, Deputy Director for Clinical Services of the Abramson Cancer Center at the University of Pennsylvania and leader of the Cancer Quality Program for the University of Pennsylvania's health system. He's also the former chair of ASCO's Quality of Care Committee, former chair of the Commission on Cancer's Quality Integration Committee, and currently the chair of the Commission on Cancer.
Today we'll be discussing his recently published paper, Survival as a Quality Metric of Cancer Care, Use of the National Cancer Database to Assess Hospital Performance. Larry, thank you so much for joining me today.
Thanks for having me.
So can you give us a little bit of background? What are the some of the challenges in measuring quality of care, and what led you eventually to do this study?
Well, a decade ago, we were doing very little to measure the quality of cancer care in any respect. And then ASCO, in the early 2000s, started the QOPI program. And at around the same time, the Commission on Cancer began a quality program as well. And as you mentioned, most of the quality metrics that are included in those two programs are process measures, that the patient with a certain stage of disease get the appropriate treatment and so on. And those are very important metrics, and we've learned a lot from measuring those.
But at the same time, people have complained, including the public and the regulators, that we really need to know outcome quality measures. And the most important outcome measure for many people is survival. And survival is really, presumably, a culmination of all the aspects of care, not just whether you gave a particular treatment, but the other aspects of care that help patients to either do well or not do well. So that appeared to be an important measure.
I will say that there are a number of centers around the country that published their own survival metrics on their website with a variety of comparisons. And we were concerned that that was not really truth in advertising, and we wanted to understand measuring survival at the hospital level and also at the a hospital type, the class of hospitals. And that's what led us to do this study.
And that makes perfect sense. I mean, ultimately, when you're just measuring metrics, perhaps through ASCO's QOPI program, ultimately you're making an assumption that that's leading to better outcomes. But it would be nice to have some proof that that was true.
So could you please walk us through your paper a little bit? So what were you trying to accomplish with this particular study?
We queried the National Cancer Database. The National Cancer Database includes cancer registry data from 1,500 hospitals across the US that are accredited by the Commission on Cancer. And our estimate is that that covers about 70% of the cancer patients in the country. So this is a very robust database, and currently there are about 36 million patients in this database.
So we decided to look at patients with two different diseases, and we had very specific reasons for including them. We looked at patients who had Stage 3 breast cancer, and we did that because those patients ordinarily receive surgery, systemic therapies, and radiation. And we wanted to assess some disease where all the modalities were involved.
In addition, most of the technologies that we need to treat breast cancer patients are available at hospitals throughout the country, community hospitals as well as academic centers. The capability to do good breast surgery, give the types of systemic therapies we give for breast cancer and radiation, are widely available.
We also chose advanced non-small cell lung cancer, and we did that because that's a changing paradigm. The use of genomics to identify patients who have targetable mutations is not widely used throughout the country, and we wanted to see whether there were differences that we could assess in different hospitals and different hospital types.
And so what we did was we looked at both unadjusted survival, which is basically how many patients were cared for with a particular disease, and the death rates. And then we also looked at risk-adjusted survival because the patient populations are not the same at all hospitals. And so we risk adjusted for a number of variables, including age and gender, ethnic background, socioeconomic status, comorbidities, and insurance status.
So I won't ask you to delve into the details of how the analysis was done, but what did you find?
So we found three major points. One is that we looked at survival at the individual hospital level across the 1500 hospitals. And when we did risk adjustment, we found that very, very few of the hospitals had survivals of their patients that were either statistically better or statistically worse than the mean. And in fact, it turned out to be about 15 hospitals out of the 1500 hospitals that had survivals that were statistically better or worse.
And there are two reasons for that. One is that the survivals were in a pretty tight distribution, so that there wasn't a wide splay of survival differences among the different hospitals. And secondly, even large hospitals, comprehensive cancer centers, have relatively few patients with a particular stage of disease that can be assessed for survival.
So we felt that at the individual hospital level, this would not be a good quality metric to distinguish levels of care, and that some of the people who have argued for using survival to ultimately assess the quality of care of a hospital, this is probably not where we want to go.
The second thing that we found was that we looked at hospital types. So we aggregated hospitals into four groups. One was NCI-designated Comprehensive Cancer Centers. The second were academic cancer centers that were attached to a medical school and a training program. Third was large community hospitals with more than 500 new cases a year. And the fourth were small community hospitals with 500 or less cases a year.
And what we found was that when we aggregated those hospitals by these categories, there was a difference in survival. And the best survivals were seen at the NCI centers, followed by those in the academic cancer centers. Third were the large community hospitals, and fourth were the small community hospitals.
We spent a lot of time with the editors of JOP trying to assure ourselves that the statistical evaluations were valid. And in the end, they felt they were valid enough, obviously, to publish the manuscript. So I really think that the findings are real. And the question is why is there a difference in survival by hospital type?
And this study doesn't answer that question, but I think it gives us enough …