How accurate are hospital report cards?

A key statistic that consumer groups and the media often use when compiling hospital report cards and national rankings can be misleading, researchers report in a new study.

The statistic is called the mortality index. A number above 1.0 indicates a hospital had more deaths than expected within a given speciality. Lower than 1.0 means there were fewer than the expected number of deaths.

The study by Loyola University Health System researchers in the Journal of Neurosurgery illustrates how the mortality index can be misleading in at least two major specialities - neurology and neurosurgery. The index fails to take into account such factors as whether a hospital treats complex cases transferred from other hospitals or whether a hospital treats lower-risk elective cases or higher-risk non-elective cases.

'A hospital with a lower mortality index may not be a better hospital for patient care, but rather a place where the patient mix has been refined or limited,' said senior author Dr Thomas Origitano, chairman of the Department of Neurological Surgery, Loyola University Stritch School of Medicine.

There is no 'definitive or reliable source for rating the quality of overall neurosurgical care,' Origitano and colleagues wrote in the Journal of Neurosurgery, published by the American Association of Neurological Surgeons.

Researchers examined neurosurgical mortality data from 103 academic medical centres in the University HealthSystem Consortium. Hospitals with the worst mortality index tended to be Level 1 trauma centres with busy emergency rooms and a high percentage of Medicaid patients.

A Level 1 trauma centre with a busy ER is more likely to treat severe and complex cases such as head and spinal injuries from car accidents, injuries from falls or gunshot wounds. And the reason a high percentage of Medicaid patients is associated with a high mortality index is likely because Medicaid patients are more likely to have 'poor access to medical care, are poorly educated in health and hygiene, are uninsured and present only once their symptoms have become severe,' researchers wrote.

The study also found that in hospitals with the lowest mortality index, at least 87 percent of the neurosurgical cases were elective in nature. Elective surgery includes cases such as back surgery or decompression of a pinched nerve. Patients deemed to be at too high a risk do not undergo the surgery.

By contrast, non-elective surgery for such conditions as head injuries and spine infections generally has to be done even when the risks are high.

Researchers cited several other problems with rating systems. For example, report cards typically lump neurology and neurosurgery into one category, neurosciences. 'Although both services treat many of the same pathological processes, their performance at any given institution is by no means shared,' researchers wrote. 'This can be misleading if the neurology aspects of the rating system misrepresent the neurosurgical service or vice versa.'

Another common practice is using reputation as one of the main ranking criteria. This practice 'is at best subjective,' researchers wrote.

Researchers wrote that misleading information in report cards and rankings 'may falsely direct patients and their families to hospitals providing a lower level of neurosurgical care, or direct them away from hospitals providing a high level of neurosurgical care.'