U.S. News Hospital Rankings Delayed Due to Data Errors

Photo by Meredith Foley

Massachusetts General Hospital must wait to learn whether it defended its place in U.S. News and World Report’s influential annual hospital rankings.

In a blog post published late Monday night, Ben Harder, U.S. News’ chief of health analysis, admits to finding errors in the data used to compile the yearly list. The results, which were set to go live on August 1, will need to be thoroughly reviewed, delaying their release until August 8.

U.S. News creates its list using physician surveys and patient safety and survival data, plus information about nurse staffing and other hospital processes. According to the post, “rankings in the 12 data-driven specialties that hospitals received on a preliminary basis on Tuesday, July 11, 2017, were incorrect.” The errors were apparently related to efforts made this year to analyze socioeconomic profiles of patients and other nuanced factors that affect care and outcomes.

“We improved the way we evaluated the care of low-income patients, patient safety and hospitals that accept very sick patients from other hospitals,” Harder writes. “Despite the complexity, we are confident that these changes allow us to provide patients with better information about their hospital choices.”

Editor and Chief Content Officer Brian Kelly adds that, “we publish rankings and information for the benefit of patients navigating complex health care decisions, so ensuring the accuracy of that information is of utmost importance.”

Still, the hiccup may provide ammunition to critics who already feel that rankings try to cram too many complex data points into a neat rating system, putting them at risk of feeling arbitrary.

Even Harder told Boston last year that the Honor Roll—U.S. News’ widely shared top-10 list—presents only a surface-level view of hospital performance, directing comparison-shopping patients instead toward specialty-specific ratings.

“[With the Honor Roll,] what you end up with is an impressionistic sense of how good a hospital might be,” Harder said, “but you’ve just obscured all the variation within the hospital that’s actually very important.”

Nonetheless, the medical community is likely eagerly awaiting this year’s revised list. The rankings have traditionally treated Boston hospitals quite well: Last year, Mass General took third overall, while Brigham and Women’s Hospital took 13th. A number of local institutions, including Mass Eye and Ear and Dana-Farber Cancer Institute, also took home honors in specialty care.