Medicare’s first public effort to identify hospitals with patient safety problems has pinpointed many prestigious teaching institutions around the nation, raising concerns about quality at these places but also bolstering objections that the government’s measurements are skewed.

The Cleveland Clinic, Mount Sinai Hospital in New York City, Geisinger Medical Center in Danville, Pa., and Massachusetts General Hospital in Boston were among those places having substantially more complications than the average hospital, according to data evaluated by the Medicare program. Major teaching hospitals were nearly 10 times as likely as other hospitals to be rated as having high rates of serious complications, a Kaiser Health News analysis of the data shows.

The new ratings are part of changes being phased in from the 2010 federal health law. A central tenet of the law was to tie Medicare reimbursement to a variety of measures, including how patients rate their stays, readmission and mortality rates and how closely hospitals adhere to basic guidelines for care.

The effort begins in October with a few initial metrics, and Medicare is poised to add the patient safety measures to the mix as early as the fall of 2013. The Obama administration believes adding such financial incentives into Medicare, the nation’s largest insurer covering 47 million seniors and disabled, will prod hospitals to lower costs and improve treatment.

But in January, a panel created by the National Quality Forum, a nonprofit organization that advises Medicare, recommended against using the patient safety measure for payment “due to concerns about the reliability of the data sources.” Dr. Atul Grover, head of public policy at the Association of American Medical Colleges, a Washington trade group, said, “They need to either revisit and refine it or drop it completely.”

The federal Centers for Medicare & Medicaid Services is already publishing the information on its Hospital Compare website for consumers to see. And some insurers may add it to the factors they consider when deciding how much to pay hospitals. “I am encouraged they are expanding the measures,” said Sam Ho, chief medical officer at United Healthcare.

Officials at many of the hospitals listed as having high rates of complications say the measures are distorted in ways that exaggerate problems at hospitals that treat lots of complicated cases or very sick patients.

“Not all of these metrics are ready for prime time,” George Blike, who oversees safety at Dartmouth Hitchcock Medical Center and Mary Hitchcock Memorial Hospital in Lebanon, N.H., which Medicare ranked as having a high rate of complications. “It’s unfortunately going to create a lot of confusion for the public.”

The Medicare data show high rates of serious complications for elderly patients at 3 out of 10 major teaching hospitals, including some of the biggest institutions in Boston, Philadelphia, Los Angeles, Cleveland and Chicago.

The data cover only Medicare patients, but the government believes that is indicative of a hospital’s overall quality. Medicare calculates each hospital’s overall, or “composite,” rate of complications, by looking at the frequencies of punctured lungs, blood clots after surgery, wounds that split open after an operation, bedsores, catheter and bloodstream infections and broken hips from falling after surgery. Shaheen Halim, who directs Medicare’s division measuring hospital quality, said the indicators appropriately factor in the different mix of patients a hospital sees, “leveling the playing field for comparisons.”

Dr. Donald Goldmann, a senior vice president at the Institute for Healthcare Improvement, said the Medicare billing records – the basis of the patient safety evaluations – are not refined enough to properly distinguish between the various levels of illness and health problems among patients. Goldman’s view is notable because his organization was created by Dr. Donald Berwick to promote quality based health care. Berwick was Medicare’s administrator when the agency adopted the patient safety indicators.

“These patient safety indicators, they’re not really well risk-adjusted,” Goldmann said. “We’re using crude, not very well validated administrative data to make comparisons.”

The indicators were never intended to compare hospitals, said Gina Pugliese, a patient safety expert at Premier, an alliance of 2,300 hospitals. “They were developed and intended to help hospitals flag events they could drill down and spend more time on,” she said.

Dr. Gregg Meyer was one of the people who helped create the formulas a decade ago when he was working at the federal Agency for Healthcare Research and Quality. Now in charge of patient safety at Massachusetts General Hospital, he said he looked into Medicare’s report of high rates of complications for his hospital and found no actual harm occurred.

“They were supposed to be hypothesis generating,” he said of the patient safety indicators. “It is to my horror that they are used to create conclusions. Using this as a mechanism to report something conclusive about safety is what we would term politely an off-label use.”

Even leaders of some hospitals rated as better than average have misgivings about the comparisons. “The use of the data to compare one institution to another is flawed,” said Dr. Evan Benjamin, a senior vice president for quality at Baystate Medical Center, part of Tufts Medical School. Baystate, located in Springfield, was the only Massachusetts teaching hospital rated as having lower complication rates.

In its summary rate for complications, Medicare identified 190 of 3,330 hospitals as having very high levels. Of those, 82 were major teaching hospitals, according to the KHN analysis. While 31 percent of major teaching hospitals were categorized as having high rates of complications, less than 4 percent of other hospitals—those with no residents or just a handful of them — were flagged as having high rates.

Barbara Rudolph, senior science director at The Leapfrog Group, which evaluates hospitals, said the high complication rates at big teaching hospitals might be real. “It’s much more difficult in a large institution to adequately train everyone to do the right thing,” she said. “You tend to have more residents and fellows flowing through.”

That’s not what Cleveland Clinic officials say they discovered when they delved into the data. Dr. Shannon Phillips, a quality and patient safety officer at The Clinic, said the clinic’s high rates of accidental tears and lacerations and serious blood clots were because “people are careful at documenting, almost to a fault, things that are incidental to the case.”

At Mary Hitchcock Hospital, Blike said one case was billed as an infection when physicians noted redness and swelling at the place where a catheter had been inserted. The doctors ordered a blood sample to test for infection. “It turns out three days later this patient didn’t have a spike in fever, didn’t need antibiotics and” the blood test did not find anything, Blike said.

Dr. Ira Nash, chief medical officer at Mount Sinai, said academic medical hospitals perform more of the kinds of procedures that can result in the complications Medicare is counting. For instance, he said electrophysiology studies, which help determine if the heart’s electrical system is working properly, require the insertion of electrode wires. Those can puncture the lung. While it might be fair to look at the percent of electrophysiology studies that resulted in a punctured lung, Medicare looks at the rate of such punctures among the entire hospital population, he said. “It completely ignores the difference in the kind of work we’re doing and the kind of patients we care for,” Nash said.

Not all the hospitals ranked as having high complication rates dispute Medicare’s assessment. “We’re perfectly fine with the way CMS does public reporting data,” said Dr. John Bulger, chief quality officer at Geisinger. “At Geisinger, we would never shy away from the number and say, ‘We don’t need to get better than this.'”

But Mass General’s Meyer predicted that many hospitals will react to the publication of the patient safety data either by instructing those who fill out the billing records to change what they include, or by lavishing staff attention on the areas flagged by Medicare even if they aren’t a real problem. “You’d be throwing resources that would really be better spent taking care of patients,” he said.

Medicare’s move toward using the measures in payment has already prompted some hospitals to pay much more attention to how they are billing.

Janis Orlowski, chief medical officer at Washington Hospital Center in the District of Columbia, said hospital officials examined the cases that led Medicare to rate her hospital as having a high rate of accidental cuts and lacerations. They found most of those cuts had been intended by the surgeon, but erroneously billed to Medicare under the code for an accidental cut. Orlowski emphasized that the hospital didn’t gain financially by the error, as both codes pay the same amount.

“Even though on the [Hospital Compare] Web site today we’re worse on national rates, we can tell you 100% we have no problem with accidental cuts and punctures,” Orlowski said.

This article was produced by Kaiser Health News with support from The SCAN Foundation.

jrau@kff.org