The Pioneer accountable care organizations have long been the shining stars of the Affordable Care Act’s strategy to rein in the country’s out-of-control spending on health care.

The 32 organizations are part of a Medicare pilot project called for in the health law that could revolutionize the health system by paying doctors and hospitals for quality of service rather than  the volume of services. Pioneers would be offered a bonus for giving patients high quality care at a reduced cost. If they failed to hit certain quality targets or did not manage to reduce the cost of care, they would be dinged accordingly.

For the first year of the program, everything seemed like smooth sailing. But the pioneers appear to have hit their first pothole—and the administration is scrambling to make sure the project goes forward.

The problem: That pesky little part about accountability.

In February, the Centers for Medicare and Medicaid Innovation (CMMI) sent out the 31 proposed quality targets that pioneers would have to meet if they wanted to receive bonus payments. Bonuses would be based on their ability to meet those targets beginning in 2013.

But then, last week, 30 of the pioneers sent a letter to CMMI complaining that at least 19 of the quality targets had too little data behind them and were therefore unfair, unreasonable and even arbitrary. In light of the flawed metrics, the pioneers requested that CMMI wait until 2014, when CMMI would have another year of good data collection to set benchmarks, to start basing Pioneer pay on quality, the hallmark of the program.

“Pioneers need to know before April 2, 2013, if you accept our recommendations, so that we can make informed decisions regarding our ongoing participation,” concluded the letter, which was first reported by Inside Health Policy.

Opponents of the health law seized on the letter immediately. “This is what federally run delivery system reform actually looks like: a lot of failure and frustration,” wrote Peter Suderman, a senior editor at Reason magazine, in a blog.

Emily Brower, executive director of accountable care programs at pioneer Atrius Health, which signed the letter, called Suderman’s interpretation “completely unfair.” The ACO program “will work, and we want to work with CMS to make sure it does,” said Brower.

She said that the program has “been a total home run” for the health system so far. The pioneers have biweekly conference calls with CMMI, she explained and regularly work together to address problems as they arise. “We want to be measured on outcomes. We do well. This is very important to us,” Brower said. “There is no threat” that the pioneers will drop out of the program, she added.

The administration has staked much on this project and President Barack Obama even highlighted the concept in his State of the Union speech last month. “We’ll bring down costs by changing the way our government pays for Medicare,” he exclaimed. “Because our medical bills shouldn’t be based on the number of tests ordered or days spent in the hospital – they should be based on the quality of care that our seniors receive.” CMS Acting Administrator Marilyn Tavenner has scheduled a meeting with the pioneers to address the issue.

“This is a big deal,” said Erik Johnson, vice president of the consulting firm Avalere Health. “It would be a blow to the administration if one of these programs goes down. The pioneers obviously know that and have some leverage” to change major aspects of the program, he said.

Johnson says that the timing of the letter was “odd,” but the pioneers do have a legitimate complaint that the quality metrics are not ideal. CMS has not been entirely transparent on how they picked the targets, he adds. The issue is particularly important because the targets will also be used to measure the outcomes of the more than 200 ACOs participating in the Medicare Shared Savings Program.

“You have to wonder whether CMS miscalculated,” said Michael Millenson, president of Health Quality Advisors LLC, much as it did with the initial ACO draft regulations that sent health systems scurrying. “The administration has to show that they can make this work in real life with the people who are most invested in trying to make this work,” he said.

The ACO programs are “unchartered waters,” he said, and are bound to hit some bumps along the way. After all, said Millenson, social and technological changes take time. “How often were your calls dropped in the first few years of cell phone service,” he asked. The problem here is that the administration is under enormous political pressure to show savings and quality quickly, “and as we know the health system does nothing quickly.”

Despite the hurdles, the administration remained hopeful about the program. “The Pioneer ACOs are our partners in the critical effort to devise new ways to deliver better care at lower costs,” said CMS spokeswoman Emma Sandoe in a written statement. “We value that partnership and are committed to an ongoing dialogue with them. We are confident that the Pioneer ACOs will continue to participate in a model that has the potential to generate real improvements in care for Medicare beneficiaries.”

The important thing is that both the pioneers and the administration are still at the table, said Anne-Marie Audet, vice president for health systems quality and efficiency at the Commonwealth Fund. In a program model as complex as the ACOs, “you have to assume that many issues will come up. What’s really important is how you resolve the problem.”

What the spat perhaps best reveals is that measuring quality is a difficult task, even for organizations like the pioneers who do it best.

“I think there’s gonna be a lot of closed door meetings and heated phone calls,” said Johnson of Avalere. “But eventually, everyone will come out holding hands.”