Over $7 billion of taxpayer money went into creating the health law’s insurance marketplaces, and about 8 million people signed up. But some experts say it’s actually still too early to declare these markets a success or failure. So, what can we say about what the public is getting for its money?

The marketplaces, including Healthcare.gov and 14 state-run exchanges, are long-term investments. They have to work in the short-term, but also be desirable places for buyers and sellers to get together for years going forward.

This flea market in Colorado Springs, Colo., and the Obamacare exchanges are similar. They are both marketplaces where sellers display wares and shoppers browse and buy (Photo courtesy Colorado Springs Flea Market).

In that sense, they’re a lot like other marketplaces, like this flea market in Colorado Springs, Colorado. Buyers and sellers flock here, because it’s an easy place for them to find each other and make deals. On a recent Saturday, Dillon Keller was looking to move some roller skates.

“My mother in law works for a roller rink,” he explains, “and these are the ones people left behind for over a year. So, she said, ‘get rid of them!’ “

Keller drove across town and is spending $20 for a little square of asphalt here to sell the roller skates. He could have just had a garage sale, “But it’s not as successful,” he says. “You don’t have near the amount of people. For every one person at a garage sale, you probably have at least 50 out here.”

The flea market does a lot to draw customers in. It has a big billboard next to its highway exit, a food court, and even hires a live band on weekends.

The exchanges are supposed to be desirable marketplaces for health insurance. It’s even the same principle behind the New York Stock Exchange, says Economist Douglas Holtz-Eakin.

“You could look at health insurance and see the same thing: Do people want to sell their policies on there? Or is it growing in magnitude? Those are sort of standard metrics of success for marketplaces, and we should apply them to [the exchanges],” he says.

Eight million people showed up to buy insurance on the exchanges in the first six months. But will it be a good enough deal to bring them back? Will more insurers jump in? Holtz-Eakin says we don’t know enough yet to conclude which state’s exchanges are successful.

“I think we know the outliers, we know the big failures in Oregon and Maryland, for example. We know what appear to be the great successes in places like Kentucky, which is quite cheap and have a big fraction of people signed up. In between, it’s pretty hard to make the case we know who’s better than someone else,” he says.

One way it’s tempting to figure out which states were successful is to divide the federal grants sent to the states, by the number of people the marketplaces signed up. That gives a a cost-per-customer.

But comparing states based on that is, “really not a meaningful calculation,” says Jonathan Gruber, an economist at MIT.

He says insurance exchanges are long-term investments. They should last for decades and every state that set up its own had similar up-front costs, like hiring technology companies to create their online shopping sites.

“A lot of the programming, the computer set up is the same regardless of how many people enroll. So, by that number, California is going to look better than Rhode Island,” Gruber says. “If you infer from that that California is better than Rhode Island, it’s sort of like inferring that we should never let small states do anything.”

It’s also nearly impossible to compare Healthcare.gov states with those that did their own exchanges, based on the financial information the White House is releasing so far. And some states got millions or tens of millions of dollars in outside help from private donors. Even celebrity endorsements.

There’s just not enough information right now for economists like Holtz-Eakin pass judgment on the Affordable Care Act’s marketplaces. He says they’re only part of the picture of the federal health law’s success anyway.

“I think the focus on the exchanges per se is a natural fallout from Healthcare.gov melting down at the beginning,” he says, “but it’s really not the right metric for success or failure.”

Economist Jonathan Gruber agrees.

“Success can really only be defined ultimately as a significant improvement in the well-being of the US public – uninsurance rates drop, seeing health improved, seeing bankruptcies drop.”

The public won’t know whether that’s what they got for the tax money they spent on exchanges until well after this November’s elections.

Next year, though, federal grant funding for state exchanges starts running out. The challenge for these new marketplaces now is to grow on their own. Most have still only enrolled a fraction of the uninsured people in their states.

This story is part of a reporting partnership between NPR and Kaiser Health News.