Diagnosing the ObamaCare Glitches: Who Farted and Is Pointing at the Dog?

We’re fast approaching the end of Week 2 of Obamacare, and like the dysfunction between the Congress and the President, there’s still no end in sight. Healthcare.gov underwent major code fixes over the weekend, but is still rejecting logins, failing to load menus and hindering millions of uninsured Americans in 36 states from accessing the system.  Cleaning up these issues for the 9 million-plus who’ve visited the exchanges thus far is going to cause several operational tsunamis for health plans in the weeks ahead.

The Obama Administration took down key portions of the website last weekend for some urgent fixes, which helped but didn’t solve the problem. The Wall Street Journal led the week showing how Healthcare.gov’s significant glitches stem from “design and software problems” which rendered it unable to handle a persistent and massive influx of traffic since October 1. What’s maddening about Healthcare.gov’s woes is that 14 states and Washington, D.C. operate their own portals and many of them, like CT and KY, have successfully handled enrollees, while others, like NY and MD, suffered glitches similar to the federal website.  So what went so wrong in the CMS health reform shop in the runup to October 1, when the Administration insisted it was good to go?

Several hundred pro-ObamaCare developers (including a couple of my own geeks) and veteran Federal tech contractors are picking apart the code on Reddit and have found that Healthcare.gov’s thorniest issues to come are most likely in the back-end code that handles the registration process, which no one can see.  Slate, in an excellent piece this week, noted that “The site’s front end (the actual Web pages and bits of script) doesn’t look too bad, but it is not coping well with whatever scaling issues the back end (account storage, database lookups, etc.) is having.” The collective conclusions were that there were several factors behind ObamaCare’s messy launch:

• A patchwork quilt of vendors who started late, and apparently rarely communicated or collaborated.
• Lame project management, as evidenced by clear lack of testing of the end-to-end process, with vendors responsible for pieces of the workflow only testing their portion.
• A badly broken Federal procurement system that rewards vendors for contracting experience, not successful delivery of results.
• Code which buckled under predictable heavy traffic.  If you have spelling errors in the consumer-facing pages (e.g., ‘required Feild’), you can only imagine what a mess the behind-the-curtain code must look like.

The Wall Street Journal continued its terrific reporting on the exchange troubles today. The initial breakdown last week, they concluded, was due to a bad decision to require visitors to Healthcare.gov to establish an account before they could go window shopping for plans.  An HHS spokeswoman said the agency wanted to ensure that users were aware of their eligibility for subsidies that could help pay for coverage, before they started seeing the prices of policies.  I get that: help consumers avoid sticker shock in their first encounter with ObamaCare.  But sticker shock might have beaten an error screen.  It’s estimated about 125,000 consumers have now successfully created an account on Healthcare.gov.

So who farted and is pointing at the dog?  Front-end architect Development Seed and back-end developer CGI Federal, a Canadian company and one of CMS’s biggest IT contractors who also screwed up the Vermont exchange website.  Then there’s Maryland-based Quality Software Services, a United Health Group subsidiary which built the data hub; Serco, the British firm tasked with eligibility determinations, whose contract wasn’t even signed by CMS until July; and last but not least, Oracle, whose identity manager was fingered as the software component responsible for the bottleneck, and who sent a fix team to Baltimore Wednesday.  Little if anything was heard from these companies all week.  CMS officials will decide today whether to tear out the registration system this weekend.

What’s clear is that we had at least four development teams, isolated from each other, building pieces of a system that had to fit together seamlessly, which didn’t because no one owned the end-to-end process.  There were isolated tests on their respective pieces of the process, but likely little or no full capacity testing in the runup to October 1.  One of the Redditors, who actually worked on Healthcare.gov, posted the following:

“Problems with setting up an account are the least of the website’s worries. The site has to have interoperable channels of communications with the department of health, treasury, social security, state agencies, employers, health insurance companies, and consumers; and present accurate information in a timely enough manner so that consumers have an optimal experience. Unfortunately, each of those stakeholders or entities have their own legacy systems, their own technical architecture that supports their own bylaws and policies, their own suits who manage their own IT contractors, blah blah blah. It’s like dominos. I can’t test this critical component of the project without having 2 other deliverables precede: (both of which are) completely out of my control, and is under the jurisdiction of another agency.”

Epic.  Fail.  And points to the bigger problems with ObamaCare functionality still ahead, like the next wave to hit: the subsidy eligibility maze.