October 1, the Affordable Care Act (Obamacare) website http://www.healthcare.gov finally went live today.
I was eager to personally review what was being offered and cut through the hoopla and criticism. I had previously written FMS Receives Health Insurance Premium Refund from the Affordable Care Act, so my expectations were high.
From the previously published rates for Virginia, the cost of insurance premiums for individuals and families was considerably lower than what FMS currently pays for our group plan. Business plans aren’t available yet, but the individual plans should be a good indicator. I wasn’t interested in the subsidies; I simply wanted to know the prices for the different plan options.
Applying for Coverage
So I went online to Healthcare.gov around 5:30 AM to apply for my family and see what it would cost. As expected, you create a login with email confirmation, and fill out a Wizard to select the options. It’s similar to many other instances I’ve applied online for credit cards and other forms of insurance. How tough could it be? Technically, it’s a very simple data entry application that should generate a quote at the end.
What a Mess!
Unfortunately, what should be a simple process is a complete software technology disaster. The logical flow of the application to register, login, and fill out the data for a family was horrendously inefficient. It seemed like the person who designed it, had never used it. Or maybe didn’t have a family which required filling out the same information for each member of the family.
Just the initial process of creating a login required multiple secret questions and other unnecessary data for getting a quote. Sure that may be necessary for the final acceptance, but it’s a complete waste of time and web resources initially. The system should expedite the process as much as possible to get people a quote without subsidies, then ask for more information to calculate the subsidies if desired. Since I later discovered it never generates a quote, it may not really matter anyway. What were the designers thinking?
Overly Complex Data Entry
As for my family, I not only had to identify my spouse, my two kids, their relationship to me, but also their relationship to my wife, and even their relationship to each other! What? Given the prior information, obvious defaults could be offered. The selection of race was also more complicated than it should be. Here’s an idea that may not have occurred to the designers: Maybe the kids should default to inherit their parents’ races. That’s how inheritance works. And does race impact pricing? If not, why ask?
The system crashed several times for me and had problems when I logged back in. It seemed like the system wasn’t even tested. Here are some screenshots:
Screenshot 1: Gibberish
(click the graphic to see an enlarged version)
What the hell is that? How could that get through testing much less production?
Screenshot 2: Error form with no data
Having error handling to catch unexpected crashes is a Best Practice in application development. It should tell the user what went wrong, what to do next, and gracefully exit the system. This page does none of that. The error message and error number are blank. Who knows what went wrong? Useless and amateurish. They do have a Live Chat button. I wonder what I would chat with them about with this crash.
Screenshot 3: Cascading errors
In this screenshot a series of errors appear to be triggered without meaningful explanation. Embarrassing.
Logging Back in and Repeating
If anything, I’m persistent. I not only had my original goal to see the premium prices, I was now intrigued to discover how poorly designed, developed and tested this application was. Eventually, I was able to finish. Took about an hour.
However, rather than receiving a quote immediately, it’s now being “processed”. For what? It shouldn’t be held up for pre-existing conditions which ACA eliminates. I would expect it to be some mathematical, logical formula that would generate the results. I presume it’s because that part of the application isn’t built yet. Although my application is submitted, given the crashes, I’m not sure what data it has. We’ll see.
Authors of Healthcare.gov
A few months ago, I read this article about how the site was being built and was impressed: Healthcare.gov: Code Developed by the People and for the People, Released Back to the People
In hindsight, it appears the authors have a philosophical bias toward OpenSource and “people power.” That’s all fine and dandy if it works, but this site doesn’t. To deliver such low quality results requires multiple process breakdowns. It just proves you can create bad solutions independent of the choice of technology.
Technical Software Conclusions
What should clearly be an enterprise quality, highly scalable software application, felt like it wouldn’t pass a basic code review. It appears the people who built the site don’t know what they’re doing, never used it, and didn’t test it.
I actually experienced many more problems than the screenshots I captured. Had I known I was performing a Quality Assurance assignment, I would have kept better documentation of typos, unclear directions, bad grammar, poorly designed screens, and other crashes. My bad!
It makes me wonder if this is the first paid application created by these developers. How much did the contractor receive for creating this awful solution? Was it awarded to the lowest price bidder? As a taxpayer, I hope we didn’t pay a premium for this because it needs to be rebuilt. And fixing, testing, and redeploying a live application like this is non-trivial. The managers who approved this system before it went live should be held accountable, along with the people who selected them.
Our Professional Solutions Group has created many mission-critical, custom software applications where scalability, reliability and quality are paramount. For instance, we built the Logistics Support System for International Humanitarian Relief for the United Nations where lives are dependent on accurate, timely data on a global scale.
We’ve also created a database link analysis program for the intelligence and law enforcement communities.
I know what’s involved in creating great software, and this ain’t it. Healthcare.gov is simply an insurance quote system. As a software developer, I’m embarrassed for my profession. If FMS ever delivered such crap, I’d be personally inconsolable. This couldn’t pass an introductory computer science class.
This is going to be a huge public relations mess that could doom the whole initiative. Maybe they can blame the problems on too many users even if that weren’t the real cause, but it’s not going to be fixed with a few weekend tweaks and throwing more hardware at this. The application process asks too many unnecessary questions and repeatedly crashes. Since 9 AM and as of this evening, the site no longer lets you apply. I presume it got overloaded or someone finally discovered how broken it is and pulled the plug. Given what I experienced, it needs to be offline until it’s corrected. Meanwhile, I’d be highly concerned about the security of the data people enter given all the crashes I encountered.
Of course, software problems with the application process are not the reason to abandon healthcare reform. As a small business owner, we face the highest premiums for the lowest coverage. I applaud the efforts to reform health insurance and look forward to working in a constructive, rather than destructive, manner to improve this. I presume once these issues are resolved, I’ll have more options for my company and employees than I did before. In the big picture, this website is much easier to fix than health insurance. We’ll see.
Because I don’t like to complain without providing solutions, I added this blog post on October 14th:
Creating a Healthcare.gov Web Site that Works
New Attempt on October 5
I thought I’d give it a new try on October 5th to see if my initial impressions were wrong. I decided to create a new account to start all over. I had forgotten how difficult it was to simply create a login.
First it requires an overly complex login name that: “must contain a lowercase or capital letter, a number, or one of these symbols _.@/-”, which is confusing based on how you apply the OR clause. A clearer and simpler instruction would be to say: “must contain letters AND at least one number”. The unclear instructions cannot be blamed on too many users.
Then, it required three secret questions just to create the login. Not sure why any would be necessary. On Day 1, I got an email to confirm my login. Today, I got this screen after the last screen: Your account could not be created at this time. The system is unavailable
It lost all the data I entered. This is a clear web interface and database design problem if they didn’t store any of my information from multiple screens prior to the Finish button. A basic rule in database applications is to never lose data. Maybe you can lose data if the current screen crashes, but there’s no excuse for losing data from previous screens….unless of course, you haven’t thought about it before.
Another Attempt on October 7
I must be a glutton for punishment. I went back in to create my login. Unlike two days ago, I didn’t get the system unavailable message. It told me it was sending an email to confirm my login information, and I actually received the email. Woohoo!
Unfortunately, because I’m trying to run a business, I didn’t respond to the email until 30 minutes later. When I clicked on the link, I got this Oops. You didn’t check your email in time message:
Are you kidding me? It would have been nice to mention that in advance. Did they really invest programming resources to design and implement a feature that requires a response to the email confirmation right away? How user hostile can we get? Most sites would offer at least 24 hours or FOREVER to respond since nothing secure has been entered yet. I need to re-enter everything again. This just adds to their user load.
FYI, my submission on the Healthcare.gov site on October 1st remains IN PROGRESS. No price quote, no email. Nothing. Just the same status online:
October 15th: I visited the site again and saw the same screen. I didn’t realize it, but if you click on the text to the left of the status, it’s a hyperlink that brings you to another page:
Imagine my shock to read “Your application is incomplete”. Darn! Could I have missed something on the first day that required me to re-enter or add more information? I don’t consider “Incomplete” the same as “In Progress”. It should have emailed me if it needed more information and certainly shown it on that screen.
I clicked on the “Continue Application” button to see what was missing. It turns out the same irrelevant, time consuming questions were asked for myself and each family member. I also found a few more bugs with problems going backwards, and selecting the address of each family member from a growing list of identical addresses. I also encountered new questions asking whether I am a Native American, and some strange question about my children’s relationship to each other (asked for each child) that I still don’t understand:
Can I get a sponsor for my child? I didn’t bother to investigate the definition of each of those terms before selecting “None of the above”, but it was bizarre and confusing. I definitely don’t remember answering this before. Eventually, I got to the end and was able to submit the completed application with a digital signature page I didn’t recall before. So maybe my application is going to be processed now. I went back to the status form and saw that I was “In Progress” again.
Then I clicked on the hyperlink next to the left of the Status and discovered the same “Your application is incomplete” screen. Did anything change? I think the message should say “Our application is incomplete”. Ugh!
Update for October 28th
My application is still “In Progress” but also incomplete. I went through the process again. Some bugs appear to be fixed since the last visit (duplicate lists of addresses are gone along with misnumbering family members). Still had many unnecessary screens that were shown but could not be edited (for instance, relationship between family members).
The administration has announced that the system will be fixed by the end of November. My belief has been that if the right team were in place and they could control the end-to-end process and design, this system could be built in a month. I hope they get it right and their families understand why they won’t see them over the Thanksgiving weekend. Good luck!