Finally Here
October 1, the Affordable Care Act (Obamacare) website http://www.healthcare.gov finally went live today.
I was eager to personally review what was being offered and cut through the hoopla and criticism. I had previously written, FMS Receives Health Insurance Premium Refund from the Affordable Care Act, so my expectations were high.
From the previously published rates for Virginia, the cost of insurance premiums for individuals and families was considerably lower than what FMS currently pays for our group plan. Business plans aren’t available yet, but the individual plans should be a good indicator. I wasn’t interested in the subsidies; I simply wanted to know the prices for the different plan options.
Applying for Coverage
So I went online to Healthcare.gov around 5:30 AM to apply for my family and see what it would cost. As expected, you create a login with email confirmation, and fill out a Wizard to select the options. It’s similar to many other instances I’ve applied online for credit cards and other forms of insurance. How tough could it be? Technically, it’s a very simple data entry application that should generate a quote at the end.
What a Mess!
Unfortunately, what should be a simple process is a complete software technology disaster. The logical flow of the application to register, login, and fill out the data for a family was horrendously inefficient. It seemed like the person who designed it, had never used it. Or maybe didn’t have a family which required filling out the same information for each member of the family.
Just the initial process of creating a login required multiple secret questions and other unnecessary data for getting a quote. Sure that may be necessary for the final acceptance, but it’s a complete waste of time and web resources initially. The system should expedite the process as much as possible to get people a quote without subsidies, then ask for more information to calculate the subsidies if desired. Since I later discovered it never generates a quote, it may not really matter anyway. What were the designers thinking?
Overly Complex Data Entry
As for my family, I not only had to identify my spouse, my two kids, their relationship to me, but also their relationship to my wife, and even their relationship to each other! What? Given the prior information, obvious defaults could be offered. The selection of race was also more complicated than it should be. Here’s an idea that may not have occurred to the designers: Maybe the kids should default to inherit their parents’ races. That’s how inheritance works. And does race impact pricing? If not, why ask?
The system crashed several times for me and had problems when I logged back in. It seemed like the system wasn’t even tested. Here are some screenshots:
Screenshot 1: Gibberish
(click the graphic to see an enlarged version)
What the hell is that? How could that get through testing much less production?
Screenshot 2: Error form with no data
Having error handling to catch unexpected crashes is a Best Practice in application development. It should tell the user what went wrong, what to do next, and gracefully exit the system. This page does none of that. The error message and error number are blank. Who knows what went wrong? Useless and amateurish. They do have a Live Chat button. I wonder what I would chat with them about with this crash.
Screenshot 3: Cascading errors
In this screenshot a series of errors appear to be triggered without meaningful explanation. Embarrassing.
Logging Back in and Repeating
If anything, I’m persistent. I not only had my original goal to see the premium prices, I was now intrigued to discover how poorly designed, developed and tested this application was. Eventually, I was able to finish. Took about an hour.
However, rather than receiving a quote immediately, it’s now being “processed”. For what? It shouldn’t be held up for pre-existing conditions which ACA eliminates. I would expect it to be some mathematical, logical formula that would generate the results. I presume it’s because that part of the application isn’t built yet. Although my application is submitted, given the crashes, I’m not sure what data it has. We’ll see.
Authors of Healthcare.gov
A few months ago, I read this article about how the site was being built and was impressed: Healthcare.gov: Code Developed by the People and for the People, Released Back to the People
In hindsight, it appears the authors have a philosophical bias toward OpenSource and “people power.” That’s all fine and dandy if it works, but this site doesn’t. To deliver such low quality results requires multiple process breakdowns. It just proves you can create bad solutions independent of the choice of technology.
Technical Software Conclusions
What should clearly be an enterprise quality, highly scalable software application, felt like it wouldn’t pass a basic code review. It appears the people who built the site don’t know what they’re doing, never used it, and didn’t test it.
I actually experienced many more problems than the screenshots I captured. Had I known I was performing a Quality Assurance assignment, I would have kept better documentation of typos, unclear directions, bad grammar, poorly designed screens, and other crashes. My bad!
It makes me wonder if this is the first paid application created by these developers. How much did the contractor receive for creating this awful solution? Was it awarded to the lowest price bidder? As a taxpayer, I hope we didn’t pay a premium for this because it needs to be rebuilt. And fixing, testing, and redeploying a live application like this is non-trivial. The managers who approved this system before it went live should be held accountable, along with the people who selected them.
Our Professional Solutions Group has created many mission-critical, custom software applications where scalability, reliability and quality are paramount. For instance, we built the Logistics Support System for International Humanitarian Relief for the United Nations where lives are dependent on accurate, timely data on a global scale.
We’ve also created a database link analysis program for the intelligence and law enforcement communities.
I know what’s involved in creating great software, and this ain’t it. Healthcare.gov is simply an insurance quote system. As a software developer, I’m embarrassed for my profession. If FMS ever delivered such crap, I’d be personally inconsolable. This couldn’t pass an introductory computer science class.
Overall Conclusions
This is going to be a huge public relations mess that could doom the whole initiative. Maybe they can blame the problems on too many users even if that weren’t the real cause, but it’s not going to be fixed with a few weekend tweaks and throwing more hardware at this. The application process asks too many unnecessary questions and repeatedly crashes. Since 9 AM and as of this evening, the site no longer lets you apply. I presume it got overloaded or someone finally discovered how broken it is and pulled the plug. Given what I experienced, it needs to be offline until it’s corrected. Meanwhile, I’d be highly concerned about the security of the data people enter given all the crashes I encountered.
Of course, software problems with the application process are not the reason to abandon healthcare reform. As a small business owner, we face the highest premiums for the lowest coverage. I applaud the efforts to reform health insurance and look forward to working in a constructive, rather than destructive, manner to improve this. I presume once these issues are resolved, I’ll have more options for my company and employees than I did before. In the big picture, this website is much easier to fix than health insurance. We’ll see.
Because I don’t like to complain without providing solutions, I added this blog post on October 14th:
Creating a Healthcare.gov Web Site that Works
New Attempt on October 5
I thought I’d give it a new try on October 5th to see if my initial impressions were wrong. I decided to create a new account to start all over. I had forgotten how difficult it was to simply create a login.
First it requires an overly complex login name that: “must contain a lowercase or capital letter, a number, or one of these symbols _.@/-“, which is confusing based on how you apply the OR clause. A clearer and simpler instruction would be to say: “must contain letters AND at least one number”. The unclear instructions cannot be blamed on too many users.
Then, it required three secret questions just to create the login. Not sure why any would be necessary. On Day 1, I got an email to confirm my login. Today, I got this screen after the last screen: Your account could not be created at this time. The system is unavailable
It lost all the data I entered. This is a clear web interface and database design problem if they didn’t store any of my information from multiple screens prior to the Finish button. A basic rule in database applications is to never lose data. Maybe you can lose data if the current screen crashes, but there’s no excuse for losing data from previous screens….unless of course, you haven’t thought about it before.
Another Attempt on October 7
I must be a glutton for punishment. I went back in to create my login. Unlike two days ago, I didn’t get the system unavailable message. It told me it was sending an email to confirm my login information, and I actually received the email. Woohoo!
Unfortunately, because I’m trying to run a business, I didn’t respond to the email until 30 minutes later. When I clicked on the link, I got this Oops. You didn’t check your email in time message:
Are you kidding me? It would have been nice to mention that in advance. Did they really invest programming resources to design and implement a feature that requires a response to the email confirmation right away? How user hostile can we get? Most sites would offer at least 24 hours or FOREVER to respond since nothing secure has been entered yet. I need to re-enter everything again. This just adds to their user load.
Application Status
FYI, my submission on the Healthcare.gov site on October 1st remains IN PROGRESS. No price quote, no email. Nothing. Just the same status online:
October 15th: I visited the site again and saw the same screen. I didn’t realize it, but if you click on the text to the left of the status, it’s a hyperlink that brings you to another page:
Imagine my shock to read “Your application is incomplete”. Darn! Could I have missed something on the first day that required me to re-enter or add more information? I don’t consider “Incomplete” the same as “In Progress”. It should have emailed me if it needed more information and certainly shown it on that screen.
I clicked on the “Continue Application” button to see what was missing. It turns out the same irrelevant, time consuming questions were asked for myself and each family member. I also found a few more bugs with problems going backwards, and selecting the address of each family member from a growing list of identical addresses. I also encountered new questions asking whether I am a Native American, and some strange question about my children’s relationship to each other (asked for each child) that I still don’t understand:
Can I get a sponsor for my child? I didn’t bother to investigate the definition of each of those terms before selecting “None of the above”, but it was bizarre and confusing. I definitely don’t remember answering this before. Eventually, I got to the end and was able to submit the completed application with a digital signature page I didn’t recall before. So maybe my application is going to be processed now. I went back to the status form and saw that I was “In Progress” again.
Then I clicked on the hyperlink next to the left of the Status and discovered the same “Your application is incomplete” screen. Did anything change? I think the message should say “Our application is incomplete”. Ugh!
Update for October 28th
My application is still “In Progress” but also incomplete. I went through the process again. Some bugs appear to be fixed since the last visit (duplicate lists of addresses are gone along with misnumbering family members). Still had many unnecessary screens that were shown but could not be edited (for instance, relationship between family members).
The administration has announced that the system will be fixed by the end of November. My belief has been that if the right team were in place and they could control the end-to-end process and design, this system could be built in a month. I hope they get it right and their families understand why they won’t see them over the Thanksgiving weekend. Good luck!
You have no idea. I have seen at least 8 different crashes and errors including grammatically incorrect text that an elementary school student would find. Funny thing is my list is completely separate from yours. So the list is at least double what you have. As of the moment, the site is down completely!
John,
Thanks for sharing your experience. Unfortunately, I didn’t capture screenshots of all the problems I encountered because I didn’t realize I was assigned a Quality Assurance job. I wonder if they even had a test plan. To run into so many bugs so quickly indicates this is rotten at the core. They don’t know what they’re doing. But they do have nice graphics.
It’s October 4th at 11PM EST and I still can’t get into the “apply screen” to laugh at all the blunders. Having the technical and testing background necessary for ay good launch is crucial. Am I surprised????? Yes, I am! I never dreamed of a snag in this website. Too many chiefs in the kitchen and no none tto test the soup or clean up the mess afterwards. Way to go!
“Too many chiefs in the kitchen and no none tto test the soup or clean up the mess afterwards. Way to go!”
itsmy business,
Don’t worry about the mess being cleaned up. A friend of mine working for CGI said they have a dedicated team to fix software release issues. This team is seperate from the developers that create problems. This team is about to wrap up all the May 2011 software release issues for another CMS project, maybe they will be availble to work on this. Something isn’t right for CMS to continously accept this level of work and pay two to three times for everything CGI does. I question not only the competency of CGI but CMS management.
The grammatically incorrect text is the first clue that this site was created using H1-B workers, possibly offshore. If so it should be another scandal in that a mission critical and politically sensitive IT project was done using marginally skilled people, and needless to point out, not US citizens. I’m guessing that’s why they are reluctant to name the company (or am I missing that it has been identified?) Using visasquare.com you can verify how many H1-B people they sponsor. Contrary to assumptions many of these people only go through a 90 day Java bootcamp before being hired, their only asset being that they work cheaper than an experienced programmer.
I work in healthcare IT, am exempt but as a veteran am explicitly exempted from purchasing insurance under the ACA and am not going to go onto it to further clog the works. I think it is the best idea possible under the political circumstance.s
It’s our understanding that it’s a Canadian contractor. Regardless, the people in the government that approve contractor deliverables should have tested and validated text. I think we can all agree grammar is not caused by too many users on the system.
Luke,
You are correct CGI Federal’s parent company CGI is a Canadian company. For projects where they are the prime contractor they also utilize subcontractors which adds another layer of quality problems. They have a lot of H1-B contractors that move from one contracting firm to another depending on who wins a goverment contract. So a lot of the goverment systems are developed by a core group of the same managers, BAs and developers. This is why so many goverment systems have delays and problems. CGI does what ever it takes to implement anything by a milestone due date to receive award fees. Once so much time and money has been spent on a project they know it is impossible to change horses in midstream. It will be interesting to see how much more tax payer money they will be awarded to fix this mess.
Come on. Writing software is hard and you know that very well. “As a software developer, I’m embarrassed for my profession. If FMS ever delivered such crap, I’d be personally inconsolable. This couldn’t pass an introductory computer science class.” I have nothing to do with the site, but I’m upset that you decided to speak about any developer’s work that way. We ALL make mistakes when writing software. Yes, even CRITICAL bugs that somehow manage to get through testing. To act otherwise is silly.
“Our Professional Solutions Group has created many mission-critical, custom software applications where scalability, reliability and quality are paramount.”
And I’m not sure there are NO usability and quality issues on your site? Like if you go to: http://www.fmsinc.com/Consulting/assessment.aspx and put in html characters, does the site blow up with an error because you still have RequestValidation turned on? I bet it does. As an end user, if I were typing a lot of text into that textbox and then it blew up, I’d be upset. But I wouldn’t start saying how I’m embarrassed for my profession.
Does the software have bugs? Yes. Should they have been caught during testing? Yes. But then, what bug *shouldn’t* have been caught during testing?
Just to address one of your nits:
“Maybe the kids should default to inherit their parents’ races. That’s how inheritance works.” OK, but now what happens when one parent is one race and one parent is another race? At that point you don’t default to anything, correct? That’s not a big deal, but it’s more code that you need to add: if (father.race == mother.race). I agree, it’s not a lot, but still. A bigger issue is, people are extremely sensitive about their children. For adoptive children, if you default their race to be the same as their parents, and they *aren’t* the same race as their parents, I can see certain parents getting upset. I’m not saying they *should* get upset, but people do. “Why does the federal government seem to think that all families should look the same!!!” etc. So, weighing those factors, you need to ask, “OK. well, this is a website that someone is intended to use *once* in their life. The average family has 2 kids. Is it worth the trouble. Or should we just not default to anything.”
Anyway, cheers!
Thank you for your feedback. I’m certainly not saying that software development is easy. Far from it, I think it requires the ability to properly design, develop, and test the system. I don’t think any of those things were done properly for the Healthcare.gov site based on the errors I and others have experienced which have nothing to do with too many users on the system.
A well designed system will get the basic information it needs to generate a result. In this case, it gathered lots of irrelevant information, made it difficult to provide that information, didn’t streamline the most basic inquiries, and didn’t produce a result. How much farther off target could they be?
That’s before all the crashes that are not volume related. Of course, all bugs should be caught during testing, but I wonder if they even had a test plan because some of the problems I encountered should have been caught long ago. At the very least they need an error handler that anticipates unexpected errors and gracefully exits. How can that be missing on a mission critcal application?
I think the work they’ll need to do to fix this is far more than anyone has publicly stated so far. These are not minor bug fixes. There are serious design problems that need to be addressed and should have been recognized long ago.
Now if your original post had said *that*, I wouldn’t have taken any issue with you. The thing that bothered me about your original post was the tone, e.g. “If FMS ever delivered such crap, I’d be personally inconsolable.” There’s absolutely nothing wrong with legitimate criticism, but if I had worked on the website and read your post, I’d be defensive and just blow you off because it’s written in a *very* condescending way. If the developers had asked your opinion of the site over a cup of coffee, would have you said everything the same way? I wager that you wouldn’t have. Just because you’re writing on the intertubes about people who you don’t happen to know doesn’t mean you need to treat them with any less respect than you would if you were speaking to them in person.
/CloseSermon 🙂
Hmmm…..I thought I was being objective and gentle in sharing my findings 😉
If this had really happened with my team, I’m not sure what language I would have used. Probably unprintable. However, I would have been most upset at myself for having a process that allowed such poor quality to slip through. Point taken though. Thanks.
I agree with Luke. Anyone who has been building and deploying web based software has their share of horror stories, but the errors on healthcare.gov are egregious. Newbie mistakes that should have been caught in the first round of testing on the happy path.
I presume their budget was decent – a good QA team would hit that damned site hard, do massive load testing, and look at the UX with a fine tooth comb. Maybe some real user testing with some real user groups?
And honestly, compared to some sites we have built for e-commerce or online data management, this site isn’t even particularly hard. 90% of the site is content.
Create a username/password, capture demographic data, run an algorithm. The core challenges are scale (they had to know 5M people would hit it the day it went live) and UX (because it has to be usable by all people – including those for whom English isn’t their first language, who may not have a lot of experience with the internet, or who don’t have strong reading abilities).
Honestly, the site is so bad, I am a bit concerned about the quality of their security and privacy protocols. After all, people are providing PII.
Luke, I tried to get information about small business exchange premiums on day one – and had an error report saying that “the data was marked as private”. I went back two days later, and was able to get an iFrame of ALL sample premiums available – despite the fact that I had already said I was in VA, I got a list of every single state.
The UX for filtering in the web form was impossible to use, so I ended up downloading it to excel, which worked pretty well.
I find this so frustrating because I support ACA and I want it to work. This is just embarrassing.
The difference between fmsinc.com and healthcare.gov is that HC.gov is one of the most important website launches of the year, with tens of millions of eyes turned on it and analyzing it from the get go.
They simply couldn’t afford to f**k it up. But they did anyway.
The other difference is that the bugs in HC are rampant, amateurish, and make the site unusable. If he has a RequestValidation error on this domain, that will affect maybe or or two people, total and less than 1% of site visitors.
The HC.gov problems are affecting millions of different people, and undermining confidence in the federal government to competently figure this problem out.
Thanks. It’s common Best Practice to have settings to minimize dangerous content, SQL injection, etc. I would hope the healthcare.gov site is protected from that too. That would be an advanced feature that’s way beyond the existing functionality that’s failing.
Luke, you have no idea how glad I was to read this. I think I finally found someone who shares my shock, disgust, and sheer amazement that this was allowed to go into production. This is the most important software release in the history of – well, ever. There are the lives of literally every American on the line. For this to go into production is like the Apollo rocket blowing up on the landing pad. I talk a little more about that at my blog, which I politely invite you to review at http://tinyurl.com/oyx5mpb
What this really is – and this is where I admire your apolitical, been-through-one-too-many-bad-code-reviews view of the system – is a scandal. And as a reflection of the IT profession, it is embarrassing.
I’m tracking how all this plays out; please keep in contact at my blog, and I at yours.
Because in the end, you need to get insurance for your family. And you can’t, at least not yet, and that sucks.
I didn’t even get to create an account before the system crashed. I do have a SW suggestion to the developers. Allow the user to create the secret questions. The choices presented were far from memorable for me. If I recorded the secret ? answers, I wouldn’t need the password reset either. I’d have recorded the P-word
Luke,
Well said! Nice to hear a direct approach to communication. There is so much politically correct rhetoric, listen to a politician, that nothing is ever said.
Yes, if you put out a product like that, you would not be in business.
I applaud you for a commentary that actually does tell it like it is, and I would hope that those responsible for this project might see this article and hopefully learn something.
Just added today’s experience. Unbelievable.
Here’s a nice article that goes into technical details about the Java problems that may be hampering the website: Yes Bad Javascript Will Shut Your Site Down from Rob Conery.
And another nice article about the dynamics between the separate development teams Err Engine Down: What really went wrong with healthcare.gov? from Slate.com by David Auerbach.
Ultimately it’s about the skills of the developers and management team.
Luke, it’s absolutely about the skills of the developers. Some of the JavaScript code is DailyWTF level awful – stuff that would get you failed in a 101 level class. Check this out:
https://www.healthcare.gov/marketplace/auth/global/en_US/js/ee/eeState.js
Just look at this awful function and the hilarious comment that precedes it.
//technically we should look at the length of the stateData and not use 50
function getStateCodeFromState(state)
{
var stateCode =””;
for(var x=0;x<50;x++)
{
if(stateData[x].stateName===state)
{
notFound = true;
stateCode = stateData[x].stateCode;
}
}
return stateCode;
}
Yes, you should use stateData.length – especially since the length of the stateData array is 51 (DC is included). Why does the function not return as soon as it finds the state it's looking for? And why is there a notFound boolean that is set to true as soon as the state IS found? The code is also riddled with incredibly embarrassing comments, like this one (found here https://www.healthcare.gov/marketplace/auth/global/en_US/js/ee/eeCommon.js):
//TODO: add some error checking, don't have time right now
There is NO excuse for this.
Sorry, I’m going to be devil’s advocate… I saw the piece on CBS and although I realize editors can trim things down to fit time allocated, it sounded very irresponsible… Lots of claims of poor coding, no testing, “rotten to the core,” etc. with no evidence to support that claim so I decided to check out your website to see what you consider top quality. I would strongly encourage you to revisit your home page and decide whether or not you think that’s a “good website.”
However, I found your blog and you support your claims with factual evidence and I appreciate that. Yes, some of those issues you identified are legitimate things that should have been found during testing and some are design choices that you don’t agree with. However, you cannot make the assertion that because you found “so many in such a short time” that the system is “rotten to the core.” Do you realistically think that your organization could have developed a bullet proof web application to provide health care to all of the country with the timeframe and hard deadline these people were given?
I’m not excusing them for poor programming, nor do I think the system was ready for prime time on Oct. 1 but when you have a hard deadline, you do the best you can to get it as ready as possible. I just felt that the strong language you chose to attack a major application deployment like this was irresponsible.
Thanks for you feedback. I’ll let my assessment stand on its own and reiterate this is a management problem. The people who designed the system were clueless and the higher ups somehow supervised and approved it. They could have done MUCH less work to accomplish what they needed to do, which is basically a Wizard data entry form. The design to collect all the information necessary to apply for coverage versus just collecting the information necessary to provide a quote quickly is seriously flawed. That’s not a programmer mistake. That’s management.
@sibaily
Sorry but Luke Chung has it right. His experience matches mine and his analysis and assumptions are quite on track. I’ve been building sites for the big boys like Pepsi Cola and have over 25 years experience designing and developing. I have yet to even be able to create a successful login on healcare,gov after many days of trying. I did not even begin to try until the second week just to give them a break. A truly scalable solution that is properly written will ramp up with more hardware. They have had almost two weeks now to add the hardware, and the site is no better. It is not a just a scalability issue – scalability issues cause lags and lack of service not lot’s of errors. I get error message trying to create accounts but later get emails to verify my email address for the accounts created. Those email verification links indicate that account does not exist or I waited too long (just minutes in some cases) and I cannot log on. Retrieving my password by providing my userid gets an email sent so they clearly have my userid and email linked. The email for the password reset gives an error indicating the account does not exist. I cannot try to get an account with the same userid as it indicates the account ID already exists. Which is it – clearly the account exists but will not allow me to logon or update my password. This site is pure s#!t. Transaction processing and error rollbacks seems to be non existent such that the DB is now so full of partial or abandoned accounts it must be a huge mess and full of junk data now after millions of users trying (and still trying) to just create accounts to get pricing info. Clearly the DB has pieces of accounts like the userid and email. This is real rubbish I as well would be embarrassed to have been a part of it. I could have done the DB and web code myself in a year with IT support on hardware and networking and it would have worked fine. I’ve done much more complicated things than this in less time. Luke Chung has it right, if I were you I would just be quiet and listen to him – you might learn something.
I was hoping to learn more about how Healthcare.gov was built, but the information session that was scheduled weeks ago for today was canceled: A New Way to Build Websites: Healthcare.gov
The link provides some information on the people involved with the development of the site. Did they really think they had some Best Practices to showcase?
I’m not interested in the site for myself, so I haven’t, and won’t, visit it.
That said, I have to agree wholeheartedly with Luke’s assessment, based on his experience.
Anyone who wants to excuse sloppy coding, inadequate testing, and all the rest because, as one comment said, “We ALL make mistakes when writing software. Yes, even CRITICAL bugs that somehow manage to get through testing. ” are missing the point entirely. These flaws are NOT edge cases that no one anticipated and didn’t think to handle.
This is BASIC design stuff, BASIC coding errors, BASIC lack of design, BASIC lack of testing. BASIC stuff that anyone should be embarrassed by.
When that happens, the responsible development manager should be embarrassed and potentially liable for ensuing damages.
But there’s plenty of blame for the government side, too. I have to think that no one on the client side, i.e. the people in the government responsible for this system bothered to test anything here. It couldn’t be this bad and not have someone notice. That is, if anyone took the time to look.
The wonderful systems architecture and brilliant development tools are clearly explained at this webinar:
http://www.howto.gov/training/classes/new-way-to-build-websites
A New Way to Build Websites: Healthcare.gov
This will tell you all about the wonderful technology that makes the ACA website a marvel of the Computer Age
uhhhhhh …. unfortunately, there is also the comment:
In the event of a government shutdown this event will not take place.
However, do check out the webpage to see the bios of the parties involved, and what they were planning to discuss.
this is a public service of Kludge Komputer Korporation
Pasta Division — Spaghetti Code is our Specialty
Presenter: Dave Cole, MapBox & Development Seed
Brian Sivak, HHS
This event will focus on an innovative and powerful new model for building and maintaining entire websites without complex Content Management Systems, and without compromising the benefits. Better security, scalabity, (sic) and load times, coupled with lower costs and future-friendly content management get Web teams out of the business of maintaining systems and focused on creating better content.
Join HHS and GSA for an overview of how one of the most prominent sites, a relaunched Healthcare.gov, joins a series of projects hosted using the lightweight technologies of Jeykll, Prose, and GitHub. These technologies maintain entire websites easily and quickly while maintaining the benefits of static HTML. This site will serve as an example that other agencies can easily follow. No matter where an agency is at maintaining or overhauling its Web presence, this session will provide useful context and answers for the common needs that every government agency has.
So, how extensive is the repair ?
Is the code salvageable/repairable with the current architecture ?
If getting a User Name and Password beyond the capabilities of the system, what about validating plan eligibility and options.
What about the system to authorize medical procedures (will Luke be approved for a hysterectomy?)
How about Claim Adjudication ?
John McAfee On Obamacare: “This Is A Hacker’s Wet Dream”
Wonder if they’ll next attempt to rewrite it all under DOS 6.22 using Paradox or dBase.
Here are two interesting articles on Slate.com by David Auerbach with more details of the development process that created this train wreck:
I guess we shouldn’t feel so bad as national taxpayers. 😉 Apparently, Vermont is paying the same amount to the same contractor to build their tiny state’s healthcare web site. This is after the contractor missed their deadlines and were able to double the size of their contract. It’s described in this article a few days before the national web site went live:
Builder of state’s health care exchange misses key deadlines
Hi Luke – very astute blog article – your experience on healthcare,gove and your analysis and assumptions match my own – very nice job! Thanks for posting this blog.
byerh
Rather than just complaining, I’ve published a new article that offers suggestions for Creating a Healthcare.gov Web Site that Works. Hope that guides us in the right direction both functionally and technically.
Updated my blog post with my discovery today that my application was incomplete. Not sure if it was or not. Gave me another opportunity to see how badly designed the system is, caught a few bugs, and got more confused. I thought I completed everything but it appears, it’s still incomplete. Was it really incomplete earlier? Who knows?
It looks as if the application was intentionally made to fail in order to sabotage Obamacare. While the web app gets fixed, the Health Dept. should provide forms at Post Offices, Libraries, etc., in the same way IRS Tax forms are distributed. Remember that not every one has a computer, or knows how use one.
To assume this failure was due to sabotage by the opponents of ACA give the opponents too much credit. I’m afraid all the evidence points to a self-inflicted wound.
Just a theory because the app is so obviously disfunctional, poorly designed and untested.
You’ll just have to accept that the administration really thought they were ready to go. I don’t think they would dispute that. For a week, they thought the only problem was too many users.
Mine is the same way “in progress”. When I click “view my application” it says “incomplete”. When I click on “continue application” It takes me back to the beginning and I have to start all over. However, if I click on the “eligibility” button on the left, it says “your detailed eligibility results are ready”. But when I click on the “view eligibility results” button, the button doesn’t work. Nothing happens. The girl on live chat said they’re still determining my eligibility and to just wait. When I called the 800 number and gave them my ID# they said they could see my registration but had no record of my application. She offered to enroll me over the phone but I decided to wait on that.
HHS thinks it’s a military operation and has implemented a surge to fix HealthCare.gov. Why do I feel that strategy won’t work if they don’t redesign the site and put in another management team? Maybe they did, but they don’t say: Doing Better: Making Improvements to HealthCare.gov
From personal experience, adding more developers to a sinking software project rarely turns out well. I guess we’ll see how it works in this case in a few days.
Contractors See Weeks of Work on Health Site [Sharon LaFraniere, Ian Austen & Robert Pear, New York Times – 10/20/13] <<< People who wrote the article have no idea that "weeks of work" is a massive understatement for a software project which hasn't even been thru alpha-testing.
Some specialists working on the project said the online system required such extensive repairs that it might not operate smoothly until after the Dec. 15 deadline.
…the login problems, though vexing to consumers, may be the easiest to solve. One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.
“The account creation and registration problems are masking the problems that will happen later,” said one person involved in the repair effort.
One major problem slowing repairs, people close to the program say, is that the Centers for Medicare and Medicaid Services, the federal agency in charge of the exchange, is responsible for making sure that the separately designed databases and pieces of software from 55 contractors work together.
Insurers have found that the system provides them with incorrect information about some enrollees, repeatedly enrolls and cancels the enrollments of others, and simply loses the enrollments of still others.
…disarray has distinguished the project. In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.
CGI Federal, a unit of the CGI Group, based in Montreal, has the biggest contract and is responsible for the architecture of major parts of the system, but not for its integration. Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system.
The identity management system from Q.S.S.I., which also taps into government databases to retrieve users’ personal information, was a particular source of trouble when the exchange opened. Change orders show that on Oct. 4 — after millions of people had been trapped in technological loops trying merely to log in — the government asked CGI to help it devise a new identity management system to replace the one provided by Q.S.S.I. But specialists said that approach was abandoned as too risky. Ultimately it was decided to fix the current identity system.
http://www.nytimes.com/2013/10/21/us/insurance-site-seen-needing-weeks-to-fix.html
Obamacare Website Violates Licensing Agreement for Copyrighted Software: Company to pursue action against HHS for using copyrighted web script. [Jeryl Bier, Weekly Standard – 10/17/13]
DataTables was developed by a British company called SpryMedia which licenses the open-source software.
…At the Healthcare.gov website, however, the copyright and all references to the author and SpryMedia are deleted.
http://www.weeklystandard.com/blogs/obamacare-website-violates-licensing-agreement-copyrighted-software_763666.html
Apparently, they enrolled six people on HealthCare.gov on Day 1. That’s actually six more than I would have expected based on the website I experienced on that day. I wonder how those six got through the system? My application has been “In Progress” for a month and I still haven’t been given a screen to see what I can choose from and the unsubsidized prices.
Here’s today’s Washington Post article about the turmoil inside the administration and development team on those first days: Obamacare’s launch looked even worse from the inside
“”Maybe the kids should default to inherit their parents’ races. That’s how inheritance works.””
Not if the kids were adopted, given that they will have millions of families signing up, there will be a few families whom happen to have adopted kids of different races, or parents from different races. And yes, race does affect premiums, the reason is because people from Pakistani/Indian/Bangladeshi are much more likely to get type two diabetes than say white person from the UK, so insurance companies have added that as a risk factors, there some others as well, at least in the UK.
For race, I was just suggesting they could set the defaults, not permanently assign. My point being, if you aren’t going to set the values of one screen based on the input from earlier screens, then why have multiple screens? Far cheaper to build, easier to use, and less costly on load resources to ask multiple questions in one screen.
As for race and pricing, it’s illegal in the US to charge different prices based on race even if you can statistically support a cost difference. I think the pricing of the policies is based on a handful of factors like age, gender, family status, smoker status, etc. Not sure why that’s not public.
Luke, what is your take on this latest development?
http://www.weeklystandard.com/blogs/claim-obamacare-user-sent-other-peoples-eligibility-letters_765771.html
Wouldn’t surprise me. Based on the bugs I saw on the intiial screens, I would expect many more bugs and more serious bugs deeper in the application. It was designed, developed, and tested by people who didn’t know what they were doing. We now know they insisted on using Open Source technology rather than existing business software and commercial cloud providers to handle a simple data entry and database system. One would expect to find significant database, security, and communications problems beyond the user interface bugs.
Luke :
It was designed, developed, and tested by people who didn’t know what they were doing.
What leads you to believe that this system was ever designed or ever tested ?
This is more like a movie based on the Fredrick Brooks book The Mythical Man Month
I wouldn’t place the blame on open source technology. Thats how most sites are built. The question is what technology was used, and was it the right tool for the job. EG, Ruby, python, php, perl, java, etc. As a self employed web developer, I use open source software almost exclusively, but i’ve learned all open source solutions are not good (some are awful), and knowing what works and what doesn’t is key. In my opinion, this site was overcomplicated. The system simply collects data, attempts to verify through experian and the government (i’d be willing to bet experians system is not the issue), and is supposed to filter your information to show you a “marketplace”, or what i would call an commerce site. This is basic web application development, and its costing hundreds of millions of dollars?
Totally agree. Inexperienced developers can create crap independent of the technology, and there are plenty of good solutions with all sorts of technology. The profit motive may have skewed some of the selections such as having Experian get paid for everyone who signs up, choosing Marklogic as the backend database which limits the number of people who are qualified to build solutions with it, creating the hosting platform rather than using a cloud provider, etc. Decisions like these probably added tens of millions to the cost.
Meanwhile, administration officials seem to think this required the skills of Silicon Valley as if it were some space project. This is a bread and butter, database web application that most business oriented software consulting firms like FMS can handle. I’m sure there are hundreds within a 20 mile radius of DC that can do it. Unfortunately, the big IT government contractors dominate the process and tell the government decision makers what to do. And the government people think they’re on their side sharing the same goals and values. Not exactly…
Apparently, the builders of the site do not want to use commercial cloud provides such as Microsoft Azure and Amazon EC2. These platforms would easily scale to huge numbers of simultaneous users. Is it because the contractors could charge taxpayers more by building their own host?
Amazon, Microsoft Turned Down By Obama
I find it bizarre that the HHS CIO had nothing to do with the project.
HHS’ Chief Technology Officer Bryan Sivak replied to Amazon by email on 8 October: “I wish there was. I actually wish there was something I could do to help.
HHS’ Chief Information Officer Frank Baitman replied to Amazon on 7 October, “Thanks for the offer! Unfortunately, as you know, I haven’t been involved with Healthcare.gov. I’m still trying to figure out how I can help, and may very well reach out for assistance should the opportunity present itself.” This is an ongoing story, stay tuned..,
Yup. I wonder what could have been more important on his priority list. It’s absolutely embarassing to watch the President and Secretary Sebelius talk about web site design. Something they clearly don’t understand. The CTO and CIO are supposed to take care of things like this and/or making sure the contractors take care of it. Just sad.
What’s the latest? Have you tested the site again to see if, nearly 7 weeks out, it’s usable?
BTW…had to go a few laps around the track here to log in here to comment. The system wanted all my private info from FB.
Pingback: Designing a Data Entry System Properly; Overhauling the Healthcare.gov Web Site