Oct 01

Healthcare.gov is a Technological Disaster

Healthcare.gov

Related Blog Posts:
Who Thinks the Relaunched Healthcare.gov Performance Metrics are Acceptable?
Too Big to Fire: How Government Contractors on Healthcare.gov Maximize Profits
Creating a Healthcare.gov Web Site that Works
Media Coverage for Changing the National Discourse on Healthcare.gov
Testifying Before the House Committee on Homeland Security about Healthcare.gov
Designing a Data Entry System Properly; Overhauling the Healthcare.gov Web Site

Obamacare
Finally Here

October 1, the Affordable Care Act (Obamacare) website http://www.healthcare.gov finally went live today.

I was eager to personally review what was being offered and cut through the hoopla and criticism. I had previously written FMS Receives Health Insurance Premium Refund from the Affordable Care Act, so my expectations were high.

From the previously published rates for Virginia, the cost of insurance premiums for individuals and families was considerably lower than what FMS currently pays for our group plan. Business plans aren’t available yet, but the individual plans should be a good indicator. I wasn’t interested in the subsidies; I simply wanted to know the prices for the different plan options.

Applying for Coverage

So I went online to Healthcare.gov around 5:30 AM to apply for my family and see what it would cost. As expected, you create a login with email confirmation, and fill out a Wizard to select the options. It’s similar to many other instances I’ve applied online for credit cards and other forms of insurance. How tough could it be? Technically, it’s a very simple data entry application that should generate a quote at the end.

What a Mess!

Unfortunately, what should be a simple process is a complete software technology disaster. The logical flow of the application to register, login, and fill out the data for a family was horrendously inefficient. It seemed like the person who designed it, had never used it. Or maybe didn’t have a family which required filling out the same information for each member of the family.

Just the initial process of creating a login required multiple secret questions and other unnecessary data for getting a quote. Sure that may be necessary for the final acceptance, but it’s a complete waste of time and web resources initially. The system should expedite the process as much as possible to get people a quote without subsidies, then ask for more information to calculate the subsidies if desired. Since I later discovered it never generates a quote, it may not really matter anyway. What were the designers thinking?

Overly Complex Data Entry

As for my family, I not only had to identify my spouse, my two kids, their relationship to me, but also their relationship to my wife, and even their relationship to each other! What? Given the prior information, obvious defaults could be offered. The selection of race was also more complicated than it should be. Here’s an idea that may not have occurred to the designers: Maybe the kids should default to inherit their parents’ races. That’s how inheritance works. And does race impact pricing? If not, why ask?

The system crashed several times for me and had problems when I logged back in. It seemed like the system wasn’t even tested. Here are some screenshots:

Screenshot 1: Gibberish

healthcare

(click the graphic to see an enlarged version)

What the hell is that? How could that get through testing much less production?

Screenshot 2: Error form with no data

healthcare2

Having error handling to catch unexpected crashes is a Best Practice in application development. It should tell the user what went wrong, what to do next, and gracefully exit the system. This page does none of that. The error message and error number are blank. Who knows what went wrong? Useless and amateurish. They do have a Live Chat button. I wonder what I would chat with them about with this crash.

Screenshot 3: Cascading errors

healthcare3

In this screenshot a series of errors appear to be triggered without meaningful explanation. Embarrassing.

Logging Back in and Repeating

If anything, I’m persistent. I not only had my original goal to see the premium prices, I was now intrigued to discover how poorly designed, developed and tested this application was. Eventually, I was able to finish. Took about an hour.

However, rather than receiving a quote immediately, it’s now being “processed”. For what? It shouldn’t be held up for pre-existing conditions which ACA eliminates. I would expect it to be some mathematical, logical formula that would generate the results. I presume it’s because that part of the application isn’t built yet. Although my application is submitted, given the crashes, I’m not sure what data it has. We’ll see.

Authors of Healthcare.gov

A few months ago, I read this article about how the site was being built and was impressed: Healthcare.gov: Code Developed by the People and for the People, Released Back to the People

In hindsight, it appears the authors have a philosophical bias toward OpenSource and “people power.” That’s all fine and dandy if it works, but this site doesn’t. To deliver such low quality results requires multiple process breakdowns. It just proves you can create bad solutions independent of the choice of technology.

Technical Software Conclusions

What should clearly be an enterprise quality, highly scalable software application, felt like it wouldn’t pass a basic code review. It appears the people who built the site don’t know what they’re doing, never used it, and didn’t test it.

I actually experienced many more problems than the screenshots I captured. Had I known I was performing a Quality Assurance assignment, I would have kept better documentation of typos, unclear directions, bad grammar, poorly designed screens, and other crashes. My bad!

It makes me wonder if this is the first paid application created by these developers. How much did the contractor receive for creating this awful solution? Was it awarded to the lowest price bidder? As a taxpayer, I hope we didn’t pay a premium for this because it needs to be rebuilt. And fixing, testing, and redeploying a live application like this is non-trivial. The managers who approved this system before it went live should be held accountable, along with the people who selected them.

Custom Database Software Application Development

Our Professional Solutions Group has created many mission-critical, custom software applications where scalability, reliability and quality are paramount. For instance, we built the Logistics Support System for International Humanitarian Relief for the United Nations where lives are dependent on accurate, timely data on a global scale.

Link Analysis with Sentinel VisualizerWe’ve also created a database link analysis program for the intelligence and law enforcement communities.

I know what’s involved in creating great software, and this ain’t it. Healthcare.gov is simply an insurance quote system. As a software developer, I’m embarrassed for my profession. If FMS ever delivered such crap, I’d be personally inconsolable. This couldn’t pass an introductory computer science class.

Overall Conclusions

This is going to be a huge public relations mess that could doom the whole initiative. Maybe they can blame the problems on too many users even if that weren’t the real cause, but it’s not going to be fixed with a few weekend tweaks and throwing more hardware at this. The application process asks too many unnecessary questions and repeatedly crashes. Since 9 AM and as of this evening, the site no longer lets you apply. I presume it got overloaded or someone finally discovered how broken it is and pulled the plug. Given what I experienced, it needs to be offline until it’s corrected. Meanwhile, I’d be highly concerned about the security of the data people enter given all the crashes I encountered.

Of course, software problems with the application process are not the reason to abandon healthcare reform. As a small business owner, we face the highest premiums for the lowest coverage. I applaud the efforts to reform health insurance and look forward to working in a constructive, rather than destructive, manner to improve this. I presume once these issues are resolved, I’ll have more options for my company and employees than I did before. In the big picture, this website is much easier to fix than health insurance. We’ll see.

Because I don’t like to complain without providing solutions, I added this blog post on October 14th:
Creating a Healthcare.gov Web Site that Works


New Attempt on October 5

I thought I’d give it a new try on October 5th to see if my initial impressions were wrong. I decided to create a new account to start all over. I had forgotten how difficult it was to simply create a login.

First it requires an overly complex login name that: “must contain a lowercase or capital letter, a number, or one of these symbols _.@/-“, which is confusing based on how you apply the OR clause. A clearer and simpler instruction would be to say: “must contain letters AND at least one number”. The unclear instructions cannot be blamed on too many users.

Then, it required three secret questions just to create the login. Not sure why any would be necessary. On Day 1, I got an email to confirm my login. Today, I got this screen after the last screen: Your account could not be created at this time. The system is unavailable

healthcare5

It lost all the data I entered. This is a clear web interface and database design problem if they didn’t store any of my information from multiple screens prior to the Finish button. A basic rule in database applications is to never lose data. Maybe you can lose data if the current screen crashes, but there’s no excuse for losing data from previous screens….unless of course, you haven’t thought about it before.


Another Attempt on October 7

I must be a glutton for punishment. I went back in to create my login. Unlike two days ago, I didn’t get the system unavailable message. It told me it was sending an email to confirm my login information, and I actually received the email. Woohoo!

Unfortunately, because I’m trying to run a business, I didn’t respond to the email until 30 minutes later. When I clicked on the link, I got this Oops. You didn’t check your email in time message:

Oops You Didn't Check Your Email in Time

Are you kidding me? It would have been nice to mention that in advance. Did they really invest programming resources to design and implement a feature that requires a response to the email confirmation right away? How user hostile can we get? Most sites would offer at least 24 hours or FOREVER to respond since nothing secure has been entered yet. I need to re-enter everything again. This just adds to their user load.


Application Status

FYI, my submission on the Healthcare.gov site on October 1st remains IN PROGRESS. No price quote, no email. Nothing. Just the same status online:

healthcare-status-NoID

October 15th: I visited the site again and saw the same screen. I didn’t realize it, but if you click on the text to the left of the status, it’s a hyperlink that brings you to another page:

Healthcare Application incomplete

Imagine my shock to read “Your application is incomplete”. Darn! Could I have missed something on the first day that required me to re-enter or add more information? I don’t consider “Incomplete” the same as “In Progress”. It should have emailed me if it needed more information and certainly shown it on that screen.

I clicked on the “Continue Application” button to see what was missing. It turns out the same irrelevant, time consuming questions were asked for myself and each family member. I also found a few more bugs with problems going backwards, and selecting the address of each family member from a growing list of identical addresses. I also encountered new questions asking whether I am a Native American, and some strange question about my children’s relationship to each other (asked for each child) that I still don’t understand:

healthcare-sibling-relation

Can I get a sponsor for my child? I didn’t bother to investigate the definition of each of those terms before selecting “None of the above”, but it was bizarre and confusing. I definitely don’t remember answering this before. Eventually, I got to the end and was able to submit the completed application with a digital signature page I didn’t recall before. So maybe my application is going to be processed now. I went back to the status form and saw that I was “In Progress” again.

Then I clicked on the hyperlink next to the left of the Status and discovered the same “Your application is incomplete” screen. Did anything change? I think the message should say “Our application is incomplete”. Ugh!

Update for October 28th

My application is still “In Progress” but also incomplete. I went through the process again. Some bugs appear to be fixed since the last visit (duplicate lists of addresses are gone along with misnumbering family members). Still had many unnecessary screens that were shown but could not be edited (for instance, relationship between family members).

The administration has announced that the system will be fixed by the end of November. My belief has been that if the right team were in place and they could control the end-to-end process and design, this system could be built in a month. I hope they get it right and their families understand why they won’t see them over the Thanksgiving weekend. Good luck!


Jul 03

Inspection Software for the National Archives and Records Administration (NARA)

National Archives and Records Administration (NARA)With the upcoming 4th of July celebrations, we at FMS are proud to have worked with the National Archives and Records Administration (NARA) over the past year to help them better maintain and preserve the important documents of our nation. Here’s what we did in our new case study: Inspection Software for the National Archives and Records Administration (NARA)

About the National Archives

The National Archives and Records Administration (NARA) is the record keeper for the United States. Of all documents and materials created in the course of business by the United States Federal government, only 1%-3% are important enough for legal or historical reasons that they are kept by NARA forever.

Natonal Archives Building in Washington DC

Background

To ensure the quality of work performed by their Facilities Management service providers, the National Archives and Records Administration performs both random and targeted inspections of completed work orders.

Problem

Inspection findings were documented on paper, which ironically, wasn’t efficient for the NARA. Reports were manually created to generate the service results. This manual process was time consuming and prone to human error.

Solution

FMS was selected to create a professional, multiuser system to collect the inspection results electronically and generate a variety of management reports.Within two months, we deployed our solution which offers data entry screens to replicate a variety of existing forms and many new management reports. An intuitive user interface made it easy for users without requiring extensive training. More importantly, we established a solid database foundation to improve NARA processes both today and into the future.

Operational Impact

  • Stores inspection results into a shared database
  • Increases efficiency and accuracy of the collection and reporting process
  • Gathers information and performs statistical analysis in ways that were previously not available
  • Eliminates the need to maintain paper files
Apr 25

Boston Marathon Bombing and the Use of Social Network Analysis (SNA) Software

Sentinel VisualizerWhile we at FMS are best known for our Microsoft Access add-in products, we are also a leader in the Big Data analytics world with our Sentinel Visualizer product from our FMS Advanced Systems Group.

Sentinel Visualzer helps analysts mine their data to find hidden relationships among people, places and events. Built with Visual Studio .NET on a SQL Server database, Sentinel Visualizer provdes advanced data visualization through link analysis, geospatial mapping, timelines, social network analysis (SNA), advanced filtering and decluttering, and many other tools to maximize the value of data.

MIT Technology Review Article on Social Network Analysis and Boston Marathon BombingAs the rampage and manhunt in Boston, Cambridge, and Watertown transpired, MIT Technology Review published an article that mentioned our Sentinel Visualizer product. In the April 19, 2013 article, David Talbot describes the rise of technology to detect the activities of criminals over the Internet and connect seemingly unrelated people, places and things. For more information, visit Building a Picture of the Boston Marathon Bombing Suspects through Social Network Analysis

Sep 27

Leveraging Technology to Enhance Teaching for the 21st Century

For the last few years, I've had the honor to serve on the Fairfax County Public School Superintendent Dr. Jack Dale's Business and Community Advisory Council. It has given me an opportunity to learn about the challenges of leading one of the largest and best public school systems in the country with a budget of $2.2 billion and 180,000+ students.

Last week, we had our first meeting of the school year. Our existing education system remains tied (some say hobbled) to early 20th century techniques, yet students need to be prepared for the 21st century. It's not easy to create and apply new teaching techniques on real children. I was pleased to witness a presentation on FCPS taking a leadership role in trying and testing new teaching techniques incorporating new technology. FCPS is forming a partnership with the George Mason University School of Education to create a "laboratory" to test these ideas to see what techniques are effective at providing the services without increasing the budget. A PDF of the presentation is provided here: Patriot Innovation Academy. This is well beyond the discussion stage. The Academy is being created at Lake Braddock Middle School with an initial group of 200 students. The principal, teachers, and parents met over the summer to plan the changes which are expected next school year. A lottery will be created for admission which is expected to be open to everyone.

Here are my impressions. First, I think it’s great that FCPS is revamping education for the 21st century and adding technology to help. That's important to teach more effectively and the reality of future budget constraints. Second, I'm impressed that FCPS is willing to attempt such an entrepreneurial venture. Education is a very risk averse culture, and with good reason since the futures of children are at stake. It would be easy to continue to repeat what has worked in the past and change gradually. Instead, innovation is being embraced with a willingness to fail since not every new approach will be successful.

But applying technology effectively is not easy. We have had great advances with technology over the last few decades without students making similar advances. Yet, technology can be used to provide personalized learning with immediate feedback. We also need to teach 21st century skills and not use technology to teach 20th century skills better. Here are some future trends I think we need to consider and address:

1. Facts are Available Instantaneously, Everywhere
People can already look up information on Wikipedia from their smart phones; this has already changed the way people argue. Future technology will search information automatically on a device that is already monitoring the conversation or what you are reading (think smart glasses). The implication is that the memorization of facts will be much less valuable. Knowing where to get it, and how to validate it will be more important. That means teaching history must be focused on WHY and not what events occurred. Life is turning into an open book test, or more accurately open Internet access. This transformation is similar to the advent of the written language which eliminated the need for elders to orally pass information to others.

2. Science is Multi-Disciplinary
I applaud the effort to teach subjects in multi-disciplinary ways. This makes the content relevant. Science is often taught in a cold, isolated manner that is difficult to connect with the real world. We need to transform teaching science from word search (looking up specific facts in book) into active synthesis and idea generation. Hopefully, they'll also include computer science as part of science, technology, engineering and math (STEM) or at a minimum, allowing computer science to fulfill the language requirement, since this is relevant to all fields in the future.

3. Babel Towers are Crashing
This is where I feel the proposed academy is short-sighted. Teaching "world" languages is a great pre-21st century skill. English is THE world language today. It’s not like Thomas Jefferson needing to learn other languages because everything important was written in non-English languages. Being able to read other written languages is a challenge already solved for free by Google Translate, Microsoft Translate, and many other providers. No one can learn all the languages these online services offer instantaneously. Writing in other languages is also becoming a commodity. Speaking remains a challenge, but it is a matter of when, not if, this is solved. We can then be trained and speak in all sorts of languages one phrase at a time. There's no need to waste thousands of dollars and hours to train a child to perform worse than what a free device will provide for a dozen languages before they graduate from college.

On a related note, requiring all high school students to devote 3 years to learning another language is a huge waste. People say it's important to learn other cultures. I don't dispute that. If that's the goal, let's teach that rather than memorizing the narrow vocabulary and grammar of one language. If we really wanted to teach cultures through language, then make it a year each of Chinese, Hindi, and Spanish to cover most people in the world rather than being mediocre in one language and ignorant of most.

4. Non-STEM Subjects are Important
STEM subjects are important, but that's not all our society needs. Writing and public speaking are critical for conveying one's ideas and influencing change. Fields that let kids push their individual limits such as playing an instrument, art, and drama should also be available. Analyzing philosophy, ethics, and moral reasoning are critical life skills, highly analytical, and important. Middle schoolers will make dumb decisions. The question is whether they learn how to recognize those situations in advance, to minimize them in the future.

5. Online Teaching is Good and Getting Better
More and more high quality online teaching is available for more and more subjects. Much of this is free, and it's getting better and better each year. Being online, the content is available 24/7. This trend will not reverse. In fact it is accelerating. Over time, school districts and traditional teaching cannot be competitive with this online content. Whether it's the Khan Academy or edX and its Harvard and MIT content, students can watch and practice on a platform that's much more interactive and comfortable than classrooms.

Local schools can also leverage this. Numerically half the teachers (and students) are below average. There are good teachers and great teachers. With the dropping costs of video, storage, and transmission, schools should be recording their best teachers' instructions so they can be replayed later and shared. It is not fair for students who are not assigned a great teacher to lose out on the experience. It's not the same as being in the classroom, but kids in other classes and schools should be able to benefit. Other teachers can also learn from them.

For a teacher to provide the same content year after year, is a huge waste of time and resources. Why not do it just once or just have the best teacher do it once for everyone?

6. Teachers are Evolving into Coaches
Recording great instructors and replaying them scares some who think this will replace teachers. That will not happen. Teachers remain critically important, but their roles are evolving into coaches in a world where information is freely available. Online training will only provide a portion of the solution. Just like teleworking is not replacing offices and face-to-face interactions, online teaching will not replace classrooms. Teachers can help and motivate students in a way that impersonal online videos can't.

Teachers should supplement technologically provided instruction (facts) with hands-on focused refinement that can't be provided by a recording. Technology does not support social interactions and the skills necessary to present ideas and convince others. It is also very weak in supporting creativity. One could argue it actually prevents creativity.

No football team is considering eliminating the role of the coaches. Technology helps them take their instructions to a higher level. Over time, this trend may even help teachers earn more because they can be more productive by delivering more value and serving more students.

7. We Cannot Predict Future Careers; We Need to Teach the Tools to Achieve Success
Many of the fields today's middle school students will work in probably don’t even exist today. My whole career (PCs and later the Internet) didn't exist when I was in college, so it’s hubris to think we can predict what middle school kids today will face. What's important is a child's ability to set high expectations, a willingness to try new things, and understanding that failing is a key part of learning to be successful. No one gains self-esteem and confidence by being told they did a good job when they know they didn't. Self-esteem comes from working hard, overcoming obstacles, and achieving goals. Kids do this very well with video games. We need to transform academic instruction similarly. Rather than focusing on a particular mistake, it's the response to the failure that's most important and helping students learn from them. Teachers/coaches play a critical role in helping students achieve higher than they originally expected. That's what is critical to life and acquiring a resiliency that prepares youths to confidently face challenges their parents and teachers never
imagined.

Conclusions
Overall, I think the attempt to revamp educational delivery with technology is a move in the right direction. Creating a separate academy is the correct method since it shouldn't be squeezed into the existing system. Applying existing technology and anticipating future technology that will impact and improve teaching is very critical. I'm concerned that the STEM emphasis will prepare kids for today's tech jobs at Northrop Grumman, Microsoft and IBM. Nothing wrong with that. Unfortunately, the future jobs are being created at technology companies like Apple, Google and Facebook where non-STEM skills have played a prominent role.

Will those companies remain leaders in two decades? We want our students to be engaged and successful in fields that don't exist yet. We want them prepared to analyze and adapt so that when opportunities arise, they recognize them and are willing to try, fail, and succeed. There is lots of work and many issues to consider. Our country spends over a quarter million dollars to educate each student through high school. Coming up with innovative ways to gain a higher return on those taxpayer investments is critical to our country's future. I'm glad to see our county playing a leadership role and taking action. What we learn from the academy should be quickly shared across the county, state and nation.

Sep 20

FMS Receives Health Insurance Premium Refund from the Affordable Care Act

Without stepping into a political quagmire beyond this topic, I just wanted to share our experience at FMS related to the Affordable Care Act, better known as Obamacare.

stethoscopeBackground

I’m sure small businesses can do much more with that money than padding the pockets of the executives and shareholders of the big insurance companies. We still see healthcare costs rising, but it’s good to know the rise is tied to actual increases in services being provided rather than profit margins. We look forward to the insurance pool that Virginia is creating for local businesses. From my contacts in our Republican governor’s office, Virginia is not resisting this initiative like some other red states and is moving forward with ways to reduce our cost of doing business here. Practical decisions like this keep Virginia business friendly.

Overall Impact

For companies like FMS that already provide health insurance to its employees, the obligations created by the new legislation don’t impact us since we were already doing them. We also don’t qualify for the small business tax credits which are targeting firms with lower compensated employees. I really like the lifting of the lifetime cap because after all, that’s when you really need insurance and it wasn’t an option previously available to us. I dreaded the idea that one of our employees would have faced a terrible health problem with financial ruin even though they were insured. I also like the moves to reduce the number of healthy people who choose not to buy insurance, yet end up using healthcare services we subsidize. Overall, it’s looking like a win-win for FMS and our employees. It certainly doesn’t solve everything, but it’s a good first step from our experience.

Luke Chung, FMS President


Follow Up

For our experience using the Healthcare.gov website the first time it launched on October 1, 2013, check out: Healthcare.gov is a Technological Disaster