We’ve completely revamped our Microsoft Access to SQL Server Upsizing Resource Center. Find links to all our related white papers and Microsoft resources to help you with the SQL Server upsizing process, from deciding why and which Microsoft Access databases to upsize, the different options, and using SQL Server Express.
We have several new and updated resources:
Microsoft SQL Server Express: Version Comparison Matrix and Free Downloads
For the first time, all the different versions of SQL Server Express from 2005 to 2014 are shown, compared, and referenced with download links. This content required extensive research to document the details of which operating system each SQL Server version supports. Just because Microsoft web pages list the versions they support doesn’t necessarily mean it works when you actually try to install it.
When and How to Upsize Microsoft Access Databases to SQL Server
The original version of this was written for Microsoft when they selected us for a joint national campaign a decade ago to promote Microsoft Access to SQL Server Upsizing. We’ve updated it to better explain why and why not people should upsize their Access databases and an overview of what the options are and what to do.
“Instead of being user-friendly, the original website was user-hostile”
Basics of Data Entry Systems
We at FMS have created countless database systems where data entry played an important role. Unlike fancy graphics filled systems that look nice, data entry systems must be designed with a focus on ease-of-use by the end-user to enter, review, and update their information. If there are many questions and complex relationships, users need to be able to see as much of that on one screen as possible. If multiple screens are required, being able to move back and forth between screens without losing data and having changes in one screen reflected on others is critical for an efficient and intuitive user experience.
Data Entry Systems Should Target Users with Large Screens
For complex tasks such as writing a paper or working on a large spreadsheet, computers remain the preferred platform for getting work done where people can have one or multiple large screens. Serious data entry applications should target that user.
Mobile Apps Have a Role, but Not for Serious Data Entry
While mobile applications have a place, it’s not appropriate for complicated data entry since one question per screen is very inefficient. Not being able to see previous entries and pressing Next and Back for each question drives users crazy. The original designers of the Healthcare.gov web site designed it as if it were a simple, consumer mobile app meant to be filled out with a few finger clicks. They were either paid by the screen or just clueless about what a business data entry system requires.
Initial Request for Information Should be Anonymous
The purpose of the public facing Healthcare.gov website should be focused on helping prospects with the buying process. People need to quickly browse the health insurance options that are available to them in their state and cost estimates. The initial data entry should be the minimal anonymous information necessary to produce those results such as gender, age, zip code, family size, etc. Nothing personal such as names, social security numbers, email address, etc.
Automating a Paper Form
Only after customers have made a decision to buy should they be required (and expect) to provide more detailed information. This application feature is the core of the public facing Healthcare.gov website and is simply the automation of a 12 page paper form. It shouldn’t be that difficult.
We at FMS have automated paper forms for decades. Recently, we did this for a series of paper documents at the National Archives. The cost of doing this was in the tens of thousands of dollars, not the hundred of millions that Healthcare.gov cost.
Separating Data Entry from Complex Validation
A high volume, data entry system like Healthcare.gov should be designed to collect the user’s information as quickly as possible without trying to validate everything with other government systems in real-time. The cross-validation of information against IRS, HHS, Homeland Security, and other databases should happen in a background process that can withstand slowdowns or down times of dependent systems. This separates the complexity and risk of linking multiple systems together, manages the load on the other systems, and lets the user get done quicker. If a problem is detected later, an email can be sent to the user to fix the mistake or invalidate their application. Regardless, none of that needs to happen while the user is entering their data. After all, it’s not as if they were going to get insurance immediately upon pressing Submit.
It remains shocking to me that it cost taxpayers hundreds of millions of dollars initially for the broken Healthcare.gov site, and hundreds of millions dollars afterwards to the same contractors to fix it. The procurement process and incentives are completely inverted for creating and delivering quality software. It’s outright theft, but no one seems to be held responsible for it, and lots of people profiting mightily from it.
Conclusion: Data Entry Systems Aren’t Difficult If You Know What You’re Doing
I’ve contended that we at FMS could have created the public facing Healthcare.gov site for $1 million. Some people scoff at that, but in our world and that of our customers, $1 million still goes a long ways. We created an international humanitarian relief logistics system for the United Nations for half that amount, and it supports full language localization as it’s deployed in 80+ countries. Healthcare.gov didn’t even support Spanish when it debuted, and that was one of its original requirements.
Creating a good data entry system is not rocket science. This is not something that needs to be done in Silicon Valley. What’s needed is a team who’ve done it before and know what they’re doing. Creating this type of solution requires a solid database foundation, understanding the user needs, creating an intuitive user experience, and building it so that it’s maintainable over time. It’s not something that can be created by people on their first paid programming job, but it’s not a rare skill. I’m proud that my development team at FMS have been with me for decades and continue to deliver systems that just work.
Total Access Analyzer is the most popular Microsoft Access add-in of all time. Analyzing all the objects and code in your database, Total Access Analyzer generates detailed documentation and detects 300+ ways to fix, improve, and speed up your Access applications.
We have released free updates of Total Access Analyzer 2007 and 2010 to existing customers:
Total Access Statistics is the most advanced data analysis program for Microsoft Access. It extends the power of Microsoft Access queries with a wide range of statistical calculations including percentiles, frequency distributions, correlations, regressions, rankings, running totals, financial cash flow analysis, data normalization, crosstabs with Chi-Square, t-Tests, ANOVA, non-parametrics, probabilities, and more.
Total Access Statistics is now available for Microsoft Access 2013. Total Access Statistics 2013 includes many enhancements since the prior release of Total Access Statistics 2010:
Support for the 32 and 64 bit versions of Access 2013 with separate add-ins for each
New redistributable runtime libraries to support Access 2013, 2010, 2007, and 2003
Support for Windows 8 (and all Windows versions supported by Access)
Improved performance when analyzing large data sets
For Percentiles, when assigning percentile values to a field in your table, you can specify calculations such as quartiles, quintiles, octiles, deciles, etc. rather than just percentile
Field format is set to Percent for percentage fields in the Frequency, Crosstab (when percentages are in columns), and Chi-Square details tables
When tables are generated from the add-in, the field column widths are resized to show the entire field name and data
This error occurs in a Microsoft Access database that seems to work fine on every other machine but one. The MS Access database actually loads and runs, so the code is compiled and functional. Then it dies on some very common code such as CurrentProject.Connection for ADO to open a table or query recordset:
The “Class Not Registered” is very confusing. It implies code that won’t compile or broken library references but that’s not the true cause. Is the Access database corrupt? No.
The Microsoft Access team has released videos of their presentations at the SharePoint Conference from Las Vegas, NV.
With Access 2013, Access web solutions are hosted in SharePoint and rather than using SharePoint lists as they did in Access 2010, they use a real SQL Server database hosted in SQL Azure. The database can also be linked from desktop copies of Access to create hybrid solutions that serve both the web and Windows.
The Microsoft Access program managers presented these four sessions:
Don’t Forget System Administration for Microsoft Access Database Solutions
A professional Microsoft Access database application needs ongoing system administration. It’s an area that many MS Access developers neglect and causes problems when things go wrong (database corruption, missing backups, disaster recovery, etc.):
Microsoft recommends periodic database compacts and repair to maintain optimal performance and avoid database corruption.
Enterprise Quality System Administration with Audit Logs and Email Alerts
Total Visual Agent has provided an Enterprise Quality solution for almost 20 years by giving organizations a reliable way to perform their critical tasks on a 24/7, 365 days a year basis. A detailed audit log documents each action that is performed, and sends emails if errors are encountered.
Schedule Events, Databases, and Actions to Perform
Total Visual Agent automates and schedules Microsoft Access tasks. It ensures repetitive tasks are completed reliably. Tasks such as database compact and repair, zipped backups, rolling backups (e.g. 7 copies for each day of the week), running macros, running Windows command lines, making copies of table data, collecting database statistics such as size and record counts, etc. Easily schedule tasks for the middle of the night and know they’ll be completed.
Events can be scheduled every X minutes, hourly, daily, weekly, monthly or just one time. You can specify the days of week and time periods that it runs to limit processing to off-hours. Select the databases and directories (including subdirectories) to manage with support for workgroup security and database passwords.
Includes a Windows Service for Secure Processing and Reliability
Once defined, the events and tasks can be run by our Monitor program that is a standard Windows program.
Alternatively, Total Visual Agent includes a Windows Service, so you can run your tasks without having anyone logged on the machine. The Windows service is a more secure, robust solution since it can automatically restart if the machine reboots.
New Features in the 2013 Version
A huge number of new features were added in this 2013 release from the previous 2007 version:
Support for Microsoft Access 2013 and 2010, plus 2007
Support for 64 bit Operating Systems
Simplified Startup and Easier Management of Multiple Microsoft Access Versions
Import Settings from Multiple Versions of Total Visual Agent
Test All Actions for an Event, Database, Directory or Task Group
Create Events that Run Every X Minutes
Create Events that are Limited to Periods Spanning Midnight
Process Directories with Managed Databases
Data Extract Tables are Keyed
Run Macros for Database Password Protected Databases
Pause for a Fractional Minute
Compressed Archive File Names Support Multiple Extensions
More Detailed Activity History Log with Deletions
More Detailed Database Statistics with Deletions
Add Your Comments to Events, Directories, and Actions
Technical Evaluation of the Relaunched Healthcare.gov Web Site
On December 1, a updated version of Healthcare.gov was deployed which offers considerable improvements over the original October 1 launch. The administration and contractors issued some press releases and the general public and press just accepted it without really understanding the technical issues. Here’s my technical assessment of the published statements.
My Assessment of Healthcare.gov, Version 1, on October 1
As I described in my original blog post about Healthcare.gov, the site on October 1 was a technical disaster. I received a lot of criticism with my original assessment from those who thought I had a political agenda against ACA or people who simply wished the site was functional independent of the facts.
My assessment on October 1 was eventually vindicated. It took a few weeks for the media, general public, and administration to recognize that the issues were far more problematic than the politically attractive excuse of having too many users.
Unfortunately, the contractors who delivered the flawed system on October 1 were rewarded with additional contracts and funds to fix the mess they created. Our federal government procurement process actually gives them more money from their failure than if they did a good job. It’s no wonder that large IT government contractors continue to deliver technically mediocre results. As long as they make sure their lawyers are more powerful than the government lawyers, they can deflect ALL blame so they can continue to use their “unblemished” past performance to go after new contracts. We will see if any of the contractors here are held accountable for this fiasco when they seek future business. This contractor behavior extends across all branches of government.
It would be amazing to interview the developers who actually worked on the original project, discover what their prior experience was, what they were being paid, and how much the taxpayers were billed for their “expertise”. The contractors are enforcing confidentiality rules to prevent those people from talking to the press in the “interest” of protecting taxpayers. I thnk it’s pretty clear which interests they’re trying to protect.
Enhancements in Version 2
When the administration recognized the technical disaster, they brought in Jeff Zients to lead the disaster relief team. It’s a small world. Mr. Zients and I actually worked at the same firm (SPA/Mercer) before I started FMS, though I left a few years before he joined. Through his leadership, he added some experienced people and reorganized the team while using the same contractors. They issued a Progress and Performance Report which summarized their work:
System Stability: Uptime consistently above 90%
Reduced Error Rates: per page system time outs or failures from 6+% to 0.75%
System Capacity: 50,000 simultaneous users, 20-30 minutes per user for 800K per day
Software Fixes: 400+ Bugs Eliminated
Real-time Monitoring: Dedicated team focused on site monitoring and instant incident response
Improved Response Times: from 8 seconds to under 1 second
There was also Improved Window Shopping for users.
To a layman, these results seem adequate. To anyone familiar with commercial software development, they are far below what we or any of our clients would consider acceptable. This is not what professional software developers should deliver, nor what taxpayers should accept.
Review of Relaunch Accomplishments
I’m quite surprised others haven’t provided a technical review of the December 1 relaunch:
System Metrics: 90% up time (One Nine Availability is Awful)
Why do people think 90% availability is acceptable? Even their data showing 95% is awful for a web site. That’s not equivalent to an A in class.
90% up time means it’s down 10% or 2.4 hours per day. 95% is still down an hour. Most web sites have hosting uptime based on the number of 9’s. For instance, 3 nines means 99.9% up time. There are 8760 hours per year (365 days x 24 hours per day). A 99.9% availability means it’s down 8 hours a year. 99.99% availability is less than one hour down per year. High volume commercial web sites strive for 5 nines or less than 10 minutes of down time per year.
I have never heard of any web site or client expecting or satisfied with one 9 availability.
Error Rates Below 1% is Still Pretty Bad
99% sounds good for a class exam, but it’s not good for software. How can a production web site have a 0.75% error rate? The rate seems to be based on the number of pages which is far worse than users. If it’s based on users, with the 50,000 capacity, that’s 375 errors. But when it’s based on pages, assuming each user goes through 50 pages, 18,750 of their 2.5 million pages fail. That means 37.5% of users crash (18,750 divided by 50,000).
Of more concern is the cause of the errors. Software either works or it doesn’t. It doesn’t randomly fail. Is the platform failing 0.75% of the time without knowing why? That would be disturbing and could indicate lots of different bugs. If the contractors don’t know what’s causing the crashes in their buggy code, that raises very serious security implications.
Or do they know if people perform certain tasks that the system will always crash, and they expect people to do that only 0.75% of the time? Still not good, but better.
Beyond crashing bugs, the site may run without crashing but fail to perform properly such as the problems submitting accurate data to the insurance companies. Those non-crashing failures aren’t even part of this error rate which is already too high for a production system.
Capacity of only 50,000?
This is a very strange metric. One usually measures website traffic based on number of page views or transactions. The number of users can be supported by adding more bandwidth and instances of the application on more servers. The capacity issues comes from what people are doing. If they are browsing static pages (not entering data), the number of simultaneous users should be much larger. Even if they are entering data, the capacity to save the data should be much higher than 50,000.
It’s not clear what is causing the 50,000 bottleneck. It shouldn’t be the front-end web application. That should be designed to efficiently save user inputs. The users aren’t entering a lot of information in the grand scheme of data entry systems.
A well designed application would separate the real-time user experience from the more capacity constrained data lookup requirements that may have bottlenecks caused by slow legacy systems at the IRS, HHS, INS, etc. This simply means that the user would enter their information quickly, the system would process it offline, and an email would notify them when the verifications were complete.
Capacity Limitations are Odd
The Healthcare.gov web site begs the use of a commercial cloud provider that can automatically support the fluctuating volumes of users. A web site needs to accommodate the largest number of users, not the average. The large volume spike is ahead of us on the deadline date December 23rd. Volumes would drop considerably after that. By using a commercial cloud provider like Microsoft Azure or Amazon EC2, there would be no need to buy hardware to accommodate huge spikes in users or unnecessary after peak times.
We suspect it’s more profitable for the contractors to buy the extra hardware and configure it poorly than to use commercial cloud providers who would provide a better service for lower costs and profits. The contractors may have also implemented features that for “security reasons”, prevent the use of a commercial cloud provider. It could have justified the creation of their own private system even though it probably decreased security given the crashes they’ve experienced.
Software Fixes and Test Plan
Fixing over 400 bugs is obviously a very good thing, but is that enough? How did so many bugs slip through a Test Plan? And what critical bugs remain that they decided not to fix?
What was the test plan before October 1?
Were the tests conducted and what bugs were known before October 1?
How did they decide to release Healthcare.gov with those bugs?
How many bugs were found after October 1 and how were they identified?
Is the current Test Plan adequate?
What bugs were allowed in the relaunched version?
How are known and new bugs being handled?
Software development never reaches perfection but a good test plan covers the expected extremes to ensure the features work, unexpected errors are gracefully trapped, the system is scalable to support the expected number of users, and the site is secure.
In our experience, buggy software inevitably creates and reveals more bugs as bugs are addressed. Known problems with transmitting data to the insurance companies were already acknowledged. This implies this final step of the process was poorly tested, probably because all the preceding steps were failing. This would indicate many unknown issues that still need to be found and fixed.
If the original developers didn’t know what they were doing, trying to fix their work could be a waste of time. An experienced development team may be able to create a better solution in less time than fixing shoddy design and code from unqualified personnel.
Hardware: Do they Have Development, Testing and Staging Platforms?
The only reason I can see for such low availability is the lack of proper development, testing and staging environments. When we create web sites, our software developers need their own hardware to create and test their work without disrupting the production system. Testers need a separate platform to do their work and report back to the developers about the problems they encounter. And a staging site is necessary to review what’s about to be deployed. When the decision is made to release the new version, a switch can be made to make the staging site the new production one. In a modern host, the switch can be done almost instantaneously. Maybe it’s down for a short period to verify the new site is working, but it’s not down for extensive testing because the testing and staging environments already handle that.
Based on the information before the October 1 debut, it was clear that the standard software environment of development, testing, staging and production did not exist. How the managers of the project could have neglected this fundamental part of software development is beyond me, especially for the amount of money spent to build this site.
Without the proper platforms, it indicates the people didn’t even consider how they’d enhance and maintain the system over time, and further supports my contention that the people who created and managed this website had never been paid to build commercial database web sites before.
It really is software malpractice to not have the proper development, testing, staging and production platforms in place. The contractors should be liable for such neglect and reimburse the taxpayers.
Why Wasn’t the Site Redesigned for Simplicity, Performance, Scalability and Security?
There were many opportunities to redesign the site to make it more consumer friendly, reduce the amount of development and testing resources, support more users, and improve security. I list these missed opportunities based on what I have seen:
The account creation page should be one screen not three. We create multiple pages if entries in one screen impact the following screens. For the Healthcare.gov site, that’s not the case. For instance, if on the first page, you enter an email address that already exists in the system, you’re not told it’s invalid until you finish the third page and are forced to restart. That just adds load on the system. There’s also no need to create a different user name. Why not just use the email address? Most web sites have a one page account creation page, but we understand how having more pages is more profitable to the contractor.
The See Plans feature is a huge improvement for shopping. However, when someone finds and wants to buy a plan without a subsidy, there isn’t a way to do so without creating an account in the system. The site should simply direct the customer to the insurance company since the government is not involved with providing a subsidy. In addition to improving the customer experience, that would reduce the load on the Healthcare.gov web site so they can serve more customers. Get them off the site as quickly as possible!
There’s no need to ask for information that isn’t directly tied to calculating the subsidy. The “nice to have” questions on race can be discarded to improve response time, reduce the time it takes users to fill out the application form, and increase the number of users the site can support. It also increases capacity.
Over the years, we’ve helped lots of organizations design their software solutions, select technologies, specify architectures, and deliver solutions that are reliable, scalable, secure and maintainable. So much of the Healthcare.gov site seems to remain quite fragile.
I don’t mean to slam the many people worked hard to salvage the awful work of the initial developers. I’m sure they didn’t get to spend much time with their families over Thanksgiving. The relaunched site is definitely much better than the original version. But it only looks good when compared to that technical disaster. Can anyone claim the new metrics are acceptable for an enterprise quality, nationwide public site as important as this?
How Could the Federal Government Spend So Much and Get So Little?
The government contractors in the Healthcare.gov project continue to make fortunes after delivering a technical disaster. Unfortunately, this is common among IT projects delivered by large government contractors. Each year the government spends billions for poorly designed or non-functional systems that never even get deployed.
When I wrote my original blog post about Healthcare.gov, I thought the web site was created by incompetent people. Now I believe that in addition to being incompetent or inexperienced, decisions were made to maximize contractor profits.
This is philosophically different from the way we think at FMS. We always want to deliver functional systems on-budget and on-time. We take pride in creating solutions that don’t require additional work to fix them. As a small firm, we’re held accountable for our deliverables. If we fail, we would never be invited back. For large government IT contractors, it’s a totally different world.
Over the years, many large IT government contractors have abused taxpayers so often that they forgot the public would actually use Healthcare.gov and judge their performance. Even now, with their legions of lawyers and media spin, they are deflecting the story to blame government officials, other contractors, etc. without taking any responsibility. I agree there were problems with other parties, but that’s in addition to their own behavior.
Charge Extra for What Should be Included
Government contractors are experts at adding change orders and generating more revenues for features that should be already included. For instance, we are now hearing about problems with data security for the Healthcare.gov site. Security should be implemented from the beginning. Anyone with any experience collecting personal information such as social security numbers and birthdays knows that. However, the government contractors who won these contracts based on “past performance” suddenly suggest others are to blame for not specifying it earlier. That’s like buying a car and discovering brakes were an add-on. No, it should be included without asking for it. Storing data requires doing it securely.
Too Big to Fire
Given the awful work delivered on October 1, there’s no chance that the same team can be trusted to deliver a functional system. They already showed the world what they considered shipping quality, yet they remain. Unfortunately, our existing procurement system keeps these large government contractors because they are simply Too Big to Fire.
No Accountability for Large Technology Contractors
A small government contractor that performed so badly would not be allowed back into these agencies. The large ones can deflect the blame and legally challenge any attempt to hold them accountable. They never issue refunds, and in fact, profit from their mistakes with awards of new contracts and change orders. They are effectively not held accountable for their awful past performance, so the disasters repeat themselves whether it’s at HHS, FBI, Air Force, IRS, etc. The federal government is littered with expensive projects that were never used or functional, but highly profitable for the contractors.
Our Government Contracting System Encourages This
What happened with Healthcare.gov is exactly what our system encourages contractors to do. Had the contractors finished on time and properly, they would have made less money than delivering a flawed system. The government has tried to privatize its services by using outside contractors. Unfortunately, these government contractors are specialists at getting government contracts and milking taxpayers more than their technical ability. They would never survive in the private sector.
Policy Makers Now Have Political Risk for Technology Decisions
This is the first time an administration has paid such huge political cost for mismanaging technology. Prior to this, Presidents understood they were responsible for the economy, jobs, wars, terrorism, crime, responding to natural disasters, etc. They never realized there was political risk with technology. President Reagan wasn’t blamed for the Space Shuttle exploding, but this administration has become responsible for this web site disaster.
Contractor Goals and Values Do Not Align with Policy Makers
Frankly, I don’t think the politicians have any better handle for designing rockets or web sites. They relied on contractors and these contractors misled them. The policy makers don’t realize the goals and values of the contractors differ from theirs. Watching our leaders say the self-serving things their contractors tell them is even more embarrassing. Those contractors are not your friends!
Lobbying and Post-Retirement Jobs Drive Business
Some government officials are swayed by promises of post-retirement jobs at the contractors they supervise. Look at all the former program managers, contracting officers, Congressmen, admirals and generals at these government contractors and their lobbyists to understand how business is done.
Contractors get on contracting vehicles like the IDIQ for Healthcare.gov (termed “Licenses to Hunt”), then send in their well connected people to get contracts directly or wire contracts by “helping” draft requests for proposals (RFPs) favoring their organization. Perfectly legal. Not taxpayer friendly.
Bipartisan Reform Required
This is a bipartisan issue because unless IT government contracting is reformed, this is going to bite future politicians/policymakers of both parties. They do not have the training or experience to manage these technology projects. Especially when the contractors are run by used car salesmen who say, “You should get the undercoating” and the government people are technically unqualified to say “No”.
Technology Accountability Office (TAO)
We need the creation of a Technology Accountability Office (TAO), similar to the GAO to help agencies properly manage and buy these solutions, or an agency that manages large IT projects so the Best Practices are dispersed across the agencies. Right now, politicians have no clue whether a project should cost $1 million or $100 million, and whether it can be done in 3 months or 3 years. It’s total chaos and taxpayers are paying the tab.
Here are a few opportunities where I’ve spoken about these issues.
November 30: Fox & Friends Live Interview with Clayton Morris
A one-on-one interview with Clayton Morris for four minutes discussing how large government contractors profit from delivering systems that don’t work: “If we follow the money, we’ll see the stink in the system…Too Big to Fire”
November 26: On the Record with Kimberly Guilfoyle
Greta van Susteren is on vacation, so I chatted with Kimberly who was in New York City while I was on Greta’s studio in Washington, DC.
“Over time, I’m beginning to see that these government contractors who took over this project have essentially made every decision that favors them as much as possible – to maximize the cost to taxpayers, to maximize their profits.”