Slashdot Mirror
What Developers Can Learn From Healthcare.gov
An anonymous reader writes "Soured by his attempt to acquire a quote from healthcare.gov, James Turner compiled a short list of things developers can learn from the experience: 'The first highly visible component of the Affordable Health Care Act launched this week, in the form of the healthcare.gov site. Theoretically, it allows citizens, who live in any of the states that have chosen not to implement their own portal, to get quotes and sign up for coverage. I say theoretically because I've been trying to get a quote out of it since it launched on Tuesday, and I'm still trying. Every time I think I've gotten past the last glitch, a new one shows up further down the line. While it's easy to write it off as yet another example of how the government (under any administration) seems to be incapable of delivering large software projects, there are some specific lessons that developers can take away. 1) Load testing is your friend.'"
No accountability of the contractors, no accountability of those who were to oversee the contractors and no accountability of the people who were to oversee those overseeing the contractors.
and I was ønce bitten by a møøse nø realli!
A feeling of having made the same mistake before: Deja Foobar
Nothing shows up the sheer arbitrariness of a government shutdown than some sites like Healthcare.gov being up, and others being forced to shut down at extra expense when they could have just been left running (and the servers that are there just to tell you the site is shut down are still consuming power and bandwidth).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Let's have our great media investigate if this is poor planning...or good planning if once the initial load gets through then they didn't overspend on equipment they don't need.
Or if there is a secret effort by the people who want this to fail to hire botnets and hackers to DDOS it... I wouldn't put it past them.
Would be something to see a considerable amount of traffic going out from Newscorp ip addresses into the healthcare.gov servers.
nothing unusual, aside a few million malformed packets...
A feeling of having made the same mistake before: Deja Foobar
Canadian firm hired to build troubled Obamacare exchanges
A Canadian tech firm that has provided service to that country's single-payer health care system is behind the glitch-ridden United States national health care exchange site healthcare.gov.
CGI Federal is a subsidiary of Montreal-based CGI Group. With offices in Fairfax, Va., the subsidiary has been a darling of the Obama administration, which since 2009 has bestowed it with $1.4 billion in federal contracts, according to USAspending.gov.
The "CGI" in the parent company's name stands for "Conseillers en Gestion et Informatique" in French, which roughly translates to "Information Systems and Management Consultants." However, the firm offers another translation: "Consultants to Government and Industry."
The company is deeply embedded in Canada’s single-payer system. CGI has provided IT services to the Canadian Ministries of Health in Alberta, British Columbia, New Brunswick, Quebec and Saskatchewan, as well as to the national health provider, Health Canada, according to CGI's Canadian website.
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
The biggest takeaway though, is that the way that the federal government bids out software is fundamentally broken. There are clearly companies in the industry who understand exactly the kind of problems that healthcare.gov needed to address. Intuit’s online TurboTax is much more complicated than the sign-up process for healthcare, and it works under heavy load. Amazon and Google both handle crushing loads gracefully as well. Why can’t the government draw on this kind of expertise when designing a site as critical to the public as healthcare.gov, rather than farming it out to the lowest bidder?
Although it's not entirely right.....government contracts are more complicated than 'going to the lowest bidder.'
"First they came for the slanderers and i said nothing."
From the list, one of the items casually mentions that usernames require numbers. What? I've never heard of a requirement like that from any other consumer system, ever.. they may suggest it (like YourName024 when a prior user has already used YourName) but do not require it.
If they worry about uniqueness, just use email addresses as logins.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
If you can get it, get a government contract to implement some huge IT system; you can have cost overruns up the wazoo, miss your deadlines, and create unusable interfaces; there will likely be few consequences, the customers can't run away from you, and the pockets of the government are infinitely deep to cover whatever you want.
If you can't get in on such a boondoggle as a vendor, vote against any kind of politician who promises to solve problems with some huge, government-paid IT system; they rarely are cost-effective solutions or very usable.
"Load testing is your friend."
In this case, any testing at all would have been friends to both developers and customers.
That would be an even more stupid idea than Newscorp buying MySpace.
Project Managers can learn giving only minimal time for QA, at the very end of the project, with no time allotted for corrections is bad practice.
"Launch" suggests that it actually, you know, worked.
When a quarter million people hit a game company's servers and only half of them get to play, it's a disaster of unrivaled proportions.
When millions of people hit billions of dollars in government investment and a few thousand of them actually get the site to work at all, it's a "learning experience."
Never attribute to malice that which is adequately explained by stupidity.
I'd have a hard time believing that the servers have been this consistently overwhelmed with traffic. A more likely explanation is that a poorly designed system was patched together from components hastily built from a thousand different vendors. The web-app equivalent of a diesel engine held together with duct-tape and baling wire was then rolled out without any real testing.
The only time, "Good enough for government work," has ever escaped my lips was when I was confronted with a marginally functional mess of spaghetti code.
An internal system operation returned the error "The operation completed successfully.".
I thought the consensus from the last story about the shutdown was that the web sites were closed because a server that's turned off is less likely to get 0wn3d without anyone there to fix it.
If they worry about uniqueness, just use email addresses as logins.
That's exploitable when you leave your ISP, someone else claims your username at that ISP, and your old ISP-provided e-mail address now points to another person.
Yep, let's turn over even more money and power to the same government that gives us the NSA, TSA, Patriot Act, and who knows what else that hasn't been leaked.
Yeah, that governement is going to help us, right?
GTA V? Sim City? Final Fantasy? Battlefield?
Turns out millions of users who start using something on the same day often don't follow the expected and tested for behavior.
Anyone who launches a service like this should expect to spend the first week in triage mode, and the first month making adjustments. I'd like to say proper planning would mean that never occurs, but the only way to insure that would be to spend 10x what is really needed. People would hate the government even worse if they did that.
This is not news, yet. It will be news in a month if it is still fubared.
That would be an even more stupid idea than Newscorp buying MySpace.
Project Managers can learn giving only minimal time for QA, at the very end of the project, with no time allotted for corrections is bad practice.
"Are we meeting with some network engineers, tech writers and systems analysts?"
"No, we are meeting with a bunch of appointees who know next to nothing about the guts of the project.
"Great... we may as well watch cartoons."
A feeling of having made the same mistake before: Deja Foobar
The Washington State's exchange website, for which the state paid $54 million to Delloite LLC, hasn't been a rollicking success either. I'm trying to wrap my head around why it costs $54 million to set up a pretty straight-forward website (costs evidently do not include hardware, just people/time/software). I believe that cost was over half what the state received from the feds to set up the exchange. Details here (such as they are).
When your boss says you're going to launch on October 3 no matter what, you get whatever you've got.
I've occasionally (thankfully not often) had to turn out things I'm not proud of for customers who have no idea how to schedule and won't hear otherwise. Stuff like the front end/back end error handling is high up the chopping block.
I've got a personal gripe about folks who think that 'developer' is code for 'guy who's expected to do everything in the project'. Outside of small projects, that's not how it should work in a healthy software development lifecycle.
Developers architect and write code, and some of the topics covered in that short editorial are relevant; use of AJAX necessitates good error handling on the front end, and synchronization of client and server side validations. Sure, they may have a broad skillset besides and understand databases, and graphical design, and so on, but there's no guarantee they're the ones meant to provide those skills.
For example, QA encompasses an incredibly large set of skills, familiarity with a wide range of products, and to be fair, seems to attract folks with a different life philosophy than those who identify themselves as developers. To talk about load testing - which itself is not a simple unit test to be added to a build - as a developer's responsibility, and ignore the vast, separate set of specialized knowledge and experience required to pull it off is ignorance. To include UX and UI design, and say these too are in the developers purview is equally misguided. (in fact, most developers are really, really bad at UI/UX, for some reason)
Not that a developer couldn't do those things, or will automatically lack the knowledge or skills, but those are separate roles and separate disciplines.
So, tell a project manager that they should make sure the QA team does load testing, and tell the project manager that the UI/UX team needs to provide descriptive error messages when validation fails, and so on. Very little of this is important to someone who's currently wearing the 'developer' hat.
The devs are in a pretty interesting situation that you don't see too often.. They're tasked with developing an application that generally can anticipate a low load level, except for one (and only one) extreme peak load. Do you develop for the general case, or the (very important) exception? Remember that the difference between these two options would make a difference in the basic structure of the app. Do you use a traditional RDBMS (perfect for the low load case), or some sort of no-SQL system (possibly necessary for the peak load case)? Remember that you can't leverage any commercial cloud resources either -- these are government records, and there are laws saying they'll have to be housed on government computers.
Odd, in my state it worked fine...no, wait a minute, it's only Oct. 4th, who in their right mind with technical savvy or experience would access such a new product in the first week of it's availability?
I live in one of the most population dense states. My current health insurance is paid up through the end of the month. I won't be accessing the exchange for three weeks yet because everything in the article is obvious, but even if implemented within the time constraints to the best of their ability, will still probably have issues in the first few days.
Duh.
I didn't make it very deep into the web site. I was mainly interested in reviewing the rates for my county. What a surprise that there was a list with all the states's counties together! I was expecting to fill in my zip code possibly or enter the state and county to get a list of available policies. The resulting table was large enough to generate bandwidth problems. One stupid error in design could saturate their network! A good design would be easier on the users, the network and the servers. Now sometimes you have to trade server time and convenience for user time and convenience, but this was apparently not thought through. Surely someone in the government must realize that good design works better than bad design. If a web site is to be used by millions, it obviously needs a good design.
Ray Seyfarth, ray.seyfarth@gmail.com, http://rayseyfarth.blogspot.com
Did a little sleuthing and discovered they're using an F5 load balancer in front of it (at least my state exchange is). I'm rather shocked that they chose a classical client/server architecture and not say, a cloud architecture for this. This could have been written on Google's cloud or Amazon's or OpenStack even and probably done a much better job of handling this load.
I would surmise that HIPPA requirements may have made cloud architecture problematic.
I checked out California's exchange web site and it's running fast. No problem registering or logging in and it's well designed to let you see your options.
California is a state which is completely in control of the Democrats. We don't have the tea baggers who are trying to destroy government here. (I think there are a few but we ignore them and they are powerless.) Government (and most other things) work better here.
I don't read your sig. Why are you reading mine?
It is worth noting that a raspberry pie computer could handle the work load of all the requests for healthcare.gov with correct load balancing and queuing. For PR you would need to set some expectation such as estimated wait time to get into the system, however your customer base would at least know that the system is working and that they just need to wait their turn due to the high demand. It is incorrect for most systems to be architected to assume everyone who accesses your system gets helped right away. The only exception to that would be an emergency response system where peoples well being is at stake. Look at many classic support call lines for major companies. Though they often have certain shortcuts based on how much you pay them, they have queuing systems that means regardless of how many millions of dollars a second you are losing, they may not be able to help you for x period of time.
It is reasonable that healthcare.gov could not complete transactions with each person visiting immediately. It is not reasonable for a lack of a system advising users of either estimated wait time or at minimum notification that they have a place in line and that they will be helped at some point in the future with no further requirement of action on their part. Any developer or software/system architect creating a transactional system big or small would be wise to first code-in this mechanism. It will save them headaches, and maybe a weekend, at some point in the future.
There is or can be built a machine that can simulate any physical object. -Church-Turing principle
If a web site is rushed into place on October 1st but there's no reason to sign up until January 1st, wait several weeks before you try use it.
It's not slashdot. There's no advantage to getting FIRST POST!!!
Oregon paid millions to Oracle for their own solution. It was a disaster. It did not work for me as I kept getting errors. And Oregon actually opted for a simple solution where you could not actually sign up for a plan online. You only received information about available plans. 3 years and millions of dollars later, they could not make that work reliably. As a developer I am baffled.
"Why would we believe they could accomplish something on this scale?"
Because they are the only ones who actually have successfully created healthcare systems on that scale, specifically medicare, medicaid, and the VA system.
Never attribute to malice that which is adequately explained by stupidity.
I'd have a hard time believing that the servers have been this consistently overwhelmed with traffic. A more likely explanation is that a poorly designed system was patched together from components hastily built from a thousand different vendors. The web-app equivalent of a diesel engine held together with duct-tape and baling wire was then rolled out without any real testing.
The only time, "Good enough for government work," has ever escaped my lips was when I was confronted with a marginally functional mess of spaghetti code.
You needn't source from multiple vendors to get a system that falls apart under load - single vendor solutions are also susceptible to such problems.. Even if you specify load testing in the contract, that doesn't mean that their load test had any relation to actual real-world load. Of course, the hard part is predidcting what load to expect, especially with a system that has a potential audience of 100+ million people.
Not one god damned fucking thing.
Not true, I learned that the portal is as useful as a politician. Considering the failure to balance the budget, reining in of these arrogant bastards who declared war on the American people. Over time will one understand the uselessness of these politicians and their insurance industry written healthcare policies. These CONgressMEN are as bankrupt as the nation they supposedly lead.
The mind conceives, the body achieves, the spirit manifests.
Everyone goes on the assumption that scale is "just make it bigger". I'd like to add some of my own notes on why this launch was doomed from the start.
I used to work for an adult internet company who had massive traffic. We were serving millions of people daily before 2000. We would exceed 10M daily viewers about once a week. That fluctuated by rather consistent calendar influences, like the day of the week, part of the month, and part of the year. Sept 11, 2011 dropped 3/4 of our traffic for almost exactly 2 hours. So we knew how long huge news event would impact us.
To handle 10M customers without a hiccup, we had to consider a lot of things. We didn't do much dynamic content. That's a killer. There were some elements that had to be dynamic, such as the voting/polling systems, message forums, etc. Otherwise, we had to try to keep the pages (html and images) as light as possible.
The hardest abused system we had was user authentication and authorization. We only had a few million users that hit it, but there were thousands of hackers (and script kiddies) that wanted to try to get something for nothing. Come on, it was cheap porn, just pay for it. We could easily see over 10M auth requests per hour. In time, we fine tuned the system, and outright blocked abusive users at the firewall.
The advantage we had was, when I was first in control over the IT work, we'd only see about 1M/day, so we had the luxury of growing it out. We'd watch for the problematic parts, and fix them. What works on your test bed where 10,000 users try it, even if they try hard, it doesn't mean you can put it on 100 servers and expect it to work for 1M users.
healthcare.gov has some other severe disadvantages. From what I understand, they are hitting the SSA database. I don't know if that's an online query to the SSA, or if they're provided a static file to import periodically. I'd assume all kinds of government organizations have put their 2 cents in too. What are they checking identity against? Drivers licenses, SS cards, voter ID, green cards? That means they could be hitting 151+ more databases run by other organizations. Does DHS get the information? Is it fed back to them when a users accesses? Are the checked against law enforcement databases? Only those directly involved in the development will know. You can disregard anything in the privacy statements. You're not going to see a friendly note in the FAQ "If you're a wanted felon, information will be transmitted to the law enforcement organization looking for you." That kind of defeats the purpose.
Depending on load testing never replicates what real users will do. Real users do weird things, just because they can. No amount of planning and testing will give you everything. There is always a lot of reactive work to be done. Shit, everyone reads the FAQ 14 times before logging in? They 20% of the people go through the login screens, back out to the 2nd page, and try again?
I'm stuck on the same non-functional healthcare.gov site as everyone else is. I signed up. I never got an email confirmation or email address verification.
My girlfriend got the verification and signed up again. I was able to present my user:pass and it did seem to say it was valid, but stayed there until I was thrown the overloaded message. Later, it said my user:pass was invalid. Is it really invalid?
I tried to do the username and password recovery. Neither sent me anything, so I assumed my account wasn't made. When signing up again, it said my combination of email, username, and real name was not unique. Ok, so I'm at least partly there.
I signed up again with a different username. This time I received the email verification, and clicking it did say I was confirmed to be a user. I still can't get in. It says my user:pass is wrong. Is there som
Serious? Seriousness is well above my pay grade.
Why would you want to do this? If you had an income that fluctuated each year, would you not save in the good years so you could maintain a reasonable quality of lifestyle in the barren years? Or would you downsize your house and sell your car every other year as your income fluctuated.
Balancing the budget is not the challenge. The real challenge is finding a government that can save when the going is good, and convincing the US electorate of the need for a rainy-day fund, rather than giving it all back and more in tax breaks.
It's not a challenge at all. Texas does it. We're required by our state constitution to have a balanced budget, and we only let our legislature meet for 150 days every other year. The result: once they are in session, they're working to hammer out the new budget and fix the real problems, instead of constantly being in session feeling the need to legislate something, messing things up, and wrecking the economy.
It works so great that our economy in Texas attracts a constant stream of refuges fleeing the charred ruins of California's economy and its legislature that occasionally takes a two week break between sessions of wrecking the state.
This was a strange comparison. Amazon often loads very slow for me, and pages fail to load completely on a regular basis.
irs.gov used to be a good example of a fast site. It is not as fast as it used to be, but still about 3x faster than Amazon, probably due to a static design with few images.
Simon's Rock College
How about this one, hire an Indian firm to run a government level oracle database without actually testing it or including load-balancing and you're gonna have a bad time.
Blame your horrendous failure on user volume and then call it glitches and you're gonna have a bad time.
List of known issues in order of appearance:
01. security questions not loading.
02. security answers failing validation.
03. email validation tokens timing out instantly.
04. correct passwords failing
05. password reset emails not providing clickable link for reset
06. password reset link loads page which doesn't find the profile it just emailed to.
07. EIDM server crashing and throwing system down errors.
08. oracle server errors.
09. network gateway timeout errors.
10. oracle account manager loading towards public
All of this excluding the actual waiting pages for a website.
This is either gross incompetence or sabotage.
They're using their grammar skills there.
definitely not a DDOS. LOIC is going to cause the kinds of errors I've seen.
They're using their grammar skills there.
The Affordable Healthcare Act is not affected because it was fully funded. The budget Continuing Resolution is for things which are not already funded.
And? Sites that require no funds to keep open because they are just sitting there 24x7 are being closed down. Privately funded areas around national parks, that are fully funded and privately owned, are being told they must shut down also.
Perhaps YOU have not heard or read it, but the federal government is shutting down everything it can, even if it's already funded, or COSTS MONEY to make inaccessible.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It's not a challenge at all. Texas does it. We're required by our state constitution to have a balanced budget, and we only let our legislature meet for 150 days every other year. The result: once they are in session, they're working to hammer out the new budget and fix the real problems, instead of constantly being in session feeling the need to legislate something, messing things up, and wrecking the economy.
Yeah. They never feel the need to legislate something, right? Only work to fix the real problems? They'd never decide that they needed a bit of extra time to legislate something just because they felt the need, right?
I'll just leave this here for people who maybe aren't absolute morons:
http://en.wikipedia.org/wiki/Wendy_Davis_(politician)#2013_filibuster
Tuesday I did the signup process, filled in all the information 3 times. Then I figured out that I could just hit the "back" button to go back to the security questions page and hit submit again. Finally got registered about 9PM, then got the validation email and clicked on that several times until it was finally accepted at 10:30PM.
And I've been trying and failing to login ever since.
So why should I have to go through all that just to get prices and find out which doctors are in their plan? On Ebay, Amazon, or just about any ecommerce site I can get the product description and price straight from a Google search. I only have to go through the registration/login hassle if I actually want to buy something. If they would just provide the plan information with a simple static html page I could get the information I want, stop hammering on their servers, decide what to do, and come back next month if I decide I want to buy.
* Off-topic: If the program is even moderately successful, I suspect certain politicians will regret working so hard to ensure that Obama's name is forever attached to it.
Tonight I tried to see what the site had to offer. First it took me an hour+ and three tries with the "Live" chat to get real responses from a human. Then while verifying my account and trying to login I was given an error code and told to contact Experian by phone. Experian! Hilarious fail, except for wasting my time and being completely useless.
This is a hacked account, for which the owner can not be held responsible.
"Now it doesn't mean I can't go see my current doctor, but my $4,500 out-of-pocket, is going to turn into a minimum of $26,000 out-of-pocket to see the doctor that I've been seeing the last seven years," he said. http://www.abc15.com/dpp/news/region_northeast_valley/fountain_hills/fountain-hills-man-dropped-from-health-insurance-because-of-new-regulations
Exactly.
If you did not think load testing was a good idea before this....
well, you are pretty hopeless.
Never answer an anonymous letter. - Yogi Berra
Why are they even writing user authentication code and associated functions? Been done a million times. Should be able to get a plug in that's been running for years. Hell, they could have bought the code to Apple's site or a similar site and just modified it. Selling shit is selling shit. Computers with many options are no different than policies with many options.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
I just successfully logged in. to a blank page.
They're using their grammar skills there.
I'm suggesting that the funding of healthcare.gov is through a separate bill and is thus not affected by the lack of a continuing resolution for fiscal 2014.
All of which is offset by the huge number of bat-shit crazy, bible-thumping, teabagger Texans.
Thanks anyway, I'll stick with California.
Wendy Davis was protesting a law that represents yet another act by the right wing to advance their agenda regarding abortion with specious regulations that serve no legitimate purpose. Those lies to support it aren't fooling everybody.
Then again you started your own protest with an empty complaint about Wikipedia.
Oh well, try this on for size...electoral turnout in Texas is terrible.
The better approach would have been if they let people enroll online over the last few months. Traffic would have ramped up quickly, but it would have been less than 10M people on the first day. 10M is the number I read somewhere that they got on the first day of operation. Allowing a slower ramp-up, even if the authenticated area only showed some pretty stats on the number of carriers, doctors, registered users, would have been something for them to prepare themselves with. Now is too late to hope for a graceful start.
Good strategy, nice thinking.
"First they came for the slanderers and i said nothing."
Don't vote for tyrants.
Contribute to civilization: ari.aynrand.org/donate
...if I hadn't once lived in California and now live in a state with a functional state government. If you think Cali has anything but a horribly dysfunctional government with bottom of the barrel public schools, badly maintained roads, ridiculously high taxes (income, sales...) and unfair and arbitrary justice system, well, I think your standards are low.
Texas has the federal government to fall back on in case of, for example, natural disaster. The federal government doesn't have such a safety net; it must self-insure. On top of that, the federal government has to be prepared for contingencies such as war that do not really apply at the state level.
The period of time, one year, is arbitrary. Requiring a balanced yearly federal budget would be like requiring a balanced personal budget every two week pay period, even though my biggest expenses occur monthly.
What we really need is some way to balance the federal budget over a much longer period of time, a decade or two perhaps, spanning a full boom/bust cycle. This is, of course, much easier said than done.
A large scale website is not free to operate, you have to pay for the electricity to run the servers, and for the bandwidth used.
Just tried to get on again.... down again for maintenance.
Have been trying around the clock since 13 minutes after it went into effect.
I have called in 3 times and have inched along.
Right now I am at the impasse of it can't confirm my identity for the last couple days and have been unable to upload my document to confirm it.
Called and had my identity verified, but still my process says it can't verify my identity.
Or CBC/BBC for the Canadian and British take on american news. Within the US, maybe NBC, ABC, CBS?
That's like saying that all the hold-up victim had to do was give up all their money and they could have avoided being hit during the mugging.
Uh, yeah.
Wendy was a voice of reason during that debacle; everybody else just glared at Texas and shook their heads.
Texas: what a country.
Need Mercedes parts ?
You missed other issues that *also* appear:
Failure to communicate with users when errors occur.
Failure to communicate when the system is broken.
Failure to communicate critical errors to your own staff.
Failure to create a realistic timeframe for necessary repairs.
Seriously. Why, when the system won't work at all, should I have to spend my time trying time and time again to even create an account? When I talk to the help and they say "keep trying", I act like a bloody fool by trying again. When I talk to the help and they say "It will be up in two or three hours", I assume that they've been told information that is roughly accurate!
---jstlook ---For that is the way of Elves, for they say both yes AND no, and mean every word of it. --- J.R.R.T.
The military isn't required by your constitution. Rather it's permitted, with an explicit requirement that its funding should need continual reauthorisation.
The navy would seem to escape that restriction by virtue of being separately listed without it; presumably because a ship could be at sea for longer than that.
I predict the way you're using two digits to count the errors is going to turn into a scalability limit.
From my experience the way you write bad code is to follow the "accepted" practices. Modern software engineering, and I use engineering very reluctantly, does not teach good coding practices. A rule which is never taught, at least I've never seen taught, is how to test your code properly. You need to do things like load testing, refresh testing, sequence testing and pattern testing. There is a reason why any good development company will have a full testing department because you really do need that level of attention to it.
The other practice that is often overlooked is to hire programmers and not software engineers. A programmer is the guy / girl who can sit down and bang out a tasty script or program to do a task with out needing to pseudo code or flow chart it or use big bulky IDE's or do code reviews and etc, they literally program. Engineering software is like trying to engineer baking, you can put the all the theory you want behind it but in the end I just want an apple pie. You'll get better code 95% of the time if you don't hire the software engineer. I can't even tell you how many times I've had to ask the professional software engineer to rewrite his / her code because it sucked.
Finally don't go live until you know the code is mostly flawless, you'll never get all the bugs out but you can get most of them out. When a website or program is released and it's full of bugs / issues, it just shows that the people behind it, management included, should not be involved with programming. If we went back to the old days, the 90's, then we'd see better code across the board. The only thing software engineer has done is brake good coding and turned it into a joke.
No kidding. They have a billion dollar advertising budget to get people to sign up for Obamacare.
They had three years to set up the online exchanges, but apparently didn't really start working the IT angle until this Spring...
The load on the Obamacare servers, while large, is nothing compared to something like Amazon or even Travelocity.
Note that all of the games you cite, while having problems, managed to actually work for most of their users.
Final Fantasy XIV:ARR is a great example. While they had issues, the biggest problem is that they had about twice as many people sign up at the start as originally planned, and they still managed to have the game working for most folks. After restricting logins to keep the individual server loads down to a manageable level, they added server capacity and fixed a few bugs. A week after launch, the wait times dropped to a few minutes at most.
The Obamacare sites are, quite frankly, terribly programmed. They should have a clean interface which asks just the right questions and drops them down to the servers. Instead they went JavaScript-happy, and each time you load the site, you get dozens of little independent scripts loading in your browser (11 CSS and 62 JavaScript files per PAGE) - a (probable) big part of the reason for the disaster is the 62x overload in JS loads.
The system was originally supposed to hit the secure IRS and HHS databases, but they couldn't get that to work, so they dropped the IRS hookup - which means your reported income is on the honor system.
There's also the recurring rumor that they might not be cleaning their database inputs quite thoroughly enough. That alone means I'm not going anywhere near this mess for at least a couple of months... or much, much later.
. . . to Infosys (India). . . .
Have you tried calling the helpline:
1-800-F**CKYO (1-800-318-2596)
http://dailycaller.com/2013/10/03/need-health-care-coverage-just-dial-1-800-fuckyo-to-reach-obamacares-national-hotline/
There's also story doing the rounds that if you don't already have health-insurance, then they'll put you on a list, suspend your driving license and place federal-liens against your home.
http://www.motherjones.com/mojo/2013/10/obamacare-conspiracy-theory-lien-house-debunked
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
I signed up again with a different username. This time I received the email verification, and clicking it did say I was confirmed to be a user. I still can't get in. It says my user:pass is wrong. Is there something really wrong, or is it still totally broken? I don't know.
Exactly this. The site may be overloaded, but when it does come up there are software bugs. I apparently have created an account but it won't let me log in. How do I know I created an account? When I can't log in I use the lost password affordance with my username, receive the lost password email, but the link back to the site throws an error page. I could not release a broken project to a customer in this state.
I can't imagine any period longer than 2 years working, unless we increase Congressional terms to match. Otherwise Party X who has control now will blow through the budget like there's no tomorrow, and leave Party Y holding the bag (and quite possibly unable to spend any money) during whatever the last two years of the cycle are.
Benford's Corollary to Clarke's Law: "Any technology distinguishable from magic is insufficiently advanced."
So why should I have to go through all that just to get prices and find out which doctors are in their plan? On Ebay, Amazon, or just about any ecommerce site I can get the product description and price straight from a Google search. I only have to go through the registration/login hassle if I actually want to buy something. If they would just provide the plan information with a simple static html page I could get the information I want, stop hammering on their servers, decide what to do, and come back next month if I decide I want to buy.
This is exactly what I have been telling my friends. The heathcare.gov site is ridiculous. You should be able to select your state, and go immediately to a static page with a table the plans sorted by age, etc., with a phone number to call to sign up. No registration nonsense. I've heard there are about 160 plans from 3 companies in my state. Who knows when I will get to see them? I still can't log into the site, even though I made it through the registration process.
And it's not just that, it's also about handling both expected average load and initial max load. If you focus on just max load, you'll wind up capacity that just sits around. If you focus on just average capacity, you run into problems like this (which is assuming they can get roughly the right estimate in the first place.) So a good plan needs to have something where it can account for max load (perhaps offload image hosting to CDNs and non-privacy-related data processing to AWS or something?) but not have a bunch of extra, idling capacity when the initial furor dies down.
I used to work for an adult internet company who had massive traffic. We were serving millions of people daily before 2000.
Was it first served first come?
How about this one, hire an Indian firm to run a government level oracle database without actually testing it or including load-balancing and you're gonna have a bad time.
For the amount of money they spent on healthcare.gov they could have hired call centers in low cost southern states to handle all of the signups by phone. Unglamorous and low tech, but effective. What matters here is getting the job done, not how pretty and mobile friendly your awesome website is (except that it doesn't work). In fact, the failed launch of healthcare.gov is a perfect metaphor for the Obama administration itself, it looks good and has plenty of style but it has a hard time getting anything useful done.
> I predict the way you're using two digits to count the errors is going to turn into a scalability limit.
Not if the error sequence number follows the convention used in IBM RPG/400 1.1.4.4. "Sequence Numbering of the Listing after a Compile" ... "The high order 2 digits of the sequence number are made up of the characters A through Z and 0 through 9 in the following order: A, B, C, ..., Z, 1, 2, ..., 9, A0, AA, AB, ..., AZ, A1, A2, ..., A9, B0, BA, ..., ZZ, ..., Z9, 10, ..., 99. This structure allows for up to 1295 different increments of the high order sequence number. " ... it is worth noting that this counting sequence does not sort properly in ASCII or even native EBCDIC [A9,B0,BA] which leads Real Programmers away from the messy realms of real-world problems into the comfortable zone of devising elaborate workarounds for problems they had created.
Sometimes delving into the structure of ancient computer architectures and programming languages yields new and clever insights into old problems. This is not one of those times.
<blink>down the rabbit hole</blink>
Here's a little something I posted to others.. I only figured it out 2 nights ago. I wrote it for a more general audience. I hope it helps you...
---------------------
I found a problem with the healthcare.gov site. If you aren't able to log in at all, read the following... Share as you wish.
Ignore what the instructions say. I'm showing 3 tries. The third try worked. Obviously, I'm not showing the real values I used, just values following the same pattern.
For the username, use mixed uppercase, lowercase, and numbers.
No more than 9 characters.
1st try: johnsmith_99
2nd try: johnsmith_9
3rd try: JohnSmith99
For the password, use mixed uppercase, lowercase, and numbers.
No more than 8 characters.
1st try: A0b1c2D3e4^hIJklmnoP
2nd try: A0b1c2D3e4hIJklmnoP
3rd try: AbCd3fG4
After creating the 1st and 2nd account, for the first couple days I couldn't get any automated messages when I tried the username or password retrieval.
Tonight I got responses for the password retrieval, which said the accounts didn't exist. I tried to sign up again with the 1st and 2nd username, and it said the username already existed. (Hint!)
I don't know what the real limit is on the username or password length. I used 8 characters, because it's "strong enough", and deals with the ancient crypt() problem. Always use random strings. Never use real words in any language or "31337" character substitution! (zero for O, 1 for i, 3 for e, 7 for t, 8 for B, etc). If I ever find out you use p@$$w0rd, I will personally hunt you down and slap you in the head. ... and this is what happens when a sysadmin/programmer tries to figure out how someone elses system is horribly broken.
Serious? Seriousness is well above my pay grade.
What these fail to mention is that there are usually security guards and security camera monitors protecting the monuments and parks from 9/11-like attacks and other riff-raff. It's not just a matter of opening gates.
Table-ized A.I.
Always time the launch of your web site when everyone is looking in the other direction?
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
Nonsense!
Congress does not get to soley decide how money is spent. They just start the process.
Any bill needs to be agreed upon by the congresss, senate, and president.
Didn't you watch schoolhouse rock?
Otherwise, whats to stop them from not paying for anything they dont like? Revoking the salaries for judges that dont agree with them? Or not paying for fuel for Air Force 1?
Congress needs to pass a budget that will pass senate and not get veto'd. Knowingly passing a budget that will not complete is purely political. Its normal and accepted for most things, but for the budget it is downright evil.
oldhack: "Security is a waste of money until shit hits the fan. 5 minutes later, it becomes waste of money again. "
Wait, according to wikipedia[1]:
"the bill would ban abortions after 20 weeks of pregnancy, require abortion clinics to meet the same standards that hospital-style surgical centers do, and mandate that a doctor who performs abortions have admitting privileges at a nearby hospital"
This is a right wing agenda bill? These seem like VERY reasonable medical protections for women, and a fetus' viability starts around 23 weeks...
[1] http://en.wikipedia.org/wiki/Wendy_Davis_(politician)#2013_filibuster
-=Lothsahn=-