Slashdot Mirror
Internal 'Set Of Blunders' Crashed Australia's Census Site (cso.com.au)
Slashdot reader River Tam explains the crash of Australia's online census site, citing the account of a security researcher who says IBM and the Australian Bureau of Statistics "were offered DDoS prevention services from their upstream provider...and said they didn't need it." From an article on CSO:
The ABS and IBM gambled on a plan to ask its upstream network provider to block traffic from outside Australia in the event that a denial-of-service attack was detected... Offshore traffic to the site was blocked in line with the plan, however, another attack, for which the ABS had no contingency to repel, was directed at it from within Australia. The attack crippled the firewall and the census site's operators opted to restart it and fall back to a secondary firewall. However, they forgot to check that it had the same configuration as the primary firewall. That crippled the census site.
In an unfortunate confluence of events, IBM's security warning systems started flagging some unusual activity, which indicated that information on the ABS servers was heading offshore. The site's operators, thinking the DDoS activity was a distraction, interpreted the alarms as a successful hack...these were little more than benign system logs and the technical staff monitoring the situation poorly understood it. Amid the confusion they naturally erred on the side of caution, [and] decided to pull the plug on the site...
In an unfortunate confluence of events, IBM's security warning systems started flagging some unusual activity, which indicated that information on the ABS servers was heading offshore. The site's operators, thinking the DDoS activity was a distraction, interpreted the alarms as a successful hack...these were little more than benign system logs and the technical staff monitoring the situation poorly understood it. Amid the confusion they naturally erred on the side of caution, [and] decided to pull the plug on the site...
blunders from down under.
Let me guess, "the technical staff monitoring the situation poorly understood it" were needful-doers from IBM.
I still haven't seen any mention of evidence that there was any attack at all. Well, except in the negative sense, as in "Global DDOS sensors failed to register any attack".
From the server's point of view, what exactly is the difference between "a DDOS attack from within the country" and "ten million users trying to log on to the site within one hour"?
No. People have until the 23rd of September to complete the census and you won't be fined if you didn't complete the census on the night.
There's some good news here. This ABS blunder sets the likelihood of paperless and/or online voting happening in Australia back another decade or so.
It's probably weird that as a technology geek I'd be a fan of paper voting, but paper forms are a lot harder to hack or manipulate without a trace.
In Australia the phrase 'Social License' is starting to register with the wider community. Issues such as the coal seam gas mining and a range of unpopular but otherwise legally compliant initiatives are feeling the backlash from ordinary people.
People may think that the 'Brexit' phenomenon is new, however there is a growing discontent among the wider population with the small but influential groups that ignore the views of the community affected by these schemes.
I wouldn't support the alleged DDOS attacks on the ABS web site, however the ABS has moved ahead with changes to its data retention policies without considering the associated risks, and even well known politicians are refusing to cooperate with the Census.
You can imagine the executives at the ABS discussing their planned changes and asking "what will people do if they don't like the changes" - well now they have seen what could happen.
It's more than likely that the Chief Statistician (on over $700,000 a year) will be asked to resign. It's difficult to sack him (a quirk of the legislation that created the ABS) however you would not expect that a person on such a salary would show such poor judgement.
The 'Brexit' phenomenon has only begun to unfold, and you can only hope that people look past the technology issues surrounding the ABS Census debacle and start asking the question - if you don't have community support is your idea actually any good?
In previous years, they had been quite careful to inform people to pre-fill their form before census night, and submit after. This year they were expecting only a minor increase in peak traffic.
Then they go and blast the message, "Fill in your form online, ontime or face massive fines", all over the media.
So what did we all do? When the majority of 9-5 workers got home, we all tried to login and submit at about the same time.
Sure they screwed up their network config, but it was a combination of poor planning and poor communication that triggered the whole mess.
09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
It is all about location, location, location...
My employer is on a state-wide network that connects, among other things, a ton of colleges and universities. After some recent BLM events, there were sympathy DDOS attacks from anonymous or whoever, so the state just spent millions on fancy new anti-DDOS gear on the external side of all of their POPs.
A few weeks ago, I had an opportunity to ask the state's Chief Information Security Officer what their plan was to handle internal attacks coming from the colleges, which are inside the perimeter, and typically have incredible switching and routing capacities (as part of I2), far in excess of anything our rural fiber rings could handle. It took him a few seconds to review the topology of the network in his head before he realized that we'd be screwed.
I have some sympathy for Australia. DDOS is a hard problem to solve, even if you've got millions to spend on the newest, shiniest gear.
See that "Preview" button?
30+ hours to get your password / receipt emailed to you. 20 minutes, maybe 1 hour is acceptable.
Started my census return on the 12th at 17:05 and requested the password be emailed to me.
The password email was sent by the ABS servers in the 14th at 03:50.
If I was depending upon the password to resume doing the census I would have had to wait an additional day.
Additionally the forms really didn't handle doing "Father", "Daughter", "Wife". Had to go and delete all the data entry for my daughter. Add my wife then re-add my daughter.
The prime minister Malcolm Turnbull went on the record to say that he will punish those responsible.
Yet it was the coalition government that cut the ABS budget by $68m, left the department leaderless for a year, and also poked the bear with talk of selling citizen information to make money which may have prompted the attack in the first place.
The only question is who will be the scapegoat.
IBM are masters of under quoting costs to win the business then when the organisation finds out they either payup or kick IBM out. a LOT choose to pay up as admitting failure is far worse for many than cost overruns. some of the worst I have seen is the extremely poor competence of many of the technical guys that turn up. (disclosure, we frog marched every IBM consultant off the premise after they were 2 million over budget already with no end in sight). IBM today is a poor shadow of its former self.
This sounds similar to Dutch police, who put out a press release that there website was having trouble because they where being hacked.
In about half a day they found out that they added a 40 MB JPG on there front-page and scaled it to a thumbnail using CSS....
I call BS on the whole story. What happened was the website fell over when most of the Australian population tried to log on at the same time. Did anyone else on the same network suffer similar outages?
Give yourself to the dark side. It is the only way you can save your privacy.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
It's okay, it isn't like anyone ever tells the truth on the damn thing.
According to it, this time, I have changed my religion from Jedi to Sith.
I don't know about Australia. In the UK, the whole point of the Jedi thing was to point out that the only part of the census that was optional and didn't need to be truthful was the religion question
That's what happens when you make fun of the Jedi.
Seven puppies were harmed during the making of this post.
Until today. Well ok it will be in a few weeks and be some low level public servant but the cliché will be broken nonetheless.
Why are you entering anything more than: 1 female, under-18?
Does the form record full contact information about your daughter? And you voluntarily supplied it?
No the form does not record her contact information. The only contact information was mine.
The relationship information however is not collectable unless you enter the people in the household in specific orders. i.e. the form was poorly defined as there were no instructions as far as I could see about entering people in specific orders.
I see, cheers for the explanation.
I've filled in many Government issued forms where the instructions are not very clear. Unsurprising to hear the online ones are no better.
SUSPECTS?
How is "Our source suspects" proof? Other articles have been referring to IBM hosting that lot because the ABS just does not have anything close to the infrastructure to do it in-house and a proposal to acquire more servers was denied last year.