Slashdot Mirror
Internal 'Set Of Blunders' Crashed Australia's Census Site (cso.com.au)
Slashdot reader River Tam explains the crash of Australia's online census site, citing the account of a security researcher who says IBM and the Australian Bureau of Statistics "were offered DDoS prevention services from their upstream provider...and said they didn't need it." From an article on CSO:
The ABS and IBM gambled on a plan to ask its upstream network provider to block traffic from outside Australia in the event that a denial-of-service attack was detected... Offshore traffic to the site was blocked in line with the plan, however, another attack, for which the ABS had no contingency to repel, was directed at it from within Australia. The attack crippled the firewall and the census site's operators opted to restart it and fall back to a secondary firewall. However, they forgot to check that it had the same configuration as the primary firewall. That crippled the census site.
In an unfortunate confluence of events, IBM's security warning systems started flagging some unusual activity, which indicated that information on the ABS servers was heading offshore. The site's operators, thinking the DDoS activity was a distraction, interpreted the alarms as a successful hack...these were little more than benign system logs and the technical staff monitoring the situation poorly understood it. Amid the confusion they naturally erred on the side of caution, [and] decided to pull the plug on the site...
In an unfortunate confluence of events, IBM's security warning systems started flagging some unusual activity, which indicated that information on the ABS servers was heading offshore. The site's operators, thinking the DDoS activity was a distraction, interpreted the alarms as a successful hack...these were little more than benign system logs and the technical staff monitoring the situation poorly understood it. Amid the confusion they naturally erred on the side of caution, [and] decided to pull the plug on the site...
blunders from down under.
Let me guess, "the technical staff monitoring the situation poorly understood it" were needful-doers from IBM.
.... dollarydoos will this little blunder cost?
If I were an Aussie, I'd want my data back. I'd have no confidence in the govt keeping their new privacy-destroying data secure. While I'm at it I'd take my tax back too as a vote of no confidence in anything they do.
I still haven't seen any mention of evidence that there was any attack at all. Well, except in the negative sense, as in "Global DDOS sensors failed to register any attack".
From the server's point of view, what exactly is the difference between "a DDOS attack from within the country" and "ten million users trying to log on to the site within one hour"?
$9million for hosting static pages.
Where can I bid for these contracts?
There's some good news here. This ABS blunder sets the likelihood of paperless and/or online voting happening in Australia back another decade or so.
It's probably weird that as a technology geek I'd be a fan of paper voting, but paper forms are a lot harder to hack or manipulate without a trace.
In Australia the phrase 'Social License' is starting to register with the wider community. Issues such as the coal seam gas mining and a range of unpopular but otherwise legally compliant initiatives are feeling the backlash from ordinary people.
People may think that the 'Brexit' phenomenon is new, however there is a growing discontent among the wider population with the small but influential groups that ignore the views of the community affected by these schemes.
I wouldn't support the alleged DDOS attacks on the ABS web site, however the ABS has moved ahead with changes to its data retention policies without considering the associated risks, and even well known politicians are refusing to cooperate with the Census.
You can imagine the executives at the ABS discussing their planned changes and asking "what will people do if they don't like the changes" - well now they have seen what could happen.
It's more than likely that the Chief Statistician (on over $700,000 a year) will be asked to resign. It's difficult to sack him (a quirk of the legislation that created the ABS) however you would not expect that a person on such a salary would show such poor judgement.
The 'Brexit' phenomenon has only begun to unfold, and you can only hope that people look past the technology issues surrounding the ABS Census debacle and start asking the question - if you don't have community support is your idea actually any good?
In previous years, they had been quite careful to inform people to pre-fill their form before census night, and submit after. This year they were expecting only a minor increase in peak traffic.
Then they go and blast the message, "Fill in your form online, ontime or face massive fines", all over the media.
So what did we all do? When the majority of 9-5 workers got home, we all tried to login and submit at about the same time.
Sure they screwed up their network config, but it was a combination of poor planning and poor communication that triggered the whole mess.
09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
It is all about location, location, location...
My employer is on a state-wide network that connects, among other things, a ton of colleges and universities. After some recent BLM events, there were sympathy DDOS attacks from anonymous or whoever, so the state just spent millions on fancy new anti-DDOS gear on the external side of all of their POPs.
A few weeks ago, I had an opportunity to ask the state's Chief Information Security Officer what their plan was to handle internal attacks coming from the colleges, which are inside the perimeter, and typically have incredible switching and routing capacities (as part of I2), far in excess of anything our rural fiber rings could handle. It took him a few seconds to review the topology of the network in his head before he realized that we'd be screwed.
I have some sympathy for Australia. DDOS is a hard problem to solve, even if you've got millions to spend on the newest, shiniest gear.
See that "Preview" button?
To be fair - the actual implementation of the census site, when it worked, was quite good, and a hell of an advance over the old pen-and-paper process of 4 years ago. It's a shame that this census will be remembered for it failing to handle the security / meltdown, rather than be lauded for pushing forward with better ways of gathering census data in a more modern and updated manner.
When I think of modern-day IBM, I think of two things: A company with excellent scientific research, and a company that has lots of problems with its software contracts. It seems weird that it's the same company.
It's okay, it isn't like anyone ever tells the truth on the damn thing.
According to it, this time, I have changed my religion from Jedi to Sith.
The prime minister Malcolm Turnbull went on the record to say that he will punish those responsible.
Yet it was the coalition government that cut the ABS budget by $68m, left the department leaderless for a year, and also poked the bear with talk of selling citizen information to make money which may have prompted the attack in the first place.
The only question is who will be the scapegoat.
This sounds similar to Dutch police, who put out a press release that there website was having trouble because they where being hacked.
In about half a day they found out that they added a 40 MB JPG on there front-page and scaled it to a thumbnail using CSS....
I call BS on the whole story. What happened was the website fell over when most of the Australian population tried to log on at the same time. Did anyone else on the same network suffer similar outages?
You don't believe them about it being unhackable anymore? Why, if they see even the slightest bit of server traffic, they'll take the entire thing offline. Good luck hacking into that!
Give yourself to the dark side. It is the only way you can save your privacy.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
It's okay, it isn't like anyone ever tells the truth on the damn thing.
According to it, this time, I have changed my religion from Jedi to Sith.
I don't know about Australia. In the UK, the whole point of the Jedi thing was to point out that the only part of the census that was optional and didn't need to be truthful was the religion question
The facts here are even more mundane than DDoS attacks or hacking attempts, which are just routine these days for any moderately high-profile website.
The problem was simply that some idiot politicians got together with some equally ill-informed managers and PR types and decided to have 'census day'. I can guarantee that everywhere throughout the developer food-chain for the Aus Census website there were many, many workers saying loud and clear, "this is a bad idea" - who in their right mind would expect a website to stand up to an entire nation hitting it at exactly the same time (after dinner) and what bunch or morons would engineer this exact scenario? Well, the politicians did and shot themselves in the foot, especially when it subsequently transpired that people had until September to complete the survey despite the advertising delivering a clear message that "there is one census day, and that's the day when you need to complete the census".
Yhat's what they did and the inevitable happened. All the subsequent media talk of hacks and DDoS here has come with the clear evidence that nobody in media actually knows what they're talking about when it comes to this issue.
That's what happens when you make fun of the Jedi.
Seven puppies were harmed during the making of this post.
I've seen that happen with every company I've worked for. The most recent was HSBC when they sold off some of their business to CapitalOne and needed to split off one of their data centers. Switched the firewall over on the weekend and forgot to configure the VPN routes to go with it. Of course, this all happened in the middle of moving 30% of staff to work from home. So glad to no longer be working for those idiots.
Until today. Well ok it will be in a few weeks and be some low level public servant but the cliché will be broken nonetheless.
if they were running windows 10, hahaha, it would have been even worse!
SUSPECTS?
How is "Our source suspects" proof? Other articles have been referring to IBM hosting that lot because the ABS just does not have anything close to the infrastructure to do it in-house and a proposal to acquire more servers was denied last year.