Slashdot Mirror

← Back to Stories (view on slashdot.org)

Australian Census Website Shut Down On Census Night After 4 DDoS Attacks (smh.com.au)

Posted by BeauHD on from the timing-is-everything dept.

Heart44 writes: News sites are reporting that the Australian census website has been shut down until further notice. This happened on census night, Tuesday (Australian time), August 9th, 2016. This is the first attempt at an online census where [the internet] is the default data collection method. You had to call an often busy number to get a paper form. This is on top of a long running controversy that the Australian Bureau of Statistics will keep the names and addresses of everyone for five years. I presume more useful links will appear over time. "The site was targeted by four denial of service (DoS) attacks," chief statistician David Kalisch told ABC radio. The Sydney Morning Herald reports: "The first three caused minor disruptions and did not stop more than two million census forms from being 'successfully submitted and safely stored,' he said. But the site was shut down after a 'gap' in the system's security measures was found during a fourth attack (AEST), Mr Kalisch said. 'After the fourth attack, which took place just after 7:30pm [on Tuesday AEST], the ABS took the precaution of closing down the system to ensure the integrity of the data,' Mr Kalisch said. 'I can certainly reassure Australians the data they provided is safe,' he said."

UPDATE 8/09/16: Many reports are contradicting Kalisch's claim that the website was shut down from DDoS attacks. User @mhackling on Twitter tweeted a screenshot of Digital Attack Map showing "nothing unusual DDoS wise for Australia and yesterday."

74 of 129 comments (clear)

  1. Yeaaaaaaa by Anonymous Coward · · Score: 5, Funny

    'I can certainly reassure Australians the data they provided is safe

    If you believe that I have some ocean front property in Alice Springs I will sell you...

    1. Re:Yeaaaaaaa by Michael+Woodhams · · Score: 1

      A DDOS attack does nothing to attack the integrity or security of the data. The success of a DDOS attack only indirectly calls data safety into question - if they were not able to defend against DDOS, perhaps they're also not good enough to maintain security.

      As an aside, I'm currently living in Australia, and the site worked fine for me at about 6pm.

      --
      Quattuor res in hoc mundo sanctae sunt: libri, liberi, libertas et liberalitas.
    2. Re:Yeaaaaaaa by donaldm · · Score: 3, Insightful

      A DDOS attack does nothing to attack the integrity or security of the data. The success of a DDOS attack only indirectly calls data safety into question - if they were not able to defend against DDOS, perhaps they're also not good enough to maintain security.

      As an aside, I'm currently living in Australia, and the site worked fine for me at about 6pm.

      What you said is certainly true. I tried at about 7:45 PM and from then on every 30 minutes and eventually I just gave up since the site was so busy or under DDOS attacks.

      What would be interesting (ABS take note) is how many of those DDOS slave machines were running a version Microsoft Windows and what version was the most compromised. I am sure we could think of a few more statistics to highlight but unfortunately, most people won't learn.

      As for security. If people have installed (err! Updated) or purchased a PC with Windows 10 and by default Windows 10 has telemetry including a keystroke logger then those people have effectively given Microsoft all their information. What about Google Chrome? Well it does like to collect information if you let it (it's pretty easy to turn off) however it does not log your keystrokes and if you are worried about it then use a web browser that is reasonably secure.

      For those people who used the Edge browser to fill out the Census. Sigh!

      --
      There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
    3. Re:Yeaaaaaaa by alex67500 · · Score: 1

      There was a time not so long ago where a DDoS was simply called being slashdotted...

    4. Re: Yeaaaaaaa by Dashiva+Dan · · Score: 1

      The official claim is that they intentionally took the site down as they found a security issue while trying to mitigate the DDoS. Not exactly inspiring confidence.

      Source? I saw they said: http://help.census.abs.gov.au/...

      Just after 7.30pm, the following confluence of events occurred:

      A fourth denial of service attempt
      A large increase in traffic to the website with thousands of Australians logging on to complete their Census
      A hardware failure when a router became overloaded
      Occurrence of a false positive, which is essentially a false alarm in some of the system monitoring information.

      i.e. no security issue. Their systems got overloaded, melted down, and flagged an alert for a possible issue that didn't exist, so they shut it down.

      --
      "lt;dr" is the correct response to most of my posts.
    5. Re: Yeaaaaaaa by dbIII · · Score: 1

      Those assurances are being brought to you by the same government that announced that they had found the missing airliner in the Indian Ocean last year.
      If they say something you have to wait until someone else confirms it before you can treat it as anything other than gossip.

  2. Never assume malice when stupidity will suffice by Anonymous Coward · · Score: 5, Insightful

    Never assume malice when stupidity will suffice.

    At this stage all reports indicate that the ABS cocked things up big time. The DDoS angle seems to be furious spin doctoring.

    1. Re:Never assume malice when stupidity will suffice by Heart44 · · Score: 4, Informative

      Yes, this link does not show any large DDoS attacks on Australia or in Australia. Interesting to look at what China is doing to Saudi Arabia at the moment.

    2. Re:Never assume malice when stupidity will suffice by TheGratefulNet · · Score: 1

      At this stage all reports indicate that the ABS cocked things up big time.

      so, the anti-lock(out) feature didn't work correctly, then?

      --

      --
      "It is now safe to switch off your computer."
    3. Re:Never assume malice when stupidity will suffice by bloodhawk · · Score: 3, Insightful

      It is pretty bad spin doctoring. They have just been ranting for the last week on how good the security measures implemented for the census are, either they were too stupid to put in mitigations for the most obvious and likely attack vector (DDoS) or their countermeasures were inadequate or they are lying to cover up for other security flaws or incompetence. None of those options inspire confidence, especially given the previous week of boasting that those that did not want to trust the site with information were just conspiracy nuts. Personally I took the risk of putting in fake names and DOB and dodgy address, I know that in theory makes me potentially liable for a large fine, but a fine can easily be fought or paid, identity theft because the morons at the ABS can't do security is much harder and more expensive to rectify.

    4. Re:Never assume malice when stupidity will suffice by quenda · · Score: 1

      At this stage all reports indicate that the ABS cocked things up big time. The DDoS angle seems to be furious spin doctoring.

      ABS decided a while ago to outsource the hosting to IBM, paying $10 million for development (simple webforms) and hosting (the hard part).
      Given IBM's record in Australia, you might argue this choice was a cockup.

    5. Re:Never assume malice when stupidity will suffice by bloodhawk · · Score: 1

      It is a $180 a day fine not just a flat $180 fine which caps at $1800. The fine for false data is $1000, So getting caught with fake data is actually cheaper than not providing it.

    6. Re:Never assume malice when stupidity will suffice by Anonymous Coward · · Score: 1

      It sounds like a sort-of-real DDoS - as long as you consider the expected usage of the people of Australia hitting the census site to be a DDoS. It's distributed over the whole country!

    7. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      The fine for false data is $1000, So getting caught with fake data is actually cheaper than not providing it.

      No, not quite.

      The maximum penalty for providing false or misleading information is 10 penalty units ($1800) (see s15 Census and Statistics Act 1905 (Cth)) The serious offences (see s19), with a maximum penalty of "120 penalty units ($21600) or imprisonment for 2 years, or both" is for an ABS officer divulging census information.

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    8. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      you are far less likely to be caught [if you enter fake names] than should you leave them blank

      Please explain.

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    9. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      I should add, that there is an argument that "the ABS has no power to commence prosecution action for Australians not providing 'name'". This from the former Australian Statistician Bill McLennan.

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    10. Re:Never assume malice when stupidity will suffice by bloodhawk · · Score: 1

      Either way still far better off with wrong details as that is far harder to spot than blanks which they have repeatedly stated they will chase people for (though maybe they won't with so many senators also saying they will leave blanks). So basically the fine is the same (Seems $1000 was for last census, this one is $1800)

    11. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      Either way still far better off with wrong details as that is far harder to spot than blanks ...

      For questions apart form name (and possibly address) that may well be the case. With the name/address fields different considerations apply. Not that there's much use crying over spilt milk, but for those of us who participated in last night's DDoS ... err I mean were unable yet to complete our census, these things are worth considering.

      Firstly if McLennan is correct (see my other post to you) there is no liability at all for failing to disclose 'name.' So you might be infinitely worse off putting a false name (yes I know it's not actually infinite but undefined). I say 'might' because the most obvious line of defence would be that lacking the power to collect names compulsorily false name information should not be not covered under s15. Worth a shot anyway.

      Moreover the name/address information was traditionally used to check compliance. Given the census number was delivered (paired) to a specified address a ensuring there is no mismatch between the delivery address and the supplied address should be a routine error check ... if they were competent.

      Seems $1000 was for last census, this one is $1800

      I think in 2013 a penalty unit went from being $110 to $170. I was surprised that it is now $180 (not surprised it's gone up, surprised that it has gone up only by $10).

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    12. Re:Never assume malice when stupidity will suffice by Anonymous Coward · · Score: 1

      Well the problem they have is they claim the data isn't used for data matching, in order to catch out a false name or Date of Birth they have to be using the Data in precisely the manner they claim it is not and cannot be used for. So in effect ABS would be openly stating they are liars and in breach by prosecuting someone for a false name.

    13. Re:Never assume malice when stupidity will suffice by donaldm · · Score: 2

      Personally I took the risk of putting in fake names and DOB and dodgy address, I know that in theory makes me potentially liable for a large fine, but a fine can easily be fought or paid, identity theft because the morons at the ABS can't do security is much harder and more expensive to rectify.

      Oh! really clever aren't you.

      When you get the ABS letter for your address it has a unique number on it which makes it incredibly easy to know which address that number is from. So putting in a bogus address is sure to raise a huge red flag and a please explain from the Government.

      If you think all the people in the ABS are morons then think again. Some have Master's and PhD's in Mathematics and Statistics as well as computer science, so it would be very easy to track you down. Let's put it this way. "Did you fill out the census from your home or mobile?" - you did well say hello to a fine.

      Personally, I am still trying to get onto the census website since it is so busy and when I do I will be doing the census from a Linux operating system using a more trusted web browser such as QupZilla (comes standard with Fedora 24). If you have done the Census from Windows 10 and using the Edge browser congratulations you have just given a foreign country your information even though some of it may be fraudulent.

      --
      There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
    14. Re:Never assume malice when stupidity will suffice by donaldm · · Score: 1

      you are far less likely to be caught [if you enter fake names] than should you leave them blank

      Please explain.

      When the Census letters were mailed out all had a unique number related to each address they were mailed to. Basically, if you have a physical mailing address then the Post Office and conversely the government knows about it as well. In addition, the Government knows via the Births Deaths and Marriages Office (also include immigrants) your name. By the way, Governments in all countries in the world do this and have been doing this for a few thousand years now.

      If you live in a society chances are that society has a form of government that needs to know the people who live in it. Yes, you could call it Big Brother but unless the government knows something about the people that live in the area under their control then how can they assist or even oppress them.

      BTW. When I mention "oppress" it is a really stupid government that thinks that this method is the best unless they want to end up swinging from a rope or at the wrong end of a firing squad.

      --
      There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
    15. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 2

      The link you provided also states that the AGS and Bureau disagrees with his conclusion, I would not put faith in what a statistician says on legal matters over what the lawyers are saying.

      I agree. That is why I wrote "there is an argument." Moreover the argument, to wit, that a name is not 'statistical information' for the purposes of ss8,9 & 12 of the Act (if I understand Mr McLennan) is not hopeless IMO. Which is far from saying it would prevail.

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    16. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      How does this establish the contention that "you are far less likely to be caught [if you enter fake names] than should you leave them blank"?

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    17. Re:Never assume malice when stupidity will suffice by bloodhawk · · Score: 1

      from the same page. "The Census and Statistics Act 1905, ensures that Census data is never released in identifiable form, or released to any court, tribunal or other agency. This will not change. No identifiable, private or confidential data will be shared by the ABS with anyone. "

    18. Re:Never assume malice when stupidity will suffice by Anonymous Coward · · Score: 1

      Also not relevant. The issue here is whether they "data match" internally. I asked where it is they guarantee that they don't. (The answer should be in the form of a URL, it is not to be found on the page I cited).

      And if they don't data match internally: how will they check that "no household is missed in the Census" using the collected addresses?; and additionally how will they check that "the Census covers the entire population" using the collected names?

    19. Re:Never assume malice when stupidity will suffice by Capsaicin · · Score: 1

      And yet the very reason name/address data has traditionally been collected is to ensure compliance. As has been noted below, the ABS explains that one of the main reasons of collect names remains "to ensure the Census covers the entire population" and one of the main reasons to collect addresses remains "to ensure that no household is missed in the Census." How would a fake address satisfy them that your household has complied?

      I also pointed out elsewhere "given the census number was delivered (paired) to a specified address a ensuring there is no mismatch between the delivery address and the supplied address should be a routine error check ... if they were competent."

      As donaldm (despite confused in his response to me) put it below: "When you get the ABS letter for your address it has a unique number on it which makes it incredibly easy to know which address that number is from. So putting in a bogus address is sure to raise a huge red flag and a please explain from the Government."

      Similarly with fake names: when your census id is paired with a household of people who simply don't exist in any other data source (albeit that these connections are to be established via anonymised links however that is supposed to work)... it might raise a few suspicions, no?

      Perhaps not so much with a fake name, but surely a fake address should raise that red flag.

      --
      Better to be despised for too anxious apprehensions, than ruined by too confident a security. --Edmund Burke
    20. Re:Never assume malice when stupidity will suffice by dbIII · · Score: 1

      Given IBM's record in Australia

      These days IBM has little more than a shopfront in Australia while most of their workers are in mainland China. Source: a few ex-IBM guys I know who flew to China a few years back to train their replacements.
      As for their record, one of the things the above poster is referring to is a payroll system fuckup so bad that it was the major cause of a government getting voted out of office for three years despite the alternative being a bunch of corrupt idiots.

    21. Re:Never assume malice when stupidity will suffice by bane2571 · · Score: 1

      Because the form can be matched to the ID number on it and that ID is directly linked to the address the form was mailed to.

      Therefore the address can be cross-checked to find deliberately false entries.

      Since the penalty for false entries is higher than non submission it would make sense to use limited enforcement resources to target those that were deliberately false. All that said, given the whole debacle I expect there will be no fines or prosecutions out of this whole mess - it smacked of scare tactics in the first place anyway.

    22. Re:Never assume malice when stupidity will suffice by dddux · · Score: 1

      They could recognise your handwriting? ;)

      --
      "It is no measure of health to be well adjusted to a profoundly sick society." - Jiddu Krishnamurti
  3. credit card by Smiddi · · Score: 2

    I got stuck at the "Please enter your credit card details" question.

  4. How can you tell? by Anonymous Coward · · Score: 1

    What's the difference between a DDoS attack and 4 million people all trying to submit their census all at the same time?

    1. Re:How can you tell? by Smiddi · · Score: 4, Insightful

      Its better politically to blame "overseas hackers" than admit they screwed up.

    2. Re:How can you tell? by PPH · · Score: 3, Funny

      Four million people?!! Crikey! We didn't know there were that many. I guess we should have counted them or something.

      --
      Have gnu, will travel.
    3. Re:How can you tell? by Dashiva+Dan · · Score: 1

      What's the difference between a DDoS attack and 4 million people all trying to submit their census all at the same time?

      Nothing, aside for that it's a distributed attempt to get service, not denial attempt, so probably even more effective at clogging the system. They spent about AU$400,000 on load testing (Should've been more than enough). They don't want to admit this was wasted money, and their IT guy said "With this many people trying to fill it out at once it's just like a DDOS attack!" so they've just gone with it.

      --
      "lt;dr" is the correct response to most of my posts.
    4. Re:How can you tell? by quenda · · Score: 1

      What's the difference between a DDoS attack and 4 million people all trying to submit their census all at the same time?

      In simplest terms, real census users would peak on the order of 1000 new sessions per second.
      A large botnet DDOS can do a million connection requests per second.

    5. Re:How can you tell? by Neo-Rio-101 · · Score: 3, Insightful

      Its better politically to blame "overseas hackers" than admit they screwed up.

      but even that is a crappy excuse.

      There's no reason at all for the rest of the internet outside of Australia to even have access to the Census website.
      They could have at least geo-blocked any IP address originating from outside Australia.
      Such a simple solution to that problem, that *not* doing it makes them look incompetent.

      --
      READY.
      PRINT ""+-0
    6. Re:How can you tell? by maglor_83 · · Score: 1

      That works out to 3.6M requests per hour.

      They boasted themselves that their servers could handle 1M submissions per hour. I can't believe they said that, because it's obviously not enough when they're expecting 12M submissions in one evening.

    7. Re:How can you tell? by Barny · · Score: 2

      "Black Friday Online"?

      You want to burn half of Victoria, virtually?

      --
      ...
      /me sighs
    8. Re:How can you tell? by MrKaos · · Score: 1

      Nothing, aside for that it's a distributed attempt to get service, not denial attempt, so probably even more effective at clogging the system. They spent about AU$400,000 on load testing (Should've been more than enough).

      Evidently they didn't do the load testing properly. If they can't get that right how can anybody expect them to secure personal data properly.

      Yet they're forcing mandatory retention of personal data.

      They don't want to admit this was wasted money, and their IT guy said "With this many people trying to fill it out at once it's just like a DDOS attack!" so they've just gone with it.

      By claiming it's a DDOS it just proves even more that they can't secure anything. How can they be trusted to keep sensitive data if they can't get something so basic functioning properly?

      --
      My ism, it's full of beliefs.
    9. Re:How can you tell? by quenda · · Score: 1

      I'm just saying that a DDOS can be orders of magnitude bigger than even a nation census.

      And it is 12 million responses total, maybe 6 million online, spread over days and weeks.
      Time is not especially critical.

    10. Re:How can you tell? by Anonymous Coward · · Score: 1

      It's worse than that - they're actually simultaneously saying that they blocked any overseas traffic from just before midday, *and* that an overseas DDOS took the system down. What a load of shite.

    11. Re:How can you tell? by jaa101 · · Score: 1

      They're saying the DDOS took down the geoblocking service. This would appear to be the "hardware failure" that is being blamed but it seems more likely that the geoblocking service couldn't handle the load.

    12. Re:How can you tell? by trawg · · Score: 1

      According to their PR people that is apparently what they did.

      This timeline of events suggests that the second DDOS (or "a significant increase in traffic") occurred at 11:46am local time.

      At 11:50am local time they blocked all international traffic. This somehow lead to a "short system outage" (which I assume means the whole thing collapsed).

      At 4:58pm there was another increase in traffic, "automatically defended by network fire walls". One must assume then that this was all local traffic if we assume that all international traffic was blocked - so either local DDOS impact, or, maybe, new demand from legitimate users.

      At 7:30pm though is where things get interesting. There's another "significant" denial of service. This coincides with a lot of legitimate traffic as we enter Australian peak Internet hours. (Again, we can wonder if the DoS was actually just legitimate users smashing their application, but there's no data to decide one way or the other.)

      But the fascinating part is that this incident was "significant" because their "geo-blocking service fell over". This apparently then caused a router failure.

      First of all, what?! Secondly, from this description it sounds like they were using a server-side geoip mechanism to block the international traffic that was responsible for the DDOS. This will obviously not help in cases where the sheer volume of DDOS traffic is overwhelming the network (which, in Australia, is most of them).

      So the question is: was their DDOS mitigation plan limited to simply blocking the DDOS on the server side? Did they not have a contingency to contact their upstream network providers and block entire international routes (which would have cut the impact of most DDOSs off at the knees)?

      Sadly most of this information (I think) came from a non-technical press conference, so there's not a lot of hard technical information available yet.

      I hope that the ABS will make a lot of their information public - not so that us nerds criticise this whole train wreck (though that will be fun too), but so everyone can learn from the mistakes that were made and we can build better infrastructure.

    13. Re:How can you tell? by marka63 · · Score: 1

      Except when the CDN network geolocates you in the US rather than in Australia and you can't even get the webpage to display.

    14. Re:How can you tell? by Dashiva+Dan · · Score: 1

      Nothing, aside for that it's a distributed attempt to get service, not denial attempt, so probably even more effective at clogging the system. They spent about AU$400,000 on load testing (Should've been more than enough).

      Evidently they didn't do the load testing properly. If they can't get that right how can anybody expect them to secure personal data properly.

      Yet they're forcing mandatory retention of personal data.

      They don't want to admit this was wasted money, and their IT guy said "With this many people trying to fill it out at once it's just like a DDOS attack!" so they've just gone with it.

      By claiming it's a DDOS it just proves even more that they can't secure anything. How can they be trusted to keep sensitive data if they can't get something so basic functioning properly?

      My first part was a little bit tongue in cheek. half a million to a company that specialises in such should have been enough but clearly wasn't.
      However you seem to be harping on the security of the data - There was no "security breach" - No one got access to their systems. They simply got overloaded (blew up a router, etc) and shut it down because it simply wasn't robust enough. But zero security issues. Keeping a server up and running and able to support a predictive load is one thing, security of data is another thing entirely. Those responsible for the server being able to handle the traffic have nothing whatsoever to do with those ensuring the security of the data.
      Then again, don't trust anyone to keep any data secure and you'll be better off. Government requires we submit this data - and it can - so either fill it out and suck it up, pay the fine, or leave the country, but never assume perfect security.

      --
      "lt;dr" is the correct response to most of my posts.
  5. Re:dear ieee reader by Anonymous Coward · · Score: 1

    The Internet itself is the only necessary technology. Connections have limited bandwidth. As soon as you get multiple endpoints trying to hit a single endpoint with a capacity that exceeds that endpoint, DDoS will ensue.

  6. Re:Canada Australia by Strider- · · Score: 1

    Well, we did crash it because of demand when the cards were mailed out... Sometimes we canucks are such geeks...

    --
    ...si hoc legere nimium eruditionis habes...
  7. Lie down with pigs..... by Anonymous Coward · · Score: 1

    The web server setup was supplied by IBM - the Bureau of Stats had a $9.6million deal with IBM.

    http://www.itnews.com.au/news/ibm-wins-96m-to-host-ecensus-in-2016-397613

    Perhaps it's time to declare IBM and its officers persona non grata in Australia - they were also involved in the Queensland Health
    payroll fiasco a few years ago.

    1. Re:Lie down with pigs..... by dwywit · · Score: 2

      To be fair to IBM, Qld Health signed off every stage of the project, and:

      http://www.abc.net.au/news/201...

      It was mostly the fault of the senior public servants involved.

      My involvement with IBM in Queensland in the mid-to-late 1980s and early 90s taught me a few things:

      1. IBM solutions cost a lot more than other peoples' solutions
      2. IBM at its best was a thoroughly professional and competent group of people
      3. IBM at its worst is still expensive

      --
      They sentenced me to twenty years of boredom
  8. IBM wins $9.6m to host eCensus in 2016 by Lefty2446 · · Score: 5, Informative

    http://www.itnews.com.au/news/...

    ABS ditches in-house plans in favour of outsourcing.
    The Australian Bureau of Statistics has opted not to build its own private cloud to host the 2016 eCensus, instead awarding a $9.6 million outsourcing contract to existing partner IBM.

    Australia’s national statistics agency first offered Australians the option to avoid completing the Census via its traditional paper-based form with a web-based eCensus in 2006.

    It partnered with IBM in a $9 million deal in 2005 to develop and support the web-based eCensus application - which is hosted on IBM’s AIX operating system and a WebSphere application server, out of the company's Baulkham Hills, Sydney data centre.

    But the agency later virtualised its server infrastructure (with VMware’s vSphere) to create its own private cloud with the intention of hosting the 2016 eCensus.

    Running the Census in-house would help address security perceptions arising from the data being handled from a third-party, the ABS said at the time. It said it also made sense to outsource the project to a third-party rather than deal with the one-off high traffic spike internally.

    The agency became 95 percent virtualised after cutting 300 physical servers to 70, which hosted 1500 virtual machines.

    But the Bureau of Statistics today confirmed it had decided to once again partner with IBM for hosting of the 2016 eCensus in order to ensure the expected high volumes would be properly managed.

    The ABS expects the percentage of Australians completing the census online to double in 2016, forecasting a 65 percent take-up compared to 33 percent in 2011. For the first year of the eCensus, 10 percent of Australians submitted their form online.

    “The ABS virtualisation project was successfully completed providing a very efficient platform for ongoing ABS operations, including supporting a number of components of the digital Census in 2016,” a spokesperson said.

    “However, due to the peak volume of the online form during Census 2016 it was decided that contracting IBM would provide the best value for money and management of operational risk.”

    Duncan Young, head of the 2016 Census within the ABS, said IBM had been contracted through a limited tender after proving it could offer the best value for money.

    “This contract capitalises on the investment in the existing online Census system,” Young said in a statement to iTnews.

    “Our existing solution has shown itself to be robust, and can be expanded to manage increased volumes. Using a known platform will reduce the risk of costly development and integration issues.”

    The IBM contract will expire in October 2016.

    1. Re:IBM wins $9.6m to host eCensus in 2016 by _Sharp'r_ · · Score: 1, Insightful

      Yeah, this sounds as much like a DDOS as the Healthcare.gov rollout.

      Guys, it's not a DDOS just because people are trying to use the web site and it sucks so bad that they can't...

      --
      The party of stupid and the party of evil get together and do something both stupid and evil, then call it bipartisan.
    2. Re:IBM wins $9.6m to host eCensus in 2016 by LordLucless · · Score: 1

      http://www.abc.net.au/news/201... [abc.net.au]

      Now they are saying it's not been attacked from overseas.

      Nah, they're still saying they were DDoSed, they just don't want to use the word "attack" (despite it being an attack) because it makes it sounds like they lost (which they did). Just the usual political weaselling.

      Personally, I believe they were DDoSed, and it didn't show up on the maps because the attack was minuscule, but managed to take down their servers anyway, because it exploited a flaw (say, an expensive operation they could trigger) that gave it a potency beyond its scale.

      --
      Just because you're paranoid doesn't mean there isn't an invisible demon about to eat your face
    3. Re:IBM wins $9.6m to host eCensus in 2016 by thegarbz · · Score: 1

      http://www.itnews.com.au/news/...

      ABS ditches in-house plans in favour of outsourcing.
      The Australian Bureau of Statistics has opted not to build its own private cloud to host the 2016 eCensus, instead awarding a $9.6 million outsourcing contract to existing partner IBM.

      This would be the same IBM that one of the states of Australia has blacklisted from IT contracts for the government.

      Yay consistency.

  9. Re:Canada Australia by c-A-d · · Score: 4, Funny

    I got to do the damned thing twice this year. Once because they thought my PO Box was an apartment. Another because they sent one directly to my home. I filled out both truthfully and marked "0" as the number of residents at my PO Box. The other, I filled out with less than clear answers.

    --
    some karma... and kinda lukewarm about it.
  10. Re:The DDOS attack was conducted by... by Lefty2446 · · Score: 1

    I wish I had Mod points :-D

  11. Re:The DDOS attack was conducted by... by xxxJonBoyxxx · · Score: 2

    Whad'ya expect from an island of criminals and reprobates?

  12. Not hacked. Just bad capacity planning by Neo-Rio-101 · · Score: 4, Insightful

    http://www.abc.net.au/news/201...

    Now they are saying it's not been attacked from overseas.

    How hard would it have been to "do a Netflix" and block IP addresses based on location anyway? - That would at least stem the amount of foreign intelligence services from trying to hack the website which contains information on Australian citizens.

    I read that they tested the system to 150% capacity, where 100% capacity was estimated to be 1 million forms processed per hour.

    http://www.abc.net.au/news/201...

    That estimate was a gross underestimation of the numbers of sessions needed to handle an estimated 16 million households - all of whom most likely would have logged in during a 4-6 hour period in the evening. You don't have to be a rocket scientist to calculate that the system didn't have the capacity to deal with this spike in traffic.

    The capacity should have been somewhere in a ball park of 5-10 million forms processed per hour, or more.
    Couldn't have been cheap to have load balancers maxxed out trying to maintain that many accelerated SSL sessions.... but there you go.

    --
    READY.
    PRINT ""+-0
    1. Re:Not hacked. Just bad capacity planning by well_in_theory · · Score: 1

      I believe they did block foreign IPs earlier in the day anyway, but my (limited) understanding is that doing so doesn't really help with DDoS - you still have the traffic banging on your door and need to reject it.

      I'd like to know who was actually in charge of load balancing/capacity - ABS or IBM?

    2. Re: Not hacked. Just bad capacity planning by sexconker · · Score: 1

      No one believes that shit.

    3. Re:Not hacked. Just bad capacity planning by AHuxley · · Score: 1

      Re 'You don't have to be a rocket scientist to calculate that the system didn't have the capacity to deal with this spike in traffic."
      Expert US firms exist that can plan for millions of people clicking and entering small amounts of text on an encrypted web site over a few hours.
      They do it well and their clients globally have no issues...
      Buy bandwidth, talk with telcos, ensure national backhaul is ready, rent, scale and test. Why was an expected and totally captive user count so to understand and plan for?
      This was not some random on the day global event or unexpected spike in users numbers. The population of computer users was very well understood, the data size and flow was set and very well understood. Most nations do have a grasp of how many homes have internet, use wireless, adsl, optical or other networks... Been a 5 eye nation, ongoing telco capacity should be the one task thats with in the grasp of experts at the national level over the decades.

      Re "... calculate that the system didn't have the capacity"
      What was the clog? A big computer network could just not keep up with the only task it was designed for and had some time to be fully tested for?
      Or a big 3rd party pipe provider was just not tested or ready?

      --
      Domestic spying is now "Benign Information Gathering"
    4. Re: Not hacked. Just bad capacity planning by well_in_theory · · Score: 1

      Well, sure, if they've 100% absolutely verified that position, then I guess there's nothing further to discuss.

      Except, I guess... http://www.news.com.au/technol...

    5. Re: Not hacked. Just bad capacity planning by AHuxley · · Score: 1

      AC Census 2016: Attack ‘not work of hackers’ says minister (August 10, 2016)
      "ABS census security was not compromised. I repeat, not compromised and no data was lost"

      --
      Domestic spying is now "Benign Information Gathering"
    6. Re:Not hacked. Just bad capacity planning by thegarbz · · Score: 1

      How hard would it have been to "do a Netflix" and block IP addresses based on location anyway?

      Ask Netflix since it never actually worked for them.

      But then there's a question of does a DDoS originate from another country or from compromised machines within a country, and also does the solution justify cutting off Australians who are temporarily overseas after threatening them with a fine for not completing the census.

    7. Re:Not hacked. Just bad capacity planning by complete+loony · · Score: 1

      They blast all over the media that we *have* to do it on one particular day, or be fined for every day late. Just what did they expect us to do?

      --
      09F91102 no, 455FE104 nope, F190A1E8 uh-uh, 7A5F8A09 that's not it, C87294CE no. Ah! 452F6E403CDF10714E41DFAA257D313F.
  13. Re: Never assume malice when stupidity will suffic by sexconker · · Score: 2

    For those who don't know the rest: IBM farmed out all labor to the 3rd world and it the product was delivered in a busted, useless state.

  14. Results of census by kevingolding2001 · · Score: 1

    Details about the results of last nights census available here:
    http://www.theshovel.com.au/20...

  15. Re: Yay Freedom! by Anonymous Coward · · Score: 1

    I'm completely OK with censuses the way they used to be done in an anonymous manner. Accurate statistics is very important for policy , budgeting and service provision. What I'm *not* ok with is the move to make this non anonymous. With the rise in authoritarianism in governments globally (and in some cases outright fascism) do I really want to be on some shady government department as an atheist greens voting guy with a pol sci degree and history of causing activist havoc for state govt (well they probably already know that bit). Well no , and reassurances by the govt mean nothing. Even if I trust this prime minister (and I don't) who's to say the next one isn't some maurauding SiÅ(TM) joh clone sending the spooks after people with dissenting opinions. Pass!

  16. Re:Canada Australia by innocent_white_lamb · · Score: 1

    It was actually because the folks behind the server didn't test it with the graphics enabled. The Stats Can webform could handle something like 60,000 concurrent connections when they tested it. Then after testing, they added the graphics to make it look pretty and didn't do any more testing.

    Put it live, and BLAMMO.

    --
    If you're a zombie and you know it, bite your friend!
  17. Re:dear ieee reader by GloomE · · Score: 1

    In this case the multiple endpoints were likely the legitimate users. They tested for around 1M submissions per hour (reports vary from 750,000 to 1,500,000). There are 15M total expected users. My bet is that some significant amount got home from work, had some dinner, then attempted to use the site.

  18. Yes it was a DDOS by smeg+for+brains · · Score: 1

    But not an attack

    They designed the system to handle 1,000,000 submissions per hour

    Trouble is, 70% of the population live on the east coast, and I'm guessing many people decided to do their civic duty after dinner

    So, several million people all tried to log on at the same time from different location, this is distributed - causing catastrophic failure as the system was overloaded - a denial of service

    Government claim the "switched off" the site down to protect the data, (although they also say the data was never at risk). They also say that they wouldn't be bringing it back up that night - yet, less than 3 hours after the failure, I managed to log on successfully - on a site the government said was "switched off"

    Politicians lie - in other breaking news, the sky is blue, water is wet and Phelps wins a gold medal.

    --
    Watch out, there are Llamas!!
  19. Basic Electoral Fraud by rsborg · · Score: 2

    Never assume malice when stupidity will suffice.

    At this stage all reports indicate that the ABS cocked things up big time. The DDoS angle seems to be furious spin doctoring.

    Basic Electoral fraud starts with gerrymandering - an input of which requires census data to be amenable to the district hacking.

    --
    Make sure everyone's vote counts: Verified Voting
  20. Apply Grey's Law by rsborg · · Score: 1

    Such a simple solution to that problem, that *not* doing it makes them look incompetent.

    Incompetence at large scale is indistinguishable from malice in the outcome. Insiders should be suspect in such a clear case of fucking up.

    Gray's Law
    http://wikidumper.blogspot.com...

    "Any sufficiently advanced incompetence is indistinguishable from malice."

    --
    Make sure everyone's vote counts: Verified Voting
  21. Re:dear ieee reader by dbIII · · Score: 1

    Since the advertising all used the term "census night" it's very likely that most people logged in a bit after sunset no matter what their work commitments were. I was under the mistaken impression that the site would not be live in the morning based on that advertising.

  22. They did try that, but ... by dbIII · · Score: 1

    They tried at some point but the geo-blocker fell over and then ONE router owned by a different company (so thus untouchable until their staff arrived) fell over.
    Isn't outsourcing to the "cloud" wonderful?