Slashdot Mirror


Australian Bureau of Statistics Doesn't Like Direct Downloads of Census Data

Bismillah writes "The ABS has released the census data for the country under a Creative Commons license, but instead of making it easy to get, they've put in Javascript to obfuscate file paths and more. All commented in the source code of course." At first glance, it's an attempt to get people to pay $250 for a DVD with the data instead.

36 of 136 comments (clear)

  1. Bit torrent by DiSKiLLeR · · Score: 4, Insightful

    Sounds like an excellent use for Bit Torrent? I assume someone will download the whole dataset and make a torrent out of it before long....

    --
    You can tell how powerful someone is by the magnitude of the crime they can commit and be able to get away with.
    1. Re:Bit torrent by gagol · · Score: 2

      Must be time for another crocodile dundee movie.

      --
      Tomorrow is another day...
    2. Re:Bit torrent by Opportunist · · Score: 4, Insightful

      Careful there, we still remember what happened to the last guy who tried to make public data public...

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    3. Re:Bit torrent by Darinbob · · Score: 5, Funny

      Ha, I won't visit sites with goat in the name. Fool me once...

    4. Re:Bit torrent by cheater512 · · Score: 4, Interesting

      Actually the census data has a whole pile of interesting nuggets in it.
      I do have portions of it right now.

      I didn't notice all the javascript however and thought the download process was straight forward.
      Kudos to the ABS for using Creative Commons.

    5. Re:Bit torrent by purpledinoz · · Score: 2

      I'm glad I'm not the only one with this fear.

    6. Re:Bit torrent by Opportunist · · Score: 2

      Maybe you can shed some light onto this, I still don't get what crime he actually committed.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    7. Re:Bit torrent by deimtee · · Score: 2

      Paul Hogan's "financial adviser" (offshore "tax minimiser") ran off with his fortune recently.

      Fair dinkum? Strewth, I bet he's madder 'n a mallee bull.

      --
      I'm guessing that wasn't on their radar screen...
    8. Re:Bit torrent by Anonymous Coward · · Score: 2, Informative

      He annoyed those in power when he made the PACER documents available for free.

    9. Re: Bit torrent by hawkinspeter · · Score: 2

      Huh? It is released under the Creative Commons license.

      --
      You're a temporary arrangement of matter sliding towards oblivion in a cold, uncaring universe
    10. Re:Bit torrent by houghi · · Score: 2

      You don't have to commit a crime to be convicted.

      --
      Don't fight for your country, if your country does not fight for you.
  2. Link to torrent by sdreader · · Score: 5, Informative

    Link to the torrent of the census data from the article:
    http://blog.angrygoats.net/2013/04/12/2011-australian-census-release-3/

    Since the data is available for free (obfuscated or not) and was released under a CC license, technically this should all be considered legal, right? Not that it should be necessary of course.

    --
    Apparently being anti-Steam is grounds for insults, even if there's basis. I shall learn to keep my mouth shut.
    1. Re:Link to torrent by mjwx · · Score: 2

      Link to the torrent of the census data from the article:
      http://blog.angrygoats.net/2013/04/12/2011-australian-census-release-3/

      Since the data is available for free (obfuscated or not) and was released under a CC license, technically this should all be considered legal, right? Not that it should be necessary of course.

      The obfuscation is probably because hosting and bandwidth are not cheap in Oz and some inventive public servant (stop snickering, they do exist, there aren't many of them but they do exist) came up with a way to reduce the bandwidth bill. With the current emphasis on public service spending and impending election, this wouldn't surprise me.

      Either that or some hopeless public servant coder has no idea what they've done.

      Could be either case really, I've seen both.

      --
      Calling someone a "hater" only means you can not rationally rebut their argument.
    2. Re:Link to torrent by mjwx · · Score: 3, Interesting

      There may be inventive public servants, but I highly doubt they are inventive enough to make a stupid obfuscated download system just so that some guy would bittorrrent it, and thereby save the government a small amount of money on bandwidth. I mean really.

      You've never worked in the APS have you. The fewer people you have to serve, the better your balance sheet looks. If someone else can do it, why not.

      --
      Calling someone a "hater" only means you can not rationally rebut their argument.
    3. Re:Link to torrent by GerryHattrick · · Score: 3, Informative

      Usual reason for doing this with official data is to avoid sensation-seekers 'hotlinking' to specific data without noting the disclaimers, statistical cautions, changes of basis etc. which moderate any interpretation.

    4. Re:Link to torrent by SgtChaireBourne · · Score: 3, Insightful

      There may be a usual reason to ensure disclaimers, etc are read, but javascript is definitely not the way to go. You can very easily require a specific http referrer URL by configuring Apache to require it for a file or directory. Or you can simply have a plain old README or LICENSE file included in the tarball. Javascript just hurts usability and makes things over complicated and broken.

      --
      Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
  3. Re:Criminals and retarded monkeys by Noir+Angellus · · Score: 5, Funny

    ... descended from criminals and retarded monkeys.

    No, we're not all descended from the English let alone Americans.

  4. Excellence in Government by edibobb · · Score: 3, Interesting

    I think that is really funny! They declare the data free and then make an inept attempt to force people to pay for it. It's almost as bad as copyrighting public laws.

    1. Re:Excellence in Government by Opportunist · · Score: 2

      Funny, isn't it? Laws are the only thing corporations invest a lot of money in that they don't try to copyright or patent.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    2. Re:Excellence in Government by Anonymous Coward · · Score: 4, Interesting

      It's almost as bad as copyrighting public laws.

      I'm not sure if you're joking here but the Australian government actually DOES copyright legal documents. For example to comply with telephone wiring regulations requires access to a document released by "Standards Australia" which costs about $200 last I checked. I don't doubt that the document was developed using public funds. I'm sure this shit happens a lot more than people realise.

  5. Damn Streisand Effect by istartedi · · Score: 4, Funny

    Thanks a lot Slashdot. Now I have a sudden urge to know precisely how many married couples with the husbands between the ages of 30 and 32 inclusive have children in Queensland, and what the genders of and ages of the children are.

    --
    For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
  6. If you're wondering why the data are licensed... by Anonymous Coward · · Score: 4, Informative

    Remember that in many countries works of the national government are not automatically in the public domain like they are in the US. In Commonwealth realms this is called Crown Copyright.

    https://en.wikipedia.org/wiki/Crown_copyright/

  7. You Know What They're Up To? by Greyfox · · Score: 4, Interesting
    This follows on my "best method to get tech support from a computer person." You don't ask "How do you...?" You assert, loudly, within hearing range of the computer person "This is the absolute beset way to do it!" and provide a woefully incorrect method of getting to the result you're trying to achieve. One of these methods will have the computer person falling over himself to help you. Guess which one. Have I mentioned that I'm Evil lately?

    Anyway, they're pulling the same thing here. They want someone to gather up their data and present it in a nice package for free. The best way to do that is to drop an ineptly-presented steaming pile of crap on the internets. There'll probably be 15 open source projects to slice and dice it on github by the weekend, and it didn't cost the Australian government a dime! It's brilliant!

    --

    I'm trying to teach myself to set people on fire with my mind... Is it hot in here?

    1. Re:You Know What They're Up To? by martas · · Score: 2

      Yes, because it takes a lot of technical know-how to create a zip file and making it available as a torrent... /s

    2. Re:You Know What They're Up To? by Gaygirlie · · Score: 2

      Damn, I would've never thought of that. I gotta keep this in mind, knowing how to manipulate people is an exceedingly useful skill.

  8. Re:What would Aaron Swartz do? by gagol · · Score: 2

    This is the idiot tax. By taking wealth from people unable to think, we restrict their ability to feed children and slow down the epidemic. Brilliant!

    --
    Tomorrow is another day...
  9. Do not attribute by charlism · · Score: 2

    to conspiracy that which can be explained by incompetence. The real goal could of had nothing to do with "hiding" the data.

  10. Re:s/months/decades by Anonymous Coward · · Score: 5, Informative

    Actually, he was facing over 50 years in prison.

  11. I can see the conversation that happened.. by gregmac · · Score: 2

    From the code:

    // Also, generate a random number, which we append to the URL, to make it appear as if a complex
    //key is required. This is a pathetic attempt to discourage someone from downloading the ZIPs
    //directly (ie. without having to login), if they deduce the URL pattern.

    Translation:

    Coder: "Here's the census web application."
    PHB: "Great. But wait..I can just type in these other names and download them really easily! People will hack us and we'll be out possibly a COUPLE THOUSAND DOLLARS! "
    Coder: "It is Creative Commons data, so of course we added no protection. Changing that now will be a massive rewrite and take months."
    PHB: "So let's add some random numbers to the end so it looks really complex and people can't guess how to get in."
    Coder: "But they still will eventually see the links because they do actually have to download it, so this is not really doing anything."
    PHB: "Psh, no one is smart enough to figure that out. I read about this GUID things and they're really hard to guess. It will work. This is your job today."
    Coder "..Ok, fine. I'll do it exactly the way you asked."

    --
    Speak before you think
    1. Re:I can see the conversation that happened.. by robot5x · · Score: 2

      ya the code snippet provided in parent post indicates to me that this was something forced upon a level-headed coder by some moronic middle manager.

      --
      Hej! Nasi tu byli!
  12. Re:No copyright on facts by FireFury03 · · Score: 2

    What is the point of putting a creative commons license on data that is not copyrightable. Anyone can take the data and do anything they want with it and there is nothing anyone can do about it. If it were otherwise, no one would be able to broadcast the temperature without permission from the weather office. How well would that system work?

    You can't copyright facts, but there are copyright-style laws covering a collection of facts organised into a database. That said, creative commons probably isn't the right licence for the same reason it wasn't the right licence for open street map (who have now migrated to a different permissive licence designed for databases of facts).

  13. Re:Criminals and retarded monkeys by lisaparratt · · Score: 3, Funny

    I think you've answered your own question.

  14. Re:Why on earth did they waste time and money by TheMathemagician · · Score: 2

    It's because bureaucrats everywhere have a visceral belief that THEY own the data and it should never be released to the public without the maximum of foot-dragging, time-wasting and hoop-jumping.

  15. Re:Government is here to help by drsmithy · · Score: 3, Insightful

    And we all know all forms of central planning always fail at everything. That's why centrally planned, hierarchical organisations like religions, corporations and military forces have never been successful at anything.