Slashdot Mirror
Australian Bureau of Statistics Doesn't Like Direct Downloads of Census Data
Bismillah writes "The ABS has released the census data for the country under a Creative Commons license, but instead of making it easy to get, they've put in Javascript to obfuscate file paths and more. All commented in the source code of course."
At first glance, it's an attempt to get people to pay $250 for a DVD with the data instead.
Sounds like an excellent use for Bit Torrent? I assume someone will download the whole dataset and make a torrent out of it before long....
You can tell how powerful someone is by the magnitude of the crime they can commit and be able to get away with.
Link to the torrent of the census data from the article:
http://blog.angrygoats.net/2013/04/12/2011-australian-census-release-3/
Since the data is available for free (obfuscated or not) and was released under a CC license, technically this should all be considered legal, right? Not that it should be necessary of course.
Apparently being anti-Steam is grounds for insults, even if there's basis. I shall learn to keep my mouth shut.
... descended from criminals and retarded monkeys.
No, we're not all descended from the English let alone Americans.
I saw the title text and thought the census data was being provided through bittorrent. A few games including the popular World of Warcraft distribute their updates through the protocol, seeing it adopted in other areas to reduce the bandwidth costs seems like a good idea.
Wouldn't something the the Open Database license make more sense? I mean, census data isn't exactly a creative work. OpenStreetMap made the change when they realised something similar about map data.
I think that is really funny! They declare the data free and then make an inept attempt to force people to pay for it. It's almost as bad as copyrighting public laws.
Thanks a lot Slashdot. Now I have a sudden urge to know precisely how many married couples with the husbands between the ages of 30 and 32 inclusive have children in Queensland, and what the genders of and ages of the children are.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Remember that in many countries works of the national government are not automatically in the public domain like they are in the US. In Commonwealth realms this is called Crown Copyright.
https://en.wikipedia.org/wiki/Crown_copyright/
Isn't this data already paid for with people's dime?
Your dataz, you can't hide them.
Anyway, they're pulling the same thing here. They want someone to gather up their data and present it in a nice package for free. The best way to do that is to drop an ineptly-presented steaming pile of crap on the internets. There'll probably be 15 open source projects to slice and dice it on github by the weekend, and it didn't cost the Australian government a dime! It's brilliant!
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
Only effective as long as there exists nobody who is interested in the data, able to crack the obfuscation, and willing to distribute it.
(... and willing to overlook that Aaron Schwartz died for something quite similar, I suppose.)
Magnet link for the lazy:
magnet:?xt=urn:btih:EE2DEAA27287952089AE257EC8B009E382598239&dn=2011%20Datapacks%20BCP_IP_TSP_PEP_ECP_WPP_Release%203.tar.xz&tr=udp%3a%2f%2ftracker.publicbt.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce
or
http://mgnet.me/DTyE
or torrent: http://grond.angrygoats.net/torrent/2011%20Datapacks%20BCP_IP_TSP_PEP_ECP_WPP_Release%203.tar.xz.torrent
No, Br'er Rabbit, don't tell the world we are hiding our data, they might get copies and make sure the whole world has access to it, no, don't do that Br'er Rabbit.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
No, no, you're right, the US government is always transparent and forthcoming with information
(yes it was sarcasm)
to conspiracy that which can be explained by incompetence. The real goal could of had nothing to do with "hiding" the data.
Crikey, they're a bunch of naughty little critters for doing that!
I was promised a flying car. Where is my flying car?
"Flight of the Conchords" aren't funny in the slightest. I never understood how they got so popular in the US.
Nice theory, until they threaten you with fines and jail time. (You're legally required to fill out census forms, truthfully even.) And then the fuckers have the hide charge money for your data that you had to supply to them for free.
"The ABS is constantly looking at ways it can simplify the website and enhance the user experience," iTnews was told via email.
Stop hosting it on Lotus Domino servers and you won't have to worry about how many people download the damned data.
What is the point of putting a creative commons license on data that is not copyrightable. Anyone can take the data and do anything they want with it and there is nothing anyone can do about it. If it were otherwise, no one would be able to broadcast the temperature without permission from the weather office. How well would that system work?
Actually, he was facing over 50 years in prison.
From TFA:
"The ABS is constantly looking at ways it can simplify the website and enhance the user experience," iTnews was told via email.
Stop hosting it on Lotus Domino servers and you won't have to worry about how many people download the damned data.
U crazy? After millions paid for the Lotus servers and zillions in staff training (or... was it train stuffing? in the context, the results would be the same), you want the IT dept head to... well, lose her/his head?
Questions raise, answers kill. Raise questions to stay alive.
From the code:
// Also, generate a random number, which we append to the URL, to make it appear as if a complex
//key is required. This is a pathetic attempt to discourage someone from downloading the ZIPs
//directly (ie. without having to login), if they deduce the URL pattern.
Translation:
Coder: "Here's the census web application."
PHB: "Great. But wait..I can just type in these other names and download them really easily! People will hack us and we'll be out possibly a COUPLE THOUSAND DOLLARS! "
Coder: "It is Creative Commons data, so of course we added no protection. Changing that now will be a massive rewrite and take months."
PHB: "So let's add some random numbers to the end so it looks really complex and people can't guess how to get in."
Coder: "But they still will eventually see the links because they do actually have to download it, so this is not really doing anything."
PHB: "Psh, no one is smart enough to figure that out. I read about this GUID things and they're really hard to guess. It will work. This is your job today."
Coder "..Ok, fine. I'll do it exactly the way you asked."
Speak before you think
Easiest way to get the javascript obfuscated data is to write a simple perl script using wget plus spidermonkey to run the javascript. News Ltd used to try the same thing with their online tv guide, but it was trivial to circumvent.
I don't understand why certain organizations want to publish their data yet not publish it at the same time. They cannot have it both ways. If they publish it then they are allowing everyone to have it.
Have gnu, will travel.
You almost had me, but the phrase "simple Perl script" is a dead giveaway.
... descended from criminals and retarded monkeys.
No, we're not all descended from the English let alone Americans.
Az cornvicted monktard, am mad you cumpare wif Americalfs or Englushes.
Why on earth did they waste time and money obfusticating something that is licensed on the creative commons. All someone has to do is either buy the DVD or reverse engineer the site once and they can put it up on their own website
I think you've answered your own question.
In Commonwealth realms this is called Crown Copyright.
Did someone say apply the CC licence and not specify which?
More reason to argue for a republic.
wow! I found the "hard to find" register page within 15 seconds. I must be some kind of eentelectual sooper-wonder... or portuguese... wait! Yes, portuguese...
I gather this is data being published by a government agency. As all agencies are funded by taxpayers, all records -- with exceptions for security and privacy -- should already be open to the public. Creative Commons seems inappropriate here; the correct notice should be "Public Domain", or is Aussie law different in this respect from US law?
"We receive as friendly that which agrees with, we resist with dislike that which opposes us" - Faraday
I have never filled out a census in my life, I hear all kinds of threats of fines and jail time, yet when I laugh and tell them to "fill their boots" nothing happens.
Javascript? who the f*** still uses that? And is that even considered a layer of security? Whoever is in charge of "security" there should move beyond Windows 98.
As census collectors we're told to say that. Having done it several times now, the worst part is morons getting on their high horse about it. We don't care what reasons you have for not doing the census, they're probably illogical and tin-foil hat anyway. There is a process we have to go to (to ensure everyone is accounted for as far as humanly possible) and report to our supervisor, so that's what we do. Telling you there is a fine and handing out the related material is part of that process.
Besides, I fail to see how completing a census implies central planning works. The data is used for more than just that, and by the time any personally identifying information is made public, you will be long dead buried and forgotten.
And we all know all forms of central planning always fail at everything. That's why centrally planned, hierarchical organisations like religions, corporations and military forces have never been successful at anything.
Was certain I had read those comments before. Yep
Everybody uses broad generalizations.
Yeah but white upper middle class techno-geeks really are pussies, so they get scared that easily into believing it's true.
The Australian government is excellent at selling to the public products that the public have paid to produce. In the US (at least in theory) products of the US government are in the public domain and not eligible for copyright. I know, there are tons of things the US government produces that are exempt from this. I know this is a simplistic view, but Australia should not be selling things to people that they have already bought with their tax dollars.
The real goal could of had nothing to do with "hiding" the data.
If you could read, you might have seen that phrase spelled "could've", which is a contraction of the phrase "could have". Instead, you heard it spoken out loud and parsed it incorrectly as "could of". What the !@#$ does "could of" even mean?!?
You're welcome.
"Tongue tied and twisted, just an Earth bound misfit
Americans are significantly more intelligent than most other nationalities.
How is it that census data is creative enough to merit a copyright?
I explicitly release the above into the public domain.
same way a phone book is, I guess.
It's the http://creativecommons.org/licenses/by/2.5/au/ licence (attribution 2.5 australia).
You are free:
to copy, distribute, display, and perform the work
to make derivative works
to make commercial use of the work
Under the following conditions:
Attribution — You must give the original author credit.
With the understanding that:
Waiver — Any of the above conditions can be waived if you get permission from the copyright holder.
Public Domain — Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
Other Rights — In no way are any of the following rights affected by the license:
Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations;
The author's moral rights;
Rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights.
Notice — For any reuse or distribution, you must make clear to others the licence terms of this work.
ie. as long as you say it's ABS data you're peachy
What's a DVD?