Slashdot Mirror
Making Sense of Census Data With Google Earth
mikemuch writes "Imran Haque has developed a mashup of Google Earth with data from the U.S. Census Bureau, called gCensus. The app uses the XML format known as KML (Keyhole Markup Language), which can create shapes and colors on the maps displayed by GE. Haque had to build custom code libraries (which he's made available as open source) that could generate KML for the project. He also had to extract the relevant data from the highly counter-intuitive Census Bureau files and store them in a database that could handle geographic data. gCensus lets you do stuff like create colorful overlays on maps showing population ages, race, and family size distributions."
Imagine what applications you cook up with this .....
Perhaps there's a way to fuse the presentation possibilities with Gapminder?
The Census Bureau has meticulously documented its data files--in a 635 page PDF file.
Wow, now thats a file format.
Libertarian Leaning Political Discussion Forum.
At least yell 'First Post!'
The Census is equally important as voting. Special interest groups representing minority organizations work closely with state and local governments when they draw up political districts. What an awesome tool to hold those officials accountable and give other groups a voice - open access for everyone.
I hope, when they die, cartoon characters have to answer for their sins.
A lot of this has already been done, although the site hasn't updated since google changed their API's: www.gcensus.com.
don't most mayors manage their cities in EA/Maxis SimCity? Those mayors have had that data available since the mid nineties.
We at slashdot are scientists, specialists and kernel hackers. Your FUD will be found out.
Just remember kids, Google Earth is not the first to do anything orignal, they just have a bigger PR engine.
I tend to use the Free AnalyGIS mash-up for Google Maps http://demo.analygis.com/google/default.htm For those who don't know much about AnalgGIS http://www.analygis.com/ I suggest checking out their web site. It is a pay service but the best of the bunch IMHO.
We are currently down while we upgrade to Google Maps API version 2. We hope to have this finished soon, thank you.
I wonder if when using this mash-up some areas will start showing up as giant sinkholes?
I'd like to see maps of the disparities between exit-poll and actual vote tally numbers, one map per election. This will make it possible, and not just "possible": once someone has putatively done the work, it'll be easy to check, because the raw data are available from trustworthy sources (cue cynicism in 3) so anyone can redo the map to check for distortions.
This makes whole classes of questions easier for mere mortals to answer, and simultaneously makes their answers easier for mere mortals to understand. It's huge.
As always, all IMO. Insert "I think" everywhere grammatically possible.
When I go to work, I don't make something meant to absorb butter and gravy.
I hope that google is bidding to support automation and data gathering and analysis of the 2010 census. I'm sure there must be contracts going out now to support the process. So far, I think that google would be the most competent at managing the volume of data yet keeping the granularity.
Also, with google support. I think that census data can be published far more quickly than the 2000 census was.
I seriously doubt that, though
I fear that this will enable unscrupulous realtors, etc. to make use of income data and geographic positioning to clearly delineate which areas are "bad" and therefore make further attempts to exploit those areas.
stuff |
Given that this uses Google Earth, a 3d tool (unlike google maps) it might be useful to be able to do things like play with the elevation to represent different types of data (doable, I'm sure, if not that useful) or alternatively (and more practically) plot bar graphs on the area which they represent- height would represent one value, colour another- you could even segment the graphs vertically. I did a quick previs of what this might look like if anyone is interested
When the posters fear their moderators, there is tyranny; when the moderators fears the posters, there is liberty.
interesting: you could accurately target each and every neighborhood to cause max damage with a dispersed attack of nearly any sort. or target just the wealthy or the white. look what a weapon you've made for terrorists you idiot. theres a reason the census stuff is hard to access.
"The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn pleases."
-sir josiah stamp, inland revenue department of England, 1896-1919
stuff |
Thanks a lot, jerk. You've ruined everything.
That's going to dilute my filling out the census for political motivations.
Read this book and understand how useful GIS and statistics are.
but can it color bewb^H^H^H^Hgender.
Guess you didn't realize that Microsoft's OOXML (the one they want us to use instead of the ~600 page spec of ODF) is more like 6,000 pages.
And they still don't describe many of the Microsoft formats they include by reference, like WMFs, or the screwball hacks they use to avoid proper backwards compatibility, like those tags saying to format the document the way ancient versions of WordPerfect or MS Office for Macs did...
... it's just very complicated. The amount of information recorded is mind-boggling and not readily accessible mainly due to its size. Also, many of the types of information recorded need to be handled in certain ways - I've seen many beginning GIS students bungle up census data by misinterpreting Hispanic data and unemployment data because they don't understand how it is tabulated.
what that? Is that still alive or is it again? How is race determined in the US, do they still have the "pencil test" for curly hair?
From his site, it looks like he's entering data by hand, and hence it's time consuming so he's got very little of the census data up.
Seems like it shouldn't be too much of a problem to have something spider the census site, rip the pdfs off the site, & parse the data out of those pdfs then enter it into whatever format he's using.
Almost certainly less time consuming to code something to do that than actually manually entering everything by hand.
-
He's also short on space... once again google could come to the rescue, with open source apps you can mount gmail accounts as drives because their api's pretty open. Simply make some public accounts on gmail, store the user/pass within an app used to mount these as drives, and ditrubute that app to grab any data needed.
My company is looking for a passionate developer who wants to work on a very similar project with Google Earth and overlaying data that originates from a Drupal/MySQL install. This project could lead to a product that would leave the developer with residuals on future sales. The development work would be paid for, though on a flat/retainer basis. If you're interested, e-mail me. chris [at] chriswinnconsulting.com. THANKS!
The next comment I write will be ready soon, but subscribers can beat the rush and see it early!
Xix.
"Everything is adjustable, provided you have the right tools"
gCensus is a great website - it spurred me to create a similar project in Ruby. I'm writing Ruby code to pull Census boundary & data files and parse them into KML files for Google Earth display (height adds a great dimensionality) and Google Maps. I'm posting my progress here: http://censuskml.blogspot.com/.
You do realize, don't you, that *every time the political maps are redrawn* (i.e. every ten years) that gerrymandering is extensively used by every local state legislature? Gerrymandering is, after all, the process of redrawing the Districts so as to maximize partisan advantage. You don't need this tool to catch gerrymandering -- it's ubiquitous!
Rant aside, this app could certainly be a useful tool. The ideal -- nonpartisan -- political map would be drawn in such a way as to have the *sum of the sides* of *all* the districts to be a *minimum* while having the population of *each* district be within 1% of the average District population size for that state. (The Supreme Court has held that *some* variance in the population of districts is OK; I think that >1%, though, is *too much*.)
For example, say a state has 6.5 million people and 10 Congressional districts. Then each District must contain 650,000 +/- 1% (i.e. 1% here equals 6,500 people) and the sum of the sides of all the districts together must be a minimum. This leads to roundish districts and no possibility of gerrymandering (which, because of the torturous way districts are drawn, tends to *maximize* the length of the sides of districts).
The 'drawback' of this method, of course, is that only population -- and not historical voting patterns -- is taken into account, thus making it impossible to ensure that all the Democrats or Republicans or minorities are concentrated into just a few districts, as is done now by partisan legislatures. On the plus side, this would make more Congressional districts *unsafe* -- Congresspeople would actually have to get out there and *earn* their seats, instead of just sitting back and taking it for granted that their particular seat is safe because of the way that the Districts are drawn.
However, this scheme is unimplementable at the present time, due to the recent reauthorization of the Voting Rights Act (for the next 25 years), which actually *requires* gerrymandering when it comes to creating so-called "minority majority" Districts. (This ensures that minorities have adequate representation in Congress, rather than having their power diluted by being split among a number of other districts.)
For example, here in Michigan, blacks constitute about 12% of the population, but are highly concentrated in Detroit. If the above Voting Rights Act stricture was *not* in place, unscrupulous politicians could redraw the Congressional Districts in such a way (for example as long thin areas that had one end in Detroit and the other end outstate) as to ensure that blacks had *no* majority districts, and were a minority in *every* District that they were in -- clearly a violation of the Equal Protection clause of the Constitution.
So my logical plan for nonpartisan redistricting is -- unfortunately -- unlikely to come to fruition anytime soon -- for Congressional Districts anyway. However, since the case that was adjudicated by the Supreme Court *only* addressed issues of *Federal* redistricting, it might be possible for individual states to implement this plan as a way of making elections more contested and, hence, more democratic.
DNA is a Turing machine. You, however, being dynamic and emergent, are not.