Looking At Pretty Graphics Of Dot Com Demographics

chris_robison writes "A talented guy wrote a great example of the quality of talent that is going to waste in these economic "hard times". An unemployed friend of mine put together a kickass tool which lets you query a database of craigslist statistics and generate graphs comparing various things such as job postings and apartments for rent. Although the stats are geared toward San Francisco and the Bay Area, it does make for some interesting reading, even if you aren't from around there. Here's his explaination of what he did (included below)" "hey kids.

I'm unemployed and bored.

With all the talk about dotcom booms and people moving away and all these really general statements people are making; i was curious as to what's really going on now, what went on four years ago and what effect did the boom really have on this community.

So.. i wrote a script that went through the craigslist archive on egroups and tallied the number of postings each day for each category then i stuffed it in a database and made a grapher tool.

It's kinda neat cos you can basically see some somwhat hard data on what the san francisco bay area has been doing over the past four years. you can graph job postings from various industries against things like apartments for rent or housing wanted postings, also for sale postings and resumes.

Some of the interesting things I found is that the number of housing wanted postings seems to be slightly down recently but pretty much unaffected by the drop in jobs. perhaps people are always in a state of wanting to move to a city.

One thing to bear in mind, this data isn't one hundred percent accurate for a few reasons... people do repost their information multiple times in a given month, sometimes people post a number of avaliable apartments in a single posting and craigslist has become significantly more popular over the years in question. however, I do still think that it makes for a decent general indicator of trends in our community. (I'm considering doing some kind of normalizing based on the total volume of posts- I need to think about it some more)

(Unfortunately, personal listings aren't archived. I guess that's a good thing... but I guess it also would have been cool to graph activity on the personals categories against some of the categories that reflect the general state of the economy. (maybe when people aren't tied up in career they start to think about more important things, or maybe not...) )

Either way, the data is there, you can look at it for different time periods and categories. draw your own conclusions and have fun!

The url is here:

http://www.signal11.com/charts/chart-o-m atic.cgi

take it easy..

--adam "

Comments

Re:Not just silicon valley

[sinster]

The job crunch must be pretty close to global. I'm sure there are isolated areas with employment spurts, but in average, everyone is having trouble.

I make this claim based only on a report in the Wall Street Journal on Friday (7 Sept 2001) morning that claimed that almost every stock market in the world closed down at least 1%, and that if the trend continues through the next quarter, that this would "meet the classic definition of a recession". And not just a recession, but a global recession. Of course, the value of this claim is proportional to how much you trust the WSJ to accurately report financial news and how much you believe that the stock markets influence employment rates.

Sorry, I don't have a link: I get my WSJ fix through my palm pilot with AvantGo.

site slowness

[ajdub]

the slashdot effect is in effect. we underestimated the traffic it would generate and are moving the script to a bigger machine now.

if you can't get to it, please bear with us and try again in a few hours.

thanks again

--adam

Re:Chart doesn't mean much...

[ajdub]

Well...

The resume is on there at the suggestion of someone on craigslist who said that I should put it on there. I thought it was a bit slimey but at the same time I figured that it couldn't hurt.

As far as it getting posted to Slashdot, I didn't post it and I *do* know the guy who posted it but I did not ask him to do so, nor did he tell me before he did it. (and his text is a bit cheesy, no?) :)

As far as useful statistics and normalizing based on total number posted. I am planning on adding that stuff. For the second version, I'm planning on adding normalizing, moving averages by day (right now it's only tallied by month) and possibly some other stats. (I've recieved a few good ideas from a few people and I've got some emails out to some math folks about how I might be able to do some interesting forecast graphs)

--adam