Slashdot Mirror

When you draw the internet while diagramming on a whiteboard, what do you draw? Most people draw a cloud as an abstraction for the stuff "out there". I've never seen anyone draw a spiderweb when drawing a diagram that includes the internet.

I've tried to explain the cloud to slashdotters before. If you don't like the word cloud, you don't have to call it that. A less buzzword-y and perhaps more accurate term would be "utility computing". Turn on the faucet and out comes your data.

Here are some good papers about Google's technologies:
Sawzall (simplified scripting on top of MapReduce)
MapReduce (Google's massively parallel system based on the concept found in functional programming. The system takes care of managing jobs, parallelism, and fault tolerance, allowing engineers to more quickly produce code.)
GFS (Google's File System)
Google's Cluster (An older paper describing how Google's search cluster works. The cluster described in this paper is a few generations out of date.)
BigTable (Google's semi-structured database. There haven't been any papers released, but this is my write up based on a talk given in October 2005.)

And here are some videos:
The Google Linux Cluster. This is an older video where Urs Hoelzle talks about their system and focuses more on the hardware side of things.
Google: A Behind-the-scenes Look. Jeff Dean gives an overview of most of the technologies mentioned in papers above. I thought the demonstration of Google's internal word clustering was interesting (and funny).
Perspectives on the Information Industry. This is a technology-light (IIRC) talk given by Eric Schmidt.
BigTable: A Distributed Structured Storage System. The talk from which I created my BigTables notes (above).

Andrew

Here are the links to the two SRB cameras (hopefully this works):

right_aft_srb_camera.mov.torrent
right_forward_srb_camera.mov.torrent

There is something wrong with my MIME types, so save the file as and, if necessary, rename to .torrent.

Here are the links to the two SRB cameras (hopefully this works):

right_aft_srb_camera.mov.torrent
right_forward_srb_camera.mov.torrent

There is something wrong with my MIME types, so save the file as and, if necessary, rename to .torrent.

Google is a a company that afaik writes everything in python, on linux boxes.

Hardly. Remember the story just a couple of days ago about which operating system and browser different companies' employees use? Google employees mostly use Windows! (Insert huge disclaimer about the unreliability of these stats here). Most of Google's software is aimed at Windows users. Native Linux support often comes much later.

As for writing 'everything in Python'? Python is a great language but I doubt if all that much of their code is written Python. A lot of their work is C/C++/Java/Javascript/Ajax/etc...

I know that on the Python homepage it says:

"Python has been an important part of Google since the beginning, and remains so as the system grows and evolves. "

-- Peter Norvig, Google

I would actually be interested to know what products (if any) they have that are powered mostly or entirely by Python. Does anyone know?

Based on the stats this article is based on they're mostly windows (72 out of 74 are Windows): http://andrewhitchcock.org/companystats/ (someone linked to this above).

I'd guess that shows how accurate these stats are.

According to http://andrewhitchcock.org/ (which seems to be the source) the search for "Britney Spears" is well above the search for "porn", and I just dont buy it.

Oh wait, now that I think about it he did not include the search for "lesbians caught in the act while I was walking my ferret". Which Specifically does not include the word "porn". I begin to see the issue...

Anyhow, this Andrew guy has articles dateing back to 2001. Its mostly trivial stuff relating to his life until recently. And then it relates to google. So my guess is that people who do a search on google sift through the pages of results and end up on his site. The way I figure it you pretty much have to be interested in google or Andrew before you could wind up there. So his statistics are probably correct. However, the test is screwed to begin with.
So in the end there are two flaws. The fact that Nick Farrell does not seem to care about what he writes as long as its antagonistic (I use this one sample only as evidence) and the second flaw is that we are talking about it.

Besides, I didnt see my searches for "lesbians" anywhere in the statistics, which doesn't seem quite right.

Additionally, Google is allowed to crawl more of my site than MSN or Yahoo. However, based on the search terms used, I don't think this made that big of a difference (the majority of visitors from all these companies came looking for my BigTable article, which ranks highly in the big three search engines).

This is a moot point, because The Inquirer should have done their job and mentioned Philipp Lenssen's blog post where he uses a much larger sample set and gets similar results.

Additionally, Google is allowed to crawl more of my site than MSN or Yahoo. However, based on the search terms used, I don't think this made that big of a difference (the majority of visitors from all these companies came looking for my BigTable article, which ranks highly in the big three search engines).

This is a moot point, because The Inquirer should have done their job and mentioned Philipp Lenssen's blog post where he uses a much larger sample set and gets similar results.

My original sample was very small (maybe 20,000 hits in total, with only some of them being from the companies in question). However, Philipp Lenssen over at Google Blogoscoped took a much larger sample and got similar results.

Of course, when you get your news from the fourth tier of information (one not particularly known for respectability in the first place), you are more likely to get some misinformation. In this case: my website->Google Blogoscoped (where more content was added)->Tech Web->The Inquirer.

Andrew

PS: This has gotten way more coverage than I ever imagined. First it was dugg and now slashdotted... wow.

If you had read TFA you would have found the link to the real article which links to the original source, and found this:

http://andrewhitchcock.org/companystats/

Firefox has just under 10% from Microsoft, and about 80% from Google.

If you RTFA and follow the links all the way to the data you will see that the conclusion is based on a whopping total of 105 hits from Microsoft, over 7 months. That is one hit every 2 days folks.

The disclaimer, "Note: My website gets limited traffic, so this data has a large margin for error. The data was taken from my website traffic between 2005-11-14 and 2006-06-17." doesn't even come close to cover it.

Google probably keeps all the data created via its services in a form similar to the Semantic Web -- just a proprietary version of it.

I believe Google stores most of its data in BigTable, their homegrown database system. BigTable seems to be built on the philosophy of the Semantic Web.

However, each service (Maps, Reader, Base, etc.) runs its own BigTable cluster, so it might be a little difficult to share information between services.

Only outlaws will have

#!usr/bin/perl

tshirts http://andrewhitchcock.org/index.pl?page=perl

Hmm, really? I e-mailed them a problem with some of the videos on the iTunes Music Store and they gave me this response. I told them that the aspect ratio of some of their TV shows is wrong (noticeably so), and after two e-mails where they ignored what I was saying, they finally responded by saying they had an engineer look at it and found nothing wrong. Basically, they told me it was in my head. I was upset that they dismissed me, so I made a page which clearly shows the problem.

I was about to buy the 16 episode plan, but I previewed the episode and noticed that both TDS and TCR both have problems in the encoding. The videos are are 320x208 resolution, which is horribly non-standard and causes the stretching of both videos (well, more accurately, squishing, but they have the same end effect), making everyone look fat. I have a blog post with picture comparing Jon Stewart's head in the video with how it should look.

I've been up all night writing a widget. Let me tell you this: the development environment is a pain in the arse. Using their developer widget, it gives you the option of not caching the various widgets. I tried using this tool to not cache mine (so I could reload easily and see changes to the code), but it didn't work reliably. Whenever you moved the widget around to a different spot, it would go to a new revision of the code... and it was almost never the latest revision. This was so frustrating. What I ended up doing was renaming my widget every revision (also a huge pain in the arse). I ended up renaming it over 40 times during the course of the night. Then you have to add the widget back to your personalized home and go through all the steps again. Blast. I'm not bitter :).

If you want to see the culmination of my night's work, plug this into the widget manager: http://andrewhitchcock.org/musicmobs/w.xml

My widget pulls data from musicmobs. You can look at similar artists or find interesting playlists. If you visit that page, you can upload your iTunes library to make the recommendations more accurate, and it gives you the ability to upload your own playlists (which then become visible in the widget). Check it out!

Andrew

Netscape 3.0? Too high tech for me. My friend reads Gmail with NS 2.0.

Yes, that is a 64K binary being emulated for PPC in Classic (IIRC).

I saved some pictures. They are not of Alaska (which was amazingly distorted, but they were sufficiently distorted to make it noticeable. It wasn't really a mistake, it was just the projection they chose, but I imagine they got lots of complaints and changed it. But yeah, the satellite imagery hasn't ben reprojected yet.

If you interested, I've tried to make a full list of changes on my website. That website has side by side picture comparisons of the old and new. Some things they did as well: darkened minor streets, changed the projection, added one-way arrows, added some major building landmarks, and changed the size of the images being sent.

My iChat/AOL IM icon is my head in a bald cap and painted blue. Before the WWDC keynote I joined the AIM chat room AppleInsider to talk about it. I thought nothing of it, because I use that icon a lot, but then this conversation happened and it amused me:

chat person 1: who teh fk is the guy with the blue face
chat person 1: thats scary
chat person 2: thats an intel man
chat person 1: how apt

Hehe, I found it amusing. I didn't realize it at the time, but I was a harbinger of things to come.

Here is my quick attempt. Some of the ones out there are really neat, especially the one has three screens overlapping on flickr. I thought it would be fun to do one of these, and then GIMP something into the image. Have my PowerBook looking out the window, and then GIMP a picture of the Eiffel Tower off in the distance. I'm too lazy, but someone else could do it :)

This was the first browser I thought about checking in. Here you can see, Gmail running in Netscape 4.7.

It seems Google has been linking to answers.com for a little while for select users. Over the past few weeks, I was occasionally sent there. It appears to have started in January, judging from the Alexa data.

They are still experimenting with putting Google Image search on the front page. I experienced this last night when Google showed me this page.

Finally, I don't really like the new Google invite text box, I think it makes the interface a little ugly. And why must they switch it every month? I swear, that is like the dozenth place I've seen it.

Hehe, I've noticed a few myself, and fortunately I often have a camera around when it happens. It isn't always BSoDs, though, there are often other error messages that are still funny.

A whole bunch of BSoDs on some kiosks at Key Arena. This is particularly funny, because I occasionally see one of the three most famous people from Microsoft there (Gates, Allen, and Ballmer).
Unable to load traffic data from a website on a public computer (running IE)
Error message on a display in an elevator in Vegas

Andrew

Hehe, I've noticed a few myself, and fortunately I often have a camera around when it happens. It isn't always BSoDs, though, there are often other error messages that are still funny.

A whole bunch of BSoDs on some kiosks at Key Arena. This is particularly funny, because I occasionally see one of the three most famous people from Microsoft there (Gates, Allen, and Ballmer).
Unable to load traffic data from a website on a public computer (running IE)
Error message on a display in an elevator in Vegas

Andrew

Hehe, I've noticed a few myself, and fortunately I often have a camera around when it happens. It isn't always BSoDs, though, there are often other error messages that are still funny.

A whole bunch of BSoDs on some kiosks at Key Arena. This is particularly funny, because I occasionally see one of the three most famous people from Microsoft there (Gates, Allen, and Ballmer).
Unable to load traffic data from a website on a public computer (running IE)
Error message on a display in an elevator in Vegas

Andrew

Hehe, I've noticed a few myself, and fortunately I often have a camera around when it happens. It isn't always BSoDs, though, there are often other error messages that are still funny.

A whole bunch of BSoDs on some kiosks at Key Arena. This is particularly funny, because I occasionally see one of the three most famous people from Microsoft there (Gates, Allen, and Ballmer).
Unable to load traffic data from a website on a public computer (running IE)
Error message on a display in an elevator in Vegas

Andrew

Hehe, I've noticed a few myself, and fortunately I often have a camera around when it happens. It isn't always BSoDs, though, there are often other error messages that are still funny.

A whole bunch of BSoDs on some kiosks at Key Arena. This is particularly funny, because I occasionally see one of the three most famous people from Microsoft there (Gates, Allen, and Ballmer).
Unable to load traffic data from a website on a public computer (running IE)
Error message on a display in an elevator in Vegas

Andrew

This is one of the reasons I rarely go to movie theatres these days.

A group of my friends convinced me to go see a movie at the Bellevue Galleria. I really didn't want to go, but they all wanted me to. We arrive and sit in the theatre. While you are sitting down waiting for it to begin (where they used to have semi-unobtrusive advertisements (from slide projectors)), they have this thing called "The 20." It is the lamest amount of crap ever. It is full motion video and like featurettes. These are the things that mask themselves as behind the scenes, but are really just poorly hided advertisements for the movie. I never watch this on DVDs anymore. Once the movie is set to begin, they play commercials. This wouldn't be too bad, except they are straight from TV, so the sound and video both suck and they are so damn annoying. After the minutes of commercials, they get into trailers. Trailers aren't one minute each, they are three to four minutes. These are theatrical trailers, not teaser trailers or the ones shown on TV. Finally, by the time you have forgotten what movie you are about to see, and ready to leave because you have been sitting down for nearly an hour with these videos playing, the movie actually begins. Fortunately for me, the movie was interrupted by a fire alarm, giving me an excuse to leave. I didn't need to hear stupid middle schoolers shout at the screen how hot the 19th century era clothing made the actress look. Stupid fucking kids (hell, I'm 18 myself and I find these pricks annoying). I used to go to movie theatres all the time, but now that I value my time more and don't want to put up with this crap, I never go to movies anymore. I can't believe anyone would put up with nearly 50 minutes of ads before a movie, it is unbelievable. Yet these same people run back again and again. My friend even went to the worst theatre in the area to see Lord of the Rings, just because it was close. Ahhh! I want to scream.

Now compare this to the best theatre I have ever been to, the Cinerama. First of all, it has the biggest, clearest, most flicker free projection I have seen (outside of IMAX, which is a separate category. Second, they treat you like movie fans, not consumers. Before the movie begins, the screen has a curtain in front of it. At the most, they sometimes play the movie soundtrack in the background. Once it is set to begin, they play only trailers, no TV commercials. Although, this isn't guaranteed. A few times I have been there, they didn't show trailers at all! They skip right to the movie. That is class. The Cinerama is a class act (thanks Paul Allen!). There are only two complaints I have with them: they got rid of the free refills on popcorn and sometimes there digital projector is broken (like when I went to see Star Wars). If my biggest complaint is not seeing the movie digitally (and instead having to settle for the best film projection system I have ever seen), well, then I think I am set. This movie costs a few dollars more (10 dollars for a standard ticket), but for how little I see movies in the theatres, I say a few extra bucks is definitely better than a few hours of stress.

Andrew

All the articles on /. mentioning going after the server operators say they are going after BitTorrent trackers. After reading TFA, I didn't see any mention of going after tracker operators, instead they are going after the websites that host the .torrent files. Please be more clear about this in future. I host a tracker on my website that has been used primarily for legal torrents (I can't think of any illegal torrents that it has hosted, but I don't watch too carefully). I hosted the Maestro torrents and someone unbeknownst to me hosted a NASA video on there. There is no way I can be held responsible for what people use my tracker for, I can only ever see the MD5. This article says that the MPAA is going after people hosting the .torrent files. I host some of them, but they are all illegal. I don't host any illegal .torrent files on my server, even if someone did set me as a tracker for their illegal file. I am in the clear. These articles need to make that distinction, it sounds like it was written by someone who doesn't know how BT works.

Andrew

PS: My BT server is http://andrewhitchcock.org:6969/.

Ah, they grow up so fast!

You're lucky, when your little penguin is ready for junior/senior prom, at least you won't need to rent a tuxedo.


rm -rf /bin/laden http://andrewhitchcock.org/index.pl?page=binladen

I've done a little bit of my own stress testing. However, I've done it a little bit different. I wanted to see how Gmail handled huge conversations. I e-mailed my brother and we spammed each other back and forth in the same thread, seeing if we could influence the Ads. After a while we started adding more people to the conversation (our current test thread has nine people). We started out by hitting Reply All and saving the quotes from the previous e-mail. It became a huge list of >>> near the bottom and eventually Gmail clipped the messages. After a few hundred replies, opening the thread became slower and slower. When it reached 426 replies, it took me a week to finally get into it. With that I made one last reply and closed the thread. Hey, just out of curiosity, I opened the thread now and it loaded pretty easily. I wonder if they have optimized their behind the scenes engine to make it faster for large conversations. Maybe I'll continue the thread. Also, if you want to be part of the new test thread, just send an e-mail to adpowers@gmail.com.

Anyway, here is my Gmail stress test.

Also, you'll notice I have a few mailing lists on the side. I only read the Freenet one, but I subscribed to the Linux Kernel list and some others because I know them to be high traffic. Gmail is pretty impressive and they seem to be optimizing it even more.

http://andrewhitchcock.org/images/gmail-slashdotte d.png

http://andrewhitchcock.org/images/gmail-slashdotte d.png

I posted my address offering invites to people on slashdot yesterday. It was a third level comment (a reply to a reply of a comment or 'grandchild' post). Even without being modded up, I got a page full of requests. Not nearly as bad as you, though. I took a screenshot after I got back to my computer.

http://andrewhitchcock.org/images/gmail-slashdotte d.png

Hey guys. Sorry, but I'm out now. If you e-mail me, I can add you to the queue, but there are already 40 people in line.

Just to see what a slashdotted e-mail address looks like, click to see a screenshot of my gmail inbox. Hehe, ouch.

Andrew

Exactly. I go to Oregon somewhat frequently and I am always amused seeing the Intel codenames all over the place. I have some pictures from my most recent trip to Portland.

Yamhill Street
Advertisement for rafting on the Willamette river

As for Cascades, give me a break. As others have said, that should have been obvious if you know the names of all those rivers.

Exactly. I go to Oregon somewhat frequently and I am always amused seeing the Intel codenames all over the place. I have some pictures from my most recent trip to Portland.

Yamhill Street
Advertisement for rafting on the Willamette river

As for Cascades, give me a break. As others have said, that should have been obvious if you know the names of all those rivers.

Haha, I finally managed to find a place to get the DVD zip. Here is the torrent for it. If you have downloaded part of it or all of it, then you can help seed. Too bad we couldn't have had this when the intial slashdotting took out the server.

Hahaha, I remember that. Thanks for mentioning the name. I was talking about the Google thing on my website and I mentioned how this was something reminiscent of the dot-com days. I used your service for a while, I remember uploading a gig or two to the servers by leaving the connection on overnight. I believe I was on dialup at the time, so that was a big feat. In the end, I never really used it that much.

That 'compression' you speak of sounds interesting, care to reveal details?

It is probably a good thing you didn't get noticed by the RIAA (or did you?). If they had a hissy fit of the MP3.com streaming thing (which I also tried), they would probably be up in arms against this too.

Andrew

Wow, that went down fast. That'll teach you to post a image gallery directly to /. :).

Anyway, I grabbed what I could and I figure it is better than nothing. It is mostly thumbnails, but I managed to get 1.5 higher resolution images.

Here is my very partial mirror

Here are the mirror links for the program and the data update in case telestra.org goes down again. There is nothing posted there besides this list anyway.

Maestro for Windows XP/2000/Me/98

Download from NASA Download from Freecache Download from USF FTP (Florida) (Internet II - university students start here) Download from LibertyOutreach Download from KNCL FTP (Texas) Download from Lakewebs (Oklahoma) Download from NJIT (New Jersey) Download from UALR (Arkansas) (Internet II - university students start here) Download from Emporia State Univ. (Kansas) (Internet II - university students start here) Download from TU-Budapest (Hungary) Download from TU-Berlin (Germany) Download via BitTorrent (what's this?) Download via ed2k (what's this?)

Maestro for Mac (requires Java3D)

Download from NASA Download from FreeCache Download from USF FTP (Florida) (Internet II - university students start here) Download from KNCL FTP (Texas) Download from Lakewebs (Oklahoma) Download from NJIT (New Jersey) Download from UALR (Arkansas) (Internet II - university students start here) Download from Emporia State Univ. (Kansas) (Internet II - university students start here) Download from TU-Budapest (Hungary) Download from TU-Berlin (Germany) Download via ed2k (what's this?)

Maestro for Linux

Download from NASA Download from Freecache Download from USF FTP (Florida) (Internet II - university students start here) Download from KNCL FTP (Texas) Download from Lakewebs (Oklahoma) Download from NJIT (New Jersey)

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org

Get your your bittorrent files here:
Maestro for Windows & Mars Dataset #1

Maestro for Linux & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Solaris & Mars Dataset #1
(tar -xzvf dataset immediately above your "JPL" directory)

Maestro for Mac & Mars Dataset #1
(Requires Java3D)

Maestro User's Guide (pdf)
BitTorrent stats

Provide feedback to these folks: maestro [at] telascience [dot] org