Google's Academic TB Swap Project

eldavojohn writes "Google is transferring data the old fashioned way — by mailing hard drive arrays around to collect information and then sending copies to other institutions. All in the name of science & education. From the article, 'The program is currently informal and not open to the general public. Google either approaches bodies that it knows has large data sets or is contacted by scientists themselves. One of the largest data sets copied and distributed was data from the Hubble telescope — 120 terabytes of data. One terabyte is equivalent to 1,000 gigabytes. Mr. DiBona said he hoped that Google could one day make the data available to the public.'"

Comments

Should we be continuing this fallacy?

[garcia]

One terabyte is equivalent to 1,000 gigabytes.

Uhh, no it isn't. It's really 0.9765625 terabytes.

Re:Should we be continuing this fallacy?

[Cristofori42]

umm a terabyte is really 1 terabyte. Though 1 terabyte = 1024 gigabytes not 1000... but whatever.

Re:Should we be continuing this fallacy?

[wizzard2k]

From wikipedia:
(a contraction of tera binary byte) is a unit of information or computer storage, abbreviated TiB.

1 tebibyte = 240 bytes = 1,099,511,627,776 bytes = 1,024 gibibytes

The tebibyte is closely related to the terabyte, which can either be an (inaccurate) synonym for tebibyte, or refer to 1012 bytes = 1,000,000,000,000 bytes, depending on context.

Re:Should we be continuing this fallacy?

[garcia]

Thanks for pointing out that I should have been hitting Preview instead of getting First Post :)

1000GB = 0.9765625 TB, not 1TB.

Re:Should we be continuing this fallacy?

[wizzard2k]

Oops. /. doesnt do <sup>...
240 bytes = 2^40 bytes
1012 bytes = 10^12 bytes


My bad.

Re:Should we be continuing this fallacy?

[AchiIIe]

Nope, that's wrong

see: http://en.wikipedia.org/wiki/Tebibyte
* 1 Terabyte = 1000 Gigabyte
* 1 Tebibyte = 1024 Gibibyte

Re:Should we be continuing this fallacy?

[Anonymous Coward]

I'll die stubborn and proud before I start saying words like "gibibyte" out loud. I appreciate the want for accurate language so that our poor little brains don't have to infer what is meant by the context, but "they" could have picked a less stupid-sounding word.

Re:Should we be continuing this fallacy?

[Professor_UNIX]

* 1 Terabyte = 1000 Gigabyte * 1 Tebibyte = 1024 Gibibyte

Yea, yea, yea. And you also believe a hacker isn't someone who maliciously breaks into computer systems, it's just a curious innocent person right... crackers are the criminals! Give it up. The general public is never going to adopt "Tebibyte" into the language because terabyte sounds much more fucking cool.

Re:Should we be continuing this fallacy?

[wolff000]

WHO CARES?!? I have worked with mathematicians that did not squabble over these terms so why the hell are we?!? My mother who can hardly turn a computer on knows damn well that 1000 megabytes is roughly 1 gigabyte. Now lets get back to the topic. It seems Google would have some brilliant way to push a terabyte through the "tubes" instead of just mailing drives, how archaic.

Re:Should we be continuing this fallacy?

[Anonymous Coward]

Very dumb people have convinced themselves that the entire disk industry has been cheating them out of storage space for decades. They seem to believe that if disk manufacturers printed the "real" size on the box, the disk drives would somehow cost less.

Re:Should we be continuing this fallacy?

[vidarh]

Just because some morons decided to randomly invent new names and try to get people to change established usage doesn't give them any authority... Whenever I see someone pushing the *bibyte crap the only thing it achieves is to annoy me.

Re:Should we be continuing this fallacy?

[servoled]

Makes you wonder why some morons decided to do it in the first place when they tried to redefine kilo, mega, giga, etc... to be 2^x instead of 10^y.

Re:Should we be continuing this fallacy?

[guruevi]

I'm old and interested enough to know what REALLY happened through the history:

First, as taught in any school book and computer manual through history (see Apple, Amiga, Microsoft, Commodore): 1024 bytes = 1Kilobyte, 1024 Kilobyte = 1 Megabyte etc. because the computer could only calculate in exponents of 2 (1 and 0) and 20MB (20480 kilobyte) was about the largest size hard drive you could get.

A Kilobyte is 1024 (2^10) bytes. A Megabyte is 1024 Kilobytes or 1,048,576 bytes (2^20) and a Gigabyte is 1024 Megabytes or 1,073,741,824 bytes (2^30), some background data on what comes next can also be found here: http://www.actionfront.com/hdtech1.html

As soon as the hard drives began to hit 1000MB and the price/capacity got obfuscated as well as the need to count with cylinders, headers and sectors and calculate your partitions based on that (yes, we used to do that for minimal loss of capacity and performance), people started to forget about the extra 24MB on the hard disk, because now it was not 1000MB but 1 Gig and of course the manufacturers started using 1000MB as 1GB etc. And then we could avoid talking about Int13h for addressing hard disks, we got translating BIOS'es and everybody forgot that we used to have 1024 bytes in a kilobyte. As you know, RAM still uses the correct count because we're still not mainstream on >4G of RAM but I imagine manufacturers will start 'forgetting' too and labeling 1000MB as 1GB

Re:Should we be continuing this fallacy?

[General Wesc]

The 'morons' are the IEEE and their standards are recognised by the ISO. What you consider 'established usage' was not established enough to be accepted by the hard drive manufacturers, which was the primary place where the prefixes were in use. We're not going to go back to the inconsistent, ambiguous de facto standards just because the new ones annoy you. Do you also reject the redefinition of the foot to a standard length?

Re:Should we be continuing this fallacy?

[Anonymous Coward]

For people who "can hardly turn a computer on" it may be enough to know what a measurement "roughly" means. For people working with computers it isn't. Everybody please stop using those undefined pseudo-units.

Re:Should we be continuing this fallacy?

[Anpheus]

That's not the problem, the problem is, when you buy a X GB drive, you don't know what you're getting until you find the fine print. Some manufacturers provide different sizes of the same labeled drive, differing only in whether it's "1 GB = 1,000,000 KB" or "1 GB = 1,000,000,000 B"

So if you buy a set for RAID one day, the next day they may no longer stock the drive you need and your vital information is put at unnecessary risk because... what, because the hard drive manufacturers can't decide whether they want to screw you out of 7% (using 1 GB = 1 billion bytes) or 5% (using 1 GB = 1 million kilobytes, which they curiously agree on equaling 1024 billion bytes. What a coincidence that KB is 2^10, but GB is 10^9?)

Think about that for a moment before you lambast the argument for proper labeling of drives.

Re:Should we be continuing this fallacy?

[ResidntGeek]

Why do you insist on keeping the metric system down? Are you keeping Atlantis off the maps too?

Re:Should we be continuing this fallacy?

[HTH NE1]

The annoying part for me today is that flash memory is in powers of two (64 MB, 128 MB, 256 MB, 512 MB, etc.), be it for cameras or in USB thumbdrives, yet the units are metric, not binary (stating 1 MB == 1 million bytes on the packaging).

When I see a power of 2 next to the units, I expect the units to be in a power of 2 too.

Re:Should we be continuing this fallacy?

[cheater512]

Its called Context.

Re:Should we be continuing this fallacy?

[cheater512]

Not sure about you but I get pretty pissed off whenever I buy a new hard drive.

You pay for a 120gig hard drive but they give you a 108gig hard drive.
You pay for a 320gig hard drive but they give you a 298gig hard drive.
You pay for a 750gig hard drive but they give you a 698gig hard drive.

Mathematicians generally dont buy hard drives so they have no reason to complain about it.
Your mother also doesnt seem to be the type who buys drives.

Re:Should we be continuing this fallacy?

[cheater512]

Thats only because hard drive manufactures can boast bigger numbers by using decimal instead of base 2.

Re:Should we be continuing this fallacy?

[Anonymous Coward]

They seem to believe that if disk manufacturers printed the "real" size on the box, the disk drive's packaging would reflect its content, allowing an informed purchase.

 
Patched up right nice, didn't she?

Re:Should we be continuing this fallacy?

[jonbryce]

The other primary place where the prefixes are in use is RAM chips, and they do use 2^10 rather than 10^3.

Re:Should we be continuing this fallacy?

[Vihai]

So, why do you think the general public has to think that one kb is 1024 bytes instead of 1000?

Re:Should we be continuing this fallacy?

[toddestan]

Actually, they were first, and defined 1 kilobyte to be 1024 bytes, 1 megabyte to be 1024 kilobytes, etc. The people who redefined the terms were the harddrive manufacturers, in order to make their drives appear larger than they really were.

Re:Should we be continuing this fallacy?

[blackicye]

"So if you buy a set for RAID one day, the next day they may no longer stock the drive you need and your vital information is put at unnecessary risk because... what, because the hard drive manufacturers can't decide whether they want to screw you out of"

I'd sincerely hope that anyone who is considering building or who is running a RAID array would know what they were doing, at least to the point of getting drives of the same capacity (and preferably from the same manufacturer.)

If they don't whomever is setting up their RAID Array or expanding it should know better and advise them appropriately.

There are several misconceptions among RAID Neophytes that I've commonly heard:

- Being unaware that RAID 0 Striping offers no redundancy whatsoever, if a drive fails, you lose data from the entire array.

- Being unaware that the performance increase from running RAID 0 is usually not more than 5% - 10% and less when using an
    onboard non-dedicated RAID Controller. Such performance gains are usually offset in part by the CPU usage of the onboard
    controller.

- Running RAID 1 Mirroring is a good way to ensure data integrity over the long term, harddrives have moving parts, all
    devices comprising moving parts have a chance of failure and will eventually fail.

- Using the onboard RAID controller is good enough, when it actually is usually a really bad idea, especially with larger arrays,
    and especially if you actually care about the data you're storing.
    If the motherboard fails, you're going to have to usually find the exact revision of motherboard with the same firmware version,
    this it the primary reason I use a Dedicated RAID controller card.

- That 32-Bit Windows XP Pro has a limitation to 2 TB per "physical drive" You need to be running an OS compatible with GPT
    to address arrays of over 2 TB.

I contest that anyone who is likely to purchase multiple drives for the purpose of creating a RAID array should know better, or should most definately seek advice, or do their homework before they purchase.

Re:Should we be continuing this fallacy?

[wolff000]

"Mathematicians generally dont buy hard drives so they have no reason to complain about it."

They do if they build their own pieces like all the ones I worked with did. Most mathematicians I know are geeks and they like computers just as much as #s. So maybe you just aren't hanging out with the right mathematicians.

Re:Should we be continuing this fallacy?

[Anpheus]

Maxtor released two sets of 160GB drives, each with different capacities.

And they aren't the only ones to do it.

Problem

[Anonymous Coward]

I get sperm in my keyboard. Can't lick it out. Any suggestions ?

Re:Problem

[Anonymous Coward]

Talk about physical transfer of terabytes of DNA info!!

How do you get the sperm separated from the semenal fluid before it hits the keyboard?

[mod -10 troll ;-)>]

1TB = 1024 GB

[adityapk]

1 TB = 1024 GB, for gods sake

Re:1TB = 1024 GB

[91degrees]

Why?

Why is a Kilobyte 1024 bytes, if "Kilo" means 1000, both according to the SI and the greeks (Kilo is derived from khilioi). If 1 kg = 1000g, 1 kV = 1000V, 1 km = 1000m, why should hard disks break the pattern?

When we're talking about addressable computer memory, approximating the kilobyte to 1024 is a convenience, but since Terabyte gives such a huge error, and makes absolutely no sense for data transfer or disk sizes, it's really time we stopped this illogical naming convention just because some engineers found a term convenient 40 years ago.

Re:1TB = 1024 GB

[NinjaTariq]

Use the kibibyte if you have a big problem with it.

But I have long since buried my problem with using the SI prefix with byte to mean a power of 2, actually not sure i ever had one, I just accepted it. I am happy with the 1024b=1Kb, 1024Kb=1Gb and 1024Gb=1Tb. The usable space is lower in the case of non-volatile storage anyway, 1Tb never means 1024Gb might be closer to 1000Gb (i don't know).

Re:1TB = 1024 GB

[91degrees]

No. I'm quite happy to accept that a Terabyte is anywhere between 1,000,000,000,000 and 1,100,000,000,000 bytes for general use, simply because it doesn't matter. It gives an idea of the amount of storage, which is all we need. If I was specifying I'd use neither and just say 3.7*10^12 bytes or whatever.

I just get a bit fed up when people insist that the illogical, and deprecated usage of terminology is correct and a usage that has been accepted for quite some time (and long before marketting got involved) is incorrect.

Re:1TB = 1024 GB

[0123456]

Because only real nerds have a problem with 1KB being 1024 bytes rather than 1000 bytes, and kibibytes or whatever you want to call them is a really stupid name. Who wants to have to deal with buying 1.073741 gigabyte DIMMs for their PC when we can just agree instead that a gigabyte is a power of two, not a power of ten?

As for why it's different for disks to RAM, disk manufacturers discovered a long time ago that they could make more money by using SI rather than binary measures for disk size, because it artificially inflated their size. Hence people now complain that they buy a 'one terabyte' drive and it actually only holds 900 gigabytes and change.

Re:1TB = 1024 GB

[Vandilizer]

Simple we don't we just work in a different base:

2^10 = 1024 bytes

See:
http://en.wikipedia.org/wiki/Kilobyte

It's not illogical it makes perfect sense to anyone who programs, well anyone who dose lower level programming. If computers were to work in base 10... Sorry I can not even go there.

True false

v.s. the classic

True Maybe False

v.s. The new base 10 computing

True
Could be factual
Might be accurate
Maybe right
Slightly correct
Slightly fake
Maybe phony
Might be counterfeit
Could be wrong
False

(Ok maybe not this bad)

Re:1TB = 1024 GB

[91degrees]

It's not illogical it makes perfect sense to anyone who programs, well anyone who dose lower level programming. If computers were to work in base 10... Sorry I can not even go there.

If we want to worry about that then use KiB and MiB. But that doesn't make a huge amount of sense. 1KiB = 400h bytes. 1MiB = 100000h bytes. Powers of 256 would make a lot more sense.

Re:1TB = 1024 GB

[vidarh]

Byte isn't an SI unit, so what makes you think we care?

Real geeks have no problem with overloading.

Re:1TB = 1024 GB

[General Wesc]

Simple we don't we just work in a different base: 2^10 = 1024 bytes

But if it's merely a change of base, 1 kilobyte should should be 23 bytes, not 210 bytes. You're changing the base to binary and the exponent to something that makes sense in decimal

It's not illogical it makes perfect sense to anyone who programs, well anyone who dose lower level programming. If computers were to work in base 10... Sorry I can not even go there.

I've done Assembly. I use C. I don't think it makes sense. As you pointed out, computers don't work in base 10. My ints are never 210. They're always raised to a power of two. That makes sense to me. But you're using a hybrid system using both and by doing so you're redefining a 212-year-old term that has at least a 2000-year history behind its meaning. Yet for some reason the IEEE and I are the ones people insist are doing the redefining.

Re:1TB = 1024 GB

[91degrees]

Well, the IEC and IEEE as well as the CIPM and NIST all agree thatthere are 1000 bytes to a Kilobyte and 1024 bytes tothe kibibyte. So there:P

Re:1TB = 1024 GB

[DukeToma]

Why is a Kilobyte 1024 bytes, if "Kilo" means 1000, both according to the SI and the greeks (Kilo is derived from khilioi). If 1 kg = 1000g, 1 kV = 1000V, 1 km = 1000m, why should hard disks break the pattern?

because Kilobyte means 2 ^ 1000

Re:1TB = 1024 GB

[insignificant1]

Just use these for binary:

http://en.wikipedia.org/wiki/Binary_prefix

And use the SI prefixes for powers-of-ten, and all will be okay. The more who do this, the more accepted it will be, the fewer who won't understand what these mean, and less confusion will result.

Re:1TB = 1024 GB

[l3v1]

When we're talking about addressable computer memory, approximating the kilobyte to 1024 is a convenience, but since Terabyte gives such a huge error, and makes absolutely no sense for data transfer or disk sizes, it's really time we stopped this illogical naming convention just because some engineers found a term convenient 40 years ago.

Yes, it's so funny when all these guys just keep arguing why 1024bytes should really be 1000bytes because they don't want to care that it's history, it's practical, it works, and anyway why the hell should be 1000, let's make it 999. Now you go calculate. So, it's funny when they just keep arguing about that but just wait and see how they react when you bring up the mile/feet/pound issue (which really is an SI-issue btw, unlike the byte) from "40 years ago", or well, a bit more so what gives.

Fact is, we who care about 1MB being 1024B, we don't really care how mister joe wants to call a megabyte and how much he wants it to be. We know what they mean, it's their freaking problem that they have created this non-issue for themselves so they won't know what we mean.
 

Re:1TB = 1024 GB

[hackstraw]

Why is a Kilobyte 1024 bytes, if "Kilo" means 1000, both according to the SI and the greeks (Kilo is derived from khilioi). If 1 kg = 1000g, 1 kV = 1000V, 1 km = 1000m, why should hard disks break the pattern?

Read all about it here http://en.wikipedia.org/wiki/Byte

The thing that gets _really_ confusing is that a byte does not have to be 8 bits. I don't know of any modern computers that don't use 8 bit bytes, but there were 7 and 9 bit byte machines back in the day.

Harddrive manufactures have their version of bytes, networking people speak in bits, and all of it is a mess.

A friend of mine is older than me, and when he was in school light came in angstroms, but today light come in nanometers. (Anstroms are deprecated because they are not in the SI power of a thousand rule).

The moral of the story is that standards are not that standard, but when the standard becomes standard the standard is subject to change at any time :)

Re:1TB = 1024 GB

[Anonymous Coward]

Dude, think about what you post.

Try multiplying 2 x 2 x 2, 1000 times.

Large datasets

[BWJones]

This is absolutely the most cost effective way of transferring large amounts of data like this. If you do the calculations on terrabyte size files, sneakernet (of FedEx net) is actually faster and less expensive. We also went to one of Jim Grey's seminars when he was here giving an Organick Memorial Lecture and he made an incredibly compelling demonstration using a variety of data types. We ended up talking with him for some time after about new projects we are engaging in that will also be generating terrabytes of data and his suggestion was to pass applications rather than data which was interesting.

This is becoming more and more the norm in scientific research and Google's work is quite welcome.

Re:Large datasets

[Sobrique]

Never underestimate the bandwidth of a lorryload of backup tapes traveling at 60 miles an hour.

Latency may leave something to be desired though :)

Re:Large datasets

[dmayle]

I remember an article I read on this I think back in the year 2000. The was a research scientist who built a standardized platform (That is to say, a specific PC case with a certain number of hard drive bays, and certain network cards) so that he could exchange data with other universities. They would fill up the data on the networked PC, and they could ship it to any of the participating projects, knowing that they'd get back the same hardware in return.

I remember at the time thinking it was just one of those smart little details that just make working together easier. It's not some great leap of genius, but enough of a well crafted idea that it could really help.

Re:Large datasets

[BWJones]

Yeah, there have been a number of folks using variations on this theme for a while now. It's been interesting that network performance really has not followed the same performance curve as storage and CPU throughput. Add to that the growing amount of data being pushed through "consumer" pipes from people obtaining broadband and pushing sources such as YouTube and company and you have the makings for a bandwidth crunch. This of course is the reason for separate academic and government Internet paths, but it is still a limited commodity. In fact, at some universities engaging in data intensive projects, it is not uncommon for them to occupy the entire bandwidth of the university in off hours to transfer data around the country to various collaborators.

Re:Large datasets

[Agent Orange]

Yup. There was a paper a few years back entitled "terascale sneakernet", by jim gray and a couple of guys at MSFT research division on this. You can find it in the arxiv.

This concept has also been applied to such things as the Sloan Digital Sky Survey. Astronomers do tend to generate a lot of data with large surveys such as this.

Re:Large datasets

[SuperMog2002]

As the old joke goes, never underetimate the bandwidth of a station wagon full of magnetic tapes. Or a Fed Ex plane full of hard drives. Your choice.

Re:Large datasets

[corbettw]

Wait, I'm confused, what happened to the tubes?

Re:Large datasets

[TubeSteak]

In fact, at some universities engaging in data intensive projects, it is not uncommon for them to occupy the entire bandwidth of the university in off hours to transfer data around the country to various collaborators.

Even using the full bandwidth between Internet2 connected Unis, it would still take 2~3+ days to transfer 250Tb of data.

10Gb/s is close to the max you can do with one frequency. That will all change once they start pumping multiple colors down their fiber. Their bandwidth will explode & Google's sneakernet will become useful only to the groups that aren't plugged in.

Re:Large datasets

[jcnnghm]

Internet bandwidth hasn't kept up, but local bandwidth definitely has. My network throughput is more than capable of transmitting data faster than my hard drives are able to write it. And I wouldn't even agree about the net bandwidth. I have a 15mb connection where I used to have a 56k.

Re:Large datasets

[Duncan3]

Shhhh... It's Google we're talking about, THEY came up with this groundbreaking shift in how data is handled.

Praise the Google, don't point out they are just doing the same thing as everyone else.

Google is watching.

Re:Large datasets

[chrisd]

That might have been Queue's interview with Jim Gray? Check it out here:

http://www.acmqueue.org/modules.php?name=Content&p a=showpage&pid=43

Chris

Re:Large datasets

[ezzzD55J]

Never underestimate the bandwidth of a lorryload of backup tapes traveling at 60 miles an hour. Close enough.. This is attributable to Andy Tanenbaum according to http://www.bbc.co.uk/dna/h2g2/A678576 (and one of his books I read).

Another ontopic remark.

Google either approaches bodies that it knows has large data sets I know people who also approach bodies that they know have large 'data sets', but that doesn't get them a lot of 'bandwidth' ;)

Never underestimate...

[bjpirt]

But are they using station wagons?

Oblig.

[RyanFenton]

Never underestimate the bandwidth of a station wagon...

Still very much applies today.

Ryan Fenton

Re:Oblig.

[Paulrothrock]

The page you linked to had a smart idea. Rather than just have the raw disks, create some sort of architecture inside to allow for rapid transmission of the data from the vehicle upon arrival. I could see specialized vehicles that have been hardened against an accident with an inverter to power the drives that have external fiber optic ports hooked up to massive, high speed RAID arrays to rapidly dump the contents to another system at the location and upload content for the next destination.

Then a GPS system in the front automatically generates a route for the driver and after a few hours of waiting for the data to transmit, off he goes!

Re:Oblig.

[Anonymous Coward]

I don't reply to Anonymous Cowards.

And yet, we reply to you...

Like days of old

[tulmad]

This sounds almost like stories of scholars trading/copying books from long long ago. It's actually a somewhat interesting plan.

Re:Like days of old

[meringuoid]

This sounds almost like stories of scholars trading/copying books from long long ago.

According to what I'm told every time I watch a DVD, these scholars were in fact stealing books.

Re:Like days of old

[Anonymous Coward]

Considering the laws at the time vs. the laws now, you have no point. I'm sure that doesn't stop you in your quest for free entertainment, I just wanted to point out the facts aren't actually on your side.

How long until...

[gEvil (beta)]

How long do you think it will be until some maroon somewhere plunks a hard drive into an unpadded envelope and drops it in the big blue mailbox on the corner?

Re:How long until...

[maxume]

If I had to choose between sometime in the future and sometime in the past, I would go with sometime in the past. I think that quote about the universe inventing better fools works at a rather quick pace.

fixed

[Dance_Dance_Karnov]

gonna fix your summary for free... "...one terabyte is equal to 1024 gigabytes..."

Re:fixed

[Anonymous Coward]

Technically that would be a tebibyte. Tera does indeed mean 10^12.

I'm glad they explained what terabyte means though, I doubt the slashdot crows would be familiar with such a term!

Re:fixed

[Anonymous Coward]

Wrong. One tebibyte is equal to 1024 gibibytes. One tarabyte equals 1000 gigabytes. If you're going to correct someone, do it right.

Re:fixed

[Macthorpe]

Wrong. One tebibyte is equal to 1024 gibibytes. One tarabyte equals 1000 gigabytes. If you're going to correct someone, do it right.

You meant 'terabyte', not 'tarabyte'. If you're going to correct someone, do it right.

Re:fixed

[rice_web]

If you're going to reply to parent, at least reply to the right one.

Re:fixed

[Anonymous Coward]

I doubt the slashdot crows would be familiar with such a term!

Most of them have TBs of pr0n running on a RAID in their mom's basement.

Re:fixed

[fourchannel]

um...he did.

Re:fixed

[Anonymous Coward]

A tarabyte is a half a small cucumber sandwich with the crusts removed, served at tea-time on the Plantation.

Re:fixed

[Anonymous Coward]

"A tarabyte is a half a small cucumber sandwich with the crusts removed, served at tea-time on the Plantation."

Tarabyte is part of the evening meal with father and mother at the palace in Helium.

Re:fixed

[l3v1]

Actually, even tarabyte sounds better than tebibyte :P

so..

[mastershake_phd]

Whos going to own the data? I hope Google isnt going to say they do like they want to with the old books theyre scanning. Everytime you download a hubble picture will it have a google watermark?

Re:so..

[99BottlesOfBeerInMyF]

Whos going to own the data?

As always the people of the world own the data. The copyright holders are, however, given a short term monopoly on making copies of it, with certain exceptions.

I hope Google isnt going to say they do like they want to with the old books theyre scanning.

Google has not, as far as I know, claimed "ownership" or even copyright on anything they've scanned. They have, however, created their own database of metadata about the works, which they use to enable people to more easily find specific items in the original data.

Everytime you download a hubble picture will it have a google watermark?

Umm, maybe. Why do I care if they add watermarks to it? If they are in the way, I'll just get them from another source that does not add watermarks. Google can also provide free copies of public domain pictures from other sources with Google advertising slogans on them if they want. It's called "freedom."

Re:so..

[cfulmer]

The ownership of data is presumably a case-by-case thing that depends on what the data is and how it was acquired.

For example, Google does not own the copyright on out-of-copyright books that it scans in (nobody does, by definition.) At best, it might own the copyright on the scan that it did, but that's really unlikely--copyright protects creative expression and a straight scan doesn't add any.

However, they probably have some rights under unfair competition law because they have gone through a lot of work acquiring all this data and it would be unfair for somebody else to piggyback on that work to compete with them.

Recognize also that many of the "Hubble Pictures" you see are colorized versions of raw data that incorporates non-visible parts of the EM spectrum, assigning colors to things you can't see with your eyes. That assignment of colors to create something pleasing to the eye is certainly creative expression. So, if Google takes the raw data and does that color assignment itself, well, the result is theirs.

Re:so..

[oneiros27]

So, if Google takes the raw data and does that color assignment itself, well, the result is theirs.

I'm not so sure that the result in theirs, necessarily. They'd need to properly attribute it. Many science archives have rules about how to properly attribute their work.

Don't get me wrong -- many of the scientists want people to use their data (eg, see The Astronomer's Data Manifesto), but they also want to know who's using it, because it's how they justify the value of their projects, and the costs incurred from distributing the data (especially for non-active projects).

The science community is also working on the Science Commons (an equivalent of the Creative Commons for marking scientific data) and various federated search engines (eg, night time (astronomy) virtual observatories, as well as other space and earth science discipline specific VOs.).

Re:so..

[cfulmer]

Attribution is different from copyright. For example, say you have a novel scientific idea which you write about in some scientific journal and that I read your article and publish my own article, using your idea without attribution.

Now, what I've done would reasonably upset you, but there is no law (at least in the US) that requires me to attribute your ideas to you. In fact, under those facts, I completely own the copyright in my article and you have no legal remedy. Now, there may be repercussions--I may lose my job or be publicly disgraced--but the question is ownership of my work, which is not affected by such repercussions.

Re:so..

[electrosoccertux]

Everytime you download a hubble picture will it have a google watermark?

Umm, maybe. Why do I care if they add watermarks to it?

Because there's no water in space! Obviously then there shouldn't be any marks indicating as such on a picture the Hubble telescope took!

Old Truth

[Nom du Keyboard]

It was said some time ago that the fastest way to transfer data was in a station wagon full of backup tapes traveling down the Interstate. I guess we now update that now to a mini-van full of hard drives...

Re:Old Truth

[Anonymous Coward]

The latency sucks, though.

Re:Old Truth

[HTH NE1]

The latency sucks, though.

They're working on that part of the problem by subjecting two trucks of hard drives to quantum entanglement.

Never underestimate ...

[boyfaceddog]

The bandwidth of a moving van full of disks.

Looks like Google is hoarding data. Seems they at least are equating information with power and money. And them that has the power and money makes the rules.

Re:Never underestimate ...

[Paulrothrock]

They're not hoarding the data. They're storing it online in open formats, at least according to the article.

Re:Never underestimate ...

[veektor]

I first heard it as "never underestimate the bandwidth of a station wagon full of magtapes".

There, two archaic conveyances in the same cliche.

Re:Never underestimate ...

[boyfaceddog]

I first heard is as "wagon full of punch cards", but I thought I'd update it.:-)

Re:Never underestimate ...

[wfWebber]

Well, actually...

I Mail Externals

[moore.dustin]

I mail my external hard drives to different friends a few times a year. I have several, but one specifically for mailing to friends and co-workers. I thought this was somewhat of a common practice.. I have never had a fellow geek gawk at the idea, rather it seemed like the only logically way to get what we wanted to do done.

Google is doing something cool by getting and hopefully displaying the data, but the method is not really anything newsworthy is it? I mean, this is the same as using a flash drive to transfer files real quick, this is just on a much larger scale :)

Re:I Mail Externals

[zappepcs]

There are more uses than just sending data. I'm using removable hard drive trays instead of dual-booting my machine. Swap the tray, reboot, I'm running Ubuntu. Repeat and its XP. I only keep that one as it came free with the PC, boot it up now and then to keep it updated. It makes life easy when you know that you can't possibly fsck up your regular installation when playing with a new distribution or whatever. Never needed to send one to anyone else, but that might be a huge support possibility for family? Never thought of that.

Mea culpa.

[Anonymous Coward]

Google is transferring data the old fashioned way -- by mailing hard drive arrays around to collect information and then sending copies to other institutions... Old fashioned??? What about sneakernets?

In Other News

[UnknowingFool]

FedEx delivered what appeared to be a ton of broken office chairs to Google headquarters this morning. When asked for the sender's ID, the severely beaten FedEx courier would only reply that the sender wished to remain anonymous.

Re:In Other News

[Kuvter]

I hate when senders check the Anonymous Coward button.

Google vs Microsoft

[spazmolytic666]

Whos going to own the data? I hope Google isnt going to say they do like they want to with the old books theyre scanning. Everytime you download a hubble picture will it have a google watermark?

In 10 years google will own just about all data worth owning. Then slashdoters will be railing on them instead of microsoft... or maybe google and MS will merge and collect our taxes too.

Wha?

[Anonymous Coward]

Um, I do not think that word means what you think it means:

maroon (plural maroons)

1. An escaped negro slave of the Caribbean and the Americas or a descendent of escaped slaves.
2. A castaway; a person who has been marooned.

So, were you making some kind of racial remark about escaped negro slaves being stupid with computers? Or maybe you are going to claim you heard it from bugs bunny or your mother?

Re:Wha?

[Anonymous Coward]

http://www.randomhouse.com/wotd/index.pperl?date=1 9970731

Re:Wha?

[Anonymous Coward]

What a maroon.

The ironing is delicsious.

Other Uses for Mass Data Transfer

[Anonymous Coward]

Moe: Say, Barn, uh, remember when I said I'd have to send away to NASA to calculate your bar tab?
Barney: Oh ho, oh yeah, you had a good laugh, Moe.
Moe: The results came back today. (reading a printout) You owe me seventy billion dollars.
Barney: Huh?
Moe: No, wait, wait, wait, that's for the Voyager spacecraft. Your tab is fourteen billion dollars.

Hubble Data

[Ikyaat]

120 TB of data from the Hubble telescope? I wish I was paid to go through that. And this picture is of a...star and this one is a star And a star another star OMG its a FRICKIN STAR

Re:Hubble Data

[Anonymous Coward]

Don't forget to give each one some rediculous and meaningless name while you are at it.

Re:Hubble Data

[Chris Burke]

Don't get too complacent...

"Star. Star. Star. Damnit, star. Star. God this sucks. Star. Star. Space ship. Star. Star. Star. God nothing but fucking stars! Fuck hubble, useless piece of shit!"

Re:Hubble Data

[nharmon]

"That's no moon"

Re:Hubble Data

[LakeSolon]

My God, it's full of stars.

Re:Hubble Data

[Anonymous Coward]

Star... star... star... star...

OMG it's full of stars!

star... pony... star... star...

I hope they're not using ...

[LaughingCoder]

SUVs to transport those hard drives. That would be evil.

Isn't TB... dangerous?

[Qubit]

I don't know what the article title conjured up in your head, but when I saw:

Google's Academic TB Swap Project

...the first thing I thought was "why are they swapping around samples of a dangerous infectious disease like tuberculosis?"

Re:Isn't TB... dangerous?

[sckeener]

...the first thing I thought was "why are they swapping around samples of a dangerous infectious disease like tuberculosis?"

I'm glad I wasn't the only one!

Re:Isn't TB... dangerous?

[Anonymous Coward]

No. No. That would be Microsoft.

Re:Isn't TB... dangerous?

[BForrester]

Obviously as part of their distributed viral marketing program.

Re:Isn't TB... dangerous?

[Anonymous Coward]

Undo that Redundant mod. Check the timestamp: Qubit quipped it before I did. Simply posting to the thread will undo it, even as an AC as long as you're logged in, moderator.

And the story shouldn't be tagged with it either. People searching for articles about tuberculosis on slashdot will end up getting this article as a false positive.

Dangerous precedent

[DebateG]

Don't say I didn't warn you guys about this "don't be evil thing." First they start swapping TB for "academic" purposes, then maybe some avian influenza in some apartments around Mountain View, and next thing you know, they'll be a smallpox outbreak and we will coincidentally receive advertisements on gmail that we can buy the cure for a few thousand dollars from one of their Adsense "partners."

Units...

[alexhs]

One terabyte is equivalent to 1,000 gigabytes. Hey, where do you think you are ? It's Slashdot here ! Everyone knows that ! What people here want to know is how much that does in Library of Congress...

The only thing you're getting by saying that is a flamewar between 10 kinds of people, whose who count only in MB (and disagree with you) an those who count in both MB and MiB (and agree with you) !

For my take on the issue, see this precedent post of mine.

Re:Units...

[nbritton]

Actually it 1024 gigabytes using binary units (base 2), we use binary units because formatted capacity is measured in binary units. For exampe: 1 Exabyte = 1(1024) Petabytes = 1(1024)(1024) Terabytes = 1(1024)(1024)(1024) Gigabytes and so on... The formula to convert si units into binary units is si_unit * (125/128) which comes out to 0.9765625. For example: a 750GB hard drive is 750(125/128) = 732.421875 Gigabytes. Also don't forget reserved space... On FreeBSD it's 8% of the format capacity, so 732.421875 * 92% = 673.828125 Gigabytes of usable space.

The Library of Congress is estimated at 3 petabytes, or 3(1024) terabytes:

http://www.lesk.com/mlesk/ksg97/ksg.html

Re:Units...

[alexhs]

we use binary units because formatted capacity is measured in binary units. It seems you haven't read my previous post I was linking to. Please do :)
Your affirmation is wrong. The correct affirmation would be "we use binary units because some OSes reports formatted capacity in binary units".

Proof I've read your post in its entirety is that I was going to write "MS Windows" (like I did in the aforementionned post) instead of "some OSes" :) . My server at home is a FreeBSD, I launched fdisk and it reports size in "Meg", neither MB nor MiB. So I can't say :) What command did you enter to get your MiB size as MB ?

The formula to convert si units into binary units is si_unit * (125/128) which comes out to 0.9765625. For example: a 750GB hard drive is 750(125/128) = 732.421875 Gigabytes. Also your formula isn't accurate, 10^(3n) / 2^(10n) ratio depends on n. Your estimation only works for n=1 (KB/KiB). For GB/GiB, n=3, and ratio is approximatively 0,93.

So a 750 GB HD is only 698,49 GiB.

Re:Units...

[nbritton]

Umm no I think most of what I said is true, except for the conversion formula... We are working with base 2 systems so it makes logical sense to report everything in base 2. Sure the SI unit abbreviations are for base 10 systems but the the formal definition of a byte is 8 base 2 digits. Given that only base 2 systems can use these definitions what do we accomplish by changing them? We accomplish nothing, in fact we backpedal on progress by confusing people and wasting peoples time because they have to debate what the fuck a gigabyte is. I'd like to highlight this fact now with an example:

One of the array's at work is 300GB * 8 drives. 2 drives are for parity and 1 is a hot spare so this leaves us with 1.5TB, converting that to TB (10^12/2^40) gives us 1.36424205265939235687255859375TB. Now we need to convert that to bytes, 1.36424205265939235687255859375TB = 1.36424205265939235687255859375TB(1024)(1024)(1024 )(1024) or 1500000000000B. lets check what FreeBSD says:

> diskinfo /dev/da0 /dev/da0 512 1499999764480 2929687040 182364 255 63

Hmm, it says I only have 1499999764480 bytes, a difference of 235520B... 230KB, Since it's Looks like maxtor screwed me out of 46KB per drive, on that's right we still need to convert it into KB... 47.104KB.

Thanks for wasting my time.

If they've got TB...

[oskay]

I really don't want to share, whether it's academic or not!

As the old sayng goes

[nweaver]

"The moral of the story is: Never underestimate the bandwith of a station wagon full of tapes hurtling down the highway."

-Andrew Tannenbaum

Re:As the old sayng goes

[Compulawyer]

DOH! Beat me to it ....

Re:As the old sayng goes

[monkeypuzzle]

One of my favorites. It still figures into our disaster plan, if you replace station wagon with helicopter...

Google != Open Source

[xxxJonBoyxxx]

Mr Dibona, who is a long-standing Linux evangelist, said: "I am comfortable with where Google is operating. People are often upset and feel we should be releasing more.

"And I agree; I would love to release more. It's more a function of engineering time, than it is a function of desire."

I call B.S. "Lack of engineering time" is why we haven't seen the source to the core search engines or gmail?

Re:Google != Open Source

[the_B0fh]

That's kind of stupid. Just because they want to help and release lots of open source software doesn't mean they have to release the family jewels.

Just because I want to show and teach you how to fish doesn't mean I'm going to give you the plans to my power boat.

That's how I got Linux

[jtownatpunk.net]

My first copy of linux was received on a tape mailed across the country. We dragged (drug?) our CPUs down to the campus computer lab where we pulled the files off the tape with a VAX then transferred the files to our PCs using a null-modem cable. (We couldn't afford NICs in those days.)

/Misty watercolor memories

Now I just my own PB HD.

[kabocox]

I've been thinking that the only home use app lots of HD storage space would be A/V. Now, I guess when 10 PB of HD are $100-1120, then we'll be able to get copies of these 120 TB of hubble data or TBs of other datasets to fill up those future home PB HDs. One day we'll need home exabyte HD to store and play around with public PB datasets.

I can only hope that bandwidth can keep up. How long would it take to transfer a 120 TB bit torrent file over either cable or dsl?

Well, maybe we'll have small TB USB flashdrives that we can just mail those around instead of upgrading our bandwidth.

Just waiting for the day...

[Billosaur]

...that a researcher sends them all the printouts of his/her data... on greenbar...

Bark! Bark! Bark!

[ColdWetDog]

I'm so tired of this stuff. Byte me!

Re:Bark! Bark! Bark!

[AchiIIe]

> I'm so tired of this stuff. Byte me!

I'm sorry, that's wrong too:

* 1 byte == 2 nibbles
* 1 byte != 1 bite

--
Byte nazi police, proudly serving since 2^1025

Re:Bark! Bark! Bark!

[greenguy]

Hey, it could be worse. I thought they were swapping tuberculosis.

Mod parent up

[ari_j]

Here's what happened when I FedExed my RMA to Newegg, packed very carefully. Note the bent motherboard - I didn't even know you could do that. The good news is that FedEx paid part of my claim ... they paid $100 plus the $8.33 that the FedEx store charged me to fax in the claim forms. The bad news is that they did not refund my original shipping or pay more than $100 on the over $280 of damage that they did. It also took about 4 hours of phone calls to even convince FedEx that I was not the seller, and then they lost my claim in their e-mail system (and did not reply to my e-mails) and closed it out for inactivity after a month or so, until I called them and asked what happened.

• box front - doesn't look bad from this angle
• box rear - run over by a truck?
• formerly a CPU - this was packed inside the box with lots of noodles
• motherboard top - not terrible-looking, but note the USB/Ethernet header
• motherboard end view - here you can see the curvature of the motherboard
• another picture of the CPU and motherboard boxes - almost certainly run over by a truck

On a side note, don't bother with UPS insurance. I insured something when I sent it to myself once, and they broke it and the insurance remedy was to return it to the origination address and ask to see an original purchase receipt to award the insurance claim. If you happened to make something yourself or even received something as a gift, don't insure it when you ship it. And hire a private courier (unless someone has found a common carrier that doesn't suck).

Re:Mod parent up

[monkeydo]

The bad news is that they did not refund my original shipping or pay more than $100 on the over $280 of damage that they did.

Did you buy additional insurance over the $100 you get by default?

Re:Mod parent up

[ari_j]

Of course not. None of these companies has ever honored their insurance for me in the past when I've shipped something other than as part of a sale. Moreover, insurance becomes largely irrelevant when you get into the "run over with a truck" territory that this particular shipment was in. Also, FedEx never offered me insurance when I told them what I was shipping and its value. Furthermore, I have had major problems with FedEx in the past, including "overnight" deliveries sitting on a truck in my city for over a week (including food products that were thus rendered worthless). I wasn't about to pay anything extra when I reasonably believed it'd make no difference in how my shipment was treated, before or after its destruction. I'd roughly estimate that FedEx has caused me at least $800 worth of uncompensated losses in the past 5 years, regardless of what insurance or delivery terms I paid for, because of unprofessional and incompetent behavior. Don't ask why I used them for this particular shipment - I will plead insanity and amnesia.

On a side note, apparently FedEx "express" gets you consequential damages whereas regular FedEx ground disclaims them. At least keep that in mind if you use FedEx. I have had better experience with UPS, notwithstanding them not honoring their insurance. And so far, I have had the best experience with DHL, with no problems to date, but I suspect that may largely be due to the frequency with which I use DHL compared to UPS. (That said, I use DHL more than FedEx, so either DHL is way better than FedEx or I am living in a statistical anomaly, a possibility I won't deny.)

Re:Mod parent up

[winnabago]

the insurance remedy was to return it to the origination address and ask to see an original purchase receipt to award the insurance claim

Sorry to nitpick, but this scam has been around for ages - you broke something, oh no! I'll send it to myself and pretend UPS did it. Hell, I even saw it in Seinfeld. Not that you were doing this, but what you tried is pretty suspicious to an outside observer.

They need SOME proof of value or even that the box was actually full to fight this type of fraud, and the original merchant is one way to do it. Also, what are you doing sending packages to yourself? It is cheaper than taking it with you?

And always buy the extra insurance, or instruct the shipper to declare it properly. Newegg is usually pretty good about this, if they provided the return label. If FedEx limited it to $100, you should have definitely added to the base value. Remember that FedEx Express, FedEx Ground (sometimes called home delivery), and FedEx Smartpost are all separate entities, with varying policies, too. You probably used their ground service, which is a conglomerate of private couriers, with little to no accountability to FedEx corporate.

Re:Mod parent up

[evilviper]

Moreover, insurance becomes largely irrelevant when you get into the "run over with a truck" territory that this particular shipment was in.

Why? You said they did pay your $100 claim after all.

Also, FedEx never offered me insurance when I told them what I was shipping and its value.

No idea what you're talking about. You generally fill out the form yourself, and select what insurance you want.

I'd roughly estimate that FedEx has caused me at least $800 worth of uncompensated losses in the past 5 years, regardless of what insurance or delivery terms I paid for, because of unprofessional and incompetent behavior.

Your experience is most definitely atypical. You must be shipping unbelievable numbers of packages to get that much damage. I have yet to have one package seriously damaged by UPS or Fedex (or DHL).

Re:Mod parent up

[ari_j]

Customers in general ought not to be held to know FedEx's corporate structure. I did indeed use the Newegg-provided label. As to my prior shipment broke by UPS, of course I realize that there is the potential for scams. I was shipping Christmas presents to myself because it was cheaper and, on average, safer than trying to check them on my return flight. See my other replies in this thread for more on the FedEx $100 insurance situation.

Re:Mod parent up

[ari_j]

Or maybe you are just lucky. I don't ship that often, and FedEx has to date managed only once or twice to get a package through without undue delay or damage. As to the $100 they paid this time, I had never before that had any of their insurance honored. I wasn't about to pay for it again on the off chance that it'd work out for once. You can hardly hold the end result against my decision, obviously made without knowledge of the outcome. Besides, insurance is meant to cover damage due to normal mishandling, such as dropping a box by mistake, not the kind of (at least nearly) intentional damage that must have been involved in my case. Or maybe you have a theory of how my box got squashed that badly in the normal course of FedEx's business. (It was the original box and packing material that Newegg sent to me, by the way, so this isn't a question of quality of packaging.)

As to the form I filled out - FedEx stores vary wildly with how they do things and how knowledgeable their staffs are. I acknowledge that I could have taken additional steps to insure the package, but given my prior experience doing so seemed to be wasteful at best. At least Newegg treated me well throughout the process, helping me find the phone numbers to call to make a claim, e-mailing me the pictures of the damaged merchandise, and holding onto the goods for FedEx to inspect.

Re:Mod parent up

[winnabago]

Customers in general ought not to be held to know FedEx's corporate structure.

I don't know if, in this age, this is wise. With so many corporations buying up major parts of our lives like food, communications, salaries, and transportation, I would challenge you to take a look at the structure of the different entities that affect you daily. The unfortunate fact is that every decision you make needs to be researched to find the most appropriate course of action based on who is behind the marketing. Sure, it might have the FedEx name on it, but it's not FedEx.

Flickr is NOT Yahoo and Cingular is NOT AT&T, Mazda is not Ford, and certainly PayPal is NOT EBAY. Again, I'm not picking on you, but there is a level of information that is available to anyone choosing shipping companies (I realize you didn't have a choice here, but this is in general), or any service, and the internet makes this available to you instantly. Use it.

Re:Mod parent up

[Anonymous Coward]

I bet the motherboard performed better like that than a normal ECS one.

Re:Mod parent up

[MajinBlayze]

As a former UPS employee, (I worked as a package handler, the guy that beats the shit out of your boxes as he loads them on the truck) I will never ship anything of value without paying extra for the insurance. when you do that, a couple of things happen:

1. the item goes into a big bag (by itself, not mixed with other items) with red/white stripes, so employess know not to mess with it)
2. it gets hand-carted to the destination truck, and is the last thing to be loaded, and first unloaded
3. only seasoned workers ever touch your package, and generally care about the state that it's in
4. finally, they are good about paying up if the item arrives damaged.

did I forget to include ???? and Profit!

Re:Mod parent up

[evilviper]

Besides, insurance is meant to cover damage due to normal mishandling, such as dropping a box by mistake, not the kind of (at least nearly) intentional damage that must have been involved in my case. Or maybe you have a theory of how my box got squashed that badly in the normal course of FedEx's business.

I still don't know where you get that idea. Insurance is meant to handle any kind of damage, including being completely destroyed in plane crashes, car accidents, train derailments, theft, loss, and anything else that could possibly occur.

For your package, I imagine it was run over by one piece of equipment or another. Forklifts, tractor trailers, etc. Or it may have been some sort of freak accident with equipment in their automated package handling system. I certainly don't have any reason to believe it was intentional, unless you have some reason to believe you've seriously pissed-off your local fedex office employees beforehand...

Re:Mod parent up

[amuro98]

I used to work for a company that made 1U web servers.

Our RMA department got back one of our boxes, unopened, with a large dusty truck tire print across it. The customer refused delivery, and was asking for a replacement. Inside, not only was the server's steel casing horribly bent, but the motherboard, RAM and HDD(!!!) were all bent as well. The PSU was totalled, but the board managed to still POST despite its new twist and bent RAM chips. The HDD didn't spin up, of course.

Re:Mod parent up

[ari_j]

I would agree with you, except that I don't think that the average consumer should be held to that level of sophistication. This is mostly a cheaper cost avoider issue, for me. Who can more efficiently discover the relevant information? Clearly, the answer here is FedEx.

Re:Mod parent up

[ari_j]

What I mean is that being run over by a truck is not within the realm of what a person buying FedEx insurance contemplates. When you ship something, you can assume a certain level of negligence is possible - such as dropping your package from a height of 4 feet or setting a package that leaks liquids on top of it. You don't normally think that FedEx will be so careless as to run your package over with a truck.

I use the word "intentional" because it wouldn't surprise me if the kind people at my local FedEx depot apply the same attitude toward package handling as they do toward customer service.

Re:Mod parent up

[llefler]

Selecting air service gets your package special handling too. I regularly ship $3000 (insured value) packages. They go out UPS blue, come back FedEx ground. Never had any problems.

Re:Mod parent up

[PB_TPU_40]

My only problem with DHL is the fact that they left $3000 worth of LCDs on my door step with no signature. Please note if i wasn't so honest I would have gotten another matching set at DHL's or Dells expense claiming I never took delivery. But seriously, I have had to deal with both UPS and FedEx on insurance claims. FedEx has always given me problems and delayed shipments. UPS has always been on time, I know my UPS driver pretty well and he met the plane when it landed an hour drive away for some stuff I had shipped next day. My insurance experiences have taught me its not worth it on small items. The key is having proof of condition when shipped and value. Cameras work great and hold on to receipts. If you don't have receipts, make a manifest of what it would cost if you had to replace it. Make sure that the insurance will cover that price. Read the form you fill out for shipping too to make sure it is the values you declared. The key is to do your homework and not be complacent. An extra 15 minutes of extra work pays off when a $600 package is damaged and you're not stuck with the bill.

Re:Mod parent up

[monkeydo]

I'm confused. Did you not get insurance because you believe it is worthless, or because they didn't offer it? Of course if you think it's worthless, then you know it exists, and whether or not they offered it is moot isn't it? Of course insurance is worthless 99% of the time. That's why they call it insurance. Most of the people in New Orleans didn't think they needed flood insurance, because it had never helped them before. Look how well that worked out.

Either buy the insurance (and the right to bitch), or don't and accept the risk of damage. It really is that simple.

Re:Mod parent up

[Profane MuthaFucka]

Is the treatment better if you buy $5000 of insurance instead of $201 of insurance?

Re:Mod parent up

[kv9]

fedex ftw

Re:Mod parent up

[Anonymous Coward]

My only problem with DHL is the fact that they left $3000 worth of LCDs on my door step with no signature. Please note if i wasn't so honest I would have gotten another matching set at DHL's or Dells expense claiming I never took delivery.

Good luck with that. Unless the delivery service level specifically required customer signature, it was considered delivered the moment the driver marked it as such in their system. Once dropped off, any packages without signature required are the customer's sole responsibility to secure against theft/damage. If you're at work and someone swipes it off your front porch before you get home, tough shit. The shipping company won't give a damn.

You don't think other people have already thought of this scam? That's exactly why they won't do a damn thing for you if you get regular (non-signature) delivery. They could require signatures for all deliveries, but that's way too inconvenient for most people, not to mention more costly as multiple delivery attempts must be made. Any shipper implementing that would go out of business in a heartbeat. So YOU, the receiving customer, must make a choice: either require signature at extra expense and possibly hassle (being available to take delivery) or take the risk that your shit gets stolen or damaged after delivery. It's a tradeoff.

Re:Mod parent up

[ari_j]

Apparently I bought the insurance without knowing about it. I did declare the actual value of what I shipped, and provided receipts and invoices later when filing my claim. I also accepted the risk of shipping damage - not the risk of running over my package with a truck, which is beyond garden-variety negligence. You'll note that I'm not really bitching, either. I'm mostly warning people that FedEx runs packages over with their trucks, so that other people either insure their shipments or take other precautions. They sound the same but my intentions are good. :)

Re:Mod parent up

[MajinBlayze]

Not that I'm aware of. you just had to have more than the default.

Re:Mod parent up

[MajinBlayze]

Most uninsured air service packages go through the same handling as uninsured ground. I remember on more than one occasion coming across air packages mixed with ground that was supposed to go to air, and it already being after the truck had left for the airport. there are several measures to catch these packages, but they don't get "special treatment" as such.

Don't Use 'Maroon' Negatively

[Anonymous Coward]

That's the only instance of anyone claiming it's a jocular misspelling of 'moron.' other sites point out why it shouldn't be used as a derogatory name. I suggest gEvil beta refrain from using that word in a negative light considering what that word (when used as a noun) has meant for a long time for many people.

That excuse is about as weak as George Allen's.

Re:Don't Use 'Maroon' Negatively

[treeves]

I have a pretty good vocabulary, yet I was unfamiliar with the 1st definition provided above.

I assumed the Bugs Bunny interpretation as well. There, now you have at least two instances.

This is NOT good news

[Anonymous Coward]

I really don't like the idea of a "private" (yes i know its publically traded) company having control of this public information. The data was paid for by tax payers. Google will inevitably make money from this otherwise they wouldn't be doing it.

This is not right.

Re:This is NOT good news

[99BottlesOfBeerInMyF]

I really don't like the idea of a "private" (yes i know its publically traded) company having control of this public information.

You do know many government agencies already outsource IT and other projects to "private" companies who have all this government generated information, right?

The data was paid for by tax payers. Google will inevitably make money from this otherwise they wouldn't be doing it.

Yeah, and right now Microsoft makes money off of selling them the OS and office suite. This isn't a question of if the government will be paying for the ability of their employees to do word processing, it is just a matter of how much and which companies will be getting the money. I don't trust Google any less than I do MS, who currently supplies the OS and the networking and the word processor. I don't trust them any less than the contractors the government already exports this data to. If they can save 75% of the current cost I pay in taxes, I'm all for it.

I'd probably rather they saved 50% of the cost and implemented Linux and OpenOffice in house instead, which would solve both the security issue and the finance issue, but given a choice between their current solution and going with Google, I don't see how Google is any worse.

Re:This is NOT good news

[astanix]

So, what you're saying is that this public data shouldn't be copied? It's not like they're taking all of the data and destroying the originals. They are obtaining copies of all of the public data that they can.
I'd just like to point out Google's Corp Info page. http://www.google.com/corporate/index.html

Company Overview

Google's mission is to organize the world's information and make it universally accessible and useful.

Re:This is NOT good news

[Anonymous Coward]

Google's mission is to organize the world's information and surround it with advertisements.

Fixed that for you.

Re:This is NOT good news

[HTH NE1]

So, what you're saying is that this public data shouldn't be copied? It's not like they're taking all of the data and destroying the originals.

There's destroying and then there's locking away. There are people pushing for laws that say one person's copy of a public domain work is copyrighted by that person for the typical term and that no one else may make a copy from that copy without permission. It's specifically about granting broadcasters copyright over their rebroadcast of a public domain work, but it is laying the foundation for perpetually renewable copyright on demand. Whosoever can preserve the originals long enough gets renewable copyright for eternity.

Re:This is NOT good news

[that this is not und]

It's not like they're taking all of the data and destroying the originals.

Actually, they probably are, at least in some instances. Many libraries, when they bought into the 'microfilm' Koolaide swindle, cut the spines off their books and permanent hard-bound copies of publications, shot the books onto film, then pulped the pages. It is much, much more expensive to hand place each page, one-at-a-time, of a bound volume, than it is to slash the book to sheets and feed the sheets.

So, yes, Google is most likely destroying a lot of originals.

People always get glassy-eyed about 'digitizing' processes. Remember, a lot of times the 'digitization' process is justified in terms of the 'big savings' that comes from eliminating the storage/retrieval cost of the paper originals.

So...

[Perseid]

...what does this new P2P technology mean for me? I guess the RIAA is really in for it now.

Um...no...

[idugcoal]

The reason that hasn't been released would be "trade secrets."

Relax. Think before you call B.S.

...why not tapes?

[Penguinisto]

I understand the whole "HDD w/ a common filesystem = more compatibility" thing, but wouldn't it be easier to simply send along some tapes of a type appropriate to the format/type that the scientific institution uses? LTO-3 can do 800GB compressed, SDLT can do up to 600... and neither is susceptible to data loss when it gets bounced too hard by FedEx/UPS/DHL/Whatever. (plus it would make for a lighter package, wouldn't require some poor IT schmuck to disassemble a server or wait forver for USB to transfer all of it, etc...)

I'm not criticizing or anything; just curious is all.

/P

Re:...why not tapes?

[Laur]

wouldn't it be easier to simply send along some tapes of a type appropriate to the format/type that the scientific institution uses?

There are basically two reasons one would choose to use HDDs over tapes: compatibility and price.

Compatibility: Sure, one scientific institution may have standardized on a specific type of tape, but what about all the rest? Pretty much everyone in the world can read a standard HDD formated with a well known filesystem.

Price: what is the cost of HDDs vs. tapes per gigabyte? What is the cost of the hardware to read them? What are the actual differences in shipping costs between the two?

susceptible to data loss when it gets bounced too hard by FedEx/UPS/DHL/Whatever.

Hard drives are shipped all the time, how do you think they get to stores in the first place? Besides, if the price difference is great enough, and damage is infrequent, perhaps you can afford to re-send damaged HDDs when necessary, and still come out ahead of tape.

wouldn't require some poor IT schmuck to disassemble a server or wait forver for USB to transfer all of it

They have removable trays for disks. Or you could use something external like firewire. Or you could do what has been suggested elsewhere, and just ship a standard configuration of PC loaded with drives, so that you just have to plug it into your network to get at the data.

Re:...why not tapes?

[kulover]

The reason for not using tapes is exactly because of the compression. The time it takes to compress that data and then send the data to the tape takes a lot of time. That same process would have to be repeated on the other end.

Besides, using HDD for transfer means immediate access to the same data on the other end with speeds that are unmatched with tape backup systems. It might also be worthy to note that data sets that large usually are stored on large RAID systems like this one from LSI Logic, http://www.lsilogic.com/storage_home/products_home /external_raid/6998_storage_system/index.html, and are not installed into a computer like you may be thinking. It provides unmatched speed and reliability. A single rack system can sustain 1,600 MB of transfer to attached hosts, which is how Google will probably use it anyway. I highly doubt a single computer will be looking at that much information.

Re:...why not tapes?

[K8Fan]

The "TeraScale SneakerNet" paper posted earlier anticipates and answers that. They ship a fully assembled computer with processor, RAM, OS and network interface. Plug it in to the wall, plug it in to the network and assuming you had previously agreed on a networking protocol, you're rolling as soon as it boots! No restoration, no decompressing, immediate access to the data.

Does anyone have a Linux distro for this specific purpose? Preferably tiny enough to fit onto a USB key and optimized for bandwidth, preferably with a web server interface for configuring the discs and network?

rsync

[G3ckoG33k]

We have been sending two DVDs, with about 6-8 GB data, around every month for updates. Now we are trying rsync, which in our view has been more convenient.

Re:rsync

[Laur]

We have been sending two DVDs, with about 6-8 GB data, around every month for updates. Now we are trying rsync, which in our view has been more convenient.

The article and the GP is about sending large amounts of data, as in terabytes. In this discussion, 8 GB is tiny, and is easily downloaded much faster than even express mail. Besides, rsync won't really help if all your data is unique (such as astronomical data). Rsync really helps when very little of your data set changes between updates, such as backups.

Re: Google's Academic TB Swap Project

[HTH NE1]

I'm just happy they're not swapping tuberculosis.

Re: Google's Academic TB Swap Project

[Anonymous Coward]

that's what I read it as!

This is NOT off-topic!

[Anonymous Coward]

It's a clarification of the confusing post above. Mod appropriately.

I told my wife about the new Hitachi TB drives

[wwphx]

She's an astronomer, said the Sloan Digital Sky Survey produces about a terabyte of data a year. Not as much as the Hubble, but still pretty cool.

120Tb is 100 SAIT tapes

[Colin Smith]

1.3Tb each or so. About $150,000. the drive is about $5500. $155,000 in total. A 750Gb hard disk costs about $1000. so it'd cost about $160k to do the same with hard disks.

Hubble data on google?

[kavalec]

Why not? Today Google Earth, tomorrow Google Universe!

Re:Hubble data on google?

[Anonymous Coward]

Don't laugh, but my first comment to Google about Google Earth was:

"Why not do the same for Mars, Jupiter, Saturn, etc...?"

Sounds like they went one step further... The known Galaxy.

Great idea!

Nope

[sheldon]

How you measure a terabyte depends on whether you are buying disk, or monitoring disk usage on your server.

The disk manufacturers define it as 1000 megabytes which is 1000 kilobytes which is 1000 bytes.

The OS measures it as 1024 megabytes, which is 1024 kilobytes, which is 1024 bytes

Why? Because when you're buying a drive, 750 Gigs sounds bigger than 698.5 gigs.

Good Hard Padded Cases

[chrisd]

We ended up buying a bunch of these to ship the arrays around in. Cardboard == bad :-)

Re:Good Hard Padded Cases

[monkeydo]

But what everybody wants to know is, "Do you insure them"?

Re:Good Hard Padded Cases

[ari_j]

It sounds like Google uses the SKB insurance policy. Smart move. :)

FedEx, UPS, insurance.

[Kadin2048]

I'm with you, although I have seen FedEx and UPS both damage a lot of packages. I think that their automated systems are a lot rougher on packages than AirBorne Express / DHL or the USPS's Parcel Post. But if you don't insure it, you're accepting that risk when you give them the goods.

A while back I bought a radio-controlled airplane, pre-assembled. It came in a big box, most of which contained the wing. So it was fairly fragile, but well packed, in tri-wall. Got it sent UPS, with insurance for the full value.

They ran it over with a forklift.

To their credit, they called me right away and basically said "uh, so we may have damaged your package a little bit, you might want to look it over." So I went and took a look at it, and it was mangled pretty much beyond recognition. I was a little surprised they had actually bothered to deliver it. But I called them up, told them the stuff inside was ruined, and they sent me a check. (I think that if they hadn't been aware that it was broken already, they might have come and picked it back up, but as it was, they didn't.)

The only problem I have with the way they do insurance, is that they always want the SHIPPER of the goods to file the insurance claim, rather than the receiver. So if you ship something to me, and it arrives to me basically destroyed, and I call UPS, they're going to say "hey, we can't do anything except ship it back, and that guy has to file the claim." It takes a lot of arguing and escalation in order to explain to them, that sometimes things just don't work that way.

I think this is because they're used to working with big businesses and retailers that want to get damaged goods back, and then send out new ones, but for eBay and private shipments, where the RECEIVER is absorbing the transit risk, and the shipper is just basically saying "hey, I'm selling this to you FOB, whatever arrives at your door is your problem" (which is the eBay standard), it creates a big problem. The last thing the shipper wants is for the damaged goods to come back at him, because from his perspective, he washed his hands of the whole business when he dropped it off at UPS.

So overall, I'm not hugely dissatisfied with them, they just need to get through their heads that it's not always the shipper who's going to initiate a claim, and that in many cases, it's going to be the receiver of a shipment who is purchasing the insurance and who is the one at risk if something gets damaged, and it's going to be them who's filing a claim for loss.

Now, when I have fragile stuff that I want to send, I pretty much always use DHL, because I haven't had them mangle anything yet, but you can't beat FedEx Ground for being dirt cheap. You just have to be prepared for a lot of bureaucratic hassle when they drive over it.

The other thing I learned, is to always take a photo of the shipping label, or note the tracking number, on everything. Both UPS and FedEx are absolutely worthless unless you have a tracking or waybill number, and oftentimes, shippers won't bother to keep records of that on their outbound stuff. (Which means if it gets lost, everybody's hosed.)

Re:FedEx, UPS, insurance.

[amuro98]

I've seen UPS deliver at least 2 items that had been run over either by the forklift or the delivery truck itself.

What I want to know is, HOW IN THE WORLD DO YOU RUN OVER A PACKAGE?!? I can almost see it if the package was VERY thin, but a fully assembled model airplane?! Forklifts aren't know for their off-roading abilities here. They have next to no ground clearance. How do you run over a box that's probably taller than the wheels of the forklift?!

At least they admitted to the problem. I've seen UPS deliver crushed boxes - tire print and all - to customers and then claim there's nothing wrong with them.

Re:FedEx, UPS, insurance.

[that this is not und]

They didn't have to run it over. It would be far easier for a forklift to RAM it up hard against something that wouldn't give. The 'give' then is your package.

So you're saying "Microsoft = open source", then?

[xxxJonBoyxxx]

Just because they want to help and release lots of open source software doesn't mean they have to release the family jewels.

If the average Slashdotter applied the same flawed logic to Microsoft, you'd have to say they're big open source sponsors too. After all, Microsoft has released GB of free source code for utilities, etc. for decades. Sure, the code mostly only works with their proprietary "family jewels" (the OS and development tools), but why quibble?

Re:So you're saying "Microsoft = open source", the

[the_B0fh]

Actually, at least the earlier versions of MS-DOS *WAS* open source - iirc, Microsoft actually distributed the source code (or at least made it available) of some of the early 1980s MS-DOS.

Not acording to NIST

[Ernesto Alvarez]

If you want to be strict, the SI defines the "tera" prefix as 10^12, so 1 terabyte = 1000 gigabytes.

If you want to use the binary values, you might as well use the correct "tebi" prefix. NIST says you should, and it looks like the IEC, IEEE and BIPM agree.

Re:Not acording to NIST

[rolfwind]

I agree - science is about universal standards, and this issue gets so confused because in every other context kilo means 1000. Saying that a kilo is 2^10 just because the result (1024) is close to a thousand is arbitrary and confusing.

(And wikipedia seems to agree with the terabyte/tebibyte difference).

TB swap???

[Lord Ender]

TB is killing people all over Africa, and Google wants to see it swapped around our schools, too?!? I knew those liberal, heathen, California commies would be the downfall of this great nation!

Who needs Google for that?

[Torvaun]

I've got Celestia

Hubble Earth?

[Anonymous Coward]

Why don't they make a 'google earth' that uses hubble data. Instead of looking down at the earth you could look up and away, allowing zooming just like google earth but with pics of the universe. I'd put up with adsense to be able to browse that kinda interface.

You got it right

[Anonymous Coward]

It's "dragged"