Slashdot Mirror
Linux Breaks 100 Petabyte Ceiling
*no comment* writes: "Linux has broken the barrier with the 100 petabyte ceiling, and
doing it at 144 petabytes." And this is even more impressive in pebibytes, too.
← Back to Stories (view on slashdot.org)
Hm, that can't be right, I swear I heard it was supposed to be two raised to the power of 50, multiplied by 128.. hm.
when the rain comes, they run and hide their heads. they might as well be dead.
..."but Linux recently became the first desktop OS to support enormously large file sizes"...
My other account has a 3-digit UID.
Help us out here...
What's a petabyte?
...but who gives a toss?
Oh arse
How about the people who filled their 30mb drives in seconds a few years ago? The more things a computer can do, the bigger the files get. All I need now is a slightly bigger hard drive to take advantage...
The slashdot writeup here is pretty terrible. doing what at 144 petabytes?
From the text, it seemed as though someone build a Linux machine with 144pb of Ram or something.
autopr0n is like, down and stuff.
I sacrifice my karma to you.
Tom.
Oh arse
Now I can finally rip all my cds at the bitrate they deserve!
How about a kernel that's secure and stable so we can have a nice secure webserver. All many of us want is a nice secure, stable, linux webserver, that will run apache, php, mysql happily and forever. No more junky root exploits its getting tiresome.
I won't say that these file sizes won't be used in the future, but can someone point me to an application that would really benefit from it right now?
"We almost forgot to mention this, but Linux recently became the first desktop OS to support enormously large file sizes."
So what about non-desktop OS then?
If you have the source, you have the whole world...
I really don't have any pr0n larger than 600MB, but thanks for asking...
This would be handy for over 8200 years of DVD video.
Portable versions of Firefox, GIMP, LibreOffice, etc
I learned about theregister.co.uk a long time ago becuase of slashdot. Now we have a billion choices for slashboxes but how come we dont have one for the reg? Every /. reader worth their salt reads the reg and its time it got a slashbox. While we are at it lets add theinquirer.net as well.
This would be a boon for every linux and hardware buff.
Ascii artist &
I Couldn't care less. Thats just like saying my
...
processor has 2^64 bytes of address space; will
I ever use it all?
- note: this comes from a guy who never ever
thought that he would see one of his computers
with 2^32 bytes of ram storage
They claim a petabyte == 10^15, but that's not true. Just as a kilobyte = 2^10 and a gigabyte is 2^30, a petabyte is 2^50.
So they really got 128 petabytes, if you ask me.
And what's this "2^48 * 512" deal? Why didn't they just say "2^50 * 128" which is more to the point?
Anyone Know of a flavor of linux that offers a simple Stable Secure Linux Webserver without all the junk piled in?
This could tatally be a "good thing" with regards to video. Just think of taking all your DVD's and merging them in to one big file that just plays... okay, thats what a playlist is for but surely an insanely high quality video format would need this sort of filesize.
Of course, we all have eyes capable of appreciating this insane quality... dont we?
You don't know pain until you try and compile on a 386, each line goes by, and you cheer because you weren't sure if i
This is what we've all been waiting for!
Now Linux can really own as a legitimate desktop OS!
Seriously though...Isn't there a better place for someone who has the time to contribute? I'd rather see a better desktop environment, a better E-mail package, etc...
(Flame away, all of you running on 200Mhz machines with a four gig drive who will post about how awesome this new support is!)
i finally have a place to put all that porn!!!
The IDE driver supports such rediculously large files, but no filesystem that I know of currently does, not to mention the buffer management code in the kernel.
So does linux support 18pb files? kind of -- pieces of it do. But the system as a whole does not.
This is great! Now all I need is a 144 terabyte hard drive.
2.197265625 trillion Commodore 64's.
98.7881981 billion 1.44 meg floppy disks.
1.44 million 100 gig hard drives
or
3.5 trillion 4K ram chips (remember those?)
If tits were wings it'd be flying around.
remember when 640k was enough for everybody?
well i for one am scared by the fact that oneday soon 144pentabyte files will seem small
- Lord of the Rings is boring. There is a distinct lack of giant robots in it. Good movies have giant robots, bad movies don't. -james
-------
Drink Coffee - Do Stupid Things Faster And With More Energy!
Can we see a SLASHDOT version of linux that's made to be a secure stand alone webserver (apache, mysql, php) Forget the banner ads I'd pay you some money if you could create an ISO for us /.'ers
You guys already know how so why not share?
I am just wondering here, is there some sort of performace hit in addressing normaly files while adding in support for this "petabyte" feature.
Surely the amount of bits needed to address this is going to increase and more data for addressing means less data for good ol file transfer.
Is this going to be a noticeable difference or am i just beeing a bit whore?
You don't know pain until you try and compile on a 386, each line goes by, and you cheer because you weren't sure if i
http://oss.sgi.com/projects/xfs/features.html
XFS is a full 64-bit filesystem, and thus, as a filesystem, is capable of handling files as
large as a million terabytes.
263 = 9 x 1018 = 9 exabytes
In future, as the filesystem size limitations of Linux are eliminated XFS will scale to the
largest filesystems
There are about 10^10 solar masses of mass in a large galaxy like our own. At ~10^33 g/ solar maxx, and 10^23 atoms per gram, That's 10^66~2^219 particles in our galaxy. Beleive me, scientists will make use of as much computing power, RAM, and storage space as they can get their hands on. If only the limiting factor were operating system limitations rather than the more practicalities realities of funding and costs of hardware.
"144 PB should be enough for anybody."
- Bowie J. Poag, November 7, 2001
Bowie J. Poag
this might be usefull for some very large database tables (assuming you don't use rawdevices).
that said, this is when i turn this into a mini ask-slashdot:
while i have no problems writing/reading large files (i.e., >2GB), most regular linux software can't deal with them
for instance, i can't upload them with ftp. i'm having this problem with a mysqldump file that's part of a system backup.
right now it's not a real problem since i can gzip the file and the size goes down to 250MB aprox, but how do you guys handle large files in linux anyway?
No, not really. But you must remember, that if you have a need and you wait until you have that need to develop a solution for it, you have developed the solution too late. Remember "640K sould be enough for anyone?" That's an example of not planning for the future adequately. So, no it you're looking for a here and now reason for it, you're not going to find it. But remember, that that's not the point.
Touch everywhere, even when inappropriate.
Just a side note: BeOS has support for files up to 18 exabytes, not 18 petabytes, as stated in the article. This is roughly 18,000 petabytes, or 2^64 bytes.
Just wanted to set the record straight.
- Mike
Now, I can really imagine someone that buys a 144Pb drive (array) and will use IDE?? I would personally go for SCSI there ;-)
What I am really wondering is: is there at the current moment ANY company/application/whatever that required this amount of storage? I thought that even a large bank could manage with a few TB's
Not intended as a flame, just interested
but still, this is a Good Thing (r)
Fighting for peace is like fucking for virginity
So you whip out your two hundred million cd recordables, and start inserting them. Let's say you get 1 frisbee for each 25 700Mb CDs.
This leaves you with eight million frisbees.
That's a stack 13 kilometres high.
So who needs this on a desktop OS again?
Free Java games for your phone: Tontie, Sokoban
FreeBSD had it first. For over a month. Read the committer CVS Logs and weep, penguin boys.
e v/ ata/ata-disk.c -> version 1.114
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/d
Sounds like all they are saying is that the new
IDE driver can support 48 bit addressing. With 2^48 seconds of 512 bytes, you get 144 PB. But there are a LOT of other barriers to huge filesystems or files.
For instance, the Linux SCSI driver has always support 32 bit addressing, good enough for 2 terabytes on a single drive. But until recently, you couldn't have a file larger than 2 gigabytes (1024x smaller) in Linux. I think that the ext2 filesystem still has a limit of 4 TB for a single partition.
So while the IDE driver may be able to deal with a hard drive 144 PB in size, you would still have to chop it into 4 TB partition.
Well, it's good to see that Linux has caught up, but the article is not correct that Linux is the first OS to support 48-bit ATA; FreeBSD has had this support for over a month now.
See for example: this file which is one of the files containing the ATA-6r2 code, committed to FreeBSD on October 6.
Now that's a lot of porn. Time to hit the newsgroups!!
Slashdot # 199661 the number that's the same upside down and right side up
10^23 atoms/gram => avg molar mass of 6 (lithium). Is this a bit light?
Free Java games for your phone: Tontie, Sokoban
http://www.cacr.caltech.edu/~roy/dataquan/
George W. Bush
President, United States of America
I'm looking at the Linux XFS feature page, which states:
My understanding is that the 2TB limit per block device (including logical devices) is firm (regardless of the word size of your architecture), and unrelated to what Mr. Hedrick did. Am I wrong? Does this limit disappear if you build the kernel on a 64-bit architecture?And, on 32-bit architectures, there's no way to get the buffer cache to address more than 16TB.
It should be 128 PB as 2^48 * 512 kB = 2^48 * 2^9 B = 2^57 B = 2^7 PB = 128 PB. Q.E.D.
(2^48 is the number of blocks (since 48 bit addressing is used) and each block contains 512kB of data.)
Real life is overrated.
Is 1 petabyte 1000^5 or 1024^5? (i.e. is it 10^15 or 2^50)?)
:-)
If 1kB = 1024 Bytes, then I've always assumed that 1MB = 1024kB (instead of 1000kB), 1GB = 1024MB, and so on.
Normally this doesn't make that much difference, but when you consider the cost of a 16 (144-128) petabyte hard drive, then the difference is more important
... but a couple of years ago, I was investigating OODBMSs. The sales bloke for (I think it was) Objectivity claimed that CERN were using their database for holding all the information from the particle detector things - which I can see being a shedload of data (3d position + time + energy). He was suggesting figures of 10 petabytes a year for database growth (so it must be frigging huge by now).
Of course, this was probably salescrap. Does anyone know the truth on this?
This sig made only from recycled ASCII
Then why are you here? Ignorant troll.
The drive size limitations for IDE drives has been driving me nuts for years. First we had 0.5G, then 2G, then 8G, 32G and finally 128G (or 137G actually)... every time the barrier was moved forward to be 4 times larger than before, which meant we needed a knew kludge every 2 years. At least now it'll take a little bit longer, before we need yet another addressing scheme. By the way, this would be an excellent opportunity to nuke the old DOS partition table format (happily I know *BSD never needed it) once and for all, as well.
Btw, don't get messed up with two distinct things: 1) Being able to address 2^48 sectors on an IDE disc, and 2) having a filesystem that can handle files as large as 2^48 sectors.
Do you wish to save "Entire knowledge of the Universe.txt" ? This may take some time.....
You clicked YES. Thankyou, please get along with the rest of your life while I save your file....
From my perspective, while obscenely large limits on file system sizes are no bad thing, I'm more interested by the prospect for scalability in the context of realistic problems. I see much larger challenges in establishing systems to maximally exploit locality of reference. I'd also like to see memory mapped IO extended to allow direct use to be made of entire large scale disks in a single address space using a VM-like strategy ... but I guess this will only be deemed practicable once we're all using 64 bit processors. Are there any projects to approximate this on 32 bit architectures?
Heavy elements are pretty rare compared with hydrogen & helium, so an average atomic weight of 6 is feasible imho.
I would still like to see where the figures come from though.
While we can access disks as file, the question that arise is, even though the IDE drivers have such large addressing capabilities, are the file handles and the file system able to support files, and I really say files here, larger than 4GB?
I remember creating a tarball file (due to a bug in tar) larger than 4GB and I couldn't even access/delete/unlink it anymore from my hard drive, linux would simply not allow it.
EKS - Dave Poirier Bandai Kaosu Jikuu!
It amazes me how UNIX vunerabilities are never /.
highlighted with as much fanfare on
I think the moderator just proved your point:
-1, Offtopic !
Shhhh! mustn't let anyone know.
Before you start thumping your chest about how superior or cutting edge *Linux is, go look at these two links
A slashdot story pointing out how without the FreeBSD ATA code, the Linux kernel would be 'lacking'
The FreeBSD press release announcing the code is stable
If The Reg actually researched the story, Andy would have notice it is not a 'first' but more a 'dead heat' between the 2 leading software libre OSes. Instead, The Reg does more hyping of *Linux.
If it was said on slashdot, it MUST be true!
could you imagine the bloat Microsoft would put into Office XP 2002?
it would come on 20 DVDs and Word itself would be a 22Gb install. the paperclip would be all nVidia 3-D rendered and exponentially more useless and annoying.
the missing link
If it was said on slashdot, it MUST be true!
>>It amazes me how UNIX vunerabilities are never /.
>>highlighted with as much fanfare on
>
>I think the moderator just proved your point:
>
>-1, Offtopic !
>
>Shhhh! mustn't let anyone know.
LOL!
Yah! I noticed. Can't say it suprises me though.
" A collective illusion of security has get to be a good thing " (TM)
Well, according to the IEC standard, one petabyte is 10^15 (or 1e+15) bytes, while one pebibyte is 2^50 (or 1.125899e+15) bytes.
So 144 petabytes is 1.44e+17 bytes or 127.89769 pebibytes. Can't say that's more impressive tho. :P
Maybe it's the same thing as with billions? (10^9 while wearing a gun, but 10^12 while smiling into a CCTV camera)
From Kill-a-byte to Tear-a-byte to Pet-a-byte.
How many of these files can Linux support? Is there a limit on the size of the disk that can be used?
144 Petabytes should be enough for everyone!
Hmm.. wait a second...
BTW, it may also re-open the debate:
Trolling using another account since 2005.
How much plaintext is that?
Enough to store the Library of Congress in a single file?
Chas - The one, the only.
THANK GOD!!!
I'm already fed up of the time it takes to back up large disks to tape. Drive transfer rate has not improved at the rate of disk capacity in the last few years and is becoming a bottleneck. It was unimportant when the backup time of a single disk was well below one hour (our Ultrium tapes give about 40Gb/hour).
Just figure that if you want to transfer 144PB in about one day, you need a transfer rate of the order of 1TB/s. Electronics is far from there since it means about 10 terabits/second. Even fiber is not yet there. Barring a major revolution, magnetic media and heads can't be pushed that far. At least it is way further than the foreseeable future.
Don't get me wrong, it is much better to have more address bits than needed to avoid the painful limitations of 528 Mb, 1024 cylinders etc... But, as somebody who used disks over 1Gb on mainfranmes around 1984-1985, I easily saw all the limitations of the early IDE interfaces (with the hell of CHS addresses and its ridiculously low bit numbers once you mixed the BIOS and interface limitations) and insisted on SCSI on my first computer (now CHS is history thanks to LBA, but the transition has been sometimes painful).
However, right now big data centers don't always use the biggest drives because they can get more bandwidth by spread the load on more drives (they are also slightly wary of the greatest and latest because reliability is very important). Backing up starts to take too much time,
In short, the 48 bit block number is not a limit for the next 20 years or so. I may be wrong, but I'd bet it'll take at least 15 years, perhaps much more because it is too dependent on radically new technologies and the fact that the demand for bandwidth to match the increase in capacity will become more prevalent. Increasing the bandwidth is much harder since you'll likely run into noise problems, which are fundamental physical limitations.
<Insert Poster's Name Here>
<Insert Sig Here>
The real advance here is that the disk drive weenies have at last realised that they need to come out with a real fix for the 'big drive' problem and not yet another temporary measure.
Despite the fact that hard drives have increased from 5 Mb storage to 100Gb over the past 20 years the disk drive manufacturers have time after time proposed new interface standards that have been obsolete within a couple of years of their introduction.
Remember the 2Gb barrier? Today we are rapidly approaching the 128Gb barrier.
What annoys me is that the disk drive manufaturers seem to be unable to comprehend the idea of 'automatic configuration'. Why should I have to spend time telling my BIOS how many cylinders and tracks my drive has? I have a couple of older machines with somewhat wonky battery backup for the settings, every so often the damn things forget what size their boot disk is. Like just how many days would it take to define an interface that allowed the BIOS to query the drive about its own geometry?
Of course in many cases the figures you have to enter into the drive config are fiddled because the O/S has some constraint on the size of drives it handles.
We probably need a true 64 bit Linux before people start attaching Petabyte drives for real. For some reason file systems tend to be rife with silly limitations on file sizes etc.
Bit saving made a lot of sense when we had 5Mb hard drives and 100kb floppy drives. It does not make a lot of sense to worry about a 32bit or 64 bit file size field when we are storing 100kb files.
If folk go about modifying Linux, please don't let them just deal with the drives of today. Insist on at least 64 bits for all file size and location pointers.
We are already at the point where Terrabyte storage systems are not unsusual. Petabyte stores are not exactly commonplace but there are several in existence. At any given time there are going to be applications that take 1000 odd of the largest disk available in their day. Today that means people are using 100Tb stores, it won't be very long before 100Pb is reached.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
I figure that at ATA-100 speeds, it would take 49 years to read the entire file.
144 * 2^50 # n bytes
/ 100 * 2^20 # bytes/sec ATA-100
= 1.44 * 2^30 # n I/O seconds
/ 60*60*24*365 # ~ secs/year
= 49.03 # n I/O years
That's what it's for!
You back up the entire internet on the drive, and you can search in it using grep.
It's what we've all been looking for, isn't it...
Oh thank god!
I was afriad that my ever-expanding-highly-illegal-eleet mp3 collection would soon be bigger than linux could handle!
This should do it - for a while!
144 Petabytes doesn't sound like a lot. When putting it into writing:
144,000,000,000,000,000 or 144*10^15
it's impossible to comprehend.
Here's a way to visualise it - although it's also mindboggeling:
Take a sheet of paper with the squares on it. If you put a single byte in each 5mm by 5mm (1/5" by 1/5") square and use both sides, you'd need:
3,600,000 km^2 of paper to have room for those 144 PB. That's roughly 1,325,525 square miles for you people who don't use the metric system.
So when people say "it doesn't sound like a lot", you know how to get them to understand that it really IS a lot.
We do not live in the 21st century. We live in the 20 second century.
"The 144 Petabyte figure is obtained by raising two to the power of 48, and multiplying it by 512. A big arse number."
How did this fellow become a journo?
Who cares about this except jerkoff Linux fanboys that have no use for such a feature anyway? Um, there's a reason Linux is in front here--no one else has been dumb enough to waste time implementing a similarly useless "feature".
Yeah, I know, what a rich tapestry of vision, solidarity, bazaars vs. cathedrals, free beers, etc. this represents. Whatever, blah blah. Instead, let's spend our limited time adding features to Linux that fscking matter, please.
The only certainty is entropy.
Not to ask aa stupid question, but can anybody think of a reason at their point you'd need a petabyte file? Other than a databse of everyone on earth's DNA, or everywebcam in the world being saved to one file, I can't think of a sginel use.
Mod point free since 2001
Keeping an archive of Slashdot. As the solar system's population grows and grows, it won't be long before every little news story gets a thousand comments per minute. There will be so many moderators that law of averages suggests that every comment will be modded up to 5, and in an ironic twist Slashdot will be flooded. Still, it's Slashdot, and no self-respecting high-bandwidth nerd will be without an up-to-date archive of Slashdot.
Leeching Aminet. By the time we actually have these monster size drives, processors will finally be fast enough to properly emulate an Amiga, WinUAE will have been perfected and bandwidth will be so plentiful that we can all enjoy the latest Amiga software, whether we want it or not.
Freaking out newbies. Remember your scriptkiddie days when you would h4x0r some dude's Windows machine and pop up something resembling the Matrix? Simply add a little matter-to-energy technology, and you can download the newbie onto his computer, FTP him along (resumable downloading, now, we don't want him to materialise with missing parts!) and rematerialise him in your fridge. He'll think he's been transported to some crazy ice planet. Just like in sci-fi, eh folks!
Somewhere to keep all your Pokémon hentai! Don't try and hide it, man. I've seen your sick pictures of Misty and Bulbasaur.
You'll finally have enough diskspace to install Windows 2024. Naturally, you'll be using Linux instead, but it's nice to brag that you could, if you wanted.
Think *Cinema*.
Current codecs already do a pretty decent job of compression of smaller(resolution) streams. However, what if I want my linux box feeding my HDTV projector at high resolution? This might be one more step in my vision of the ultimate entertainment center.
You can't legislate goodness. Let each to his own destiny, by will of his freely made choices.
Of course, I won't be able to use the damn thing for years to come, but I could just have it around for bragging rights.
Oops, I can imagine the cops coming down on this one....
Suppose you copy at full PCI bus speed
You can cut a few years off that figure with AGP storage. Because most servers don't need excessive video performance, such systems can use an el-cheapo video card sitting on the PCI bus, leaving the AGP port open. Storage makers have developed high-speed storage solutions that take advantage of the insane throughput of AGP (1 GB/s and beyond).
Will I retire or break 10K?
I had a peta byte once, I shot it
This limit is for a SINGLE IDE disk. Now, if you use Logical Volume Management (which is in the standard 2.4 kernel, no patches required) you can combine multiple disks into one.
Since my machine has 2 IDE controllers, with 2 buses each, and 2 drives per bus, you could make a system with 8 144 pB drives, put an XFS partition on it, and have 1152.92 pB of storage.
And for meaningless statistics sake: I make my MP3s (from CDs that I own, thankyouverymuch) at an average of 160 kb/sec. At that rate, the specified drive array would store 1826693 YEARS of MP3s. None of which would be Brittany Spears.
www.eFax.com are spammers
I've sort of found it funny that as disk space goes up compression has been making files smaller. The amount we actually need to store keeps getting smaller while the amount of hard drive space people have keeps getting larger. If you were to honestly go through your archive of whatever it is that you archive and get rid of every file you haven't opened in the past 6 months you'd probably go down to less than 5 gigs anyhow. Of course not everybody. Some people run 24 hour a day porn slide shows.
M$ Word 2003 XP/Professional Extended Special Edition
... I'll just sit here and wait a while ...
...
10 OPEN "c:\bigassfile.txt" FOR APPEND AS #1
20 WRITE "Fill 'er up!" & chr$(13) & chr$(10), #1
30 GOTO 20
... tumteetum
How completely hypocritical...must be extremely boring since you responded! :-)
Derek Greene
What about the file systems for such an amount of data? I think FreeBSD's FFS has had petabyte capability for quite some time now. I don't know aboud FBSD's disk driver support for it though..
http://www.freebsd.org/news/newsflash.html#2001Nov ember3:1
... look at that 48 bit addressing ATA drivers are now working? Wow... maybe FreeBSD people should run around making bogus claims too. FreeBSD invented the Question Mark! Wooo hoo.
Hey
I also think you can use Vinum to mount such a petabyte sized file system fairily easilly.
Really FreeBSD doesn't get enough credit for work that's been done. I know linux has a lot of good marketing for technical features but you also have to believe everything you read to fall for it.
Seems to me, if this support comes to Work in the SCSI subsystem, then it will be an exciting thing for EMC who is about the only Company capable of Building such a drive array. And Oracle who is like the only company I can think of that could fill such an array with data at the moment, anyway.
In practical Terms how soon could we reach this data size...Humm, EMC's largest Cabinet holds 384 Disks...right now the largest single drive I have heard of anyway is 160GB, but I keep hearing rumors of a Seagate 256GB Drive on the near Horizon...so lets play with that number as the practical largest for the forseeable future(about 10 Minutes in Computer Terms) 384 drives * 256 GB/1024 = 96 Terabytes...we have a long way to go at the moment!
Power Corrupts,Absolute Power Corrupts Absolutely, leaving one person(group)in charge is absolutely corrupt.
Where does this go, on a petabyte file?
how about lseek(handle,MAX_LONG,SEEK_SET);
My point is that the location is specified by a long, which on intel systems is 4 bytes. It doesn't reach past this, so what good is a petabyte system that allows larger files then can be accessed?
I must be missing something. Anybuddy with a clue out there?
That is not waht the artical said it said it was the first to support more that a 100 patabyte file. But they were not really clear as they seemed to run the two things to gether. Plus linux has had 48bit ATA support for awhile as well.
One day people will learn the folly of Winbloze, Linux Rules!
I was just doing an order of magnitude estimate to illustrate the point that scientists will use as much computing power, memory, storage, etc. as they can afford. I wasn't paying attention to factors of 6 here or there.
Obviously, there are more relevant issues. For example, how are you going to store X bits of information using Y particles? At least for classical computing, you have a problem if Y is orders of magnitude less than X. Hence storing 8 numbers for each atom in the galaxy would be impossible if you were confined to using only the atoms on the earth, at least in classical computing. (I beleive with quantum computing in principle you can be clever and get around this, but I don't know enough to say for sure.)
But to answer your question since over 70% of the baryonic matter is hydrogen, nearly all the rest is helium, and less than 2% is heavier, the average molar mass of baryonic matter in the universe is less than 2.
Thanks! FreeBSD is sometimes like Rodney Dangerfield. "No respect."
Actually, I think the confusion about what open source is lies firmly in the collective mind of the open source community itself. On one end of the spectrum, you have people who like open source because they simply like to hack on whatever feature they personally like, as you've mentioned. Fine. On the other end of the spectrum, there are those like me who hope that open source is a method for creating self-organizing software projects with the ultimate goal of those projects being better than they otherwise could be.
Of course, all the confusion arises from the definition of "better"--what is the proper fitness function for open source software in general, and Linux in particular? For me, I see the complete lack of coherency in Linux and many other open source projects as their fatal flaw. People want to have Linux be a hackers paradise, yet topple Microsoft's OS monopoly by supplanting the desktop. Think about it, are these two goals really compatible?
The former requires no vision whatsoever of what Linux should be, and in fact thrives on its absence. The latter requires almost nothing but vision--the technical aspects are trivial. Unfortunately, the former goal tends to undermine efforts for the latter, and I frequently wish the Linux community as a whole would wake up and see what's in all of our collective best interest. Let people dick around with useless features in their spare time, but for God's sake, please have them more than that! Applause for the sort of nonsense described in this article just encourages more crap, more wasted time.
It's very frustrating to want to see Linux go somewhere important, yet see trivialities picked up by a bunch of fanboys as something important. So your proud of Linux, big deal, get on with it.
Exactly. So let's see the fanboys excited by this news go out and make Linux a viable desktop OS, or shut the hell up about it replacing Windows.The only certainty is entropy.
First this that occurred to me was not astronomical numbers, but ones like these: the human brain. Assuming that a true AI will have approximately the same complexity, petabytes may just not cover it, quite.
yes, we have no bananas
You use the three 144 petabyte drives for RAID 5. Sure, you only get about 100 petabytes space that way, but you really increase your throughput, assuming that the controller can keep up with it. If you'd rather have the space, I guess you could do raid mirroring...
However, there might one day be information processing systems to which 1.44 petabytes is a small amount of information. In a sense, these systems will have a richer experience of the world than human beings. I wonder if human consciousness would seem marvellous or valuable to such a machine.
That's only about 177 years' worth of 640x480, 24-color, 30fps uncompressed video.
Sheesh. I at least want to be able to chronicle the entire history of mankind in uncompressed video on my Linux box. Right now I'll have to settle for the history of the Industrial Age, or split my documentary into several smaller files.
dinner: it's what's for beer
We use large files (one servey file can span 20+ 40Gb ddr tapes). We have multiple sets. We are a university (we don't play with the real thing - just the "toy" sets). We could easily fill several terrabytes with todays servey technology. New technology will likely increase the amount of data for any given servey at a polynomial or exponential rate. We don't need it today, but the old cluster may need it in the future to process (and store) the files.
Spell check? Why bother. That is what grammer/spelling Nazi freaks who waiste band width posting "spell right" are for.
Well... There was MULTICS, about 30 years ago it threated memory-segments and files the same way. Swapping and VM in one!
/. ate my <
It should have been:
7,686,810 km^2 < 2,175,600 km^2?
-no broken link
In the first paragraph, they state that Linux recently became the first desktop OS to support enormously large file sizes
and then went on to mention BeOS, which also has support for abnormally large files. Just because Linux has a theoretical limit higher than BeOS(about 5 times theirs), does not mean that suddenly linux is "Linux recently became the first desktop OS to support enormously large file sizes".
Or is BeOS (the most user freindly and consistent OS I've ever used, BTW, I can't wait for the BeOS Clone projects to get a working OS) not a "Desktop OS"? If it isn't then Linux REALLY isn't!
It's been a long time.
what's 144 petabytes over a cable modem?
(kidding, for all the humor impaired)
A: None. The Universe spins the bulb, and the Zen master merely stays out of the way.
The point is, if I have the choice, I will choose to develop on a system where I have access to the source, for a number of reasons, only partially technical. There is no "collective mind". Developers are highly independent and like to work on what interests them. If you're interested in reaping the rewards from something, you sometimes need to earn them. Whether this is actually contributing code, funding development, etc.
Open source platforms were created by hackers, for hackers. And typically we don't give a damn about widespread acceptance or overthrowing microsoft's dominance of the desktop. We just want something that works well for what we need. Try to understand it from that perspective and you'll do better.
People want a lot of things, but the only people who really matter here are the people implementing this system. See, thats the great part, if you want it to be something it's not, make it that way. And personally, I do think they're possible. If it weren't for legacy applications, Linux would likely be on a lot more desktops than it is. I know any clueful sysadmin would much rather maintain a bunch of linux boxen than windows boxen. From a management perspective, Linux is lightyears beyond windows. Especially considering if something doesn't work right, instead of looking for a kludge or trying to get a vendor to include the needed functionality (usually a combonation of the two), you can locate the problem, isolate it, and correct it. I know of at least one place I've worked where this ability would have saved the company literally millions of dollars. Theres nothing wrong with cheerleaders to keep the team motivated. As a matter of fact, if you really think about it, recognition is the sole form of payment quite a few oss developers receive. Its all about the right tool for the job. If you want to play the latest and greatest games, linux isn't a good desktop choice for you. For the people maintaining 5000 corporate PCs with custom apps, it becomes a very sensible desktop OS.Personally, I run FreeBSD and a mix of NT/2000. Windows is still a requirement for me (a couple of addictive games, and some apps that my job requires). And the majority of the time I'm in windows, I have emacs/tcsh/python windows up (Exceed is a dream here). I personally would LOVE to get windows off my desktop, but it's the applications that keep me there, applications are key.
Thanks for the thoughtful reply :)
no, they said "The first desktop OS which supports enormously large filenames" basically, though I may have a few words off, the meaning remains intact.
It's been a long time.
Patrick Doyle
I mod down every jackass who puts his moderation policy in his sig. Oh, wait a sec....
This is just geek fodder....Hey, we're using 48bit addressing. Which means you can have 10 petabytes of pr0n now!!! It just sounds cool is all, it doesn't mean anything practical.
But, impracticality is much more interesting, isn't it?
1024 bytes = 1 kilobyte
1024*1024 = 1 megabyte
1024*1024*1024 = 1 gigabyte
1024*1024*1024*1024 = 1 terabyte
1 petabyte is 2^50 bytes or:
1,125,899,906,842,624 bytes!
so 144 petabytes would be...
162,129,586,585,337,856 bytes
not
144,000,000,000,000,000 bytes
or 144115188075855872 bytes (2^48 *512)
I don't know where they get 2^48 * 512?
if it really is 2^48 * 512, well, I'm sorry but
that is only 128 petabytes!
A lot of people are complaining about backups, if I had such a system, I would have no conventional backup plan, such as to tap or CD. I would RAID/mirror the heck out of my disk arrays and put them in a huge fire resistance/magnetic resistance cage.
------ Curiosity killed the cat. {satisfaction brought it back | it didn't die ignorant | lack of it is killing mankind
Would be nice if they put the extra bits in where they could do some good.
if (!scp->r_io)
...
goto failure;
BASIC kicks ass
codegolf.com - smaller *is* better.
Australia is counted as a continent and not an island. If you count australia as an island, why not antarctica or north or south america?
We do not live in the 21st century. We live in the 20 second century.
Yet not just IMHO only. The current disk layout, still using platters and sectors and all such just is not up to scratch anymore. Of course there have been minor improvements, yet years ago already studies were done to simulate other layouts at some German institution and yes, a different approach would improve performance. In short a TB, PB or whatever EIDE hack may sound nice, yet it is absolutely worthless. If you need something decent, than do not be skimpy and EIDE was designed with cheap in mind. Even SCSI is going to be a far cry from being good enough to handle such an amount of data in the future.
Why do you care what the BIOS thinks? Set it to NONE and be done with it.
Any "modern" OS doesn't use what the BIOS thinks anyway. Try it with Linux sometime. Stick a 60 gig disk into an old 486 that can't handle it, set it to none, boot up Linux and watch it tell you that there's a 60 gig disk there, and more importantly, watch it WORK in all respects. Watch it have full access to the whole thing. Be careful, if the BIOS is set to something other than NONE, it *can* lie to Linux when the kernel asks for the size of the drive. But I've done this with systems that have a 32 gig limit on disk size, and have it work just fine.
Windows can do this too, sometimes. Not exactly certain on the details there, but having done this myself with Linux, I know that much of it does work.
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
Most large internet companies produce already a couple of terrabytes per day. Thinking that the 144 Petabyte is about 144.000 TerraByte, it would be barely big enough to hold more detailed information.
EMC is already planning in the next comming 5 years to have a Petabyte system running as a central databse for all information.
To big will soon be already to small.
And filling it up wouldn't be a problem since this will happen in parallel over thousands of computers.
Think as of it as 576 Milion people having 250 MByte of online diskspace.
No single disk can support 48 bits of data, so the statement that Linux has broken some sort of barrier is rather academic. In any case, OSes with 63 bit addressing effectively get large file capability by spanning multiple disks merged into one logical striped/concatenated disk. This isn't anything new, and Linux isn't the first.
That single drive could hold... everything!
Let's all pitch in and buy a big fat bandwidth pipe and fancy hardware interface and an array of these drives and we can store everything we want.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
All this talk about PETA and bites is amking me hungry. Anyone want to go out for a burger?
It's impossible to conceive of Linux _needing_ that big a hard drive. But think of how fast Microsoft code bloats. Every few years M$ has to invent a new file system to properly handle the larger drives needed to hold Windows & Office. And so who knows how big common disk drives will be in 10 years? But Linux is ready NOW... ;-)
now i can create the worlds longest porno video!
BeOS supports 18 exobytes.
I'll buy this. I was erroneously under the belief that New Zealand was part of the Australian continent, but not the Australian country. Plus I bought the marketing stating that Australia is the island continent.
I find it kind of odd that New Zealand isn't part of a continent.
-no broken link
From The Beast:
That's a 64 bit addressing space, and yes, it's supported on the consumer desktop OS that's been shipping by same for just a little while now.
is freebsd even considered a desktop os? I always hear about how great its network stack is, but as far as drivers go, (even proprietary ones) I figured it was better to consider it as a server only os. I mean the ports system is great and all, but anyone using freebsd is going to end up compiling all of their apps anyways. I don't really think freebsd is a desktop OS, but is a strong industrial strength server os, as most bsd zealots like to point out how great the network stack is. Now if they could only improve the smp scalability.
There are seven continents:
Africa
Antarctica
Asia
Australia (sometimes called Oceania)
Europe
North America
South America
From Atomica.com:
Australia
"Australia, smallest continent, c.2,400 mi (3,860 km) east to west and c.2,000 mi (3,220 km) north to south, only continent occupied by a single nation, the Commonwealth of Australia (1995 est. pop. 18,322,000), 2,967,877 sq mi (7,686,810 sq km). Subdivisions of the nation include the offshore island state of Tasmania; the five mainland states of Queensland, New South Wales, Victoria, South Australia, and Western Australia; and the Northern Territory, Australian Capital Territory (containing Canberra, the federal capital), and Jervis Bay Territory (until 1988 part of the capital territory). External territories include Christmas Island, the Cocos (Keeling) Islands, the Coral Sea Islands, Norfolk Island, Heard and McDonald islands, and the Australian Antarctic Territory."
Oceania:
"The islands of the southern, western, and central Pacific Ocean, including Melanesia, Micronesia, and Polynesia. The term is sometimes extended to encompass Australia, New Zealand, and the Malay Archipelago."
You could also look up continents in your own favorite encyclopaedia. Here's what Atomica had to say:
"The large parts of the surface of the earth that rise above sea level. The seven major continents are Africa, Antarctica, Asia, Australia, Europe, North America, and South America."
Of course, this IS an online encyclopaedia, so it has to wrong, but feel free to check with any three somewhat new (younger than 1980) encyclopaedia, and you will probably find, that the above holds true.
So, if you find it odd, that New Zealand isn't part of a continent, it's because it's not located on the Australian continent but probably on some minor continent.
And before I forget:
Greenland:
"An island of Denmark in the northern Atlantic Ocean off northeast Canada. It is the largest island in the world and lies mostly within the Arctic Circle. Inhabited by Inuit peoples as early as 3000 B.C., it was discovered by the Norwegian navigator Eric the Red in the tenth century A.D., became a Danish colony in 1815, and was granted home rule in 1979."
Also 'island':
"island, relatively small (compared to a continent) body of land completely surrounded by water. The largest, in descending order, are Greenland, New Guinea, Borneo, Madagascar, Baffin Island, Sumatra, Honshu, and Great Britain. Islands are either continental, caused by partial submergence of coastal highlands or by the sea breaking through an isthmus or peninsula, or oceanic, originating from the ascension of the ocean floor above water through volcanic activity or other earth movements. Tropical oceanic islands sustained above sea level by coral growth are called atolls (see coral reefs)."
Repeat after me:
"I will double check my facts, so I don't make a fool out of myself."
Now of course, you did get your facts from a commercial and as we all know, those are never wrong. I'll make sure that Encyclopaedia Brittanica and National Geographics are made aware of the fact, that Australia is an island and not a continent.
We do not live in the 21st century. We live in the 20 second century.
This would be a much better article if the headline read "IBM breaks 100 petabyte barrier." Or Maxtor. Or Western Digital. Or perhaps Quantum. See what I mean?
The Register updated their article. It now acknowledges FreeBSD as being the first Unix to support multi-petabyte filesizes.
However, NTFS 5.0 (the filesystem that is used by Windows 2000) has had 64-bit addressing since Windows 2000 was released. This yields a maximum capacity of 16 exabytes, which is 8388608 Petabytes. That's right, Windows has supported files eighty thousans times larger than Linux with an experimental patch for the past few years. Still, by the time people actually start needing this kind of storage, I don't think it'll actually matter much...
Lets see:
FreeBSD is core to Mac OS X. How many people claim Mac OS X is NOT a desktop OS?
GNOME don't ship unless it compiles on FreeBSD.
(Is GNOME considered a 'server' product?)
You *DO NOT* have to 'compile all your Apps'. Packages takes up 4.7 gig. 4.7 GIG of pre-compiled stuff.
Finally: FreeBSD - The Desktop edition
(and a reply to the troll about the SMP)
At least Linux FINALLY decided on a VM system. Just the other day, in fact. Given most of the Intel systems are one processor boxes, not 64 way SMP boxes, it doesn't bother me that SMP support with FreeBSD isn't up to the level of Solaris.
If it was said on slashdot, it MUST be true!
NTFS volumes can theoretically be as large as 16 exabytes (EB)
Never mind.
Wouldn't a file size of 1 PentaByte be kind of useless? I mean - it seems like that one file would be like a partition of it's own, inside that file you would have to have logical breaks to make it searchable kind of like seperate files... Just an idea.
Trying to wrap my head around that big of a file...
This space available at a low monthly rate...
In the article it stated that FreeBSD had the changes checked into their CVS tree on October 6th. Linux did not have the updated drivers until November 5th. That's a whole month ahead.
An entire day of a human life could be recorded in perfect detail (with no compression) on a 120 GB disk.
Let me guess - you're using roughly the bitrate of DVD, extrapolating over 24 hours, and fudging the numbers. Well, either you or whoever came up with this figure.
Let's look at this from a cocktail napkin perspective. At the CURRENT resolution and audio sampling rate etc for DVD, 24 hours is about 50GB of storage. Only problem is, this assumes that DVD catches every single bit of visual/audio information that is out there. Well, just ask your dog how well 44Khz records high pitched noises. And then remember that not everyone has as poor eyesight/hearing as the masses. So even fudging this number by a factor of 2 or 3 starts to hit and overtake 120GB.
Oh wait, this assumes that all we care about is what the eyes see and the ears hear. Too bad that things are happening all around you. Also too bad that you have 3 other external senses, plus several other internal ones (balance comes to mind) that are continually inputting data into your brain.
Estimates like this really make me shake my head, as they assume artificial limitations that just aren't there in the real, ANALOG world.
Endless arguments over trivial contradictions in books written by ignorant savages to explain thunder in the dark.
All versions of Windows NT and Windows 2000 filesystems support a maximum filesize 16 exabytes ... now when was win nt released?
s /Q 93/4/96.ASP
http://support.microsoft.com/support/kb/article
Now THAT'S a LOT of pr0n!
Whats the big deal about something like this? So, Linux can hold files that are larger than any hard drive is, or will be for a while now. Not even the largest databases in the world are this big, with the exception of maybe the U.S. Governments. Other than that, I see no reason to make such a big deal over something that doesn't mean much. Maybe 100 years from now this will be useful.
-Vic
I can run Windows XP (c)
After several large cups of coffee and a bit of luck i did some maths, and worked out a feasible answer.
then i threw that away and just guessed a random number.
so 144 petabytes, thatd be one 5minute track at... ooh....
230584300921369395.2 kb/sec?
sounds good to me :)
Now Microsoft has no excuse not to port the next version of Office to Linux.
so would this make linux an OS for petaphiles?
Yeah, that deal where pointers are assumed not to need segments or offsets - that kind of puts a damper on the ability to provide userspace with more than 32 bits of space at a time. Even though the Pentium Pro supports 36-bit addresses, there's no way you could use these directly in user space in Linux, due to this assumption.
I'd be rather surprised - at least if anyone is trying to do this with Linux. It's not like 64-bit CPUs are either expensive or hard to find these days - MIPS R1x000, Alpha, SPARCv9, POWER3, PA-RISC-2.0w, take your pick, or IA-64 if you swing that way. Of these, at least the R10000, Alpha and UltraSPARC have been out for several years.For applications where you really need to be able to address more than 32 bits at a time, there is really no excuse to use a 32-bit CPU. Application compatibility? Nah, we're talking specialised stuff, not off-the-shelf. OS availability? Choose any major Unix brand, or OpenVMS. Money? Your memory and secondary storage requirements will dwarf your investment in the CPU and mobo. Company politics / policy? Not a good enough reason IMHO, which of course is easy for me to say....
<flame>Oh - ever heard of demoronizer?</flame>
"How can you claim that you are anti-crack, while still writing a window manager?" — Metacity README
Well, actually it's not file sizes; FreeBSD has had 64-bit file size forever, and Linux has that too thesedays I believe. That's a completely separate issue to the ATA driver being able to address 48-bit physical media, which is what the support is for. And actually the Linux ATA driver maintainer only released these patches a few days ago (and they're not yet integrated into any of the kernel distributions), so they are much more recent than FreeBSD's support.
File sizes
Filesystem sizes
Partition sizes
Physical media sizes {-- this is what we're talking about: the limitations on these four things are more or less independent
Whoo time to NFS mount my brain through that gigabit ethernet port wired into the back of my head!!! Linux can read my mind for another 20+ years without needing to break it up into multiple files whoo hoo!
I wrote a simple file system that supported 256-bit addressing. I can support 2^256 byte files.
The question is: is it useful? NO. Duh.
Might i just point out that, holy christ on toast, dose anybody actually have files this big? i can imagine *my poor brain!* a database that might extend to this size but a single file?
Pirates go ARRRRRRR, Dinosaurs go RRRRRRRRRR
Yah, Yah, Yah, but what I really want to know is "What is the largest array of disk drives attached to a single linux system that is operational somwhere in the world today?"
I see. Being that by your research mainland Australia is actually smaller than the continent (as it includes the offshore island state of Tasmania), the mainland is an island. Thanks for pointing that out.
-no broken link
A slight correction:
s /Q 93/4/96.ASP
Remove the space between the "Q" and the 93 - this is in error, and interestingly enough that is *exactly* where the textbox for this submission to slashdot cuts-off and puts the rest on a new line.
If I were a linux-o-phile and was posting something on a predominantly-Microsoft site, I'd start crying "Conspiracy" because my link got modified by the site's posting engine.
The link:
http://support.microsoft.com/support/kb/article
Linux: the Academian's choice - and that's why it will *never* be accepted and used in mass quantities in the business world, and the only reason that it has made any kind of foothold in the business/corporate environment is that the people that implemented it in those corporations have been only out of college for 4 years or less and want to feel "comfortable" while working in the stress-ridden corporate world.
Peace, y'all
ScottKin
I don't give a rat's behind about "karma" here or anywhere else. Don't like what I have to say here? Deal with it!
And you obviously don't get it.
By definition (you know, something raised to be a fact) Australia is a continent. By some weird coinsident, the country covering the entire mainland of this continent happens to have the same name.
If a person happens to be called, say Washington, does that mean, that if he moves to England, that the city and state moves as well?
If you could present individual research, showing that my research (which indeed 99.9999% of the established scientific community supports) is false, then by all means do so. Like I said in a previous post, I'm quite sure that the likes of Encyclopaedia Brittanica and National Geographics would love to hear your reasoning.
But to elaborate on just why Australia is in fact not an island, here's my guess (an educated guess at best):
The crust of the earth is made up by several tectonic plates, seven of which are so called major continental plates, since they have large protruding landmasses on them. The largest body on each of these continental plates has the pleasure of laying name to the continenetal plate. Since the country of Australia happens to cover the entire main body of it's continental plate, this continental plate is conviniently called Australia. Since an island by definition cannot be the mainland of a continent, this excludes Australia from the running as the largest island of the world. If this definition of an island did not exist, the largest island wouldn't even be Australia (as it is the smallest of the continents) but Asia, which is the largest continent. Not only is Asia the largest continent, but it is also joined at the hip with the second largest continent, Africa, but also with Europe, giving this "island" a combined landmass of 84,994,050 km^2 - an impressive 62.5% of the entire landmass of the earth, 135,916,252.5 km^2. How's that for a big-assed island? But it's not an island, because a continent is defined as something else than an island, and guess what - Australia fits the definition of a continent. But hey - I'm probably wrong, and you can probably prove this by some other mean than useless rhetoric like, say your own small piece of recearch. But let me guess - it's too hard to find any evidence that Australia isn't a continent, since there are seven continents, and Australia just keeps being listed as one. Go figure.
It's better to keep your mouth closed and be thought an idiot, than to open it and prove it to be true.
We do not live in the 21st century. We live in the 20 second century.
The fact is that I agreed with you two posts ago. That's because I looked up the definition of island and it said in two different sources (dictionary.com and m-w.com) that its a land mass surrounded by water that is smaller than a continent. But then you responded to me saying that australia, the country is made up of more than the mainland, and that continent contains the country. It's still very vague as to what the continent really contains. If it just contains the mainland, then the mainland isn't smaller than a continent (it IS a continent). But if it includes more than that is is smaller than a continent, and thus an island. Your Eurasia theory doesn't fit because it isn't smaller than a continent (indeed bigger than one). Besides, I never claimed that Australia was the biggest. I just claimed it was bigger than Greenland.
-no broken link