Slashdot Mirror
Ext4 Filesystem Enters Experimental Kernel Tree
An anonymous reader writes "Looks like the next version of the venerable Linux 'ext' filesystem is just around the corner. Andrew Morton has added an early version of ext4 to his 2.6.19-rc1-mm1 tree, enabling Linux to support storage volumes up to 1020 petabytes in size, and to write files in 'extents,' or contiguous, reserved areas. According to an article at Linux-Watch, ext4 will be ready for production use within six to nine months, if all goes well. On the downside, the new ext4 filesystem will offer only limited backward compatibility with ext3-aware Linux kernels."
Unfortunately, this will just murder Reiser4.
1 Exabyte!
Not to be confused with Excitebike, which is something entirely different.
The Slashdot Paradox: "100% Overrated"
EXT4-fs warning (device sdb1): ext4_journal_start_sb: Detected tasteless ReiserFS jokes - hahahaha!
biopowered.co.uk - catalytically cracking triglycerides for home automotive use since 2008. Just say no to big oil!
By now you don't even now what to do with 1024PB. Just as we couldn't imagine filling a 250GB harddrive 15 years ago when 500MB were considered huge.
What will happen? We store our digital photos in raw format, not JPEG. We store our songs in raw format, not artificially crippled. We will store high-definition video, possibly even in raw format, not MPEG4 or the likes.
And, woosh, 1024PB will be nothing leaving us wondering how we could ever survive with a measly 250GB drive -- just as we ask ourselves today how life was with nothing but 170kB disk drives.
Wake me up when chunkfs hits the kernel. I don't even want to think about fscking all those petabytes ...
It could make doing backups interesting too.
>> 1020 petabytes
My porn collection will now be complete.
Lindsay Blanton
RadioReference.com
How does ext4 perform when compared to, say, reiserfs 3.6 or 4? What new features there are?
Who is John Galt?
I thought file systems were typically constructed in, for and with software.
My question is why they don't mention why it is better to use ext4 then XFS.
XFS can do 9 exabytes (exabyte = 1024 petabytes).
They mention that ext4 is not faster than other filesystems.
Ofcourse people can do whatever they want, but why not spend their time making XFS easily resizable for example?
If you mod this up, your slashdot background will turn into a beautiful sunset!
So the solution is to limit the storage capacity of computers, to ease the work of keeping data organized? Wow, why didn't anyone think of that when we had 500 MB hard disks? Organizing data would be a breeze for us now; but no, instead we are stuck with our large capacity systems.
....now I can make use of those 1400 1 terrabyte drives that I have sitting in my basement. I knew they'd be useful for something someday. How long until the RIAA sues because Ext4 will allow for even MORE music to be illegally downloaded and stored?
enabling Linux to support storage volumes up to 1020 petabytes in size
Now, is there anybody who still believes that porn does not drive innovation?
From lwn current issue(you have to subscribe for the full article ;):
Also merged is the developmental ext4 filesystem, which includes a number of enhancements, including support for extents and 48-bit block numbers. See the ext4 documentation file if you are interested in playing with ext4 (and have good backups).
Will we back all this data up?
I'm honestly more interested in someone coming up with cheap, long term archival storage. Hard disks have gone so far past our ability to archive information it's beyond comprehension.
Clear, Dark Skies
This is exactly how I'm filling up my space. I got a new computer with a 160 GB drive, thinking it would be "enough". Started storing all my CDs in FLAC, and I'm currently transfering all hte movies I downloaded in AVI to DVD so I can watch them easily on my home theatre. Once you start working with video and sound that isn't compressed to nothing, you start to realize just how fast you can use up all that space. If my camera did RAW i'd probably use that to store my photos. I usually save any edits I do in PNG or TIFF so that I don't have to worry about the lossy encoding. Granted I still have space to spare, but I could see very easly using up a Terabyte drive if I had it, and a faster internet connection.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
Meh.
find "$dir" -amin +"$time" -print0 | xargs -0 rm -fr
Please correct me if I got my facts wrong.
Who needs ext4? I'm perfectly satisfied with my 640k, and so should you!
Well, that depends on what your expectations for the future are. I don't think it is impossible that demands on multimedia will reach high enough sometime. Let us as an example consider a movie file from the Future (tm). Given better and bigger screens (perhaps covering whole walls) a frame dimension of 3000 x 2000 pixels is not inconceivable. Each pixel might consist of three RGB values of 16 bits each. Such a movie, if two hours long and running with 25 frames / second, would require about 6.5 TB in raw format.
framedimensions = 3000 x 2000
framebytes = framedimensions x 6
moviebytes = framebytes x 25 x 60 x 120
moviebytes / 10^12 ~= 6.5
"and to write files in 'extents,'"
Aright! 1970s mainframe technology, here we come!
Coder's Stone: The programming language quick ref for iPad
to no longer use ReiserFS as its default FS (orig. reported on OSNews.com...don't think I've seen it here yet). I think this came out before the whole Hans Reiser affair, BTW.
SuSE contrasted the ease of upgrading ReiserFS and ExtFS versions:
Carousel is a lie!
What I don't onderstand is that this is merged into the 2.6 kernel tree today. What has happened to the concept of -stable (2.6) and -experimental (2.7) trees? This would be aperfect opportunity to open the next experimental branche..
To Terminate, or not to Terminate, that's the question - SCSIROB
Interesting.. That'll be UHD-DVD then
You don't need to store your audio as FLAC. Only weirdo audiophiles do that [unless you own the masters then it's ok]. Use 192 to 256kbps M4A and be done with. As for the movies, you should be able to convert ON THE FLY to DVD so you don't need to store uncompressed frames or whatever. As for the camera, just use high bitrate JPEG.
Unless you are publishing your audio, videos or photos you don't need a 100% representation. Often that 97% quality but 1/10th the size copy will be fine for your own enjoyment and use.
Tom
Someday, I'll have a real sig.
You forgot the 16 channels of 192KHz 32-bit audio you need to. That's 84GiB on it's own!!! :-)
Tom
Someday, I'll have a real sig.
Other Reiser issues aside, the SuSE folks at Novell are looking to leave the nearly unsupported reiserfs3 (in maintenance support, which isn't enough for them) and move to ext3 as their default FS. Why? They feel ext3 is a lot more mature & better/wider supported then reiserfs4, is an easier migration, and appreciate that there is a solid roadmap from ext3 to ext4.
Of course this would also be the week that (coincidentally) Andrew Morton gives reiserfs4 the green light for eventual mainline kernel inclusion.
I don't read ACs: If a post isn't worth so much as a nom de plume to its author then I wont bother either.
The limits are set by our senses, more concretely, our ears and our eyes.
Our ears are only capable of hearing up to about 20Khz (less than that for most people) and 16-bit samplings are enough that most people cannot hear the difference with anything more. Thus CD-quality is, if not perfect, then good enough that further improvements are ignorable for most people. CD-quality losslessly-compressed music is around 300MB/hour.
In a year, there's 8760 hours, so you'd need on the order of 2.5 TB to store a year worth of around-the-clock never-repeateing losslessly-compressed music. If computers keep getting replaced at the current rate, this means you'll never need more than about 10TB to store sound. This assumes you don't store more than you listen to, if you choose to for example store all music ever produced for convenience, despite never listening to more than a tiny fraction of it, then this requirement goes up by a couple of orders of magnitude. Still, there's good reason to suppose that 10TB will suffice for most peoples sound-storage needs. (even if you wanted to store all the sound you've *ever* heard in your life, including traffic at nigth, that'd still only be 200TB or so)
The real killer is video. We can take in a *lot* more data with our eyes. 10GB/hour is in the ballpark of what you'd need for the sort of quality a modern cinema can deliver. (and there's no particular reason we couldn't go higher.) That works out to 100TB/year, more or less. A lifetime of high-quality video is thus on the order of 10PB.
In short, it is unlikely that an individual (or family) will be able to fill a 1000PB disc with sound and video-recordings. Infact it's unlikely they'll be able to fill it with anything, if that anything is to be consumed only trough their 2 eyes and 2 ears.
That doesn't mean it won't happen. Only that it'll be filled with something more. Once we fire up the holodecks all bets are off. I don't even want to try to estimate the bandwith needed for that kind of immersive experience.
Just mirror it over the internet, you silly goose! Bandwidth is cheap(ish)
it's unlikely they'll be able to fill it with anything
:-)
People always used to ask me if I had the internet at home. Maybe when I can get my hands on a 1020 PB hard drive, I will be able to download it all for local access...
Any fool can talk, but it takes a wise man to listen.
I don't need to store any uncompressed video on my hard disk, but having a couple 15 GB of VOBs floating around that I haven't had time to burn yet takes up well, 15 GB. As far as i've found the easiest way to go from AVI (divx, etc) to DVD requires encoding to MPG, while the files are 3-4 GB, and then creating the VOB files from that, again 3-4 GB, and then burn those. I don't think I could go straight from AVI to burn, because it usually takes a few hours to go from AVI to MPG.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
Also, I store my audio as flac, not because I can hear the difference between 256 kbps M4A, but because i lose/scratch the CD, I still want to have a way to get a perfect copy back.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
VOBs are just MPEG-2 streams. Also nothing says you have to do your movie as one huge 15GB VOB. You could break it into 1GB parts and then delete the temps as you go.
...].
mencoder can transcode pretty much anything, unfortunately only into AVI [the MPG output sucks bad], but ffmpeg can do some other streams [notably though it doesn't like 5.1 AC3 streams though
Tom
Someday, I'll have a real sig.
If you can't hear the difference between a high bitrate m4a and the original ... does it matter?
That's the thing, the "difference" is only in your mind [not sensory]. Recall the whole point of psychoacoustic based encoders is they take advantage of the disparity of S/N ratios on various bands. If you have a 10dB masking on a given band, encoding it with full 96dB range [e.g. 16-bit PCM samples] doesn't make sense.
Imagine you can't see the colour red. Would adding more bits to the red channel make the picture better? If you can't hear that masked tone, adding more bits won't make it any better.
At the point where you are encoding at >=192kbps the encoder doesn't have to cut out resolution you can perceive so you're essentially home free.
Tom
Someday, I'll have a real sig.
Insightful? Yes. Informative? Certainly not. Finally the Funny mod hits (what took so long?) This is the funniest thing I've read on Slashdot in ages, on so many levels.
On second thought, maybe it is Informative, since I was not previously aware you could cram that many puns into so few words.
Karma: Incomprehensible (Mostly affected by posting at +5, reading at -1, and metamoderating everything unfair.)
If you can't hear the difference between a high bitrate m4a and the original ... does it matter?
Yes, it does, mainly for re-encoding. What if he wants to convert them to 128kbps for a portable player. Or what if in 5 years a different compression is used? Personally, I ripped about 120 of my cds to mp3 this year and it's not something I plan on doing again soon. I can completely understand someone never wanting to have to redo it.
My question is why they don't mention why it is better to use ext4 then XFS.
Simple: ext4 is a backwards compatible, evolutionary change from ext3, while XFS is a different file system and codebase. XFS doesn't offer sufficient advantages to overcome that built-in advantage of ext4 (after all, neither XFS nor ReiserFS managed to succeed even against ext3).
Now my p0rn won't be fragmented!
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
When you need to edit the damn thing, it is quite important to have it in a lossless format. If you loose 5 % everytime you save the file, by the fifth save the quality may be lost forever.
Use open source software.
... um ... if the 256kbps audio is INDISTINGUISHABLE from the original ... it might as well be the original. Sure the bits are different but the sound produced is perceptible as the same. I've re-coded stuff before for my portables and frankly I can't really tell the difference. Specially since you typically listen to portables in noisy environments where the noise floor masks most subtle noises.
As for going down in bitrate
Tom
Someday, I'll have a real sig.
This reminds me of a conversation I had about ten years ago with a coworker. One of us brought up the new 1-gig drives that had been released. He let out a slow whistle, and with a grin on his face, said "Boy.... that is big. It would take me at least three weeks to fill that up."
Oh, you're not stuck, you're just unable to let go of the onion rings.
84 GB is a drop in the bucket compared to the 6.5 TB of video data. Crazy, but true. :-)
Help! I'm a slashdot refugee.
Oh! If only they could have squeezed an extra 4 petabyes on so that we could get a true exabyte. What am I to do with my last 4PB of... um... art?
When our name is on the back of your car, we're behind you all the way!
Incidentally, what tools do you use to convert mpeg4 avis's to DVD? This is something I've been meaning to investigate for a while now.
When you lose something irreplaceable, you don't mourn for the thing you lost, you mourn for yourself. - Harpo Marx
It doesn't matter if you use OSS or Closed Source, it's a lossy format. Sure you might not be able to hear the encoding on a 256kbps audio file, if you've only just encoded it, but try transencoding a file from aac to ogg to mp3 all at 256kbps and see if you can't hear the compression.
FLAC is the only logical choice if you think you might have to reencode later. Or burn back to a CD, or remix the track, or make a home movie with it, etc. etc. etc.
"Build a man a fire warm him for a day, set a man on fire and warm him for the rest of his life."
It isn't nearly that simple. Oftentimes, in audio production, you work with audio that's much higher than 16-bit... maybe 24-bit, or often internally in software, 32-bit. Can I hear the difference between 16-bit and 24-bit? No. However, if you're going to be processing it heavily, you can hear the difference after it's finished. This is especially true when doing dynamics processing. The more bits an audio encoder has to work with, the better (within reason).
If anything I would say that your resolutions are way to low. For a full wall display of ultra high quality I would guess at 10000x4500 (for roughly anamorphic). I would also up it to 50 FPS for a really smooth display at 48 bpp a two hour movie is ~100TB for the video alone. Trouble wouldn't be storing it - it would be getting it off the device fast enough.
I used to have a better sig but it broke.
Regards,
Exabyte Corporation
By the time people have hard drives this huge, I bet there will be even better lossless compression algorithms available for video and audio data. I don't know why someone would want to store huge amounts of A/V data WITHOUT using some sort of compression. It just becomes silly after a certain point.
- Excitebike sounds like exabyte
- 1020 petabytes != 1 exabyte
- 1 exbibyte != 1 exabyte
- In terms of scale, 1 exbibyte and 1 exabyte are completely different, the difference in this kind of mistake could equal several hundred thousand copies of Wikipedia, as opposed to confusing KiB/KB or MiB/MB, which are different, but not earth-vs-sun different.
- The post was moderated Insightful and then Informative before it was modded Funny
- I'm counting the excellent sig too
That's 6, and I'm sure there are more I'm missing... Sorry if you think I'm stretching, but I really think this is one finely crafted layer-joke, and I just enjoy that kind of thing.Karma: Incomprehensible (Mostly affected by posting at +5, reading at -1, and metamoderating everything unfair.)
And what about going up in bitrate? Or switching to a different codec?
128kbit CBR MP3 used to be good enough for everybody and "indistinguishable from the original." Then it was 192kbit VBR mp3. Now it's m4a. What will it be next year? When will the next big lossy audio player roll around with an even better psychoacoustic model? Who knows, but I'm sure there'll be one.
FLAC means never having to re-rip my CDs. No matter what the next big lossy format is, I'll be ready for it. Considering that hard drives cost about $.50/gigabyte and I can fit about 3 losslessly-encoded albums per gigabyte, I opt to store them as-is and avoid throwing away data until I need to transcode my music for portables and so forth.
Ita erat quando hic adveni.
The article says "On the downside, the new ext4 filesystem will offer only limited backward compatibility with ext3-aware Linux kernels."
Ext4 is going to be the MOST compatible with Ext3, relative to ANY other option out there.
Upgrading to Ext4 is NOT going to involve a dump and restore from Ext3, likely a tunefs -j or similar command, just as the ext2 -> ext3 migration worked. Ext4 will be able to mount ext3.
If older versions of software could use the new format, you wouldn't need the new format. Yes, upgrading to Ext4 means your 120 petabyte raid array will not be compatible with your old "ext3 aware kernel". But it is PRECISELY because such an array is not possible under ext3 that ext4 is going to be introduced.
And does this submitter think other fancy new filesystems magically work on old kernels? Of course not. Does the submitter know if ext4 will be backported and made available to older releases? It doesn't look like they gave that much thought either.
Please read this for a more detailed description of what is happening.
Slashdot's always good for a smile.
"/Yeah, I can believe MS would commit murder to steal code."
...
REALLY??!
I saw it on Slashdot, it must be true!
Given petabytes of storage, we would take a very different approach to media. For example, we might make much more significant use of webcam capture, perhaps putting webcams in every room to catch anything interesting that might happen (especially for people with children). We also might capture everything that comes over the airwaves or television cable. We also might cache HD movies forever. We might move to a software caching mode whereby we have all of Microsoft's or Red Hat's software on our computer (auto-updated during the night) and we just turn it on when we need it. A lot depends on the ratio of bandwidth to disk space.
Think of it in terms of archive vs use format. If storage capacity is cheap, it makes sense to store all your media in the highest quality available (e.g. FLAC or other lossless if you rip from CDs, VOBs from DVDs). While MP3 and MPEG2 will be with us for a long time, when the codecs of choice for player devices change it makes sense to have the highest possible quality source available when you re-encode. It is perhaps even more important if you want to edit the media (trying to use YouTube or Google Video as source for a mashup is ugly).
With HD content available just around the corner, the storage requirements for full quality archive copies will go up a lot. A PVR set to capture digital broadcasts of all your favorite TV shows can easily fill a lot of disk.
(which reminds me - anyone have a howto on how to build a cheap and reliable Linux NAS?)
If J.K.R wrote Windows: Puteulanus fenestra mortalis!
Or do you mean murderers...
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Most Hollywood movies using digital negatives are burned to film at 2k (~2000 x 1250, varying slightly based on aspect ratio). Postproduction people might use 4k or 6k frames while working with the footage, but that's downsampled on output most of the time. Note that this is not terribly different from the size of a 1080p24 HD frame.
Movies are also generally scanned, mastered, and encoded in 10-bit or 12-bit per channel log. 16 bpc really isn't necessary, as at the top of the brightness range there's not a lot of subtlety in the colors anyhow. Again, post-production people might work with 32-bpc float internally, but there's no reason to have that much color definition in the output.
Also, movies are shot at 24 fps, not 25 (which is the PAL framerate). I'll spot you the 120 minute assumption to keep my answer in the same order of magnitude as yours, but it really should be 90 minutes.
So for most movies, (assuming uncompressed 1.75:1 2k frames):
framedimensions = 2048 x 1170
framebytes = framedimensions x 4.5
moviebytes = framebytes x 24 x 60 x 120
moviebytes / 10^12 ~= 1.8
Woo... snapshots. *blink*
(okay, so apparently yes, ext3 now comes with the 'lvm enhancement' patches that allow it to flush and lock operations during snapshot creation to avoid inconsistency after the fact)
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Your estimate is a bit off. People are already working on screens with horizontal pixel counts in the range of 8k. Let's assume we don't go any higher than that, and use a 16:9 format, so that's 8192x4608
We're also fairly likely to see 8 Byte/pixel screens, as 64 bit/pixel is an increasingly popular format.
60 frame progressive displays are also quite popular (already), so let's guess that we'll see recording at that speed not far into the future.
And a lot of movies run up to about 2.5 hours, so let's give 150 minutes to cover most movies without having to go to a second disk.
8192 x 4608 x 8 * 60 * 60 * 150 = 163074539520000 = 151875 GB = ~148 TB
I guess a 1020 Petabyte drive will hold a decent number of these.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
The thing about standards is that they can't just be good enough for "most people" they have to be good enough for EVERYONE to get traction. CDs will be around for a while to come, but I can see it being replaced with something better.
Higher sample-rate, more channels (surround sound) etc. Maybe even include a direct copy of the studio multichannel recording, so people can either listen to the pre-mixed version, or mix their own...
Not just higher, but MUCH higher. Film has great resolution, but 24fps is ridiculously low. Upgrading to 120fps seems an obvious step even now.
Besides that, like DVDs, it's not unreasonable to assume that with every movie you get, you also have to keep many GBs worth of special content and features you don't really want... When there's enough space, expect the dozens of commentary tracks to include video instead of just audio. And in-depth coverage of every aspect of every minute of every film...
And all that's not to mention what will happen to video data-rates when holographic displays become practical, and movies go 3D. (holodecks are much further off)
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
In the eyes of doctrinaires, this is true. Us others continue to use power-of-two units for data storage, and power-of-ten units for other purposes. This is no more difficult than having "mile" mean different things on land and at sea.
1 kilometer = 1000 meter
1 kilobyte = 1024 bytes
1 kibibyte = 1 pedant
--
*Art
He can say exabyte without referring to your corp.
And the percentage of people touting FLAC vs. the # of people doing mixing is?
I agree there are uses for FLAC [and the like]. I just disagree that most people need to use it [and thus require TB storage arrays].
Most people are probably like me, buy a CD or DVD, make a rip for their digital archive [e.g. multimedia box] and play it back in their homes or portables. I seriously doubt most people rip stuff then make a DJ mix, etc...
Tom
Someday, I'll have a real sig.
It depends on various things like your speakers/amp, sound card, room acoustics, noise floor, etc.
For most people lame with q=2 and 192kbps joint-stereo is as good as the original CD even with good acoutics and gear. 128kbps never cut it as far as I'm concerned for anything but portables or noisy environments. M4A is just a step up [e.g. better quality at the same bitrate] but even without it MP3 is fine.
The problem I have with storing mass amounts in FLAC is when I want to share or transfer the music it takes forever and you really have to plan ahead. I'd rather hack a stockpile of m4a's and just cp them as required.
Tom
Someday, I'll have a real sig.
The reason that most people who rip to flac do it is so that they can FORGET about the original CD.
.rm? Use the flac. Want to burn it to CD? Use the flac. Gave the CD to a friend, and they want to make OGGs? They can use your burned CD, without having to boost their bitrate to compensate for the cumulative loss of going CD->MP3->CD->OGG.
Need an MP3? Just use the flac. Need a
With hard drives available in the sizes that they are now, and at their current prices, it's plenty worth it to rip to flac and not have to worry about the original CD at all. 300-350MB is NOTHING. I regularly download individual episodes of TV shows that are that size or larger.
Yeah, until you hit multi TiB collections then have to sort out how you backup, or transfer RAIDs when the drives are old.
Also, if you just encode it once you don't have to worry about "OMG I NEED A RM FILE!! OH NOES" cuz you already have an mp3 or m4a or whatever. Why would you need the same audio in more than one or two formats?
Tom
Someday, I'll have a real sig.
You can fill your drive with a few really really big rips, or be like me and fill it with many many quality rips.
"drives as big as they are" is no excuse unless you only plan on having like 50 albums in your collection...
Tom
Someday, I'll have a real sig.
Most people touting FLAC are not editing or mixing. They're just really impressed that they can waste space and pretend they can hear signals below the S/N levels. :-)
Tom
Someday, I'll have a real sig.
Super computers? Once, maybe - not today, and not for the last decade or so. There are a bunch of companies (I'm working for one of them, now) that will quite cheerfully sell you a storage system that spans hundreds of disks. Assuming your OS won't flake out when it sees a 500+ TB volume, you could mount it on your desktop, if you want. There's absolutely no need to conflate processing power (super computers) with storage capability (NAS, SAN, etc.)
"Great men are not always wise: neither do the aged understand judgement." Job 32:9
"It ain't a war against drugs.it's a war against personal freedom" --Bill Hicks
And do not forget the future-future where movies store 3D information.
For example: I used to work in a business that felt it was reasonable to store information for a 100 years or more; just because medical research was done in the 19th century doesn't mean the results might not be useful today. But modern research generates far too much data to simply print it out on acid free paper. Technology for their needs simply doesn't exist today.
More personally: I've already had CD-Rs fail after just a few years, showing me that my digital photos are at at least as much risk as my old film negatives, if not more. I want to make sure my great grandkids can look at these shots, but hard disk interfaces and tape drives all become obsolete in just a few years.
Clear, Dark Skies
And what happens when the luser is you?
No one can call themselves a "real" sysadmin until they've accidentally reformatted the root partition.
Clear, Dark Skies
who wanted to delete some of the preference files in his home directory, so he typed
.*
rm -rf
It never occurred to him (or me) that ".." matched that pattern. He worked his way right up the directory tree and back down again...
Clear, Dark Skies
I agree that more than 44 kHz/16 bits doesn't make much sense for music playback, but I could imagine an upgrade from stereo to 40-channel surround sound that enables a full immersion into a three-dimensional sound field. Of course, most people wouldn't want to mount 40 loudspeakers in their living room. :)
Avantslash: low-bandwidth mobile slashdot.
I think they mean hysterical raisins.
Xenu loves you!
As a movie producer, you will still need to pay a human or a team of humans to do the additional recording and editing. I don't expect the producers to increase their investments in the future uxhd-dvd with a factor 1000 just to fill up the available space.
Avantslash: low-bandwidth mobile slashdot.
You're meant to divide by 2^40, not 10^12, it yields a very different result on larger numbers . . . or unless you intended metric TB and not TiB (tebibytes).
And before anyone steps in and notes that there's audio hardware and software that deals in higher sampling rates (e.g. 88.2kHz up to 192kHz) and higher dynamic ranges (e.g. 24-bit vs. 16-bit fixed, or 32-bit float)... these rates are targeted at quality improvements during the production process, not so much at improved quality of distributed music.
This is a bit OT, but I don't understand why people would abandon a filesystem whose creator is suspected of murder, but they're totally cool with a filesystem whose creator is convicted of multiple patent infringements, false testimonies in court, and anticompetetive business practices.
Did you miss that I was explaining a pun?
You just go right ahead using the same word to mean different things and calling it pedantry. I'll call it a pun when it's funny, and a crime against humanity when you are actualling making important calculations based on exabytes without a care in the world whether it's base-2 or base-10.
Karma: Incomprehensible (Mostly affected by posting at +5, reading at -1, and metamoderating everything unfair.)
I think by that point we'll be digitizing our pron *people*, not pictures and storing them..... think holodeck.
why the incident in question occurred in 1994, on a Data General AViiON, that wasn't running Linux and was using csh as the default shell.
You seem awfully sure of yourself for someone who doesn't even realize there are other shells besides bash.
Clear, Dark Skies
will be a bitch tho :)
I have nothing to lose but my bindings.
What about 3D content? Assuming your calculations are correct, add another 2000 for z resolution, and you're getting to around 10PB for a 2 hour movie.
I'd expect that to fill up such a hard drive fairly fast...
Ask me about repetitive DNA
The forced fsck is because hardware and unrelated software can make errors. (failing disk, bad RAM, loose cable, buggy controller, an unrelated buggy driver like audio, electrical noise from the power supply, overheating CPU, a cosmic ray, an alpha particle emmission...)
If you totally trust the journal, eventually your filesystem will be trashed.
Very few filesystems are more durable than ext3. We will expect the same of ext4. Of the non-journalling filesystems, ext2 was relatively durable. This is a design team that knows their shit.
Much less durable: reiserfs, reiser4, jfs, xfs, hfsplus, ntfs
Slightly less durable: ufs, ufs2
Possibly more durable: zfs
With ext2/ext3/ext4 you get data structures in fixed locations on the disk. This makes them much harder to trash. With most filesystems, the loss of a single critical bit (in a pointer to the root of a tree) will wipe out the whole filesystem. The odd beast here is zfs, using replication to protect an otherwise-fragile tree structure.
I don't now why I should bother with *lossless* compression especially on streaming media. That will, if ever, give compression rates of 25-50%, taking a buttload of computation power to compress. Lossless compression works only on data with low levels of entropy such as text or stupid zero-filled formats coming from Redmond.
All what lossless compression can do is finding a more suitable alphabet to code this very data into, and identifying identical/repeating regions so that only "repeat this!" is stored instead of the actual data.
There was a time when even 25% compression rate was a great thing cause it made things fit on a 170kB/360kB/720kB/1.44MB disk drive, but today it's just not worth the hassle anymore -- only for very big data, which usually are multimedia files and therefore a very bad application for lossless compression algorithms.
Do you care whether a file has 2MB or 1MB these days? I don't. It takes a second longer to download, but apart from that you don't feel the file size anymore. You do care, though, if a movie can be shrunk from 4GB down to 700MB -- cause then it fits on a CD, cause then it takes just a fraction of time to download, to burn, plus 3.3GB is a difference you still feel on today's harddrives.
As for lossy compression, there is always a catch. I'm one of the people out there who are able to actually hear MP3 artifacts. I also easily spot DCT-related artifacts. Besides, lossy-compressed data is harder to postprocess. (If you want to hear some really strange effect, do a 128kbit-MP3-coding of Art of Noise's "Moments in Love" and feed it through a Hush surround sound processor. However, these side-effects are not always that pleasant/interesting.)
So, to make it short: you don't want to archive in a lossy format, cause when the newer, better format appears, you transcoding makes no sense anymore cause you're stuck with the limitations of your original format. Whatever bad that format did to your data will remain. You can't "repair" lossy-compressed data. And no, low-pass-filtering ("blurring" for you video guys) is no option.
In a couple of years we all are networked with gigabit fibers, we have several Terabytes home storage. Then we just don't care about these 3.3GB as we today don't care whether a file is 1 or 2MB in size. By then we probably care that a HD++ video takes just 50GB instead of 300GB...
You might already know about these but: FLAC for audio (works great), ffvhuff (from ffmpeg) for video in the YUV colorspace. Those are the two I can think of that currently work pretty well.
Especailly with ffvhuff (a modified Huffyuv) for a 704x480 stream at 24 minutes, I will get a file that's about 14GiB (results will vary, of course.) The uncompressed size of that would be (using a YV12 colorspace at 29.97 FPS, that's 1440 seconds or about 43158 frames) 506880 bytes/frame * 43158 frames = 21875927040 bytes = ~20.37GiB. So, no, you won't be putting that onto a single CD, but for lossless compression of video, that's pretty significant. Even more space is wasted as the total number of frames and resolution increases without some form of compression.
Oops, wrong numbers. 34 minutes of video with lossless compression (704x480, 29.97 FPS, still YV12 with ffvhuff): almost 14GiB (results will vary.) Same thing without compression: 506880 bytes/frame * 61140 frames = 30990643200 bytes = ~28.86GiB. Quite a difference. Didn't get this right the first time because I was in a hurry.
Yes, but will we be able to conveniently carry around that much data on small cards, send it over a network quickly, etc?
If not, then compression will still have its uses.
By now you don't even now what to do with 1024PB
I can barely fill a 250GB drive, and most of it is taken up by barely-touched pr0n.
The people who use massive disks now and in the future will be businesses and government agencies, if for no other reason than bureaucracies like to keep records.
"I don't know, therefore Aliens" Wafflebox1
Where?
But even if it were 32-bit pixels, that would mean the movie would "only" be 74TB.
"I don't know, therefore Aliens" Wafflebox1
It's also been said that high quality DVD and CD ROMs last a lot longer than the cheap ones, so my current strategy is a single hard disk paired with DVD roms, but I'm getting ready to switch to a NAS with a mirrored array.
Clear, Dark Skies
OK, fine, I too can perhaps see an upgrade to say 5.1 sound becoming common, but even that is "only" a factor of 5 larger.
Besides, I was assuming lossless compression. That is, frankly, overkill for end-users. A CD (uncompressed) is on the order of 10MB/minute. With modern codecs you can compress that down to 320kbps or even 256kbps and the quality is still such that even the critical listener, in a silent environment, with excellent components, is unlikely to hear the difference.
Witness SACD. There's no doubt whatsoever that it is "better" than standard audio-cd. But the advantages are irrelevant to 99% of all listeners. I predict it'll *never* become dominant over CD, despite being technically superior. The same may very well be true for Ogg versus Mp3. (If Ogg *does* win, it'll likely be because of political issues more than because of any property of the codecs used or flexibility of the file-format)
I too can see CDs being replaced with "something better". There are lots of disadvantages of CDs that *are* relevant to normal people, if a new format comes along that fixes a significant numer of those, without introducing new worse problems, there's no reason it couldn't take over. Problems include:
Witness how SACD fails to solve basically all of these real-world problems. In contrast, for most people, the 16-bit sampling-depth and the frequency-limitation of 22Khz are *NOT* significant problems, nor is the 2-channel limitation. The fraction of people that *do* consider these three serious problems is shown in the uptake of SACD-players versus mp3-players. (the latter solves most of my problemlist, but actually typically *degrade* soundquality)
The logical choice that solves all of these and then som problems is the non-physical downloaded unencumbered mp3 or ogg-file. People love those. Even enough to use them in millions despite a very limited array of legal ways of aquiring them. Hell, most people I know that *do* still buy CDs do it explicitly for the purpose of converting them into files for their mp3-players.
I agree with you that it's quite reasonable to expect video-bitrates significantly higher than 10GB/hour in the next few years.
I wouldn't just call 99% "most", I'd call that an overwhelming majority, or "almost everyone".
"Most" is a much lower percentage... And when you have several millions people unwilling to accept the lower standard, the higher standard eventually gets accepted, baring major limitations.
It's technical improvements are hampered by it's natural and artifical limitations.
Actually, I'd say audio CDs are a very good example of my point... Though many people are willing to buy a few tracks of lossy compressed AAC files, their much poorer quality will prevent them from eliminating CDs. The lossy compression, or the bitrates, will need to be improved significantly before it's suitable enough for almost everyone to be happy with them. And until that happens, the standard will remain decades-old CD technology.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Of the people I know that refuse to buy (or in practice don't buy) anything from iTunes, I've only *once* heard quality-concerns even mentioned. What I *have* heard, repeatedly, is:
Notice how none of this has anything to do with sound-quality. If sound-quality was the huge selector you make it out to be, 128Kbps mp3 would *never* have been the stellar success it actually was (and is)
The fact that you failed to mention sound quality... or more specifically that the people you heard from failed to mention it... doesn't mean it's not an issue.
MP3s are a bit like copying songs off the radio. People accepted the quality because they were getting them for free. Unencumbered 128k MP3s didn't replaced CDs, nor will they.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant