Slashdot Mirror
Bitrate Peeling with Ogg Vorbis
Yort writes "Thought this might be interesting to some audiophile /.ers - there's been some discussion on the Ogg Vorbis lists, summarized in the most recent Ogg Traffic, about "bitrate peeling". In short, it's where you can simply "peel off" the high resolution data from the ends of an audio stream packet to come up with a smaller, lower quality stream. Brings up a number of geek-cool opportunities."
That may be true if it were peeling off the copyright instead of the bitrate.
Though it does sound like a pretty cool way to downsample a stream.
-hero.
I don't know of any that play Vorbis files yet, but it would be very handy if I could take an OGG I had encoded at a high bitrate (for playback on my nice home stereo) and make it smaller for use on a walkman-type player for the gym or whatever.
Wait a minute, I thought audiophiles always wanted to improve sound, not deteriorate it! Maybe the coders, like me, like this stuff, but my audiophile side is not interested
This is just sig!
You could do the same thing with simple transcoding but this could be a lot more scalable.
It would be even cooler if they could disperse the signal in time (i.e. across several packets) and just drop every n'th packet in order to create a lower bitrate stream without making it sound choppy.
"I have opinions of my own, strong opinions, but I don't always agree with them." -- George H. W. Bush
Bitrate peeling is a briliant idea, and would be a major win for Vorbis if they ever actually provide an implementation of it. It's something that the format supposedly supports, but right now it's still just a hypothetical application.
Let me know when they've got something working THEN I'll be impressed
my sig's at the bottom of the page.
this brings to light the mp3/ogg hardware compatibility issue... ie: no portables/car stereos/dvds/home stereos supporting ogg, even though it's a superior format... i'm scared that it's going to suffer the betamax syndrome...
personally, i'm converting over from 256kbps mp3 to 128kbps vbr ogg and i'm saving TONS of space and not really sacrificing any quality... does anyone know of a petition or something similar to get mainstream hardware manufacturers to include ogg support in their hardware?
similar to bit reduction decimation (look about halfway down the page for a quick explanation of decimation as an audio effect).
Smaller and lower quality does not belong in the same sentence with the word audiophile. :)
Cool idea though.
Q.
I call BS. The parent poster is making all that stuff up.
Or am I the one that's bluffing?
~Idarubicin
Lately I've been finding all I can download off P2P programs like Direct Connect and Furthurnet. Its mostly live shows, and they are all in .shn format, which is a lossless compression format that restores to the original .wav file.
These communities shun both compressed files like .mp3 and trading anything that has been released commercially. What you do get is great recordings of live music from bands like U2, DMB, Grateful Dead, etc., all ethically traded and in their full audio glory.
The audiophiles I know pretty much don't listen to mp3, ever.
No, Thursday's out. How about never - is never good for you?
Bitrate peeling is not a good idea because it seperates the audio signal, with hi frequency data being seperate from low and mid frequency data.
I'm not sure if it's frequencies that are being separated here - the idea seems similar to wavelet image compression, where refinements to the original data arrive over time.
Note I know nothing about Ogg, so it probably isn't based on a wavelet approach, but the idea sounds more like you get low resolution data (covering the full frequency range) followed by higher resolution data. I.e., you're increasing the sampling rate over time - but the range of frequencies you sample is the same regardless.
Nae bother
I always kind of felt the first and third bit of every byte were kind of unnecessary. I'm not to fond of the F6 key either!
I don't think that's how it works.. from what I understand, it simply removes bits... ie:
I = one bit
----A 128k stream----
IIIIIIIIIIIIIIIIIIIIIII
----A 96k stream-----
IIII IIII IIII IIII IIII
It simply removes some of the bits, not any specific freqency of the high/low/mid.
and it looks to be an impressive streaming method
"Beni Cherniavsky mentioned a very intriguing counterpart to bitrate peeling. If you have a peeler that saves the bits it chopped off, you could reconstitute the higher quality files by adding the missing bits to the lower quality file. This idea could lead to a music download service where you can download a low quality preview version of a song, and if you are interested, download the missing bits to make it a high quality version."
Or, Slightly modified, You could share all your high quality oggs on a P2P network, and have your client peel it down to 'future-legal-to-share' low quality files.
Pain lasts, kid. Its how you know you're alive. Sometimes I think this growing up thing is just pain management-TheMaxx
Sounds more like it would be unique to OV, if they implemented it.
The point is, nobody does it now. Perhaps this is because there's really no need for it. Consider the list of "very sexy applications:"
- In a streaming situation, the server would store only one high quality stream, and dynamically peel it down to the client's bandwidth. Not useful. If you instead stored a hundred separate files, each optimized for its bitrate, with each being half the bitrate of the previous one, you'd still have a set of files less than twice the size of the largest file. Plus, you'd have no bit-peeling overhead. If you're streaming 100GB audio files, maybe there's a benefit, but if you're doing that, you can probably afford a second 100GB file for all the smaller files.
- You could store high quality Ogg Vorbis files on your PC for your Audiophile home theater setup, and peel them down to "good enough for lousy headphones on a noisy train" portable files. You can do that now, without this high-tech. Such low-quality files could be easily made from the original OV high-quality files, without much extra artifacting due to the re-encoding. And again, how low a quality are you willing to accept, if you're going this far anyway? Wouldn't you just buy a higher-cap memory card?
- Download a low quality preview version of a song, and if you are interested, download the missing bits to make it a high quality version. Another non-benefit. Suppose the full file is 10 meg, and you download a 1 meg sample. Are you really going to opt to download the 9 meg "patch" file, rather than the 10 meg complete version?
A clever idea, and it sounds cool on the outset, but it seems to me that this is a solution seeking a problem.This is exactly the killer app that Ogg needs for acceptance: a program that syncs songs to your portable player at a lower (user adjustable) bitrate. Even better: You pick out X number of songs. Each time you add a song, it re-calculates what bitrate to shave them all to, to maximize the bitrate used, thereby using all the RAM on the player but getting all your songs in.
I can't wait til this one hits.
If you read many of the audiophile magazines such as Stereophile, etc., you'll see reviews of equipment such as external DACs for CD players as well as many high-end CD players. Among those there is the legendary Linn Sondek, a CD player which retails for around 21k.
Why would you need a 21k CD player, you ask? If a CD player is playing back an exact digital file, than shouldn't all CD player's sound the same? The answer is simple: let your ears be the judge.
I was initially skeptical when I was first shopping for stereo equipment, but there is a world of difference between a consumer CD player obtained at the chains like Best Buy, Fry's, etc. and an audiophile CD player. The difference is primarily in the level of clarity or resolution that you can hear from a quality CD player. The difference is subtle yet dramatic, you can hear instruments and detail that you simply could not make out before.
To make a long story short, the quality of mp3s is typically even below that of a CD played on a cheap consumer CD player. No "audiophile" will listen to them as a primary audio source. That said, I have an mp3 player that I use when running and I have mp3's on my computer. Everything has its place, but the place of mp3's or ogg is not in audiophile stereo systems but in the world of music sharing where file size is a critical issue.
Musi withou hig frequenc is lik wods mising leters.
-braxton
Just jam a crayon up you nose. They you will understand the joy of paying $100.00-$500.00 per foot for speaker cable...
of a Slashdotter comment that twisted an old expression...
"Those who sacrafice sound quality for hard disk space deserve neither."
Cool idea to throw around though.
Why, o why must the sky fall when I've learned to fly?
One thing that hasn't been discussed here is that a lot of people feel that Vorbis is transparent at something like quality setting 4. Other people think it's transparent at quality setting 3. Others think it's great at 1. I release my stuff at 4, but bitrate peeling will let people peel those down to what sounds good to them. Maybe they want to monkey with it, and maybe they don't, but the option to do this without re-encoding is sexy.
It's not just a 'chop it down for modem folks' thing, it's also a letting people choose for themselves situation that I think is more important.
Features are cool, but features that give people options apart from 'use it or not' are even cooler.
That's it for me. Please donate to Xiph.org, and then go listen to some tunes. Enjoy!
What about LIVE streaming? Should the audio source encode 10 different versions at once, and send double the bandwidth to the repeaters? Bitrate peeling is a great benefit for live streaming, it will reduce the upload to a single stream and take processing power requirements away from the encoder.
Scalable techniques like this are very cool, but hardly novel.
MPEG-4's scalable profiles provide a similar effect (albeit in the other direction, with enhancement layers). Some of the higher end audio codecs (beyond CELP and AAC), like ER BSAC (Error Resistant Bit Sliced Arithmetic Coding) do exactly this. The idea in this case is that the server will in real-time only provide as many bits as the connection can currently provide. Very nice for wireless.
QDesign's QDX format does almost exactly what is described for Ogg, with arbitrary bitrate peeling down to the 1 Kbps level. The idea is that you could copy as much data as you want to your mobile player, and it'd dynamically thin to the data rate that would fill up your device.
And still image codecs like JPEG have used progressive modes for years, where additional data adds more detail to the image.
My video compression blog
Okay, if each one is double the one before, that means you'll have a 2^100 ratio between lowest and highest data rate. Thus, if your lowest is 1 Kbps, the highest would be... Not going to happen that way.
Also, you assume the sweet spots are 2x the one before. In fact, jumps of more like 1.25 are likely to be optimal (albeit with a lot fewer jumps!).
My video compression blog
It would be great for multicast-type situations (including, but not limited to IP multicast.) You could send one high-quality signal out from a central point and then shave off bits to fit the stream down to the quality needed by the end-user.
For instance, users on a 56k modem could listen to the same multicast stream as a broadband user (ie, no need to send out multiple, separate versions from the source)-- this assumes the presence of routers (or conversion boxes) capable of doing the peeling as needed.
All in all, very useful.
Could it possibly be used to progressively download an Ogg file and begin listening to it before it's 100% done? For example, have the 32kbit quality at the start of the file, then next have the next 32kbit (starting at the end of the file), then the next 64kbit, etc...
Luke-Jr
Most lossy compression schemes like JPEG (regular) work off the concept of using a domain transform, only saving what are considered to be the most significant coefficients after said transform. This makes almost every lossy compression scheme out there is a canidate for bitrate peeling of some sort of another. I'm just surprised that on one does it; having not read the JPEG standard though, I agree that it might be technically challenging (or simply not much in demand).
Having studied wavelets, I can state that they definitely could support a technique similar than this. Given a properly ordered stream of discrete-time wavelets, one could chop off the last few cycles and similarly lose just a bit of quality.
He was bragging about his expensive (and therefore wonderful) setup, when he mentioned having "super-high quality" digital cables, which cost him $2,000. I asked him how much "low quality" digital cables cost, the answer? About $15!
I asked him what the difference between the two was, he claimed that the high-quality digital cables gave the music more "body"!
It may have been cruel of me, but I just couldn't help but explain what digital actually meant.
The moral of the story? That much of the audiophile community are simply the blind leading the blind, pseudo-techie alchemists, who assume that expensive means better.
"On my Complainophile Sony Type-R Limited (bling-bling) audio set up, 32kbps sounds like ass because I didn't realise I could encode at higher bitrates. Except Windows Media because there were no options... then it scanned my hard drive, told me I had really poor taste in music and automatically deleted all my .mp3 files.
Bitrate peeling is not a good idea because I say so and need to look important. Transcoding is such a better idea because it introduces no artifacts and takes a much shorter time to do. Not that you should do that because nobody uses a narrowband internet connection anymore, at least nobody that I care about. Comparitively RealAudio is great! No, really! I enjoy giving my personal information to someone just so I can listen to commercials and kill pop-up advertising before it asks me if I want to upgrade, then crashes my computer!
While I conceed it's possible.. nobody else knows what they're talking about! Um... look over there! (exits stage right...)"
Yes. Thanks. Really persuasive argument.
There aren't ANY free lossy codecs that can do bit-peeling right now. Some non-free codecs allow you to overlay data (like progressive JPEG), which is NOT the same thing. That's just like transmitting deltas at higher compression rates, which could be done with a simple side-band for any existing method, MP3 even.
The only option is transcoding, which compounds compression errors (decode, reencode). I often wished the MPEG group would have been more intelligent in the design of their bit-allocater so that you could "thin out" the quantization of the power bands by looking at the "right parts" of the MP3 frame. Alas, this is not possible.
But the Vorbis designers have made this possible, thus making it possible to have high-quality and low-quality versions derived from the same source file without additional processing. I imagine you have certain restricted choices, due to how quantization information is bundled up/packeted. But it isn't just sexy, it would be stupid to NOT DO IT. It takes just a little forethought on how to lay out the information in a hierarchial fashion. What makes anyone think that this is any harder then decoding/reencoding. I guarantee it has a time complexity on the order of a straight copy.
Hell, formats like SHN and FLAC can do it, just substitute short codes for long codes at a certain rate; it'll add a bit of wide-band energy on decode, raising the noise floor in proportion to the space savings you gain.
So anyway, "poop on you" to all these wanna-be audiophiles/slashdot-know-it-alls who don't no a good thing when they see it.
Don't like it, keep sucking on that Layer III.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Speaking of such, are there any Ogg-supporting portable players, or players in development?
(Granted the Hardware Support page at Xiph has some info, but I'm curious if there's anything else known)
Alex Bischoff
HTML/CSS coder for hire
Frigging.
As in the song, "Frigging in the rigging". Somewhat rude.
Looking at your example of storing the variable bit rates as seperate files as an example, let's do some theoretical math:
Original High Quality file: 10mb
3/4 Quality file: 7.5mb
Half quality file: 5mb
Quarter quality file: 2.5mb
Total for all variations without peeling: 25mb
Or, store the High Quality 10mb original only and dynamically peel. Savings, 15mb (1.5 times more files)
No need te reencode a new file for each device (and some of us have many!)
Just FYI
You can have it fast, accurate, or pretty. Pick any 2.
Speaking of WMA files, how does Ogg compare with WMA, quality and size wise? My dad who works in the radio production industry swears by WMA over MP3, and I have tried to tell him about Ogg because it's free from all of the MS $trings, but I dont have any hard evidence to prove me right (I may very well be wrong)
(B) + (D) + (B) + (D) = (K) + (&)
but what you just said has nothing to do with whether a 128kbps Ogg sounds like a 256kbps mp3.
It's commonly observed that oggs of lower bitrate compare to mp3 at higher bitrates.
There are a number of problems with the parent post.
1. Keeping hundreds, or even ten separate files as described, each half the size of the previous, is not plausible. I'd assume the author was a troll, but since no one else has mentioned it, perhaps the obvious fallacy with that idea is slipping past even the sharper readers. A 10MB file can be split in half at most 23 times before it is only 1 byte long, far fewer before the quality level is unacceptable. Secondly, the idea described in the article, provides for dynamic bitrates, not simply half the original bitrate. To provide even similar functionality, one would need files in ranges from 1MB to 10MB in relatively small increments, totaling well in excess the "twice the size of the largest file" as suggested. Even so, this would be deficient in that the bandwidth could not be throttled mid-stream.
2. Second, decoding and re-encoding the same file with a different bit rate will almost certainly result in poorer quality than the technique described. The safer, more straightforward solution, is to perform reduction operations on the transformed data, rather than the decompressed waveform. Otherwise, amplified artifacts from the original compression will be present in the new file.
3. Third, the strength of the poster's argument lies entirely in the choice of ratios. Downloading a 5MB file rather than a 10MB file leaves only 5MB remaining. To paraphrase, are you really going to opt to download the 10MB complete version when your software can download the remaining 5MB in half the time?
There are a number of problems which bitrate peeling address, not the least of which are 1) reduction of storage space as described previously, 2) dynamic bandwidth regulation of audio streams for streaming radio, future cellular phones, VOIP, and network appliances running on congested networks, 3) file size reduction without transcoding, 4) user-specified bandwidth on demand, 5) automatic preview generation from source without any extra administrative overhead.
I'll even add my own... the ability to download a very high quality file and start listening to it immediately at lower quality without interruption. By the time the file has played through, the download may be only 50% complete. If I decide not to continue with the download, I have wasted no more time than that necessary to listen to the file. If I want the file, I have only 50% remaining.
In some ways, this is similar to the rationale behind interleaved images, except that it is unlikely that you will need to listen to the same file repeatedly at progressively higher bitrates. Nothing prevents this of course.
-Hope
You could start here or here.
I'm no audiophile. I think the long protracted discussions about cables, wiring and shit are a joke. (And convincing me to spend $40 on a OPTICAL cable to run 6ft ain't going to happen, I'll take the $15 cable. ($15C)
But even *I* think 128k MP3s really aren't that great when compared to the original music.
No comment on ogg at 128, never use it.
which is why I ASKED if the person honestly thought that OGG was twice as good ?? I really don't care on my end which format I use, I just want the closest to true sound. If an OGG track was actually as good as an MP3 of twice the bit rate it would be time to begin re-ripping things to Ogg Vorbis. As for space vs quality that is ALWAYS going to be an issue...
errr....umm...*whooosh* *whoosh* Is this thing on ?
That's how DTS was able to add a discrete surround channel (DTS ES) without causing problems with older receivers. Dolby can't change their header without breaking backward compatibility, which is why their extra surround channel (DD EX) is matrix encoded.
* As is generally the case, my opinions do not reflect those of my employer.
I think that the iPod does a very bad attempt at this. MP3s, encoded at 320k play back really badly on the iPod, far worse than 128/192k ones. I suspect that the iPod hardware hasn't enough horsepower - and it is discarding bits that it cannot decode fast enough. The MP3s sound fine on the pc, or decoded to wav and then played back on the iPod. But played back (as MP3) on the iPod, the result is dismal - there's a 5Hz "wobbling", rather like a steel band, and lots of distortion. (Apple won't help, but I have replicated this problem on multiple setups both Linux and OSX - it would be interesting to see if any /.ers have seen the same thing. You need a good recording of a classical CD with very large dynamic range eg Mahler 8, part II to demonstrate it - listen to the quiet bits.)
[I have some demo files, but can't link them - I'll get slashdotted off the net !]
...but how many vacuum tubes do you have in your system?
The true measure of an audiophile.
"And like that
How does my quip "iPod'ers with 20 gig drives wouldn't lose sleep over it" become quoted as "20G should be enough for anyone!" in your post?
You must work for CNN.
Peel layers from a data file thin enough that no layer is independantly recognizable. Scatter the layers throughout a P2P system. Now no individual user possesses the whole file, but all users can reconstruct a working copy when they want to play it.
That Jesus Christ guy is getting some terrible lag... it took him 3 days to respawn! -NJ CoolBreeze
A-bloody-men!
All this audiophile nonsense, what with the insanely expensive power cables and such, smacks of simple elitism. "Look, I can spend twice as much as you! Hence, my audio is better, and I can appreciate it more than you can!"
I consider my low standards for audio a blessing. (Especially since I can't afford 'good' equipment.) If I don't hear a problem, then I define my solution as "good enough". It's cheaper, easier and I bet I get just as much out of my music as anyone else does.
And besides, if audiophiles were so nuts about quality, why don't they do more double-blind tests? Scared they'll discover that their A$240 power cable doesn't present an improvement over the freebie they picked out of the trash bin?
I swear, these people are crazy.
--grendel drago
Laws do not persuade just because they threaten. --Seneca
The idea is that quality is lost over multiple generations if people re-encode using the same lossy codec. Consider the following situation: Person A downloads/trades/whatever a show which is MP3 encoded, transfers it to CD and then dumps the MP3. Then, some time later, someone asks for a trade for the show, so the person re-encodes the audio. Now we have a second-generation version, which now has compounded errors from the first and second encodings. This process could continue indefinitely. The result is that, over time, the quality of the shows would decrease. So, the rule is, only trade shows in a lossless format, and never, EVER, trade a lossy-enoded show.
The player could strip its own files on-the-fly. If you wanted to add one more song to the player, it could strip a tiny bit off of each song (or maybe just strip the higher bitrate songs), and make enough room for the new file.
HIV Crosses Species Barrier... into Muppets
As reported by Palm InfoCenter, "A group calling themselves Aerodrome Software have released a public beta of an Ogg/Vorbis media player for the Palm Tunsgten T handheld. The player inititally supports only ogg/vorbis encoded files, a new open source audio format, but promises to have mp3 support in the near future."
I'm curious as to whether they used the integer-based "Tremor" code to achieve this.
Alex Bischoff
HTML/CSS coder for hire
peel off the high resolution data from the ends of an audio stream packet to come up with a smaller, lower quality stream
Newsflash: removing data from a packet makes the packet smaller, and as a totally unrelated bonus, the content of lower quality! This obviously deserves space on the front page of the world's largest geek magazine.
Independent artists and labels are free to license their music however they want.
Even if the performer is not with a major label, the songwriter still gets a fixed amount per copy distributed. Performers who write their own music have no easy way to verify that they didn't accidentally infringe the copyright of an existing work, which George Harrison found out the hard way.
Will I retire or break 10K?
it simply removes bits
It's a lot more sophisticated. It's not lowering the sample rate, and it's not droppping the least signifigant bits of the samples.
It's hard to explain without getting into the complex math of Fourier Transforms, but every bit has information from the entire file rather than merely a small piece of information from one spot. The bits are naturally in the order of importance, and dropping the bits at the end drops the least important information everywhere.
-
- - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.