Slashdot Mirror
Breakthroughs In HTML Audio Via Manipulation With JavaScript
jamienk writes "Imagine if you could grab and manipulate audio with JavaScript just like you can images with Canvas. Firefox experimental builds let you do just that: crazy audio visualizations, a graphic equalizer, even text-to-speech, all in JavaScript! Work in progress; you need a special build of Firefox (videos available), being worked on via W3C."
All I can think of is <BLINK> </BLINK> turned into audio!
Of course this has good uses (blind users with scripts available), but I can see how this will end badly for the rest of us.
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
FYI, Quicktime is not awful at all on Mac OS X, nor is Safari and iTunes. And almost everything from Microsoft and Adobe sucks on Mac OS X.
I don't know why you associate Quicktime with online audio, so what you probably meant to say was "The quicker we can get away from our reliance on that god awful Flash, the better."
Be prepared for another "Firefox vs the World" with this, however: Vorbis vs MP3/AAC.
No, that was Flash, and it caused lots of ire, particularly in combination with Firefox plugin that loaded the Google homepage in the background. I hope there'll be a sound=off option, because I still remember the 90s and web pages with MIDI sounds.
All these breakthroughs in Canvas and Javascript are useful, but will they be used?
Until IE implements even half these features, we will be stuck with "Quick"Time and Flash for quite some time...
Taxation is legalized theft, no more, no less.
music to my ears. Finally I will be able to listen to midis without a plugin.
Once you start despising the jerks, you become one.
This will be fantastic for aiding those that have vision impairments. The 503 compliance will end up including this if it is ever standardized. (w3c not known for speed)
You mean annoying like people who put their comments between <code> tags?
Isn't mp3 set to be free soon?
awesome, you could do that in Flash 5 years ago.
even text-to-speech, all in JavaScript!
WooHoo! I'm gonna have hidden fields in my web pages so this "text to speech" will say one thing while the text on the screen will say another!
Yes it would seem still no native mp3 support.
The real fundamental question is, however, do we really want to use java for audio?
mp3 is a patent-encumbered standard. We don't want any of that in your open standards, now do we? Also, javascript is not java; it really has nothing to do with java, except the ill-chosen name.
GAAH! MY PRINTER IS ON FIRE!!! PUT IT OUT! PUT IT OUT!
The game and all the movements were Javascript. The sounds were done with flash.
Not really. Vorbis has about the same quality per bit as AAC (unlike theora vs h264), and it's established long enough to not have patent issues. There's no reason not to implement support for Vorbis, and it's plenty good enough to be the default codec. What's more, Youtube's behind Vorbis (it's part of the WebM spec), and since Flash has pledged support for WebM, they'll have it too.
the future os is the browser
the future os programming language is javascript
enjoy!
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
You mean that you expect this to actually be used?
IE doesn't even have support for canvas, Firefox has had it from 3.0 at least, and I think even 2.0 had some support.
If IE still has more than 30% worldwide marketshare, and doesn't have basic requirements for this, its not going to be used. Period.
Canvas has been around for ages and is there even a single practical example on a site people use daily? Yeah, there are about a million tech demos but very little actual use because IE doesn't support it.
Taxation is legalized theft, no more, no less.
Well, he's a disciplined programmer, he puts comments in his code.
Ezekiel 23:20
And from what I've read, there is no-floating-point software decoders that don't require much CPU power either, so software-only solutions won't drain the battery significantly. I don't doubt for a second that Opera would also support Ogg Vorbis. The problem would be to get Microsoft and Apple to add support to their products.
However, if we look at the past:
- Microsoft: BMP screen captures (Microsoft format)
- Apple: PNG screen captures (Open format)
- Microsoft: Windows Media Audio/Video (two Microsoft formats)
- Apple: H.264/AAC and MP3 support (non-Apple formats, already established industry standards formats)
Apple would probably go with MP3, AAC and Vorbis audio files in Safari 5.
Microsoft would probably go with MP3 and WMA in Internet Explorer 12.
I thought at first that themp3 standard was already "free":
Open standard. Developed by the Motion Pictures Expert Group (MPEG), Coding of audio, picture, multimedia and hypermedia information.
However, mp3 is not free...yet. Some of these patents are set to expire on their 20 year time frame in a couple of years it would seem.
We should start a new Slashdot and return control to the geeks. It actually wouldn't be that hard to get some users to
Also, javascript is not java; it really has nothing to do with java, except the ill-chosen name.
Valid point...I see where the 'Offtopic' tag came from now, thanks :)
We should start a new Slashdot and return control to the geeks. It actually wouldn't be that hard to get some users to
You know, there are already PLENTY of applications out there that blind users can use to turn text on screen to speech? And that a lot of people prefer to disable javascript from running?
Last thing we need is an "on Hover" Javascript ad, you mouse over, and audio gets maximized "FIND YOUR MATCH @ ADULTFINDER.COM!"
Vorbis has about the same quality per bit as AAC (unlike theora vs h264), and it's established long enough to not have patent issues.
Age is not an issue, MPEG-2 still has patent issues! The issue is that Vorbis has not been adopted by enough deep pockets for the patent trolls to come out yet. Already they are licking their lips over VP8.
From what I've read, there was an off-screen flash helper playing the audio, and I'm not surprised. It's nigh impossible to get reliable audio with JavaScript only. People have been doing some experiments with stuffing data urls into , , and tags for a while now (heck, I've been doing it since last fall) in order to programmatically generate audio, but it generally suffers from performance issues and various glitches for a while now. You can mostly pull it off only if you stick with MIDI or you use Flash or Java to deliver the audio.
Here's something I didn't get until recently: Audio is in some ways harder than video. It's more timing sensitive. With video, you're generally slinging a lot more data per frame, but you can get away with *much* lower frame rates... 24 fps works, 60's not bad, and you don't have to get much higher than that to pass the threshold of perception. With audio, very small "frames"/samples will hold adequately resolved data from one point in the signal, but you have to move a lot *more* of them (44,100 of 'em per second for CD quality audio) and move them *reliably* in order to get sound fidelity. JavaScript timing might be millisecond reliable except in IE, but it's not microsecond reliable. Totally realtime programmatic sound is probably out of the question for a while.
But, programmatic generation of the audio *data* is possible now, and you can probably precompute enough things ahead of time that if they can work the bugs out of the audio tag or something else to enable microsecond level starting/stoping/looping of playback, then pure JavaScript audio would get real wings.
Tweet, tweet.
Comment removed based on user account deletion
Vorbis has been used by a number of major game publishers for game audio, precisely because it has no license fees. I think WoW would be a big enough target for the patent trolls, how about you?
> However, mp3 is not free...yet. Some of these patents are set to expire on their 20 year time frame in a couple of years it would seem.
Yes, the next MP3 patent expires this Sunday. The longest patent seems to expire in 2018 but that appears to be MPEG-2 LSF and only required for low sample rate MP3s. So the next furthest date looks like April 2017 but it may be worth double checking the dates on those around 2014/2015.
http://wiki.multimedia.cx/index.php?title=MP3_Patents
Humorously, I actually thought the game was silent when I first viewed it. Once again, ClickToFlash delivers an improved user experience.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Example case - trailers from trailers.apple.com - the ones hosted at apple typically demand that you have Quicktime installed (obviously since Quicktime is their technology). Quicktime might be dandy on the Mac but is most definitely god awful on the PC - much worse than Flash, in fact I'd put it in the same category as Adobe Reader or Real Player plugins.
.mp3 link in Firefox, the Quicktime plugin fires up and starts playing it - and its player sucks. Also, since it's impossible to install iTunes without Quicktime, every time I install or upgrade iTunes I have to go through and decrap-ify all the mime associations for Quicktime.
Further more, Quicktime's browser plugin assigns itself as the default player for many types of media - including mp3 files. So every time I click on an
Yes I will be happy when Quicktime dies.
Why would you even bring up BMP in this context?
It is barely a format. It is not encumbered by patents.
Nerd rage is the funniest rage.
This:
was meant to be
Tweet, tweet.
Third, unlike Theora and H.264, Vorbis is actually much better than MP3.
We use Flash for video. I see no reason to not use JavaScript (or Java) for audio.
I'm not sure of the speech packs but I remember seeing in the older MPQ's for WoW that the shorter sound effects were in WAV and longer songs in MP3.
It was Chris Rock, who said, "You can do it, but that don't mean it's to be done."
There are no karma whores, only moderation johns
awesome, you could do that in Flash 5 years ago.
awesome, you could do that in a Win32 app 15 years ago.
Last thing we need is an "on Hover" Javascript ad, you mouse over, and audio gets maximized "FIND YOUR MATCH @ ADULTFINDER.COM!"
Can't that be done now with Flash?
The marketshare for Safari is only going when new iPhones or iPads come out. If you look at the w3schools.com listings, you'll see that when new Apple products aren't coming out, the marketshare for Safari actually went down from 3.8% to 3.7%, whereas the number a year ago was a little over 3%. Considering the number of Apple handhelds entering the market, you'd think the number would be going up faster than that.
In the same period that Safari went from 3.1 to 3.8 percent, Chrome went from 4.9 to 13.6 percent.
You are welcome on my lawn.
Microsoft is not blocking WebM (just not shipping with it). And they have always allowed whatever codec installs for WMP/IE in the past. Contrast this with Safari and Apple's stance on Theora.
-]Phreak Out[-
In all honest ignorance and curiosity, how so, aside from the patent legal issues? Are the file sizes smaller? Is playback processor load less? Is playback quality at a set sampling rate audibly better? Etc., etc.
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
Actually, I'm building an enterprisy application right now that uses the canvas tag to emulate a 3D environment. We support IE via the Google Chrome Frame. So there's one way around your problem and one example of people using canvas in the real world.
"Those that start by burning books, will end by burning men."
Do you have reason to believe that Apple will include Vorbis support for Safari but MS will not support it for IE12?
As a content creator, I really like Ogg Vorbis. The limited number of devices and apps that support it is a drag. I hope it becomes more widely accepted. I like FLAC too, especially as an archiving format, but it can be an even bigger pain to get people to accept because support is so poor (especially on Apple). There are lots of Windows apps that support FLAC, but precious few for Apple. Fortunately, Sansa has been pretty friendly to both formats, so I recommend those players to my clients. Now if Sansa would only come out with a large storage device that has a screen at least as big as a Zune (and hopefully as big as an iPod Touch). I haven't heard anything about them coming out with such a device, though. Android devices look hopeful for the better formats.
You are welcome on my lawn.
For fuck's sake, when will people understand that the listings on w3schools.com ARE ONLY FOR THAT SITE AND NOT REPRESENTATIVE OF THE GLOBAL MARKET SHARE?!
I brought it up to show that when Apple needed a file format for the screenshots of Mac OS X, they went with PNG (which is open like Vorbis) while Microsoft created BMP.
My reason to believe this is the two lines you didn't quote from my post:
- Microsoft: BMP screen captures (Microsoft format)
- Apple: PNG screen captures (Open format)
If Apple was so anti-open-formats, they would never have picked PNG as the default format for screenshots. Microsoft has a history of always creating their own formats even when a multitude of alternatives already exist (kind of like Sony with their half a dozen Memory Stick types).
But I guess Apple could have used FLAC instead of creating Apple Lossless. There may be some technical or legal details we're not aware of. For corporations, it's not always the best technical solution that gets used because of legal reasons, licensing costs, etc.
Since the patents for MPEG Layer III audio have expired/are about to expire, shouldn't we use that instead of Vorbis or AAC?
Aren't trailers from that website encoded in H.264 video, AAC audio in a standard MP4 container? Shouldn't that play on anything from Windows Media Player to VLC?
No, if IE implemented it we'd be complaining that it's proprietary technology, when Mozilla implements it we just complain that it's a stupid idea.
I am TheRaven on Soylent News
Correction: That was HTML5 (JS, audio and Canvas) on whichever systems could hack it. Flash was the fall-back.
If IE still has more than 30% worldwide marketshare, and doesn't have basic requirements for this, its not going to be used. Period.
People who use "Period" to end their sentences are close minded absolutists with no capacity for argument, contrary views or the small possibility that they may just be completely wrong. Period.
Seriously dude, this is a technology news site. Period. A good proportion of readers will see this as being cool and may knock up a few 'tech demo' pages to do some tricky stuff. Period. That means the technology has been used. Period. Someone may come up with 'the next big thing' that the commercial sites decide they cant live without. Period. MS will eventually patch in canvas support and you can go and declare your authoritative prophecies on other ideas. Period.
Next time, try the following, less inflammatory statement and watch the world explode into smiles, cuddles and happiness:
If IE still has more than 30% worldwide marketshare, and doesn't have basic requirements for this, its probably not going to be used on any significant or commercial scale.
Uh-huh, it just doesn't work very well as an example, what, with there being a 10+ year gap between the selections of the formats and png not existing when Microsoft "created" bmp (it would be reasonable to say that they specified bmp, it is not very complex, it is basically a way of organizing raw bitmap data, with no support for any compression, so most of the bits in the file map directly to the display).
Especially considering that the paint.exe that shipped with Windows XP has (at least some) support for Jpeg, GIF, PNG and TIFF.
Using stupid examples doesn't really add any weight to your argument.
Nerd rage is the funniest rage.
IE9 will have Canvas
Climate Progress - Hell and High Water
Any chance google will help Microsoft along again? (Heh, I'll bet MS *loves* that!)
BMP was around long before OSX. What was Apple using before OSX?
PICT
Have you published your experiments anywhere?
I talked about them a little bit at the December BayJax Meetup, and I've been meaning to turn that presentation into a web page since then, but I haven't.
However... I've also been meaning to create an audio-focused JavaScript mailing list so that when I do put it up, I'd have somewhere to announce it. And thanks to your comment, I just got off my butt and created the js-audio Google Group. :)
http://groups.google.com/group/js-audio
I'd love to have you or anyone else join and share what you've observed and experimented with yourself, and I'll be posting my experiments there in due time.
Tweet, tweet.
Apple picked PNG because Microsoft didn't. And screencap image formats were hardly going to be an industry deal breaker. BMP files are so obvious as to really not give any leverage to Microsoft. Any patent on the silly thing would be overturned("Method for displaying a picture by lining up the colored pixels in order?").
Apple is pro-open-formats only if there's no way to get a lock-in. Apple is a patent holder with MPEG-LA, hence their preference for H.264.
Remember that there's nothing preventing you from installing Theora and Matroska codecs on your Windows box, same as with your Mac. Sure, media player will push you towards wma, but Apple will push you towards Apple Lossless.
"Microsoft has a history of always creating their own formats even when a multitude of alternatives already exist"
- iPad USB dongle
- iPhone/Pod connector
- Their old practice(no longer the case) of locking their RAM so only their RAM would work on their hardware
- The stupid one button mouse(yes, it's no longer the case, but it was).
Not saying MS is good. They're not. But neither of them are. Trying to say "At least Apple..." these days is a tricky game.
Also, MS has stated that IE8+ will handle WebM just fine as it can be installed via Media Player. Opera stated support. Mozilla breathed a sigh of relief and quickly said the same. Apple(or at least Turtleneck Prime) is the only one that seemingly indicated they wouldn't support it by throwing some lawsuit FUD around. They may yet, but the response wasn't exactly positive.
Actually, wtfjs.com is all about syntax quirks and nothing about the current APIs being an "amalgamation of junk" - and some of them are just "proof of concepts", not real problems, like using a reserved keyword as a variable name.
Besides, who is doing videoElement.play() or videoElement.pause() bad? It's a very clean API, imho.
Dilbert RSS feed
Indeed, Activision Blizzard, with revenues of $4 billion last year, is big enough for patent trolls....if they actually use Vorbis.
Having worked very closely with the source, including walking through the entire audio subsystem, I can assure you that the audio was always flash, and the rest of the game was always JS and Canvas, at least for the first day or so. They may have changed things late in the game.
Stylish sheet to fix many problems in Slashdot's D3: https://gist.github.com/801524
The boundary is being breached by the form field on your right column. One's eye just jumps right to it. I'm using Firefox 3.6.4.
...that was needed to be able to actually make a decent 3D game in JavaScript. Sure, I'd prefer OpenAL bindings, but this'll work.
Don't thank God, thank a doctor!
Microsoft: Windows Media Audio/Video (two Microsoft formats)
Microsoft has already pledged H.264 support (and AAC for audio) in IE9 out of the box, and WebM (VP8+Vorbis) if codecs are installed.
The current generation of AAC (HE-AAC/aacPlus/etc) is undeniably vastly better than Vorbis. In fact, the parallels with H.264 are stunning. HE-AAC is part of the MPEG-4 standard, it's not particularly impressive at high bitrates, but vastly superior at low bitrates, etc.
The only difference is that people have been rushing to put H.264 everywhere, while they've been reluctant to put HE-AAC anywhere, limiting their devices and software to simple-profile AAC, so it doesn't (yet) have the same death-grip.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Now if Sansa would only come out with a large storage device that has a screen at least as big as a Zune (and hopefully as big as an iPod Touch).
If you want an "Android Pod Touch", get an Archos tablet.
awesome, you could do that in a Win32 app 15 years ago.
You have to be a member of the Administrators group to install a native app. You do not have to be a member of the Administrators group to run a Flash object, a JavaScript program, or a Java applet, because the web developer can safely assume that every PC already has players for one or more of these.
Why didn't Microsoft simply use TIFF or TGA for the graphic format? Why invent BMP at all?
I'm amazed someone hasn't just entered a patent for "Stuff that does shit. Fuck yeah." Cause, given the state of the USPTO, it would probably hold up.
I scream. You scream. I assume that means we're both acquainted with the problem. We proceed.
The market share for Firefox is far from stable.
Here. If you dig a little on that blog you'll find the actual sources, which are a mix of reliable web statistics providers.
http://www.google.com/pacman/
Sorry to be pendantic, but it's JavaScript we're talking about here, not Java. Both are quite different things.
And I'm guessing that there is absolutely no point to this audio manipulation via "JavaScript" when it only works with one specific build of one specific browser's particular JavaScript implementation.
It's called a proof of concept.
If I have nothing to hide, you have no reason to search me
>> even text-to-speech
This is great. Now porn websites can save bandwidth by sending dialogs and love sounds as text, and letting them the T2S engine speak them out on your side.
They even play fine in Flash, funnily enough.
Demoscene in my browser!
: If IE still has more than 30% worldwide marketshare, and doesn't have basic requirements for this, its not going to be used. Period.
IE is being retrofitted:
~ Canvas: http://code.google.com/p/explorercanvas/
~ HTML, CSS, PNG: http://code.google.com/p/ie7-js/
When did Windows prevent normal users from just running executables?
Software Restriction Policies started in Windows XP. This is similar to the UNIX feature of mounting /home noexec.
Also worth pointing out that the majority of HTML5 browsers already supports both Vorbis and Theora. I think Safari is the ONLY HTML5-using browser that doesn't have support for Vorbis out of the box. I believe Chrome ALSO supports vorbis and theora (not just aac/h.264), and between Firefox and Chrome that's got to be at least, what, roughly 60-80% of HTML5-supporting browser traffic?
Since Apple® Quicktime® can actually be taught to understand Vorbis, Theora, and all of the other Xiph codecs system-wide (i.e. not just inside the web browser) with the installation of a single component (XiphQT), I think support for Vorbis is already better than people give it credit for...
Hacker Public Radio is our Friend
Good to know, thank you for the reply.
I have a sizable existing library of MP3 files. Would it be worth the trouble to transcode these into Ogg? Would this process be problematically lossy? And will iPods will play Ogg files? (This last is curiosity; I don't use iTunes, so it's not too much trouble to wipe my iPod and put something in it that plays Oggs.)
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."