Slashdot Mirror
Linux Cluster For Processing DSP Effects?
SpLiFFoRd asks: "I'm a MIDI musician and songwriter who seems to be constantly running out of processing power for my VST effects when I'm working in CubaseVST. Coincidently, I've also been working on explorations into Linux clusters and parallel processing. As I watched my CPU meter in Cubase hit 95% one night, I began thinking...'What if there was a way to farm out the work of my single processor to an outside Linux cluster with multiple processors to speed things up as well as enable me to run more simultaneous effects without straining the system?' I've been looking around, but don't seem to see any others who might have had this thought. This would be a tremendous real world application to me, and probably to others as well. Do you know anyone who might want to tackle such a project?" Sounds like a worthy project, but sound processing is still in its infancy under Linux, and while the horizon looks good with projects like GLAME and ecasound, I'm wondering if it may be a while before something like this will be feasible. That issue aside, however, I think something like this would be really cool. What about you?
"As a side note, when I say 'VST effects', I mean both the original and third party plugins for audio effects such as digital delay, reverb, tapped delay, pitch shifting etc. that work with Cubase."
See here for my earlier comment/response.
Some links to mLAN resources, by no means complete, please follow-up with more URL's if you find them (I don't have time):
Pavo (leader in 3rd party mLAN development): http://www.pavo.com/
Yamaha's mLAN (pretty pictures): http://www.harmony-central.com/Newp/2000/mLAN.html
Details on mLAN products (Japanese) http://www.miroc.co.jp/Tango/qrys/NEWS/yamaha/mlan .html
mLAN stands for "musical LAN", which is exactly what it sounds like. There is no point us re-engineering this for Linux, we simply need to work on making Linux a good platform for mLAN development, and we'll get to the point where we can route audio just like we route IP, and thus do things to it just like we do with the content fields of IP packets...
If you want realtime DSP-rendering farms based on Linux, then keep an eye on the work being done to add Firewire/IEEE1394 support to the Kernel. This is where it will start.
-- Jay Vaughan - jv@teklab.com
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Yes, I was going to post something along these lines myself.
Your main problem is going to be latency. Even a few milliseconds of latency is glaringly noticeable when working with audio. Many of the effects that you are wanting to use even use delays by this small amount to achieve their goals (phasing using around 3 ms, and flanging a little bit longer delays, chorusing slightly longer, etc.).
The second issue I was going to bring up is why use a cluster of CPU's at all? You do realize there are _MANY_ DSP PCI boards available that can handle the tasks you want. They aren't cheap, but neither is a cluster...
Creamware, Korg, and Digidesign all make boards that perform the tasks your talking about with several DSP's on a board. Going with a Pro Tools system from Digidesign allows you to add more boards as your processing needs increase.
Going with a computer cluster is the wrong way, if you need DSP's, by god use DSP's!
Okay, I get the difference you're pointing out.
:)
This doesn't make any sense in the context of audio, though. In the context of audio work, as *long* as things are being done in realtime, at a minimum, then you're alright. I'm speaking purely from a professional musicians perspective - not a compsci/compmus type researcher or engineer - because I believe that is the stance from which this article was originally posited.
Offline faster-than-realtime rendering of audio is typically not all that useful, since audio is a linear medium that must be perceived serially.
It is useful as a backend process - calculating reverb tails and filters, when designing a realtime DSP engine, really does benefit from faster-than-realtime processing, but it doesn't matter to the end user how fast it's done, as long as it sounds the way its supposed to sound when we listen to it in realtime.
You very rarely get a scenario where you're listening to how your compression sounds by 'skipping through the audio' in non-1:1 ratio's - at least, in my experience, this hasn't been very useful...
As *long* as the results are done in realtime, we're cool. And as long as the architecture supports realtime results, we're also cool.
So in my scenario, having the Linux "DSP FX boxes" accessible with realtime mLAN-type networking is acceptable and workable.
The typical musician usually just wants to add *more* effects to his mix, not more *complicated* effects (at least not in the midst of production of a track - in experimental mode, yes, sure, give me all the weird granular effects I can handle, but when the time comes to refine a track, I'll stick with what I know so far...), and thus being able to quickly and easily do this by adding a $600 P3box with mLAN/Linux support any time he wants to would be a productive architecture.
And this is what mLAN will give us, ultimately. The day may come when the average musician points at a rack of blinking lights and go "that's my PC-based effects farm, I add to it by buying cheap PC's and I get more effects busses instantly over mLAN"... I look forward to it.
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
"This seems like an expensive and inefficient solution for the problem you describe. PCs are actually pretty bad at doing signal processing; there's a whole class of chips (DSPs) that are optimized for it. There are almost certainly DSP-based hardware boards/widgets that will handle a lot of these effects for you, and it wouldn't surprise me to learn that there was a general-purpose programmable DSP card on the market too. Researching this before you spend money on a cluster would be a Good Idea."
y mbolic_Sound/PR/Kyma5.html
Yup. Here's some links:
Symbolic Sounds Kyma system, a DSP-based programmable hardware solution for audio.
Details on the Kyma system (drool!):
http://namm.harmony-central.com/SNAMM00/Content/S
Hardware DSP hacking tools:
Analog Devices EZ-SHARC/EZ-KIT DSP experimentation board (which screams for Linux support, incidentally)
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Two years ago the horizon looked good with projects like Audiotechque and Koobase. Today the horizon looks good with projects like Glame and Ecasound. Who knows why the horizon is going to look good two years from now. We always know the horizon looks good even if nothing materializes.
Unless the purpose is to study clustering methods it's usually cheaper to get a faster computer.
OK. There are a couple of possibilities here.
One - you're using real time effects processing. In this case, the latencies across a network would kill you. It is generally accepted that the human sense of hearing can distinguish sounds within 3 milliseconds of one another (Bracewell's Sound For Theatre.) 3 milliseconds latency is hard enough for software on the host PC to meet as a goal - keeping the response of all the computers within a cluster to under 3 milliseconds distance from one another would be difficult. A trained musician (I live with one and have tested this, FWIW) will get very confused if monitoring latency causes them to hear what they're playing more than about 10 ms from when they play it.
In the second case, non real-time post-processing, a cluster would be more useful. Essentially, it would be a parallelization process - each computer in the cluster gets a chunk (perhaps even allocated by some measure of speed) and they all grind away seperately and return their results to the host, which reintegrates the data into a continous whole. However, problems arise when the effect you're using is a time-domain effect. Frequency effects (filters, EQ) and amplitude effects (compressors, envelopes) won't have problems with being split, but time-domain effects like reverb and echo will have an issue. Imagine if the echo from one segment spills over into the next? I suppose you could extend the echo beyond the given data, and the host machine could mix them with the next machines data for a result. But what about multitapping echoes, where the echoes can themselves be echoed? Problems arise even with non-realtime processing.
The ideal solution is dedicated hardware, either internally (programmable DSPs with VST plugins for them) or externally (banks of I/O with external reverb and such). Both probably cost more money than software, but you pay a price for realtime. mLAN is just an implementation of the latter idea, using Ethernet for the I/O and the dedicated DSPs for the effects.
Finally, for all you Linux weenies who are crying "Why Windows for audio?" - I hate to break it to you, but Linux isn't very good for real-time. In fact, in some ways it's worse than Windows. I'm not a programmer - these are the words of a friend who is, and manufactures timing-critical show-control hardware and software. For a long time (as long as it was practical) they used Amigas, because the RT support was better. Eventually, they moved over to NT, because it was the best available.
So, for the foreseeable future, it looks like the best investment is still decent analog gear - after all, the EQ on my old RAMSA analog mixer doesn't add 5 ms of delay to my signal.
---
Mod me down, you fucking twits. Go ahead. I dare you.
(I read with sigs off.)
I'm not an expert on clusters but it may be simpler to use something like MOSIX
http://www.mosix.cs.huji.ac.il/
MOSIX uses transparent migration of processing amongst other nodes on an Ethernet network [read the site for more particulars]. It's also pretty easy to set up [basically just run the install script and make sure the nodes can locate each other].
Might be helpful if Beowulf is a bit much for your needs.
http://www.yamaha.co.jp/tech/1394mLAN/mlan.html-- detailed mLAN specs.
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
This seems like an expensive and inefficient solution for the problem you describe. PCs are actually pretty bad at doing signal processing; there's a whole class of chips (DSPs) that are optimized for it. There are almost certainly DSP-based hardware boards/widgets that will handle a lot of these effects for you, and it wouldn't surprise me to learn that there was a general-purpose programmable DSP card on the market too. Researching this before you spend money on a cluster would be a Good Idea.
Assuming that the software's there, dedicated DSP hardware should by far be the cheapest and easiest solution.
OTOH, good luck finding the software.
Addendum: Actually, I think that the effects generators on most modern sound cards are just DSPs, RAM, and some firmware. You might be able to find (or write or contract) hacked drivers that would let you arbitrarily reprogram this for your own purposes.
Just a few thoughts.
It's at:
http://www.digidesign.com/ptfree/
I don't wanna press my luck, but how about a free linux version, Digidesign?!! ;)
W
-------------------
-------------------
This is my SIG. There are many like it, but this one is mine.
I do recommend Digi, though I don't actually use it- I use hardware analog mixing and limiting (heard on my most recent album, the tracks named after airplanes). I do think that with enough skill and dedicated analog gear you can top the quality level Pro Tools will give you (though if I had Pro Tools- I could use _that_ as well and do even better. So even then, Pro Tools is desirable). However, I have to seriously confirm all that Funkwater says here: you don't want a Linux cluster for DSP. Maybe you want Linux support _for_ the hardware DSP you can already get, so you don't have to run a Mac or Windows. But you don't want a Linux cluster, unless you have some sort of non-realtime arrangement that can make use of insanely demanding 128-bit calculations to slowly 'render' a final track far better than even modern DSP allows. However, we're talking audio- that's hard to even imagine, and the DSP _is_ out there and very capable.
As it just so happens, I'm the chief tech at a service that does basically this. I think we're within a month of being able to launch our public beta. Contact info's on the site if you wish to use our services. We're doing Maya rendering first, but we would be more than happy to add support for other formats (possibly including processing music - we hadn't thought of that one - and definitely including stuff like Premiere) on request, though of course we'll need a bit of time (hopefully not more than a week or two, but no promises yet) to add more formats.
Yeah, so this post is a shameless commercial plug. It's also what was requested.
Exactly...mLAN will help this out tremendiously, but it is only a start. Saying that mLAN will be the cause of Audio Rendering Farms is a bit premature though. Right now, it is little more than a fast network dedicated to audio, though I've seen some really fricken cool stuff come through in the last few week. I too am stuck under too many NDAs to talk about this and kinda wish that this topic had come about in a month. NAMM, the music industrys premier showcase of technologies, is almost on us (doh! gotta finish up some demos soon) and I can guarentee we will be seeing some of the technology that will enable this kinda processing to happen.
The greatest thing about mLAN is that, unlike MIDI, it will allow both the Audio and Data layers to be transmitted syncronously and we will soon be looking at other paradigms that us electronic musicians (and probably the folks that have stayed away from electronic music because it was too geeky and technical). Yeah, I'm just reiterating what was said, but few outside the closed music communities know whats coming up.
The only real problems I see right now is everyone is focusing their attention on Mac and Windows. The few cool pieces of software that I would consider to be professional level are getting their developers eaten up by these Mac/PC Only developers. Our staff has been heavily pursuing one company to produce a version of their software that could be run from a modified linux distribution from simply a LCD and command line. Once we can get that, run it in server mode and let the server parse the different tasks to different machines. I believe jMAX, which is a derividation of IRCAM / Opcode / (MSP???)s MAX application...only running from a Java front end and running in Server Client mode. Heh...I haven't been able to get it working on Mac OS X yet, but it is technically what we all want (I hope...I've only read about it so far as I've already got the regular MAX running and haven't put the time to it). If that could run in Server Client mode, whats to split the polyphony and have multiple servers for the parts of the audio that don't need to modulate the other parts.
blah blah blah...ask this question again in 4 weeks...I'm sure a few of us will be able to open our mouths (if'n these guys actually produce...still waiting for a few products from the last winter NAMM to be announced at least from a development sake...)
clif
The correct question is "Would I be better off writing this with vi or Emacs". In the open source world, the idea is not to get people to give you free software, it's that people want something done so they do it. If your idea is interesting enough, other people will get involved.
Rich
that kindof thing would paralellize well...
tagline
... hi bingo
I've been wanting something exactly like this for a long long time.
I do editing using Adobe Premiere on my Macintosh. When I finish a project, or finish a "stage" of it, I have to render the project overnight. My next editing machine will likely be a dual processor PC, but for now I'm using my Mac.
In addition to my Mac, I have four other computers available, all of which will happily run Linux. Setting up a Linux cluster would be a good project, and is definitely feasible. But my Macintosh has no means to offload the rendering to a cluster...
I think this would be a fantastic product, I can see it maybe as a wholly separate product. It would run separate from the actual editing application and distribute its rendering load to some sort of cluster (I'd assume a customized Linux cluster).
Computers are cheap now, but it is expensive to buy very fast machines every few months... why not allow for clustering of your old machines (even if you do replace the primary one every few months)? Then you could still use those extra CPU cycles, and maybe you could actually use your money effectively.
Isn't the problem faced by distributing computing such as this the limitation of the network bandwith between machines? Assuming you connected machines via 100MB/s, wouldn't you top out real quick? The processing speed compared to what it can send of the wire would make for a few delays I would think.
Anyone have any feedback on using distributed computing for such "Real-time" things such as Video and Audio?
If you did your effects in Mpeg 4 structured audio instead of MIDI, you might get considerably more performance.
Why? Because there is considerable research in compiling MP4-SA to C and then running the native code, to get greater performance out of arbitrary effects, filters, etc.
More info is available here
Nicholas C Weaver
nweaver@cs.berkeley.edu
Test your net with Netalyzr
This is a natural for parallel processing.
--Mike--
Friends don't let friends use multiple inheritance.
Yeah? Well, you use "420" in your domain name, so I guess you must be a stoner.
Stupid assumptions, made liberally.
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
... all it takes is someone willing to devote the time to add mLAN support.
... once this works, I'll consider doing mLAN driver work for Linux.
http://www.yamaha.co.jp/tech/1394mLAN/mlan.html
Can't do much better than that, honestly. And mLAN hardware is out there right now, there's no reason enterprising developers can't get started as of today.
I'd do it, but I'm working on stuff that sits on top of mLAN first on Mac/PC
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
It's more efficient but not more cost-effective on a large scale. 16 or 32-processor computers like SGI Origins cost about $1,000,000. A 128-node beowulf cluster costs a mere $700,000 about. So which one's more cost-effective?
# debian/rules
That clears it up a bit, but I would still be worried about the network performance between machines. If you're maxing out your 100MBs link, doesn't that create some sort of delay with the machines synchronizing themselves together? Or is the delay negligible enough given performance gains.
For effects on a file the machine would be able to wait. But what happens when this theory gets applied to real-time things like Video Editing?
Without a fast enough hard drive, you loose frames. Without a fast enough backbone/backplane between machines would a similar effect happen?
... before MP4-SA can begin to function, and I mean *function* in a professional studio in lieu of MIDI.
... which, by the way, is one of the reasons I'm a very strong advocate of musicians *staying* in the hardware realm for now, and leaving the software plugins alone.
There's a lot to say about MIDI - sure, it's slow, sure it's limited, but as far as serial control protocols go, it has *definitely* weathered the test of time.
Anyway, this slashdot article is not a problem of MIDI, but a problem of i/o architectures.
The problem with realtime Cubase effects is that they're still very processor/architecture/OS bound. There's no functional way of getting data in and out of a Cubase effects module other than by using the strict architecture path that the effects software has been written to use - this problem exists for Cubase plugins, ProTools/TDM plugins, DirectX plugins, etc.
A better alternative to investing energy into the MP4-SA side of things (which is not a new technology - for a similar example, check out Thomas Dolby's Beatnik, which is a fairly similar implementation) is to instead investigate Linux' support for mLAN, which is Yamaha's "musical LAN" protocol that sits on top of Firewire/IEE1394.
mLAN is the solution to this article's stated problem.
By adding really good support for mLAN in Linux, it's really *NOT* unreasonable to say that Linux clusted boxes could be used for REALTIME DSP processing tasks. mLAN represents audio, video and media-data as individual streams on the same wire - a very good implementation of mLAN/Firewire in the Linux kernel would allow you to separate these streams, route them off to different processes/processors, and stick them back on the mLAN wire in realtime.
This will happen in the PC world.
There are already Windows-based realtime DSP effects boxes that use mLAN as the routing interface, currently in *development* by a number of large music companies (I can't drop names here, I'm under very tight NDA).
With good Firewire support, Linux (and other OSS OS'es) can keep up with these developments, which will hit the market, I'd predict, by Q3 2001. Put the audio out on the wire, route it wherever you want to route it, and get it back off the wire, and we'll never have these limited 'plugin' problems again
The audio world has *ALWAYS* been about interoperability, open specifications, and shared protocols. MIDI would never have come so far, and given us so much, if it weren't for the fact that hardware manufacturers took the time to figure out good ways to work together, in spite of competition. Audio effects architectures have always been very well documented - the average professional mixing desk has been designed from the ground up to be able to work with gear from *all* sorts of manufacturers, using simple protocols and specifications for audio signal routing. (XLR, balanced, etc)
This changed recently with the advent of software plugins, where hardly *ANY* companies are truly working with others to build a common platform - Oh sure, Digidesign have their TDM system, as do Microsoft, as do Steinberg, but pretty much all of these systems were developed from the ground up to be proprietary and to be used in giving 'market edge' to the respective owners.
Next year, software plugins won't be as badly implemented, and the audio world will actually be able to work cooperatively again... because of mLAN.
So, we should work on getting good mLAN/Firewire support in Linux, and develop open API's to solve the problem. Then you can build your Linux audio-rendering farm, hook it all up with mLAN, and forget about proprietary crappy interfaces for your audio work in the future...
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
I think that such a thing would be awesome. I'd like to see *all* processing done by the cluster, using the host recording PC simply for I/O and possibly as the storage locale. (Perhaps file storage could be sent to the cluster as well...)
Regrettably, I think the host program (CubaseVST in your case) would have to be heavily modified to allow it to understand that Cubase itself will not be processing the requests.
Perhaps someone currently beginning development on a Linux Multitrack environment from scratch would be interested in this idea. I think it would be much easier to implement from the start than try to convince existing manufacturers to chnge their philosophy.
Another multitrack function which is badly needed is a global file management. (More like a document management system, but for sound events.)
I have roughly 30 GB of WAV material ranging from small loops (50K) to whole mixdowns (50MB). It gets hard to find the sound one is after.
How do I find an example of your current Cubase material?
If the effects are being done in real time, Beowulf probably won't work very well. The inter-node latency would be a killer.
If you're doing non-real time (from a score/cue sheets etc), a beowulf could be helpful. The best bet would be to lay the local network out as a ring (two nics in each node, crossover cables interconnect the nodes in a ring). Each time segment could get processed by each node in turn like an assembly line until it ends up back at the master where it goes to the output file. The software would be 'interesting' to write, but doable.
The only probblem would be setting up for road shows. but for studio work, it would be great.
Maybe I can build me a wall of sound like Raymond Scott did.
(side note: Raymond Scott invented the sequencer, and was a teacher of Robert Moog. His novelty tunes were used widely by Carl Stalling for themes in many Warner Bros cartoons.)
"It is a greater offense to steal men's labor, than their clothes"
Given that network bandwidth would quickly become your bottleneck, it seems that going SMP would be a *much* more efficient (and cost-effective) solution.
Slashdot: come for the pedantry, stay for the condescension.
..for distributed computing.
Where you can use any other type of OS, and send a work order to a clustered network, for processing.
Of course, for this to become reality, it would require software manufacturers support. Which is where things fall out, not because of ignorance, but it just wouldn't be cost effective (why spend extra money on a feature few users would be able to use?).
This then presents another problem, inhouse development by individual companies based on need, with no outlook for distribution of this work. It's cool that some companies would be willing to sacrifice resources to develop a distributed solution for a product that they need, but there's no guarentee that they'll release it to the public afterwards.
Cheers,
leroy.