Slashdot Mirror
Software Based Echo Cancellation?
"For some more information, my office is trying to get a small system up and running that will allow multiple locations to video conference together. We have some specific requirements and have a fairly good handle on the entire video part of the problem.
However, we are running into problems with parts of our audio mix. The first issue is something that (I believe) is called 'mix minus.' This means that in a group conference, speakers do not have audio sent back to their location. (This is important for various psychology and network latency related issues.) There are several hardware based solutions that are available and we have some software based options.
The larger problem is echo cancellation. As many people may need to speak at once (and to avoid the requirement of having individuals constantly muting their microphones), we would like an echo cancellation component. The ideal would be a software solution that we could run locally, perhaps in conjunction with the same code running on the remote systems. However, most of the solutions we have found are hardware based (DSPs, ASICs, etc.).
The technology used on the studio side as well as the host side will involve various operating systems. We are trying to avoid avoid relying on specific OTC hardware solutions (namely, sound cards) as we would like to be able to create a solution that would function over time, particularly as specific hardware solutions tend quickly to horizon. So, having nice code that could be compiled on different systems would be a plus. Ideally, we would like to minimize the amount of hardware necessary, so an echo cancellation algorithm that could run in conjunction with other processes would be nice, but it is not a requirement."
wouldn't the echo cancellation itself propogate over time and create artifacts in the sound? Or am I totally misunderstanding the issue altogether?
Jobs? Which jobs?
HTH !
The reason you're finding more hardware tools than software tools for echo cancellation, among other things, is that the telecommunications industry demands these sorts of things with much more fervor than the average consumer. Echo cancellation devices (for example, a codec with echo cancellation built in, running on a DSP) are used extensively in cellular telephones, voice routers, and this sort of thing. Your best bet, in this respect, is to find a company that is willing to release the source code to the software that is running on your hardware.
Alas, I do not know of any software, especially open source or free, that provides a full suite of audio processing utilities. Why is it that you're against using hardware, in the first place? Too expensive? Those are the breaks.
Hey, at least he didn't leave the caps lock key on again.... I had a headache for that entire week...
As many people may need to speak at once (and to avoid the requirement of having individuals constantly muting their microphones),
Why not just install a noise gate at the microphone inputs?
For the non-audio-inclined slashdotters, a noise gate sets a minimum sound level threshold before the signal is transmitted.
sed 's/^echo/#echo/' /etc/inetd.conf >/etc/inetd.new /etc/inetd.new /etc/inetd.conf
mv
kill -HUP $(ps -ef |grep root.*inetd|grep -v grep|awk '{print $2}')
no more echo
Slashdot always italicizes quotes.
Get with the program.
--
Does anyone remember
There's an excellent open-source PBX called Asterisk. Among other things, it provides an MMX-optimized echo-canceller. Look here
Maybe someone at /. will find an answer for you, but I would be surprised to see this implemented in any kind of stand alone SW package. Because it is a specialized real-time application requiring fast feedback, it makes sense to implement it as an embedded system (i.e. in hardware).
Back in the day when 56k modems were taking off, there was a large piece of software people were coding into drivers called 'Ring Cancelation'.
:-)
These were added because when you send data down an analog line at high speeds, you begin to hear an audible sound which sounds like ringing. The modem drivers needed to be able to tell the difference between this ringing sound and the actual data.
I think a good place to start if you cannot find any software is perhaps hacking these drivers or something along those lines.
It's a good start at least. Hope this helps
This is a great question you are asking, and I would love to see a good answer. The shame of it is, I'm expecting to see a bunch of posts in response to this saying "If you need one then write it yourself".
Is it just me, or does it seem like the open source offerings for things related to audio/video are lacking in general? I wish I had time to make improvements myself, or the money to contribute to the developers, but it seems like we need more in this area to be able to be more competitive with proprietary solutions.
Sigs are out of style, so I'm not going to use one...oh wait..
$count = 0;
while ($count {
if((strtoupper($parent_post[$count]) != $parent_post[$count]) || ($parent_post[$count] == " " && $parent_post[$count + 1] == " ")) echo $parent_post[$count];
$count = $count + 1;
}
?>
There, it removes your echo and your lame allcaps...
Err. Ignore the above. This browser sucks like Hoover.
If you think about it, most of the front page is usually quoted (since the editors try to minimize their own writing), so italicizing the whole page is probably a pretty close approximation.
Karma: Good (despite my invention of the Karma: sig)
You're probably not going to find a free software echo canceler. I've been working on audio processing for a cell phone design and AEC (automatic echo cancelation) is budgeted for a lot of time and money. It's a pretty involved signal processing function that is well-suited to optimization on a specialized processor, hence the number of solutions you're seeing for ASICs and DSPs. The bottom line is that I would suspect that you're probably going to have to pay for it.
Echo cancellation is a classic application of adaptive filters. Every reference ever published on the subject discusses it. I like Haykin's book myself.
I just did a search on Google and came up with 4000 references.
The underlying theory is pretty hairy, but the implementation of an algorithm like LMS is straightforward.
...laura
Are you talking about acoustic or electrical echoing? Have you taken a look at ITU G.168?
It is as far from the open source as it gets, but the Windows Messenger includes voice features and has a software echo cancellation. Little known fact: Messenger 4.6 does not need Passport for operation, in closed enviroments any SIP server will do just fine. Also, encoded media stream it outputs uses standard encoders and standard RTP protocol. So Messenger can interoperate with many applications.
The Access Grid is a project started at Argonne National Lab's Math and Computer Science Division to build a mostly open videoconferencing system over the Internet, using multicast audio and video streaming. You may want to take a look at their technology to see if they have ideas you can use.
Anyway, a "node" on the Access Grid consists of a room with at least three computers: a multihead box running Win2k for display to several video projectors, a computer running Linux for audio capture and playback, and another running Linux for video capture. The audio capture machine usually runs into a Gentner AP400, which does echo cancellation as well as phone bridging.
I don't know of anybody who has software that does this; sorry.
"My life's work has been to prompt others... and be forgotten." --Cyrano de Bergerac
From the article:
.
Echo-cancellation software algorithms are embedded in programmable DSPs
Though the article refers to "software" I think this is still refering to the kind of embedded solution the poster was hoping to avoid.
I remember when I worked at Tellabs we had a product, EC-8000 Digital Echo Canceller Might be worth a look.
Notably, a lead such as: http://www.nist.gov/speech/tests/ctr/h5e_97/echoca n.htm
Sig: What Happened To The Censorware Project (censorware.org)
A common algorithm (I believe it's used in modems) is the Widrow Noise Filter. I had to write one in a CS class. It's a type of neural net. The general algorithm for noise filtering is sort of machine intensive, so it's reserved for high end $$$ equipment (if I recall the lecture properly). This filter is a quick write though, and will get you set straight.
I was amazed at all the processing going on inside the DSP back when I tested, and later debugged/coded, modems for a living. When you mentioned echo cancellation, it was the first thing that popped into my head.
A logical extension of this for your application would be to try to get your hands on some source code from a "Soft" modem. The idea was to move the most intensive processing out of the DSP and onto the PC processor since they were, in theory, becoming powerful enough to handle all the operations in real-time. Actual performance of these types of modems is a completely separate story, but the echo cancellation code is out there somewhere. At thing point it should just be a matter of getting your hands on it.
Check out Analog devices; their 2181 demo has echo cancellation as a part of the included software; source included.
Is that site for real? The only story I read had like 6 completely clueless comments (e.g., xmms skips because you need a faster hard drive, or because 2.4 is bloated...huh?). While the S/N on slashdot is admittedly low, at least there is a signal in there somewhere.
Karma: Good (despite my invention of the Karma: sig)
runs on windows (NT/etc.) and linux, OS X port is in the works, IRIX is also supported. it's fast and very flexible, and will support a simpler solution - like gating, matrixed switching (if you've got separated inputs at hand), whatever - if you find one. the pd-list is quite supportive.
[|]
We are trying to avoid avoid relying on specific OTC hardware solutions (Emphasis added.)
Looks like this message was dictated with intermitent echo!
Education is the silver bullet.
...is because it doesn't exist.
Realtime processing, AFAIK, be it audio or video, is astonishingly processer-intensive. It doesn't surprise me that DSPs are being used for this reason: they may be the only thing that can cut it in a cost-effective manner.
i.e. you may be able to build a high-end workstation, and write some real-time software to handle this task, but since it probably wouldn't be able to do anything else at the same time doesn't that qualify as a hardware solution?
Perhaps instead of going to extreme lengths to remove echos, perhaps you just need to work harder to prevent them in the first place? Pro audio mags have tons of ways to reduce echo and other unwanted effects in small (usually home) studios. Have you looked into this?
Michael-
You catch enchiladas by picking them up behind the head and holding them underwater until they don't kick anymore -VeGas
Look here for my echo-cancellation code:g z
http://speex.sourceforge.net/audio/sndio.t
It's bundled with open-sound calls to read and write audio in real-time, while removing acoustic echo from the input. There's not much doc, but the test2.c program is quite simple. Feel free to contact me at jean-marc.valin@hermes.usherb.ca. Note that there's no real project (sourceforge or other) assiciated to it but if you find it useful, I may create one.
Opus: the Swiss army knife of audio codec
1) Use directional microphones, or else throat mikes. This will make the neigbour's microphone only pick some one up very quietly, if at all.
2)if there is still some echo problem, it should be quiet enough that simple (software) noise gates should solve the problem.
god forbid anyone use the software for something. He doesnt want to rip off the code, he wants to make USE of it.
I've been hearing about some new technology from Macromedia that might make your life a *lot* easier. Apparently the Flash 6 plugin supports hooking into both webcams and mics (after the user OKs it) as well as special socket-based connections to a new piece of server software codenamed TinCan. In addition they've talked about the server supporting shared objects as well.
From what it seems you're able to put code on both the client and server and both are based on ECMAscript. This would let you do a lot more than nearly every other solution I've ever seen. I don't know when the server is supposed to be released, but if you check up on the recent interviews with MMs CTO Jeremy Allaire on C|Net or The Register you'll see that they seem to be hinting that it will be available later this year.
A|Q|U|A
Most solutions offered by Ditech, Telogy, etc. cancel the electrical echo caused by an impedance mismatch 2 to 4-wire hybrids in the analog part of the Old Telephone Network. You seem to develop a packet-based videoconferencing system, which has no hybrid in it, so you must want to cancel acoustic echo, caused by reflection of the sound produced by the speaker-phone on the walls of a conference room.
This is a very hard problem, because you have to modelize the environment of each conference room. You will have to guess mathematically (with the LMS algorithm for example) the echo response on a tail of at least 128ms for each room, which would take at least a few minutes to one hour on a P4 2GHz system.
And what about if a door is suddenly closed in the conference room? Or what if the speaker phone is moved? You will have to re-modelize your echo response each time that happens, because the geometry of the room will have changed.
The solution is surely not a software echo cancellation system, at least not before 2010.
Think about a hardware solution, DSPs or ASICs (http://www.octasic.com)
You just need a better microphone. Use a teleconference unit from plantronics. As a side comment, looks like you're reinventing the wheel. There are a lot of off-the-shelf video conferencing systems out there that'll probably cost less than all your TIME spent tinkering.
In other words, either (1) exchange the room speakers for earpieces or headphones or (2) exchange the room microphones for lapel or boom microphones. Clearly the degenerate case has everyone essentially speaking into a separate telephone receiver which probably defeats the purpose of the system altogether.
Of course, it would be way cooler to have a setup where the room microphones are aware of the room speaker output and automatically cancel it out. The trouble is it's way cooler because it's so difficult to do.
--- Jason Olshefsky
Karma: Poser (mostly affected by adding this line long after everyone else did)
Just say to yourself "There is no echo" until you believe it.
I'd love to see a list of all the other software you have found.
Why not? While at a given speed, a DSP will outperform a Pentium (assuming you aren't doing something dumb like trying to do floating point on a fixed point DSP), I haven't seen any GHz DSPs out there. Also, keep in mind, this is the kind of thing that SIMD architectures (like MMX, Altivec, etc.) were designed to speed up.
But those are not echos. Those are real signals picked up by other microphones. It might be difficult to DSP that.
I guess I don't see why NOT routing the audio back would be a problem, or maybe I don't understand the question.
Otherwise, save your paper towel rolls, and hand them out before a meeting. I don't do this for a living, so YMMV.
well, since you asked for a 'soft'-ware solution, i suggest using foam padding on the walls surrounding your conference room. altho i don't understand why you're so bent on the material being soft. foam is soft and from my very-basic understanding of acoustics, it dampens and reduces echo. wait, oh... that software!?
I doubt you will find an open source echo canceller, since acoustic echo cancellation is pretty difficult (and has generated many, many patents). Nearly everyone uses a different, proprietary algorithm.
If you want to make one yourself, set aside about 10 months.
Lots of people were modded up in this discussion proposing wrong, irrelevant or incomplete solution. Typical /. ...
cheers
Assuming you want software, I would think the best way to go would be Realtime Csound. If you're not familiar with the csound language, its very powerful for dsp and was designed for manipulation of audio. It runs on most platforms and processors. Different implementations of csound designed for realtime use exist for both cisc and risc architectures. Assuming you can design the appropriate algorithm for echo cancelation, csound may be ideal.
Another option, also assuming you can design the algorithm, would be SuperCollider, which is another audio processing language that tends to have better realtime performance than csound. Only runs on macs tho.
Finally, in terms of techniques, you might think about 'shooting' the various rooms (by recording a balloon popping) and using the resulting impulse data to remove (rather than add, as in conventional convolution) the echos on voice audio from that same room. FFT might work here also, although probably less effective. SuperCollider can do realtime convolution; csound might be able to, depending on how high of a sample rate you need.
#define AdaptationRate 0.99
;-)
// Basic adaptive LMS FIR algorithm.
float EchoCancellation(float Sample)
{
static float History[MAX_ECHO_DURATION+1] = 0;
int i;
float AdaptationRate;
float EchoAmpl;
for( i=0; iMAX_ECHO_DURATION; i++)
{
EchoAmpl = History[i]*Coef[i];
Coef[i] *= AdaptiationRate*(Sample-EchoAmpl);
History[i+1] = History[i];
}
History[0] = Sample;
return Sample-EchoAmpl;
}
That's all the "basic" science.
You might find, that for 40kHz and 250ms echo this is too computationally intensive for a single Pentium. You may need some 1200 MIPS.
You may then:
1. Use Athalon
2. Convert it to pointer arithmetic
3. Convert it to integer arighmetic
4. Skip some samples for echo estimation, sometimes
5. Contact me to use more clever algoritm (IIR?)
(Petrus.Vectorius@ied.com)
if you are creating your studio then you need to make the studio fix the problem first, dont try to compensate for a crapy studio in the recording hardware/software.
#1- Sonex, sonex, sonex. If you dont have sonex or the crappy sonex copy or even just carpet on the walls (Yes wall carpet looks good) along with the roughest texture ceiling tiles you can buy at the home-depot (or better yet the $90,00 a 2foot square city scape audio ceiling tiles) then you are wasting your time. it takes very little to make a room acoustically deadened to the point that properly set up microphones wont pick up any perceptiable echo. (Note: if you have you're mic's set so your artists or voice talent is farther away than 3 inches from the P popping screen then you have it set wrong. also dont let the talent talk quietly, make then talk or sing loud to overcome room acoustics.
start with the low tech, then add your high tech bandaid filters.
Do not look at laser with remaining good eye.
Pure Data is a real time sound-manipulation program that runs on Linux, Windows, and Mac Os X. You'll be able to design any kind of sound processing algorithms you like but be warned: only Linux gives you input-output latency as low as 3 milliseconds. In Windows you'll have to settle for 300 ms unless you buy some fancy audiophile soundcard that supports Steinberg's ASIO.
Hell I helped build one. And while there is a LOT of noise cancellation and "echo reduction" software on the market (Cool Edit Pro has a few nice plug ins) the sound quality after applying such a filter could at best be called "fair". Unfortunately your best solution is to find a high quality mic with a bit of noise cacellation (and the higher end ones can be "tuned" with a hardware equilizer) and just suck it up and BUY THE FOAM. I know it's ugly. I know it's a pain in the ass. I know it's only effective if the studio is designed well, but nothing that I have personally seen (well under 40k that is) beats the stuff. Acoustic dampening foam is your cheapest option that will still maintain audio quality to a reasonable degree.
Find out about my new childrens book: SS Death Camp Criminal Batallion Go To Monte Carlo For The Massacre
thorbo is right. The speed of sound at sea level is something like 700 miles per hour (Mach 1). If I've done the math right, that is 1,026.67 feet per second. A 10-foot separation will cause a 10 millisecond delay. I believe a 50-millisecond delay is heard as echo. Can anyone help about this?
http://www.intel.com/software/products/perflib/
...it is quite clear that this gentleman is talking about feedback and not room acoustics.
;) In this case then the use of high quality mics/headphones is totally unnecessary. This also offers the users individual monitoring control.
In my experience such hardware or software solutions do more harm than good because they cannot distinguish between wanted and unwanted sound (unlike one's auditory system).
Taking the time to set levels and proper mic/speaker placement should be sufficient. But to totally remove the problem then headphones really are the simplest option - just don't forget to turn the speakers off too!
If I understand correctly, by "echo cancellation", you mean that, when several folks are talking at once, you want each person to hear everyone except themselves. If that's the situation, I would suggest flipping each speaking person's outbound signal 180 degrees out of phase with their inbound signal. That way, their own voice will cancel itself out in their personal conversation "mix". Make sense? To flip an audio signal out of phase, you delay it by twice its frequency. In other words, if your signal is 15 kHz, that's 15,000 waves per second. Or, 1 wave per 1/15000 of a second. If you mix in another copy of that signal which has been delayed by 1/30000 of a second, the two signals will be 180 degrees out of phase, and will cancel each other out. It is important that you only flip the speaking person's voice out of phase with the incoming signal, so that her voice is only removed from her inbound mix.
@sshatrack
We implemented an echo cancellation algorithm for a GSM phone. It required somewhere around 2-4 MIPS from our 26 MIPS budget. You should be able to find some app notes you can use to implement on a very inexpensive TI or ADI development kit.
DSPs are historically very very slow in frequency when compared to other embedded CPUs (although this is far from true today). Until the past few years, DSPs have been lingering in the 10s of MHz range, while embedded MCUs were venturing into the 100 MHz range.
You can probably implement on any MCU that runs around 20MHz. This should provide you the low latency you need and more than ample MIPS capacity.
best regards,
mega
Guys.... Actually this does not require any computing power at all.. Get some highly directional microphones (directional cancellation)...and if that doesn't work, well, you can always get two microphones..... very near each other you point one at the sound source (person speaking) and one away (use two Shure SM-57's)....you using the balanced audio pins 2 and 3 you hook pin 3 of the away mic to pin 2 of the person mic and likewise with the other two, now hook the two combination up to your input...walla! Analog, non-powered background noise cancellation using the miracle of 180 degree phase cancellation. this equates to.... person + noise - (noise) = person talking... person + feedback - feedback = person.. I know this is hard to type out, but it's a technique I've been using in the pro-audio world since about 1980....and it still works...of course you could (and I usually do) use a very small mixer with variable levels on both mics.... that way you have some control over the '3 dimensional' 'sound' of the input....natural reflections from the source and natural slight delays and echos from the rooms environment. John
the link is interesting though misspelled. Correct would be:a re/legacy/fir_echo_canceller/
http://www.isip.msstate.edu/projects/speech/softw
I haven't tried it myself (I obviously know very little about the matter altogether), but it seems to fit the description of the original problem pretty well. Doesn't look like quite a finished product yet, but, nevertheless, very interesting.
Jobs? Which jobs?
That third link was to the Jack Project. And it was supposed to end that sentence and not go on and on and on.
I must admit that I've never dealt with echo cancellation as a large obstacle. It is generally something taken care of by the sound engineers along with everything else. And artists: when selecting an engineer, don't pay attention to name-dropping or even reputation if you haven't heard his (or her) work yourself. In this business, a DAT is worth a thousand words, and listening to what your engineer has done in the past will not only help you avoid difficult hires, but will give you context when working with this person.
As for digital effects, I have had some experience with Akai products, but mostly regarding live performances. Again, I tend to leave the hard labour to my engineer(s). After all, I do not ask them to sing songs for me.
"I'm a rocket man / Rocket man burning out his fuse up here alone." - Sir Elton John
If you sent the audio from each source as an individual (multiplexed packetized) stream, at each source you could just not play the stream (any packets) originating from that source.
Simple and Effective, but would require sending more data than mixing one version at a central location and sending that version to all conections (but that would be silly anyway due to lag).
But generaly you wouldnt have more than one person talking at a time and when it happens if you are using a decent compression targeted to voices (look at the compression schemes used in digital mobile phones (both GSM & CDMA)) this shouldnt be to much of a bandwidth problem anyway
If what this comment describes is correct, it wouldnt be touch to code this yourself. Find a decent Sound API you can use and cache the sound as it is received. Then remove that sound from the outstream with a small delay, which you might have to make optionally tweakable.
Question
http://www.ironfroggy.com/
Most of your open source software that you'll find relating to this issue will be based on adaptive filtering, which when considering time delays over 1 second, this becomes infeasible, even at a 8000 Hz sampling (I imagine we have a few channels to process on each end). As a result, you're going to have to either buy some more hardware or write it yourself. Or you could find some geek willing to write a small package that does the following:
Generate an impulse test signal on one end, and using the mic/speaker set up at all the other points of the VTC, measure the return. This will give you a bunch of ranges in time delays that you're interested in. You might want to do it for each other destination seperately. Then, to be more accurate, send out some sort of PSM sequence (this is only slightly more complicated than calling random() ). Correlate this signal against itself for a while (to account for any variations) to get a precise range of delays.
You might be suprised to find the delays are extremely constant, if not dependant on the destination. A simple delay, invert, attenuate and mix will suffice at this point. Otherwise, you need to use an adaptive algorithm, but since you know the delay (roughly), you can just buffer up a bunch of data but only process a small window of it.
Black holes are where the Matrix raised SIGFPE
You could use a VST fx host, and use VST software gate plugins...
Buckets,
pompomtom
"There's an exception to every rule. Except for some rules"
Good, but Few comments:
1. The speaker signal should be close to unit pulse.
2. There is no need to subtract the signals.
3. There is no need to do any tranform.
4. There is no single echo to be handled, but infinite numbers of echoes from every point of the room superimposed together. You need sort of "Echo Profile.
5. Subtracting the signals in frequency domain is not better, but worse and more computationaly intensive.
The best is:
1. Record the echo response to the impulse into E[i].
2. Keep history S[i] if N previous samples.
3. EchoCancelledSignal = Input - sum(R[i]*S[i])
One problem:
The echo might change as the source of sound and other objects or people move about the room. You might want to make the R[i] coefficients adaptive to the changes of echo as posted in the eralier algorithm.
Petrus Vectorius
P.S.:
Go to comp.dsp newsgroups, it is actually quite trivial (as algorithm, implementation might require quite some processing power)
In your terminal emulator configuration, deselect "local echo".
Give me my freedom, and I'll take care of my own security, thank you.
- Its seriously compute intensive. You need two buffers: a circular echo buffer containing the last N samples where N is the sample frequency times the longest echo you want to cancel, and another buffer of N samples which describes the echos you are hearing. The echo cancellation to be applied to each sample is the sum of the products of the elements in the two buffers. For radio you want to be sampling at 20KHz (giving a 10KHz Nyquist cut-off), so if you want to cancel a 0.1 second echo then you are having to do 2K multiplications per sample, at 20KHz, giving 40e6 multiplications per second. Conference phones (the kind of thing that sits on a table) can do this more cost effectively by handling only 8k samples/sec, which gives much saner numbers. In parallel to this you have to be doing a much slower (say, 10 or 20 times per second) bit of DSP to tell you what the echos are as people move around and change the acoustic properties of the room. I'm not sure about how thats done, but it probably involves an FFT somewhere.
- The whole thing is the hardest of hard real time. The echo correction for each sample has to be fed back into the audio at exactly the right time to be out of phase with the original echo. If you get the time wrong then you will be in phase for at least one frequency in the echo, and that frequency will be amplified around the loop.
This is simply not a job for stock hardware on stock OS. It needs dedicated DSP. Also I suspect you can save on the DSP by doing the buffers and multiplication in analogue hardware, but I don't know if that is cost effective in practice.Paul.
You are lost in a twisty maze of little standards, all different.