Nonlinear Neural Nets Smooth Wi-Fi Packets

mindless4210 writes "Smart Packets, Inc has developed the Smart WiFi Algorithm, a packet sizing technology which can predict the near future of network conditions based on the recent past. The development was originally started to enable smooth real-time data delivery for applications such as streaming video, but when tested on 802.11b networks it was shown to increase data throughput by 100%. The technology can be applied at the application level, the operating system level, or at the firmware level."

Comments

Improves network performance

[miguel_at_menino.com]

But does this improved network performance allow me to predict if I will get a first post based on my past inability to do so?

Re:Improves network performance

[Too Much Noise]

it does, but only if you apply the algorithm to /. and notice everyone else is busy posting to the last Longhorn article ^_^

Checks, Governing circuits, etc.

[Anonymous Coward]

Is anyone else slightly alarmed by this news? "Neural-net" technology that shows some degree of intelligence (if you consider making fuzzy predictions intelligence). I think that checks or governing circuits should be put in place for this kind of technology so that it doesn't get out of hand by, oh I don't know, burning out transmitter circuits or something. Remember the documentary "The Terminator"? Yeah. I do. I don't want something like that to happen.

Re:Checks, Governing circuits, etc.

[ookabooka]

Wow, because this neural net has like 10 nodes, and the average human brain a few billion. I made a chess engine, to me thats a lot smarter than traffic handling, and it doesnt use neural nets. . . .i would say we are a little ways off from making true AI. now a distributed neural net. . . thats interesting. . .

Re:Checks, Governing circuits, etc.

[grouchomarxist]

The Governing circuits have already taken over California. Soon they will take over the rest of the country.

Re:Checks, Governing circuits, etc.

[Thing 1]

Dude, it's too late -- your Terminator is already a "governing circuit."

Re:Im no programmer, but...

[wpmegee]

Firmware and/or software:

The neural network is one of three modules in the company's WiFi Speedzone system which has reached the beta test stage. A second module monitors and analyzes network traffic and the third handles the packet-sizing operation. WiFi Speedzone can be implemented at the application level, the operating system level or as firmware embedded in an 802.11b device, the company said.

And next time, please RTFA, m'kay?

this hurts my geek cred

[fjordboy]

When I see the headline: "Nonlinear Neural Nets Smooth Wi-Fi Packets" and I only understand the words nets, smooth and packets...and none of them in relation to each other... - I have to be a little concerned that my geekiness is dwindling...

Re:this hurts my geek cred

[pla]

When I see the headline: "Nonlinear Neural Nets Smooth Wi-Fi Packets" and I only understand the words nets, smooth and packets...and none of them in relation to each other

Simple 'nuff, really...

Neural net - An arrangement of "dumb" processing nodes in a style mimicing that which the greybacks of AI (such as Minsky and Turing et al) once believed real biological neurons used. Basically, each node has a set of inputs and outputs. It sums all its inputs (each with a custom weight, the part of the algorithm you actually train), performs some very simple operation (such as hyperbolic tangent) called the "transfer function" on that sum, then sets all of its outputs to that value (which other neurons in turn use as their inputs).

Nonlinear - This refers to the shape of the transfer function. A linear neural net can, at best, perform linear regression. You don't need a neural net to do that well (in fact, you can do it a LOT faster with just a single matrix inversion). So calling it "nonlinear" practically counts as redundant in any modern context.

Smooth - A common signal processing task involves taking a noisy signal, and cleaning it up.

Wi-Fi - An example of a fairly noisy signal that would benefit greatly from better prediction of the signal dynamics, and from better ability to clean the signal (those actually go together, believe it or not - In order to "clean" the signal without degrading it, you need to know roughly what it "should" look like).

Packets - The unit in which "crisps" come. Without these, you can't use a Pringles can to boost the gain on your antenna to near-illegal values. ;-)

There, all make sense now?

Re:this hurts my geek cred

[Anonymous Coward]

I believe packet smoothing refers to taming rapid swings in packet output rates, so the network can adapt in a timely manner and thus drop fewer packets. In the OSI protocol layer model, I guess it's usually best accomplished in the protocol timers of the network and link layers. It would have nothing to do with receiver signal filtering, which is a physical layer process, performed before converting the received signal into packet data bits.

Re:this hurts my geek cred

[pla]

So, I take it this means that the "greybacks of AI" no longer believe this to be true? What is the new thinking?

I put that in the past tense for two reasons...

First, at least the followers of Minsky have apparently deemed connectionist learning models as passe. In fact, as far as I can tell, the very field of artificial intelligence has shifted away from the "intelligence" part, preferring to focus on (the far more marketable) automated problem solving and classification, rather than trying to mimic aspects of actual consciousness.

And second, neurophysiology (rather than AI researcher) has all but obliterated the hope that any basic variation on the standard multilayer feedforward neural net really does all that great of a job at modeling the brain. It seems that real neurons do some pretty impressive processing, each having a local store, exceedingly fine-grained delay lines, self-feedback (at the signal, rather than just the obvious neurotransmitter level), and some degree of actual flow control. And that just mentions what we know, they may have quite a good many more secrets waiting for someone to notice... For example, recently, a few bright folks noticed that glia, the non-neuronal cells making up literally half of our brains, might do more than just sit there and take up space.


Please confine your answer to words of less than ten syllables :)

I apologize for the length of words in this domain, but I didn't make them up, we all just inherited them from people who liked Latin and nominalization waaaaaaaay too much. <G>

Re:this hurts my geek cred

[12357bd]

Just one point.

'One way' classical layered models can be said to be 'passe', but recurent o 'looped' conectionist models are far from being understood, in fact, are a great source of advances.

What's in a sig?

According to the website

[Anonymous Coward]

This technology can increase throughput 200-800% in networks of 3 Asian people and 1 doctor.

Hahaha

[Anonymous Coward]

This sounds like a scam. The CEO is a furniture salesman, the CTO was a consultant to DEC with an EE from the University of South Maine.

Re:Hahaha

[JessLeah]

It seems that CEOs always end up in fields they have no experience in. Remember Sculley (sp.) of Apple shame? He was, what, a Pepsi or Coke exec?

Re:Hahaha

[Jeffrey Baker]

There's lots of scam giveaways in this article. If the protocol "can be implemented" at the application layer, the network layer, or the MAC firmware, that means it *hasn't* been implemented in any of those places at all.

Damn...

[AvoidTheNoid]

Words I undestood in the headline:

1.Smooth

Fuck...

Re:Damn...

[ixplodestuff8]

Nonlinear Neural Nets Smooth Wi-Fi Packets

You don't know what nets are??

How do you catch stuff?

Re:Damn...

[antic]

True (I knew that but was being silly).

Anyway, I'm only replying because you're being "insigful" and it made me laugh.

"Insighful" would be moderation for when a comment makes you sigh. Hmm..

Re:Im no programmer, but...

[burns210]

the technology can be executed at any of those level to be effective, not all 3 at once. So that means linux could get support for it at the kernel level... someone could write an application for windows, and palms could use an updated firmware and all 3 would effectively take advantage of the algo.

Re:could be handy..

[wpmegee]

Not necessarily. This improves throughput, but as a general rule wireless always adds 20ms to your ping. so 50% of that would still be a 10ms penalty.

I'm not a network engineer, but latency is more important than bandwidth for ping times and such.

For an example pay a half-life game, open the console and type net_graph 3. That'll show you your fps, ping, and in/out bandwith used.

Re:Im no programmer, but...

[Pyro226]

There would be no reason to implement the algorithms in all three. The point is that the implementation isn't so low level or complicated that it requires wireless cards to be designed for it. It can instead be implemented in firmware, hardware, or software.

Now, a speed increase sounds good to me, but as most of my wifi usage is for internet use, and I don't think i've ever been on an internet connection faster than my wifi connection, I'd like to know if it helps with range, and I'm too lazy to RTFA.

Why Neural Networks?

[women]

I'm curious as to why they are using Neural Networks for this application? In the last 10 years or so, most machine learning applications have moved away from Neural Networks to more mathematically based models such as Support Vector Machines, a generative model (e.g. Naive Bayes), or some kind of Ensemble Method (e.g. Boosting). I suspect they used NN because the Matlab toolkit made it easy or someone in research hasn't kept up. I'd look for a paper to come out soon that improves the accuracy by using SVM.

Re:Why Neural Networks?

[Anonymous Coward]

I'm curious as to why they are using Neural Networks for this application?

Well, it's quite obviously because a Support Vector Machine is inherently linear, and to make it nonlinear, you must insert a nonlinear kernel which you need to select by hand.

If you'd read the article, you'd see that they are using a recurrent-feedback neural network; good luck finding a recurrent-feedback nonlinear kernel for a SVM....! You can't just plug in a radial bias function and expect it to work. In this application, they are looking for fast, elastic response to rapidly changing conditions as well as a low tolerance for predictive errors--something an RFNN is ideal for, and that a SVM is absolutely terrible at.

Re:Why Neural Networks?

[zopu]

I hear all too often from people in the field of machine learning who get their favourite solution (SVMs and NNs are the most common) and then they go hunting for a problem.

It might not be exactly the best technique, but if at the time it was the easiest to understand and use, and gave really good results, then the right decision was made.

Is that the difference between theory and practice right there?

Re:Why Neural Networks?

[jarran]

I expect at least part of the answer is that neural networks are trivial to understand and implement compared to support vector machines.

You might be able to build SVM implementations relatively easily on a real computer using off the shelf libraries etc., I doubt many of these would run on a WiFi card.

Neural nets have also been around for quite a while, so they have gained acceptance. Although SVMs have been known to the machine learning community for quite a while now, they have only just started being noticed by the wider world quite recently.

Re:Why Neural Networks?

[semafour]

Well, it's quite obviously because a Support Vector Machine is inherently linear, and to make it nonlinear, you must insert a nonlinear kernel which you need to select by hand.

Not true.

"This invention provides a selection technique, which makes use of a fast Newton method, to produce a reduced set of input features for linear SVM classifiers or a reduced set of kernel functions for non-linear SVM classifiers."

Re:Why Neural Networks?

[Anonymous Coward]

Actually the whole point of SVMs is that they can be used to model non-linear decision boundary. Contrary to the above post selecting the non-linear kernel isn't a big deal because the three common ones (polynomial, radial basis function and sigmoid ) generally produce similar classification results in most applications. Also SVMs are pretty damn fast to train and update since only the support vectors need to be remembered and changed. Just check the literature...

I figure the real reasons they use NNs are much simpler. Firstly, its really easy to implement NNs that predict numeric values instead of classes and even more importantly they work. Research usually involves trying everything under the sun and reporting/patenting/exploiting whatever worked best.

Yahoo Serious Festival

[Hatta]

I know those words, but that sign doesn't make any sense.

Re:Im no programmer, but...

[I_Love_Pocky!]

Umm... it isn't so simple. You are missing the basic idea of a layered architecture. This is actually really cool that it can be implemented at any layer. Sometimes there are things that can't be done at the application layer because of the constraints created by the layers below it. For instance, it is pretty worthless to do routing at the application layer if you are using IP, because it is already taken care of at the network layer.

So to say that it is all just "software" misses the fact that there is a significant difference between how these peices of software work. It is really cool that this can be done at the application layer, because it will allow applications to be developed to take advantage of it with out even changing the drivers for your wi-fi card.

Re:Im no programmer, but...

[I_Love_Pocky!]

This is a big deal to me, because I live in an appartment complex that offers Internet access over Wi-Fi. Because there are so many people using it the connection is pretty flaky (due to collisions). I get high download rates, but poor response times. If this provides better collision avoidance I will get a significantly better connection (lower ping times and such).

Skeptic

[giampy]

Very often the term "neural network" is used
just as a selling point because it sounds
like something extremely advanced and "related
to artificial intelligence".

usually the neural network is just a
very simple, possibly linear, adaptive filter
which means that really contains no more
than a few matrix multiplications ...

yes it has some success in approximating
things locally, but terms like "learning"
are really misused

After RTFA (the second) it actually
seems that they did try two or three
things before, but really i wouldn't
"welcome our new intelligent packet sizers overlords"
just yet.

Re:Skeptic

[batkiwi]

Are you posting
that from a mobile
phone or do
you just like to
hit enter after
every couple of
words as some
sort of nervous ti
ck?

Re:Skeptic

[hawkstone]

usually the neural network is just a very simple, possibly linear, adaptive filter which means that really contains no more than a few matrix multiplications ...

The simplicity of the calculation does not mean it is not a learning algorithm. Real neural networks are quite simple, as each "neuron" is simply a weighted average of the inputs passed through a sigmoid or step function. However, en masse they perform better than most other algorithms at handwriting recognition. They take a training set and operate on it repeatedly, updating their parameters, until some sort of convergence is reached. Their performance on a test set is a measure of how well they have learned. This is a learning algorithm.

Even linear regression is a learning algorithm. You give it a bunch of training data as input (i.e. x,y pairs), iterate on that data until it converges, and is then used to predict new data. There happens to be an analytic solution to the iteration, but this does not make it any less of a learning algorithm.

I think maybe your definition of "learning" is unnecessarily strict. The simplicity of the computation is not what defines this category of algorithms.

Re: Skeptic

[Black Parrot]

> usually the neural network is just a very simple, possibly linear, adaptive filter which means that really contains no more than a few matrix multiplications ...

No one in their right mind would use a linear ANN, since ANNs get their computational power from the nonlinearities introduced by their squashing functions. Without the nonlinearities, you'd just be doing linear algebra, e.g. multiplying vectors by matrices to get new vectors.

As for the computational power of ANNs,

• A simple feed-forward network with a single hidden layer can approximate any continuous function on the range [0,1] with arbitrary accuracy. (Or is it s/continuous/differentiable/ ? - can't remember.)
  
• Certain architectures of recurrent ANNs are equivalent to Turing machines, if the weights are specified with rational numbers.
  
• An ANN with real-valued weights (real, not fp) would be a trans-Turing device.

Goggle a paper by Cybenko for the first result, Siegelmann and Sontag for the second, and Siegelmann (sans Sontag?) for the third third.

> yes it has some success in approximating things locally, but terms like "learning" are really misused

"Neural network" and "learning" are orthogonal concepts. A neural network is a model for computation, and learning is an algorithm.

In practice we almost always use learning to train neural networks, since programming them for non-trivial tasks would be far to difficult.

Prisoner's Dilemma applied to networks flows

[doc_brown]

To me, this sounds like (in the simplest form) that this is a variant on the Tit for Tat strategy that is usually applied to the Prisoner's Dilemma.

Re:Prisoner's Dilemma applied to networks flows

[ThatGuyInTheHole]

Back in the days when computers were large hulking monsters best kept under a desk, some college had a contest matching two computer programs playing the prisoner's dilemma game with roughly equivalent outcomes. A lot of famous computer scientists submitted programs, some many pages in length, but it turned out a really simple program won: Tit for Tat. The program begins with silence, but if it is betrayed, in the next round it will betray you, then switch back to silence. That's pretty much it. TGitH

Chartsengrafs

[NanoWit]

Heres a graph that I ripped out of some lecture notes. It shows how much of a problem congestion is on 802.11b networks.

http://web.ics.purdue.edu/~dphillip/802.11b.gif

For a little explaination, where it says "Node 50" or "Node 100" that means that there are 50 or 100 computers on the wireless network. And the throughput numbers are for the whole network, not per host. So when 100 nodes are getting 3.5 Mbps that's .035 Mbps per host.

Thanks to professor Park

Why wireless only

[Old Wolf]

Why isn't there something like this for normal internet? Even the "old days" of Zmodem's big packets if it was going well, and small packets if it wasn't, is better than the fixed MTU/MRU we're stuck with now.

Re:Why wireless only

[Anonymous Coward]

Why isn't there something like this for normal internet? Even the "old days" of Zmodem's big packets if it was going well, and small packets if it wasn't, is better than the fixed MTU/MRU we're stuck with now.

The normal internet has far less collisions & errors than wireless ethernet. And ethernet switches are now so cheap that it isn't worth your money to buy ethernet hubs.

And the advantage here is that it is (allegedly) a successful predictive model of whether to use big or small packets, and not reactive.

If you react to errors, you can resend, but you've already wasted bandwidth. If you can avoid the error in the first place, it's much better! :)

Re:Why wireless only

[mesterha]

And the advantage here is that it is (allegedly) a successful predictive model of whether to use big or small packets, and not reactive.

If you react to errors, you can resend, but you've already wasted bandwidth. If you can avoid the error in the first place, it's much better! :)

It predicts based on past performance therefore it is reacting. The savings on switching packet size is based on resending small packets instead of resending large packets. Losing a single small packet is not nearly as bad as losing a large packet. Of course, as the packets get smaller there is more overhead... Hence you need to optimize the size based on the current noise conditions.

Quick summary for those too lazy to RTFA

[Anonymous Coward]

It's a new way of determining the optimum packet size on the fly so that collisions, errors & retransmissions are minimized, greatly boosting overall throughput.

QED

Re:What?

[Anonymous Coward]

In average geek terms please!

Not gonna happen. The poster was just using random random terms that have nothing to do with this article, trying to sound smart, and is probably laughing as the post gets moderated up.

Everything the poster mentione, such as Naive Bayes and Support Vector Machines are used for static tasks, like classification, not for realtime feedback situations. They learn once and predict forever. They don't learn iteratively and keep changing. Follow the Google links I just gave and peruse the first few sites that come up if you're not sure. They are used for things like text classification, handwriting recognition, voice recognition, etc., i.e., "train once use often."

Re:What?

[zopu]

I'm sorry, but that's not quite true.

There are online methods using both the techniques you mention. The theory is usually a little more involved, so you're not likely to get a good tutorial from page 1 of google results.

Try MIT's open courseware (Machine Learning course) for some better explanations of this stuff, if you can handle the maths, ughhh.

EE Times Article

[FreeHeel]

Wow...not 30 minutes ago I read this article in this week's EE Times on the same topic.

This sounds like a great improvement to 802.11x technology...now let's open-source it so we can all benefit!

Buzzword-powered network

[identity0]

Gee, let's see how many buzzwords we can cram into a technology:

"Introducing iFluff/XP: An XML-based Object-oriented neural networking system that will synergize the modular components of your SO/HO WAN protocols, while minimizing TCO and giving five 9's reliability by branch-predicting streaming traffic through your SAN, NAS, or ASS.

iFluff/XP allows you to commoditize and monetize the super-size networkcide as rogue packets from black hats and white hats and clue bats compete for cyber-mindshare of your Red Hat hosts.

Secure your Homeland LAN and manage your digital rights with dignitude and affordability with the help of iFluff/XP's bytecode-based embedded operating system protocols interfacing through broadband Wi-Fi connectivity and virtual presense frameworks.

A user-friendly GUI is provided through an XSLT module interfacing to leading industry applications such as Mozilla, .NET, Java 2 USS Enterprise Edition, and GNU/Emacs - soon to include POP, IMAP, P2P and B2B functionality for enhanced productivity.

When you're thinking of buzzword-compliiant, ISO9001 conformant, remotely-managed turnkey security solutions, remember iFluff.... TO THE XXXTREME!"

Oh god, my brain hurts now.

Re:Buzzword-powered network

[OldManAndTheC]

streaming traffic through your SAN, NAS or ASS.

(sigh) I've had that last one, and it ain't fun. Imodium usually clears it up quick though...

Re:could be handy..

[The Unabageler]

I couldn't agree more. Last summer I was working with a satellite link from a third world country. Now the pipe on those guys is pretty big, we were working with 3MBps. 24Mbps. However, latency on the tcp packets from the south pacific to space and back to sunny CA is such a large problem it slowed an asynchronous transfer to about 20KBps. I had to write a special segmented upload package to be able to send video feeds sychronously and maximize the bandwidth utilization.

In other words, bandwidth will do you zero good with a traditional tcp/ip stack if your latency is too high.

LOL

[pyth]

Saying "Non-linear neural network" is like saying "Non-purple hamster". I mean, how often do you see a linear NN? Like, never.

Re:LOL

[I'mJVC]

hehe, I just pictured a purple hamster in my mind...

Re:which end?

[Kris_J]

Interesting question. I would assume that any given existing 802.11b adapter can receive packets of any size, given that there's a protocol to packets that lets the receiver know how big they should be or when they've finished. Thus you could just deploy a new access point and get a boost from it to the computers. Similarly, you could install a new NIC in a particular PC and boost the transfer rate from it to the access point. For benefits in both directions you'll have to upgrade both ends.

Re:which end?

[wed128]

if it is packet-sizing, i suppose it would only have to be on the transmitting end...so you'd basically double your upstream...and if the AP or router has it implemented for ITS upstream, then you get your full speed boost.

Wireless net congestion...

[Ayanami Rei]

does not suffer from intense negative feedback as does the stock market.

I think part of the ease of predictability may have a little to do with the kind of protocols used to collision detection/TDMA in congested 802.11 nets. If they are suffciently simple a single node could outmanuver the others.

Some questions...
What is the behavior of this algorithm as the number of enabled clients increases and the bandwidth demand of the clients exceeds the channel capacity? Does it degrade gracefully? Does it unfairly compete with non-enabled clients?

Looks sketchy to me

[marksthrak]

Now, I'm an AI guy, not a networking expert, but some of this seems sketchy. Their website says:

Because all data packets that are now being sent across packet-switched networks are in fixed-size data packets, SmartPackets' "variable-sizing" packet technology can positively impact the performance of a very wide range of technologies, applications and protocols.

I'm pretty sure that's not the case. Besides, if the technology you're pushing boils down to 'variable-sizing', seems like someone's thought of that before.

As far as neural networks are concerned, a sufficiently complex neural network can adapt to learn reasonably complicated functions. Backpropagation networks and radial basis function networks can make good filters and make sense of noisy data. A network that doesn't adapt its structure boils down to a few matrix operations, so it's easy to script in Matlab.

With all that in mind, shouldn't they have looked at Kalman filters?

Re:Looks sketchy to me

[Hast]

I'd recommend that you read the second article, from EE-times. It actually has some content which is something their own site is quite completely void of.

And people have been doing this before. The EE-times article mentions that. Apparently no-one has either not made so much progress or just not made so much of a fuss over it before. A quick search for "variable packet length and wireless" turns up quite a lot of results though. I'm fairly confident that you can find previous research in this area if you look around.

I'm not entirely convinced that Kalman filters would do a good job though. Or rather, they may need more additions in order to be efficient in this particular problem space. Thus making it bigger and less desireable to put into eg firmware. It's not unreasonable that they haven't considered it though. But they did seem to try some other systems, some of them seem a bit too optimistic to be reasonable in this problem space though. According to EE-times they did attempts with database lookup and expert systems eg. Seems like both of those could be "trivially" rejected to not be adaptive and accurate enough.

It's not impossible that there exist methods which are a lot more efficient than NN to solve this problem. NN seems to do a good job though, so I guess it warrants some more looking into.

not really

[Trepidity]

The whole point of SVMs is that they can be used to model a linear decision boundary. They were developed to find a maximum-margin hyperplane separating positive and negative training instances, and the kernel methods to allow them to work on non-linear boundaries were a later addition.

Re:could be handy..

[ComputerSlicer23]

Actually, all you have to do is tweak the parameters of the TCP/IP stack. As I recall, Linux as a specific parameter for this. I want to say, it's the transmit window size. They document it as something you only change on a long haul line, which a satallite feed should count as one.

Specifically, you want to allow a lot more packets to be outstanding then a normal TCP connection will allow. This is a bad idea on a low latency connection. It has something to do with windows, and buffering. Also, if you use advanced IP tools to ensure that ACK's get sent before anything else, you'll be much happier.

This thread on the LKML seems to have useful information on it: LKML Thread

Kirby