Hardware Headaches Inevitable?

JaneWalker6847 writes "Don Becker, co-founder of the Beowulf project, describes the inevitability of hardware administration headaches and warns users not to expect a silver bullet to solve these problems." From the article: "We're about to see another revolution, which is in network adapters -- that we [will] talk directly to [them] from application level. That's a massive change in how you interface with them. And that brings about a new round of device drivers completely unlike the device drivers we had 10 years ago. So, that part of the world isn't going to stabilize anytime soon."

Comments

revolution indeed

[b1ufox]

FTA

"We're about to see another revolution, which is in network adapters -- that we [will] talk directly to [them] from application level.

I hope ioctls do perhaps the same job as long as there is a module properly written to handle a specific ioctl

Or is it like controlling network firmware directly from Application ? !!!

Sounds like weird...

Imagine a malicious program kicking your Network Adapter's butt :) ...

Re:revolution indeed

[edashofy]

What they're likely talking about is technology like Chimney, which, barring lawsuits, will be coming out in or around the time of Windows Vista. Effectively, instead of the TCP/IP stack coming from the OS and running on the main processor, the network card will have a processor and memory and run the TCP/IP stack there. This increases efficiency and reduces reduces latency because the main CPU doesn't have to get involved as much. In the future we will probably see things like SSL encryption being performed on the card as well.

Re:revolution indeed

[addaon]

Try 10Gb ethernet.

Re:revolution indeed

[Masa]

In the future we will probably see things like SSL encryption being performed on the card as well.

If I remember correctly, there was once upon a time encryption accelerator cards for servers to release some stress from CPUs. They were discarded as obsolete because the CPU power kept growing and in the end, the pure CPU calculation power was much more greater than with these accelerator cards. It would have required constant hardware investments to keep the cards up to speed with ever-growing CPU speeds.

It seems that these old cards are on sale on the eBay and [when I look those other search results] also some are sold as new, so I guess that this is still viable technology in some places: http://www.google.com/search?q=ssl+encryption+acce lerator+card

Anyway, I just have this feeling that it's not a good idea to integrate any kind of encryption technology directly to the card.

Hmm... Do I smell WinModem?

Re:revolution indeed

[modeless]

Why would you do that when CPU cores are just about to become plentiful? I think multi-core computing should be the death of dedicated acceleration cards. When you have all these cores just lying around you don't need a dedicated sound DSP, network accelerator, or even (eventually) a video accelerator. On, say, a 64-core system, you could do real-time raytracing without any specialized hardware at all, and still dedicate a core each to network and sound if you wanted.

This kind of general-purpose architecture is far more flexible and in the end faster than dedicated accelerators.

Re:revolution indeed

[hpavc]

I think the downfall of the cards is the SSL standards, the card is tricked out for a small subset of calculations. So it can do IPSec + VLANS (in the case of that Intel server cand) and the cypto cards can do SSL for webservers and VPNs (but the implementation would be whatever the card did, not what apache could do -- though I assume apache could fall back).

Re:revolution indeed

[the unbeliever]

Individuals? Not many, at least not on a regular basis.

In a data center environment? Quite often.

Re:revolution indeed

[Detritus]

The traditional problem with doing this is that when you put TCP/IP on the NIC, you still need a protocol for the operating system to communicate with the NIC, and the CPU on the NIC is much slower than the main CPU. I used to have a box full of smart NICs that people had discarded because they were more trouble than they were worth, even though they had paid premium prices for the onboard protocol processing features.

Re:revolution indeed

[Short Circuit]

Transferring large files is not the primary use of high-bandwidth connections. Instead, streaming data and clusters get big bang out of it. 10Gb isn't targeted at consumers or gamers, it's targeted at businesses with lots of fresh data to push around.

Here's a short list of companies for whom 10Gb Ethernet likely comes in handy:

• Google
• Pixar
• IBM
• Amazon.com

And then there are systems on the lower end of the Top 500 supercomputers list.

Re:revolution indeed

[hackstraw]

Hmm... Do I smell WinModem?

I'm not sure what that smell is, but its familiar.

Yes, TCP/IP offloaders, crypto offloaders, physics offloaders, FFT offloaders, have all existed. The only accepted offloader that has succeeded is the GPU, and that is because it was subsidized by the high end graphics people and people with game addictions. The cost/benefit of the other offloaders has not proven itself. Especially when you consider the rate of increase of the CPU speeds and the bottleneck of getting the data to the offloaded chip and back again. The Linux kernel mailinglist has been pretty much anti TCP/IP offloading because the time spent optimizing the drivers and the performance increase was often surpassed in a few months with a faster general purpose CPU and a generic driver that worked with all cards. This is also confounded when you have OS level TCP/IP stuff like ipchains or iptables that need their code in the OS and not on the card. FFT and physics offloaders have not taken off because of the cost/benefit loss when one takes into account the speed loss of getting the data onto the card and then the cost of the cache RAM to put on the card (if its ever enough), and then the time to get the data off of the card.

Now, what will make these things work?

A bus that is near or at the speed of memory bandwidth or a problem where the data does not need to go back to the host system. A GPU falls into the second half there. The display needs to know what its told, the computer does not need to know the details. TCP/IP offloaders have failed to catch on because of the price/performance benefit and their lack of ubiquity and commonality. Crypto offloaders are cool. Especially for the geek factor of having the keys stored on the card and zeroed out when tampered with. But again, the cost of writing specific drivers vs known CPU drivers for doing crypto and the rate of increase of cheap commodity CPUs over time often exceeds the cost/benefit of the crypto card.

Now, give me a fast bus and the ability to use my generic system RAM with one of these offloading cards, and things could change, but until then I expect this battle between the CPU and specific PUs to continue with no real winner.

Re:revolution indeed

[SchwarzeReiter]

Because dedicated cores can solve the same problem more cost efficiently than general cores. Imagine how many general cores would you need to get the capabilities of a GeForce card. And also I'm not an expert in graphics, but I think you would need a truck load of 64-bit cores to be able to do real-time raytracing.

Re:revolution indeed

[Eivind]

Unlikely. "tcp offload engines" and similar crap come and die regularily.

The problem is that general-purpose cpus grow in power so quickly that the offload-engines get ever-larger problems beating them. And they get the aditional problem that they don't get packet-filtering or anything else that is not custom-written for that particular card (if it's even possible to convince the card to do it!)

It's also nothing new -- these cards have existed for literally decades, and haven't managed to make any kind of inroads, not even in specialized servers.

Have a look at this year-old Lwn-article for an example listing some disadvantages.

Re:revolution indeed

[raynet]

Except that people are doing real-time raytracing already with todays computers. First real-time raytracing was running on 80486 or Pentium using somewhat limited 160 x 120 resolution, but best raytracers today run on dual CPU AMDs at VGA/SVGA resolution with 100000+ polygons at about 10-20 fps. And because raytracing can be parallised easily and gains almost linear speed up you can use new dual core CPUs to get even higher fps or resolutions. I also recall reading a paper claming that software raytracer on Opteron was faster than best NVidia GPU when using 5+ million polygons.

Re:revolution indeed

[Tower]

It is all a question of tradeoffs, and for most situations, the tradeoff of a little extra CPU against extra money on an adapter (particularly if it can increase latency) is a no-brainer. This is the same way with crypto offload.

That being said, if you are trying to scale (think a dozen gigabit cards running at high utilization) or a significant number of high-throughput IPSec/VPN clients, then the offload hardware can really show up as a big gain. Even the OTS gigabit ethernet cards these days support offload of some type - usually TCP checksum offload and some support large send offload which save quite a bit on the CPU, since these checksums are cheaper to do in HW than in software.

If you are running a layer on top with its own checksum or CRC (think iSCSI), this can use a very significant amount of CPU, or it can be handled in offload hardware and really save. Again, a multi-proc Xeon can certainly handle a full gig wire with iSCSI with all of the CRCs enabled, but it can't really handle 4 of them without some serious help.

Another issue is that the bulk of older TCP offload engines were firmware based - good path TCP/IP can be handled in ASIC logic instead. Expensive to deisgn and test, but much faster and more capable than a small processor trying to handle those kind of speeds. Great for scaling to many adapters under a single OS image - generally too expensive for a simple home / SOHO type of setup where the demands aren't that great.

As always, everything depends upon what kind of traffic you want to send, how much, and over how many interfaces.

Re:revolution indeed

[iamcadaver]

This logic does not follow through when you think of GPU's.

That's what this sounds like, giving the network card the kind of specialized bus and direct communication channels like the Graphics subsystem.

Which gives a whole new meaning to...

[patrixmyth]

Imagine a BeoWulf Cluster of these #$*&#@ drivers!

Ok, but seriously, maybe someone can answer me this. Why do we still need to construct massively parallel computing architectures at the platform level? Not saying we should toss the whole concept, but for the foreseeable future won't it make a lot more sense to stick with the Amazon model of chunking up into virtual machines? I know the FA says that this view is a mistake, but he doesn't explain why. Can anyone else?

Why?

[Anonymous Coward]

Isn't having a stack of software between the network card and your application a _good_ thing? Personally I like to leave the network configuration up to the OS and focus on developing the functionality of my apps.

Besides, what about hardware abstraction? If we're talking directly to the network adapter, isn't this taking a step back into the past (remember when you had to hand-code ASM to talk to various video/sound cards back in the early days of PC demos and games?)

Re:Why?

[jonwil]

What happens now (on windows) is that applications talk to winsock. Winsock sends the data to kernel mode code including tcpip.sys. From there, it ends up in ndis.sys and then the driver for your network card before being sent to the card.

What this new thing means is that applications send the data to winsock which hands it directly to a new kind of network card/driver which takes the data and header info and creates the TCP/UDP and IP packets on the card itself in card firmware. From there, the card wraps it up in the lower level protocols and then puts it out over the wire (or air if its wireless)

Re:Direct from the application?

[Forge]

TFA gave no indication that Don thoght it wise or even knows why it is being done.

He just stated what is. "My boss now insists that we all wear a tie to work. In a call center."

Same tone.

Standarization

[bn557]

I don't believe that most places that would benefit from an 'advancement' like this would tolerate a moving standard. If a company revises a card because the standard changes, it sure as hell had better support whatever the previous standard is because some company running 10,000 of these cards in some massive distributed application won't accept anything but plug and go.

The idea that your original hardware vendor can provide you with a drop in replacement for a failed card is nice, but any decent manager is going to ask you what the plan is for when that vendor goes under. You need to know that you can just buy some card off the shelf and put it in and have it work. At least with our current driver/hardware structure, you know that a 3com nic is going to work. Or if 3com goes under, you can drop an intel nic in. You may have to install the driver for the card, but it's not going to (unless you start messing with dirty cheap hardware) have compatability issues.

I guess my point here is, there's no way a bunch of companies would target big business with a product like this without there being SOME standard interface. Who wants a multimillion dollar migration to 100% proprietary hardware?

nil nove sub sole

[spectrokid]

Come on, we have seen the same before with modems. First, modems did everything by themselves, then we started seeing Winmodems which pushed a lot of the work back to the main processor. Intelligent network cards will be more expensive than letting the CPU do all the work. Of course, if Moore's law becomes harder to uphold in the future, then decentralising the computing work might be the only way to make computers run faster. Until then, it will be a niche product for computers where TCP is a significant part of the CPU load (webservers). Just like 3D engines on video cards are a niche product for gaming PC's where rendering is a significant part of the CPU load. Oh and a little question: how easy will the upgrade to IPv6 be? Especially if it is not just the OS being involved? As far as I understand, Vista will have a nice soup where IPv4 and IPv6 are mixed in the same driver?

Be the first to pre-order your essential apps!!

[MrFlannel]

Taking pre-orders now for exciting new products to be released soon!

Only $50 each!
Choose from the list below, and many more!

• MS Word Firewall
• MS Excel Firewall
• MSIE Firewall
• Windows Explorer Firewall
• and many more!

If you purchase 10 or more, we'll give you a COMPLIMENTARY Firewall Commander (a $100 value), to simplify the process of setting up rules for multiple applications!
</advertisement>

Seriously, why would I WANT to have to update all of my programs because a hole was found in the networking code (that they all share - because it's a full featured drop in library - BSD licenced and everything)?

Did anyone think this through? Or, is this a follow up to the "OS of the future" article?

What really worries me

[Moraelin]

What really worries me are the metaphors mixed in the same sentence in the summary. So we should expect headaches, but not a silver bullet to solve them? I'm just having a mild headache at the moment, and just the thought of curing it with a bullet seems... somehow not that tempting ;)

Not WinModem-like at all

[Kadin2048]

Hmm... Do I smell WinModem?

Except that WinModems are the exact OPPOSITE of the philosophy that's being espoused here with crypto offload engines, intelligent network cards, etc.

The WinModem was an attempt to take traditional modem functions and move them onto the CPU, in software. Rather than actually having a box full of circuitry that did the hardware handshaking, data compression, and all that good stuff, you just replace it with a simple device that barely connects the analog telephone line to the computer, and have the computer do all the heavy lifting.

I think the justification behind this approach is "software is cheap, hardware is expensive." Therefore, you put the 'brains' in software, and your dumb-hardware/smart-software combo is cheaper than the traditional combination of dumb-software/smart-hardware.

It's a pretty radical departure to essentially go in the opposite direction, from WinModems to these kind of "intelligent network cards," which seem more like a traditional serial modem in philosophy; they do all the work themselves and basically present the computer with a standardized data stream.

The only way that I could see this whole business being "WinModem-like" is in it being tremendously difficult to program for on non-Microsoft OSes. But that's not a consequence of the design per se, but of how I suspect MS will choose to implement it.

This guy is worth listening to

[vtcodger]

The article doesn't mention it, but Donald Becker is, I'm quite sure, the guy who wrote most of the Linux NIC drivers. I think that anything he says about hardware interfaces and their future is probably worth reading.

It looks to me like he's telling us that drivers are not likely to go away as an issue any time soon. Too bad, but if Becker says so, he's very likely right.

memory bandwidth

[Gr8Apes]

This comment makes me think again about AMD's acquisition of ATI. Would AMD put an ATI graphics core in the CPU package? (HTT allows for all the bandwidth the GPU could handle - no separate cache needed). Need a faster GPU? By the time you do - there'll be a faster CPU with a new GPU included, and this packaging might be less expensive than the current high end cards.

This combination would also work fine for 90% of the world's computer users, and possibly be much cheaper. Think Sempron with RAM and a miniscule motherboard with ports. The $100 laptop might drop in price.