Slashdot Mirror
TCP Equipped Ethernet Card
Josh Baugher writes
"
A 100 megabit ethernet card
with a TCP/IP stack built in. They claim
to be able to do 9 megabytes/second with only 2% CPU load (compared to
4.5 megabytes/second at 98% receiving CPU load using Windows NT TCP/IP
( read about this on "geeks" mailing list.) "
Seems cool. But How much will they cost?.
Also, whats Linux (or Open/FreeBSD's) system load on an equiv OS TCP/IP stack situation?
It seems it would be an I/O bottleneck in this. Even with an Ultra-DMA HD
I'd also like to know if they're going to open the API, provided this is what it claims to be.
<ponders 400Mb webserver on a p166>
Ulta-DMA HD? Since when do file servers use Ultra-DMA hard drives? The last time I checked most (serious) hardware configurations with 100 Mb (and higher) NICs in them are designed for server applications. Those same (serious) hardware configurations usually employ the use of a SCSI subsystem and more importantly RAID. Thus you have all kinds of storage-IO and, unlike your desktop, your IO bus probably doesn't represent a serious bottle neck (not at 100 Mb anyway).
9 MB/second = ~90Mb/second. That is a MASSIVE transfer rate. Few hard drives can keep up with that in actually continuous read rates. Although of course Linux' stack just MUST be better than NTs.
Bah.
Max win95 is 40kbyte/sec? I've got a win95 box sitting next to me now that I've seen do sustained 600 kbyte/sec over cable modem (10bT).
Let's see some credible URLs please.
My Linksys 5 porter is still going strong after 9 months. Couldn't beat it for the price!
who said it had to be server-only?
:)
What about geek-toy
You only passing on information, its not as if you have to give credit to anyone who has ever done an artical on such a topic.
It seems logical to me that you should quote the best source of information you have as the refernce.
That said, im not a logical person either and plagarism would be fine by me (claim it as your own hehe).
youve obviously never tried to run linux 2.2.6
on a 386SX-16 with 4 megs of ram and a 43MB
HDD.
I don't know how you got those numbers. I routinely get over 1MB/sec with my ftp server over the 10bt lan here. I only have a cheapo 10/100 DEC 21140 based netgear card.
How is 900kbytes slow? 900kbytes is 75% of the maximum rate of 1.25Mbytes/sec.
Typical Slashdot "news". I get more reliable information on the wall of a public bathroom.
Of course they want people to think, "if this board can do 9MB/sec on an Aptiva, it must be good". Never before have I heard of anyone running NT on an Aptiva. Heck, you can do a benchmark of 386 with a GUS versus a 386 with an SB16 and Timidity, and completely miss the point that on a decent system, both sound the same.
In regard to Aptiva model numbers and CPU speeds, it's possible that IBM updated the chip before you bought yours--even though they're both E56's, they probably have different model/type numbers.
Anyway, none of this means diddley due to the crappy "benchmark". TCP on a gigabit ethernet card would be more worthwhile anyway, since that's when the interrupts really flood your CPU.
that simple, or do you think that when you get a load average of 5 it the CPU is doing 500% (of course not!)
The load average is just the average number of processes waiting on the cpu in order to get work done. Each measures separate things, so don't be surprised if they differ.
Yeah, but for transfering a single large file from an unfragmented HD, UltraDMA/33 is going to be comprable to SCSI, unless you have some serious striping.
What do you mean by hardware-only memory management? The '386 already has a complete MMU, and the allocation/configuration algorithms really do belong in software for integration with the rest of the OS.
In fact, the company I worked for, has several
486sx25 machines with linux-2.2.x kernels. Of course, compiling the kernel on those is out of question. We tried 386sx16-s too, but they
were too hard to find.
Sure, your point seems valid (and mostly is), except for one thing you seemed to have looked over: 9 years ago you weren't using 100Base-T, were you? If their claims about the processor usage are true, then umm, this would provide a signifigant improvement. As for the good reason they haven't gone mainstream, /.'s only been around for about 3 years, right? Hehe
-me
well - the 21140 is cheap as in not expensive. it is a good piece of hardware though - I use the Kingston KNE100TX cards (21140 based) in a few machines - works great.
Well, if I remember well, the curve knees out
somewhere between 55 or 65% of the nominal
throughtput (assuming there are multiple nodes
contending -- single node networks are a different
story).
Not true. Hardware is interesting, even if it doesn't (currently) work under linux. The availability of PC NIC's that have on board TCP/IP could be a real win for linux, too.
Face it, all new PC hardware is going to only support M$ stuff.
Who cares if it runs on NT. It's a good idea, even if it is one revisited or overhyped. I don't think /. should be passing up stories because of anti-NT sentiments. If it's cool, it's cool. If it's a good idea, it's a good idea, regardless of the OS.
/. has a heavy linux bent to it (duh). But I also like the fact that /. doesn't ignore the other OSs. First, it's good overall coverage and understanding of what's changing--technology, marketing, design. Second, it should be reminding us that we, the /. community, do not have a monopoly on thought out there. Good changes can come about from "bad" companies.
Yes,
Heck, just look at KDE and gnome. Even they have some efforts to mimic known desktops; I would imagine part of the reason is because they want to make it easier for people to migrate to linux, but also because there are good things to be learned from other OSs.
yep, it's the old wheel of reincarnation...
application-specific hardware gets replaced by
more flexible software as soon as microprocessors get fast enough.
Silicon TCP has the potential to scale further than software based TCP, particularly on systems which are CPU bound, not bandwidth bound. Would you rather spend your CPU cycles doing real work, or building packets?
That said, I don't know how much putting the TCP stack in hardware really saves you vs. a reasonably intelligent NIC that just computes the checksums for you (which already exist)
Bandwidth utilization is not directly attributable to octets os bits of payload data moved. It's actually quite a dry calculation. And I can guarantee that very rarely will two collision domains show the same benchmark results. I've seen enough poorly designed repeaters that blantantly break IEEE 802.3 spec in the way they handle runts, long events, collisions etc. Also, there's lots of NICs that do the same.
Take the same 10 boxes and do a bench mark in one collision domain of one manufacturer's repeater, place them into 9 different collision domains and repeat the same benchmark with a different repeater. I guarantee that they will not be the same.
The scenario you mention above is a rather ideal one. I'm sure it could more or less be reproduced, but be careful not to make false blanket statements in arguments against others of the same ilk.
Try 1.2.13 - it runs nicely on that level of hardware
1 mbs is one milli-bit per second ;)
I don't know where you're pulling these numbers from. Where I work, we regularly get 800 kB/sec and higher over switched 10 Mb ethernet between Windows machines.
"Sigh. That old myth."
As the previous poster mentioned: "be careful not to make false blanket statements in arguments against others of the same ilk."
(Mea culpa: up front) I have not read the specific paper you have referenced (and will do so (I promise ;)). However, I have studied ethernet congestion formally, and there are concrete mathematical reasons why a contention based algorithm cannot produce 100% of nominal throughtput (and there is an obvious common sense argument as well (but it doesn't pin down a specific number).
This really isn't a religious argument, nor is it some kind of 'insult' towards Ethernet. Stating that there is a fundamental performance limit in Ethernet (CSMA/CD) is not equivalent to a criticism of the design.
In fact, Ethernet is a brilliant design and (along with tcp/ip, html, dns and others) it is a brilliant example of two design concepts: .
(1) worse-is-better, and
(2) complex systems are doomed by complex solutions: instead, complex systems _require_ "simple" basic principles --
Cheers
Depends entirely on them, and how well they document their hardware. Considering the product info on their "web site" for prospective customers is only available in Word format (spit), I don't hold much hope.
These guys patented "Silicon TCP." If they now exclusively own the notion of hardware that knows how to do more of its job, I may have to finally go postal.
A new kind of coprocessor, tied to the Internet Protocol, is certainly News for Nerds. Even if proprietary (and it may not be; it'd be easier to tell if only their Web site weren't so useless) - it doesn't have to be *good* news....
My PRO200 server can almost saturate 100Bt (11.25MBytes/sec) using knfsd off of a SOFTWARE eight disk raid-5 array!
Hey man, the highest call so far was 8MB, meaning 80Mbits, so your 1mb doesnt match the pot. Now are you in or are you out?
Then this means that the real future of TCP/IP lies in fast ethernet cards built of somekind of FPGA cards.
Part of the Great Wheel of Being. Someone notices that you can do something faster in hardware, and ships it. A few years later CPUs are faster, and someone discoveres you can save a few bucks on hardware by doing it on the general-purpose CPU via software.
It would be interesting to see how it holds up under multicast. Lots of cards filter mulitcast groups only at the OS/TCP stack level, and even those that do some filtering at the hardware level usually have to pass at least some groups up to the OS. That puts a load on the CPU, even when the box isn't actually subscribed to any multicast groups. I've seem some 166 pentiums with 3Com cards, subscribed to no mcast groups other than ALL-HOSTS, on an ethernet that has heavy mcast traffic, suck up 15-20% of CPU just discarding packets not addressed to it.
> Using a raw packet driver on a 200MMX notebook with a SMC 100Mb/s Full Duplex card and using remote loopback we are unable to get much more then 15Mb/s sustained. (That's 15Mb/s going out and 15Mb/s comming in) .
This was using raw Ethernet packets, no TCP/IP or other protocol.
'remote loopback' hopefully implies that no Ethernet packets were _ever_ sent - just the ip code realizing to loop back to itself. So, what you're describing is either a really broken implementation, or you misspoke. Which is it?
C'mon. Get real. Thousands of things like these get tossed out on the 'Net all the time...but usually people don't fall for them this badly. Let's see. A little tiny company, only two or so years old (try looking for "interprophet" with altavista or something, and whois shows their domain name registered in Dec '97) suddenly (Altavista didn't find bunches of pages yammering on about how great whatever they were making was going to be during development time...expected, if they're a startup looking for funding) comes out with a pretty darn impressive engineering feat (a full TCP stack is a pretty large thing to be putting in hardware). Not a lot of technical info. I've seen other scammers make *much* more plausible web sites than that. If they really had the mony to put into doing that much hardware development, they could afford to at least take pretty pictures to push their products...not trashy shots in a barren office or whatever.
:-(
Yeah, it's *real* hard to doctor screen shots.
Over 100 comments so far, and I haven't seen anyone who has realized that this thing isn't for real.
Are you sure your cards were running in full duplex? With a crossover cable I could only get half. And yes, I did it right for full duplex and even made one the "wrong" way and it still did only single. I think the cards need to talk to a switch to go full duplex. Full duplex is pretty important for tcp/ip on local networks because of the ACK's sent back.
.25 seconds but puts FreeBSD transfer speed up to 2 seconds! Maybe IP in silicon isn't such a bad idea after all. I will try my setup with your tests though.
With NT, Microsoft has a KB article at http://support.micro soft.com/support/kb/articles/q169/7/89.asp which supposedly helps tcp/ip speed. Not sure if it's going to make a difference...
Also, a little benchmark of my own (which is not anywhere as good as yours) is sending a 4.5 meg file between a Linux 2.2.7 and a FreeBSD 4.0 system (standard ftp daemons and clients as installed) via ftp. FTPing from the linux system to FreeBSD transfers the file in approx 1.3 seconds. Going the other way (FreeBSD doing the sending) takes 0.4 seconds... I'm on a 1/2 duplex 100 mbps network. There is a 3c905 PCI in the Linux box (PPro 180) and a DEC 21040 in FreeBSD (p5-200).
The most intersting observation: running rc5des seems to slow down the transfer rate! Although running at nice -20 priority, it seems to affect receive speed by approx
P.S.
Where do you get your 21040 cards? I can only find the 3com's which cost way too much or these VIA Rhine cards which leave much to be desired. Got a couple 21040's a while back which work great on all OS's.
How about... A sound card with an MP3 player easily loadable into it's onboard DSP to offload that task from the processor? Or better yet, a means to use the DSP to encode MP3s without signifigant processor load?
It's because of the busmastering! The ISA bus can only adress 16 MB of memory, if the data buffer is for example at 18 MB the driver needs to do some tricky stuff each time data is sent to and from the NIC, that's also the reason why no one should ever buy a ISA busmastering SCSI card for a system with more than 16 MB of RAM.
That 10Mb/s has to carry all the TCP/IP packet
headers etc, so you can't expect to get the
entier 10Mb/s for data transmissions (like the
way you can't get 2Mb of useable data on an
HD floppy)
Does anyone have some figures as to how much
bandwidth is eaten by TCP/IP -- I would suspect
that it can partailly account for the 7.2Mb/s
observed speed.
Linksys Sucks ;-) Hell, I've got the generic DEC Tulip chipset in my alphastation and a 3c509 card in my ppro and I can easily get between 800-900Kb/sec consistently. My laptop's Intel EtherExpress PRO/100 card (16 bit) can easily sustain 800Kb/sec. On the other hand, transfers to my other machine with a generic ne2000 card in it are around 150-200Kb/sec.
You're in college?
Someone already mentioned how the number of stations effects the % but another overlooked one is physical net size. If you have active nodes at oppisite ends of the max size net they produce many collisions which drives down the % quite effectively. Someone just needs to solve that pesky speed of light problem now.
Maybe for old decrepid SCSI II, but compared to U2W, UltraDMA is a complete joke.
Indeed, it might well be possible that with 100 Mpbs/1 Gbps cards and very small packet sizes the processor usage may hit the roof. However, TCP/IP is not heavy to process, so automatically you should hit a bottleneck from your server process before having to much TCP/IP to process in the kernel. Looks what happens on Slashdot :-)
"Denial of Service." You know, the way some OS' TCP stacks would tie themselves in knots when receiving a flood of bogus SYN packets (every one representing a "connection request" that allocates resources in the stack) or a "super source quench" ICMP redirect ("your new route to me is ... yourself!") or similar attacks.
Everybody seems to be talking about TCP/IP performance on a machine with no load and an empty network. Now test a WinNT database server running 10,000 data base request, that many dynamic request should jack your cpu usage to about 98% (estimate) now were is the processing room for your NIC? What is your average transfer rate at that kind of load?
To be fair try this on Linux also, start up something like MySQL, give it 10,000 request and test the average transfer speed on that.
Also on both machines set your packet size to something small (500) and see how that effects performance.
All these test on machines not really doing anything is not going to prove much, I don't know why they performed these test on regular PC's when the equiptment that is going to benifit from a technology like this would have a large riad subsystem and a ton of ram.
Just something to think about
i was surprised that no reference to memepool was made... Of course there's going to be a lot of overlap of "cool" info... but if everyone tries to take credit for it as their own, we all lose.
From using nmap, it is clear that Microsoft does not use the standard Windows TCP/IP stack on their main web servers, like www.microsoft.com (though they do the standard Windows TCP/IP stack on some of their less popular servers). So what TCP/IP stack DO they use?
Your data is interesting ; but to go further it would be interesting have a tcpdump trace to see how the TCP/IP dynamic interacted. Because Windows and Linux are probably implementing different variation of TCP/IP (there are numerous behavior possible, depending of the options implemented). Note also that getting super-high performance with one flow between two machines, and getting good performance on a big network/Internet are not necessarily completly related. Because of even if you don't saturate, that means that more bandwidth will be free for others machines.
There is no advantage in making webserver in hardware because you'll have to use micro-code in the hardware anyway, with a persistant storage for code. Basically to code a webserver, you need a general purpose computer, and there is no point in putting a webserver on a low-end RISC subprocessor, on a machine where you have a powerful processor (multiple way, with much cache).
Maybe you are implying that soon, say, your monitor itself alone will host a web server, but that will be because your monitor would have a powerful RISC processor and have bigger software.
Would you believe 3D graphics were once done in software?
There have been several cycles to put graphics processing to hardware to CPU to hardware to CPU etc...
For instance when the first 3D game was out (Wolfstein 3D), on 386 a multiplication was taking something like 20-50 cycles ; nowadays on MMX machines you could do 4 16-bit multiplications per cycle. I'm not sure but we could well be close to be moving 3D stuff back to CPU. If you consider Voodoo 1/3D cards prices, an indicator of the die surface necessary to implement the 3D processing, you'll see that probably putting the 3D processor on the same chip as the CPU would be even doable now (it wasn't when the first Voodoo 1 was out).
You should note also that for sound, the CPU has already took the task back ; the CDROM quality , 16 bit, is now doable even on low end cheapo cards (I remember when SB 8 bit was costing $100), audio card makers don't now what to invent now to continue to sell expensive cards, and audio compression (MP3) can be done in software (but not MPEG2 video yet right now, although...).
The extra time with the distributed.net client should actually be easy to explain...
... on some OS's it varies (the less the cpu load, the longer the timeslice on some systems).
When the RC5 client get's a timeslice, it most likely is using the FULL timeslice, whatever it is set to. It could be 200ms, 300ms, 150ms
In addition, while FTPing, the ftp process may be waiting on I/O for some reason or another -- but has nothing else to do. So it'll give the RC5 client another timeslice. Which it uses...
...my thoughts on the matter.
I'm with you. Moving common tasks to dedicated hardware is always a winner. TCP/IP today is like WinModems--the software has to do most of the work.
Will you Linux kiddies stop spreading FUD about Microsoft products? You are looking silly!
In the file kernel/sched.c find the function
count_active_tasks and remove the tests for
(*p)->state == TASK_UNINTERRUPTIBLE || (*p)->state == TASK_SWAPPING
Rob
Wow, you seem to know a lot about the internals of the Microsoft stack.
Which parts of the source code are exactly inefficient, or is this more FUD?
Our studies show that just doing the TCP and IP checksums in hardware more than doubles throughput. In theory, one should be able to turn off checksums and acheive the same performance gain, but in practice, the NT4.0 registry entry that is documented as disabling checksums appears to have not effect... I presume in Linux it is actually possible to disable checksums?
I'm using Samba between Windows 98 and Linux 2.2.6, and my speeds are significantly lower than
that (~ 100 kb/s). I have a decent 3Com card in
the Linux box and a cheapo Linksys card in the
Windows box, along with a Linksys hub. Is there
anything I can tweak to make it go faster?
I never though CPU load was an issue, if 100 Mbit netwwork cards have that sort of cpu load then whats the go with gigabit ethernet ?
Maybe they have some wierdo network cards that dont use DMA.
I think ill stick my 10mbit NIC for now
There are at least a half dozen companies developing/offering routers on silicon. Most have highly parallel and exotic (e.g. hypercube) architectures with hardwired advanced pattern matching algorithms. The future of TCP/IP is in dedicated hardware circuits.
Banging bits out an ISA bus isn't exactly a fast process.
Ok, Ive got two computers with SMC etherpower II 10/100 cards running under Linux 2.2.4, connected by a crossover cable in full-duplex mode, with little other traffic. ie: ideal conditions.
/dev/null
host 1: nc -l -p 5050 >
host 2: dd if=/dev/zero of=/proc/self/fd/1 bs=4096 count=102400 | time nc utrk 5050 -w1
output from host 2:
102400+0 records in
102400+0 records out
0.70user 7.70system 0:46.10elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (140major+22minor)pagefaults 0swaps
since netcat had a 1 sec delay, we get a total of 45 seconds for the transfer of 419430400 bytes = 9320675 bytes/sec = about what they are quoting. CPU load was higher that 2%, obviously (actually, about 25%), but the computer was still usable (it is the disk IO that would kill it, anyway)
Yeah, this is in fact one of the "other uses" that toshiba is planning for the chip that is going to be used in the "Playstation II"
Um, just wanted to point out, that the samba protocols specifies that for most machines out there, you can only send 65,535 bytes in one go. And then you have to make sure that the other side heard you. So, ftp will always be faster than samba.
Remember to keep in mind the differences between bits and bytes, while its probably true that you don't _notice_ a performance hit at 900K/s, what you really mean is 900 kilobits/s. That's nearly 80 times less than 9 megabytes/s. So if the kernel spends 1% of CPU time on 900kb then its easy to see why it might spend a whole lot more time on 80 times more data.
My two cents.
oh yeah, interested people should take a look at
D. Clark, V. Jacobson, J. Romkey, and H. Salwen,
An analysis of TCP processing overhead, IEEE Communications, Vol. 27, No. 6, pp. 23--29,
June 1989.
Classic paper! (well, kinda)
You must be *very careful* assessing the load on a Linux system, as the measurement of the load is far from optimal or complete!
/proc/stat are also useless for the kind of tests required in this case, because the load imposed by interrupt handling is added to the process that happens to be running when the interrupt comes in. When the interrupt comes when the idle process is active, the interrupt handling time is counted as idle time.
The "load average" you see displayed in the output of the "w" or "uptime" commands is quite useless in this case, it shows the average number of user processes that are "ready to run", i.e. they would take the processor if it were available. This number in no way reflects the load by interrupt handlers. It is also badly computed, as it includes processes that are waiting for the disk.
(this error was introduced into the kernel long, long ago and I have mailed Linus about it, but he did not want to change it as this figure better reflected the general feel of load on the system, in his opinion. In my opinion, it makes the whole "load average" useless, so I always patch this away when I compile a kernel)
The numbers in
This means that when a test is run that does not use much user-space processing and heavily relies on interrupt handling (like a networking test where a test process sends useless data over a TCP socket! same when it tries to use a serial port), the load results obtained from Linux will be far off the realistic load on the processor.
Rob
Although writting a driver would be fairly easy, this would break all sorts of features that have come into existance due to open source protocol stacks (not being able to do IP chains stuff to an external IP stack is a definite step backwards).
They would either need to open source the stack and make it downloadable from an OSS driver (like some of the SCSI cards out there) or the card will never get within 10' of my boxen.
Somone already pointed out the security implications. Personally, I don't want to be yanking a card in and out of production just becuase someone built the next teardrop and the vendor is slow to fix it.
As for NT, well, this is obviously the tact that MS is pursuing to gain equal performance with other OSS operating systems, but it has certain implications that will keep NT in the second fiddle chair. The card will probably weaken NT security initially by breaking fixes that are covered by current hotfixes and service patches. Additionally, I can think of no better way to fill up somone's disks than by having improved transfer retes across the net, operating system and disks. Worm designers should re-joice as well. Now, you can design worms that can consume more resources without being noticed. If you're really tricky-trick you could design a worm that existed only within the context of the the TCP/IP stack and if the board has NVRAM... well, a box could stay compromised for years. How long before a Microsoft Weapons System sees daylight?
-- "Most decently written TCP/IP stack applications have NO buffer overrun problems" - an anonymous programmer at Fort Mead
-- "ALL TCP/IP stack applications are a long way from being mathmatically correct." -- A mathmatician's retort.
-- "Our job is to find the differences between one and the other and keep this information from the public as long as possible." -- A manager who successfully defused the situation..
Heh, the sites been slashdotted, they should use their own cards or something... :)
Moving the stack into hardware is an interesting idea, though. Unfortunately it has some negative (and admittedly positive) implications for those concerned about security.
First, it will be impossible to tell what operating system a computer is running by using TCP fingerprinting. This is both good and bad in that it will thwart script kiddies to some extent by not revealing the platform, thus making it more difficult to take advantage of well known exploits. On the other hand things like Netcraft and the Internet OS counter will also not be able to take surveys properly.
Second, and entirely negative, is the possibility that their hardware implmentation of TCP/IP may be sub-standard. It may have scads of DOS loopholes and other weaknesses. Unless they make the thing software upgradeable as holes are found, and make the software Open Source, I don't see it gaining much marketshare against the cheap and plentiful cards we have now.
I submitted the news item, and thought it to be a bit overwhelming to cite more than one source. What is the appropriate limit as to how many sources to cite?
I think going 1 level down is fair -- I got the item from 1 source, the geeks list.
What if an item popped up on the geeks list that came from site1 that came from site2 that came from site3...?
Shoud geeks, site1, site2, site3 ALL be cited?
Opinions?
- A.P.
--
"One World, One Web, One Program" - Microsoft Promotional Ad
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
- A.P.
--
"One World, One Web, One Program" - Microsoft Promotional Ad
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
- A.P.
--
"One World, One Web, One Program" - Microsoft Promotional Ad
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
I've seen ftp reporting 1.1e+3 kbytes/s on my 10Mbit Ethernet. Of course, there wasn't really any other traffic on the network at the time.
\\'
First of all, many threads here have been talking about writing drivers for it for Linux, etc, and second, many threads have talked about makeing it software upgradable to fix security holes.
Consider that BECAUSE it was posted to Slashdot the makers of this card could be slashdotted with email and offers of help for making drivers for alternative OSs. Plus, if slashdotters show enough intrest they'll make OS drivers because they can see that it will increase their sales.
I think this kind of thing is EXACTLY what should be posted to Slashdot because we'd be able to make a difference.
I worked with an expensive Intel NIC 9 years ago that had an i960(I think) and an OSI protocol stack on board. Never did any benchmarks, but I'm guessing the complex OSI protocol stack plus wimpy ISA '386 boxes made putting intelligence on the NIC a good idea at the time.
I figure there must be a good reason these things haven't gone mainstream in almost a decade. The proliferation of simple TCP/IP plus faster CPUs might be one reason.
It is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail. - Abraham Maslow
Not all of your bandwidth is going to go to data. Remember that you have protocol overhead to deal with -- ethernet packets have headers, TCP packets have headers, and so forth. All of those headers take up some of your precious bandwidth, so you don't really have 1.25MB/sec to play with.
The PIO is killing your performance. Switch to a PCI bus mastering card, and your load will go WAY down.
Please, correct me if i'm wrong, but couldn't you
:(
use an FPGA or something similar thereby providing
reconfigurable hardware TCP.
Now that would almost certainly put the cost up and there may be security issues, i'm not too hot on FPGA's.
Anybody know if this has been tried using FPGA's somewhere else ???
Iggy
I can't wait until memory management is moved to hardware only. Maybe Linux MM would stop crashing. Looks like web servers are going to be implemented in hardware real soon. Would you believe 3D graphics were once done in software?
Winmodem people need to learn something from this. Things perform better in hardware than software. Of course, this depends on the openness of the drivers. We may be stuck with a good card and no documentation. Only problem I see is that tcp and other pieces of this layer are intended for software. So maybe this isn't a good idea (I'm tending to agree with the "what if there is a hardware bug" comment). Winmodem people seem to have taken the opposite approach and I'm not sure who is worse, but the winmodem people can definately learn from these guys.
If whoever did this is getting 900 kilobits/s, he's got SERIOUS problems :). Even 900 kilobytes/s is pretty slow for a 10Mbit (although maybe SMB is just a slow protocol?). Still, my ftpd will eat up to 5% of my CPU running at ~1100 kbytes/s, so I can see where this TCP on-a-card sort of thing might come in handy for higher bandwidth stuff.
Thanks, I never saw that before. Still there is more to consider.
When you start offloading stuff from the CPU to a processor on an add-on card, it's going to be a pretty single-purpose processor. This really limits the lifespan of it, though. What if there's some new networking protocol that obsoletes TCP? If the processor on the netcard isn't general purpose enough, then it'll go to waste. What if people start using voxels instead of polygons (hypothetical), are we going to find a bunch of Voodoo cards in the trash?
Of course general purpose processors on cards would be even worse. You'd be much better off with another CPU. But it would be nice to have some means of keeping abreast with changing protocols and APIs and the such.
Remote loopback is a term used by some Ethernet card diagnostics for a network test in which one system sends out an ethernet packet and the receiving system immediately takes it and sends it back.. sort of a link-level ping test.
With a remote loopback, you get hardware testing, but it's probably not a great test for anything having to do with tcp/ip since it's not involved.
- jon
Ganymede, a GPL'ed metadirectory for UNIX
It's a hardware story... how hard would it be to implement this on a linux box?
Just about any interesting new computer toy should be reported here. If linux (and other) users don't keep up on what's new, they're going to end up in the silicon ghetto, just like Microsoft would want.
Putting that functionality in hardware ties you to a particular implementation. Systems have been designed that would handle page faults purely in hardware. They were very complicated and rather limiting. Experience has shown this to be an area best handled by software, which is flexible and can be easily changed or fixed.
Yes, I believe that 3D graphics were once done in software. They still are, in many applications. 3D game rendering is a rather specific task, and very computationally expensive. Thus, it was worthwhile to develop specialized hardware for this purpose. Compared to modern workstation hardware, TCP stacks are fairly inexpensive, if written correctly. I question how how useful silicon TCP would be. I expect that the added cost and reduced flexibility would more than outweigh the saved CPU-load for most cases.
Then again, perhaps NT's implementation of TCP is poor enough to where this is a concern.
Oh, and I never have problems with Linux MM.
--Lenny
//"You can't prove anything about a program written in C or FORTRAN.
It's really just Peek and Poke with some syntactic sugar."
Seriously, Linux is quite capable of fully utilising 100 megabit ethernet by itself, combine that with the cheap price of processor power and this sort of gizmo looks kinda redundant. Oh, and I'll give you 2-1 odds that queso et al identify it as a *BSD OS :-)
fish and pipes
i get about a 50% cpu load on a p5-120/3Com 509/10BT when moving at full speed, messes with my mp3 playing...
Of course, I tried to post this to /. a few times and it didn't make it. Oh well. Maybe people will find it appropriate to this thread.
I've oft wonderd about this maybe time for an other ask how can the load avg be out of sync with the cpu utlisation by such a large margin?
When 3COM first started making Ethernet boards, they tried putting the protocol processing software on a smart Ethernet board. It never worked too well. Unless you had a slow system, with a brain dead operating system or small address space. The processors on the "smart" boards were cheap and slow. The on-board software tended to be buggy, limited and out-of-date. The cards were expensive. You still had to have some sort of light weight protocol for communication between the operating system and the card, adding another layer of software. With well written software, a 25 MHz 68020 host, could run TCP/IP at wire speed on a "dumb" 10 MBPS Ethernet card. The "smart" cards quickly disappeared.
Putting the protocol processing in Silicon will burn you when you need new features and algorithms in your networking stack. What happens if you need large windows, SACK, IPV6, IPSEC, QOS?
The right solution is to use an operating system that doesn't suffer from MBD and use decently designed network cards on a fast bus. 100 MBPS shouldn't be a problem for a decent system. 1 GBPS is where current hardware and operating systems fall down and need improvement.
Mea navis aericumbens anguillis abundat
See subj:
Baker's Law: Misery no longer loves company. Nowadays it insists on it
http://www.sigsegv.cx/
If I remember well, an Ethernet above a 10% usage level is considered very close to being quite dead.
-- Fast, Cheap, Well. Pick two.
Let's get those attributions right!
Peter
I've been wondering about how well a Linux PC would actually be doing on a 100Mbit Ethernet.. I'm a bit worried that it wouldn't be too good, as there seem to be real work for the CPU to do. I became aware of this when I found that an R5000 SGI O2 couldn't do more than max. 5 MB/sec, memory-to-memory TCP no disk involved! And this used more than 60% of the CPU, the system was completely kneeling and with all the other work going on the TCP became a real bottleneck :-(
IRIX 6.3 isn't the worst operating system in the world so this got me thinking.
That on-board TCP stack seems interesting, but only if it supports something else than NT of course.
But then there's the problem of embedded TCP stacks, I've yet to see one withouth strange bugs here and there. TCP stacks are notoriously difficult to get right, in practice it's only a real, open-source preferably Unix box that can be trusted to (eventually) get it Right.
TA
Who's talking out of his ass here? If you haven't got better comments than that please shut up. .sig Dave Miller used last year: //// //// ////
Here's the
Yow! 11.26 MB/s remote host TCP bandwidth &
199 usec remote TCP latency over 100Mb/s
ethernet. Beat that!
>I always patch this away when I compile a kernel) :-)
Would you mind posting a patch for us lazy people?
TA
Thanks,
TA
If you think that 900KB/sec is slow for 10bT, what do you think the average is? Ethernet is not known for it's ability to work well at high levels of utilization, and 900KB/sec is 7.2Mb/sec. At this level of utilization and up, the contention between different hosts trying to talk at once drives up the collision rate which keeps the tranfer rates down.
Such a thing has already become commonplace on the Amiga. Most of the soundcards have DSPs, and most of those are capable of doing MP3 decoding to take load off CPU. It's cool, but I would expect it to only be of interest to people who aren't able to get faster CPUs (e.g. 680x0 users). With today's PPCs and x86 chips, and clock rates approaching a gigahertz in the next year or so, it seems like the processing requirements of decoding MP3s are kinda trivial.
Don't get me wrong -- I'm always interested in custom hardware to take stuff over from the CPU. Heck, that was the whole point of the original Amiga hardware. But general-purpose CPUs are just getting so damned fast... who cares about CPU load anymore? Just get a faster processor. They're already "infinitely" fast for most people's purposes.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
hmm...does this mean that some of those poor, defenseless servers are finally going to have a /. defense mechanism?
Juiced? Or Not?
First, a couple of years is a long time in the networking industry. More impressive first products have been engineered in that time frame.
Second, don't be missled by claims that something is done in hardware. As often as not, this does not mean that the entire implementation is burned into silicon. Often it only means that processing that may have been done on the main CPU has been shifted to a dedicated processor. Often times this dedicated processor may be particularly well suited to the task at hand because it implements special instructions. It may also be on the same die as other discrete functional units dedicated to the task (like an ethernet controller).
What you end up with can be quite fast, but it still retains a bit of flexibility, so it can be reprogrammed to fix bugs, or meet new standards. (or something totally unrelated. Appearantly the engineers at Alteon programmed the MIPS CPUs they use in their gigabit ethernet switches (two per port) to crack RC5 keys for a laugh.)
Well, have you?
- There are many hardware patents out there that don't affect us that much, the question is -- is this one of them?
- is the "Silicon TCP" is indeed a step forward for NICs, and if so
- is the patent based on "prior art", and therefore unenforceable?
I would hate to see something that would work well for us regular (e.g., don't have unlimited funds to buy the next new great hardware) folks trapped within a "can only get it from one company" patent....Open Source isn't the only answer -- but it's almost always a better value than the alternatives...
> Maybe we should try to port netperf
> (www.netperf.org) to Windows and add raw
> Ethernet to it.
netperf allready runs on Win32, I've tested it on both 95 and NT. The netperf ftp site has binaries for Intel and Alpha.
Neither 95 nor NT has any problem saturating a 10Mbps ethernet, but at higher speeds NT kicks the snot out of 95 on the same h/w. I've seen NT get 90+Mbps on a 100Mbps ethernet, but don't have anything faster to test with.
The loopback tests show that Linux and FreeBSD have comparable maximum speeds, much higher then NT (on the same h/w). One assumes that this has to do with how often stuff is moved around in memory, etc. etc.
BTW, if you ever want to convince yourself that ISA sucks, just run some tests on a PCI NIC and then an ISA NIC, and watch the 60-80% increase in CPU use for the ISA card.
The systems were PII-333s with 64MB RAM running NT4SP4, and DLINK 8029 (or maybe 8019) PCI NICs.
I believe that the NT TCP/IP stack has been improved greatly since the 3.5x days, which was what I suppose you would have been running on a P90.
(Sorry for delay getting back to you, has been busy at work)
Sigh. That old myth. Look at the first paper on this page. One of the designers of Ethernet tested it. Drove it at almost 100% of 10Mbps. A bunch of workstations on the net.
They make it sound like they're using fancy servers.
:) Secondly, 48 megs of RAM seems a bit low for NT. Thirdly, the hard drive systems in there are probably also low quality to say the least.
:), although not a Linux box... and it was a server. I don't call a home system a server that will be pumping 90 megabits/sec.
They're not, they're using two proprietary IBM Aptivas, low end machines. (I should know, I have one of them, the same model they used). Firstly, the E56 has a 266 mhz K6, unless IBM lied to me too
Couldn't they have borrowed Mindcraft's server or something? At least THEY could tune an NT box
Isn't this what IO2 is supposed to accompish? This is very cool, but it would be nice for the TCP/IP stack to be configurable and upgradable. i.e. to IPv6 etc. This should be great for homebrew routers and such also. I hope linux drivers appear soon.
I just checked the troughput between a pair of my computers. I did this with a test program that used read and write calls to transfer the data and did nothing else with it. /dev/null is more expensive than the ethernet transfer. ;-)
The data transfer ran at 10Mbytes/second with a 12% cpu load on a PII-266. configuration info at end
Extrapolating to a modern 500MHz system, that would be something like a 6% cpu load. I suppose there are niches where the differense might matter, but with three 10Mbyte/sec transfers the PCI bus will choke anyway and we are looking at the difference between 18%cpu for network and 6%cpu for network.
I guess if you are stuck with a server OS where the TCP stack is a pig and you can't fix it then maybe this is a reasonable optimization, but I sure wouldn't bother with it for linux.
Footnote:It turns out the sending data to
Configuration of Receiver: DellXPS PII-266, Kingston 10/100mbit card(dec21140), Linux 2.2.6, tulip driver.
Configuration of Sender: Gateway PII-300, Kingston 10/100mbit card(dec21140), Linux 2.2.1, tulip driver.
I'm not sure if Kingston still makes these, we buy equivalent cards for $13 now.
Configuration of Network: Half duplex 100mbit ethernet with about 50 machines hanging on it. Mostly idle during test.
Procedural Note: I did run the test many times and exclude the slow outliers, both of these machines have operational services that I did not wish to disable, so many test runs were ruined by other loads on the machines.
Open Benchmark In order to preserve my credibility I will offer to allow anyone to run my tests exactly as I have in order to verify my results. No changes which may improve the results will be permitted.
Thad
The Bolachek Journals
The web page describing this NIC is really unimpressive. What I'd really like to know is:
How many simultaneous connections can this thing support, and is it slower when multiple connections are used? How's the performance when one of these cards is hooked up to a machine with a standard software TCP/IP?
Putting TCP/IP in hardware is nice, I guess, but then nasty real-world issues crop up. What happens if there's a bug in the implementation? There's also the nasty challenge of writing a driver for a card like this, but I won't claim that's a defect of this NIC design.
the big push for gig ether is campus trunks between buildings. goes up to 10km (single mode) and Cisco's Gigabit EtherChannel can aggregate up to 16 links. Really great if your org grew strangely, and you have departments in two buildings. It also helps in linking switches in a internet server fanout. Either way, it's more of a backbone technology than anything else. In fact, PCI32's theorietical peak (132MB/s) doesn't match the throughput of full duplex 1000Base-SX. And NT's networking core prevents running the network over 400Mbit peak.
Interestingly enough, on most tested unices, i think they're getting around 800-900Mbit. It'd be interesting to see how fast ftp.cdrom.com would be with gig ether to the backbone... Maybe then i'd see more than 10KBps...
First off, to all those engaged in the "I get more bandwidth than you" pissing contest: try beating 560Mb/s sustained application-to-application bandwidth between two P2/450GX systems (running NT, BTW, but not NT's TCP/IP). So you think you can beat that, eh? Try beating 2us application-to-application latency for zero-length messages. Can't do it, even with buzzwords like VIA and I2O, can you? OK, next topic.
Regarding TCP/IP on the card: as another poster pointed out, this is not a new thing but a very old thing. I once worked on putting DECnet on old 3Com "smart" cards...ick. There are all sorts of problems with doing this sort of thing on the card. First is upgradability of the network stack. You immediately become dependent on the manufacturer for upgrades - don't expect open-source firmware any time soon, even if you had the tools to compile and load it. FPGAs aren't really a good choice here because they increase the component and design costs too much. You'd be much better off using a commodity embedded microcontroller with "firmware" stored in flash memory, although this may still increase the cost unacceptably and for _real_ speed you just plain have to chuck all this stuff out the window and go ASIC. As it turns out, most systems in most uses have more CPU power to burn than any other type of resource. Some guys at HP several years ago took this observation and ran with it; they designed a card that was even more stripped down than the typical Ethernet card, doing even more of the work in software, and they actually got excellent results.
Lastly, the conversation about NT's networking code reminds me of an exchange I had with an engineer at MS a couple of years ago. He was saying that they had to sacrifice a little on TCP/IP features and error checking (e.g. not crashing if sent a source-routed frame, or something like that) to get speed. My response was that (a) not checking unusual conditions in incoming network packets is just unacceptable, and (b) NT's TCP/IP performance is piss poor, indicating that they have bigger issues to worry about than shaving a few instructions by not checking packet headers. In the time since then I have found no reason to change either observation.
Slashdot - News for Herds. Stuff that Splatters.
O common, what do you expect?
;o).
Can we please be serious about this. 1MB/s is nothing. If win95 wouldn't be able to even sustain that...
Let us know when you get >10MB/s with a 100Mb/s board.
btw. please get your capitalization correct: MB = Mega Byte, Mb = Mega bit, mb = I dunno
Breace.
Using an other system doing the remote loopback, or simply wiring TX to RX on the system that's being tested itself.
Sorry about the confusion.
Breace.
No you don't understand.
With Linux you don't need multiple processors to sustain decent Ethernet throughput.
Breace.
I've seen NT get 90+Mbps on a 100Mbps ethernet
:(
I'm sure you are right, but can you please discribe the system (CPU, RAM etc?), just for us to get an idea. Because I've never been able to see such a thing. Then again, I think I gave up on NT when we still only had P90's...
BTW, if you ever want to convince yourself that ISA sucks, just run some tests on a PCI NIC and then an ISA NIC, and watch the 60-80% increase in CPU use for the ISA card
Right. Although we had a bad experience with the OPTi 802/832 PCI chipset. It doesn't implement DMA burst modes from RAM to PCI devices, and thus limits the max datarate in that direction to about 5MB/s. That's worse then ISA! Anyways, that's an other story altogether...
Thankx for letting us know about netperf. I wasn't able to connect to their ftp server yesterday.
Breace.
Hmm, you must be confused about the D-Link. I don't think they have a 8029. They do have a Realtek 8029 clone though, but that's a 10Mb/s only chip. It's also a NE2*00 clone, which means that it's very unlikely to actually get anywhere close to 100Mb/s because of the silly I/O scheme. I imagine it was probably a DEC based board.
Anyways, that's a pretty fast system,- we don't have one in the office here. I'll certainly redo our tests though as soon as I can, because we do have NT4 now and I think you are right about us using 3.51. (Although most of our test are Raw Ethernet related, not TCP/IP)
Thankx for letting us know.
Breace.
Exactly, this card is of interest because the Microsoft network stack is terribly slow, and costs an awful lot of CPU time.
The main reason for this is: Poor design. (What did you expect?)
The M$ network stack is (as with most M$ device driver architectures) way more complex to deal with then for example Linux and as a result many of the network drivers are not well written. E.g. they are not optimized for performance. I'm sure the writers are happy if the damn thing works at all.
The network stack itself also has much more involvement with each packet going out or coming in then with Linux. In some cases packets are actually copied in RAM. This is what causes the higher CPU overhead.
We have run several tests and the results where depressing. I think we probably lost the source code, but if I can find it I will post it somewhere.
Here's some of the figures I remember: (all tests where raw network tests, RAM to RAM, no hard drives involved)
Two P90 systems using 100Mb/s Full Duplex (DEC 21140) cards. We where unable to sustain more then 35Mb/s using UDP in one direction only.
Linux on this configuration: close to 100Mb/s.
Using a raw packet driver on a 200MMX notebook with a SMC 100Mb/s Full Duplex card and using remote loopback we are unable to get much more then 15Mb/s sustained. (That's 15Mb/s going out and 15Mb/s comming in)
This was using raw Ethernet packets, no TCP/IP or other protocol.
This same configuration in a normal network is unable to receive more than around 4Mb/s UDP multicast packets, or packets will be dropped (thus not received)
I'm surprised that people have kept up with the poor raw network performance of our Redmondians. It's a disgrace, and I will NEVER use a M$ product if serious networking is to be done. Even if the Ethernet adapter handles the protocols.
Breace.
One thing is becoming obvious.
We need some serious network benchmark tools that are cross-platform usable between Linux and Windows and maybe more.
Strangely enough not too many seem to exist. Neither does someone seem to have some hard data on network performance. It entirely useless to say 'I get 1MB/s using FTP'.
A good network benchmark tool will be able to test raw Ethernet performance as well as performance through protocol layers.
Of course it would only test the network and not use the file system or hard drive. It should be very clear what sort of configuration is supposed to be used: two systems running full-duplex, one system using remote loopback or whatever. It could also be interesting to have a > 2 system test to show what collisions do.
And most important as far as I'm concerned: Open Source. I don't believe in closed source benchmarks.
The difference between raw Ethernet and TCP/IP protocol would show us how badly we need hardware assistance on what platform.
Maybe we should try to port netperf ( www.netperf.org) to Windows and add raw Ethernet to it.
Well, maybe I'll be a bit more serious about this if there's an interest.
Breace.
Unless your file is very large, writing is easier than reading, because buffering can cover the seeking latency. The dirty blocks sit in main memory for a while, then when the bus has a chance they sit in the controller cache, then in the disk's cache, and then when the head is in the right spot they finally get committed to media.
/dev/null
Also, with a heavily used disk running a good filesystem, large writes are more likely to be contiguous on disk than reads.
When I'm testing TCP networking performance, I give the web server a file that is just a hole (length huge, size just a few indirect blocks) and use a command line http client. Works nicely:
# httpget -v http://.../~ambrose/hole -o
67108864/67108864 bytes transferred in 8.26 seconds (7930.47 kB/sec).
Java: the COBOL of the new millenium.