Slashdot Mirror

Iperf is excellent for this, especially if you want to test details like packet size, port number, UDP vs TCP...

Your transfer limit is either:

1) Lack of GigE switch between the machines, or a really really poor GigE switch (80 mbps - that sounds like Fast Ethernet)
2) Poor GigE chipset(s) on the mobo.
3) Poor CPU - rsync and scp encrypt the transfer which uses alot of cpu at each end. Also, GigE generates a fantastic number of interrupts/sec. Intel's Pro/1000 GT has an "Interrupt Moderation" feature to help with that.
4) Your hard drive speed. GigE is faster than most single hard drives. Seagate's Savvio 15k could maybe push it, with a STR of 117-97 MBps. A 10k Raptor is only 84 MB/s STR.

Run iPerf between two machines and see what your hardware can really do. (Quick tutorial, run `iperf -s` on one machine and `iperf -c {ip of other machine}` on the other. Then reverse the roles. FYI for a Lan, no IP Stack tuning is necessary)

Yes, trying to get what Gig-E promises has pissed me off too.

Iperf, or something like it is what you should be using for speed tests. Set up the daemon on a machine that you know you need to access and tell it to send a ton of data a few times. See what the results are. Those speed tests test how quickly you can communicate with some random server that you'll never need to send any presentations or video files to in day to day business.

http://dast.nlanr.net/Projects/Iperf/
http://sourceforge.net/projects/Iperf/

Very configurable, and if u want GUI or network tuning.. read the FAQ, they give suggestions.

First, what you're asking isn't really about bandwidth metrics. There's really only one way to measure bandwidth. There are other factors that contribute to connection usability... notably packet loss (probability distribution) and latency. Those are not network bandwidth metrics, though.

Second, you're asking the wrong question. You seem to be presenting data to slashdot and asking them to make a decision for you. You need to figure out what you don't know, and ask those specific questions.

If you were asking for some tools with which to measure connection properties, I'd suggest:
http://dast.nlanr.net/NPMT/

It doesn't make a whole lot of sense for a development team to primarily use a source control system located thousands of miles away, particularly when they have a lousy internet connection.

First, I'd use iperf since it is designed for this kind of testing. Additionally, I'd adjust my TCP buffer limits by appending the following lines to /etc/sysctl.conf:
# increase Linux TCP buffer limits
net.core.rmem_max = 314572800
net.core.wmem_max = 314572800
net.core.rmem_default = 65536
net.core.wmem_default = 65536

# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
net.ipv4.tcp_rmem = 4096 87380 314572800
net.ipv4.tcp_wmem = 4096 65536 314572800
# number of pages, not bytes
net.ipv4.tcp_mem = 195584 196096 314572800


Then run 'sysctl -p' to read in the changes. (sysctl and sysctl.conf may not be available on some distros, in this case you'd have to echo the values into the correct places in proc; google is your friend).

"What practices and tools do you use to test your bandwidth speed and"

Download it here http://dast.nlanr.net/Projects/Iperf/ From the website: "Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss. "

ooking at this map: http://moat.nlanr.net/International/images/collab_ world_map.gif There are a lot of places that, surprisingly, are NOT The United States of America

But look at all those places that are! Like Canada, Mexico, Russia, China, Thailand, Japan...

But another website says the U.S. is just what's in red on this map. http://www.therockalltimes.co.uk/2002/07/08/world. jpg Go fig.

looking at this map: http://moat.nlanr.net/International/images/collab_ world_map.gif

There are a lot of places that, surprisingly, are NOT The United States of America. I hear that those places are prone to ignoring laws passed by the United States. I cannot fathom why those things that are not America would not follow our laws, but I do believe it would make it hard to use a United States law to get them to move thier titties and cockies to a different server.

Give Iperf a try. I used it for benchmarking my home gigabit LAN. It's got multiple versions available for many platforms (as well as source code). It generates data and sends it, not requiring any hard drive access thereby taking drive speed out of the equasion. This blog site also has some more info.

Why not use iperf, which is meant for this usage?

I might be good to start by measuring your network's performance, without hard drives or application software in the loop. I'd suggest using IPerf to accomplish this. If you measure less than expected performance with IPerf, your problem is with your NICs, switch, or drivers. If IPerf reports OK numbers, start looking at Samba and your hard drives. The bus shouldn't be a problem, because even a lowly 32 bit 33 MHz PCI bus has a theoretical 1.056 Gb/s data rate.

I'm using Intel Pro/1000MT desktop NICs and the Netgear GS105 5-port copper switch. My desktop machine is an Athlon XP 2500+ running XP and my file server is a VIA C3 at 800MHz running Linux. Iperf gives me TCP throughput of about 650-700Mbps. I'm probably not getting quite that much on actual file transfers since there's a disk involved, but it is quite a bit faster than running at 100Mbps.

For those who claim gigabit isn't needed for home use... sorry, it is. I was tired of waiting for things to transfer at 100Mbps. There's my need right there. At $30 for a NIC and $85 for a switch, it's not like I'm throwing tons of money at it, and I'm seeing a significant improvement over 100BaseT.

Netgear (and a lot of the other consumer switch manufacturers) claim wire-speed performance on their gigabit switches, even the little 5-port versions. I don't doubt it since they're using Marvell or Broadcom switching silicon. I haven't had a chance to put the Smartbits on my Netgear to prove/disprove the claim though.

Even if you have a $5000 gigE switch and a PC that can handle it, what are you going to talk to, your cable modem? The only place gigabit ethernet makes sense is when you are aggregating traffic from multiple computers to a centralized server or set of servers, and are using applications that actually require that kind of bandwidth. Even if you want to move that much data around, and have a way to do it (hint - neither scp nor samba can talk that fast), the best benefit you'll see is about double the performance you get with 100.

GigE is also useful in beowolfs, if you can't afford Myrinet.

Could you explain why scp/samba can't talk at GigE speeds? I've got a box that can spew data out at gigabit rates (network benchmarks like iperf confirm this) but I can't get samba/ftp faster than about 15MB/s. Yet the CPU load is low. Trying to figure out why, and if there's a fix.

Hi Barry,

Thanks for doing this interview :)

I'm not really satisfied with your answer to my question about dollar cost of spam, but that's OK, you don't have to satisfy me :)

I did want to clear one thing up. I had written:

"As far as I can tell, SMTP traffic is at most 2-5% of net traffic."

And you responded:

"Your figures for the percentage of bandwidth which is spam are far too low. Others have put the numbers much higher. NewsFactor cites studies putting the figure somewhere between 17 and 38%."

I totally accept that spam is about 17-38% of SMTP traffic, that sounds roughly correct to me.

My point there was that SMTP traffic is a very small fraction of total net traffic.

I haven't found any recent statistics on this -- partly because I don't think anyone publishes these numbers anymore, and partly because it's a real pain to try to find with Google. (Do a search on "SMTP NNTP HTTP bandwidth backbone" and you turn up a zillion ISPs bragging about all the protocols they support and how many backbones they're connected to.)

Here's one example of the crappy data out there, a six-year-old report from a link near a backbone showing that SMTP traffic totaled 2.2% of all network traffic:

http://www.nlanr.net/NA/Learn/popular.html

Here's another survey of a backbone, this one five years old, showing SMTP traffic as 3.3% of all network traffic:

http://traffic.caida.org/Reading/Papers/Inet98/

My point was just that if we're trying to assign a dollar figure to what spam costs an ISP, we might as well ignore connectivity charges, because SMTP itself uses so little bandwidth.

As for what all the other costs add up to... I still don't know.

Zeus really is great, it has some wonderful clustering features too, admin for the whole cluster can be done from one place. At the very least it's worth taking a look at the 30-day trial version to get an idea for how much work it would be to port the scripts across.

On a large site, you'll quite likely save the license cost by the decreased use of resources.

(AOLServer is a good server too, though it doesn't have the nice admin of Zeus there's a lot it can do and is also very efficient. I'm not sure whether it can throttle bandwidth by itself though).

Interesting that no one has suggested using tools (e.g. one, two) to actually test to see what happens to a network as TCP or UDP traffic increases. There are a variety of ways to test the way both streams perform under various/different network conditions. The tools allow you to configure these metrics to your whim. (Caveat emptor, if you use these tools, there is a VERY good chance that people will notice and either get mad or just blackhole everything form your IP address.)

----
"they said there's too much caffeine in your bloodstream..."

• In General
  
• Scope of Wireless Project
  
• Progress
  
• Challenges
  
• Security -- Careful: Word Doc!
  

Editorial control from my ISP? I think not. Your view, and that at the moronic Federal and State levels, only make sense if your ISP is really an entertianment company pushing crap down your throat. That's not what the internet is for, and it is outrageous that the public right of way is being given to people who think differently.

1. I didn't say that it was the way I thought things ought to be, I stated it as my view of the way things are. There's a difference.
2. There is no "public right-of-way" in the sense you're using it, at least not in the USA. NSFNet shut down April 30, 1995, when the backbone was privatized. Since then, the US Internet has been a collection of leased lines and custom-laid cables, each of which is leased or owned by some very large corporation, connected by a series of NAPs run either privately or as co-ops. The only reason your traffic gets to use these is that disrupting your stuff entirely would kill their business model (they don't want people to think they would do that).

Get this! I'm not paying an ISP for yet another way to get Hollywood garbage. I'm paying my ISP for communications services. That my ISP would exercise "editorial" control by keeping me from serving, and that my ISP is a monopoly carrier is OBVIOULY against the public interest. My internet connection is worth more to me than my phone, my tv and all my magazine subscriptions as it has taken their place. My desire to contribute to the public domain is shared by countless others, who get it. Blocking our contributions will destroy the web as a forum of information creation and make it worthless, much like the poorly regulated Cable TV, and broadcast media.

Last month there was a presentation by the Berkeley campus net. admin regarding the issues that are being discuessed here. It shows the traffic flows, how they increased when the students came, how problems occured when controlling traffic, and more!

In fact, you can look here to get the story on what various universities are doing to manage traffic.

One possible solution is to run SETI proxies at other universities that will route the traffic to Berkeley via Internet2, since that traffic is free and isn't being regulated/restricted. However, this may not work given that the problem is with transmitting the large data sets to clients, rather than receiving their relatively small responses.

Last month there was a presentation by the Berkeley campus net. admin regarding the issues that are being discuessed here. It shows the traffic flows, how they increased when the students came, how problems occured when controlling traffic, and more!

In fact, you can look here to get the story on what various universities are doing to manage traffic.

One possible solution is to run SETI proxies at other universities that will route the traffic to Berkeley via Internet2, since that traffic is free and isn't being regulated/restricted. However, this may not work given that the problem is with transmitting the large data sets to clients, rather than receiving their relatively small responses.

Actually they have admitted it publically. This presentation is from the Berkeley Net Admin. The problem has to do with flooding *out* traffic. They tried various solutions and ended up lowering the SETI traffic to a lower priority class.

Uh, the problem is *outbound* traffic, not inbound: See here for proof. So, the problem isn't getting the kiddies their MP3's, but getting their requests out. Also see this presentation to get the full story.

SETI@Home floods out a lot more traffic than its gets back because the data sets are larger than the yes/no answer that people send back.

Uh, the problem is *outbound* traffic, not inbound: See here for proof. So, the problem isn't getting the kiddies their MP3's, but getting their requests out. Also see this presentation to get the full story.

SETI@Home floods out a lot more traffic than its gets back because the data sets are larger than the yes/no answer that people send back.

This presentation from the Berkeley network admin (Ken Lindahl) shows exactly how the BW has increased, and the problems they encountered in rate-limiting traffic.

In fact, more presentations about the BW problem at serveral universities is here. They'd like to use traffic shapers, but traffic shapers are only designed to handle T1-level traffic, not OC3-level traffic.

I saw the presentations in person (and I'm from Berkeley). They don't want to get in the business of deciding what is valid traffic, nor investing time to block the various workarounds (e.g., HTML proxies) that people will use to get around the filters.

A temporary solution is to use proxies at other campuses to send the traffic to Berkeley via Internet2, since that traffic is free and isn't being restricted at Berkeley.

This presentation from the Berkeley network admin (Ken Lindahl) shows exactly how the BW has increased, and the problems they encountered in rate-limiting traffic.

In fact, more presentations about the BW problem at serveral universities is here. They'd like to use traffic shapers, but traffic shapers are only designed to handle T1-level traffic, not OC3-level traffic.

I saw the presentations in person (and I'm from Berkeley). They don't want to get in the business of deciding what is valid traffic, nor investing time to block the various workarounds (e.g., HTML proxies) that people will use to get around the filters.

A temporary solution is to use proxies at other campuses to send the traffic to Berkeley via Internet2, since that traffic is free and isn't being restricted at Berkeley.

Slashdotted already.

One of these days, this is going to happen one time too many. You guys really need to start thinking about how linking sites like this affects them.

If this concerns you so much, perhaps you should use your local caching HTTP proxy or hook up with a cache hierarchy.

Hey, guys, you know about the Internet Junkbuster, right? It's a proxy server that will filter cookies, ads, referer information, and lots of other stuff. It's incredibly useful if you desire privacy on the net, not to mention saving your eyes from those aforementioned strobe-light ads.

The IJB is available for UNIX, Microsoft Windows, and Linux. Configuration is just a little bit complicated, but no more so than any other standard UNIX daemon.

Alos, there's a truly wonderful program by the name of WebWasher that will do that same thing under Microsoft Windows. It's got a very slick interface, awesome features, and some very friendly guys working on it. If you have any Microsoft Windows clients, I would highly recommend installing WebWasher on them.

Definitely check out Squid as well. It's a caching proxy server that runs under UNIX and Linux. I've used it for years.

The problem may be easily solved if Bidder's Edge made use of some sort of distributed HTTP cache which would already have the relevant HTTP objects stored and would avoid requests of eBay's servers. These caches are populated by other individuals making use of them through other local caches (you ARE making use of a local HTTP cache, aren't you?)

The Web100 Project is working on putting automatic TCP tuning into the stack. This will allow a TCP connection to use all of the available bandwidth, without breaking any of the internal algorithms or stomping on other connections. It is already possible to tune most TCP implementations by measuring the bandwidth*delay product and tweaking the socket buffer size; the NLANR TCP Tuning page has instructions.

Why bother with this? Everyone should be using cacheing HTTP proxies anyway, to distribute the load.

Thats what the "dream" of a proxy cache hiarchy system was supposed to achieve.

as an example, nlanr's ircache project advocating the use of proxy cache servers all over to reduce the "load" on final servers,and reduce bandwidth requirements of border links. [any proxy cache software can be used, but they use squid



Amarillo Linux Users Group

Well, jeez. My squid wasn't able to cache it, and it's already getting /.ed for its trouble, methinks. Perhaps a nice "Cache-friendly-Apache-HOWTO" or similar should be sent to them.

Cache now! campaign and the Squid proxy server make my LAN life easier. IJB latches onto Squid quickly & easily, so less spam, more content, and lots of calamari for all!
---

I assume you looked at Squid. I may be totally wrong about this here, but here goes (Disclaimer:I have no experience doing this myself. This is my experience seeing other people doing this.)

I have a friend who is an admin who runs squid to do this very thing (sort of). Here's the deal: he runs squid on a box in front of the mail box. Since squid is a cache proxy, you can (and do) look at everything that passes through it. The mail is directed to the squid box. A few lines of perl look for attachments on the mail. If yes: delete/run as guest/send back/whatever; if no: put in the queue. The mailbox (the one that all of the users get their mail from) runs fetchmail. It wakes up periodically, looks in the queue, and pulls over any new mail. IIRC, you can set it up to deposit the mail on an NT box (if needed) so you can run Norton (or whatever). In this way, the mail is disinfected, the users run Win9X, you run *nix, and everyone is happy.

If you like, you can also use perl to pre-sort and filter the incoming mail: .doc attachments go here, .exe attachments go | /dev/null, etc. Hope this helps...

Jedi Hacker (Apprentice) and Code Poet

Which brings up the question of keeping support for multiple filesystems. MINIX has so little overhead that many still use it on floppies. The Squid caching group is working on a new VFS to put on top of the Unix filesystems it is installed on because they are so bad at handling large numbers of small files. It would be great to have an open filesystem standard for a small-file reliable filesystem for such things as caching and user document partitions. Then use ext2/3 for binary/library directories, etc. There shouldn't be a "one size fits all" filesystem we aim toward, should there? Complexity may be a pain some days, but you don't have to expose the average person to this, just those wishing for optimised performance (just like not everyone needs to know how to use RAID).

- Michael T. Babcock <homepage>

There most certainly are cases where it is very nice to have something like Junkbuster= and/or Squid in between me and remote places, as both can help keep things a bit more anonymous.

I'm looking forward to cable modems being more ubiquitous; this will mandate having personal firewall machines, and this will encourage the development of little easily-managed boxes to help with such.

Little Linux boxes would be perfect candidates for this sort of thing; a minimal distribution that has some proxying software, and something like Linuxconf or COAS that can be configured remotely through a secure connection (e.g. SSL) would be a killer app.

Squid is a full-featured, free cacheing web proxy that is most certainly what you want to look at. It is available in RPM and DEB pre-packaged form.

You might also want to look into filtering web proxies that might be what users set up to "hit," to do things like filtering out cookies and/or annoying banner ads. (Not the Slashdot ones, of course!). The "standard" one to mention is Junkbuster but there are other possibly more sophisticated ones as listed at HTTP Links.

I'd hazard the guess that you'd be able to get most of the web cacheing benefits from a 386 box with 8MB of RAM and 500MB of disk; moving up to 14.4GB isn't likely to increase performance vastly over that...

While trying to retrieve the URL: http://crack.linuxppc.org/guestboo k/status.shtml

The following error was encountered:

• Zero Sized Reply

Squid did not receive any data for this request.

NAT helps if there isn't a "socksified" client or an existing proxy. Otherwise, you are saving yourself a lot of potential IP headache...

Here are a few more links for more information about HTTP and some neat things that are being done with it...

• Get the latest dirt from the World Wide Web Consortium.
• RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1 ( text, PostScript, PDF)
• Berkeley's TranSend service is a cluster of workstations working together to act as a massive HTTP proxy. This proxy "transforms" Web pages based on clients settings. Was the basis of the ( now-commercial) Top Gun Wingman Web browser for the PalmPilot.
• The Anonymizer acts as a proxy that strips out all the unwanted/unneeded header lines that your Web browser sends.

I had started hacking together an HTTP/1.1-compliant proxy in perl that did on-the-fly compression if the client supported it, but I never got around to completing it. Initial results were impressive, especially when it was paired with a caching proxy like Squid or a CacheFlow box. Of course, with DSL and cable modems getting more widespread use, people like myself that are still pinned to a 33.6k connection are being left behind.

Caching/compressing/proxying is still in widespread usage outside North America (most notably Australia and European countries). Their problem was (is!) outrageous access prices and relatively slow overseas connections, so they've been using caching for a long time to help solve it. The US and Canada have solved their "problem" of Web pages not instantaneously loading by throwing more bandwidth at it...

TCP Tuning information for many operating systems (including Linux, BSD, and Microsoft's offerings) may be found at the Pittsburgh Supercomputing Center and NLANR Engineering Services