Slashdot Mirror
Wcarchive Does 1.39tb In 24 Hours
Josef Grosch writes
"Walnut Creek's FTP machine, wcarchive, running FreeBSD
transfered 1.39 Terabyte in a single 24 hour period. "
← Back to Stories (view on slashdot.org)
← Back to Stories (view on slashdot.org)
FreeBSD has a feature that allows you to have more filesystem than disk. It's actually pretty useful (argh, filled up / ).
Check the FreeBSD handbook entry for how to do it:
http://www.freebsd.org/FAQ/FAQ52.html#52
The sort of optimization I'm thinking of here would be along the lines of giving some daemon a hard-coded place in memory, or some similarly evil thing. (I recall hearing that NT did something along these lines; a discussion of it came up on one of the /. Mincraft threads.) Or are Unix derivatives so versatile with kernel tuning parameters and such that a single-purpose OS (ServerBSD, perhaps, or "Servix" :) would be unnecessary?
:)
Or, if you optimize it for serving pr0n, you could all it "Cervix"
Sorry -- couldn't resist (LARTing self heavily)
http://www.bullnet.com
Maybe in a 24-hour period, but so what? It just exceeded a round number. Is getting 0.9tb much different than getting 1.1tb?
I haven't laughed like this in ages.
... Same kernel, same apps ...
... great support from Microsoft. It can also play games ...
and even better...
Hah!
>Linus likes penguins
I thought the penguin was a joke, because Linus was bitten by a penguin...
The actual theoretical when you consider layer 2 overhead is about 1.07TB. Then subtract Layer 3/4 .97TB. Prior to this weekend's gigabit ether test, the record was .969TB, which seems to confirm this.
overhead, the global Internet packet loss rate (which averages about 7%), and FTP overhead, and you end up with about
-DG David Greenman Co-founder/Principal Architect, The FreeBSD Project
Round up to 954,841 and THAT'S how many floppies it would take to hold that much data. Wonder how long it would take to split that? Any estimates? =)
Well, zipping a large file to disks here, without compression, takes a fairly regular 80 seconds per disk....so....
80 * 954,841 == 76,387,280 seconds == 21218.69 hours == 884.11 days == 2.42 years (roughly).
Not bad for a days work. :-)
Umm...(Score:2,Informative) ? (ahem!)
dylan_-
--
Igor Presnyakov stole my hat
This is great. slashdoters aren't falling for the flamebait tricks! yay!
Kudos to you, sir.
The thing to remember is there's a VERY big difference between 1 drive/1 controller nad a hundered drives on 5 controllers (2 double channel + 1 single)... doing the first is trivial, doing the latter and not letting it eat up CPU isn't.
Linux VM system doesn't scale. You can tune it to a given load, but it just won't be able to adapt to different loads.
Though I suspect this issue is not relevant for wcarchive. For one thing, since it's sole purpose is serving files through ftp, it doesn't need to adapt to different load usages. For another thing, I doubt it ever swaps... Not that Linux can handle 4 Gb of RAM on ia32, but...
Nevertheless, there are still gains in the handling of memory on the FreeBSD. David Greeman _is_ one of the responsibles for the vm subsystem in FreeBSD, and has been so for a long time.
(8-DCS)
They don't seem to offer *any* other pre-loaded operating systems other than Windows NT.
They could get early market share on FreeBSD pre-loaded servers.
If they don't, I'm sure someone else will...
Any Free/Net/OpenBSD VARs out there?
Would be too hard to make it real - What would happen if tomorrow morning you read in Slashdot about such a benchmark? I can assure you that thousands of people would be requesting more and more info from the free server, and hundreds would be sending DoS attacks to the NT one... :)
FreeBSD has a new ATA subsystem under development that will bring the same performance improvement CAM brought to SCSI... to IDE!
I'm running with that code right now, and it is EXTREMELY fast!
--Alex
Topic says it all.
-- filgy
No, but using a network monitor to count the bits for us would be quite easy to do.
Just reset the count every 24 hours.
When I had an IDE disk, a Samsung 3.2 GB drive, it ran pretty well, but you have to realize that FreeBSD is really more of a server/workstation OS, and no server/workstation should really use IDE. PCs, which happen to be Linux's domain (more so than FreeBSD), however, do not need SCSI, and can do fine with IDE.
Well, let's do this together!
- -
Supposedly 1,391,836,770,942 bytes have been transferred (bout 1.39TB).
A floppy disk has 2827 usable 512-byte sectors. Do the simple multiplication and that floppy disk holds 1,457,644 bytes.
After some division fun you get 954,840.6017724.
Round up to 954,841 and THAT'S how many floppies it would take to hold that much data.
Wonder how long it would take to split that? Any estimates? =)
--Rob
-----------------------------------------------
void docrazythings(){
docrazythings();
}
You forgot the original site: The FreeBSD Token Ring Project
The biggest easily identifiable things that make FreeBSD be able to handle this load are the CAM subsystem (to serve up the data fast), which Linux currently lacks. Justin Gibbs did an excellent job of getting close to the max performance out of SCSI with CAM. Linux's SCSI subsystem is primitive and slow in comparison. It lacks good error recovery and mixes too many levels of abstraction. While it does work for most people most of the time, I would doubt if it could drive the I/O subsystem as fast as FreeBSD does.
I'm biased. I work with Justin here at Pluto, and we have Video server machines based on FreeBSD that are disk bandwidth limited. It is very fast and I'm very impressed with it.
Warner Losh
But I don't want to use any microsoft contribution, besides WHAT CONTRIbUTION?????
your M$ put alot of money on kernel research. WHO CARES, I wish you die soon for what you said here. I dont want you to post anything here, you don't belong here, Go play around somewhere else, DIE YOU MORON.
http://www.6design.com/chuck-wan.gif
Your right.
I want a gf that dares me to make posts on Slashdot.
I think that'd be cool.
I have seen the future, and it is inconvenient.
While I'm never one to bash linux, or even NT for that matter (ok, maybe a little), The last post looks pretty funny to me:
couldn't have done it without us? sounds a little like BSD envy to me.
all in good fun, all in good fun......
--atdot
(bsdie)
(Score:-1, Flamebait) -what the hell is THIS about?! I'm a pro-linux user. If you can't see the humour here then don't moderate. I was doing my best Darth Vader in light of all the M$ crap flying around here. See that name up there.."Khan"...that means "really cool linux mofo". Remember it! :P
"Klaatu, verada, necktie!" -Ash
Does anyone here know how to switch this controller to 80M/s on Linux - I can only get it to work at 40M/s on Linux 2.2.8. Not that this speed is critical - I'm not having much luck finding hard drives that'll do 80M/s, I'm just curious.
,hacker Perl another Just)'
Thanks.
perl -e 'print scalar reverse q(\)-:
Matt. Want XML + Apache + Stylesheets? Get AxKit.
Why don't we do away with such notation anyway and quote in NT units, i.e. the equivalent of X similar NT machines.
:)
IDE is okay for occasionally-accessed small amounts of data, but for things like workstations or servers (servers like wcarchive), SCSI is definitely the best choice.
"Welcome to wcarchive - home FTP site for Walnut Creek CDROM. There are currently 4963 users out of 6000 possible. This machine is a Xeon/500 with 4GB of memory & 1/2 terabyte of RAID 5. The operating system is FreeBSD. 100Mbps colocation services provided by CRL Network Service." (from ftp://ftp.cdrom.com )
Up until just recently the box was a lowly 200pro as far as i know and was still setting records.
Daniel Harvey
The subject says it all. What exactly _is_ your point?
:/
This is genuine curiousity here. Your point went right by me.
It was NEVER a lowly 200pro. It was a 200pro with a lot of RAM. Serving up 13 MB/second sustained on a system with modern SCSI hardware (disks and host adaptors), a large file system cache, and modern network cards (that do bus mastering DMA) is not a CPU bound task.
I'm on 4.0 CURRENT aswell and the only big difference I notice is less delay on bootup.
The claim came from the server generated stats.
What exactly would you propose we do? Count the bits one at a time as they leave WC CDROM?
We've heard about wcarchive setting records several times in the past couple of months. Is this really news anymore? Is it a news story when the number of hosts on the internet sets a new record?
Let's put it this way. Theoretically, we'll say that the server can use the entire 100 Mbps capacity of the connection, all day long. A day lasts for 86400 seconds.
Total theoretical bandwidth: (100 Mbps * 86400 seconds) / 8 bits per Byte == 1.08 TB (assuming
1000000 MB == 1 TB).
Of course, overhead will drop this number some, but they must be saturating their Fast Ethernet line continuously!
This IS news, but only because the record went over 1TB. If they set one over 2TB, that will be news, and the same if they do it over, say, 5TB -- 1TB, like the 1TFLOP in supercomputing (though less significant) is important, because another order of magnitude of transfers has been reached.
John_Chalisque
After all the flap about Linux-vs-NT, how about a *real* realworld benchmark -- Have the OS in question pull a 24 hour shift as the WCArchive server-- Whomever puches packets faster and crashes less, wins!
Hilary Rosen's speech was about her love of money and her desire to roll around naked in a pile of money.
Are we sure that this is flamebait? To me it's so funny that it can be nothing other than sarcasm and could pass for a story on Segfault with just a little bit of polish.
This machine is a Xeon/500 with 4GB of memory & 1/2 terabyte of RAID 5.
NetFRAME 9201 server machine provided by Micron Electronics. Please visit http://www.micronpc.com/web/walnutcreek.html.
You're confusing terabytes with megabytes, I'm afraid. A megabyte is 1,048,576 bytes, whereas a terabyte is 1,099,511,627,776 bytes.
Actually, it servers more RedHat installations than ftp.redhat.com and all other mirrors added up.
I think we can safely say this was meant to be sarcastic, why does it get a -1?
Search first, ask questions later.
Can't you get 1.44 tbytes to a floppy?
Jeez...
Let's see...it's the official server for FreeBSD and Slackware Linux, as well as being a mirror for NetBSD, Red Hat, TurboLinux, Debian and Suse (check in the Sunsite dir for Suse). They also have a lot of ID Software stuff (don't know if they're an official mirror, but it looks like it). They seem to have a lot of other things, but I haven't really explored the UNIX or Windows stuff yet.
Are you an idiot, or just kidding!
Congrats to the BSD guys on this one. :)
:)
I'm just curious what the machine config is OTHER than the gig ethernet.
_Deirdre
Is your GF from Microsoft?
Posted by Mr Spock:
I would also like to echo your sentiment.
read the file describing the configuration.
It's a Siliconrax SR-485 rack mount peripheral chassis with a built in 9" or 10" monitor
It also houses the Mylex DAC960SXI SCSI-SCSI 6 channel RAID controller, w/256MB cache.
"Yeah well
'Twas due to the Slashdot Effect after posting the Slackware 4.0 article here, right? There's too much coincidence...
In Soviet Russia, Jesus asks: "What Would You Do?"
OS design is a matter of comprimise, especially with general-purpose operating systems. Linux and the BSDs have to accomodate people using machines as web servers, or as HTTP clients, or as single -user machines, or all of these at once and more. This necessitates comprimises between such things as portability, and speed, and generality, and code-ability of features, and so on.
/. Mincraft threads.) Or are Unix derivatives so versatile with kernel tuning parameters and such that a single-purpose OS (ServerBSD, perhaps, or "Servix" :) would be unnecessary?
My question is, why not take one of these general-purpose OSes and, for some given task, hard-code in optimizations? This new OS could still remain backwards-compatible with the old OS, so it wouldn't really be a fork.
The sort of optimization I'm thinking of here would be along the lines of giving some daemon a hard-coded place in memory, or some similarly evil thing. (I recall hearing that NT did something along these lines; a discussion of it came up on one of the
WC Archive is a very well tuned FreeBSD system. It was years of experience with a high traffic ftp server that they got the performance out of it that they now get. I once read somewhere about Jordan K. Hubbard saying something about this. They also use a lightweight ftp daemon that is specialized to the task. WC archive is probably where the FreeBSD project gets alot of feedback about the performance bottlenecks in the OS. To make it fair, you'd have to let some linux people and some NT people run a high traffic ftp site so that they can learn how to tune their OSes to peak performance. But how long and how far does the "tuning" go? Is source code modification considered tuning? Or how about source code tuning of somethings but not others? You can't really compare another OS in serving ftp files to the system WC archive runs I don't think. You could possible test stock installs w/ a decent ftp daemon (like wu-ftpd in the FreeBSD ports collection). If possible, compile the same ftp server on all OSes. Then launch each into the fray.
If Hotmail couldn't do it with Microsoft's backing, I can't imagine it being worthwhile here without it.
With Linux getting such media attention, it's normal that some people in the community don't get the message or the point or whatever... It's like everything else, kids come, get bored and go away...
I'm definitely impressed with *BSD, and may well give it (them?) a go when I have a PC for it.
Anyway, the point is, we don't fight each other mind you, we just fight world domination by ONE and unique operating system... I want choice! I want the best system for the task, whichever it is... And if FreeBSD is that much better than other systems for File/Web/FTP servers, well, I'll just have to try it by myself!
Repeat after me, we're not fighting each other, we must keep an open mind and as they say over here, what have I contributed to the community today?
---
"Hasta la victoria siempre!" El Comandante
The second you have to access two disks simultaneously, IDE looses. If you make each one a master, then you are really putting them on two different controllers.
If you have lots of disk access, lots of disk, there is no way IDE can compete. Even more if you want your CPU time for other things. Ever heard of tag queues?
(8-DCS)
The free un*x programmers have put a lot of love into kernel research/development to make their kernels, that's why they are superior to NT.
Bill - aka taniwha
--
Leave others their otherness. -- Aratak
Absolutely! This is the first time any public FTP server has transferred over 1 terabyte of data.
But the mindcraft study was between Linux and NT, so in this context it would be pointless. Now if you wanted to use if for a completely new study then that is a different story.
---
Personally, I try to write "bits" or "bytes" explicitly, so that there's no possibility of confusion. (Except of course there is, because sometimes "mega" means 2^20, and sometimes it means 10^6.)
http://www.bafug.org/NewRecord.html implies that the amount of data shifted really was 1.39*10^12 bytes in a day, which I reckon comes to around 15 megabytes a second on average.
1.39 terabits/day wouldn't be news.
Actually, the connection can't keep up with the disks. This recent record (of over a tb) was due to the recent upgrade to gigabit ethernet. With the previous 100Mb ethernet, a tb was a mere dream.
Same goes for the first letter: T is the real symbol for Tera(=10^12), while M is for Mega(=10^6) and m is for milli(=10^-3), D is for deka(=10^1) and d is for deci(=10^-1). t has no special meaning, so it may also be understood if used for Tera although I would not recommend using it this way.
My PC has 32 MB of RAM, while my modem transfers data at a speed of 33 kb/sec - You see de difference.
ms
PS: I'm missing SUP and SUB for super/subskripts - Rob, pls add them
But it's more than just 13MB p/s. I'm sure almost any modern OS, including NT, could pump through 13MB p/s. But cdrom.com is doing much more than that. It's managing 5000 concurrent connections as well. I don't know that any OS could do that _and_ still pump out 13MB p/s.
Where do the benefits of the CAM system manifest? I run Linux with an AHA2940U and 7200rpm Quantum Ultra SCSI drive, and I can't really see how it could be any faster. (Seems hardware limited) Incidentally, the AIC7770 driver for Linux is derived directly from a recent version of the BSD driver.
See my Linux Server Performance Checklist at:
http://home.att.net/~jageorge/performance.html
The Linux logo is perfect, just look at Linus Torvalds. Don't you agree that he looks like some fat penguin with a bad haircut?
Hi, Im the original guy who posted this.
Yeh. It was sarcastic. Plus, my girlfriend dared me to do it
hehe. None of it is serious, I was just testing how ignorant i could make myself sound.
I believe you; I'm sure they optimize every kernel parameter they can.
I guess my question (and I feel really naieve about this, never having done any OS design of my own) would be whether it would be helpful to do "bottom-up" programming with the OS -- not just changing the application to run well on the OS, but changing the OS to run the application well. Evil things, like changing variables to constants, or hard-coding memory addresses into code.
I don't know if this sort of thing has really been possible before, since free, hard-core OSes are relatively new, and those selling them haven't been highly focused on benchmarks.
Why? Because the point of the Mindcraft survey was to denigrate free UNIX, of any stripe. Whether it's FreeBSD or Linux it sheds doubt on their veracity either way.
I agree, FreeBSD and Linux RULES!!! I'm glad to see that it is hitting the spotlight a lot more ofter. Now if we could just be a little nicer to each other......
I alway wonder how those cdrom.com manage to finance the whole thing. I really doubt that you can sell so many CD's ... btw: what do you think they'll have to pay for such a connection ?
looks like a microwave! Anyone know what it is?
The disk I/O system isn't that amazing. Linux has no trouble doing a sustained 13 MB/second, which is what is required to satisfy that load. Also, consider that every time they set a new record, it is soon after some major large thing has been released. They've got a lot of RAM, and that large thing (e.g., the SiN demo, RH 6, Slackware, etc.) fits in cache, so the disk probably isn't that big a factor.
This is my name man...c'mon, be kind
And it was a joke...I just think floppy whenever I see anything around 1.38 or 1.44 or somewhere there...
I could never get meg's confused with tb...I would hafta be extremely stupid.
Though that fact about the 900,000 disks was kinda cool...
You are most likely not running the comparisons in a fair setup. The default setup of the filesystem on Linux is to gamble with the users' data (metadata is trashed on uncontrolled reboot); the default setup on FreeBSD is to not gamble with data. If you want to compare the FS speed of Linux and FreeBSD, the following comparisons are (reasonably) fair:
- Linux default mode (async) vs FreeBSD in async mode
- Linux default mode (async) vs FreeBSD with soft updates (in this comparison Linux is unsafe and FreeBSD safe)
- Linux sync mode vs FreeBSD sync mode
The best mode for FreeBSD is usually soft updates mode; this is not the default mode due to licensing restrictions (the license the soft updates code is available under is about as restrictive as the GPL, which is much more restrictive than FreeBSD consider OK).As for the TCP/IP stack of Linux being faster: This does not surprise me; I expect it to at the very least be faster for anything that can be micro-benchmarked, and probably as fast as the BSD stack under real world load, too. Dave S Miller has done a very good job. If we (*BSD) have an advantage for real world load, I expect it due to that not being as easy to measure, so we may have an advantage due to actually having run under real world load for a long while).
As for SMP: In my opinion, neither FreeBSD nor Linux is really up to snuff. Attempting to judge between them is really pointless.
Eivind.
Doubting the existence of evolution is like doubting the existence of China: It just shows that you're uninformed.
FreeBSD also has much better SMP support than linux does, UFS is also a little faster than ext2 (so I'm told but I may be talking out of my ass). The new WC server has like 4 Xeons, and a bunch of RAM. I would guess that if it were on linux they would have to drop the user limit back to 3000. But thats just a guess.
I'm a loner Dottie, a Rebel.
Go FreeBSD! :)
Wow....Walnut Creek's FTP server is mighty popular
though.
-- queef
Well, I have no comment as to how well Linux could do in comparison to FreeBSD, but will this machine even be the same on Linux. ie. Can linux even access 4GB of RAM on x86 hardware? Last I heard it was about 2GB.
:)
_BECAUSE_ it is running FreeBSD.
Why? The machine is not running Linux, it is running FreeBSD.
---
FreeBSD is more or less linux: Same kernel, same apps, so Umm.. I think not. FreeBSD is based on 4.4BSD which was developed by Berkerly. Linux is a homemade kernel done up from scratch. Sounds like someone needs to do there homework.
Posted by LocNar:
Now you all know the Power of the FreeBSD side!
Quick, submit that machine to mindcraft! :)
--
I boot freeBSD 3.1 and Linux 2.2.* and NT 4.0 on my SMP box and theres basically no difference between freeBSD and Linux except Linux has faster disk performance (on my hardware) and slightly faster tcp/ip transfer speeds, freeBSD might have better SMP but it's to close for me to pick it. In 2 months FreeBSD has crashed once and in 8 months Linux never has, never even had X crash the _only_ thing crashable is netscape :), as for NT it can crash whenever and is pretty slow.
Realistially, what would happen if you substituted say... Linux or (god help us all) NT for FreeBSD on that machine. Seriously, would it suddenly grind to a halt? What exactly is special about FreeBSD that makes it and it alone able to dole out data at that rate? To me (and I am not an expert by any means) it sounds like most of the ability of this site is dictated by the amazing disk I/O subsystem. This is an honest question, not a flame or criticism of anything. I am just curious.
This claim is unverified and lacks the substantiation of an independent and impartial audit. It is like the questionable golfer who claims he hit a hole in one while playing the links alone at midnight.
These sorts of unscientific claims should be taken with a large dose of salt in the absence of any independent and impartial corroboration.
How does FreeBSD do on IDE disk i/o? Better than Linux or what?
Cheers,
Alex.
--
http://www.tahallah.demon.co.uk
alex.buell@tahallah.demon.co.uk
You, my friend, are clearly attempting to flamebait. What exactly do you mean that "NT clearly outperforms Linux hands down"? If that's based on the Mindcraft benchmark, you should realize that is entirely unrelated to Wcarchive's FTP serving load, not to mention the fact that the tests reflected mostly software tuning rather than anything else. If you're going to cite "stats" at least try not to totally make them up.
Love alone does not a good kernel make. You need some brains too. I wouldn't trust Linus Torvalds to find his way out of a wet paper bag.
this clearly is a joke, not flaimbait
i get the joke!
I dunno... that AlphaLinux penguin looks pretty demonic already, if you ask me....
Seriously,
I use both IDE and SCSI on my servers and workstations.
For large data stores that are rarerly accessed but "must" be online, the new large IDE drives are way-fine, plus the price can't be beat. For the price of a single 50GB SCSI drive I can throw together a computer and 80GB of IDE storage. Transfer speed of these linux boxen over our 100-Base-T network keeps up (and even outpaces) the striped 18GB SCSI drives on our NT server....
Is it as reliable as SCSI? No, but all the data is on tape. Will it hold up under heavy multiuser access??? Probably not, but not every server needs that sort of muscle......
Ditto for workstations. I'll take two or three 20GB IDE drives any day over a single 18GB SCSI drive....
On the other hand just what would it cost for a
5000 user regestered copy of win-nt4, more than the machine ?
Span...
How many of us here have gotten their shiny new version of FreeBSD (3.2), Slackware (4.0) or Red Hat (6.0)? Who was downloading game cheats and such? How about downloading Seti@Home from them? Did anyone download something from cdrom.com's XFree86 mirror yesterday? How about their GNU mirror? See my point?
Mike
--
Mike
--
"Wi nøt trei a høliday in Sweden this yër?"
Pretty soon all those people that complain, "Rob, a new Linux kernel comes out EVERY week! Is it really Slashdot news-worthy?" will be saying "Rob, cdrom.com sets a new transfer record every week! Is it really Slashdot news-worthy?"
And I love every minute of it.
I'm a FreeBSD Beastie, but I still seem to know more about Linux than you..
Linus, not Linux. Linus likes penguins, *Linux* is the kernel. As for *demonic unix roots*. What?!
But I do agree on one point, the Linux logo sucks. The *BSD logo rules.
Unscientific claims? How much more scientific can this get? These are simply bandwidth stats on how much data was transferred! They are not OS benchmarks!
I seriously think that he's kidding, and his "flame-bait" post is meant entirely tongue-in-cheek.
Linux users may be vociferous about defending their OS, but they *do* need to stop taking every single post literally.
This reminds me of that post on Usenet a year or two ago ("Why Windows NT is better than Linux") - it was highly satirical, yet people flamed the author, assuming that he was seriously comparing
Win9x favourably to Linux!
grayson@wwmerchant.com
Not AC, just lazy
120 GB for the FreeBSD tree...
13 GB for Seti@Home...
1.5 GB for cheats (seems low for a scr1pt-k1dd13 area)
1 GB for XFree86 (heh)
and at the bottom...
/UPLOADS.TXT 2k.
--
--
The Internet is the Suppository of All Knowledge. You get it in the end.
Woohoo! Now, feel the power of the Dark Si...er...I mean, FreeBSD ;)
"Klaatu, verada, necktie!" -Ash