Slashdot Mirror

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

• Disclaimer
• Why?
• Which Systems?
• What To Test?
• Details
• Results
• Conclusions
• Other Issues
• Tuning
• Future
• Thanks
• Contact

• no OS religious wars please!
• keep it simple - nobody is perfect
• there will be no "winner"
• more complex than expected
• moving targets - everything may be different tomarrow
• depends on hardware, drivers etc.
• ment [sic] as a relative comparison

• curiosity
• linux is more often used in the "classic" server area - how does it perform?
• in the shadow of linux some other free OS are more often used, too - how about their performance
• often comparisons are quite biased
• finding ways to tune a given system for more performance

• all currently available free OS which are usable for production: linux, FreeBSD, NetBSD, OpenBSD, Hurd
• Hurd did not seem to be suitable after some initial tests (1GB filesystem limit, nfs bugs, no posgresql)
• to get a feeling of how all this compares to "commercial" systems we also tested solaris/x86 (7&8)

• filesystem performance
• nfs performance
• webserver performance
• database performance
• scalability (parallel compiling)
• most tests done also in parallel
• a lot more possible

• all tests were done on exactly the same hardware (K6/400, VIA, 64MB, 6GB IDE, 100MBit 3com)
• same hardware was used for client and server c/s tests
• disks were partitioned exactly the same way on all systems
• data partitions were freshly formatted
• all the services are configured as identical as possible on all systems

• all tests were done on dedicated otherwise unloaded systems
• client server tests were done on a 100MBit back to back connection
• systems were carefully set up
• tests were run multiple times to make sure that they give reproduceable results
• client/server tests were run with two client systems: linux 2.2 & FreeBSD 4

• recent versions of all systems
• linux: 2.4.0-test1-acxx - very fresh kernel due to the big problems in the vm and buffer management in the late 2.3.xx kernels, also linux 2.2 was
• *BSD: checkout of the -CURRENT trees from spring this year (2000)
• solaris: version 7 and 8 for x86
• Hurd: not tested

• interest: filesystem throughput
• bonnie - measures transfer-rate of block- and char-wise sequential in- and output on file 2 x main memory
• for nfs: run on a nfs mounted directory
• other things possible but are not done: bonnie++, large amounts of metadata operations, performance on large directories, tests of various filesystems (reiserfs, gfs, xfs, jfs, lfs)

• interest: http access performance
• http_load - can create well customizable load of random http accesses for a given url list
• 3 times parallel: 32 big file fetches (1MB), 1024 cgi fetches (hello), 4096 page fetches (1KB)
• multiple threads and instances in parallel (2, 16, 64)

• interest: performance of sql queries
• postgresql as database target
• a few random value tables of various types loaded into it
• little mix of sql statements ran against it (selects, inserts, updates, joins, deletes, ...)

• interest: scheduling, virtual memory management
• compilation of MesaLib-3.0
• increasing number of parallel compiles
• all with a little relative time to avoid all doing the same at the same time
• why Mesa ? - compiles on all platforms tested, no configure, big enough

• #13 filesystem - char read
• #14 filesystem - block read
• #15 filesystem - rewrite
• #16 filesystem - char write
• #17 filesystem - block write
• #18 nfs - char read - linux client
• #19 nfs - block read - linux client
• #20 nfs - rewrite - linux client
• #21 nfs - char write - linux client
• #22 nfs - block write - linux client
• #23 nfs - char read - FreeBSD client
• #24 nfs - block read - FreeBSD client
• #25 nfs - rewrite - FreeBSD client
• #26 nfs - char write - FreeBSD client
• #27 nfs - block write - FreeBSD client
• #28 http - pages - linux client
• #29 http - pages - FreeBSD client
• #30 http - large files
• #31 http - cgi
• #32 sql - real time
• #33 parallel compiling - real time
• #34 parallel compiling - user time
• #35 parallel compiling - system time Conclusions

• even these simple tests show: there are differences of performance between the systems for various tasks
• NetBSD and OpenBSD are quite close to each other (history)
• at least some of the free systems are in most of the tests better than the "commercial" UNIX solaris
• linux 2.2 has some serious problems under higher loads

• linux 2.4 will most probably bring linux in nearly all tests cases to the top level
• in some cases it is really advisable also to consider the others (esp. FreeBSD for high load, NFS and IO)
• performance is not all (OpenBSD - security, NetBSD - clean design, portability, solaris - applications)
• trying to compare as many systems is more complex than you expect :-)

• SMP scalability (locking problems)
  • linux: 2.2 is ok, 2.4 is good
  • FreeBSD: 4.0 is ok, SMP project
  • NetBSD, OpenBSD: in progress
  • solaris: good
• filesystem scheduling: linux 2.2 has problems with parallel bonnie - 2.4 better
• lots of filesystems to choose from: reiserfs, xfs, gfs, jfs, lfs

• mbuf problem: for all *BSD systems the NMBCLUSTERS parameter has to be raised for any serious use
• with full http_load the *BSD systems tend to produce errors - careful tuning required
• linux nfs implementation has serious problems (async, stale file handles, etc.)

• linux:
  • /proc/sys/...
  • hdparm (-d, -Xxx, ...)
  • kernel config
• FreeBSD, NetBSD, OpenBSD:
  • sysctl kern, vm.
  • kernel config
• solaris
  • /etc/system, ndd

• filesystem:
  • which one
  • blocksize, async, soft updates
• nfs:
  • rsize, wsize, async, v3, tcp
• others:
  • optimized recompilation
  • optimized applications (apache)
  • duplex network settings

• moving target: is only valid for today
• should be continued (time ... :-)

• I would like to thank all people at innominate who helped me with all this ... especially Jens Sanders and Michael Krax and innominate itself.

Presented at LinuxTag 2000, it's a bunch of unbiased benchmarks of FreeBSD, NetBSD, OpenBSD, Linux 2.2, Linux 2.4, Solaris 7, and Solaris 8.

It's certainly more of a worthy benchmark for the front page than the ones here.

What i would like to see, is some serious testing of all current unix'es, doing real world tasks... Focusing on not only their performance, but their stability, AND load bearing... ( who cares if something is fast, if as soon as a great load comes, it crawls )

Too bad they didn't mention the presentation that showed how severely BSD kicks ass over Linux. Click for Truth

You can find the image of innominate's Business Card Rescue CD on http://www.innominate.org/~gpa/rescueCD/