It would be interesting to compare costs between a campus wide grid like this and a dedicated beowulf cluster. I believe you will find that the Beowulf cluster will still be more the efficient solution. Of course each situation would be different making it hard to get a true comparison.
Of course the grid will be less money up front, but I think you will find that performance to power consumed will be higher (Especially if you use a water cooled cluster). The Adminstration costs will definitely be higher as a campus wide grid will be a much more complex animal than a straight beowulf cluster.
Now onto the performance issues.
With such a grid you limit yourself in several performance related ways:
Parallel Code. - To achieve good performance on a cluster with such slow communications (network latency), you need problems and code that lend themselves to lots of parallelization. If your code has too many serial routines in it, your performance will suck.
Limited Network Bandwidth. Not only will a grid give you poor network latency which limits MPI performance but it also will provide very poor network bandwidth. This is a problem if you have large data sets to be processed.
Heterogeneous Hardware - This is a major issue. Because a campus wide grid is going to be made up of Heterogeneous Hardware, you are limited in how much you can tweak your code for a specific processor. Sure you can create multiple sets of code for different processors and architectures but that takes time and you will never be able to optimize your code for all the different options available.
Most Beowulf cluster are built with Homogeneous Hardware which allows for code optimization.
Also having different speed machines means that results are going to be obtained at differnet intervals so you would have to limit or remove the interdependence between the nodes.
Having a computer chain sell Linux is a good thing in some respects but I am not sure I like the idea of that linux being Linspire.
Whatever they call their pay as you download more free software model, it stinks. They want to be the Microsoft of the Linux World. If feel that they will do more harm than good in the long run.
My experience is based upon a particularly knowledgable friend of mine who had A+ and CCNA training plus years of Linux use and network administrating (on linux boxes) under his belt and was turned down there.
If his resume was that good he wasn't turned down because he wasn't knowledgeable enough. Chances are more likely that they were afraid to hire someone with too much experience / knowledge because they don't pay well enough to keep them any significant length of time.
Having a high employee turn over ratio doesn't increase the bottom line especially if you provide training to new employees.
I'll agree thats an issue for a novice but setting up the correct modules for an experienced admin isn't in issue. Besides which hardware detection has a nasty habit of going tits up on install and boot as I've seen in mandrake and redhat occasionally.
Hardware detection works most of the time. If necessary it can be disabled. When it does work though it really speeds up the install.
3) How about lack of commercial support? Yes some exists but I wouldn't be surprised if Linspire has more than slackware now.
Come on, thats hardly slackwares fault. As for oracle i used to run an Oracle 8.1 server on a slackware box. Apart from tweaking a couple of install files I had no further problems.
Granted but it is a major issue for someone running commercial software. Sure it may run on slackware but you'll never get commercial support which with some apps is needed. Also the bean counters like it when your 50 Grand software package comes with support.
>5) How about the lack of good administration tools? Sure somethings like webmin and linuxconf can be installed but I hardly call them good admin tools.
Sorry , you've lost me here. You can't claim to be a serious professional then gripe because something only has standard unix admin tools. Sure , go learn COAS or Yast etc. Fat lot of use they'll be too you on Solaris or HP-UX or AIX etc etc. Any serious unix professional uses more than one version of unix and we don't bother learning every 2 bit admin tool for each one , we learn the standard tools and use them everywhere.
Sure I am able to support different distros and Unixes but my clients aren't. We sell hardware and I need a distro that my clients can administer. It costs us money if I need to spend a lot of time supporting the software.
>I have nothing against Slackware personally. However as a Linux professional, I cannot recommend slackware as anything more than a hobbyist's distro.
I think you'll find its the other way around. Suse etc with their easy install for dummies and pretty GUI tools and for the clueless hobbiest. Slackware is for people who know what they're doing and want a fast, efficient, uncluttered system to run 24/7 unmonitored in a server enviroment.
No really I don't think so. Sure slackware is fairly stable but it is more work to administer. That's fine for small installs (much better than trying to administer lots of gentoo boxes) but it doesn't scale well to a large install. The key to managing large installs is efficiency.
Also when I say recommend, I am refering to my clients. If I was hired to admin a small network maybe I would choose Slackware.
Yet if I am recommending a distro for a department of chemistry cluster, I most certainly won't. Especially when most of the Beowulf tools are packaged for RedHat distros. Sure you can setup a beowulf cluster with any linux distro but most of my clients are university profs not Linux administrators. They have classes to teach and don't have the time to fool around getting a cluster running on slackware.
1) How about package management? No one can tell me Slackware has good package management.
2) How about lack of hardware detection?
3) How about lack of commercial support? Yes some exists but I wouldn't be surprised if Linspire has more than slackware now.
4) How about commercial application support? Do you think if I install Oracle on Slackware, Oracle would actually provide support for that install?
5) How about the lack of good administration tools? Sure somethings like webmin and linuxconf can be installed but I hardly call them good admin tools.
I have nothing against Slackware personally. However as a Linux professional, I cannot recommend slackware as anything more than a hobbyist's distro.
Yeah but since I still want a flash plugin to work, I need to stick with 32Bit Java as having both the 32Bit and 64Bit Java's installed breaks both. I really don't feel like fixing Sun's lame linked Library Issues or compiling and running static versions of 2 different JVMs.
Hmm like I deal with the symlinks myself. Chkconfig does that for me. The only hassle is if I have to write a startup script myself but they are very straight forward and I rarely have to do that.
Heck it's alot easier looking for the sendmail initscript in/etc/init.d to edit the startup than searching through one massive script.
Slackware quit being a "Big One" when it ceased being commercially distributed. Patrick's done a good job keeping slackware alive but it's become even more of a hobbyist's distro than before.
With that said if you want to learn linux slackware is a good distro to work with.
Hmm BSD init scripts are just loads of fun. Nothing like searching through one big script in hopes of finding a line of text to uncomment so you can start that special daemon on the next boot.
Yes but most of this work has already been done. The fun part is fixing code that was hardcoded 32Bit by brain dead programmers who never thought their code might be run on an Alpha or Sparc system let alone an Opteron.
You know there's an easier way to install undecorated drivers than decorating them yourself. Disable the requirement for decoration in your registry.
But either way so far there's little point for WinXP 64. No real apps yet and only a few wireless nic drivers. Yeah and I haven't tried to print yet.
Hehe. Of course if it weren't for the odd game I would even have windows installed. Running a good X86-64 Distro like FC is great. 64Bit Goodness. Now if only I could only get a 64Bit Flash plugin, 64Bit versions of all the Windows...er Mplayer codecs, and hey a 64Bit java plugin would be nice.
Oh anyone looking for a wireless nic with WinXP 64 Drivers should look at something using Ralink's RT2500 chip. They have a 64Bit driver on their website.
1) Support and customer comfort. Sometimes it's easier to sell the board and investors on hardware from a known company even if you are installing Linux.
2) Sometimes the PHBs buy the hardware but don't manage or dictate the software installed. They buy with windows server because machine needs an OS and then the IT department changes whats installed after they get their hands on it.
I worked with people who just go ahead and make purchases then ask your opinion after. Eventhough you should have been consulted first because they were authorized not qualified to make the purchasing decision.
3) Even Some Companies that offer linux sometimes have models (or lines) that only come with windows and don't sell without an OS.
4) Sometimes the purchase is made before the IT consultation who's setting up the system is even hired.
Client: Hey we both these Dell servers for a song but we don't know how to make them into a high performance cluster can you do that for us?
Linux Cluster Company: Sure, we a solution that should work for you.
Yes that happens. Often the people ordering clusters forget that they also need to pay for the environmental and structural requirements of these massive machines. This happens with every cluster and is hardly due to the cluster being made of blades.
We sold a cluster to a University in Brazil that stored the hardware for 2 years while they built the building to house and cool it.
It also doesn't count the servers from some manufacturers that come with Windows installed but end up running another OS such as Linux. Remember some vendors don't sell systems without an OS installed and don't offer Linux.
2 to 3 times? That's pretty high density. Our blades are only provide 25% more density and cost less per node than 1U servers. Since we use less power supplies than 1 per blade our power consumption per node is less for blades than 1Us so the extra heat to remove for a full rack is less than 25% more.
The amount of heat put out by a specific cpu is the same whether in a 1U or blade server. (Disregarding speed stepping issues). Thus the amount of cooling needed is always a function of the number of CPUs.
Bull. The MIPS/Watt thing is the same with 1U, 2U, 5U and blade systems running the same processor. The question is are you pushing enough air through the case to cool the system? Or are you using an alternative cooling solution that's adequate?
Most blade solutions sacrifice cooling to provide more density and thus run hotter but I can guarrantee you that our current blade design runs cooler not hotter than our 1Us.
Why? Because we push more air through the blades than we do with a 1U.
That doesn't change the fact that all these sales totals include both hardware and software. Not just hardware. The previous poster was implying that remaining 30% was mostly hardware costs and not OS costs. But in reality that's beyond the scoop of the report and all 3 OS categories included hardware costs as well. The software, hardware cost breakdown is very complicated and his generalization just doesn't consider the variables.
For instance, you will probably find that the average cost of the Windows server is less than the average cost of the IBM mainframe. You will also find that the software that comes with the IBM mainframe is a large portion of the cost of the system than the windows software. If that's the case then actually the windows servers are inflated by hardware costs more than the IBM mainframes.
I am in no way saying that's the case as it's alot more complicated than that.
Depends on the design. Our blades can be cooled just as well as a Rack of 1Us. However since our blades provide 25% more nodes than 1Us they also require 25% more air conditioning.
Slashdot Mirror
Of course the grid will be less money up front, but I think you will find that performance to power consumed will be higher (Especially if you use a water cooled cluster). The Adminstration costs will definitely be higher as a campus wide grid will be a much more complex animal than a straight beowulf cluster.
Now onto the performance issues.
With such a grid you limit yourself in several performance related ways:
Most Beowulf cluster are built with Homogeneous Hardware which allows for code optimization.
Also having different speed machines means that results are going to be obtained at differnet intervals so you would have to limit or remove the interdependence between the nodes.
Or you could just unplug the network cable if you want a temporary solutions. :P
Having a computer chain sell Linux is a good thing in some respects but I am not sure I like the idea of that linux being Linspire.
Whatever they call their pay as you download more free software model, it stinks. They want to be the Microsoft of the Linux World. If feel that they will do more harm than good in the long run.
My experience is based upon a particularly
knowledgable friend of mine who had A+ and CCNA training plus years of Linux use and network administrating (on linux boxes) under his belt and was turned down there.
If his resume was that good he wasn't turned down because he wasn't knowledgeable enough. Chances are more likely that they were afraid to hire someone with too much experience / knowledge because they don't pay well enough to keep them any significant length of time.
Having a high employee turn over ratio doesn't increase the bottom line especially if you provide training to new employees.
>2) How about lack of hardware detection?
I'll agree thats an issue for a novice but setting up the correct modules for an experienced admin isn't in issue. Besides which hardware detection has a nasty habit of going tits up on install and boot as I've seen in mandrake and redhat occasionally.
Hardware detection works most of the time. If necessary it can be disabled. When it does work though it really speeds up the install.
3) How about lack of commercial support? Yes some exists but I wouldn't be surprised if Linspire has more than slackware now.
Come on, thats hardly slackwares fault. As for oracle i used to run an Oracle 8.1 server on a slackware box. Apart from tweaking a couple of install files I had no further problems.
Granted but it is a major issue for someone running commercial software. Sure it may run on slackware but you'll never get commercial support which with some apps is needed. Also the bean counters like it when your 50 Grand software package comes with support.
>5) How about the lack of good administration tools? Sure somethings like webmin and linuxconf can be installed but I hardly call them good admin tools.
Sorry , you've lost me here. You can't claim to be a serious professional then gripe because something only has standard unix admin tools. Sure , go learn COAS or Yast etc. Fat lot of use they'll be too you on Solaris or HP-UX or AIX etc etc. Any serious unix professional uses more than one version of unix and we don't bother learning every 2 bit admin tool for each one , we learn the standard tools and use them everywhere.
Sure I am able to support different distros and Unixes but my clients aren't. We sell hardware and I need a distro that my clients can administer. It costs us money if I need to spend a lot of time supporting the software.
>I have nothing against Slackware personally. However as a Linux professional, I cannot recommend slackware as anything more than a hobbyist's distro.
I think you'll find its the other way around. Suse etc with their easy install for dummies and pretty GUI tools and for the clueless hobbiest. Slackware is for people who know what they're doing and want a fast, efficient, uncluttered system to run 24/7 unmonitored in a server enviroment.
No really I don't think so. Sure slackware is fairly stable but it is more work to administer. That's fine for small installs (much better than trying to administer lots of gentoo boxes) but it doesn't scale well to a large install. The key to managing large installs is efficiency.
Also when I say recommend, I am refering to my clients. If I was hired to admin a small network maybe I would choose Slackware.
Yet if I am recommending a distro for a department of chemistry cluster, I most certainly won't. Especially when most of the Beowulf tools are packaged for RedHat distros. Sure you can setup a beowulf cluster with any linux distro but most of my clients are university profs not Linux administrators. They have classes to teach and don't have the time to fool around getting a cluster running on slackware.
Well if you want to get into it.
1) How about package management? No one can tell me Slackware has good package management.
2) How about lack of hardware detection?
3) How about lack of commercial support? Yes some exists but I wouldn't be surprised if Linspire has more than slackware now.
4) How about commercial application support? Do you think if I install Oracle on Slackware, Oracle would actually provide support for that install?
5) How about the lack of good administration tools? Sure somethings like webmin and linuxconf can be installed but I hardly call them good admin tools.
I have nothing against Slackware personally. However as a Linux professional, I cannot recommend slackware as anything more than a hobbyist's distro.
No there's just one for different runlevels and rc.local. I ran slackware for years. I am quite aware of how it's setup.
Yeah but since I still want a flash plugin to work, I need to stick with 32Bit Java as having both the 32Bit and 64Bit Java's installed breaks both. I really don't feel like fixing Sun's lame linked Library Issues or compiling and running static versions of 2 different JVMs.
Man java vms are such crap.
Hmm like I deal with the symlinks myself. Chkconfig does that for me. The only hassle is if I have to write a startup script myself but they are very straight forward and I rarely have to do that.
/etc/init.d to edit the startup than searching through one massive script.
Heck it's alot easier looking for the sendmail initscript in
Slackware quit being a "Big One" when it ceased being commercially distributed. Patrick's done a good job keeping slackware alive but it's become even more of a hobbyist's distro than before.
With that said if you want to learn linux slackware is a good distro to work with.
Not necessarily. B could also equal zero. Then x could be anything.
Simple. No proper support for multiarch libraries.... Apt can't handle them.
Hmm BSD init scripts are just loads of fun. Nothing like searching through one big script in hopes of finding a line of text to uncomment so you can start that special daemon on the next boot.
Yes but most of this work has already been done. The fun part is fixing code that was hardcoded 32Bit by brain dead programmers who never thought their code might be run on an Alpha or Sparc system let alone an Opteron.
You know there's an easier way to install undecorated drivers than decorating them yourself. Disable the requirement for decoration in your registry.
But either way so far there's little point for WinXP 64. No real apps yet and only a few wireless nic drivers. Yeah and I haven't tried to print yet.
Hehe. Of course if it weren't for the odd game I would even have windows installed. Running a good X86-64 Distro like FC is great. 64Bit Goodness. Now if only I could only get a 64Bit Flash plugin, 64Bit versions of all the Windows...er Mplayer codecs, and hey a 64Bit java plugin would be nice.
Oh anyone looking for a wireless nic with WinXP 64 Drivers should look at something using Ralink's RT2500 chip. They have a 64Bit driver on their website.
http://www.ralinktech.com/supp-1.htm
Actually I mispoke. Officially they are planning a 2007 release.
1) Support and customer comfort. Sometimes it's easier to sell the board and investors on hardware from a known company even if you are installing Linux.
2) Sometimes the PHBs buy the hardware but don't manage or dictate the software installed. They buy with windows server because machine needs an OS and then the IT department changes whats installed after they get their hands on it.
I worked with people who just go ahead and make purchases then ask your opinion after. Eventhough you should have been consulted first because they were authorized not qualified to make the purchasing decision.
3) Even Some Companies that offer linux sometimes have models (or lines) that only come with windows and don't sell without an OS.
4) Sometimes the purchase is made before the IT consultation who's setting up the system is even hired.
Client: Hey we both these Dell servers for a song but we don't know how to make them into a high performance cluster can you do that for us?
Linux Cluster Company: Sure, we a solution that should work for you.
Yes that happens. Often the people ordering clusters forget that they also need to pay for the environmental and structural requirements of these massive machines. This happens with every cluster and is hardly due to the cluster being made of blades.
We sold a cluster to a University in Brazil that stored the hardware for 2 years while they built the building to house and cool it.
It also doesn't count the servers from some manufacturers that come with Windows installed but end up running another OS such as Linux. Remember some vendors don't sell systems without an OS installed and don't offer Linux.
2 to 3 times? That's pretty high density. Our blades are only provide 25% more density and cost less per node than 1U servers. Since we use less power supplies than 1 per blade our power consumption per node is less for blades than 1Us so the extra heat to remove for a full rack is less than 25% more.
The amount of heat put out by a specific cpu is the same whether in a 1U or blade server. (Disregarding speed stepping issues). Thus the amount of cooling needed is always a function of the number of CPUs.
Bull. The MIPS/Watt thing is the same with 1U, 2U, 5U and blade systems running the same processor. The question is are you pushing enough air through the case to cool the system? Or are you using an alternative cooling solution that's adequate?
Most blade solutions sacrifice cooling to provide more density and thus run hotter but I can guarrantee you that our current blade design runs cooler not hotter than our 1Us.
Why? Because we push more air through the blades than we do with a 1U.
Oh I am definitely not suggesting that Windows machines are cheaper to run just that the server software is or was cheaper.
That doesn't change the fact that all these sales totals include both hardware and software. Not just hardware. The previous poster was implying that remaining 30% was mostly hardware costs and not OS costs. But in reality that's beyond the scoop of the report and all 3 OS categories included hardware costs as well. The software, hardware cost breakdown is very complicated and his generalization just doesn't consider the variables.
For instance, you will probably find that the average cost of the Windows server is less than the average cost of the IBM mainframe. You will also find that the software that comes with the IBM mainframe is a large portion of the cost of the system than the windows software. If that's the case then actually the windows servers are inflated by hardware costs more than the IBM mainframes.
I am in no way saying that's the case as it's alot more complicated than that.
Depends on the design. Our blades can be cooled just as well as a Rack of 1Us. However since our blades provide 25% more nodes than 1Us they also require 25% more air conditioning.
Than who Unix vendors or Linux Vendors. The Unix Vendors still charge a far bit more for their OS than Microsoft in a lot of cases.