Slashdot Mirror
Clustering with Consolidated Physical Storage?
"It's always seemed to be stupid to convert AC to DC for the UPS, then convert back to AC for the built in power supply and then back to DC for the hardware. Does anyone know of single conversion solutions in the UPS market? Something that goes from AC to DC and then straight to the hardware with protection and backup?
And the next and probably bigger question is about hard drive storage. With the low cost of RAM these days it's very reasonable to put a GB or more RAM in a single unit. And with that much space in RAM, who needs a swap file? So my idea is to design the workhorse units with a heap of RAM and no local storage. This should in theory save power draw, equipment costs and increase reliability. But is it workable?
Linux should be able to boot from network, and I know there are Motherboards out there that support it as well. But I've never tried this. Do they still require a local disk for some other purpose after the initial boot? Would it be possible to run everything from RAM? Would the network traffic become a big problem when the units needed to read or write to the disk array?
What resources are out there that I may have missed?"
Kinda in the right ball park:r =e posfpos
:P
http://www.magnumpower.com/sectors.cfm?mp_secto
Alternatively make something yourself, you lazy person!
-Yarn - Rio Karma: Excellent
It's possible to run everything in RAM (just use memfs's), and to run without a swap, but should a runaway process or memory leak get you with no swap, your machine is as good as dead ;) As far as the power conversion...such conversion smooths the line noise out of the current, giving cleaner, steadier power to vital internals.
For UPS and Power supply discussions you can find previous ask slashot article on UPS's here and discussion on power drain from PC's here
As for network accses to storage etc. You need to be careful. People claim that ATA66 drives can transfer at 66 MB/s put this isn't the case. You need to take into account the file system and OS overheads on the file transfers. This said harddisk storage and access speed is the big bottle neck in servers. Having everything in memory would be great, if not then you need raid to ensure that you can get a decent transfer rate.
On the network front, use CAT6 cables. That way if you find 100Mb is too slow as the cluster expands you can upgrade to gigabit easily.
If you ever drop your keys into a river of molten lava, let'em go, because, man, they're gone.
What the UPS does it takes 12 or 24 or some other voltage (96 in the case of the one I just installed) and convert it to 110v (or 220 or 240) at 60hz (or 50). It does this by makeing lots of steped squarewaves and if you look at them with a scope, they look like little steps.
Your PC power supply then takes that AC and when it gets above 90 V, its starts charging a cap. when the voltage gets too high (like above 100 or so), it stops charging the cap. That energy stored in that cap is then switched at a very high rate through a transformer to charge up another cap to 5V and a different one to 12V and one for -12v etc. which goes off to your cpu and memory and hard drive.
The problem with running just 12 volts is you need to swtich it anyway down to 5 and if you put 12v on one end of a wire, it will be less on the other end. Pulling out the Disney book of electronics... Assume you get have a wire from lusers electrical supply and you put 12 v on one end and measure it at the other. Since its very bad wire, you get 11.5V. Now if you put 1000V on that wire (assume it has good insulation!), you will find the other end is 999.5V. So basicly your wires are going to have a fixed drop based on their length (the current remains the same). This is why the power compaines run 7500V on the power poles and switch it down close to your house.
Back towards the correct tangent...
If you have a large battery pack...
I've got 16 deep cycle batteries (think big car batteries) in two banks of 8. I get 96 Volts accross them in series. Now my over priced inverter takes that and chops it into nice square waves which get run into a toridal transfomer. somevoltage comes out the other side and its switched to chage up different caps at different voltages. Those caps are then switched so the voltage sort of looks like a sine wave. If I put an inductor on the line, it will look more like a sine wave but I'm not sure how the iverter will like it. If you plug in a cheap audio amp, you will hear the noise. The PC's don't care.
With the hassles I've had with UPSs over the years, I was tempted to just run the 96 volts off the battery to the PC's. The next time the UPS dies, this may happen. They don't care since they swtich on at about 90 volts, they will like the battery voltage.
When playing at home, just keep in mind that more than 48v dc can kill you and if it grabs you, you won't be able to let go.
And a final drift in the history direction...
Edison's electrical system was all DC at low voltage. He tried to show A/C (the Westinghouse & Telsa way) was dangerous because of its higer voltage and even killed an elephant which I'm sure you can find on the web somewhere.
Memory is not storage.
Storage is where the computer puts things that it isn't dealing with at the moment; disk space is the normal storage medium.
Memory is where the computer holds things that it is currently dealing with.
I like the hands analogy. The computer's memory is like your hands: if you're working with something, it's got to be in your hands. When you're done fiddling with it, you put it down (send it to storage).
This guy is just wanting to make a powerful diskless compute server, which is great except for one thing: most modern operating systems use, if not require, virtual memory (a region of storage that is used as a temporary holding area for thing that are not currently being worked on, but will be needed shortly). To the user, VM looks like memory, but it's really just storage in a funny suit. Having VM over a network would be painfully slow, I imagine. (Imagine how slow a Beowulf cluster of these diskless compute engines would be!)
That's a peak value sustained data rates are likely to be much less, probably around the 20MB/s area; in practice, you could probably get by with Gigabit to the switch and 100Mbit (~8MB/sec) to the desktop. It's still slower than local disk, but you might be able to get by that with agressive caching on the client side.
there are 2 ways you can handle booting the machines, the wrong way to do it is to have every machine boot off of a floppy drive. The right way to do it is to get network cards that can accept a boot rom(there are lots of cheap ones that can), and use etherboot.
If you really need to, you can use swap over nfs, but its not going to be fast.
I suggest at least 2 nics in each node, one for the booting and nfs, and the other for for the the actual work happening on the cluster. gigabit is a good idea. switched is a necesity.
-- free as in swatantryam - not soujanyam.
Almost all telco equipment runs on 48V DC. In particular, I know there are DC Sun Netras. Good luck affording them, though.
If your cluster nodes are wasting time by filling the IDE bus constantly, you need to move your data into a RAMdisk anyway. The reason for clustering machines is to get more aggregate *CPU* power - not to read data off of a disk. The nodes should be spending more time processing data than they do transferring it to disk. If they're not, then the solution should probably be tuned differently.
The cluster should be fine with commonly available 100Mb equipment - maybe with multiple NICs in the head node to keep network utilization down a little (since all of the other nodes will just be talking to the head node).