Slashdot Mirror
Best Motherboard for a Large Memory System?
kimanaw writes "Due to a particular infrastructure need, and increasing OS support for 64 bitness, I'm looking into building a large memory server box (at least 16 gigs, possibly up to 64 gigs, probably config'ed into a big ramdisk). I only need a single CPU, and just minimal disk; most prebuilt systems w/ large memory seem to focus on more CPus and big RAID, all of which (over)inflate the pricetag. I've searched several websites (including Tom's Hardware), and I've googled, but can't seem to locate any commercially available AMD MBs supporting more than 4 sticks of RAM, or 4 gigs. Have any Slashdotters built a big-RAM server? Any pointers, hints, and tips much appreciated."
You know there are 4GB DIMMs around nowadays. So, 4 slots = 16GB. And 2G DIMMs are all over the place and relatively cheap, so you could get 8G with 4 slots at a minimum!
I've seen a 4P Opteron system with 16 DIMMs running with 64GB of memory. So it can be done! But that much memory is EXPENSIVE (4G DIMMs are just now getting under $1/MB)!
Linux ramdisk support is actually pretty slow. I did performance testing because I was investigating the possibility of loading the 'hot' portions of an FS into memory to speed things up.
The results were less than stellar on a system with 1.5GB of DDR400.
Any good to you? You can easily get up to 128Gb in a single drive, 3GB/s bandwidth, fits in a drive bay. Who cares what "RAM" the motherboard supports? Just a though, don't know if it would be any good for your environment.
The biggest board I know of is the Thunder K8QS Pro (S4882). With 16 DIMM slots it supports 32 GB of RAM. But it is a quad processor board.
Most of the rest of Tyan's Opteron server boards do support 16 GB. Again they are dual processor.
Tyan's boards will run with a single processor installed, but only 4 DIMM slots (1 bank) will be active. All processors can see all the RAM installed, but because the memory controller is integrated into the CPU, the CPU must first be installed. Then it can pass access to the RAM to the other CPUs. Local RAM always being the fastest. (Linux with Opteron NUMA support tries to keep memory associated with the task running on a specific CPU local to that processor.)
Is it possible for you to write software which can handle this application across several machines? 4 machines with 4 gigs of ram connected to each other via gigabit ethernet would probably cost less than 1 machine with 16 gigs.
If cheap is what you care about, you've gotta use scalable software. If you want to just buy something out of the box which can handle 16 gigs of ram, you can expect to pay more.
Insightful my ass. Some of us have real JOBS that require Windows, and saying "Sorry, I missed the deadline because of problems with Wine" are a fast way to get yourself fired.
He asked about how to fix it on Windows. How about I respond to every problem with Gnome and KDE or Linux with "Maybe you should switch to a supported platform like Windows"? We'll see how often that tripe gets modded "Insightful".
If you need a very fast database (even for random access), another option would be a solid-state disk. Of course, they aren't exactly cheap, either.
That's a NUMA opteron. You need to fill all the CPU sockets to access all that memory. Given the price of 8xx's, that's not very affordable.
You will likely be better served if you just use a solid state disk for this.
why not search for a hardware device that appears to be an IDE or SCSI drive but is actually a bank of DIMMs?
Can't speak for the OP, but possibly because IDE maxes out at 1.06 Gbps (Gigabits per second) for ATA-133 and 1.20 Gbps for SATA, whereas PC3200 DDR SDRAM can push 25.60 Gbps?
(all numbers above handily supplied by http://www.forret.com/tools/bandwidth.asp )
"Go to CNN [for a] spell-checked, fact-checked summary" -- CmdrTaco
First, I would recommend going with a server vendor. I honestly do not mean any offense by this, but if you are looking at places like Tom's Hardware for recommendations (a website which is frequently incompetent even at reviewing l33t g4m3r d00ds hardware, let alone server-grade hardware), you are probably not qualified to build a system which would actually need 16GB of RAM (e.g. a corporate server which must be relied upon). I do not know what the system will be used for, but if it is for a many-person organization, my recommendation stands, and if it is just for you and some friends, 16GB of RAM is almost always going to be absurd overkill.
Now that that's out of the way, Tyan has several dual Opteron boards which support 8 DIMMs. Look at their Thunder line, and put just one Opteron in them, and 2GB DIMM modules.
Computer Science is no more about computers than astronomy is about telescopes. --E. W. Dijkstra
Rather than saying, "I need this to solve my problem, and I'm not saying what my problem is!" it is better to say, "Here is my problem, I am thinking of doing this. How can I do that, or are there better solutions that I haven't thought of?" He'll get better answers that way.
If he doesn't know where to purchase a large memory system then it is possible that he doesn't really know if that is what he wants.
Lasers Controlled Games!
Very, very good advice. PC parts (especially very high end stuff like this guy is looking for) are usually at a premium on the consumer market anyway. With an IBM server you get support and a guarantee that everything you buy will work properly under Linux. With bleeding edge high-end stuff, this is not always the case, at least not immediately. Try pricing out an equivalent machine from commodity parts; and remember that for a server you're going to want ECC registered memory. Also remember that, if this is for a business, they are essentially paying you by the hour to build a PC. It's probably cheaper to pay an assembly worker to build a computer than a system admin (and if it's not, you should ask for a raise ;) Like the saying goes, it's only free if your time is worthless.
This goes for home-built systems as well. Often it's just cheaper to buy a system built in taiwan than to build your own out of the exact same parts. I'm all for geeking it out and building something to your own specs, but for gaming machines or just basic desktop apps a pre-built system saves a lot of headaches.
> Actually it's hard for me to think of an app which doesn't require more than one CPU and needs faster
:-)
> access than a properly configured RAID can give it.
was wondering this too, how about this unlikely scenario:
1. very low concurrency application - in which you never or almost never have more than one connection at a time to a database.
2. you need the fastest possible response to the query.
3. you've got 15 gbytes of data along with 500 mbytes of indexes
4. your queries don't fit into any pattern - so normal database buffering seldom gives you above a 50% hit rate.
5. all access to data is via indexes, and is never for more than a few rows: so parallel queries won't help.
6. availability doesn't matter - you'll just get two of these, and they are read-only.
In this scenario I imagine that a ramdisk would provide a tiny performance margin over a properly designed disk-based database.
But the other 99 out of 100 scenarios would do better on disk.
I'm curious how a device like this would get 3GB/s bandwidth - what kind of bus is it using? It's certainly way past PCI. Perhaps PCI-X, or plugging into AGP or something?
Another possibility, if cost is more of a problem, and bus speed is less of a problem, is to network a couple of motherboards together, with as much RAM as possible on each of them, and either GigE or Firewire.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
> - but for gaming machines or just basic desktop apps a pre-built system saves a lot of headaches. I've got to WHOLEHEARTEDLY disagree here. If I were to buy a pre-assembled gaming rig, it would cost about 100% more than if I were to build it myself - even if it had the exact same components in it. Don't believe me? Just go look at alienware or something equivilant. Now for a desktop only machine for granny, an emachine is fine and dandy, but you'd never catch me buying a pre-built computer.
By my opinion the idea of wasting a precious, fast and costly memory for just a ramdisk of that hideous size is a product of pure lunacy and clearly indicates there is something extremely wrong with your software. For the price of the memory itself, you can certainly design and built a more adequate hardware infrastructure for your task if you use some brain first. But wait, you are not a Longhorn core developer, aren't you?
There you are, staring at me again.
Far out man, your post is just a blast from the past!
If you replace every GB with MB, it'd be soooo 1992 again. Well, except for the opteron references.... but you know what I mean, dammit.
You are in a twisty maze of processor lines, all alike.
There is a lot of hype here.