Slashdot Mirror
Best Motherboard for a Large Memory System?
kimanaw writes "Due to a particular infrastructure need, and increasing OS support for 64 bitness, I'm looking into building a large memory server box (at least 16 gigs, possibly up to 64 gigs, probably config'ed into a big ramdisk). I only need a single CPU, and just minimal disk; most prebuilt systems w/ large memory seem to focus on more CPus and big RAID, all of which (over)inflate the pricetag. I've searched several websites (including Tom's Hardware), and I've googled, but can't seem to locate any commercially available AMD MBs supporting more than 4 sticks of RAM, or 4 gigs. Have any Slashdotters built a big-RAM server? Any pointers, hints, and tips much appreciated."
http://ask.slashdot.org/article.pl?sid=05/02/22/09 30238&tid=198&tid=126&tid=4&tid=137
IBM xSeries 445. We have around 50 of these IBM servers at work and they support up to 64 GB of RAM. Any reason why you are trying to build a server out of cheap commodity parts? Save yourself the trouble and buy one of these.
Get a Sun. You can get very large memory systems. You can go up to 192 gigs of memory. Marc
Tyan Thunder K8QS Pro Quad Socket 940 Motherboard AMD Solution Motherboard - Operton Motherboard: Outpost #: 4311135 http://shop2.outpost.com/product/4311135/
-Chipset: AMD-8131
-Processor Support: Quad Socket 940 for AMD Opteron 800 Series Processors
-Memory Support: Up to 32GB Registered PC3200 DDR
-Expansion Slot: Two 64-bit 66/33 MHz PCI-X, two 64-bit 133/100/66/33 MHz PCI-X, One 32-bit PCI Slots
-Other Features: GigaLAN, Integrated Video, SCSI
if you have the money to fork over, pick one of these up at your local Fry's Electronics or try fry's online at http://www.outpost.com/
I've always read it's inadvisable even if one has a lot of RAM. Here's some quotes from this page:
Can the Virtual Memory be turned off on a really large machine?
Strictly speaking Virtual Memory is always in operation and cannot be "turned off." What is meant by such wording is "set the system to use no page file space at all."
Doing this would waste a lot of the RAM. The reason is that when programs ask for an allocation of Virtual memory space, they may ask for a great deal more than they ever actually bring into use -- the total may easily run to hundreds of megabytes. These addresses have to be assigned to somewhere by the system. If there is a page file available, the system can assign them to it -- if there is not, they have to be assigned to RAM, locking it out from any actual use.
--
Why is there so little Free RAM?
Windows will always try to find some use for all of RAM -- even a trivial one. If nothing else it will retain code of programs in RAM after they exit, in case they are needed again. Anything left over will be used to cache further files -- just in case they are needed. But these uses will be dropped instantly should some other use come along. Thus there should rarely be any significant amount of RAM 'free'. That term is a misnomer -- it ought to be 'RAM for which Windows can currently find no possible use'. The adage is: 'Free RAM is wasted RAM'. Programs that purport to 'manage' or 'free up' RAM are pandering to a delusion that only such 'Free' RAM is available for fresh uses. That is not true, and these programs often result in reduced performance and may result in run-away growth of the page file.
I'm on a road shaped like a figure eight; I'm going nowhere but I'm guaranteed to be late.
The only way to get anywhere close to 64GB is by going multi processor. I have 2 Thunder K8S Pro (S2882) motherboard with 16GB of RAM each, running Postgres 8 on FreeBSD 5.3 in production. These systems are dual opterons. They have 2GB sticks. I also have a test system with a Tyan Thunder K8QS Pro (S4882), this one is a quad opteron and supports upto 32GB with 2GB sticks. The only way I know to get 64GB with 2GB sticks is with an IWILL H8501, that's an 8-way opteron barebone system.
I think you can get 4GB stiks now, but they are very expensive still.
As to the purpose of the box, I can't say too much, except its intended as a sortof main-memory database cache solution (ala TimesTen ) to serve a potentially very large user base, with a lot of cached images associated with the cached data.
I also considered a piecemeal commodity approach, e.g., filling racks with cheap 4 gig RAM 1U's running Linux, but then I'd have to come up with s/w that can coordinate those systems (basically, a head-end to hash the requests to the right 1U). memcached looks intriguing, but I'm still concerned about interconnect latencies.
Based on my cocktail napkin estimates, the h/w cost of a big RAM system (if it can be got) would probably be about 2x the cost of the piecemeal system, but hopefully signifcantly reduce interconnect latency, and use simpler s/w (assuming I can get Linux to config a ramdisk that big).
Since latency is the prime concern, solidstate disks using the usual HD interconnects just didn't seem an acceptable solution, esp since the cost reduction doesn't seem that significant.
As for failure recovery, my thoughts were just a couple fast HDs that would ping-pong taking snapshots of the ramdisk; hopefully, the ramdisk recovery would never lose more than an hour's data (the lost data wouldn't be critical, and would be recoverable from other sources...its really the delivery speed thats essential)
But I may need to go back to my napkin and do some more queueing analysis to see if there are better commodity solutions.
007: "Who are you?"
Pussy: "My name is Pussy Galore."
007: "I must be dreaming..."
Windows XP doesn't swap, it uses a page file, which preemptively makes copies of the least used pages in your ram to your disk so that if there is an unexpected jump in memory usage, it can wipe those pages from memory quickly and just put them back from disk when they are needed. There really isn't a performance hit since it's a pretty much passive system in reference to RAM writes. Adrian's Rojak Pot has a great guide on Windows virtual memory at http://www.rojakpot.com/default.aspx?location=3&va r1=143&var2=0
-Julius