Slashdot Mirror

https://www.supermicro.com/Apl...

AMD 1 CPU
all flash!
NO PCI-E SWITCHES OVER HEAD NEEDED

2920X = $649 i9-9900K = $580

Frankly you'd be nuts to go for the intel chip when you can get 50% more cores AND an upgrade path to 64 cores for around the same price.

Intel Atom® Processor C3958, 16-core, 16-thread (no hyper-threading) low power (full load 47W), fan-less design, with 16MB cache, up to 2.0GHz, can access up to 256GB of RAM, and can be used as a edge-computing device or a dedicated web server.

https://ark.intel.com/products...

Server motherboards from Supermicro, TYAN, or Gigabyte, with CPU attached, for less than $800.

https://b2b.gigabyte.com/Serve...
https://www.tyan.com/Motherboa...
https://www.supermicro.com/pro...

When can AMD offer us something similar?

Just a quick Google search:

https://www.supermicro.com/Apl...
https://www.tyan.com/EN/campai...

Several models mention being able to use 2 TB of RAM, with dual socket ones being able to reach 4 TB.

Where is the evidence? They keep saying they have it. Why don't they show it?

Is somebody stopping you from buying one of their products: https://www.supermicro.com/pro... ...and looking for these backdoors yourself?

My thoughts exactly. Supermicro boards are normally used in servers, so their customers have certain expectations. Here is one list of Spectre patches for a bunch of Supermicro motherboards:

https://www.supermicro.com/sup...

If anyone missed the news, just recently it was discovered that the Chinese manufacturer added a very suspicious chip to a small number of Supermicro boards. That's obviously very bad news.

Don't look any further, Supermicro H11DSi-NT. Not Ryzen but Epyc thoug.

Someone pretending to be you a few days ago for your lawsuit story.. Said they had a family computer and looked up porn on it.

I saw that photo of eight blazing Core i9 CPUs, bathing in heat-sink grease and wet with cooling water, nestled in a motherboard carrying an entire 256 gigabytes of RAM. I'm still panting today!

Heh - nice :)

You think that's cute, check out a real computer: the 2123BT-HNR from SuperMicro. Yeah, baby! In 2U, you can get 8x 32-core CPUs, 8TB RAM, and 24x NVMe SSDs. Plus some M.2 boot drives and multi-gigabit connectivity.

You're totally right; I misread the table -- the G34 socket and Opteron 6300 is a Piledriver chip, not Ryzen.

AMD Epyc the new brand name that replaces AMD's Opteron server line (can I say I hate rebranding?)

It's the Epyc 7551 is ZEN architecture, uses the SP3 LGA socket, and has 32 cores/64 threads.

SuperMicro appears to have some in the pipeline, and even has a dual-socket model, with IPMI 2.0.

128 threads in a single system ain't too shabby...

Um they have 3 of them https://www.supermicro.com/Apl... for example. You can get them in a 1ru supermicro chassis for about 1500 for a functional computer.

Most manufacturers now make barebones servers specifically designed to cram in GPUs. Amazon AWS, Google Cloud and Microsoft Azure all offer virtual servers with multiple dedicated GPU's as well. Yes, your run of the mill server is still headless with an ASpeed IPMI but you can get absolutely crazy with GPU server platforms.

While Zen Server parts (Epyc) look good on paper, it reamis to be seen if there will be Adoption from server makers

https://www.supermicro.com/pro...

But one thing is to hope, and quite another thing is reality

Hence the provided link via a two word google search that describes some reality.

https://www.supermicro.com/pro...
https://blog.dellemc.com/en-us...
https://www.forbes.com/sites/p...

They sure have AMD now (or coming shortly).

Not really. Xeon with 24 cores, and you can put 8 of them into a single server.

And that's not even including the Xeon Phi 7290 that can be slapped in using PCI-e, adding 72 more cores.

264 cores per server would get Oracle / Microsoft frothing at the mouth for per-core licensing. 18? Not so much.

Fun fact: Quad sockets are SUPER-RARE.

Go ahead. Try and Google for a quad motherboard. They haven't made them since like... the Athlon 64-era Opterons.

Really? You should tell all the people that are making quad socket servers today.

• Quad socket Xeon E5 system
• Quad socket Xeon E7 system

Maybe you should try following your own advice.

Aye, dual Ethernet is the feature that seems to be missing from them. There are of course workarounds involving USB3 dongles but I'm not sure if any of the common ones using ASIX chips have support in something like pfSense, although apparently there is a driver for ESXi that will support them as NUCs have been popular lab machines for VMware setups.

Even if the dongle solution works in a given use case, it's still a cursed dongle and not the clean built-in port solution.

In addition to the second Ethernet port, I'd like to see rack mount ears made for it with space to accommodate the PSU brick and any dongles you'd consider adding.

Really though, what you end up wanting kind of already exists -- https://www.supermicro.com/pro...

I'd call that the rack mount NUC on steroids, although the price point is higher than a NUC.

You haven't given any technical explanation of why RealTek hardware isn't worth the effort, and on this page it shows support in BSD for half a dozen model numbers from RealTek. You must be talking about a different model? There could be many reasons the developers haven't spent time on it other than the quality of the design. For example, a simple lack of people.

You could explain, for example, how the DMA engine in the card is broken or that it drops interrupts (if either of those is the case). Just saying it's crap isn't very credible without technical background.

My impression was that nVidia provided a proprietary driver, thus not long-term supportable once they lose interest.

I have a number of Supermicro boards. Actually their driver and firmware support is not great, and I have complained about their boards to them, only to get the answer that they work correctly on Windows. I buy them because they have remote management, but have given up on their remote management implementations and will go with an external solution next time. My latest Supermicro board is a C7X99-OCE-F and as you can see here, they don't claim BSD support at all. On Linux they have some IOMMU issues at the moment.

http://www.supermicro.com/prod...

http://www.supermicro.com/prod...

http://www.supermicro.com/prod...

http://www.supermicro.com/prod...

If you can afford this "enterprise" SSD, you can certainly afford a Xeon or Haswell-E and LGA2011 motherboard with 40 PCIe lanes.

Yeah. The nice thing about x16 cards is that you can probably reuse graphics-designed systems. Like this one, four x16 slots in a 1U chassis, 2-way system so you have 80 lanes total:

http://www.supermicro.com/prod...

Drop in four of those cards and you'll have a pretty decent database server, I imagine...

The Backblaze implementation of top-loading drives is one well-known example. They've 45 drives in 3U (or 4U?) for many years.
https://www.backblaze.com/blog...

Nowadays you can order a 90-bay top loader off the shelf from Super Micro:
http://www.supermicro.com/prod...

I have a pfSense router built on a C2758 Atom CPU (specifically this board: http://www.supermicro.com/prod...) paired to a couple of Unifi APs (http://www.amazon.com/Ubiquiti-Networks-Enterprise-Unifi-UAP/dp/B00HXT8R2O). Its the best home network I have ever had. And that is including some DD-WRT stuff that I used to use for wifi in conjunction with some actual Cisco gear that I used to use. (ASA 5505 firewall, 3745 router, ect...) I can't see myself ever going back to a consumer grade wifi router. Sure its total overkill. But being able to set up a Site to Site VPN to my friend's place and an OpenVPN server for remote access without having to worry about CPU usage is pretty nice. Being able to have separate SSIDs and corresponding VLANs for guests and my kids and such is also nice. The Unifi APs give better wifi than any consumer grade device I have ever used. I am seriously considering upgrading to dual band AC models.

2758. Same TDP slightly better performance.

http://www.supermicro.com/prod...

I've used supermicro in the past. Very cost effective if you like to assemble your own systems.

I have no financial relationship. This is just an unsolicited testimonial.

http://www.supermicro.com/prod...

I've used SM in the past too. They are shit. - Slashdot. fair and balanced.

I've used supermicro in the past. Very cost effective if you like to assemble your own systems. I have no financial relationship. This is just an unsolicited testimonial. http://www.supermicro.com/prod...

There are no recent 8-scoket Xeon system benchmarks to compare to!

http://www.supermicro.com/prod...

Besides, there are these little things called "extrapolation" and "long division" that will let you make an actual comparison.

If all you need is a low end 2-socket box, then probably the Xeon is the better choice, but once scalability to higher end performance, throughput, etc is required, thatâ(TM)s where you won't find Xeon systems competing against SPARC.

I repeat that this misses the point of my original comment. I wasn't comparing the performance of an Intel computer to that of a SPARC computer. I was referring to the complexity and capability of individual cores. We are talking about two very different things.

By default, SuperMicro IPMI attaches to normal ethernet.

Yes, I saw a mention of that on G+ today, but I lost it. So I went to the source, I will save y'all the trouble of dicking with the PDF and jump straight to page 2-26 and excerpt the really interesting part:

The default setting is Failover, which will allow IPMI to be connected from either the shared LAN port (LAN 1/0) or the dedicated IPMI LAN port. Precedence is given to the Dedicated LAN port over the shared LAN port.

YE GODS. At least it's in the manual, which no one reads. You can select a port once you've got the system up and running, and once you do that it will stick, but until then it operates unsafely, as above. And if by chance there's no link on the management port during boot, perhaps because the management switch is also being cycled, then IPMI will appear on another interface.

There's no excuse for not firewalling that off, but it's still also unacceptable behavior.

Oh sorry, forgot to say, yes, it's easy to find all IPMI devices on your network. Please take a look at: ftp://ftp.supermicro.com/utili... - you can download the IPMIView tool from there, which will find all IPMI devices on your LAN. The default password and username for all Supermicro IPMI is ADMIN and ADMIN, so, of course, super secure.

What about Supermicro?
That setup sticks 4 dual-socket opteron nodes in 2U with an Infiniband interconnect.

By mass, the predominant component of a server is the steel enclosure. The electronics are pretty light except for heat sinks, which are aluminum or copper. Chips are predominantly ceramic or plastic, the actual circuits being negligibly small. The circuit boards are fiber and resin laminate layers with very thin copper layers. If you pull all the electronics, excluding the drives and power supply, out of a well populated 55 lbs 2U server the guts might weight 3-4 lbs, I guess. Not much. You may attribute 75% or so of the mass to steel. 21 of those in a 42U rack is 1155 lbs of mostly steel.

Supermicro 1u 64 cores. Bunch of other Mobos (some more than 1u) on this page. Cheap is relative to the buyer I suppose, but to my (admittedly very large) company these things are rather cheap unless you start stacking them with lots of dense memory.

Supermicro 1u 64 cores. Bunch of other Mobos (some more than 1u) on this page. Cheap is relative to the buyer I suppose, but to my (admittedly very large) company these things are rather cheap unless you start stacking them with lots of dense memory.

For Intel chips, the only mainstream option is the Intel S1200KP. Supermicro has a few ITX boards that support ECC but they are horribly expensive as they come with built-in Mobile CPUs. Supermicro also has a new Atom board that supports ECC, but it only goes up to 8GB. (what's the point?)

For Intel chips, the only mainstream option is the Intel S1200KP. Supermicro has a few ITX boards that support ECC but they are horribly expensive as they come with built-in Mobile CPUs. Supermicro also has a new Atom board that supports ECC, but it only goes up to 8GB. (what's the point?)

My last Asus motherboard install (about four months ago) did not support Linux due to the NIC reporting as the wrong model,

So return the MB and get a refund. I wouldn't have spent more than 30 minutes on it, after seeing the specs were wrong. I've built about 5 dozen computers.. I have NEVER had that happen. So just return it.. it's not a widespread issue.

None of the currently available motherboards make the NIC model available before purchase

Here, I don't know what you are talking about. Granted it's been almost a week since I built my last computer, but the manufacturers websites haven't changed that much.

http://www.asus.com/
http://us.msi.com/
http://www.biostar-usa.com/app/en-us/index.php
http://www.supermicro.com/index_home.cfm

Yep.. all of them list the NIC card in the MB specs.

If you're that lazy, why the fuck are you bothering me.

Supermicro is crazy. They'll cram 4 nodes in 2U with an Infiniband interconnect.

11 i3's, 2 i5's and 11 i7's
i3 CPU's are used on low power servers, hence the ECC. Google found Super Micro make a bunch of ITX sized server motherboards with the i3/i7 ECC CPU's
The Atom N2800 supports 4GB

8-way would mean 8 sockets on the motherboard, like http://www.supermicro.com/products/system/5U/5086/SYS-5086B-TRF.cfm .

I'm only really familiar with SuperMicro products, but they offer a pretty standard warranty for their servers. Since they use pretty standard components, rather than vendor-specific stuff or firmware-locked drives (see my other post), spare parts are pretty easy to come by. They had all the standard features like IPMI ("Lights Out"), redundant power supplies, etc.

RMAing broken hard disks to Sun was an exercise in frustration and delays. It literally took weeks to get a hard disk replaced under warranty.

Dell premium support (whatever they call it) for their Optiplex systems was great, but we didn't use Dell servers because they were too expensive. Only downside: their desktops used some Dell-specific variant on the ATX power supply plugs: if we had issues on an out-of-warranty system we'd have to buy a new power supply at an inflated price. It made testing potentially-broken systems considerably difficult.

I've used Supermicro equipment for years. Their 1U Atom based systems work great for firewalls, routers, or any other kind of Linux network device. Low power, mostly fanless (power supply has a fan), expansion slots, decently priced. You can go up the line to full blown Xeon based systems with all the redundancy you need.

Their support is good also. You get to talk to knowledgeable people who speak English.

Supermicro website

I cannot find an actual server (with redundant power supplies, hotswap drive bays, monitoring hardware) that has a lot of drive bays, but less than 4 CPUs, because the $CPU ones are extremely expensive and I don't need 4 CPUs anyway. The only affordable ones have 4-6 bays max.

I bought this 4U system. If memory serves me right (too lazy to check), it accomodates two power supplies. I opted for just one. The chassis, I think, is this Supermicro chassis. It has 8 hot swap drive bays (plus 2 peripheral).

Is that what you're looking for?

useing software raid is ok most boards have about 6 ports so if you want like 10 then maybe a x4 or better pci-e card may be needed.

Or, get an actual server board (this is gonna be a server, right?), like this one. That's six SATA ports and 8 SAS ports. If you flash the SAS ROM to the "no-RAID" version, the controller is recognized natively by Linux. In addition, you get lots of PCIe connectivity, a pair of Gigabit Ethernet ports, and IPMI (allowing remote power cycle).

Then, find a full-tower case with lots of 5-1/4" drive bays, and add hot swap bays. There are smaller versions, as well...just budget what you need for drives.

I use the motherboard I referenced along with an add-on 8-port SATA card (anything supported by Linux would be fine) and two of the drive bays for ten 2TB drives in RAID-10. I boot Fedora off a pair of SSDs in RAID-1 and also have four 2-1/2" 750GB drives in RAID-10. The 10TB array serves iSCSI over 10Gbit Ethernet to ESX systems that hold all my VMs, with the 1.5TB array as local and NFS storage. There's still PCIe slots available if you need more controller cards.

With this setup, the VMs are how everything is accessed, so you can pick whatever OS you want to face client machines.

    Well, it sounds like you have and will continue to spend an unhealthy amount of time collecting. It may not be stacks of newspapers waiting to crush you, but it's a obsessive behavior, and you should seek help. Hell, we could be wrong, but none of us are qualified to say that.

    But since you're asking for a review of your methodology, lets have a look.

    You have 64 drives in 4 machines. That's 16 drives per machine. What are the machines that you're using? 16 drives each is an awful lot. Like, my 4u raid chassis held 15 drives each. I have seen some machines like this that hold 16 drives or more in the same chassis as the motherboard. It's only about $1,000 for just the chassis. A bit pricy for a hobby.

    You say you have 4 machines with 12 TB each, and 16 drives each. That's 16 750GB drives, and I'd guess you spent over $6,000 for them. Probably a lot more, as you have already collected 8TB of porn, and vast quantities of other things.

    You say you spent $1,900 on the tape drive. That would be about 30 tapes (LTO4 800/1600), at about $30/ea, so another $900 on tapes. But you say you want another 100 tapes, so you can have one more generation of backups?

    You mentioned using rsync between the machines. rsync is great, but as I discovered with huge filesystems is that the memory consumption is likewise huge. That, and the fingerprinting and comparison of the files would take roughly ... forever and a day. Been there, done that, suffered the pain of it all. So you'd need a robust ordering system, and do them in pieces. That's how we did one of our huge sites. That's usually one of those things that people mention when talking about the wonders of a system they put together.

    I'd even expect you to talk about the extra power consumption and cooling requirements. I would strongly suspect that kind of load would require at least two power circuits, but probably 3 15A to 20A circuits. I would assume you did this at home, and not in a commercial building. Most residential rooms have one, maybe two, circuits. This is usually due to lazy contractors only wanting to run one cable for adjoining walls. You didn't even mention the power supplies per machine, much less these larger issues.

    And what about your network. Syncing terabytes of stuff between multiple machines is rather bandwidth hungry. Well, unless you expect it to be done this year. You wouldn't be using a regular consumer linksys/belkin/netgear switch. You'd want GigE ports on a real managed switch. That has a a pretty healthy price tag attached.

    If you had such a huge investment in large storage, I'd believe you'd have a home theater system to match. You wouldn't be watching that porn on a 15" CRT.

    Really, the 8TB of porn question that I raised, although questioning why you'd possibly want that, was really a question of if you had really done it.

    I'd say that you're thinking and dreaming of having such a system. I seriously doubt that you've spent over $13,000 for your home porn store. ($12,800 outlined above, not including motherboard, processor, memory, cables, network gear, portable air conditioner, electrical work, etc).

    This is the point that you'll argue, insist, call me a liar, probably with many profanities either written or implied.

    I could be wrong, but that would lead us down the road towards unhealthy obsessions.

European Branch

Super Micro Computer, B.V.
Het Sterrenbeeld 28, 5215 ML,
's-Hertogenbosch, The Netherlands
Tel: +31-73-640-0390
Fax: +31-73-641-6525
General Info: Sales@Supermicro.nl
Tech Support: Support@Supermicro.nl
Supermicro uses Xeons in all their current workstations. You'd have to ask for a custom job to get Opterons in a pedestal build. Nothing says you can't just run http://www.supermicro.com/Aplus/system/2U/2042/AS-2042G-6RF.cfm on its side, except it's going to sound awful in an office. If you're curious how one of those could/would spec out filling all bays and memory slots (without converting currencies),

1 x SUPERMICRO AS-2042G-6RF 2U Rackmount Server Barebone Quad Socket G34 AMD SR5690/SR5670 DDR3 1333/1066/800 Item #: N82E16816101321 $1,899.99
6 x Western Digital RE4 WD5003ABYX 500GB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive Item #: N82E16822136697 $449.94 ($74.99 each)
32 x Kingston 8GB 240-Pin DDR3 SDRAM DDR3 1333 ECC Registered Server Memory Model KVR1333D3D4R9S/8G Item #: N82E16820139140 $3,103.68 ($96.99 each)
4 x AMD Opteron 6128 Magny-Cours 2.0GHz Socket G34 115W 8-Core Server Processor OS6128WKT8EGOWOF Item #: N82E16819105266 $999.96 ($249.99 each)
Subtotal: $6,453.57 which is about the total budget.
The biggest problem with Opteron 6100's is the next faster proc costs 2x as much. I'm not suggesting this exact config, just an example. And YMWV with exchange rates and the hike to costs when importing stuff to your lil island. The UK gets hosed on hardware prices.

I tried these guys (just noticed they switched from com to nl, interesting, wtf?

They are not as refined as the big ones, but they provide bang for the buck.

Even the 750k MTBF feels bogus. In my experience actual failure rate is just a hair under 3%, which works out to about 300k MTBF. Maybe they're quoting the MTBF of a drive still in its anti-static bag, sitting in the spares drawer :)

Try shoving 36 3TB drives in one of these: http://www.supermicro.com/products/chassis/4U/847/SC847E26-R1400LP.cfm and you'll appreciate MTBF in a whole new light. My approach is simple: I take the number of drives, times 3% annual failure, times the number of years I want to keep that box in operation. The result is how many spares I'll need. For the 36-drive box, I use 32 drives for the RAID + hot spares, and leave the remaining 4 as spun-down spares. That ensures at least 5 years of hassle-free operation. Add JBODs to the mix and you get an even smoother curve, as a 100-drive array is more deterministic in its failures due to the larger sample size.

For a single drive, well, failure rate becomes a lottery. MTBF won't help you here. If you buy a 3TB drive for your desktop, you should also get a 3TB external drive to back it up, and sync them regularly. Always assume one of them will die at the time you need it the most, and be prepared to deal with it gracefully. The odds of both failing at the same time are pretty slim, certainly slim enough for most residential users.

As I mentioned, we resell Supermicro's. Those look like (but could be anything really), this model: http://www.supermicro.com/products/system/1U/6016/SYS-6016GT-TF.cfm?GPU=FM207 (or one of it's variants). Nearly all of them ship with Intel Xeon, although they do offer an AMD based range but it's not as popular (at least where I am).

Sure, 'value added' is assembly.

Example: I bought servers from a US 'manufacturer', Supermicro, who are getting every single component from other countries and then they are putting them together and reselling. Now, I did that because it made sense at the time to buy those servers from US, I was in Canada and brought the servers through the border myself, figure out why.... And people still buy from Supermicro, but the point is that they didn't manufacture any single component themselves, every single thing in those servers is made in Asia.

Same with machinery, same with cars, most tools, furniture even (I bought furniture from US as well and leather was made in Italy, frame pieces were made somewhere in South America I think and the table with leather top and the bookcase were assembled in USA, it was expensive).

Sure USA still makes stuff, but trade deficit is on the slope, less you didn't notice. That trade deficit is constantly going up. The slope over 20 year period shows the direction and grade. There is no question, the trend is to get rid of manufacturing from USA completely.

You can have 48 cores today with a Quad G34 motherboard.

They should have talked to SuperMicro.
That's just over 8 enclosures (4 nodes/enclosure) and fits in 18U.

It looks like this.

For just over a buck, it's not a bad deal.

Since it's actually $8 when you include shipping, it's not so good a deal after all. Even buying 10 only lowers the per-unit total cost to $3.

Chances are you can hit a local shop and get them for close to the same price.

I've looked, and they are about $5 in local stores. I purchased a case of 24 from Best Buy for Business for less than $30 including shipping, but that was about 3 years ago.

I use this case for my smaller servers, which allows me to have 6 hard drives by using three of the 5-1/4" bays with adapters. With this motherboard, you can put an awful lot of computing power into a small space, and it's much quieter than rack mount solutions.