Slashdot Mirror


Fitting A Linux Box On A PCI Card

An Anonymous Coward writes: "Running on Newsforge/Linux.com is a hardware review where Slashdot's Krow took a couple of OmniCluster's Slotservers and and built a cluster configuation inside of a singe host computer (and even had DB2 running on one of the card's inside of the host). Could something like this be the future of computing where for additional processing power you just kept adding additional computers inside of a host?"

41 of 137 comments (clear)

  1. Ob Beowulf comment by anticypher · · Score: 2

    Imagine...

    It would be cool to have completely separate processors in a box, so that as long as there is power, each card can run on its own. Then you could network them together into a beowulf cluster, and then make clusters of clusters

    the AC

    --
    Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
    1. Re:Ob Beowulf comment by Khalid · · Score: 3, Interesting

      There has been a project a while ago, which aim was to implement a beowulf of separate StrongArm cards to be plugged in a box, they have even managed to build some prototypes.

      http://www.dnaco.net/~kragen/sa-beowulf/

      Alas I think the project seems to be dead for some time now.

  2. Seen these for a long time by ackthpt · · Score: 3, Insightful

    I've seen these around for ages, variety of manufacturers, but usually they're priced significantly higher than just buying several cheap PC's, granted you have a fast bus between cards/PC's, unless you have a redundant powersupply, one failure brings your whole cluster down, whereas networked mobos should be tolerant of one system failing. As for future, eh, they've been around long enough, but I expect the use has been rather specialized.

    --

    A feeling of having made the same mistake before: Deja Foobar
  3. The SETI version by Wire+Tap · · Score: 3, Insightful

    Does anyone here remember a while back when that "fake" company tried to sell us SETI @ Home PCI cards? I was about to place my order, until the word came to me that they were a fraud. Kind of a funny joke at the time, though. At any rate, here is the old /. story on it:

    http://slashdot.org/article.pl?sid=00/07/23/2158 22 6&mode=thread

    It would have been GREAT to have an improvement in CPU speed on a PCI card, as I always have at least two free in every system I own. What I wonder, though, is what instructional speed would the PCI card "CPUs" give us?

    --

    Man is born free; and everywhere he is in chains.

  4. Impractical by atrowe · · Score: 2, Troll
    I don't see these things taking off for most uses because the PCI bus is limited to a measly 133 MB/S. Even newer 64 bit PCI slots found in some servers have insignifigant bandwidth to keep the data flowing fast enough to make full use of these things. I can see where they may come in handy for heavy number crunching applications such as SETI, but for web serving and DB applications, the throughput between the card and the host system is simply unacceptable.

    Also, I would imagine that the RF interference generated by having several of these in one box would be quite signifigant. PCI slots are only an inch or so apart on most motherboards, and without any sort of RF shielding between multiple cards, I can't imagine they'd function properly. It's a good idea on paper, but in reality, I'd think a few 1U rackmount servers would do the job much better. And for $499 a piece, you could get a decent single processor rackmount server for around the same price.

    --

    -atrowe: Card-carrying Mensa member. I have no toleranse for stupidity.

    1. Re:Impractical by man_ls · · Score: 2

      The way I understood it, the host system's motherboard was just a backplane supplying power to the computer, which was contained on the PCI card. IIRC, several years back, when Pentium IIs came out, lots of people wanted a way to upgrade their Pentium-I based systems. The easy answer was - make the motherboard into a holder for a very compact computer. It had, I think, a 333 Celeron, an SODIMM slot for memory, a single IDE channel and floppy controller, and onboard sound and video. Not too impressive, but the entire workings of the computer onto a single PCI card.

      Sun or SGI also has something like this, to allow SparcStation users to run Windows applications natively. Basically, a card with a 450MHz Pentium II, some RAM, video (no sound though), and the other necessities of a computer.

      I agree about the RF interference, however. I ran several computers, even in their shielded cases, in my room for a while, and it was a deadzone for our cordless phone. It would be only worse with inches, instead of feet, between the systems. Not all people have room for a rack to mount things on, however.

    2. Re:Impractical by Peter+Dyck · · Score: 2
      the throughput between the card and the host system is simply unacceptable.

      I'd suspect a radar system requires much high a throughput than web or DB serving. Here's an example of such a system. "160Mb/sec, 32 bit parallel synchronous interface" doesn't sound that high to me.

    3. Re:Impractical by morcheeba · · Score: 3, Interesting

      RF Interference:
      I don't think there will be a problem with interference. Check out these computers. They use a similar system, but instead of being on a pidly motherboard, they use the ubiquitous VME format. They really pack in the processors -- 4 G4 PPC's per daughter card, and 4 daughter cards per single 9U VME card, and then 16 9U cards per chassis, and then three chassis. (4*4*16*3=48 TFLOPS) The pitch spacing on PCI is comprable to that on VME.

      Also, I wondered about the connector on the tops of these boards. It looks like another PCI card edge. I wonder if this is a duplicate of the host PCI interface (for debug purposes), if it's a new "slot" to connect to the server's internal bus, or if it's a way to connect server cards bypassing the main PCI bus (for better performance).

    4. Re:Impractical by Knobby · · Score: 4, Informative

      don't see these things taking off for most uses because the PCI bus is limited to a measly 133 MB/S. Even newer 64 bit PCI slots found in some servers have insignifigant bandwidth to keep the data flowing fast enough to make full use of these things.

      You've heard of Beowolf clusters, right?

      Let's imagine I'm running some large routine to model some physical phenomena.. Depending on the problem, it is often possible to split the computational domain into small chunks and then pass only the elements along the interfaces between nodes.. So, how does that impact this discussion? Well, let's assume I can break up an NxM grid onto four subdomains. The communication from each node will consist of N+M elements (not NxM).. Now, let's take a look at our options. I can either purchase 4 machines with gigabit (~1000Mb/s) ethernet, Myranet (~200Mb/s) cards, or maybe I can use ip-over-firewire (~400Mb/s) to communicate between machines.. Gigabit ethernet has some latency problems that are answered by Myranet, but if we just look at the bandwidth issue, then ~1000Mb/s is roughly 125MB/s. That's slower than the 133MB/s you quoted above for a 32bit, 33MHz PCI bus.. Of course there are motherboards out there that support 64bit, 66MHz PCI cards (such as these from TotalImpact)..

      You're right that the PCI bus is not as fast as the data io approaches use by IBM, Sun, SGI, etc to feed their processors. BUT, if I'm deciding between one machine sitting in the corner crunching numbers, or 4 machines sitting in the corner talking slowly to each other through an expensive gigabit ethernet switch, guess which system I'm going to look at?

    5. Re:Impractical by Charles+Farley · · Score: 2

      Please tell me you're not trying to argue that ethernet is faster than PCI. If that's true, your PCI bus would plug into your ethernet card and not the other way about. Do the math. 100/8 is a whole lot smaller than 133.

    6. Re:Impractical by Shanep · · Score: 2, Informative

      Moderators need to mod Charles UP and the AC down.

      That's why they use ethernet for communications and just use the PCI bus for the power supply.

      There is nothing informative about this! I was supporting products like these from Cubix back in '94. Back then, those products were even using the ISA and EISA buses to carry ethernet between other Cubix cards on those buses.

      The PCI bus is just an outdated fancy parallel port.

      ROFL. ISA can carry data at 8MByte/S (8bit 8MHz = 64Mbit/S) to 32MByte/S (16bit 16MHz = 256Mbit/S) which provided a far better solution in these setups than a 10base-* NIC that was going to be plugged into ISA or EISA *anyway*!

      And PCI can carry data at 133MBytes/S (32bit 33.333MHz = 1Gbit/S)

      These cards are usually slaves to the host motherboard, not the other way around. This way they're easier to make and the assumption can be that whatever they're plugged into will be the master of the PCI bus, so no need to fiddle with master/slave config. For usage with a dumb PCI backplane, PCI master cards (one per backplane please) can also be purchased. Though I have'nt looked at this companies offerings.

      A Linux machine set up as a web server, accelerated with khttpd and one of these cards running FreeBSD serving the db would be an awesome setup. Nice and real fast. Especially with a server mobo with multiple PCI buses (buses != slots) to seperate the 100/1000Mb NIC interfaces and the db card.

      --
      War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
    7. Re:Impractical by ryanr · · Score: 2

      You know that that's megaBYTES per second, yes? Or just over a gigbit per second? If that's not fast enough for you, what is? Pretty much any solution to connect something external to the box is going to have to go through the same bottleneck. Really, the only faster buses you have on a PC are the RAM sockets, and the AGP socket. I seem to recall a special high-speed networking solution that goes through AGP, but we're talking a little bit different class of hardware.

    8. Re:Impractical by crisco · · Score: 3
      Hmm, methinks this is a very good troll.

      Aside from the nick and the sig, calling 133MB/s 'measly' is absurd. Sure, compared to servers that cost an order of magnitude higher than these do, it is a little slow. But comparted to 100 Mb Ethernet, it is pretty fast. For specific applications it is definately useable.

      And RF problems? How about24 CPUs in a 3u package, using a similar concept?

      But for a troll, its nicely done. Several detailed replies, even I couldn't resist!

      --

      Bleh!

    9. Re:Impractical by Luminous+Coward · · Score: 2, Informative

      I can either purchase 4 machines with gigabit (~1000Mb/s) ethernet, Myranet (~200Mb/s) cards [...]

      You meant Myrinet. And you meant 200MB/s not 200Mb/s. Actually it's almost 2 Gbps (2000 Mbps).
  5. Geode Specs by Ron+Harwood · · Score: 2

    Here are the Geode specs... "Speeds offered up to 266 MHz"

  6. G4 processor cards by Peter+Dyck · · Score: 3, Interesting
    I've been wondering how expensive/difficult it would be to build a multiprocessor computer for computational physics applications based on G4 PowerPC cards.

    I'd just love the idea of having a host PC (or a Beowulf cluster of them ;-) with all the PCI slots filled with G4 7400 boards crushing numbers...

    1. Re:G4 processor cards by statusbar · · Score: 3, Insightful

      'Crushing numbers' is the right term, as g4's altivec is only single precision.

      But it would be cool.

      --jeff

      --
      ipv6 is my vpn
  7. PCI card computers by hattig · · Score: 3, Interesting
    You have to remember from a certain aspect, you can add a PCI card to a motherboard which made the motherboard the PCI slave.

    PCI = PCI = PCI = CPU = PCI = PCI
    I I I I
    IDE CPU CPU CPU
    I I I
    USB PCIs PCIs
    I I
    IDE ..
    I
    USB

    I have left out memory controllers, northbridge, etc, and modern fancy chip interconnects because they are just fluff (no, not fluffers, that is another industry). In the above diagram, what is the host CPU? Is there actually such a thing as a host? The PCI bus is arguably the center of a modern PC, with CPUs and controllers hanging off of it.

    Modern motherboards are just a restriction on what you can do in reality. Reality is a PCI backplane on a case, maybe with a couple of PCI-PCI bridges. You can then add anything into any PCI card that you want - normal PCI cards, or CPUs (NB, Memory, CPU, etc).

    That is why you can configure these cards to use the 'host' IDE drive. It is just a device on every 'computer' within the case...

    I can't post a diagram though, because I must use "fewer junk characters". Bloody lameness filter - affects the real users, the people it is meant to trap just work around it. Would you call this a "lame post"?

  8. Re:Linux on PCI cards is the way forward. by Ron+Harwood · · Score: 2, Interesting

    Actually, in my LUG we've given the newbies an eval copy of VMware, and a pre-installed linux image... let's them play for a month before they have to think about installing.

  9. Isn't that the course we've been on? by JAZ · · Score: 4, Funny

    Follow me here:
    A computer used to take up a room.
    Then, computers were large cabinets in a computer room.
    Now, they are boxes in a computer cabinet in a computer room.
    So we can extrapolate the next step for computers is to be cardss in computer box in a computer cabinet in a computer room.

    It's a natural (obvious) progression really.

    --


    "Karma can only be portioned out by the cosmos." -- Homer Simpson
    1. Re:Isn't that the course we've been on? by dillon_rinker · · Score: 2

      Computer chips on computer cards in computer boxes in computer cabinets in the computer room...

    2. Re:Isn't that the course we've been on? by CarrotLord · · Score: 2

      Then chips on cards in boxes in cabinets in rooms.
      and then... ?

      salt on chips on cards in boxes in cabinets in rooms?

      hmmm

      --
      Quidquid latine dictum sit, altum videtur.
  10. Geode? by tcc · · Score: 2

    266mhz max. Their target audience is the firewall/network application.

    Too bad a Dual Athlon-based solution (on a full length PCI card) would suck too much juice... at least from the current PCI specs... AMD needs to make a move like intel did with their Low wattage PIII, I'd love to see a 12 processor (5 pci slots plus host) renderfarm in a single box for a decent price. Not only it would be space saving, but imagine that in a plexi-glass case :) a geek's dream.

    --
    --- Metamoderating abusive downgraders since my 300th post.
  11. Imagine a new kind of bus by Anonymous Coward · · Score: 2, Insightful

    Imagine if all the devices in your computer were attached to each other with 100 GB optical cable.

    Essentially there would be a switch that allowed about 32 devices to be attached.

    The devices could be storage devices, processors, audio/video devices, or communication devices.

    Storage devices would be things like memory, hard drives, cdroms and the like.

    This bus would allow multiple processors to access the same device at the same time and would allow devices to communicate directly to each other, like allowing a program to be loaded directly from a hard drive into memory, or from a video capture device directly onto a hard drive.

    No motherboard, just slots that held different form factor devices with power and optical wires attached.

    A networking device would allow the internal protocol to be wrapped in IP and allow the interntal network to be bridged onto ethernet. This would allow the busses on seperate computers to work like a single computer. The processors on all the machines could easily network together, memory could be shared seamlessly, harddrive storage would be shared and kept backedup in real time. Any device in any machine could communicate directly with any other device in any other machine. Security allowing.

    Want 20 processors in your machine? Install them.

    Want 6 memory devices with 1GB each? Add them.

    Want 100 desktop devices with only a network device, display device and input/output device that use the processor and storage out of an application server? No problem.

    Want a box that seemlessly runs 20 different OSes each in a virtual machine that are ran across 10 boxes in a redundant failover system? No problem, it's all done in hardware.

    Want the hard drives in all the desktop machines to act like one giant raid 5 to store all the companies data on? No problem. (1000 machines with 10 GB each is 10 TB of storage)

    This is the future of computing.

    1. Re:Imagine a new kind of bus by MikeFM · · Score: 3, Interesting

      I think the basic form to use is some simplified base system designed to be upgraded to the extreme. No built-in crap on the motherboard to speak of.. just lots of PCI slots. If they could share harddrive and RAM and provide a keyboard/mouse/monitor switching method similar to KVM switches but all in one box it'd be great. So rather than replacing older computers we could just add to them. Maybe perfect something like MOSIX and drop the whole stupid SMP idea. I've always imagined computers would someday be like legos where you could buy a CPU lego, a RAM lego, a harddrive lego, etc and just plug them together in any order to add to a hot system. No reboot and no case to open. If one burned out just toss it and put a new one in.

      --
      At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
  12. SUN has a similar product.. by Phizzy · · Score: 5, Interesting

    I am actually typing this comment on a Sun Microsystems SUNPCI card.. It's a celeron, I beleive a 466mhz or so, w/ 128m of ram. It has onboard video if you want to use an external monitor or can use the sun's native card if you want to run it windowed, ditto w/ ethernet. I've been using the card for about 3 months now, and other than some instability w/ SunOS 2.6 (which dissapeared in 2.8), I haven't had problems with it.. you can copy/paste between the Sun window and the 'PC' window, which is very helpful.. and though we are running WIN2000 on it (ok.. so shoot me) I don't see any reason why you couldn't run linux on it if you really wanted too.. All-in-all, the card is pretty badass..

    //Phizzy

    --
    "Most European technology just isn't worth our stealing," -- Former CIA chief James Woolsey, referring to Echelon
    1. Re:SUN has a similar product.. by Phizzy · · Score: 2

      yeah.. that, and StarOffice eats all your ram, not to mention your desktop when you run it. ;)

      The test I've run of SunPCI has convinced our management to do away w/ separate NT/2000 systems when we move to a new building in april, and just outfit everyone w/ Ultra 5s, SunPCI cards and dual-head monitors..

      //Phizzy

      --
      "Most European technology just isn't worth our stealing," -- Former CIA chief James Woolsey, referring to Echelon
    2. Re:SUN has a similar product.. by 11223 · · Score: 2
      Actually, Sun is making them now with 733 mhz Celeries in them.

      Definitely an awesome product.

  13. Audio Apps -- Digidesigns DSP Farms by namespan · · Score: 2

    I haven't been paying attention to the market... I guess things like this aren't all that rare. Apparently there's a G4 PPC computer-on-a-card as well.

    But anyway, it reminds me a quite a bit of what Avid/Digidesign do for their high-end systems.
    You see people who've got 6 slot PCI systems and 4 of those slots are filled with extra computing cards (sometimes more... some people get expansion chasis'). You can rely on your computers processor if you're not doing to many complex effects on a track of audio, but at some point (not too hard to reach... throw in a tube amp emulator and a reverb) you run out of CPU. So they have PCI cards which have a couple of DSP chips (Motorola 56xxx series, I think) on them, and the more of these you add, the more audio processing you can do simultaneously.

    At some point, perhaps people will think: hey, why add a specialized card? Why not just more general purpose computing power?

    --
    Libertarianism is rich wolves and poor sheep playing gambler's ruin for dinner.
    1. Re:Audio Apps -- Digidesigns DSP Farms by RFC959 · · Score: 2
      At some point, perhaps people will think: hey, why add a specialized card? Why not just more general purpose computing power?
      Thank you for contributing to the turning of the Wheel of Reincarnation.
  14. Re:You don't understand... by krow · · Score: 2

    Right. And the cool part (which makes them a bit different then the typical solution) is the loopback ethernet on them.

    --
    You can't grep a dead tree.
  15. Switched Bus, Multipurpose cards by swb · · Score: 2

    I'd like to see a bus that was little more than a switch, with a minimum of logic for management.

    For cards, it'd be great if each card had its own CPU and RAM. Ideally the cards would have a few universal connectors, each of which could accomodate an I/O module which would contain just the discrete electronics necessary to drive a specific device or medium (eg, video, audio, disk, network).

    Bus-Switch modules would be interconnectable to accomodate more cards, and would have switch-like manegement features for segmentation, isolation and failover type features.

    The CPU cards themselves ought to be less complicated than motherboards since there's no bus logic, just interconnect logic to the Switch-Bus and the I/O modules, and RAM.

    Since each board has its own RAM and CPU it ought improve system performance because the O/S could offload much more processing to CPU boards dedicated to specific tasks. Instead of the kernal bothering with lower-level filesystem tasks and driving the hardware, a "driver" for filesystem and devices could be loaded on a CPU board dedicated to I/O.

    The same could be true of user interfaces -- run the the UI on the board dedicated to video, audio and USB. The kernal could run applications or other jobs on the "processing" CPU board(s).

    Networking? Offload the entire IP stack to the networking CPU board.

  16. Linux on PCI a year old, dudes! by scaryjohn · · Score: 2, Insightful

    I looked at this and said... wait a minute, hasn't this already been sorta done? Despite not being a full featured box, Firecard is a PCI-card running Linux... for the purposes of supporting a firewall (as you could have guessed from the name if you'd not read the story -- Nov 14 2001... but it's cool that they've taken it to the next level.

    --
    One might ask the same about birds. What ARE birds? We just don't know.
  17. Read the article... by YuppieScum · · Score: 2

    These cards have TWO NICs, one that talks across the PCI bus and one physical RJ45 10/100...

    I'm only go to say this once, but I could copy/paste the same response to 20 or 30 posts on here...

    --
    This sig left unintentionally blank.
  18. who cares? by adolf · · Score: 2

    Stuff like this started in the PC world, IIRC, with 386s on 16-bit ISA cards.

    Nobody cared then.

    Why would anyone care now?

    Please explain your point using no more than 100 words.

    -

  19. Sunpci card by johnnyp · · Score: 2, Informative
    I've got an Ultra 5 with a PCI card which has an AMD K6/400 on it so's the Ultra can run a Windoze machine. The K6 shares the HD disk and can either use the Sun display or you can plug a separate monitor into its on-board vid. Also shares the ethernet card. It works OK, runs Win 98 fine (95 is supposed to work though I can't get it to, but I have seen one running NT 4.0) and you can cut and paste between CDE and Win 98. The only real use I find for it is firing up IE to see if webpages I've made look OK.

    I think you can pick them up pretty cheap nowadays if you like that sort of thing. I don't imagine much mileage from trying to install a.n.other unless you feel like writing the relevant drivers to get everything to talk to each other.

    --
    Johnny
    1. Re:Sunpci card by PerfectWorld · · Score: 2, Informative

      Sun limits the number of these cards which are supported in each system type, due to bandwidth limitations, but obeying the supported rule is up to you. I wouldn't put more than 2 in a desktop box (U5/10,SB100) and I think only 1 is supported. IIRC, Sun supports up to 6 in an E450.

      The current model is a 733MHz Celeron with 128MB RAM base, going up to 1GB RAM, onboard Rage something or other graphics. Supports all versions from Windows from 95 to 2K Adv Server.

      You can do some interesting things with these. Since Windows is 'installed' in an image file on the UFS filesystem, you can copy them, easily back them up, and bring differant images up on differant cards at differant times. You could have the office apps images running during the day and the Quake servers running at night ... ;-) They are also cheap enough to have a spare in the server should one go tits up.

      They won't run Linux unfortunately. They would have to add support for that to the SunPCi software.

      - Mark

      --

      Ancient Budo Master once told me: "All your bruises are belong to us."

  20. Re:who cares? lots of possibilities by adolf · · Score: 2

    But is just adding boxes not cheaper, more flexible, and less prone to catastrophic failure?

    By my reckoning, half-width 1U rackmount PCs stack more densely than mid-tower cases each with a half-dozen cards in it. Overall reliability increases, as well: you don't need to rely on a single motherboard and power supply to keep things going.

    Putting a bunch of rackmount PCs into a single (portable, or not) box is also a very trivial exercise.

    Is realestate at the desk really at such a premium that anyone need care about these things?

  21. wait a minute by dakoda · · Score: 2, Insightful

    this is exactly what many good video cards do, but in a specialized manner. same with high end sound etc. the idea of putting powerful cpu's on cards is probably as ancient as cards themselves.

    as has been noted before, this would really be useful if the pci bus was extended (faster/wider). of course, making it faster/wider gives you what sgi has been doing for a while too (also mentioned above).

    perhaps the most dissapointing thing is that all that power goes to waste on users playing solitare, running windows, aol, and quake, not on something that will actually need the power to perform the tasks. well, maybe quake isnt so bad...

  22. days of future passed by hatless · · Score: 3, Informative

    Can this be the, uh, future?

    No, not if it's existed for decades. It's what's referred to as a "mainframe". You know. An expandable number of processor boards running under an operating system that can treat them as any number of single-processor or multiprocessor machines, with the ability to reassign processes between CPUs.

    The Unix world has had them for a long time, too. Modern examples include Sun's higher-end servers, which support hot-swappable and hot-pluggable processors and memory.

    Doing it with x86 processors and standard x86 OSes like x86 Unixes and Windows is less common but I believe Compaq and maybe Unisys can sell you machines that can do it, too, with one or several instances of the OS running at once.

    This hatdware approach is not quite the same as VMWare's server products, which do it via software and don't limit you to one OS per processor or block of processors. It in turn mimics other decades-old mainframe operating environments in approach.

  23. This is nothing new by mrdisco99 · · Score: 2

    IBM AS/400's have offered integrated Netfinity adapters for years. These are PCI cards with processors, memory, and console/network connections which share power and storage on the AS/400. You can fit up to 16 of these in a single machine. Check it out

    --

    +++
    NO CARRIER