Slashdot Mirror
Buses and Interconnects: The Next Generation
mkarpinski writes "ExtremeTech has posted a nice overview of the next generation of peripheral buses and interconnects including PCI-X, InfiniBand, 3GIO, and HyperTransport. From the article, "All these future interconnects and buses have a few things in common. They use packet-based, point-to-point connections; in fact, InfiniBand implements a full switch fabric. They provide bandwidth in multiples of that offered by PCI. They decrease latency significantly, with HyperTransport and RapidIO showing the most dramatic decreases, crucial for their target communications and embedded markets. And all four strive to reduce pin counts in order to conserve power and system real estate." Open the floodgates!"
Well, IB lends itself to a lot of chassis chassis connections (SAN, failover, clustering, etc)... The others pretty much stick to planars, but IB could network globally (a stretch, but theoretically, it is all there). Since IB is an application to application protocol, it has quite a bit of reach...
Think - a generic server, boots from the IB link (possibly off of a SAN)... add more of these to the IB fabic as needed to increase the cluster. Great possibilities.
"It's tough to be bilingual when you get hit in the head."
"Many people still don't fully understand and cannot effectively differentiate InfiniBand from PCI-X from RapidIO, or 3GIO from HyperTransport, for example."
geez, what are you guys, a bunch of idiots? I can't *believe* that some people STILL don't fully understant inifiBand form PCI-X from RapidIO, or 3GIO! What is the world coming to?
Moon Macrosystems. Sun's biggest competitor.
...that only ONE of these standards actually goes mainstream and takes the place of PCI. This reminds me of the DVDR-W versus DVDR+W conflict that's going on right now. You've got multiple standards that are totally incompatible with each other, and yet neither of them has any true advantage over the other.
Hopefully one standard will emerge, so I can *safely* proceed with upgrading my PC hardware without fear of immeidate obsolescence.
If my memory serves (occasionally it does) PCI-X has been "on the drawing board" for, what?, the past 5 fricking years?
Quick with the acronyms, slow delivering anything useful...heck not delivered anything yet from what I've seen.
3GIO, Infiniband and Hypertransport...remains to be seen. But at least there is a 1 out of 3 something showing up and relieving these "low bandwidth blues".
Isn't Hypertransport supported by the AMA?
whoops, acronym abuse...Apple, Microsoft and AMD?
Seeing as Hypertransport is AMD's brain child.
We'll see, but it winds up being a matter of "the chips falling where they may", especially in the hands of motherboard makers, where they belong.
Whoever comes out with an actual working spec could say "Get on the bus, come ride with us" in a commercial and not be referring to city transit.
(can you tell I want to go home an hour early?)
If it is not on fire, it is a software problem.
RapidIO is in the PPC G5 roadmap and will be in moto's first g5-based chip. I've been drooling for some time now...
HIV Crosses Species Barrier... into Muppets
Even though we (myself included) always complain about all the "standards" there are, it really is a good thing in the long run. Even thought I hate having to deal with stuff like which keyboard to i need for this system (USB, PS2, Big DIN connector), it is good. I know the best usually wins. It's just like ISA vs. PCI vs. AGP, USB 2 vs. Firewire, SCSI vs. IDE, (yes, I know those are not apples vs. apples, but you get my point). Eventually, we just have to wait it out, and then buy whoemever the winner is. The unfortunate part is that the early adopters (a lot of slashdot readers) are the ones that pay the high price for new technology, but that's the way it goes.
However, if we look at trends in 5 to 10 year periods, we can clearly see what technology won the battle for existence and standards. The best technologo doesn't ALWAYS win (think Windows Media...), but more often than not, and that gives time to sort out the better from the good. Right now, though, we do live with a lot of different, competing standards that are quite frustrating.
Moon Macrosystems. Sun's biggest competitor.
API NetWorks just released a HyperTransport "Switch" , See the press release .
Interesting stuff. The PDF has some more info
Peter
www.alphalinux.org
This article misses one crucial standard: PCIX 2.0. While not highly publicized it has some key features that make it more likely to show up in high performance systems that 3GIO. The PCIX 2.0 standard is due to be finalized at about the same time as the 3GIO standard will.
One should not theorize before one has data. -Sherlock Holmes-
--brian
Why do hardware makers insist on using marketroid-designed names? I'm going to stay with PCI until somebody comes up with a new type of bus: the Magic Bus.
Every day, I put a request on the queue.
Ooh yeah, it's the MAGIC BUS!
To get on the bus to my CPU.
Ooh yeah, the MAGIC BUS!
That's "Mr. Soulless Automaton" to you, Bub.
The "killer application" for these technologies (at least HyperTransport and RapidIO) is mostly very high-end telecom infrastructure equipment. My last job involved working on the definition of a HyperTransport bridge product that was targeted for these types of systems. A good serialized interconnect is needed in these systems, as nothing else comes close in terms of speed and latency.
PCI-X is also becoming very useful as a telecom backplane connector, and for drive arrays in high-end servers.
Of course your message was completely stupid in the first place. Why is it that so many people on slashdot look at every technology as though the PC is the only important system on the planet? It's fine that all you use is your desktop PC, and it's fine that you wait for the most commoditized solutions to filter down to your local PC shop, but despite what you might think, the world doesn't revolve around PCs. Perhaps you'll want a HyperTransport video card in 2004, but your major Internet backbones aren't going to be connected through AGP4x today. Do yourself a favour and get a more rounded technical education.
- j
Two words: Beowulf Clusters.
That is all.
check out Mercury Computer Systems
one of the co-designers of RapidIO, they make ppc shared memory multicomputers which will use RapidIO in the future.
think 320 PPC nodes, 256MB RAM each, the whole thing in one 9u 19" rack mount chassis, running on standard power with no special cooling requirements.
The difference between Theory and Practice is greater in Practice than in Theory.
I was reading through the article, and one question kept coming to my mind. It's great that they are coming up with higher bw busses, but it seems it will only help for I/O. What about memory? I know we have the 266MHz (I think) DDR memory, but how much is that really helping? How will memory access be affected by all of this?
sigged out...
Actually, from what I've read, for the most part the interface presented to the driver will be identical (or nearly identical) to that used by PCI. I doubt that any new bus standard which drastically changes the driver interface will be very popular.
---- I made the Kessel Run in under 11 parsecs.
...the intelligent switching of devices on the bus. Having a packet-switched bus controller is important (and I've worked on the design of an Infiniband controller so it can work), but even having the application passing parameters to the bus controller is important. For example, should a network card in a workstation take priority away from the video capture card?
Consequence to the capture card: dropped frames, necessitates recapture.
Consequence to the network card: dropped network packets.
Bandwidth is important, but again - the application must drive the necessity for the bandwidth. Intelligent switching and caching in addition to the increased bandwidth are necessary.
ALSO, one important point - architectures such as HyperTransport are essentially point-to-point, so you need multiple HyperTransport interfaces per switch IC - something that will drive the costs up. Hopefully low-speed devices can all be dropped on the 1394b or USB2.0 buses and then those can be handled through specialized south bridges on PCs, for example. For high-end network apps, obviously the switching of multiple buses leading to optical driver modules is basically the application itself.
You're losing it :)
Actually, the world is moving to a fabric-based interconnect rather than a bus-based one. RapidIO and Infiniband (and Sky and Race for the embedded world) all techincally support only one slot, too. And that one slot goes to a switch, which connects to other slots and all your other resources. Ideally, each processor would get its own connection, and maybe some memory would, too. That way, processor #1 can talking to the video card to play quake, while processor #2 serves up data from your hard drive to the LAN, while data goes directly from your HD to your DVD burner... all data takes a separate path and operates at full speed - no need to time multiplex, like with a bus.
HIV Crosses Species Barrier... into Muppets
What is the difference between this stuff and what Cray & SGI did in the mid '90s? Same speeds, point to point (~ 6GB/s). I think the MSS on HPPI was large, but what are the other differences?
The main difference? It won't cost $8 million dollars.
Well for one, mechanical hard drives have had their day in the sun and probably in the next 10 years or so having say a 15GB SSD (solid state drive - basically a drive made from nonvolatile memory) won't be unheard of. In such a case we'll have "instant on" computers and then you should really see a big difference in thinks like Hypertransport vs. AGP. Right now mechanical devices are big hindrances to electronic parts, just as electronic parts are to say fiber optics.
The PC will not need bandwidth of this capability for some time. This bandwidth is needed right now in the networking/telecommunications, server and storage area. With networking technologies pushing into and beyong the 10Gbs bandwitdth arena, technologies such as PCI-X and Infiniband are here just in time. Companies such as Mellanox are exploiting these technologies (Infiniband) just in time for applications in these areas.
It also leads to other potential ideas. What about truly shared memory? Being able to put core memory for multiple processors in a distributed system? RapidIO will help us tie multiple processors together and share memory and I/O resources, and they will not need to be inches away, but possibly many many feet. The possibilities are truly endless with what we can do with this bandwidth.
Routers typically use standard I/O busses such as PCI to route packets between various I/O interfaces and the central CPUS. We now can increase the bandwidth up to OC-192 and 10GE with a standardized interface. This implies lower cost ASICs with common bus interfaces. No longer will companies need to develop their own ASICs with proprietary interfaces to support bandwidths capable of 10gigs...
Really cool technologies!
--Chris
It's because Motorola has traditionally included a lot of onboard support 'for free' in their embedded chips, such as memory controllers, interrupt handlers, SRAM, EEPROM, etc.
It doesn't mean we won't see HT used on the motherboards, or that HT and RapidIO won't both be on the motherboard.
It could be that RapidIO is used between CPUs and memory, with HT linking the video, sound, PCI-X(ugh, I think I like 3GIO better) together with the CPU-Memory systems.
As an example.
GPL Deconstructed
Apple was one of the early members of the HyperTransport consortium... what does this say about the G5's motherboard architecture?
It probably means Apple is hedging their bets.
OTOH, Apple likes nVidia, Apple likes HT, nVidia likes HT... nForce for PowerPC anyone?
Am I losing (or loosing, either works) my grip on reality or was the whole point to have all your cards belong to us and on the same damn freq/speed/clock perhaps at the FSB rather than a seperate timing?
You lost it. Having the bus clock run at an integer divisor of the FSB clock is good, but it has nothing to do with the number of slots per bus.
multiples of that offered by PCI
I sure hope that the multiples are real numbers greater than or equal to 2. I don't know if I could cope with (2+i) * PCI speed or -22/7 * PCI speed.
Washington, DC: It's like Hollywood for ugly people.
Woohoo InfiniBand. Finally small companies can build clusters that can compete with multi-million dollar SGI super systems. Beowulf clusters using Ethernet interconnects are nothing compared to a system with memory bridge level hardware interconnects. Current Linux clusters are often based around Ethernet which is very slow and has a high overhead when you want to put it to the use of letting processes on different machines talk to one another. Big Beowulf clusters have been shown to work well on embarrassingly parallel computational problems (key cracking, some types of CFD, and packet analisys like SETI@Home) but aren't really up to snuff with truely high end super computers without much faster interconnects. Some clusters (I think Microway and a few others) are built using Myrinet interconnects which are faster and actually provide the sort of connections big clusters need. Even using Myrinet you end up with alot of scalability problems because you only get your max throughput when transfering very large blocks of data to the same host. Having a Northbridge based interconnect (rather than something like a Myrinet card which is connecting to the PCI bus which in turn hooks to the Southbridge) is a very good way to speed up clustered systems. IBM, Sun, and SGI learned this a long time ago which is why they use high speed interconnects between individual boxes rather than trying to overextend network protocols.
I'm a loner Dottie, a Rebel.