Slashdot Mirror
When The PCI Bus Departs
km790816 writes: "I was just reading an article in the EETimes about the possible war over the technology to replace the PCI bus. Intel has their 3GIO. (Can't find any info on Intel's site.) AMD has their HyperTransport. There has been some talk about HyperTransport going into the XBox. I hope they can agree on a bus. I don't want another bus standard war. So when can I get a fully optical bus on my PC?" Now that's what I'd like: cheap transceivers on every card and device, and short lengths of fiber connecting them up. Bye bye to SCSI, IDE, USB, Firewire ...
Part of my PhD work is in this area and I just wanted to throw in some of the things I've seen..
First, in a nutshell, look for PCI-X. It makes PCI a switch based medium instead of a bus, will let you utilize your existing PCI devices.. Supposed to be very good for performance (Gigabytes/s) and will be out in a few months. This is the one to bank on..
The real competitor is InfiniBand. The big 7 are involved in this (includes Compaq, Intel, IBM, MS, HP..). Its huge, with an 800 page spec you can get online for $10. It may be used to replace local I/O infrastructure (ie pci), as well as comm between hosts (ie cluster computers). It functions as: an I/O replacement (pci), Storage area network solution (ie fiberchannel, smart disks), and System Area Networks (cluster computer interconnect..Myrinet, GEth, SCI). Multi-gigabit links, switched medium, memory protection for transfers between hosts, split dma transaction notions (ie remote DMA mechanisms)..InfiniBand is what happened to a lot of the smaller competing standards, btw..
A lot of people are not happy with IB. People see it as the big 7 trying to ram more hardware down our throats in a way that forces us to rewrite our OSes and do communication. Looking on the IB trade page and you will literally come across quotes along the lines of "InfiniBand(tm) is a knight in shining armor to save you from the monster of PCI".. Do we really need to have a new standard when 90% of the people out there use it for low-bandwidth soundcards, 5,000 gate ethernet transceivers....?
You MUST read this article about PCI-X vs IB:
http://www.inqst.com/articles/pcixvib/pcix.htm
grimace/Georgia Tech
Needless to say, it was the smallest, slowest stampede in history.
At least EISA and VESA slots would also take regular ISA boards (except for that full length card skirt thing).
I see even classic Slashdot is now pretty much unusable on dial up anymore.
Hey guys, don't forget that the PCI and ISA busses are more than just slots on your motherboard. Only until recently was the IDE controller on most motherboards moved over to the PCI bus. If you have a floppy drive on your computer, chances are it is still using the ISA bus... So just because you don't have an ISA "slot", don't think that the bus is gone...
---
Well, the basic answer is that AGP isn't a bus, it's a port. A bus can have multiple devices chained off a controller, but a port, only 1; each AGP slot on a motherboard needs its own controller chip. That's one reason you only rarely see mobos with multiple AGP slots. :)
There's also the basic reason that almost nothing besides gfx cards -need- the huge bandwidth and bus speed of AGP.
-- Veni, vidi, dormivi
...what he's talking about?
> Now that's what I'd like: cheap transceivers on
> every card and device, and short lengths of
> fiber connecting them up. Bye bye to SCSI, IDE,
> USB, Firewire.
Here the posting is about replacing the high-bandwidth (formerly local) bus in PC architechture, and he thinks the suggestion regarding an optical bus is to be used for the (relatively) slow I/O busses of IDE, SCSI, Firewire and USB?
I think there should be Metaeditors to handle the editors who talk before their brain starts working. Either that or Timothy should be disallowed from adding "his two cents" to a news posting.
-Chris
...More Powerful than Otto Preminger...
PCI is 32 bit at 33MHz. This is 1000 Mbits per second. The only interface device that needs or ever will need more than that is Graphics cards, and they have AGP slots. Most of the devices we use can even be run over USB.
Is that like Bill Gates saying no one will ever need more than 640kb? Frankly, I use two graphics cards in my desktop, and the only reason I don't have three is that the cost of another LCD panel is ludicrous. As soon as they come down more, I want one more panel, and then I'll be happy. I hate having to settle for a differently-branded (and usually more expensive) PCI card just because I don't have more AGP ports available. Usually the cutting edge stuff only comes out on AGP.
Granted, I'm not playing Quake on all three at once, but the only reason I'm not is because I can't. I'd love to be able to play my driving games on all 3, with the left monitor being a left view, and the right being a right view. Or a view of my nearest competitor. Or even just a big rear view mirror. The possibilities are endless.
The next thing up is storage area networking. PCI cards can't handle the biggest SAN loads, like our DVD jukeboxes at work. We can only use one 300-DVD jukebox per server, because the bus load can't handle more. Think in terms of quad Xeon servers, and it'll make sense - you can indeed shuffle a lot of load across the bus and off the fiber network if you need it. (And no, it's not a single reader per jukebox, there's lots of readers in each jukebox.)
What's your damage, Heather?
Need more CPU power or more memory? Hot-plug a module into the Infiniband Switch.
Don't gush. Infiniband, or any cabled serial connection, will never be a memory connection worth having. The reason is latency: what matters with memory isn't how many terabytes you can deliver per hour, but how many picoseconds the processor has to wait for that data that's stalling the pipeline right now. Which is very hard to reduce when the address has to be shipped a bit at a time over a serial link (64 bit times @ 10 Gb/s = 6.4 ns), transported over the cable (~50 ps/cm * ~60 cm = 3.0 ns), memory accessed (technology dependent), serialized (another 64 bits, 6.4 ns), shipped back (another 3.0 ns) and finally it gets to the processor. You've added 18.8 ns over and above any protocol overhead (usually much worse) and that's at 10 Gb/s!
Not gonna happen.
Lacking <sarcasm> tags,
There's also the basic reason that almost nothing besides gfx cards -need- the huge bandwidth and bus speed of AGP.
If that were true, then this entire thread, and both Intel's and AMD's replacement bus technologies, would be moot. I don't think they're doing it for fun. Basic (1X) AGP has the same bandwidth as 66 MHz 64-bit PCI, as found in servers and stuff; ~0.5 GBps. Obviously, that isn't enough either. Gigabit Ethernet, for example, is a prime example of something that usually sits on the bus, and that definitely needs more bandwidth (1Gb/8 = ~125 MB, which is very close to PCI's 133 MBps limit). If you want multiple Gb Ethernet boards on the same bus, the bus has to be bigger.
main(O){10<putchar(4^--O?77-(15&5128 >>4*O):10)&&main(2+O);}
That works fine in a simple Apple Macintosh which only has a couple of expansion slots and doesn't have very many devices that need interrupts. It's just not reasonable to limit the number of devices to the limited number of interrupts available. It's also not reasonable to require the computers to have a vast number of interrupts available in case someone want to use them. These resources can be safely and efficiently shared. The reason plug and play doesn't work is because of crappy hardware and driver development. I write drivers for FibreChannel boards, and my drivers can share interrupts and still perform well. Don't try and tell me that some ethernet card has to have it's own interrupt. It's simply a poor implementation.
Whoa, slow down partner. USB and Firewire have something that optical will never have, and that is power. You will never be able to send electricity down an optical cable. The only way to power something would be to send a bright light and use solar panels on the other end--not likely.
Furthermore, SCSI has direct memory access. Unless the new bus has DMA, then SCSI will still have a niche market. IDE? Well, maybe it is time to retire IDE.
Keeping
...no joke. Sure, it sounds stupid that they should, of all companies, come up with the replacement for PCI, but remember that it took Apple's decision to completely replace serial with USB and SCSI with FireWire for those two technologies to be seen as bona fide replacements by PC manufacturers. If a practical and real replacement for PCI does come along, I'm going to expect to see it on the new PowerMacs long before I'll see it on an Intel motherboard.
It might be interesting if folks started building Open Hardware (much like the Open Software movement). With Open Hardware, the specifications are open (as are the standards), but customers would still need to pay for manufacturing.
I wish we'd start doing something like this... we could then build Linux on top of it, and know that the drivers will work well. Not to mention the benefits of open peer review against the hardware specs.
And so it goes.
What about InfiniBand, which all the major PC hardware design people seem to be involved with? This takes a "switched fabric" approach to linking function blocks together, via Switches (which is where Brocade hopes to be the next Cisco). Need more CPU power or more memory? Hot-plug a module into the Infiniband Switch. Version 1.0 of the spec. is available for download at the site, for those interested.
The successor to IDE is already on the way: Serial ATA. Reportedly, PC makers like it because the thin cables allow them to build smaller systems with better cooling. V1.0 is not going to be much faster than UltraATA/100, but they say there's room for growth there.
Plus, you can have fibre channel (not fiber) hard drives right now, from Seagate (example), IBM (example), etc., and the big storage guys are heading that way too. Fibre Channel doesn't always mean Optical - these drives use a 40-pin "copper" connection, which can be a cable or a backplane (for hot-plugging). The SCSI-3 protocol is carried over the Fibre Channel interface, meaning that with a FC driver loaded, the drives look like SCSI devices.
Anyone see a trend here? It's the end of the parallel interface in all its forms, much as USB and FireWire are replacing the humble parallel port...
(this is not a
Yup, the PCI bus does ~1gbps or 133megabytes/sec, considering we have things like 1gbps Ethernet (~125meg/sec) and Ultra160meg SCSI... soon to be Ultra320, it's well under siege. I know these technologies aren't in the mainstream yet, but it certainly doesn't leave much headroom for growth when they do trickle down into general computers.
The above figures don't even include overhead either, obviously no bus performs to its optimum because no board is built for perfect bus timings.
As you said, the plan of over the last few years has been to shift everything off the PCI bus, graphics went to AGP and in most modern chipsets the south bridge has a dedicated 266MB/s link to the south bridge, rather than a standard PCI bus link. They've also took the ATA controller off the PCI bus and given it a dedicated channel to the north bridge.
Even by taking everything off the PCI bus... it's still hitting its limit, as for a bus that is nearly 10 years old, it's done quite well. It's not quite end-game yet though, remember PCI2.2 allows 64bit transfers, so they've effectively doubled the throughput and given it a little more breathing space, however this isn't a long-term solution.
That is an interesting idea - that we might replace all of our device connections (the bus) with fiber.
To take that idea a bit further, would it be possible to implement a protocol which is extendable? For example, each device connected gets a dedicated strand of fiber. The system, when polling the device, can negotiate a frequency range and transmission speed dynamically.
If I understand things correctly, this can help the system decide where it needs to put its resources, because higher demand devices would want a higher frequency range and transmission speed (hard drives, video cards etc) where simple devices like the mouse and keyboard will only take a little bit.
I think it'd be a great way to build a scalable architecture which might be unlimited in capacity, and eliminating wasted bandwidth and resources.
You can accomplish anything you set your mind to. The impossible just takes a little longer.
Just because optical has the potential to be faster doesn't mean is IS faster. You have to remember that you not only have to have a fiber, but you also have to have a very expensive source and reciever setup on either side. and fiber can not be run in parallel like electronics can. Each card would have to have a chain like device to allow communication with the next device on the bus. Aka, fiber to the first drive, that drive has fiber to the next drive, etc etc etc.. also another problem with fiber is that it is SLOWER over small distances because of the Voltage -> light -> Voltage conversion process.
Optics are mainly used as long distance communication devices (a few feet+). The reason that USB is used over fiber is that USB provides power.. 100ma if I remember corectly. Fiber can not reasonably provide power.
And all this is neglecting the cost/size considerations. Gold vias are nice because they are VERY thin and can be stamped into layers really easy. In other words, There will be no PCB like fiber for a while.. to large, way to complicated...
(Sorry.. I get tired of the 'fiber solves all speed/electronics problems' comments.)