"iSCSI killer" Native in Linux

jar writes "First came Fibre Channel, then iSCSI. Now, for the increasingly popular idea of using a network to connect storage to servers, there's a third option called ATA over Ethernet (AoE). Upstart Linux developer and kernel contributor Coraid could use AoE shake up networked storage with a significantly less expensive way to do storage -- under $1 per Gigabyte. Linux Journal also has a full description of how AoE works." Note that the LJ article is from last year; the news story is more recent.

Comments

Will it catch on?

[andrewman327]

From TFA:
Some significant caveats mean that not everyone is so keen on the technology. For a start, it's a specification from Coraid, not an industry standard. Its networking abilities are limited. And its detractors include storage heavyweights such as Hewlett-Packard and Network Appliance.

So will this ever develop into a real standard or will it remain the sole domain of one company? I do not know if I want to invest time and money into it if the latter is true. From a comp sci point of view this is a great approach to networked storage. It uses what people already have to make storage reletively cheap. I am going to wait to see where this technology goes. Maybe it will blossom and become a serious contender.

Not an iSCSI killer, here are the reasons why not

[cblack]

1) Complexity for RAID and volume management is not centralized and is pushed to individual hosts. One of the main benefits of SAN technology is that you can just carve out storage from a single interface and assign it to a server and the server simply sees it as a block device. With AoE each drive is addressed separately by the server, which means it is up to the server (and server admin) to figure out how to handle distributing over multiple drives, handle drive failures, and expanding volumes. This is huge.
2) It is not a standard and is only really supported by one vendor. This may change in the future but it is significant right now. It is registered with the IEEE but that hardly makes it a peer-reviewed standard with input/improvements from many experts.
3) No boot from SAN. Until someone makes some sort of mini bootstrap system on a CD or a hardware card implementation of AoE that can be addressed as a block device admins will be unable to host the root filesystem and/or C: drive on an AoE SAN
4) No multipath (that I can see). Perhaps I misunderstand this, but it seems like there is no way to do multipath IO with this system. That is, all the drives are single-connected to a network. If that network switch goes down, all drives on that network are inaccessible.
So AoE looks like a neat technology for pushing drives out of the box and potentially sharing them among hosts, but there is no intelligence there. It is just dumb block addressable storage with no added availability or management, and therefore is far from being an iSCSI or FC killer.

Re:Cheaper?

[Zephiris]

The really silly thing about this is that they claim it's "lower overhead" than TCP/IP because people are having to buy "expensive TCP offloading engines" for iSCSI, when a few seconds of research provided, namely on Wikipedia (http://en.wikipedia.org/wiki/ISCSI), that plain NICs can outperform the offloading ones, and sure, it's obviously going to be lighter than TCP/IP, however, ATA over Ethernet only has basic authentication (MAC addresses, which can be forged cheerily), can't be routed, and isn't very available. It's -only- usable for Storage Area Network, not really for general remote drive (or part of a drive even) access. At currently, only Linux support is available. iSCSI is supported by Windows, Linux, Solaris, among others. Even FreeBSD is working on a native implementation. Windows Vista will even include a fully built-in/native support for iSCSI. I can't imagine why they complain that iSCSI is 'more expensive' to implement, when their primary product for ATA over Ethernet is a 'special drive enclosure' (according to their documentation, you can't even use AoE with standard networking hardware, interfaces, routers, etc) with special networking hardware which can house up to 15 ATA drives. The enclosure itself (with nothing else) costs about $4000. You could build ten high-end machines dedicated to serving iSCSI requests to multiple drives each for that (five if they use actual SCSI), and still use standard networking hardware, and still have it accessable from a network across the world, with things like actual user authentication.

The whole ATA over Ethernet thing seems like trying to blow smoke up the arses of some very rich and silly people. At the same time, the technologies are rather different, too. If you just want to build a SAN? Sure, go for HyperSCSI or AoE, maybe, but if you actually want remote drive access? Why would you want any of this? They shouldn't be trying to utterly replace iSCSI. It's absurd. As far as I see it, iSCSI is more of a general and free/open replacement for things such the old 'X drive' remote service, and network filesharing like SMB/NFS. Websites can (and are starting to) offer iSCSI targets to offer remote drives for backup. It can also be used for cheap SAN, or more-or-less replacing SMB/NFS over a network. It does all of this rather well.

It seems to me that the company behind ATA over Ethernet is becoming rather desperate to resort to such claims.