iSCSI Moves Toward Standard

EyesWideOpen writes "The iSCSI technology, which allows computers to connect to hard drives over a network connection such as a company Ethernet network or the Internet, requires only minor changes before the Internet Engineering Task Force endorses it as a formal version 1.0 standard. A final round of comments has been completed on the technology according to the Storage Networking Industry Association, the subgroup that led the creation of the iSCSI, and as a result companies now can start building iSCSI products."

Comments

Just wait...

[the Man in Black]

...I give it a week or two before someone buys a patent for "Accessing digital storage devices via a network" and sues.

Jeesh.

iSCSI not ready for prime time

[IGnatius T Foobar]

I work at a mid size hosting facility, and we've done quite a bit of experimentation with iSCSI. In my opition it's not ready yet. Either that or it's just a bad idea, full stop.

We do quite a bit with our SAN -- there are a coupla IBM 2105 ESS ("Shark") boxen in the back of the data center with many terabytes of disk online. It's all about Fibre Channel. At least as fast as SCSI, effectively faster when you have all sorts of cache running on the storage side, and you have the flexibility to define exactly how much disk goes to what server, and you can add more dynamically without a power down, etc.

Unfortunately, Fibre Channel is expensive. It requires expensive host bus adapters and even more expensive switches. And of course it runs over fiber optic cable, which isn't exactly penny kit. So the industry decided to try running it over Ethernet.

Now there are iSCSI-to-Fibre gateways, such as Cisco's 5420 Storage Router (which we've evaluated), but there are just problems in general with running block level storage over a TCP/IP network...

• For one thing, it's only as reliable as your network. If you have a network problem such as a down switch/hub etc, you lose your disks immediately.
• Unlike SCSI and Fibre Channel, you can't boot from an iSCSI volume. This is because your operating system has to be loaded, and your TCP/IP stack initialized, before you can load the iSCSI driver.
• Most operating systems want to load their storage drivers before they load their networking drivers. Doing it the other way around challenges all sorts of assumptions made by various system software out there. Sounds trivial, but again, we've evaluated it, and the result ain't pretty.
• By putting block level storage on your LAN, you've increased the capacity requirements by several orders of magnitude. To get any reasonable performance you're going to need Gigabit Ethernet everywhere -- and if you're going to make that kind of investment, you might as well be doing Fibre Channel.

That's why our iSCSI stuff is just sitting around doing nothing right now.

The only place I can see iSCSI being used at this time is for really temporary quick-and-dirty setups, such as a programmer needing another 100 GB online for a one-week project. But even then, NAS seems like a better idea.

Re:iSCSI not ready for prime time

[Wakko Warner]

For one thing, it's only as reliable as your network. If you have a network problem such as a down switch/hub etc, you lose your disks immediately.

If our Brocade switches go down at work, we lose our Hitachi fiber-channel SAN, too. We also lose our StorageTek 9960. But that's a separate, redundant network, and I'm sure a properly-designed iSCSI network would be separate and redundant as well.

Unlike SCSI and Fibre Channel, you can't boot from an iSCSI volume. This is because your operating system has to be loaded, and your TCP/IP stack initialized, before you can load the iSCSI driver.

Firstly: Why would you want to? Every one of our servers that are attached to the Brocade have their own pair of internal mirrored disks for booting. What's the point of doing it any other way? I guess, if you ever truly needed to boot from an iSCSI device, those issues will be addressed by OS vendors once there's enough uptake for iSCSI.

Most operating systems want to load their storage drivers before they load their networking drivers. Doing it the other way around challenges all sorts of assumptions made by various system software out there. Sounds trivial, but again, we've evaluated it, and the result ain't pretty.

See last point made above.

By putting block level storage on your LAN, you've increased the capacity requirements by several orders of magnitude. To get any reasonable performance you're going to need Gigabit Ethernet everywhere -- and if you're going to make that kind of investment, you might as well be doing Fibre Channel.

Gigabit Ethernet is still much cheaper than FC. I can see the market they're aiming for with iSCSI, can't you?

- A.P.

Re:iSCSI not ready for prime time

[foobar104]

Are you kidding? When was the last time you used Fibre Channel? Its mostly optical now. All the new HBA's come with optical GBIC's.

You're both wrong. FC is neither "mostly optical" or "mostly copper." Devices like HBAs and switches that use GBICs (modular media adapters) can be either optical or copper depending on the GBIC used, and switched on the fly. You choose optical or copper cables depending on your environment. Copper cables have shorter runs than optical cables-- they can only run 30 feet or so, as opposed to miles for optical-- and they much more bulky. So in a data center where you have literally hundreds of FC cables, you'd probably choose optical to keep the physical size of the cable bundles from getting out of control. For connecting two devices in a rack, you can choose copper cables.

I think the "fiber optic is expensive" thing is a myth, though. I can't say for certain, but I think I remember that the outfit that sells us our patch cables sells 4-wire copper cables and optical cables at roughly the same price.

Re:iSCSI not ready for prime time

[selectspec]

FUD alert:
For one thing, it's only as reliable as your network. If you have a network problem such as a down switch/hub etc, you lose your disks immediately.

Of course a fiber channel SAN network has exactly the same properties.

Unlike SCSI and Fibre Channel, you can't boot from an iSCSI volume. This is because your operating system has to be loaded, and your TCP/IP stack initialized, before you can load the iSCSI driver. Most operating systems want to load their storage drivers before they load their networking drivers...

This is not true and has nothing to do with iSCSI but rather the iSCSI HBA. An iSCSI HBAs can have their own network stack which not only offloads the networking computes but also configures on its own.

By putting block level storage on your LAN, you've increased the capacity requirements by several orders of magnitude. To get any reasonable performance you're going to need Gigabit Ethernet everywhere -- and if you're going to make that kind of investment, you might as well be doing Fibre Channel.

Look at the figures. A 1Gb fiber channel switch costs roughly twich that of a 1GigE switch. 10GibE switchs are already available, while 10Gb FC still is being debated. The upgrade to GigE will happen naturally on a network. The cost of the switches are ammortized over the network and the switches are cheaper because they don't serve a specialized data center market.

Most excellent news!

[Jeppe Salvesen]

I applaud all such efforts. If it doesn't work, fine, we won't use it. But if it works, it could easily become yet another technology that is excellent for its uses. Think about this technology a little more deeply. With a bit of work, it would change the name of the game in file servers. All operating systems that support iSCSI and the FS would be able to share the harddrive. I can see some savings down the line in terms of maintenance, and reduced downtime. I hope I'm right. Now, we just need to figure out exactly how to use this technology.

If everyone had fiber into their homes, I can at the very least see harddrive upgrades without ever opening the box. Wouldn't that be nice, folks?

Re:hum..

[SQL Error]

The difference is very simple:

With a file server (current buzzword is "NAS" for Network-Attached Storage) the server maintains the file system, and multiple clients connect to it to read and write files. It's a shared *file system*.

With a SAN (Storage Area Network) a bunch of raw disks is made available over a network. Currently this is normally Fiber Channel; iSCSI will bring standard Ethernet to SANs, making it much cheaper. No file system is mandated by the SAN; a machine connected to the SAN gets access to one or more raw disks and can use them any way it wants. Typically, the unit of allocation is one disk, though some systems (EMC) allow disks to be subdivided and the sub-disks handed out separately. While the storage pool on the whole is shared, each disk (or sub-disk) is only connected to one machine at a time.

A SAN provides a centrally managed pool of local disk, so you don't have to run around upgrading individual servers. This is a *big* win for large corporations.

Possible uses

[Anonymous Coward]

Well, the article is useless, but this white paper clarifies some points.

One exquisite use would be for someone maintaining a lab: imagine remotely partitioning and ghosting 100's of computers from a single console through Gigabit Ethernet, or being able to repartition a colocated server.

One aspect that is disappointing is that it just looks like SCSI over IP. None of the peer to peer aspects of Firewire were mentioned, such as target-disk mode that newer Macintoshes support. It's really nice to be able to reboot, hold 't' and plug my laptop into another Mac and have its hard disk appear on the desktop as though it was an external Firewire disk.

iSCSI nearly ready for prime time

[crow]

We're starting to see PCs ship with 10/100/Gig ethernet standard. Within a year or two, it won't be unreasonable to run GigE to every desktop in the building.

Now consider what iSCSI offers the system admins. You can use the network boot option on the desktop systems and run them diskless. This means you can centralize your storage. No longer to you face the daily panic of a user desperate to recover a file they only saved on their local hard drive. If someone is having trouble with their system, you just give them a fresh boot image; if the problem persists, it's hardware. If I were a sysadmin, I would be pushing hard for iSCSI.

And from the technology standpoint of iSCSI vs. Fibre Channel, I expect that ethernet speeds will outpace Fibre Channel speeds; it's a larger market, so the R&D investment will go there first.

[Disclaimer: I work for a data storage company, but everything stated here is based on general observations and opinions, not insider information.]

what's wrong with smb, nfs, ftp, http?

[jilles]

I don't understand why it is necessary to tunnel a low level protocol like scsi over ethernet (other than to trick legacy software into remote storage). There are protocols for remote storage, why not use these?

anytime you read IETF is about ready to approve..

[keithmoore]

Anytime you read that IETF is about ready to approve something as a standard, take it with a grain of salt unless it comes from the IETF chair or the area director responsible for that group. Such statements are usually propaganda from people who are trying to encourage premature adoption, or at best they are wishful thinking. It's not unusual for working groups to produce drafts which they think are ready for approval, but which actually contain serious technical problems that need to be resolved. Fixing those problems can require months or even years.

In particular, the fact that The Storage Networking Industry Association has completed its comments on the draft doesn't have any bearing whatsoever on IETF standardization.

Someone mentioned the security issue. I haven't followed the iSCSI discussions but security is definitely an issue that was identified before the group was formed, and one which is particularly difficult to solve for iSCSI because of performance concerns. I'll be interested to see how they've addressed it. I'd consider it extremely unlikely for IETF approve the standard without due consideration of security. And saying "it's going to be behind a firewall, so it doesn't have to be secure" has traditionally not been considered sufficient.

(FWIW, I'm a former IETF area director)

No raw disk IO

[marm]

There are protocols for remote storage, why not use these?

I agree that for most network storage, low-level SAN protocols are pointless - higher-level abstractions of remote disk such as smb/nfs/etc are much better as they enforce proper filesystem semantics, and run on top of a physical filesystem. You get all the advantages of having a filesystem in the first place - locking, sane disk space allocation algorithms, journaling, that sort of thing.

However, some applications - big databases particularly - prefer to have raw access to the storage medium, with no filesystem in the way to slow them down. These applications implement their own locking, sharing and space allocation semantics which are optimized for their own particular storage use patterns.

Classic file sharing protocols don't cut it for these big databases because there's no way to get raw disk access over the network with them. Which is why these lower-level SAN protocols exist - they provide the raw disk access that the big databases want, over a network. This means you can have your database spread over multiple physical locations to minimize the risk of your whole database going up in smoke, without taking the performance hit that running the database over smb/nfs would have.

You won't see iSCSI hardware making it into bog-standard file server hardware any time soon, but I can see it being huge in big-iron database servers, where it should be considerably cheaper and easier than Fibre Channel, the current best solution.

Admittedly, there are big questions over whether raw disk access is really necessary for databases - modern general-purpose filesystems are a LOT quicker than they used to be, and MySQL, for instance, which doesn't use raw disk IO but is still blazingly fast, is turning some of the performance assumptions on their head. But the big guys - Oracle, DB2 and so forth - still prefer it, so this is why iSCSI is here.

Security of iSCSI

[XNormal]

There is an important difference between my SCSI chain and an IP network - you won't find many SCSI chains with the kinds of security threats that are quite common on networks these days. Remember that block devices live below the OS permissions level - it's deeper than root access.

I hope that iSCSI has good security measures *enabled by default*. I remember some discussion on iSCSI mailing lists about using SRP and potential intellectual property problems. I hope it's in the final standard.