Slashdot Mirror

← Back to Stories (view on slashdot.org)

Cross-Platform Firewire Networking at Home?

Posted by Cliff on from the Xfers-As-Fast-As-Possible dept.

Stahnke asks "I have two computers that I need to exchange data between. I do music-production on a Windows-based system and have everything else on a Linux system. I need to exchange HUGE amount of data (5GB at a time sometimes) between the two systems as fast as possible while clients are waiting for me. 10/100 Ethernet is too slow, and fiber is just too expensive. Can Linux (2.4) do Firewire networking with a Windows machine? If so, how, and what tools are out there? I have a Firewire card working in Linux, but I haven't had luck with TCP/IP via Firewire yet."

52 comments

  1. A Halfway point by Anonymous Coward · · Score: 2, Interesting

    Perhaps if this is not possible, in the way of direct network connections, a halfway point may be worth considering. As firewire's hot-pluggability is a feature, go for a medium sized firewire drive that can be used purely for transporting files

    Admittedly it's not quite as neat a solution at first glance as just firewire-cabling between two machines - but it sounds as good an excuse as any to buy an iPod!

    1. Re:A Halfway point by Wise+Dragon · · Score: 1

      Add an A/B switch like this, and it IS a neat solution! Just store the data on the firewire drive in the first place. http://www.signvideo.com/firewire-switch-ieee-1394 .htm

      Here's a non-destructive test to check the read speed of your linux drive (replace /dev/sda with the correct device).

      #iostat 1 > iostat.log&
      #cat /dev/sda > /dev/null&
      #tail -f iostat.log

      Watch the change from second to second in the data transferred stat. Guess at it. Multiply that number by 4096 and you have bits/second. Even with 2Gbps fibre channel, you're not going to exceed that data transfer rate because of the spindle speed of your drive. The stat will actually drop toward the end of the drive because the sectors are spaced closer at the beginning (inside tracks) of the drive. On a fast scsi drive, you may get 80000 blocks/sec=327 Mbits/sec.

      If you just store the data on the firewire drive in the first place, you don't have to wait several minutes for the copy. Just press the switch button and use the data.

    2. Re:A Halfway point by earlytime · · Score: 1

      actually,

      the "beginning" of a hard drive is the outer edge. the drive platter has a fairly constant data density, so the outer cylinders(concentric circles) have more sectors(slices of the concentric circle) than the inner cylinders. So on an outer cylinder, more sectors pass under the read/write head per rotation than on an inner cylinder.
      with cdroms the data is written from inside out, which means that the slowest part of the disc is used first. this behavior is probably a remnant of the days when the cd drive was designed to maintain a constant data rate under the laser, rather than a constant rotational speed of the disc.

      --

  2. Just One thing to say by amigaluvr · · Score: 1, Interesting

    Three Words: Firewise Drive

    1. Re:Just One thing to say by spoonist · · Score: 4, Informative

      > Three Words: Firewise Drive

      First, "Firewise Drive" is two words.

      Second, it's "Firewire Drive". That's a good suggestion.

      Third, the poster should investigate gigabit ethernet. Gigabit ethernet can be achieved over copper. Although I believe gigabit ethernet hubs/switches are still awefully expensive. However two inexpensive gigabit ethernet cards can be directly connected (i.e. no hub/switch needed) together with a crossover cable.

      Fourth, if the systems are capable of hot-swapping drives, the poster could plug a drive into the Linux box, load up the data, yank the drive and stick it in the Windoze box.

      Fifth, when writing the data out to a drive initially on the Linux box, use Raid-1 (mirroring) to write to two drives simultaneously. Then yank one drive out of the Linux box and stick it in the Windoze box. No waiting for copying.

    2. Re:Just One thing to say by lowtekneq · · Score: 1

      First, "Firewise Drive" is two words. two words it may be, but its still just one thing : )

      --
      Carpe meam simiam!
    3. Re:Just One thing to say by Polo · · Score: 1

      Just keep track of what machines can read what format disks.

      TCP/IP removes this problem, but the data access will be slower.

    4. Re:Just One thing to say by Anonymous Coward · · Score: 0

      I don't think you can connect gigabit with a cross over cable last I looked. Why? Because there is no gigabit standard that supports half-duplex modes last I looked, maybe that's changed but I doubt it.

    5. Re:Just One thing to say by CMiYC · · Score: 1

      First, its "Windows" not "Windoze."

      If you're going to correct someone for mispelling something, then you should take the time to make sure you do the same.

      (Yes, I am aware of the fact you think its "leet" to call it Windoze instead of Windows. However, mispelling it is stupid.)

    6. Re:Just One thing to say by bafu · · Score: 1

      If you're going to correct someone for mispelling something, then you should take the time to make sure you do the same

      Since he is, in effect, diss-spelling rather than mispelling I'm not surprised that he wouldn't consider the one to be the same as the other. After all, as you point out, he is almost certainly using "Windoze" intentionally for effect...

      (Yes, I am aware of the fact you think its "leet" to call it Windoze instead of Windows. However, mispelling it is stupid.)

      I'd call 'disspelling' more pointless than stupid, but that's splitting hairs, I suppose. Anyway, given how common the practice is, here and elsewhere on the Internet, I'm kind of surprised that this one mild instance bugged you as much as it apparently did.

  3. Fibre channel any good? by aegilops · · Score: 2, Interesting

    You might like to have a look at fibre channel. Not sure of the extent of support in Linux, but Windows has it already supported (as you'd imagine).

    Fibre channel is expensive but notable due to higher %age usage in real-world use. The max throughput on Ethernet (you refer to IP) is somewhere between 30% and 50%, whereas fibre channel can realistically get up to about 80% usage. So, on a 1Gb card, that's not too shabby.

    Anyway, others will have better hands on experience but it might serve as a useful alternative avenue to explore.

    Aegilops

    1. Re:Fibre channel any good? by karnal · · Score: 1

      30 and 50 percent?

      I would think that would completely depend on the hardware. For instance, if you're using a hub (and I doubt that there are any gigabit hubs out there), you don't want to see utilization much higher (in my experience) than 50-60%, because retransmits after collisions on a busy network will cause the usage rate to climb if everyone is still chatty.

      However, we've seen usage rates (peak) on our switches at work as high as 90+ percent. Given, we're pushing 100Mbit from the desktop, through Gigabit links to the Network Operations Center, and from there, into dual or quad gig links to the servers (each).

      Ethernet won't fall apart in my opinion, with good equipment. Get a decent switch, or if you're going from machine to machine, put gig cards in with a crossover.

      --
      Karnal
  4. Gigabit Ethernet by m0rph3us0 · · Score: 4, Interesting

    I would seriously look at gigabit networking. a) its faster, b) it will work. currently there is no standard for the medium the transmits your IP packets so it is unlikely for two IP stacks to work over IEEE 1394. If you can't afford the price of two gigabit nics I would wonder how much your clients time is really worth. (btw, you dont need a gigabit switch because you can use a cross-over cable.).

    1. Re:Gigabit Ethernet by Anonymous Coward · · Score: 0

      *blink*

      can you be more precise and indicate just which laptops would have GIGABIT ethernet? You may want to double check that claim

    2. Re:Gigabit Ethernet by harakh · · Score: 4, Informative


      Dont even need a crossover on most Gigabit NIC's. Intels Pro/1000 cards all have Auto-detect MDI-X.

    3. Re:Gigabit Ethernet by PeekabooCaribou · · Score: 1

      Apple's G4 PowerBooks have had gigabit Ethernet for quite some time, 12" model excluded. So has every PowerMac G4 in the past, oh, two years, I think.

      --
      "I'll say it again for the logic-impaired." -- Larry Wall.
    4. Re:Gigabit Ethernet by Anonymous Coward · · Score: 0

      Newegg.com currently has Intel Gigabit cards for $43 each. $86 + $3 cable + shipping = far less headaches than firewire. And theoretically more speed too.

    5. Re:Gigabit Ethernet by tim1724 · · Score: 1
      currently there is no standard for the medium the transmits your IP packets so it is unlikely for two IP stacks to work over IEEE 1394.

      Huh? What about RFC 2734? It's a standards-track protocol for sending IPv4 over IEEE 1394. Apple has a beta implementation for Mac OS X and I believe Microsoft has an implementation for Windows XP.

      Also see RFC 3146 (IPv6 over IEEE 1394) and RFC 2855 (DHCP for IEEE 1394).

      --
      -- Tim Buchheim
  5. Gigabit Ethernet by DiSKiLLeR · · Score: 3, Informative

    Just use gigabit ethernet.

    My laptop and my cousins laptop both have gigabit ethernet in it as standard. We connected them together with some twisted pair (you will need a crossover cable) and we got 1000BaseT.

    Mmmm, gigabit......

    I am sure you can get 2 gigabit ethernet nics at a decent price, get some cat5e or cat6 or whatever it is you need (gigabit uses all 8 wires; 4 pairs, not just 2 pairs like 10mbit and 100mbit) and you are set.

    D.

    --
    You can tell how powerful someone is by the magnitude of the crime they can commit and be able to get away with.
  6. Perhaps NINNLE is the answer! by Anonymous Coward · · Score: 0, Informative

    I have no idea which distro you're using, but check out Ninnle Linux, which is capable of TCP/IP over Firewire.

    1. Re:Perhaps NINNLE is the answer! by isorox · · Score: 1

      Yes, Stupidmods Linux also supports it

  7. Firewire Drive by mjpaci · · Score: 1

    With Firewire, can't both machines be attached to the drive sumultaneously? Bingo! Mini-SAN.

    --Mike

    1. Re:Firewire Drive by Omega996 · · Score: 2, Interesting

      yeah, but both OSes need to know who's reading and writing to the disk at a given point in time. You'd still need some kind of volume manager software, and they're expensive (in comparison to the hardware).

  8. its possible by jjshoe · · Score: 1

    i suggest you do a little searching

    if your still stuck visit #linuxhelp on efnet. there are several veteran firewire networkers who swear by it over gigabit ethernet. (which to be any good has to be done with fibre)

    state your question in full with many more details then you did for this story. such as machine types what you've tried, what docs you've read. etc etc. and then wait for someone to answer you

    --
    -- botsex is {grep;touch;strip;unzip;head;mount} /dev/girl -t {wet;fsck;fsck;yes;yes;yes;umount} {/de
  9. Ninnle Linux doesn't exist - mods on crack? by Anonymous Coward · · Score: 1, Informative

    How shameful.

    1. Re:Ninnle Linux doesn't exist - mods on crack? by Anonymous Coward · · Score: 0

      Very shameful indeed!

      Somebody should do something about this, and come up with a real Linux distro called Ninnle.

  10. Drive Speed by rawgod0122 · · Score: 1

    If you are using a single drive on either end, getting a faster interconnect will not help you. The drive will be your limiting factor.

    If you have a raid array on each side then you would not be limited by the bandwidth off of the drives, but this would be expensive.I have seen some suggestions of getting a firewire HD and just share it between the machines. I think this would be one of your best bets.

    Do not worry about the connection speed until you get more/faster spindles.

    1. Re:Drive Speed by drsmithy · · Score: 1
      If you are using a single drive on either end, getting a faster interconnect will not help you. The drive will be your limiting factor.

      A single drive, even an old one, will easily saturate a 100Mb network. Heck, brand new 7200RPM IDE drives will top 50MB/sec, which also exceeds the capacity of Firewire.

      On topic, my suggestion would be to invest in some GigE network cards and ideally a GigE switch (although the latter is unnecessary if there's only two machines). Such a solution will simply be a matter of switching from the existing network cards and then continuing as before, with essentially no reconfiguring and/or changed habits necessary and should reap at least a 3-4x performance boost.

  11. What if Firewire is the only option? by cryptor3 · · Score: 1

    I'm in a similar situation; I want to connect a Windows 2000 machine to a Windows XP laptop. However, the reason that I want to use firewire is because I'm a cheap bastard. Also note that in MY laptop, I have not been blessed with Gigabit ethernet.

    I'm not so concerned about the speed (10/100 would be fine); I just want connectivity. If I go the firewire route, then all I need to buy is a $10 1394 cable, right?

    My question is, does Windows 2000 (or Linux, for that matter) support 1394 networking? I haven't seen any definitive answers regarding which OS's support this other than Windows XP, and [sometimes] WinME.

    1. Re:What if Firewire is the only option? by decep · · Score: 1

      Windows 2000 does not and will probably never support firewire networking. Microsoft wants you to use/upgrade to XP and adding features to 2000 is not conducive to switching.

    2. Re:What if Firewire is the only option? by Lovejoy · · Score: 1

      OS X does not support it either, if I remember correctly, but see the post above about Firewire target disk mode.

      --
      Yes, it's a blog. Sorry if that offends you.
    3. Re:What if Firewire is the only option? by Anonymous Coward · · Score: 0

      I understand you don't have gigabit ethernet, but if you say 10/100 speed would be fine - why don't you just use regular ethernet?

      Am I missing something here?

  12. Firewire Disk Mode by Johnny+Mnemonic · · Score: 2, Interesting



    Although not strictly relevant to your question, I feel the need to point out that if you had Apple CPUs with built-in Firewire, you could do exactly what you're asking for. To wit: by restarting one in "FireWire Target Disk Mode", which involves simply restarting and holding down the "t" key, that unit's HD will appear on the desktop of another Mac just like any other externally mounted FireWire HD would.

    Depending on how much you need this feature, you might consider a "switch"--why not use the tools that provide the features you need?

    --

    --
    $tar -xvf .sig.tar
    1. Re:Firewire Disk Mode by Polo · · Score: 1

      This is very useful for Macs (just remember to drag the disk to the trash before unplugging!)

      However, for his other systems, I don't believe they would be able to read the disk since it's in apple's HFS format.

  13. firewire drive by marimbaman · · Score: 1

    What better excuse to buy an iPod? :)

  14. Theoretical bandwidth comparisons by pbox · · Score: 3, Informative

    FireWire 400 Mbit/s ~ 50 MB/s
    Speed close to theoretical speed. Direct PC to PC might possible. Guesstimated speed around 40-45 MB/s.

    FireWire 2(?) 800 Mbit/s ~ 100 MB/s
    New version, Apple already supports on high-end, but likely there is no support for PC, or even interface, or very expensive. Speed close to theoretical speed. Direct PC to PC might possible. Guesstimated speed around 80-95 MB/s.

    Drive Swap 1064 Mbit/s ~ 133 MB/s
    Well, you will have to csave on PC, load on the other one. That makes speed at least half of theoretical one. Drives are not physically capable of these speeds, only around 40MB/s sustained. Guesstimated speed around 20 MB/s.

    Gigabit 1000 Mbit/s ~ 100MB/s
    Direct PC to PC possible. You need 64 bit PCI cards to get colse to 100MB/s. Guesstimated speed around 90-95 MB/s.

    Ethernet 100 Mbit/s ~ 10MB/s
    Cheap, easy, cheap, but slow. No special needs. Guesstimated speed around 9-10 MB/s.

    USB2.0 480 Mbit/s ~ 60MB/s
    Cheap, high(er) CPU overhead. Need PC-to-PC host bridge (might be hard to locate). Guesstimated speed around 40-50 MB/s.

    --
    Code poet, espresso fiend, starter upper.
  15. Channel bonding by Anonymous Coward · · Score: 0

    Hi,

    You could look into channel bonding as an alternate (or complementary) technology for faster file transfers. Channel bonding is the practice of using N ethernet cards as if they were one card. Although the gains are not linear and depend quite a bit on the quality of the card, it does speed up communications.

    Bonding 100mbps ethernet cards is how we sped up intra-beowulf communications at one of my previous jobs. This was for scientific communications, and data was continuously transferred between nodes. All you need (in addition to the ethernet cards) is a good hub/switch. We used a cisco hub with two backplanes (can't remember the model number).

    You can bond gigabit ethernet cards as well. If you can afford it, this will provide greatly increased communications speeds.

    Getting bonding to work may take a little digging around on google. Just search for "bonding site:scyld.com" for information on donald becker's site or "ethernet bonding"/"gigabit bonding" for general information.

    Regards.

    1. Re:Channel bonding by Wise+Dragon · · Score: 1

      But can you do channel bonding between linux and Windows? Hmm... I don't think so.

  16. answers, by itzdandy · · Score: 4, Informative

    Linux Firewire TCP/IP is possible and their is a kernel patch for 2.4.18 and its built into 2.5.x and will be a feature of 2.6 -BUT-
    this is TCP/IP ONLY!, that means that windows file sharing will not see across this medium, you will have to use IP (//192.x.x.youipwhatever/share) which may not be a problem for you but it's something to consider.

    This does work well, i have a firewire400 link between a windowsXP machine and gentoo linux and it works very well. I use NFS for file sharing as i feel Windows is the "guest" on my network and should pay homage to the exsisting linux machines :)

    i do get very close to optimal speed accross this link for file sharing. i have susstained ~45Megabytes/second transfering large video files. on these same machines with 100 speed networking i get about 9.5Megabytes/second and with gigabit i can get about 35Megabytes/second, but only on large files.

    The firewire link is by far the fastest but i think the gigabit is being help back by immature drivers on the linux side.

    good luck.

    1. Re:answers, by Gothmolly · · Score: 0, Troll

      You obviously have no idea how Windows networking works. It's NetBIOS (the APi, not the fscking protocol) over TCP/IP. Which means if you have IP connectivity, and know what you're doing, it will work just like any "normal" net connection. Try googling for such arcane tools as samba, wins, and lmhosts. ...fscking AOLer...

      --
      I want to delete my account but Slashdot doesn't allow it.
    2. Re:answers, by misterplow · · Score: 1
      Would someone who has successfully networked a Windows + Linux machine over firewire . . . (or 2 Linux machines for that matter)

      Would you please document somewhere some of the commands and/or settings files you modified to get it working?? If I can find enough information to get it working myself I'm eager to document and contribute that howto info to anywhere/one who could benefit from it. TCP/IP networking over Firewire would be COOOOL!

      I wouldn't say that I'm a newbie (I got slashcode compiled and running on my home machine if that helps you), but honestly -- I've looked all over google, slashdot, etc. and not found the specific commands or files that I would need to change.

      Everything I've found essentially says "configure your FW card for IP" . . notta big help.

      Thank you in advance.

  17. It's not always about the network. by billn · · Score: 1

    You can perform this transfer in one swoop, with a single packet transfer with the mother of all MTU's.

    Removable, hot swappable drive bays in each machine, and then use a sneakernet to transfer your data.

    Not familiar with sneakernets? That's where you use your sneakers as the transfer medium. Transfer time algorithm works out to (size of drive / walking distance) * number of trips.

    In short, use removable drives, dummy, and walk between machines.

    --
    - billn
  18. use gig Ether by rakerman · · Score: 2, Informative

    400 Mbps FireWire makes it sound like you will get 4x better than 100Mbps Ether, but in practice you won't.

    Gig Ether will do what you want.

    If you still want more info on FireWire networking, visit my research page.

  19. Use multiple 100Mbit/sec Ethernet cards by Wise+Dragon · · Score: 1

    I realize everyone's been telling you to use Gigabit Ethernet, but if cost is an issue, this may work better, and it's WAY cheaper. It may not work well for you if you have a lot of PCI slots in use already. Just put extra ethernet cards in each machine. See this cringely article for more information: http://www.pbs.org/cringely/pulpit/pulpit20011227. html

    OTOH, this may not be very effective between linux and windows.

  20. Re:Channel bonding supported by NINNLE linux by Anonymous Coward · · Score: 0

    You may want to check out NINNLE linux, which does just this.

  21. Re:answers, IP by Anonymous Coward · · Score: 0

    of course, if you edit your hosts file, you wont need to use the full ip.
    windows nt: c:\winnt\system32\drivers\etc\hosts
    nix: /etc/hosts
    192.168.x.x name

  22. Easier way to do read timings by Andy+Dodd · · Score: 1

    hdparm -t /dev/sdX

    Times a 64MB read from the drive. Most Firewire drives will cap out around 23 MB/sec (Megabytes, not megabits) The same drive/enclosure using USB 2.0 capped out at 12 MB/sec for me.

    In my case: 7200 RPM 120GB Maxtor HD in a comboe 1394/USB2 enclosure fron www.newegg.com - Part number is ME720UF.

    --
    retrorocket.o not found, launch anyway?
  23. Real-world caps by Andy+Dodd · · Score: 3, Interesting

    Note that these will all likely be capped to 35-40 MB/sec if you have IDE drives, maybe a little bit more for 7200 RPM SCSI.

    So far I've found the 1394 networking support for Linux to be pretty slow... For some reason it seems to put the interface into 100 mbit/sec mode.

    USB2 can't even come CLOSE to theoretical max throughput. I have a combo USB2/1394 drive enclosure. In 1394 mode, hdparm -t gives a result of approx. 23 MB/sec. In USB2 mode, the same benchmark gives a result of 12 MB/sec. (For a 64MB sustained read)

    --
    retrorocket.o not found, launch anyway?
  24. Do you ask the right question? by Anonymous Coward · · Score: 0

    I think that a faster network can't help you more than a factor 2. 100 Mbit ethernet gives you 10 Mb/sec throughput while a modern harddisk has a maximum sustained throughput of about 20 Mb/sec. Switching to a faster network moves the bottleneck from the network to the disks. (And if you upgrade to RAID, the PCI bus will become the bottleneck.)
    You should consider:
    a) to accept the few minutes of waiting
    b) change the way you work (work smarter)

    You could make the data on your linux machine available to your windows machine by setting up samba (windows file sharing for linux) or NFS (with an appropriate windows client). If you realy need the files locally on your windows box you could use a protocol as rsync to keep an relatively up to date copy, which can be updated in a matter of seconds when needed.

  25. Re:Channel bonding supported by NINNLE linux by Anonymous Coward · · Score: 0

    Wow! Another Ninnle fan! I knew the Ninnle movement would grow...thats why I started it. Someday, it will be as legendary on /. as Soviet Russia, goatse, PROFIT, Beowulf clusters, and all the other running jokes that make /. such a great community to be in!

  26. Re:Mods don't exist - Ninnle uncrackable! by Anonymous Coward · · Score: 0

    What's so shameful about Ninnle? You should give it a try, and seriously enhance the security of your system. Ninnle does it all!

  27. TCP/IP over Firewire by Wise+Dragon · · Score: 1

    I know it's been awhile, but here's a link that might be of service:

    http://web.mit.edu/kkeville/www/firewire/

    also try googling for "ip over firewire"