Slashdot Mirror


Mount Remote Filesystems via SSH

eval writes "Ever wanted secure access to your files at work or school, but didn't have the necessary permissions or were thwarted by a firewall that allowed ssh access only? The SHFS kernel module allows you to mount directories from machines to which you have shell access. File operations are executed as shell commands on the server via SSH (or rsh). Caching keeps it reasonably fast, and remote commands are optimized based on the server's OS."

43 of 269 comments (clear)

  1. Great! by Anonymous Coward · · Score: 5, Funny

    Now my web hosting company will probably take away ssh access. Thanks Linux hackers!

  2. LUFS! by Santabutthead · · Score: 5, Informative

    Big deal! I've been doing this for close to a year now, with lufs (http://lufs.sf.net). It's not really the easiest thing to automate but it sure works for day-to-day computing.

    1. Re:LUFS! by TTimo · · Score: 5, Informative

      Well .. lufs is the main player in userland filesystem stuff really. It has had sshfs functionality for months. Very slick.

      The difference seems to be that SHFS does some amount of caching, which lufs doesn't do afaik. This has a good chance to improve performance.

    2. Re:LUFS! by clump · · Score: 5, Informative

      LUFS deserves a lot of credit. I now use LUFS's SSHFS to mount my remote file volumes, whereas I previously used a tunneled NFS setup. The latter is a bear to setup but wonderful when operating. LUFS's SSHFS on the other hand requires zero setup on the server, no portmapper on either client or server, and is much easier to automate and control.

      I am looking forward to trying SHFS, but currently very much enjoy LUFS and the hard work put in by its authors. And that means your work on it too, TTimo ;)

    3. Re:LUFS! by Erik+Hollensbe · · Score: 3, Informative

      It needs it because it's a filesystem driver. Somewhere along the chain, LUFS needs it too.

      BTW, if you really want to play with good ideas for kernel modules, get the cryptoAPI patches and compile openSSL to use them (which requires more patches, IIRC).

  3. Good idea but... by Vorgo · · Score: 3, Insightful

    This is a good idea, however there is one problem with the way that the problem is presented above:
    If you're at work or school, are you really going to be able to insert a kernel module on the machine you're on? Generally I would think that you do not have sufficient permissions on the local machine.

    --
    A new feature is just a bug waiting to happen. And vice versa.
    1. Re:Good idea but... by skurken · · Score: 5, Informative

      No I think it's ment to be used the other way around. This way, I can mount my UN*X school account that allows shell access on my Linux computer at home (where you usually have root access). /S

  4. Another option by Guiri · · Score: 5, Informative

    Just type fish://user@host in your Konqueror location bar ;). It works great!

    1. Re:Another option by mosch · · Score: 4, Insightful
      fish is a great idea, implemented in the completely incorrect location. This kernel module gets it right.

      Filesystems should be handled by the operating system, not the window manager.

    2. Re:Another option by oliverthered · · Score: 3, Interesting

      That's a shortfall of the kernel not KDE.

      Why arn't all the kioslave protocols in the kernel?

      camera:\\
      ftp:\\
      http:\\
      fish:\\
      etc....

      --
      thank God the internet isn't a human right.
    3. Re:Another option by sfraggle · · Score: 4, Insightful
      While I'm not a huge fan of microkernels, this is one area where a system like the Hurd has advantages over Linux.

      In the Hurd all the filesystems are done by userspace programs called "translators". So to access your local filesystem you have an ext2 translator which accesses it. You can write your own translators - I believe they have a system to access remote systems via FTP.

      Both fish/gnome-vfs and the kernel module system seem wrong to me. With kernel modules you have to be root to load them, but on the other hand its bad to be remaking the wheel by rewriting the filesystem in user space (plus, it only works with programs that are designed to use it).

      It would be nice if the kernel module was added to the main kernel and offered as a "standard" system where nonprivileged users can mount their own filesystems from userspace daemons - Linux is kind of paranoid about non-root users mounting FSes. It would appear to provide the advantages that the Hurd approach brings, while keeping the higher performance of a monolithic kernel (having all FSes in user space like Hurd does seems like a bad idea performance wise)

      --
      were you expecting to see a sig here? perhaps you'd rather see the inside of an ambulance!
    4. Re:Another option by Elbows · · Score: 3, Informative

      Implementing all of them in the kernel would bloat the kernel a lot. What KDE/Gnome should have done (and what LUFS, mentioned in another thread, seems to do), is have a small kernel module that calls a userspace daemon. All of the protocol code stays in userspace, but the kernel module makes the filesystem accessible to all programs. The overhead from the context switch doesn't matter when you're dealing with remote filesystems.

      It's a pretty slick idea, actually... maybe it will be integrated into the major DEs someday.

    5. Re:Another option by Spy+Hunter · · Score: 5, Informative
      I would say you're right, except the kernel does a lousy job of implementing filesystems in a user-friendly way. KDE IOSlaves are so much cooler for several reasons:
      1. They use URLs everywhere, which makes it easy to access local and remote files anywhere using any protocol from any application.
      2. New filesystems can be installed and activated by the user, you don't need a kernel module.
      3. You don't have to mount anything anywhere.
      4. Non-filesystem like protocols such as HTTP and POP3 can be easily implemented as IOSlaves and then used from any application.

      These features make IOSlaves much cooler than kernel filesystems IMHO.
      --
      main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
    6. Re:Another option by Nucleon500 · · Score: 3, Insightful

      LUFS or FUSE would be the module you're talking about, which allows you to do filesystems in userspace, and can let users mount them. So Linux can do all three: kernel modules for lean, fast, "real" filesystems, LUFS or FUSE for "exotic" filesystems that shouldn't be in the kernel, and kioslaves and gnome-vfs for those who like reinventing the wheel and want to further bloat their favorite GUI library.

    7. Re:Another option by Spy+Hunter · · Score: 4, Interesting

      LUFS is pretty neat, but I think IOSlaves are nicer. LUFS is still tied to the Unix filesystem, which is great for managing local files, but was never designed for anything else. Creating magic directories that cause gnutella searches to be performed is not my idea of a nice interface. IMHO, automount has always been an ugly kludge, and mapping URLs onto the Unix filesystem is not a great solution. How would you handle a URL like:
      http://user:password@host.com/search.pl?param=va lue&param2=value2
      And how would you handle HTTP caching? How would you send POSTs and other types of HTTP requests? Even if you could add all these features to LUFS, it would start getting more and more unweildy to use. And that's just for HTTP.

      --
      main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
    8. Re:Another option by Nucleon500 · · Score: 3, Insightful
      I agree the gnet filesystem is a big stretch, and automount is somewhat ugly. I think it's an issue of the right tool: LUFS should only be used for things which directly map to files. Meaning FTP, a subset of HTTP (just downloading, no caching, POST, etc.), Samba, scp, etc. And it's best for connection oriented things where mounting makes sense (otherwise, just use wget or your browser). Anyone who writes a web browser depending on LUFS should be shot, but writing one with IOSlaves or gnome-vfs is cool.

      Mapping most urls to files isn't a great solution, but neither is mapping files to urls. How do you get a directory listing? How do you change permissions? Consider FTP: do you make a new connection for each URL, or do you keep the connection open for a while in hopes of getting another? Mounting makes sense when there's overhead in establishing a connection, be it parsing a filesystem or making a remote connection.

      I guess what I'm saying is that a URL library (IOSlaves and gnome-vfs) is nice to have, and a file system (LUFS, FUSE) is nice, but they shouldn't be one and the same. The URL library can be used for browsers, downloaders, etc, but the filesystem and only the filesystem should be used for naive tools that want to run on a local filesystem.

    9. Re:Another option by 73939133 · · Score: 4, Interesting

      No, it's a shortfall of KDE developers: instead of spending time on writing Konqueror modules, they could be writing the equivalent kernel modules.

      But that isn't really anything new: a lot of the KDE effort could be written as more independent, stand-alone functionality, useful to lots of non-KDE software. Instead, KDE produces tightly integrated C++ modules that only work if you are running a large amount of KDE support infrastructure.

    10. Re:Another option by BusterB · · Score: 3, Insightful

      Then again, KDE also works in *BSD, Solaris, HP-UX, and even Windows, to name a few. Writing kernel modules that work for every one of these systems would be a bit of a hassle, no? It might not even be possible in a few of these.

      If someone were to implement all of these systems in the kernel, then KDE/GNOME, etc. can already them. In the mean time, this seems to be the best compromise between functionality and portability.

    11. Re:Another option by be-fan · · Score: 3, Informative

      KDE is an application framework. If you make things independent of each other, you loose a lot of the consistency and integration that makes an application framework so nice to program for in the first place.

      --
      A deep unwavering belief is a sure sign you're missing something...
  5. You might want to have a look at... by yanestra · · Score: 5, Informative

    avfs and lufs are much more common solutions to the "mount userland filesystems" problem. Yet, avfs makes it easy to construct your own whatever-you-want filesystem.

    1. Re:You might want to have a look at... by SCY.tSCc. · · Score: 3, Informative

      Don't forget FUSE (filesystem in userspace) by the same author as AVFS. It recently hit version 1.0 BTW.

  6. Re:If you don't have permissions... by wowbagger · · Score: 5, Informative
    f you don't have permissions to use network connections other than SSH, are you going to have permissions to mount a filesystem on the computer?


    Could be: for example, where I work I'm behind a corporate firewall, but I have admin rights on my workstation. As a result, I could very easily mount a remote file system via SSH. In fact, since I administer an FTP server that is outside the firewall, being able to mount it as a file system in a secure fashion would be quite useful.

    Just because network ingress is controlled does not mean that your workstation is controlled. In many ways, this is no different than you burning a CD of your files at home and bringing that into work - the infection/cracking risk is the same. If you are not allowed to mount an external file system then you should not be allowed to mount a local file system.

    However, just because you CAN access your home machine does not mean you SHOULD.
  7. What's with the... by Xpilot · · Score: 5, Funny

    ..margerine box at the bottom? Is it what the programmers ate during the creation of shfs? Like that apocryphal Java-drinking sessions at Sun? Does margerine have magical caffiene-like properties too?

    --
    "Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." -- Linus Torvalds
  8. Some other project does this already by vlad_petric · · Score: 4, Informative
    LUFS - userland filesystem. It's a userland "teleportation" of the VFS infrastructures (a kernel module sends all the queries to a userland daemon, which takes care of the protocol, etc).

    The advantage of this approach is that adding a new filesystem type implies modifying a user-space daemon, not the kernel. LUFS includes, besides sshfs ftpfs, gnomefs, and gnutellafs and a few others

    --

    The Raven

    1. Re:Some other project does this already by mosch · · Score: 4, Informative

      Not really. LUFS can access a machine which you have sftp access to whereas this project allows you to access the filesystem of a machine that you have true shell only access to, as is common especially in some university environments.

  9. Re:what's the point? by kcurrie · · Score: 5, Insightful

    What are you talking about? This makes a remote filesystem appear local, and all local commands work accordingly (i.e. edit a file, play an mp3, etc). With sftp you'd have to sftp a file down to do an operation on it, and then sftp it back up again afterwards, etc/

    I've used lufs with the sshfs and it works great, I've even done compiles on a remote filesystem this way-- you simply cannot do that using just sftp/scp.

    --
    -- I speak only for myself.
  10. Yet another option by BlueEar · · Score: 5, Interesting

    This seems to be beta quality code. Thus you might want to try Secure NFS via SSH Tunnel, which provides, accoding to the author Secure NFS (SNFS) via SSH2 tunneling of UDP datagrams, as suggested in the SSH FAQ.

    --
    A religious war is an adult version of a fight over who has the best imaginary friend
  11. Another option with SMB by aldjiblah · · Score: 4, Informative

    An ssh connection forwarding the remote port 139 to 127.0.0.1:139, and then doing smbmount to //127.0.0.1/<mountpoint> - works great, and is practical considering Samba is often already running on the remote side.

    --
    sig sig sputnik
  12. Or... by Greyfox · · Score: 4, Insightful

    Create VPN with freeswan or ppp over ssh, mount remote host from VPN.

    --

    I'm trying to teach myself to set people on fire with my mind... Is it hot in here?

  13. Re:My beef with firewalls.. by Ian+the+Terrible · · Score: 5, Funny

    Yeah, that's pretty stupid.

    It's a bitch to get SMTP to work over 23, too.

  14. Nothing new, been done before by cce · · Score: 4, Informative
    LUFS (Linux Userland Filesystem) already provides a nicely-developed interface to allow for userspace programs to implement filesystems over exotic protocols like SSH, FTP -- even Gnutella. Another project, FUSE (Filesystem in Userspace, part of AVFS) performs a similar task.

    Moreover, the SHFS project website admits that it's "partially based" on FTPFS; but the FTPFS website says it's now obsolete and recomends using LUFS instead.

    So the question: why did this merit an article? SHFS is just a proof-of-concept project for some kid's operating systems class, and I'll bet that despite the warning ("Warning: This is beta quality code. It was not tested on SMP machine. Backup data before playing with it!") tons of Slashdotters -- most without any kernel-hacking experience -- will have downloaded and perhaps installed it before I finish typing this post. This is dangerous.

    So -- if you want to play with (and implement your own, it's remarkably easy!) fun filesystems, try LUFS or FUSE instead.

    1. Re:Nothing new, been done before by Donald+Knuth · · Score: 4, Insightful

      Why does any software announcement that's posted here always bring out a bunch of elitist trolls? Oh, that's right - because it's not good enough unless it's yours.

      Do you know the authors of shfs, their ages, and what classes they're taking? Have you even downloaded the driver? Compiled it? Actually used it? Have you tested it, exprienced crashed, and therefore empirically come to the conclusion that it's "dangerous"? Or do you just like to play the role of Slashdot nanny?

      Wait, don't answer that. I really don't care.

      *You* should care, however, that you come off looking like a frustrated little prick by shitting on other people's work - for no reason other than to hear your own voice. Nobody wants to read your little pride-ridden, hyper-competitive, and overtly paternalistic little diatribes, no matter how much you think you enjoy writing them.

      Anyway, lighten up and take the post for what it is. And in the future, if you can't say at least one reasonably positive thing about someone else's hard work - do the world a favor and just shut the bloody fuck up.

  15. Re:what's the point? by Anonymous Coward · · Score: 3, Insightful

    I just have to reply...

    Having come from the world of satan (windows) and now firmly embedded in the world of linux I feel qualified to make this observation:

    This is EXACTLY "what is wrong with open source". Not the software, not the thousands of dedicated people who contribute countless hours doing something as a labor of love. But the attitude that usability and ease-of-use are STUPID and not worthy of anyone's time.

    To paraphrase: "If you're too lazy to do it using the extremely flexible but very arcane tool we have developed for you, you don't deserve to use linux."

    Aside from the fact that this actually makes it easy to do something WITH the files, it also makes it substantially easier to get the files in the first place.

    Also, anyone that knows anything about security knows that one of the main reasons security is circumvented is because of inconvenience. This removes that possibility.

    Seriously, if you don't want to use it fine. But why take shots at it for the sake of looking l33t on /.???

  16. Better Implementation idea... by Polo · · Score: 4, Interesting

    I think a better implementation of this might use the sftp protocol on the server side. This has been recently implemented with SSH v2. It's a subsystem within SSH (sftp-server) that supports all the common filesystem operations (open, close, read, write, seek, stat, etc...).

    This is the protocol that scp uses to read and write files and is already part of ssh.

  17. macos x by hachete · · Score: 3, Interesting

    It would be nice if this worked with Macos X and apple-type file systems. SSH works well on Macos X and I could do with an alternative to webdav and netatalk. Yes, I know that there are "issues" with apple file resources, but I wish they would just *disappear into* the shell so I didn't have to worry about them :-)

    ah well. I can dream.

    h.

    --
    Patriotism is a virtue of the vicious
  18. big deal by F2F · · Score: 4, Interesting

    we've been doing this with Plan 9 since 2000.

    from the ssh man page:

    Sshnet establishes an SSH connection and, rather than execute a remote command, presents the remote server's TCP stack as a network stack (see the discussion of TCP in ip(3)) mounted at mtpt (default /net), optionally posting a 9P service descriptor for the new file system as /srv/service.

    1. Re:big deal by ceswiedler · · Score: 3, Funny

      Everything that is cool, in a hacker sense, you can do with Plan9. You just can't do anything that's cool in an ordinary sense.

  19. Re:Neato by tzanger · · Score: 4, Informative

    I dunno, I run my NFS over IPSec and it seems to work just fine. A simple script to block any NFS access that isn't coming in on an ipsec interface and you're all set. rpcinfo and some awk, that's all it takes.

  20. Don't you remember hacking a school lab? by iamacat · · Score: 5, Interesting

    The authors might not have admin access to the server to configure secure NFS. Or for that matter an installed compiler to install samba and tunnel it over ssh. Just shell access and instructions on using pine. And a sysadmin who will need a shot of brandy after hearing about students/employees running a remote filesystem. He might even be right, considering how NFS lets clients pick a userid to access files or uses inode numbers as handles.

    There are a lot of projects like this. Linux used to have term and later a user-level PPP daemon to forward socket calls over a serial line, when the admin could have easily installed the real thing. At one point I had to write a rather complicated tool to forward incoming requests from the internet to a host inside an http-only firewall because that was the only way to test it with a client running on a cell phone.

    Now if someone wrote a daemon to run PPP (or PPPOE) over an HTTP proxy, we could all just use it and stop reinventing the wheel.

  21. PPP over an http proxy by Guiri · · Score: 4, Informative
    I used to do that at the university. Here is gow:

    1. Get Http tunnel. You have to install it inside the network with the proxy, and in another machine on the internet (outside that lan).
    2. Create a tunnel from the first machine to the ssh server of the second machine (http tunnel creates a socket).
    3. Do ssh-keygen on the first machine, and copy the .ssh/indentity.pub file from the first machine to .ssh/autorized_keys on the second host. That way you can login without password.
    4. Now configure both machines to do PPP over ssh. I wrote the explanations here , look at the comment with a subject saying "PPP over SSH". It's in spanish, but you can translate it with babelfish, and at least you can get the scripts from there. If you don't manage, look in google for "ppp over ssh" or "firewall piercing".
    5. Configure the first machine to use the second host as the default gateway (through this new ppp network device), and configure the second machine to do NAT for the first one.
    There you go, you have unrestricted access to the internet through the most firewalled network in the world, and through a proxy ;).

    You need to have root in both machines, but is worthwile, trust me! ];>> The first time it could look a little bit complicated, but afterwards you can just create a script to do the whole thing, so next time you'll only have to do "./create_tunnel" on the first machine to do the whole process.

  22. Shell over SMTP by NoOneInParticular · · Score: 5, Funny
    Big Deal! Back in my day, we ran a filesystem over smtp: sent your commands per email, have it executed remotely, and send the results back to the sender. Imagine:

    To: user+bash@host.com

    ls /usr/bin

    And get the result back by email. The tricky part was to do (insecure) copy: cat piped to uuncode etc.

    To paraphrase: it's not really the easiest thing to automate but it sure worked for day-to-day computing

  23. Everything is a file... by Ayanami+Rei · · Score: 3, Insightful

    OR... a pipe or socket.
    But as to not being seekable, etc: that's why we have the distinction in the first place!
    For things that aren't like a file, they shouldn't be a file. We need appropriate metaphors for things that are not flat and seekable.
    Duh character devices aren't seekable, that's the POINT. In fact, upon further reflection: the reason why a CD-filesystem is a bad idea is because only one process can use it at a time!!! (^_^;;;) Making some resource a filesystem implies that it can deal with at least concurrent access, right? I'm not saying you can't do it, but I don't think it's a good idea because it doesn't really get you anything special.
    After further reflection, the cd-extraction "device" almost begs to be a character file. Send it an ioctl (with a silly little command line util), then start slurping it into oggenc. The "cdparanoia" device... you might use it like this:

    $ cddactl --extract 1[1:30]-3[4:00] /dev/uld/cdda:scd0.raw /dev/uld/cdda:scd0.a

    oggenc [ some stuff ] /dev/uld/cdda:scd0.a

    In this case, cdda:scd0.raw is the file that accepts the ioctls, and cdda:scd0.a is a magically created char device corresponding to your request to read a range from the CD, and data will only start appearing from there once all other requests have been fulfilled (otherwise you get EAGAIN on read()). I pretend that uld is a filesystem exported by a magic kernel module that allows user-space libs to do some grunt work and provide device files as needed.

    Perfect way to queue up a bunch of jobs; just spawn a bunch of oggencs in parallel on the file requests, and let the kernel wake up the right process to do it.

    Or you could do something evil like this:

    $ streamctl --push /lib/modules/streams/vorbisenc.so -oq=5,oggwrap /dev/uld/cdda:scd0.a

    Then:

    $ cat /dev/uld/cdda:scd0.a > "KMFDM - Waste.ogg"

    That is definitely in the spirit of Unix!

    And whatever happened to STREAMS? That would be a perfect way to take cdparanoia + ogg and roll it into a OS abstraction.

    I'm talking about generic "uber-character devices" for asyncronously and syncronously available data, and a way to get at their individual ioctls without having to write whole C programs.

    Some kinda standard for that would kick a large amount of ass, especially if its easily accessible from user space.

    --
    THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
  24. Re:Will this hurt ssh? by sgifford · · Score: 3, Interesting

    My experience has been that if a network is configured in an idiotic way (such as allowing telnet and not FTP), it's not because its operators have made careful and well-considered decisions about what to allow and disallow, but simply because they're idiots. That sort of eliminates the whole ethical dilemna. :-)