WebDAV with a Quota?

gik asks: "I'm in the need for a quota-managing, multi-account capable, class-1 WebDAV server (for remote file storage for clients). I've been researching WebDAV for a long long time now, and have only found one all-in-one implementation: Xythos webfile server, which is a very costly (but a very good) solution. I know that some online storage companies use a hacked Apache, but as anyone who's worked with WebDAV knows, doing this with Apache can be hard. So I'm asking: Does anyone out there know of a good WebDAV server with (hopefully) quota management that is as reliable and free as Apache? Oracle's IFS, Novell Netware, and the like are acceptable as possible candidates."

Comments

IIS

[RzUpAnmsCwrds]

IIS?

Re:IIS

[NutscrapeSucks]

Shouldn't be modded as funny. Unlike Apache WebDAV, IIS impersonates users and therefore WebDAV files have the expected permissions/ownership. However, I don't know how cleanly quotes are handled.

(IIS being designed for "intranets" while Apache is designed for internet sites.)

Re:IIS

[RzUpAnmsCwrds]

"However, I don't know how cleanly quotes are handled."

They are handled just like they are with a local user or with CIFS. Sharing a folder with WebDAV/IIS is functionally equivilent to sharing it with CIFS.

Enabling WebDAV on Apache

[shufler]

This article details adding WebDAV functionality to Apache. I'm not quit sure what is so hard about it. Works in Windows, Linux, and OS X.

Re:Enabling WebDAV on Apache

[Saeed al-Sahaf]

I am ignorant. So, in all likelihood, I speak from my ass. But...

Reading the link, sounds a lot like (groan) FrontPage... Since I know this can not be the case, could one of the learned Slashdotters explain to me what WebDAV means?

Re:Enabling WebDAV on Apache

[shufler]

Reading the link, sounds a lot like (groan) FrontPage... Since I know this can not be the case, could one of the learned Slashdotters explain to me what WebDAV means?

Reading the link provides the following explaination:

In order to simplify the way you update websites, WebDAV was invented. Web-based Distributed Authoring and Versioning (WebDAV) uses extensions to the existing HTTP protocol to enable multiple users to manage and modify the files in a remote system. Using suitably enabled clients you can view, open, edit and save files directly into the filesystem of the Web site as it were of a remote website.

FrontPage does has WebDAV capabilities built-in, but I (and probably you) doubt this is the main reason someone would use FrontPage.

Bottom line: The purpose of WebDAV is to give multiple people the ability to edit and publish remote content.

Re:Enabling WebDAV on Apache

[Curtman]

Using suitably enabled clients you can view, open, edit and save files directly into the filesystem of the Web site as it were of a remote website.

Wow, FTP.. Again.

Re:Enabling WebDAV on Apache

[Saeed al-Sahaf]

But how is this different that FTP???

Re:Enabling WebDAV on Apache

[shufler]

But how is this different that FTP???

First off, who cares? The topic of this story has nothing to do with FTP. It has to do with finding a webserver which allows an easy setup of WebDAV.

That said, this is different from FTP as it uses HTTP extensions to operate. Let's remind ourselves what FTP does -- tranfers files. That's it. In order to edit remote files, you would have to download the file locally, make your changes, and upload it again. The FTP protocol does not allow the same functionality as WebDAV, until you start using an FTP client which has features that automate this process -- you select a file to edit, it downloads it to a temporary location, opens your default editor for the file type, and allows you to send back the edited file when complete. Most GUI FTP clients do this, which is probably how the confusion around "Isn't this just FTP?" has arrisen.

Note that I'm not advocating the use of WebDAV, and for that matter, I have never used it myself and have ABSOLUTELY NO IDEA HOW IT DOES WHAT IT DOES. All that I originally intented to show was that a simple Goole search returned the above cited article (context) as the second result, once again proving that another Ask Slashdot question could have been answered by simply typing "google.com" instead of "slashdot.org" into one's browser.

Re:Enabling WebDAV on Apache

[jeif1k]

It's not really any harder than configuring other Apache modules.

However, given the importance of WebDAV, I think mod_dav should become a standard, default part of every Apache install; the only thing users should have to do to enable access is set up passwords for their users.

Re:Enabling WebDAV on Apache

[Geoffreyerffoeg]

It's not very different from FTP; it's just done using HTTP. This may be easier for the user. This is the converse of using FTP in a browser to download an HTML page, which shows in the browser as if HTTP were the protocol.

Actually, it has the benefit of allowing more access controls than traditional FTP - hence the name, "authoring and versioning".

Re:Enabling WebDAV on Apache

[beerits]

Adding WebDAV to Apache is not hard but mod_dav has no quota option. This is what the original question was about.

Re:Enabling WebDAV on Apache

[Anonymous Coward]

1. You can edit files in place without downloading them first
2. WebDAV implements file locking so you don't have to worry about two people writing to a file at the same time
3. It's more convenient, because many operating systems allow you to treat a WebDAV folder the same as a normal network share (ie. iDisk, which is implemented on top of WebDAV)

Patch for quotas

[Anonymous Coward]

Here is William A.Carrel's Patch patch for Apache 2. setup info

Quota

[jrockway]

Regarding quota, can you not set up the server to save files in the user's homedir (like in public_html or something)? Then the quota will be managed by the underlying OS (and should be trivial to set up).

Re:Quota

[LiENUS]

linux quotas are managed by owner not by location, files created by apache are owned by the user the apache daemon runs as, there is a mod_setuid or something like that that may assist you however.

Re:Quota

[jrockway]

I would think Apache would know how to do something as a user instead of "www" or whatever it runs at. Otherwise a nice perl script in my homedir could wipe out everything owned as "www" (like the main webpage). I am a non-expert about about Apache, though, so I could wrong. (Likely you can't run cgi scripts unless they're owned as www.)

Re:Quota

[LiENUS]

Apache runs setuid WWW unless its run setuid root then it cant write to files owned by other users, its a basic unix fundemental.

Re:Quota

[Bitsy Boffin]

Otherwise a nice perl script in my homedir could wipe out everything owned as "www" (like the main webpage).

Yep, welcome to the world of shared web hosting.

Re:Quota

[Foolhardy]

Unfortunately, from the docs for Apache mod_dav:

In order for mod_dav to manage files, it must be able to write to the directories and files under its control using the User and Group under which Apache is running. New files created will also be owned by this User and Group.

There seems to be no support for having new files created as the user that logged on, far as I can tell. mod_dav does not handle authentication of logons itself, you have to use mod_auth_digest or mod_ssl, so it may not even be aware of what user is logged on.

I get the feeling that Apache was designed for providing uploads to clients only, not full scale IO, and that mod_dav is a bit of an afterthought for trusted users.

BTW: This mod (and Apache) specifically provide no support for quotas:

Another possible denial-of-service attack involves a client simply filling up all available disk space with many large files. There is no direct way to prevent this in Apache, so you should avoid giving DAV access to untrusted users.

Re:Quota

[LiENUS]

http://httpd.apache.org/docs-2.0/mod/mod_suexec.ht ml

Just thought I'd note if you use a dav cgi script you can potentially utilize that to achieve quotas, depending on how much you trust your script you could make the cgi script setuid root and then authenticate against /etc/shadow (bad idea on non ssl connection btw) and from there immediately setuid to the user you authenticate as. From here standard OS quotas will indeed take effect.

The problem is if someone finds a flaw in your script then they can root your server. You would also need a farely complete WebDAV implementation in perl or whatever your using for cgi (note you can use C if youd like), perhaps the best way to accomplish this is use pam for authentication and once your authenticated (Hopefully within 40-50 lines of code at the max) immediately chroot to your directory your writing to and then setuid to your user, you've potentially increased your security vs the standard mod_dav implementation at this point as users are now within a chroot which they have no way of escaping and they are setuid to a different user so they cannot overwrite other users files.

Re:Quota

[toast0]

Why not use suexec and install a copy webdav script in each user's cgi area?....

If someone finds a flaw in your script, they can ownz0r all your users, but not have root :)

Frontpage extensions vs. WebDAV

[jeif1k]

Yes, Frontpage has allowed upload of content through HTTP for a long time (it may even have been the first WYSIWYG HTML editor to support this). However, the mechanism it used to use was proprietary, had gaping security holes, and it had very limited functionality. (I don't know what Frontpage uses these days, but Windows has WebDAV client support built-in, although it has some limitations.)

WebDAV attempts to standardize this kind of functionality and make it available to many more programs and across platforms. WebDAV is sufficiently functional, complete, and efficient to serve both as a network file system protocol and as a network-based version control system.

Re:Frontpage extensions vs. WebDAV

[Anonymous Coward]

My limited understanding is that FrontPage still uses FrontPage extentions (as does VisualStudio). However, the rest of MS Office uses WebDAV.

Zope, perhaps

[Earlybird]

Zope supports WebDAV. Zope supports quota-limited mounted databases.

You would use Zope as a dumb, albeit journaled and transactional, file storage, though the files themselves will be stored in an opaque (object database) format; in other words, the only way to access the files will be through WebDAV (or FTP, which Zope also supports).

Re:Zope, perhaps

[Anonymous Coward]

You can also set up Zope to store the
files in a filesystem rather than the
ZODB (look for the File System Storage
product).

Re:Zope, perhaps

[VladDrac]

There's also a QuotaFolder product for zope, which together with Zope offers exactly what you want. You can see a working setup of all of this at FreeZope

ezPublish CMS has WebDAV capabilities

[gregwbrooks]

eZ publish has WebDAV capabilties. Here's a handy how-to on setting up an intranet using it.

Since it's based on PHP and pretty extensible, I would think getting a quota function established (if eZ publish doesn't already have one) would be easy enough.

Now, can someone help me get the damned 3.4.4 version to run on FreeBSD? ;)

Obligatory 10-second Google search

[fsck!]

http://www.needful.de/docs/projekte/webdav-quota/

iDisk

[Johnny Mnemonic]

Apple's iDisk offering is, or at least once was, WebDAV. Also incorporates quotas and is multi-user capable. Allows them to give nice hooks to publishing directly from iMovie and iPhoto, for example.

Sorry, I don't know exactly how they do it; but I do know that when it was announced (in '99?) there was some discussion about how Apple was accomplishing it. And you could probably reverse-implement their implementation in a few hours of poking.

For that matter, I think SpyMac uses the same thing.

Re:iDisk

[Anonymous Coward]

OS X server does quotas over webdav. Just give the users home folders with quotas and WebDAV respects them. Apple.com

WebDAV without Apache?

[david.given]

Does anyone know where I can find a small, simple WebDAV server that's doesn't require something as heavyweight as Apache?

My main server is a low-end notebook. It passes packets, does SMTP, file serving etc quite nicely. Unfortunately apache is just way too heavyweight for it; I use thttpd instead, which is smaller and faster.

I'd like to set up a WebDAV server. But I don't want to have to replace thttpd. Are there any small, light tools that will just do WebDAV and nothing else, that I can add to my setup?

Re:WebDAV without Apache?

[ttfkam]

Sometimes I really hate comments like this. Just how heavy to you think Apache is? With PHP/Perl/Python/Kitchensink support loaded in, sure.

Try loading Apache with just the barest essentials of modules (core functionality in Apache 2). The resource footprint ain't very big. Also, in terms of speed, Apache 2 can use the sendfile(...) API, memory map resources, and mod_cache if you load in support for it. In this configuration, you can easily saturate a gigabit ethernet connection. So unless you've got a gigabit internet pipe, use of thttpd vs. Apache is a moot point.

And besides, if raw web file serving performance is your goal, your script algorithms, internal caching model and judicious use of reverse proxies will make far bigger differences than your choice of web server.

Now then, on to WebDAV. Your comment, "Are there any small, light tools that will just do WebDAV and nothing else, that I can add to my setup?" belies a fundamental ignorance of what WebDAV is. It's an HTTP layer for file handling. By definition, anything that "will just do WebDAV" will need to be a web server. Subversion has it's own transport, but prefers the use of Apache for access to its version control repository. Why? Because Apache handles HTTP well. Why reinvent the wheel?

So in summary, in order to use thttpd with WebDAV, you would need to use thttpd as a proxy to another web server that also handles WebDAV, or you can get someone to write extensions to thttpd so that the WebDAV HTTP extensions PUT, DELETE, MOVE, PROPFIND, LOCK/UNLOCK, etc. are implemented correctly. Of course you would just be rewriting Apache+mod_dav, but suit yourself.

Re:WebDAV without Apache?

[david.given]

Sometimes I really hate comments like this. Just how heavy to you think Apache is? With PHP/Perl/Python/Kitchensink support loaded in, sure.

Stock Apache on my server came in at about 5-10MB RSS. thttpd is currently ticking away at 976kB. Apache takes >15s to start, thttpd starts instantly. Apache is too heavyweight for me. (My server is grossly underpowered, yes, but size and budget constraints mean I can't upgrade it.) Now then, on to WebDAV. Your comment, "Are there any small, light tools that will just do WebDAV and nothing else, that I can add to my setup?" belies a fundamental ignorance of what WebDAV is. It's an HTTP layer for file handling.

Yes, using a very specialised subset of HTTP extensions. It'd be entirely possible, and extremely useful, to write a special-purpose HTTP server that just did WebDAV. That would also be a useful distinction of functionality: a stock HTTP server doesn't need to write to files, but a WebDAV server would, which means it'd be easier to security audit everything.

I think the biggest problem is that you're approaching the problem for a different direction than I am. I want lightweight, low footprint, but not necessarily fast. It is a home setup, after all. You seem to be looking for something that can server vast amounts of traffic scalably.

Re:WebDAV without Apache?

[ttfkam]

Apache takes >15 seconds to start? My god man, what are you running? My house server is a 1GHz Athlon and I basically don't see the startup -- I'm serving WebDAV pages (via Subversion and Apache2) from a cold start in less than a second. Even on a 486, it shouldn't take even close to 15 seconds just to start up. Reduce the MinStartServers settings for example.

Another note, are you running out of memory on the box? Unless you're using up >95% of available memory (and thus beginning to swap heavily), what does it matter?

As for RSS (and ps reports in general), don't believe everything you read. When Apache forks processes on a good OS, a copy-on-write occurs. This means that unless a particular page of memory actually changes, a child process allocates no additional RAM (simplification of course, but effectively true). In addition, older versions of ps and some OS kernels report threads as separate processes. I'm running Apache2 with the worker MPM on a Linux 2.4 system and it looks like a dozen processes taking the same amount of memory instead of one process with a set amount of memory.

And then of course there's the issue of memory allocation vs. memory usage. Many processes (not just Apache) have pages swapped out because they aren't actively used.

So after all that, are you sure that Apache is taking up as much memory as you believe? And are you hitting swap -- the real memory litmus test?

Re:WebDAV without Apache?

[david.given]

Apache takes >15 seconds to start? My god man, what are you running?

A P133 notebook with 48MB of RAM.

So you can probably understand why I'm anxious to keep memory usage down! And yes, I know I should upgrade it, but it's harder than you might think. While I do have 128MB of laptop memory in hand, the blasted notebook can't handle DIMMs larger than 16MB. (It's got two 16MB DIMMs, plus another 16MB internally.)

There are two main reasons for using such an underpowered machine: firstly, cost, and secondly, it's an exercise in doing things the Unix Way. A P133 provides ample CPU power for doing simple stuff like serving web pages, processing email, etc. After all, it wasn't very long ago that this computer would be considered a number-crunching powerhouse. It's just a matter of utilising that CPU power appropriately. Finding the right software to run on it has led me to find all kinds of interesting, lightweight, flexible tools: thttpd serves my web pages, sn does NNTP, dnsmasq does DNS, etc. The only big server I still have is exim --- I'm on the lookout for a smaller replacement.

Apache on this machine was noteably sluggish just serving pages; whenever a mail came in, big chunks of Apache would get swapped out and then have to be swapped back in again to emit the next page. This doesn't happen with thttpd.

Also, I just find thttpd appeals to me far more than Apache. Apache's this vast amorphous blob of a program, which can do anything and everything by absorbing plugins. thttpd is small and optimised to do one thing well, and it does it really well.

Caudium

[Anonymous Coward]

How about Caudium?

public_html+webdav

[alexborges]

Simple. Turn on the mod_dav, modify the part where http://server/~username means /home/username/public_html and change it to something less cumbersome than the tilde.

Turn the gid bit on for the public_html directories.

Turn on group quotas for the partition.

Add www-data to each of the user's unique groups.

Careful though. This server will only be suitable for webdaving as adding www-data to many groups may make things very insecure if youre allowing your users to do scripting, ssh in or things like that.

On the other hand, perhaps a self compiled version of apache. Very stripped down. And look for a file creation mask/user/group in the httpd.conf options for dav.

This apache would probably have to run as root....

Okay, perhaps its not as simple as i thought. All of the above sucks.

Funny.

[FreeLinux]

You state that Netware would be an acceptable suggestion so, why not use it? Novell Netware can do what you want. Have a look at their iFolder product. But the funny part is that they use Apache and Tomcat/Jakarta to accomplish it.

With this in mind, it is probably less surprising that Novell also offers this functionality on the Linux platform using SuSE Linux Enterprise Server 8/9 and Novell Nterprise services for Linux.

Great. Now I sound like a shill.

Re:Funny.

[rdejean]

Novell has a product called Netstorage that comes with NW6.5 and NNLS. It's web-based file storage. You can use a browser or their Win32 client.

Random Thought

[Anonymous Coward]

If I run Apache as root and modify the authentication module to do setuid when they log in then the files written would owned by the user logged in.

Time to start hacking Apache!

Re:Random Thought

[LiENUS]

See mod_suexec it probably can do what you need.

Not webDAV, but why not:

[A Naughty Moose]

Is there any reason that it HAS to be webDAV? I am assuming that you don't want to use FTP because it is perceived as difficult to use. However I am going to suggest just that. Set up an FTP site, with a username/login for each client. Have them install Novell's Netdrive and talk them through the very simple configuration page. After Netdrive is configured, it will make the ftp site look and act just like a ny other local or network drive. (Pay no attention to the iFolder refrence on the Novell site. It will use it, but it works just fine with FTP services.)