Slashdot Mirror
FTP: Better Than HTTP, Or Obsolete?
An anonymous reader asks "Looking to serve files for downloading (typically 1MB-6MB), I'm confused about whether I should provide an FTP server instead of / as well as HTTP. According to a rapid Google search, the experts say 1) HTTP is slower and less reliable than FTP and 2) HTTP is amateur and will make you look a wimp. But a) FTP is full of security holes. and b) FTP is a crumbling legacy protocol and will make you look a dinosaur. Surely some contradiction... Should I make the effort to implement FTP or take desperate steps to avoid it?"
--sex
Very popular slashdot journal for adul
As long as you're willing to secure your FTP server and do the simple stuff like watch out for file permissions - FTP is much better.
HTTP is restricted by browsers, many of which will not support files larger than a certain size. Furthermore, FTP allows for features such as resume, etc...
The real question, however, is what are you trying to use this for? What's your intended application?
If it's a file repository for moderately computer literate people - FTP is definitely the way to go.
If it's a place for average-joes to store pictures, maybe HTTP is your best option. Sacrificing a bit of speed and capabilities such as resume might be made up for with ease of use..
http://www.babysmasher.com
http://www.openingbands.com
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
It's generally simpler to get to from a browser, which is where 95% of people's online life is anyway. Yeah, you can rig up a FTP URL, but it seems a bit kludgey and more prone to firewall issues.
SO YOU'RE GOING TO DIE: The Comic for Dealing with Death
And I wouldn't care about the opinion of someone who would actually judge you over what friggin protocol you use to provide downloads. Such an utter nerd is somethig that I can not relate too. Maybe after I use Linux for a few more years, who knows.
Security-wise, HTTP is a big win over FTP if only because it makes your port-filtering easier - "allow to 80" is simpler and less likely to cause unintended holes than all the things you need to do to support FTP active and passive connections. Certain FTP server software has a reputation as having more security holes than IIS, but there are FTP servers out there that are as secure as Apache.
There are two things to consider:
* protocol
* application
Neither ftp nor http are `full of holes'.
It very much depends which specific server you are using. Now, IIS is full of holes, and some ftp servers have been full of holes in the past, and still are... and there has been holes in apache as well.
As far as the functionality goes, ftp indeed provides some very useful functionality for file transfers: the ability to resume an interrupted transfer. Most servers and most clients support it nowadays and that is really VERY useful.
ftp is slightly more difficult to set up when firewalls are in effect, because of the two actual tcp channels used (data+control), which means the firewall needs to know about ftp (or it can get very unpleasant: active/passive protocols tricks will get you through one firewall, not through two of them).
Of course, if you are a complete moron, you will botch your server configuration and leave yourself wide open, or choose the wrong server, or forget to upgrade, and get security holes all over the place... doesn't matter which
protocol you use.
In reality, depending on what you want to serve, modern answers include sup, cvsup, rsync, and all kinds of other applications which leave ftp faaar behind (not even talking about http...). But then, they are less accessible and a little more 3l33t, which is maybe what you are looking for.
From my point of view (A network administrator), I provide both ftp and http servers for the same files (stick all downloads off /download or something, and set the ftp root to that). This has several benefets...
1) I've found HTTP transfers are a little faster than FTP transfers (just personally, and I can in no way prove it - it may be user error, or just the programs I'm using)
2) I've found that FTP clients are everwhere - Windows, Linux, BSD, everything I've ever installed has included a command line FTP client, but not a web browser unless I specifically remember to install one. Further more, most of the "live CDs/boot disks" that I use don't have a web browser, but do have FTP... Thus, if you're serving files that a person with out a web browser/server might need, I'd set up both...
3) FTP security is what you/your daemon makes of it. wu-ftpd has a long history of being rooted... ProFTPd dosn't. VSFtp doesn't. HTTP security is the same way... IIS has a long history of being rooted... Apache doesn't... *(Not to say that there haven't been occasional exploits for these platforms)
There is no clear "Use this" or "Use that" procedure here, it depends entirely on your situation, what you're serving, what your network setup is, etc...
-PhaseBurn Welcome to Linux country. On quiet nights, you can hear windows reboot.
I run into this dilemna constantly. I've found that HTTP is more reliable for downloads for intranet applications. When I build something in VB I try to use FTP, because I have better control of what's going on. For file uploads, of course, FTP is definitely the way to go. As for speed, I think FTP is technically faster, but HTTP throught the browser has the added bonus of showing you how much time is left on the download, making it SEEM to go faster. Basically, if the file is something that doesn't need to be secure, I'd use HTTP if speed isn't an issue (ie: intranet only or broadband connections mostly), otherwise, I'd go with FTP.
If you're looking for security, look into sftp, part of the openssh package. It uses the same encryption as SSH, is secure, yadayada. The only drawback is that windoze users have to get the sftp client to connect to an sftp server. Our school is considering adding sftp to the student fileserver so that we can access files from home without risk of attack.
I'm the Devil the Windows users warned you about.
How about implementing a webdav solution? You can get away from clear-text passwords, users can mount them like a drive on Mac, Windows and Linux (via DAVfs, http://freshmeat.net/projects/davfs/?topic_id=143% 2C90 )
Still have some of the unreliablity of HTTP transfers and slowness. But works a lot better through firewalls (and more securely since connection tracking works better with WebDAV).
I've found Passive Mode FTP to also be more unstable than standard ftp transfers.
The efficiency differences will be debated forever... the common wisdom is that FTP is more efficient, but there is also evidence to the contrary. That isn't the point.
To me, this is a problem of authentication. If you want EVERYONE to have these files, why not just use the HTTP server? If you're targeting a select few people, then why not use the built-in authentication mechanisms of FTP?
Yes I know there are authentication mechanisms for HTTP, but they're arguably harder to implement than setting up an FTP server.
Are your clients only using web browsers to retrieve these files? I'll get flamed for this, but web browsers were not designed for FTP, and thus are klunky at it. HTTP wins there again.
Don't worry about it. Just use HTTP and let the FTP bigots flame away.
There's no reason to use sftp for publically available files. This is for the exact same reason that you wouldn't use https. There's no need for encyption of something that is freely, publically available. Checksums, yes, encryption, no.
I personally would say go with http for the files, as it'll be much easier for people behind http proxies to download, it'll get cached more often by transparant proxies, and most browsers support browsing http directories FAR better than FTP directories.
FTP has generally better support for transferring plain-text. By downloading a document using the ASCII protocol, FTP automatically fixes up newline conventions to match your native OS. HTTP doesn't do that and requires user agents to respect most every ASCII-based newline convention. Unless your user agent is really intelligent in that regard and fixes up newlines automagically, you'll probably end up with text using newlines incompatible with your environment.
This is one area that I feel HTTP was deliberately weak on. I can't see any reason why HTTP couldn't require any text/* media type to be sent in a canonical format, with "network" newlines. If every Internet transport protocol did this properly, there would never be a situation where you have a file on your computer using the wrong type of newline.
HTTP "upload" is trickier than FTP upload, if that's a factor.
Computers are full of security holes. Both webservers and FTP daemons have the risk (and history) of 'sploitable holes. As to configuration-related risk, there's more inherent in the default configuration of an FTP server, since its original concept of ops is browsing a physical directory tree, but most current FTP servers can be locked down as tight as any webserver.Just can't win, can you? Only posers worry that their choice of protocol might look bad. It's not fashion, folks.If you had to support only one, pick HTTP, since I bet you're already running a webserver. And if it's really about fashion, run an exotic protocol like CORBA or SCP or something.
Welcome to the Panopticon. Used to be a prison, now it's your home.
While this does help with the evasion of an acient protocol, it forgets one of the poster's main goals. The poster worried HTTP is slower ... than FTP . SFTP is very slow considering the overhead each packet has because of encryption. -js
HTTP is stateless and FTP is stateful, yes. However, the "overhead" required for a stateful connection is not going to cause a slowdown in transfer except possibly an the onset for the authentication. Stateles IP protocols don't have persistent connections, they shoot packets out and don't care about the receiving end because they assume that IP devices in the middle or the end will ensure delivery and proper resequencing. Stateful protocols keep an active connection and this is supposed to be faster. After all, ATM's big hold over, say DS3 and T1 lines, is that they open a pipeline and shoot data downstream. It is "connection oriented," like a phone call.
My suggestion would be to go with FTP if the size is anything above a few MB for administrative reasons. HTTP doesn't allow you to limit connections and bandwidth could be incidentally mashed into virtually nothing per user. You can limit not only the number of active downloads, but even who can download if you wanted.
Think of almost every Linux distribution - they use FTP. Why? Ever tried downloading something, connected at a smooth 100k/s or something and found, a few minutes later, you're pulling 2k/s? That USUALLY doesn't happen with FTP. Once you're in, you're in.
If you have tiny files, of course, logic dictates you go with HTTP. But for anything of size, do *yourself* a favor and go with FTP.
Adam
The question does not contain enough information to form a proper answer.
Don't start with finding the solution, figure out what it is you want, what you want it to do and then find the right tool. We can not tell you which is right with almost no information about the use of it, for what and what is the average user profile etc.
HTTP and FTP can be equally insecure, but it shouldn't be much of a job to properly secure a ftp.
my sig
From the end-user's perspective, downloading via http is more convenient, especially if you're on a corporate network and have to go thru Squid proxy since ftp-by-browser-thru-squid just sucks.
As far as I know Google does not spider ftp archives. Therefore if you want people to be able to search for all the files offered (including inside word processing documents and PDFs) then you are better going with http.
:-)
I think that the ability for people to find the files using a search engine outweigh the other considerations if you are seeking to make the files public.
Slashdot Beta should die a painful death.
Connecting to multiple servers to download a file is great. Getting six connections from one client to one server is a royal pain, and is one of the reasons some admins have taken to blocking download managers. Getting multiple connections from one client can reduce the number of total users that can be served and is the biggest drawback to allowing download managers.
But... doesn't HTTP 1.1 support resuming? I've resumed files over HTTP before...
== Jez ==
Do you miss Firefox? Try Pale Moon.
And I've never understood why they think that opening up 6 connections when downloading a file would be quicker than just one. Quite apart from the fact that that insane kind of tactic can get your IP banned from some HTTP servers.
== Jez ==
Do you miss Firefox? Try Pale Moon.
You don't need to use something with encryption like HTTPS or SFTP. These are publicly available files, right? I do recommend you post md5sums for each of these files both in the download directory and on the webpage that links to each of these files.
HTTP is most likely to get cached by caching proxies. So if your target audience is likely to be behind a caching proxy (Unv campus?) and you want to save yourself some bandwidth, use HTTP on a dedicated port or box so you can control concurrent downloads. HTTP might also get you through more downloads. Then again if someone's firewall will only let them get to port 80, they're too stupid to be downloading files from me anyhow.
Personally I'd recommend FTP. It's easier to control the number of concurrent downloads, concurrent connections from a single IP, transfer speeds, and integrates well with your authenticated uploads by yourself or whatever other person will be uploading files for download. Saying that FTP is less secure than HTTP is complete BS. Both have the same level of insecurity. The most common FTP daemon (WU-FTPD) has had horiffic security holes in the past which is why no one in their right mind should use it. ProFTPd on the other hand has had very few security problems (usually long after the affected release has been surpassed by a couple new releases at least). I highly recommend you look into ProFTPd for and FTP solution. You can always offer and HTTP solution but I highly recommend you limit the concurrent number of transfer if you use HTTP.
Unfortunately HTTP is still a more secure protocol because HTTP clients can easily implement SSL encryption for passwords and sensitive data transfer. When you're dealing with a client who insists on using an outdated, but "easy" FTP client, it's not easy to tell them they have to change, and it's even less easy to tell them what they have to change to.
FTP is notoriously difficult to secure while retaining ease of use for the clueless end-user.
-Sara
You're also slightly less likely to be threatened with root exploits when using HTTP instead of FTP. Especially if you're already maintaining an HTTP server on the particular machine in use--because you already HAVE that security hole open. Rather than opening another hole, you're using one that's already there.
-Sara
It sounds like anonymous downloading of publicy available files - whatfor do we need any encryption then?
If not for the encryption, then consider what else you get: a well-defined TCP connection. It's a cinch to configure a firewall to allow sftp connections, while FTP firewalling will give you prematurely grey hair (and if it doesn't, then you're not doing it right).
Dewey, what part of this looks like authorities should be involved?
The reason to use sftp on publicly available files or hhtps is so that people <*cough*>Carnivore</*cough*> can't track what I'm doing online. Sure, they can tell what sites you are visiting, but they can't tell what content you're looking at, or which files you downloaded... Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads <*cough*>Verizon</*cough*> and the RIAA wants access to those records... If they don't know what you downloaded they can't (potentially unjustly) prosecute you...
A HTTP server is perfectly within its rights to answer 503 Service Unavailable and disconnect if too many unauthenticated requests are going on.
Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads Verizon and the RIAA wants access to those records
Why in Hell would the Recording Industry Association of America care who has software to break the DVD Content Scrambling System? They market music on CDs, not movies on DVDs.
Or did you mean to say MPAA?
Yeah, yeah, it's just a careless error, but things like this make us all sound like a bunch of idiots. If we can't even keep track of which evil corporate organization goes with which issue, then how is anyone supposed to take us seriously?
David Gould
main(i){putchar(340056100>>(i-1)*5&31|!!(i<6)<< 6)&&main(++i);}
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
I suspect that's because 99% of people are downloading from one of the FTP servers.
I put to you that would be more logical to suspect it's because HTTP is faster than FTP as a transfer protocol. It generates less traffic (and uses less CPU overhead) which means downloads end quicker.
Additionally the CPU overhead generated by FTP connections also causes many sites to limit the number of users who can connect, which often results in 'busy sessions', something much rarer with HTTP (as HTTP servers typically have very high thresholds for the number of concurrent connections they will support). The overhead on a server of a user downloading a file over FTP is much greater than that of a user downloading the same file over HTTP.
Although FTP is of course theoretically more reliable than HTTP, in practice, because of 'Server busy: Too many users' messages combined with the speed and reliability of modern connections (which in turn makes HTTP more reliable) mean the the reverse is often the case from a user perspective - which is what I think the poster is getting at.
This may be partly due to poor FTP server configuration defaults and/or poor administration, but they cannot shoulder all the blame.
The potential lack of reliability with HTTP is a very minor issue these days, and the extra overhead of integrity checking files in addition to relying on TCP is just not warranted for all but the largest of files.
This doesn't make FTP completely redundant, but it does make it make it redundant when your files are small and your users are on fast, reliable connections (though the value of 'fast' varies in relation to the size of the file, even 33 kbps is 'fast' compared to the speed of connections that proliferated when the File Transfer Protocol was developed).
HTTP is amateur and will make you look a wimp.
FTP is a crumbling legacy protocol and will make you look a dinosaur.
Why care so much what others think? Efficiency beats out looks any day.
I see too many FUD replies here:
:-)
1) HTTP doesn't support resumed downloading.
- That's ridiculous. It has since HTTP/1.1 years ago. In fact, it can even do things like request bytes 70,000 - 80,000, then 90,763 - 96,450, etc.
2) HTTP doesn't support security/authentication
- Ridiculous. HTTP has an open-ended model for authentication and security, many of which are secure and standardized. If you REALLY need security, use HTTPS.
3) HTTP doesn't support uploading
- HTTP/1.1 has had this for a while. Netscape 4.7, Mozilla 1.1, and IE 4+ support this. I must admit though, it sucks.
Several people have pointed out the real differences:
1) FTP doesn't like firewalls
- Passive FTP fixes this, but it has quirks and limitations.
2) FTP supports directory listing, renaming, uploading, changing of permissions, etc.
- This is what FTP is for
- This can be done in HTTP, but requires serious work
- If the scope creeps, shell access would be better.
I am not sure if there is really a purpose to this and am just throwing ideas around. But you could host an FTP on an internal network and route it through an HTTP CGI script. You can also use encryption over the HTTP connection to make it secure without resorting to SFTP. I just happen to use this method myself since SFTP requires special clients and is a bitch to setup...just some food for thought
I'm not totally up on HTTP and such, but why is it a royal pain? Assuming the user is going to download all 3 ISO images one way or another, what's the issue if he grabs them using 6 streams in 1/6th (roughly) of the time, or in one stream for the full stretch? It's still bandwidth being eaten. I guess it boils down to time versus users.
Caching MD5 sums for every block? Well, this may ease the load on your processor but I hope you have plenty of RAM!
A pretty distinct advantage for those of us who use shells often or have to repair machines at a command prompt.
.sig wanted: Must be concise, funny, and display my cleverness.
If it ain't broke, don't fix it.
This sig no verb.
In my case, I run an Apache server on my file server box because it gives me full 100% throughput at Ethernet speeds. That's 10+ megaBYTES/sec. The average anime episode downloads in 10-15 seconds, and when friends come over, I don't have to worry about them having the right kind of client when they want to leech.
Where FTP becomes useful is when you want people to upload to you.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
rsync wasn't written by a moron.
Some ISP's (and companies if the user is behind a corporate firewall) provide transparent caching proxies for HTTP transfers, saving you the cost of actually serving their copy of the file.
That's not true--they can tell roughly what the size of files you're looking at, and if you're browsing, which files were triggered-downloaded--for example images embedded in web pages.
All this information makes it trivial to find out which pages you downloaded, when, and how long you visited.
The only thing it doesn't do so well is listen in on data you send. But consider--if they're listening to the tcp channel to the remote site then they can listen to outgoing tcp connectivity as well.--and correlate what information came in, and what information went back out.
You think a single encrypted channel buys you any privacy? Get real.
What are you using Lynx to connect to the ftp server? That's the reason it's slow. Try a real client like NcFTP and when you do, make sure to take note of the sheer power.
mget blah[1,2,3].iso....
Get the drift? HTTP indexes are rather stupid if you ask me, it's FTP without the features. And before you whine "But I don't need the features", neither do I most of the time, but it's nice when they're there.
ftp> get mybigfile.iso
/* walk away from PC. Come back 2 hours later */
/* Doh!!!!! */
file transfer started
ASCII data transfer of mybigfile.iso successfully completed.
http support for self-signed keys is weak - it complains too much.
http upload is weak, and the method to accomplish an http upload is akward.
Why would I want to use UDP? Don't you imagine that there were performance reasons behind the choice of UDP for NFS?
cli support for http is weak - show me how you upload with wget.
Since when did Apache run chroot?
Funny, I don't remember linking in zlib for compressed http streams the last time I built apache (let alone bzip2)...
Why should you have to write a script to download files? Choice FTP programs let you queue downloads just fine.
It seems everyone talks about DOWNLOADING. How about UPLOADING? How fast and wonderful is HTTP for that?
You might consider setting up a BitTorrent tracker on your website, and making the files available that way. It's not any worse than website bandwidth, and since you're also leveraging the outbound bandwidth of sites that download the file, it could be faster.
Editor Emeritus and Senior Writer, TeleRead.org
If you're providing downloads that may reach a decent volume, I would recommend a caching appliance. A CacheFlow server (they were bought out by BlueCoat systems) or something similar should do the trick. These can cache HTTP and FTP inbound requests, and instead of having the load on your servers, these things will serve up your files instead. A good cache appliance can serve up files a lot faster/more-efficient than a web server, and you won't bog down your box with processes.
These caching appliances can also broadcast multimedia streams (Real or WinMedia) with the correct licenses.
I would do some research on some CacheFlow boxes, which, by the way, are selling on the cheap at eBay. These were the $25,000 (circa 1998-2000) priced appliances that you can now pick up for, sometimes, in the hundreds of very low thousands.
The main strengths of HTTP over FTP for file transfers are:
The other differences one sees are due to server design issues. I.e. most FTP servers are large and spawn a process per connection, which makes FTP sessions much slower than HTTP sessions. But if you want to use FTP, there are very fast FTP servers out there.
Overall, in today's world, it does not make sense to use FTP unless you have a requirement from your users. For public access to files, use HTTP or something more modern, such as rsynch, or a P2P network.
As usual, you should answer such questions by thinking about your target users and asking yourself what they are likely to be most comfortable using. Chances are it's their main tool, the web browser.
My blog
All most people would want to do with FTP can be done with HTTP.
There are hardly any practical legitimate reasons for an FTP server to shove data to some other place rather than the FTP client and other dangerous combinations FTP allows.
Uploads? Uploads had better be disabled on FTP by default. Uploads are disabled in HTTP by default, but you can configure your site to accept uploads.
Mass downloads of files? Just standardise. As is you can already download entire websites. With HTML (not javascript) the links are there. If you want files only you just have to standardise on a program readable way to figure out which links are files to download.
Resuming of downloads - works on most webservers, works on most ftp servers.
And http allows more - better compatibility with firewalls and proxies. Can be made to support SSL easily - most popular clients support SSL.
---
FTP is a bad protocol. The PORT and PASV thing is stupid. Most protocols which specify network layer data (IP address, port) in the content layer for programs to use, are bad designs.
How would PASV or PORT look with IPv6? With HTTP, you don't have to change the protocol much if at all.