Slashdot Mirror
FTP: Better Than HTTP, Or Obsolete?
An anonymous reader asks "Looking to serve files for downloading (typically 1MB-6MB), I'm confused about whether I should provide an FTP server instead of / as well as HTTP. According to a rapid Google search, the experts say 1) HTTP is slower and less reliable than FTP and 2) HTTP is amateur and will make you look a wimp. But a) FTP is full of security holes. and b) FTP is a crumbling legacy protocol and will make you look a dinosaur. Surely some contradiction... Should I make the effort to implement FTP or take desperate steps to avoid it?"
...use sftp!
I haven't really noticed any reliability issues with http anymore. If it starts loading it usually finishes, and I haven't run into any corruption problems. Maybe if you were serving huge files ftp would be a good idea, but for 1-6 mb it's probably not worth it.
--sex
Very popular slashdot journal for adul
Use telnet and screen capture the VT100 Term buffer!
But in my experiences, HTTP for whatever reason goes faster (not entirely sure why), and FTP doesnt work for some because of firewalls.
Try both - see which gets used more.
The One Rule Of Chess You'll Ever Need: Don't play someone who carries a kit in their bookbag.
rsync is a great protocol, fairly robust, can be wrappered in ssh (or not), supports resuming transmission, and operates over one socket.
seems like the best of both worlds to me.
the real question is - do you control the clients that are going to access you? or is it something like a browser (which doesn't support rsync).
Do it for da shorties
"HTTP is slower and less reliable than FTP"
I would think FTP is slower since with FTP you have to login and build the data connection before the transfer begins. With HTTP it's a simple GET request.
As far as the actual data being sent, I believe that the file is sent the same way with both protocols. (just send the data via a TCP connection). I could be wrong though.
As long as you're willing to secure your FTP server and do the simple stuff like watch out for file permissions - FTP is much better.
HTTP is restricted by browsers, many of which will not support files larger than a certain size. Furthermore, FTP allows for features such as resume, etc...
The real question, however, is what are you trying to use this for? What's your intended application?
If it's a file repository for moderately computer literate people - FTP is definitely the way to go.
If it's a place for average-joes to store pictures, maybe HTTP is your best option. Sacrificing a bit of speed and capabilities such as resume might be made up for with ease of use..
http://www.babysmasher.com
http://www.openingbands.com
do as mysql site does and others... supply links for both options
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
It's generally simpler to get to from a browser, which is where 95% of people's online life is anyway. Yeah, you can rig up a FTP URL, but it seems a bit kludgey and more prone to firewall issues.
SO YOU'RE GOING TO DIE: The Comic for Dealing with Death
1-6mb files? :P
heh, most 1-6mb files I see are on irc fserves
Those of your customers who don't have fast access to the internet may appreciate even a slightly faster standard.
Ahh, I always set hash on before command line ftping. Then I get to see the pretty #'s fly by...
They who would give up an essential liberty for temporary security, deserve neither liberty nor security
HTTP does not have firewall issues, does not need authentication, does not (by default) allow directory listings, and is the same speed as FTP. It's a good deal for general file distrubution.
FTP is quickly becoming a special-needs protocol. If you need authentication, uploads, directory listings, accessability with interactive tools, etc. then this is for you. Mainly useful for web designers these days, IMO, since the site software packages can use the extra file metadata for synchronization. Other than that, it's a lot of connection overhead for a simple file.
FTP does have one nice advantage that HTTP lacks: it can limit concurrent connections based on access privleges (500 anonymous and 100 real, etc.). Doesn't sound like you need that.
Go with HTTP. Simple, quick, anonymous, generally foolproof.
1 to 6 megs, huh? Why not use Kazaa like everybody else? :-P
"If he thinks he can hide and run from the United States and our allies, he's sorely mistaken." Bush on bin Laden
We run ftp, but we have to have people send us files, and also distribute them on a regular basis.The client software available for doing the sending and receiving on a regular basis is a lot better for FTP.. it's pretty klunky, but it is very doable for http.
We just choose to stay on top of our ftp updates.
"HTTP is amateur and will make you look a wimp"
You really gotta watch out for things like this. I know one guy that got a 'click me' sign on his back because he used HTTP instead of FTP.
And I wouldn't care about the opinion of someone who would actually judge you over what friggin protocol you use to provide downloads. Such an utter nerd is somethig that I can not relate too. Maybe after I use Linux for a few more years, who knows.
The question is, "what do you want to do?" I run an FTP server (incidentally affiliated with etree.org, lossless live music!) and I need what it can give me. Namely I need multiple classes of login, each with a different
1) number of available slots
2) speed limit
3) premission set
Some people can only read files at 60KB/s, some can read and write (to the upload dir) at the same speed, come can only browse, etc. etc. For this kind of a setup, FTP is great _IF_ you keep your software up to date; subscribe to bugtraq or your distro's security bulletin or both.
On the other hand, HTTP is great when you want to give lots of people unlimited ANONYMOUS access to something. I'm sure there is a way to throttle bandwidth, but can you do it on a class by class basis? In proftpd it's a simple "RateReadBPS xxx" and I'm set.
As always, choose the tool that fits _your_ purpose, not the one that everyone says is "best"; they both have good and bad qualities. And http can be just as secure/insecure as any other protocol.
The time of day is 29:33.
If you're only offering files to a group of users who you can give passwords to, you could even use SCP. (Secure copy...uses sshd on the server side)
It all depends on the application. I only use SCP to move files around if I have the choice, just because I like better security if I can have it.
But if you want to offer files to the public, I'd recommend offering both FTP and HTTP so people can use the most convenient.
wget does both and does it well.
http://www.gnu.org/software/wget/wget.html
Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws-Plato
Security-wise, HTTP is a big win over FTP if only because it makes your port-filtering easier - "allow to 80" is simpler and less likely to cause unintended holes than all the things you need to do to support FTP active and passive connections. Certain FTP server software has a reputation as having more security holes than IIS, but there are FTP servers out there that are as secure as Apache.
From my point of view (A network administrator), I provide both ftp and http servers for the same files (stick all downloads off /download or something, and set the ftp root to that). This has several benefets...
1) I've found HTTP transfers are a little faster than FTP transfers (just personally, and I can in no way prove it - it may be user error, or just the programs I'm using)
2) I've found that FTP clients are everwhere - Windows, Linux, BSD, everything I've ever installed has included a command line FTP client, but not a web browser unless I specifically remember to install one. Further more, most of the "live CDs/boot disks" that I use don't have a web browser, but do have FTP... Thus, if you're serving files that a person with out a web browser/server might need, I'd set up both...
3) FTP security is what you/your daemon makes of it. wu-ftpd has a long history of being rooted... ProFTPd dosn't. VSFtp doesn't. HTTP security is the same way... IIS has a long history of being rooted... Apache doesn't... *(Not to say that there haven't been occasional exploits for these platforms)
There is no clear "Use this" or "Use that" procedure here, it depends entirely on your situation, what you're serving, what your network setup is, etc...
-PhaseBurn Welcome to Linux country. On quiet nights, you can hear windows reboot.
FTP provides you with user authentication, and binary transfers (which should be faster as there is no encoding??) It can also be linked to via the web, so there's not too much hassle for the user...
On the other hand, if you don't need user authentication - and don't want to off load big file transfers from your web-server, you may as well just leave it as http.
I run into this dilemna constantly. I've found that HTTP is more reliable for downloads for intranet applications. When I build something in VB I try to use FTP, because I have better control of what's going on. For file uploads, of course, FTP is definitely the way to go. As for speed, I think FTP is technically faster, but HTTP throught the browser has the added bonus of showing you how much time is left on the download, making it SEEM to go faster. Basically, if the file is something that doesn't need to be secure, I'd use HTTP if speed isn't an issue (ie: intranet only or broadband connections mostly), otherwise, I'd go with FTP.
If you're looking for security, look into sftp, part of the openssh package. It uses the same encryption as SSH, is secure, yadayada. The only drawback is that windoze users have to get the sftp client to connect to an sftp server. Our school is considering adding sftp to the student fileserver so that we can access files from home without risk of attack.
I'm the Devil the Windows users warned you about.
ncftp is a command-line client that shows you your download progress.
Anyone who says that HTTP is slower and less reliable than FTP probably hasn't done any benchmarking. Based on my experience, HTTP is definitely more reliable if only because it tends to go through firewalls easier then the two-connection FTP protocol.
Both FTP and HTTP stream data across a TCP socket -- I can't see that streaming it over port 20 versus 80 is going to make any difference.
FTP was designed to be able to do all these neat things back when the internet didn't have so many security issues. Most of these features are either not used or explicitly disabled these days... The fact that the FTP server uses a different port means that firewall have to understand and properly be configured for this. HTTP sends the data back in response to the initial connection, so it tends to be easier to get through firewalls.
If you're concerned about looking like a "wimp", then you should offer both and let people pick what they prefer. Or... Stop worrying about what people these people think and figure out what YOU think is best.
The people who would call you a "wimp" probably aren't worth worrying about.
Sean
How about implementing a webdav solution? You can get away from clear-text passwords, users can mount them like a drive on Mac, Windows and Linux (via DAVfs, http://freshmeat.net/projects/davfs/?topic_id=143% 2C90 )
Still have some of the unreliablity of HTTP transfers and slowness. But works a lot better through firewalls (and more securely since connection tracking works better with WebDAV).
I've found Passive Mode FTP to also be more unstable than standard ftp transfers.
Which one to choose? Why, its trivial!
In the future, I would want to not be isolated from my friends in the Space Station.
The efficiency differences will be debated forever... the common wisdom is that FTP is more efficient, but there is also evidence to the contrary. That isn't the point.
To me, this is a problem of authentication. If you want EVERYONE to have these files, why not just use the HTTP server? If you're targeting a select few people, then why not use the built-in authentication mechanisms of FTP?
Yes I know there are authentication mechanisms for HTTP, but they're arguably harder to implement than setting up an FTP server.
Are your clients only using web browsers to retrieve these files? I'll get flamed for this, but web browsers were not designed for FTP, and thus are klunky at it. HTTP wins there again.
Don't worry about it. Just use HTTP and let the FTP bigots flame away.
My experience with FTP/HTTP is due to FTP's authentication it responds slower and is harder to configure, whereas HTTP has near instant authentication & is more easily multithreaded on downloads.
FTP has generally better support for transferring plain-text. By downloading a document using the ASCII protocol, FTP automatically fixes up newline conventions to match your native OS. HTTP doesn't do that and requires user agents to respect most every ASCII-based newline convention. Unless your user agent is really intelligent in that regard and fixes up newlines automagically, you'll probably end up with text using newlines incompatible with your environment.
This is one area that I feel HTTP was deliberately weak on. I can't see any reason why HTTP couldn't require any text/* media type to be sent in a canonical format, with "network" newlines. If every Internet transport protocol did this properly, there would never be a situation where you have a file on your computer using the wrong type of newline.
Come on people, use the Z-Modem protocol. It can resume transmission on a file transfer where HTTP or FTP can not. The only way a FTP or HTTP can resume transmission is with the GetRight tool.
:) HA HA!
I remember in my days of BBSes with X and Y Modem, and then when Z-Modem showed up we all couldn't be happier. When some idiot in the house picked up the phone and disconnected you from hours and hours of downloading the latest Liesure Suite Larry, I just reconnected and started to resume my downloads (but only if I had enough credit, then I might have to upload some crap).
IE will resume a d/l too, providing the temp file is still in cache
Why do we have all these new ask slashdot question that sounds like a tech with a years experience is asking how to do his job?
I vote for a new section, "How do I do my job" with a dollar bill as the logo.
Neck_of_the_Woods
#/usr/local/surf/glassy/overhead
(Cleaning up the text a bit)
Well, 2 and 4 are nothing more than acephalic punditry, unworthy of our attention, which leaves 1 and 3.
The fact that HTTP doesn't use a binary connection to transfer binary files means that, yes, it is frequently slower than FTP. Especially since your listed file sizes imply that you're not offering text files for download.
While FTP doesn't have any security holes (Yay for false generalisations!), many of the readily available ftp daemons have had shaky track records in the security area.
I don't really have an answer for you, I just wanted to say acephalic. :-) Acephalic acephalic acephalic...
Teach your kids: "C++ made baby Jesus cry."
HTTP "upload" is trickier than FTP upload, if that's a factor.
Computers are full of security holes. Both webservers and FTP daemons have the risk (and history) of 'sploitable holes. As to configuration-related risk, there's more inherent in the default configuration of an FTP server, since its original concept of ops is browsing a physical directory tree, but most current FTP servers can be locked down as tight as any webserver.Just can't win, can you? Only posers worry that their choice of protocol might look bad. It's not fashion, folks.If you had to support only one, pick HTTP, since I bet you're already running a webserver. And if it's really about fashion, run an exotic protocol like CORBA or SCP or something.
Welcome to the Panopticon. Used to be a prison, now it's your home.
It is possible to get approximately 80% of the theoretical maximum throughput of your pipe using a single FTP connection, whereas HTTP can hope for around 60% max for a single connection. The only thing faster than an FTP-based protocol (tftp, pftp) is a raw socket, and they rarely get better then 90%. Most schemes like pftp (parallel ftp, see this paper) are implemented to get as close to theoretical maximum throughput by having multiple data connections transfer the file. Of course you'll see the difference in performance more for large file transfers. The previous comment about HTTP being OK for small files is right on the mark...you will hardly notice a 20% gain when the transfers are only taking a few seconds.
I use two programs to retrieve files, wget and Mozilla. Both show the download rate whether I'm fetching from HTTP or FTP.
What bugs me is when servers won't tell me the final downloaded file size -- no ETA available. I've seen both FTP and HTTP servers do it. The same goes for servers that don't support resuming or last-modified dates. They suck.
Does my bum look big in this?
P2P?
I've written a tutorial on how you can use P2P on your website to save bandwidth, space etc. An obvious way to do this would be to run a P2P client and share the file on a simple PC & Cable Modem. This works, but it is a bit generic and un-professional. A better way to do this may be to run a P2P client such as Shareaza on a webserver. You could then control the client using some type of remote service (Terminal Services, for example).
P2P has it's advantages. Such as:
- Users who download the file also share it. This is especially useful if the client/network supports Partial File Sharing.
- When you release the file using the P2P client, you only need to upload to only a few users. Those users can then share the file using Partial File Sharing etc.
- Unlike FTP and HTTP, they aren't connecting to your webserver. Thus, it saves bandwidth for you and allows people to browse your website for actual content, not media. (Though, media is content). In addition, there is ussually "Max # of Connections" allowed to a server or FTP. Not so on P2P.
- P2P Clients have good queuing tools. At least, Shareaza does. It has a "Small Queue" and a "Large Queue". This basically allows you to have, say, 4 Upload slots for Large Files (Files that are above 10MB, for example) and one for Small Files (Under 10MB). Users who are waiting to download from you can wait in "Queue", instead of "Max users connected" on FTP.
Though, at it's core, all of the P2P I know of uses HTTP to send files etc. But the network layer helps file distribution tremendously.
FTP is a quirky, extensible protocol, great for uploading, downloading and sharing files, and you can do wonderful things with FTP and databases which web servers only dream about.
Rich.
libguestfs - tools for accessing and modifying virtual machine disk images
HTTP is stateless and FTP is stateful, yes. However, the "overhead" required for a stateful connection is not going to cause a slowdown in transfer except possibly an the onset for the authentication. Stateles IP protocols don't have persistent connections, they shoot packets out and don't care about the receiving end because they assume that IP devices in the middle or the end will ensure delivery and proper resequencing. Stateful protocols keep an active connection and this is supposed to be faster. After all, ATM's big hold over, say DS3 and T1 lines, is that they open a pipeline and shoot data downstream. It is "connection oriented," like a phone call.
My suggestion would be to go with FTP if the size is anything above a few MB for administrative reasons. HTTP doesn't allow you to limit connections and bandwidth could be incidentally mashed into virtually nothing per user. You can limit not only the number of active downloads, but even who can download if you wanted.
Think of almost every Linux distribution - they use FTP. Why? Ever tried downloading something, connected at a smooth 100k/s or something and found, a few minutes later, you're pulling 2k/s? That USUALLY doesn't happen with FTP. Once you're in, you're in.
If you have tiny files, of course, logic dictates you go with HTTP. But for anything of size, do *yourself* a favor and go with FTP.
Adam
What I don't care for with FTP is the continuous setup/teardown of data connections. What is even worse with active FTP is that the client side of the data connection establishes server ports, and the server becomes the client (I'd like to be able to use plug-gw from the TIS FWTK for FTP, but this is not possible for the data connections). However, even when enabling passive FTP, the data connections are too prone to lockup. The difficulty of implementing all of this in C probably contributes to the FTP server vulnerabilities.
Still, if you want both (optionally anonymous) upload ability and access from a web browser, FTP is the only game in town.
From the network perspective, the rsh/rcp mechanism is cleaner (in that there is only one connection), but it still has the problem of either passing cleartext authentication or establishing unreasonable levels of trust with trivial authentication. In addition, with rcp syntax you must know much more about the path to a file, and there is no real "browsing."
Many say that SSH is the panacea for these problems, but sometimes I am not concerned about encryption and I just want to quickly transfer a file. The SSH man pages indicate that encryption can be disabled, but I have never been able to make this work. SCP also has never been implemented in a browser for file transfers. I should also say that I've never used sftp, because it has so little support.
Someday, we will have a good, encrypted file transfer protocol (and reliable implementations of that protocol). Sorry to say, but ftp, rcp, and scp are not it. What will this new protocol support?
Boy, I never thought that I could rant about file transfer software for so long!
The question does not contain enough information to form a proper answer.
Don't start with finding the solution, figure out what it is you want, what you want it to do and then find the right tool. We can not tell you which is right with almost no information about the use of it, for what and what is the average user profile etc.
HTTP and FTP can be equally insecure, but it shouldn't be much of a job to properly secure a ftp.
my sig
From the end-user's perspective, downloading via http is more convenient, especially if you're on a corporate network and have to go thru Squid proxy since ftp-by-browser-thru-squid just sucks.
As far as I know Google does not spider ftp archives. Therefore if you want people to be able to search for all the files offered (including inside word processing documents and PDFs) then you are better going with http.
:-)
I think that the ability for people to find the files using a search engine outweigh the other considerations if you are seeking to make the files public.
Slashdot Beta should die a painful death.
Each has their place.
FTP was designed for interfacing with the filesystem of a remote Unix system, with the filesystem permissions that are granted to the user you log in as. FTP lets you browse the hierarchy, including examining ownership, permission, and symlink targets; pretty much the same as what you get with 'ls -l'. Apache does file listings, but only shows file names, last modification dates, and size. This makes FTP more suitable than HTTP for remote mirroring of directory trees. This also makes it easier to "browse" what an FTP server has to offer, on a directory-by-directory basis.
With FTP, the server prints a response when a client connects. Usually, the client sends a user name, password, the 'SYST' command, and asks for the current working directory, tells the server what mode (ASCII or binary) it wants, changes to the directory with the file it needs, sends a PORT command, then finally requests the file. With HTTP, the client connects, sends a request, and the server responds. That's 8 client commands and 9 server responses with FTP, as opposed to 1/1 with HTTP. Each time a command is sent, the client has to wait for the server to respond. The latency adds up, and that means, especially on high-latency connections, FTP is slower to initiate and begin downloading than HTTP. Who said HTTP is slower?
Regarding reliability, both protocols and modern implementations of their clients and servers have features to resume a broken download from where it left off. Who says one is more reliable than the other?
HTTP is more simple than FTP. As far as I can tell, in FTP "active mode", the client sends a PORT command with an IP address and port number that is listening. In "passive mode", the server sends an IP address and port number that is listening, after the PASV command. These address/port combinations are used for the actual file transfer.
Active mode doesn't work if there is NAT between the client and the server, unless the NAT system rewrites the packets so that the IP address the server sees in the PORT command is the outside, external address of the NAT system. When an FTP server is behind NAT, passive mode cannot be used without a similar kluge; it must get an outside-world IP address to connect to from the client, which active-mode PORT does. If both client and server are behind NAT, then one of these NAT kluges must be in effect for file transfer to be possible.
This address/port nonsense could be part of the security concern with FTP. I also believe older FTP implementations allowed the client (in active mode) or server (in passive mode) to specify arbitrary address/port combinations, so that the FTP server or client could be used as a proxy in an attack. Is this still the case?
With HTTP, transfers are conducted on the same TCP connection as control is on, and therefore doesn't need to concern itself with IP addresses and ports, and the people using it have fewer NAT headaches.
Connecting to multiple servers to download a file is great. Getting six connections from one client to one server is a royal pain, and is one of the reasons some admins have taken to blocking download managers. Getting multiple connections from one client can reduce the number of total users that can be served and is the biggest drawback to allowing download managers.
Let me get this straight. You went to search the web and got conflicting, likely ill-informed, and inconclusive reports. So you went to Slashdot?
As many people have said, it depends.
...)
FTP has a great advantage in that you can request multiple files at the same time: mget instead of get. Additionally, you can use wildcards in the names, so you can select categories / directories of files with very short commands. (mget *.mp3 *.m3u
Modern browsers allow you to transfer multiple files simultaneously, but they don't queue files for you - FTP will. This may be important if connections might get dropped - the FTP transfer will complete the first file, then move on to the next. In the event of an interruption, you will have some complete files, and one partial (which you can likely resume). For multiple simultaneous transfers - from an http browser - you may have some smaller files finished, but it's likely that all larger files will be partials, and will need to be retransmitted in their entirety, since http doesn't quite support resuming a previous download.
So, if you're going to have a web page with many individual links, and you think that most people will download one or two files, http will probably suffice. If you expect people to want multiple files, or that they will want to be able to select groups of files with wildcards (tortuous with pointy-clicky things), then you should have FTP.
It's not that hard to set up both, and that's probably the best solution.
- The Sigless Wonder
So we are left to be smart-assed and provide shots in the dark as to what the "best" solution is to this.
Of course, since these days most networks use switches instead of hubs, one has to wonder how much of a security risk sniffers are anyway. I guess there's always the possibility of ARP poisoning or some rogue router between the source and destination, but honestly, how likely is that? I guess it just depends on how secure you need your data to be.
And I've never understood why they think that opening up 6 connections when downloading a file would be quicker than just one. Quite apart from the fact that that insane kind of tactic can get your IP banned from some HTTP servers.
== Jez ==
Do you miss Firefox? Try Pale Moon.
is ncftp. It's got filename completion and all the file shell commands. I resort to it when mozilla chokes (which is most of the time) -- especially when getting large files.
To-do List: Receive telemarketing call during a tornado warning. Check.
Everyone disables the only unique thing FTP can do (third-party transfers) to prevent a variety of attacks by proxy. It might be worth accomodating decrepit machines that lack HTTP clients, but there are no actual advantages to using FTP.
Unfortunately HTTP is still a more secure protocol because HTTP clients can easily implement SSL encryption for passwords and sensitive data transfer. When you're dealing with a client who insists on using an outdated, but "easy" FTP client, it's not easy to tell them they have to change, and it's even less easy to tell them what they have to change to.
FTP is notoriously difficult to secure while retaining ease of use for the clueless end-user.
-Sara
I also assume the following:
I would say that you should go with HTTP for sure. Of course, you can provide both, but there are some key reasons for using HTTP.
Easier Configuration Perhaps I'm just not that swift, but I've found that web servers (including Apache) are easier to configure. This is especially true if you have any previous web server experience. Of course, the FTP server is more complex due to its additional features that HTTP doesn't have, but assuming that (c) is true, then you won't need to mess with group access control rights and file uploads.
Speed This whole "FTP is faster" stuff is not true. HTTP does not have a lot more overhead than FTP; it may even have less overhead than FTP in certain cases. Even when it does have more overhead, it is in the order of 100-200 bytes, which is too small to care about. HTTP always uses binary transfers and just spits out the whole file on the same connection as the request. FTP needs to build a data connection for every single data transfer, which can slow things down and even occasionally introduce problems.
Easier for Users Given assumption (d), your users will be much more familiar with HTTP URLs than FTP addresses. You could just use FTP URLs and let their web browsers download the files, but then you lose the benefit of resuming partial downloads.
Simple Access Controls Though some people need to have complex user access rules, you may very well just need simple access controls. HTTP provides this (look at Apache's .htaccess file), and you can even integrate Apache's authentication routines into PAM, if you are really hard core.
There are a few main areas where FTP currently holds sway:
Partial Downloads Web browsers typically don't support partial downloads, but the fact of the matter is that the HTTP protocol does support it (see the Range header.) The next generation of web browsers may very well include this feature.
User Controls Addressed above.
File Uploads Again, HTTP does support this feature but most browsers don't support it well. Look to WebDAV in the future to provide better support.
In summary, just use HTTP unless you need complex access rules, resumption of partial download, or file uploading. It will be easier both on you and your users.
We had a client a few years ago (about 1998), that we needed to set up data transfer for.
IBMGSA were running their IT systems, and they decided that FTAM (over OSI) was the way to go, citing FTP's supposed lack of reliability, and lack of features such as resume...
They were absolutely serious when they said that too.....
We eventually managed to get the whole setup working (FTAM, OSI, X.25 over an ISDN link)...but it was the most unreliable, cryptic, complicated system I've ever had to deal with...
A couple of years later, the client ditched IBMGSA, and the company that took over their systems took one look at the FTAM / OSI mess, and threw it out and replaced it with FTP over TCP/IP, and we've never had another problem.
Advanced users are users too!
just do what I do and mail people newly-obseleted floppy disks
Fully licensed blockchain psychiatrist
I would say go with http.. my reason is simple.. my college blocks ftp! There are two sides of the story, first off, the complete blocking of ftp started a week ago when they switched firewalls.. they couldn't get the ftp to work, and they decided it wasn't a high priority so they didn't do anything to fix it... After yelling at the IT department for half the day I got the priority up. But besides that.. Even before, when they allowed ftp, it was only active ftp.. I was unable to go to a website, click on a file that is served from ftp:// and download it! I'd have to use flashfxp or another client to d/l files.. I was able to figure that out quickly, but what about the other 99% of the school that wants to download say shareware from download.com or a driver from somewhere.. they are denied! FTP is a superior protocol for file transfer in my opinion, but administrators don't seem to care about it as much as they do http.
They serve different purposes, and each was designed to meet those needs. Each one has its own positives and negatives.
True, but ftp has far more negatives. The connection model ftp uses, in which the server connects back to the client for data transfers, is a horrible idea. Surely, there must be a benefit to this method (or else why would ftp have become such a standard?), but I've yet to hear it. Passive ftp (PASV) is supposed to compensate for the fact that clients behind NAT can't use ftp, and passive is pretty much supported everywhere now. But passive doesn't solve the problem of many-connections, it just shifts it: With passive ftp, there is still a seperate "data" connection for every file transfer, or transfer of directory contents. It's just that with passive ftp the data connection is initiated by the client to some high-numbered port on the server. This allows clients behind NAT to do ftp, but it makes firewalling the server a pain in the ass, especially if the server is (god forbid) behind nat itself. It's relatively easy to setup a mailserver, or webserver, or ssh server, on a machine behind nat (assuming you can forward ports on the nat gateway). But try doing the same with ftp. Passive connections won't work, unless the ftpd is configured to only use a certain range of high ports for it's data connection, and the firewall forwards that entire range. You'd think with all these connections, the protocol might support multiple connections from one login, ie list another directory while downloading. Well, thats not the case at all, you have to login again if you want simultaneous transactions.
And then if you want tunneled connections... oh boy, what a pain in the ass:
I set this all up for someone recently, and it DOES finally work, with tunneled passive ftp connections to a proftpd server behind a nat gateway, but it was a big pain. Due to the way proftpd works (not sure about other daemons...) the passive connections are directed to the interface the client is connected to. So, with an ssh tunneled connection, proftpd sees the client connection TO 127.0.0.1 (because thats where sshd is). So it tells the client, connect back to me for the data connection, my IP is 127.0.0.1. And the client tries to connect to itself, and fails saying "can't open data connection". The solution was to point the tunnel not to 127.0.0.1, but to the external IP, and have the nat gateway forward internal traffic back to itself. So traffic comes in over ssh, over to the firewallwall (where the nat rewriting happens), and then back to the same machine for ftp.
If people could just use sFTP clients, this wouldn't be an issue _at_all_ (One client connection, one tcp connection! Hey, what a concept!). But due to the integrated ftp clients in programs like bbedit and dreamweaver, I suspect we'll be dealing with that bastard protocol known as ftp for quite a long time....
Damn, reading back over this, I'm kind of ranting. BUT ITS ALL TRUE!
Sorry bout that.
[X] Post Anonymously
Because you get a larger share of the bandwidth pie. This applies both on the server end, and your ISP end. If you have bandwidth to spare, you can get 6 (or 8, or 10, depends on the client) users worth of bandwidth because each connection is treated by the server as seperate. On the other hand if you have a slow ISP, you can get a larger share of your ISP's bandwidth.
I have 3.5mbit DSL, but my ISP's performance is flaky. However, I have no problem pulling 300-350KB/s with a download accelerator.
The reason to use sftp on publicly available files or hhtps is so that people <*cough*>Carnivore</*cough*> can't track what I'm doing online. Sure, they can tell what sites you are visiting, but they can't tell what content you're looking at, or which files you downloaded... Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads <*cough*>Verizon</*cough*> and the RIAA wants access to those records... If they don't know what you downloaded they can't (potentially unjustly) prosecute you...
But the question brings up another point: FTP is a rather cumbersome protocol for transferring files. Ok, so you have two separate connections for entering commands and for sending data. Why then can I not enter any commands while data is being transferred?? Some FTP clients simulate this capability by open multiple control connections, but it really should be built into the protocol, IMHO.
HTTP was designed for small text file transfers, and not really for bigger files (although it does usually handle them pretty well).
I propose an "FTP2" protocol. This protocol will have the following features over and above the original FTP protocol:
Control connection is fully functional even when data is being transferred
Multiple data connections are allowed for each control connection (configurable by server operator)
Ability to queue files for download
Separate server-wide limits on control and data connections (for example, so I can log in and look around even if all data connections are used up. If I try to download something, it is queued until a data connection frees up. This puts an end to hammering on anon-ftp servers.)
Modular authentication -- ability to support authentication methods other than clear-text passwords
Support for encrypted connections
No server connecting to the client to establish a data connection, eliminating most firewall and NAT problems
MIME-type support, rather than clunky BIN/ASCII modes.
I'm not suggesting this is the answer to your problem, I haven't seen any GUI-based fsp clients (or a unix cli one for ages for that matter).
But it did seem very useful for what it was designed to do, serve out large files to anonymous outsiders.
Just raise the taxes on crack.
Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads Verizon and the RIAA wants access to those records
Why in Hell would the Recording Industry Association of America care who has software to break the DVD Content Scrambling System? They market music on CDs, not movies on DVDs.
Or did you mean to say MPAA?
Yeah, yeah, it's just a careless error, but things like this make us all sound like a bunch of idiots. If we can't even keep track of which evil corporate organization goes with which issue, then how is anyone supposed to take us seriously?
David Gould
main(i){putchar(340056100>>(i-1)*5&31|!!(i<6)<< 6)&&main(++i);}
Being in IT for a large Fortune 500 company that sells an operating system among other things (no, not Microsoft), I can share some of my expereinces with you. So take it for what it is worth.
Our FTP servers run both HTTP and FTP providing the same content in the same directory structure. There are five servers that transfer an average of 1-2 TB (terabyte) per month each, so they are fairly busy. On a busy month each server can go as high as 7 TB of data transferred. File sizes range from 1 KB to to whole CD-ROM and DVD-ROM images. I think the single largest file is 3 GB.
The logs show a trend of HTTP becoming more popular for the last several years and not stopping. It is currently at 70% of all downloads from the "FTP" servers via HTTP. While the remaining 30% is via FTP. Six years ago (I lost the logs from before this time, they are on a backup tape but I am way too lazy to get that data), it was completely reversed. 75% of downloads were via FTP and 25% were via HTTP. 90% of all transfers are done with a web browser as opposed to an FTP client or wget or something.
One thing we learned was that many system administrators will download via FTP from the command line directly from the FTP server, especially during a crisis they are trying to resolve. They do this from the system itself and not a workstation. The reasons for this are a bit of a mystery. Feedback has shown that we should never get rid of this or we might be assassinated by our customers. We thought about it once and put out feelers.
I would say if you don't need to deal with incoming files and you file size is not too large then stick with HTTP. Anything over about 10 MB should go to the FTP server. An FTP server can be more complicated. It seems like the vulnerabilities in FTP daemons has died down in the past year or so. Also, fronting an FTP server with a Layer 4 switch was a lot more tricky because of all the ports involved. If you want people to mirror you then go with FTP or rsync for private mirroring. In reading the feedback, most power users seem to prefer FTP, perhaps because that is what they are used to. Also, depending on the amount of traffic you might need to consider gigabit ethernet.
The core dumps being uploaded are getting to be huge. Some of those systems have a lot of memory!
It'd be nice if Phoenix and Mozilla would acquire that ability. For some reason the developers' stated position is that it won't happen anytime soon, but one can always vote for the bug anyway.
The problem with using HTTP for large file downloads is that, in most cases, it's cheaper ressource-wise to span multiple FTP simultaneous connections than HTTP connections. Of course, this only becomes a real problem if you have more than a few hundred virtual hosts on a single box. So save your httpd processes, and use FTP for large files.
In FTP the sender connects to the receiver, but client issuing commands does not have to be either of those two machines. As designed, FTP should be able to allow a client to send data between two arbitrary machines. Its an infrequently used feature, and I'm not sure if it is implemented by all clients, but it is, technically, a feature that HTTP doesn't have.
Starting multiple TCP connections for a single file download can be advantageous, because of congested network paths.
If there are 500 TCP downloads ocurring, each download will theoretically get 1/500th the bandwidth.
Therefore, by opening multiple TCP connections, you will increase the amount of bandwidth for your transfer, at a cost to everyone else using the connection. This is because you've effectively doubled the size of your receive window (one for each connection), causing the host you are downloading from to stuff that many more packets down the pipe.
The problem is, when everyone does it, it completely negates any advantage to using this method. It also leads to packet loss, since you have that many more TCP connections (each with its own receive window) fighting for pieces of the pie.
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
I suspect that's because 99% of people are downloading from one of the FTP servers.
I put to you that would be more logical to suspect it's because HTTP is faster than FTP as a transfer protocol. It generates less traffic (and uses less CPU overhead) which means downloads end quicker.
Additionally the CPU overhead generated by FTP connections also causes many sites to limit the number of users who can connect, which often results in 'busy sessions', something much rarer with HTTP (as HTTP servers typically have very high thresholds for the number of concurrent connections they will support). The overhead on a server of a user downloading a file over FTP is much greater than that of a user downloading the same file over HTTP.
Although FTP is of course theoretically more reliable than HTTP, in practice, because of 'Server busy: Too many users' messages combined with the speed and reliability of modern connections (which in turn makes HTTP more reliable) mean the the reverse is often the case from a user perspective - which is what I think the poster is getting at.
This may be partly due to poor FTP server configuration defaults and/or poor administration, but they cannot shoulder all the blame.
The potential lack of reliability with HTTP is a very minor issue these days, and the extra overhead of integrity checking files in addition to relying on TCP is just not warranted for all but the largest of files.
This doesn't make FTP completely redundant, but it does make it make it redundant when your files are small and your users are on fast, reliable connections (though the value of 'fast' varies in relation to the size of the file, even 33 kbps is 'fast' compared to the speed of connections that proliferated when the File Transfer Protocol was developed).
Dan J. Bernstein has written a fantastic, lightweight server that will serve files via either or both FTP and HTTP depending on how the client connects.
If you want to serve files to the public, this is the most secure way to do so. If you need to provide the files to only certain logins, use something else. If not, you can run this on very lightweight hardware and if it's the only server running, you won't get hacked. Period.
I see too many FUD replies here:
:-)
1) HTTP doesn't support resumed downloading.
- That's ridiculous. It has since HTTP/1.1 years ago. In fact, it can even do things like request bytes 70,000 - 80,000, then 90,763 - 96,450, etc.
2) HTTP doesn't support security/authentication
- Ridiculous. HTTP has an open-ended model for authentication and security, many of which are secure and standardized. If you REALLY need security, use HTTPS.
3) HTTP doesn't support uploading
- HTTP/1.1 has had this for a while. Netscape 4.7, Mozilla 1.1, and IE 4+ support this. I must admit though, it sucks.
Several people have pointed out the real differences:
1) FTP doesn't like firewalls
- Passive FTP fixes this, but it has quirks and limitations.
2) FTP supports directory listing, renaming, uploading, changing of permissions, etc.
- This is what FTP is for
- This can be done in HTTP, but requires serious work
- If the scope creeps, shell access would be better.
Would use sftp in a SSH tunnel over IPv6 encapsulated in IPv4 over X-25. Don't forget to encrypt, obviosuly with an unknown russian algorithm, not only the files but also the instructions on how to download'em. If you have some spare time, instead of sftp you can develop your own [undocumented] protocol.
This way you won't look like a wimp. Jaws will drop, you'll be recognized as a uber BOFH and your peers will respect you.
I am not sure if there is really a purpose to this and am just throwing ideas around. But you could host an FTP on an internal network and route it through an HTTP CGI script. You can also use encryption over the HTTP connection to make it secure without resorting to SFTP. I just happen to use this method myself since SFTP requires special clients and is a bitch to setup...just some food for thought
scp is your friend. Learn how to use it, and it will handle all of your (non-anonymous) file transfers. It is a beautiful thing.
Using your sig line to advertise for friends is lame.
I'm not totally up on HTTP and such, but why is it a royal pain? Assuming the user is going to download all 3 ISO images one way or another, what's the issue if he grabs them using 6 streams in 1/6th (roughly) of the time, or in one stream for the full stretch? It's still bandwidth being eaten. I guess it boils down to time versus users.
So, in other words, her brother is a dumbass...
A pretty distinct advantage for those of us who use shells often or have to repair machines at a command prompt.
.sig wanted: Must be concise, funny, and display my cleverness.
I run a ftp for similar file-sizes (1-6 meg) using a Windows 2000 Pro box (yeah i know i should stick to my preachings about the wonders of linux but im not 100% with my abilities to lock down linux yet) and im using Bulletproof FTP server which is hella cheap but has every feature you can need and is very secure. I highly recommend it. It handles beautifully.
"Self-destruction might be the answer" --Tyler Durden
sftp incurs a terrible CPU overhead, especially if many people are going to be downloading at once. I doubt most web servers could concurrently handle a few dozen 3des encrypted sftp connections without slowing throughput, and if you're hosting files, thats the last thing you want. FTP is supposedly more bandwidth-efficient (though I've never seen proof), but I can still get 400k/sec downstream over http, so I don't think it's a huge problem. I'd just use http for the ease of setup. Securing a public-access ftpd is a true pain.
lynx, wget, and fetch, all work over http.
Video for Online Dating Profiles
Hmm.. There's no reason why you couldn't just perform the download via HTTP if your target customer or audience has the bandwidth.
If you're concerned about the customer losing connectivity, you could always just break the file up into small chunks for download.
Both FTP and HTTP, depending on clients and servers, supports continuation-downloading(ie, recovering from where you left off.) So if one connection flakes out, you can always recover.
The long and short of it?
HTTP(S) or FTP(s/FTP), it doesn't matter. With both, you can restrict access and/or grant access to specific files. You can recover from some point or offer split files. With HTTP, you can have the information piped through a back-end piece of code so that you can monitor who does what when, but something similar can be done with FTP as well.
If you've already got a webserver going and secured, you're probably better off offering the files via HTTP. Nothing unprofessional about it.
Fact is, if your target audience is the general public, you'll fare better with HTTP. Why? Because while most people who are online will probably have access to a web browser and know how to download HTTP files, they may not have an FTP client or necessarily know how to use their browser as an FTP client.
Me? I offer photos and docs online via HTTP. It's easier and I can control the visual aspect of the "download area" to create a more professional look than that of the "industrial" FTP interface.
Nothing wrong with FTP or sFTP. I use them at work and at home. But then again, when I find a site which offers HTTP or FTP? I pick HTTP to download first. It's only when I'm still looking for something that I go with FTP... which, by the way, becomes difficult to navigate on some clients. A nicely generated webpage featuring the item via HTTP would be much more elegant.
Just my two cents.
Winged Power Photography
If it ain't broke, don't fix it.
This sig no verb.
Well there's always the option of FTP over SSH2. I'm sure you can find a Java applet that will do the SSH2 and make the tunnel needed for secure FTP.
Now when we talk about Java, there's another possibility. Some sites (cr@ck) use a Java downloader. It doesn't mean that the Java applet that downloads the file uses HTTP or FTP, it can be some sort of propriety protocol (or you can combine the best of both worlds.)
One way is to have the applet on a SSL'ed (https) page and it does some decrypting as it downloads a pre-encrypted file from your FTP. Or the person can just download the encrypted file directly and use the applet on the secured page to decrypt it. There's ALWAYS a way to have your cake and eat it all by yourself, too.
Lynx lets you browse, but you can't do globbing, so you see lots of irrelevant crap, and you have to select files to download one at a time.
For getting (possibly multiple) files whose location you don't know in advance, FTP is more flexible and efficient.
The ocean parts and the meteors come down
Laid out in amber, baby.
Most web servers allow a max number of connections. If one user is eating up six connections that is potentially five fewer people that can download the files. With the case of ISOs the distributor probably has more bandwidth than the person downloading does. Hence it is more effective to serve as many people with as much bandwidth as possible. It's really a curteousness issue to the server operator to not open six connections when one will do.
- Do you have bandwidth issues? If you are serving files to many people, FTP servers allow maximum concurrent users, which can be useful. I know you can do this with HTTP, but it's difficult to segment the downloading >1Mb files traffic from the normal site traffic. A separate service also allows you to use all the Quality of Service stuff in the 2.4 kernel nicely.
- Do you have a large array of files that the user might want to download, such that using an FTP client to ctrl+select multiple files is the right answer compared to having your users click on twenty links and have to cope with twenty dialog boxes?
- Do your users need to be able to upload files to you? This can be done with HTTP, but you'll need some PHP processing or similar on the server, it doesn't support resuming, and it won't work through many company firewalls, and therefore isn't a good option. HTTP uploading it particularly hopeless for large files, as it provides no user-feedback.
However, you should NOT use FTP if you answer no to either of these:- Are you running some flavour of unix? There just aren't any robust Windows FTP servers. Yes, I'm prepared for the flame war about this.
:) - Can you be bothered to keep your FTPd patched? ProFTPd and WU-FTPd are both frequent appearers on bugtraq. You need to stay on top of the patches, or you will be 0wn3d.
Simple, see?In my case, I run an Apache server on my file server box because it gives me full 100% throughput at Ethernet speeds. That's 10+ megaBYTES/sec. The average anime episode downloads in 10-15 seconds, and when friends come over, I don't have to worry about them having the right kind of client when they want to leech.
Where FTP becomes useful is when you want people to upload to you.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
Who's the user? If it's WIndows lusers, then by all means use http. It makes their life easier. But if the users are *NIX, Mac or other, then use ftp. These folks have a clue and can deal with it.
Warning! The above is sarcasm. But it is so apropos. Paraphrased quote from a conversation I had with a Windows luser:
"I need to find a good FTP program so I can download this software I found. Which do you think is better, ProFTP2K for $25, or should I spend more for EZDownloadFTPPlus at $50?"
A Government Is a Body of People, Usually Notably Ungoverned
After hosting an HTTP file transfer area for some time for my company, we decided to move to an FTP setup that was a bit more sophisticated.
So far, it's been a failure for two reasons:
1. IE blows as an FTP client, and users aren't comfortable dropping into the (somewhat crappy) DOS FTP client.
2. Firewall setups at the fortune 500 companies that we deal with normally seem to keep FTP access off-site restricted.
Why are you letting these clowns ruin our country?
> For getting (possibly multiple) files whose location you don't know in advance, FTP is more flexible and efficient.
:>
Ever heard of web page directories that almost every common web server uses, and most web servers automatically do when no index file is present?
I find lynx+ web page directories a lot faster to get files across my network than ftp.
The only real advantage I see with ftp is uploading files quickly and easily... but that's not for me
I just set up WeebleFM http://sourceforge.net/projects/weeblefm/
It's a PHP front end to FTP. My FTP ports are only open to the loopback interface. Users get the usablity of a clean web interface, and I get to have encrypted password-controlled FTP on a box that only has port 80 open to the internet.
WeebleFM uses mcrypt to encrypt traffic (and I'm pretty sure I could get it to work over https).
Using standard unix permissions, a careful directory schem, and vsftpd's chroot capabilities, I can have an internet filesharing arrangement with blind drop boxes, a group accessible directory and any number of world readable directories.
That's not true--they can tell roughly what the size of files you're looking at, and if you're browsing, which files were triggered-downloaded--for example images embedded in web pages.
All this information makes it trivial to find out which pages you downloaded, when, and how long you visited.
The only thing it doesn't do so well is listen in on data you send. But consider--if they're listening to the tcp channel to the remote site then they can listen to outgoing tcp connectivity as well.--and correlate what information came in, and what information went back out.
You think a single encrypted channel buys you any privacy? Get real.
FTP implementations frequently use a fixed, small window size. HTTP on the other hand will honor the system limit, almost always larger even without tuning.
t mls _tcp tune.html/ faq.html
Dramatically simplified, it means that the connection can send a lot more packets without hearing back from the far end, enabling the connection to reach higher speeds (imagine a phone call where you had to say 'okay' after every word the other person said. Now imagine only having to say it after every sentence. Much faster.)
The tiny window size of (most crappy legacy implementations of) FTP starts to affect download speed at just 25ms latency, and has a huge effect over 50ms.
A properly tuned system with HTTP can make a single high-latency transfer hundreds or even thousands of times faster than FTP.
Relevant links:
http://www.psc.edu/networking/perf_tune.h
http://www.nlanr.net/NLANRPackets/v1.3/window
http://dast.nlanr.net/Projects/Autobuf
What are you using Lynx to connect to the ftp server? That's the reason it's slow. Try a real client like NcFTP and when you do, make sure to take note of the sheer power.
mget blah[1,2,3].iso....
Get the drift? HTTP indexes are rather stupid if you ask me, it's FTP without the features. And before you whine "But I don't need the features", neither do I most of the time, but it's nice when they're there.
In Unix, you can background this operation fairly simply...
This will start downloading your file in the background, will autoresume if something happens, waiting 15 seconds between retries, and never give up until the file is completed or the source no longer has it. You'll get an email when it completes one way or another.
Help us build a better map!
I tend to agree with this, but for different reasons.
If you are downloading a file off of a remote server, then there are one of two possibilites:
1) you know the exact address to the file you are looking for... in this case ftp provides no superior advantage over using lynx or wget since in either case you could have been given the direct URL... either provided as an http url or an ftp url. Basically my point here is that an ftp url is no more or less useful or easy to remember than an http url.
2) you don't know the address of the file you are looking fore... therefore you are pretty much required to browse via http, to find the site (or page) you want to download from... so since you are already forced to browse for the site, then you might as well use the browser to download. For most people that use graphical browsers, this is great... for those of us (myself included) that use shell browsers (ie lynx and links), this poses little problem as well (unless javascript is required to download a file... I friggen hate javascript... people who use javascript in their websites and have a choice should be fired [note, I use javascript in my works' website... but they make me.. I don't have a choice]).
So I should abandon encryption and just be free to let whomever listen to traffic coming from my box? If the the images are locally referenced, they'll be transferred securely as well (talking https here). And modern browsers keep connections open for reuse, so that only thing that can be gleaned is when a spike happens, which means that you are loading content from the same site, but not how big it is, since a web page is a bundle of different elements that need to be retrieved.
Also, if you think that just knowing when data packets are sent and received tells you the contents of the data, then why hasn't SSH been broken yet? With those assumptions, it should be easy to use a statistical model of people's keystrokes and break into a session. Realistically? Not possible.
You've got to start somewhere, especially if you're going to get the world to transfer to using encryption everywhere. The mail server I use for private mail among friends has no unsecure channel going out of it. All of it's protocols are protected with SSL/TLS, including SMTP inbound and outbound. And besides, the argument I was making was for using encryption to connect to a public mirror site. Since the majority of mirrors have HUGE numbers of different files, and the amount of control data sent is never constant, knowing how much data is sent in either direction will NOT be a guarantee that you know what was transferred.
AND, I was talking about legal prosecution (or persecution, depending on how you see it), and a document that says you downloaded approximately enough bytes to be a suspect program isn't enough to stand up in court...
FTP, in my opinion is antiquated.
In the uploads arena, I could transfer any file faster on rsync or scp, or even cvs -z5. And thats over 56k.
FTP isn't so secure, it's pratically text on port 23 (wonder why the warez sites still use it, aren't they trying to avoid goverment agencies? and M$ and all that crap?).
HTTP is faster for smaller files as FTP eates up a whole lot of time doing the connection procedure.
The biggest advantage of XML is that software does not have to be changed for a different XML data format - they all use XML, the standard bracketing syntax. I know most major browsers support an XML view mode, so they don't need to be upgraded to download by XML.
I'm sure that there are people out there saying that XML is inefficient, but that's simply not the case. We can use special XML commands that allow us to include large blocks of binary data -- at the expense of portability to 14-bit computers, of course. In total, an XML download should only have about 30kb of metadata added to it. Author, guid, PGP signature, original source, license, and all that good stuff.
We can hope for the day when the need for binary transfers will be over, since everyone will be using XML files.
Why should you have to write a script to download files? Choice FTP programs let you queue downloads just fine.
If you're talking about the human engineering aspect of this discussion only, then I have no disagreement with you. However, FTP is just as technically feasible over SSL, since SSL works at a lower level on the network stack than FTP.
Furthermore, there are good FTP clients that have SSL support. For example, CuteFTP supports FTP over SSL (and has a very user-friendly interface, for the clueless end user).
There are a good number of servers supporting FTP over SSL. ServU and Sambar are some of the windows servers. Just do a google see what else there is.
Seems to be a lot of comments about firewalls and FTP from people who obviously don't work with them. Remember there are three basic types of firewall technology: packet filters, proxies, and stateful inspection.
Packet filtering alone is always a problem because you have to open up all of the high ports.
Proxy firewalls and FTP (active or passive) are a no brainer as long as either feature has been enabled. Remember that proxies "watch" the conversation so it will manage the connection if it's data coming back to the client on port 20, and will recognize the 'pasv' command in the command channel.
Stateful Inspection firewalls include proxying code for the major protocols ie FTP, HTTP, Telnet, etc. So you are covered here as well.
If you are having problems using FTP through a firewall then you are probably:
-Are being blocked intentionally
-Have a lazy security admin who hasn't updated the firewall in five years
-Have a stupid router jockey "securing" the network with router ACLs (packet filters).
As long as you are using a major firewall release like Checkpoint, PIX, Netscreen, IPTables, etc, that is up to date there will not be an issue getting FTP to work.
It seems everyone talks about DOWNLOADING. How about UPLOADING? How fast and wonderful is HTTP for that?
I have never thought of FTP as a crumbling legacy protocol full of security holes. Each protocol has its flaws, security flaws or not. HTTP isn't free from security holes.
They are two different protocols designed for two different purposes. If you plan on transfering large amounts of data, and want it to be more reliable, I would most definetely recomment FTP. FTP is made for transferring files, and that is what it is best at doing.
If you are not concearned with data integrity and plan on transferring decent sized files, I would wish you the best of luck with HTTP.
As far as security holes go? I have no idea, but I imagine both protocols have had their fair share of security issues.
For more reliable transfer of larger files, I would really recommend FTP.
---
Mike
I'm going to kick the next person that I see with their karma rating in their sig.
You might consider setting up a BitTorrent tracker on your website, and making the files available that way. It's not any worse than website bandwidth, and since you're also leveraging the outbound bandwidth of sites that download the file, it could be faster.
Editor Emeritus and Senior Writer, TeleRead.org
Serve out anonymous FTP through public file (http://cr.yp.to/publicfile.html). Then there aren't any security holes.
Really. The security holes in sendmail can be fixed by installing qmail. The security holes in BIND can be fixed by installing djbdns. The security holes in WuFTP (and most others) can be fixed by installing publicfile. There are also other good programs out there as well.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
If you're providing downloads that may reach a decent volume, I would recommend a caching appliance. A CacheFlow server (they were bought out by BlueCoat systems) or something similar should do the trick. These can cache HTTP and FTP inbound requests, and instead of having the load on your servers, these things will serve up your files instead. A good cache appliance can serve up files a lot faster/more-efficient than a web server, and you won't bog down your box with processes.
These caching appliances can also broadcast multimedia streams (Real or WinMedia) with the correct licenses.
I would do some research on some CacheFlow boxes, which, by the way, are selling on the cheap at eBay. These were the $25,000 (circa 1998-2000) priced appliances that you can now pick up for, sometimes, in the hundreds of very low thousands.
Serve up over P2P protocols, and tell people what filename to look for. If the file becomes very popular, you'll only need to serve up the page giving directions on how to download it :)
(yes, yes, I know, requiring people to use spyware-laden software is not nice, but it's a good idea in an ideal world)
Stop the brainwash
Instead of banning download managers that can do segmented downloading, why not just limit the number of connexions from a given IP? That solves both the segmenting and the "tons of files at once" problems.
I can make a dozen or more connexions to your FTP server with nothing more exotic than Netscape. Why pick on download managers when they use the same number of connexions? (BTW, Getright says right in its configuration that "some servers regard segmenting as rude" and recommends against it.) Better to limit connexions to x-many per IP address, and let the user spend them any way they wish.
BTW, if you do limit connexions, please remember that it usually takes one for browsing the site (using Netscape or whatever) PLUS one for the download manager to fetch the file. Otherwise the user who was looking with a browser has to leave the server, then wait for the browser connexion to close (which can take a while) then finally paste the link into the DLManager. So a limit of two connexions from a given IP is a nice practical minimum, and surely not a hard load for anything outside of home servers operating over dialup.
PS. I love FTP's convenience, and I always try to be extra-polite to small servers (and not rude to big ones). I do use Getright, and have segmenting disabled (which BTW is the default).
~REZ~ #43301. Who'd fake being me anyway?
Stand on your rooftop with a CD burner and frisbee CDs to anyone who calls to initiate a transfer!
The ONLY way to serve files!
The main strengths of HTTP over FTP for file transfers are:
The other differences one sees are due to server design issues. I.e. most FTP servers are large and spawn a process per connection, which makes FTP sessions much slower than HTTP sessions. But if you want to use FTP, there are very fast FTP servers out there.
Overall, in today's world, it does not make sense to use FTP unless you have a requirement from your users. For public access to files, use HTTP or something more modern, such as rsynch, or a P2P network.
As usual, you should answer such questions by thinking about your target users and asking yourself what they are likely to be most comfortable using. Chances are it's their main tool, the web browser.
My blog
"No standard directory listing"
Neither has HTTP.
"Bad error recovery, many differing standards on restart"
Erm, yeah, it's so non-standard that prozilla doesn't exist - exploiting the `resume' potential of most FTP servers to parallelize downloads starting 0, 25, 50, 75% through the file. Yeah right.
"Designed for human readable command lines not machine readability"
Oh, come *on*. When there's no standard output format at all in HTML, how can an `ls -l' output not be vaguely standard? Are you unaware of the SIZE FTP command?
"Password sent in plain text, not even concealed"
And somehow HTTP with its sending of usernames and passwords also in plaintext is "better"?? Does it look flourescent pink on your screen, or something?
"HTTP: none of the above"
Poster: no clue what he's on about. HTTP: all of the above. Readers: much annoyance.
HTTP is more easily integrated with SSL in the form of HTTPS.
Almost everyone has an ftp client. You have to install curl or wget or lftp yourself consciously in most distributions.
HTTP: no standard for file uploads, you rely on server-side configs and handlers for the purpose. How mad is that??
~Tim
--
Rushing on down to the circle of the turn
I used to run a server which distributed ~3TB/month. Initially I served these files via proftpd, but it soon became apparent that ftp daemons are far too bulky for high-volume serving.
:)
Enter apache. On the same hardware which keeled under around 30-50 ftp sessions, I could handle over 400 concurrent http sessions, with plenty of ram left over for the vital cacheing
All most people would want to do with FTP can be done with HTTP.
There are hardly any practical legitimate reasons for an FTP server to shove data to some other place rather than the FTP client and other dangerous combinations FTP allows.
Uploads? Uploads had better be disabled on FTP by default. Uploads are disabled in HTTP by default, but you can configure your site to accept uploads.
Mass downloads of files? Just standardise. As is you can already download entire websites. With HTML (not javascript) the links are there. If you want files only you just have to standardise on a program readable way to figure out which links are files to download.
Resuming of downloads - works on most webservers, works on most ftp servers.
And http allows more - better compatibility with firewalls and proxies. Can be made to support SSL easily - most popular clients support SSL.
---
FTP is a bad protocol. The PORT and PASV thing is stupid. Most protocols which specify network layer data (IP address, port) in the content layer for programs to use, are bad designs.
How would PASV or PORT look with IPv6? With HTTP, you don't have to change the protocol much if at all.
FTP traffic is given lower priority than HTTP traffic in a large number of packet shaping / DiffServ type routing algorithms.
These algorithms are based on the assumption that HTTP traffic consists of fairly short bursts and not long sustained transfers which is typically what FTP traffic looks like. Based on these assumptions, these routers give lower priority to FTP traffic than they do to HTTP.
This does not mean that you should serve large files off HTTP since it'll be "faster". Au contraire, it means that you should be fair to others and serve them over FTP, so that the routers can do the correct packet shaping even if it means a slight speed hit to you.
Think of people downloading huge files off your web server and screwing up your warcraft (/quake/whatever) game.
alot of companys only have one external ip.
that'd mean only one user in the entire company can use the server - bad.
It's pure genius! Transfer files via ICMP echo requests! Add a special byte in the ICMP header or data to make the target recognize it as a file transfer and not spam you with echo replies.
Why not implement FTP over ICMP? It will be the best ever!
It sounds like your objective is to make files available for download by the public. That being the case, your best solution is to provide both methods and let the person downloading the file determine which method is better for his/her/its needs.
Some will prefer ftp because it's faster. Others, especially those behind overly-restrictive firewalls, will find that http is a more usable alternative.
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?
There is a fairly usable client that does both SFTP and FTP over SSL.
z illa/
FileZilla
http://sourceforge.net/projects/file
--
Time is on my side