Slashdot Mirror
FTP: Better Than HTTP, Or Obsolete?
An anonymous reader asks "Looking to serve files for downloading (typically 1MB-6MB), I'm confused about whether I should provide an FTP server instead of / as well as HTTP. According to a rapid Google search, the experts say 1) HTTP is slower and less reliable than FTP and 2) HTTP is amateur and will make you look a wimp. But a) FTP is full of security holes. and b) FTP is a crumbling legacy protocol and will make you look a dinosaur. Surely some contradiction... Should I make the effort to implement FTP or take desperate steps to avoid it?"
They serve different purposes, and each was designed to meet those needs. Each one has its own positives and negatives.
...use sftp!
I haven't really noticed any reliability issues with http anymore. If it starts loading it usually finishes, and I haven't run into any corruption problems. Maybe if you were serving huge files ftp would be a good idea, but for 1-6 mb it's probably not worth it.
--sex
Very popular slashdot journal for adul
Use telnet and screen capture the VT100 Term buffer!
HTTP is the way to go, especially for larger files, because you can see the download rate and a better estimate of how much longer before the download is complete. The "HTTP is for wimps" mentality is for fools. You normally don't see the download rate when using FTP, unless you're using a special ftp GUI....which, could be argued, is for wimps.
But in my experiences, HTTP for whatever reason goes faster (not entirely sure why), and FTP doesnt work for some because of firewalls.
Try both - see which gets used more.
The One Rule Of Chess You'll Ever Need: Don't play someone who carries a kit in their bookbag.
rsync is a great protocol, fairly robust, can be wrappered in ssh (or not), supports resuming transmission, and operates over one socket.
seems like the best of both worlds to me.
the real question is - do you control the clients that are going to access you? or is it something like a browser (which doesn't support rsync).
Do it for da shorties
"HTTP is slower and less reliable than FTP"
I would think FTP is slower since with FTP you have to login and build the data connection before the transfer begins. With HTTP it's a simple GET request.
As far as the actual data being sent, I believe that the file is sent the same way with both protocols. (just send the data via a TCP connection). I could be wrong though.
I'd implement FTP if your attempting to transfer large files. There are many excellent programs to resume FTP transfers yet I do not know of any such programs for HTTP transfers. Real Download might do it, but who in god's name would want to inflict that upon themselves.
Granted I'm not an expert on the security flaws of either the HTTP or FTP protocols, but I can say that most commercial enterprises provide download services for both.
just dump passworded zips to usenet
As long as you're willing to secure your FTP server and do the simple stuff like watch out for file permissions - FTP is much better.
HTTP is restricted by browsers, many of which will not support files larger than a certain size. Furthermore, FTP allows for features such as resume, etc...
The real question, however, is what are you trying to use this for? What's your intended application?
If it's a file repository for moderately computer literate people - FTP is definitely the way to go.
If it's a place for average-joes to store pictures, maybe HTTP is your best option. Sacrificing a bit of speed and capabilities such as resume might be made up for with ease of use..
http://www.babysmasher.com
http://www.openingbands.com
do as mysql site does and others... supply links for both options
Its encrypted it is easy to get through firewalls, and it lets you run rsync (which lets you synchronise two machines automatically).
Ftp is a major pain for firewalls because unless you use it in passive mode it opens another port back after the initial connection.
IE, especially 6, doesn't seem to handle FTP well, so if you expect Windows users, consider IE performance
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
It's generally simpler to get to from a browser, which is where 95% of people's online life is anyway. Yeah, you can rig up a FTP URL, but it seems a bit kludgey and more prone to firewall issues.
SO YOU'RE GOING TO DIE: The Comic for Dealing with Death
Actually, http is faster. FTP keeps open another connection to control the file transfer. This connection also uses bandwith, just a little bit, but enough to matter.
hey! nobody likes a fence sitter.
a) it is more easily secured
b) it is easier for people to use (ftp url's tend to confuse some people)
c) it seems like it is easier to overwhelm an ftp server with traffic than it is a webserver
1-6mb files? :P
heh, most 1-6mb files I see are on irc fserves
If the right tool for the job is efficient, let
them name-call all they want.
As far as I've seen, HTTP for FTP type operations
is still somewhat new. While FTP has it's firewall and security issues, It is still used a whole lot
more than HTTP IMHO.
If this is your first FTP application and you are already running HTTP alot, you might consider just
using HTTP. If you plan to do alot of FTP hosting, I would suggest cracking the FTP server books.
HTH.
what about gopher?
I'll be leaving my current DSL provider because they will be removing FTP from the standard services they provide. (SBC to SBC Yahoo!)
Those of your customers who don't have fast access to the internet may appreciate even a slightly faster standard.
HTTP is not "slower", "less reliable" or "amateur", and the people who say that are idiots. If it was "slower" or "less reliable", than why would web sites distribute graphics, or binary downloads, or banking information, or purchasing with it ???
FTP is outdated, but useful if you want the user to be able to navigate a directory structure with a simple command line client. I'm sure a lot of free software developers prefer FTP, at least ones who were on the net before HTTP was common. But it is brainless to say that HTTP is "less reliable" or any such bullshit. This web page came up for you, didn't it ???
My god i wish there was! Every time i need it, i have to go look for putty, and download psftp. Maybe we should write a patch for mozilla?
But what am I?
HTTP does not have firewall issues, does not need authentication, does not (by default) allow directory listings, and is the same speed as FTP. It's a good deal for general file distrubution.
FTP is quickly becoming a special-needs protocol. If you need authentication, uploads, directory listings, accessability with interactive tools, etc. then this is for you. Mainly useful for web designers these days, IMO, since the site software packages can use the extra file metadata for synchronization. Other than that, it's a lot of connection overhead for a simple file.
FTP does have one nice advantage that HTTP lacks: it can limit concurrent connections based on access privleges (500 anonymous and 100 real, etc.). Doesn't sound like you need that.
Go with HTTP. Simple, quick, anonymous, generally foolproof.
HTTP file transfers are easier to get through corporate firewalls.
1 to 6 megs, huh? Why not use Kazaa like everybody else? :-P
"If he thinks he can hide and run from the United States and our allies, he's sorely mistaken." Bush on bin Laden
Last time I checked the best way to evaluate a technology was to see if it meets your needs in a secure fashion. Calling FTP a "Dinosaur" shows that you are only interested in being 3733t and not being really secure. If you really want to impress the script kiddies on IRC try something challenging like tunneling FTP through SSH or use something like SCP to transfer files.
"Freedom of speech has always been the abstract red-headed stepchild of the Constitution"
-Suck
We run ftp, but we have to have people send us files, and also distribute them on a regular basis.The client software available for doing the sending and receiving on a regular basis is a lot better for FTP.. it's pretty klunky, but it is very doable for http.
We just choose to stay on top of our ftp updates.
"HTTP is amateur and will make you look a wimp"
You really gotta watch out for things like this. I know one guy that got a 'click me' sign on his back because he used HTTP instead of FTP.
I'm really surprised at this question. Those are small files. Serve them any way you want. If I were posting ISOs or something, it seems more "natural" to serve via FTP.
However, I'm sure you'll be serving them via a "Click to Download" link, which it doesn't matter. You click, the browser downloads. End of story.
I'm really interested in Missionary vs Doggy Style. Missionary is traditional, old-skool, and makes me feel dominant. However, doggy style gives me that porn star class that I crave in my lovemaking sessions, not to mention the hairpulling and ass-slapping possibilities. But you dont see me asking Slashdot. Even though it is "stuff that matters".
I personally loathe FTP, except the times that I need to connect to my file server--which it actually does quite well. I am running a Linux box that stores all of my music, my documents, and more, and I simply connect to that machine when I'm at work or elsewhere. So, just for this reason alone, FTP is excellent. In fact, it's actually saved me a few times, when I've noticed that I hadn't printed a document or finished a website, and I needed my files.
However, the security holes of FTP would prevent me from ever keeping FTP connections open on high-traffic websites. Yes, I maintain FTP accounts on my servers to allow updating the site, but no guest accounts are active (not that my server is magically "safe"), and the sites hardly receive an ounce of traffic.
The Political Programmer
And I wouldn't care about the opinion of someone who would actually judge you over what friggin protocol you use to provide downloads. Such an utter nerd is somethig that I can not relate too. Maybe after I use Linux for a few more years, who knows.
The question is, "what do you want to do?" I run an FTP server (incidentally affiliated with etree.org, lossless live music!) and I need what it can give me. Namely I need multiple classes of login, each with a different
1) number of available slots
2) speed limit
3) premission set
Some people can only read files at 60KB/s, some can read and write (to the upload dir) at the same speed, come can only browse, etc. etc. For this kind of a setup, FTP is great _IF_ you keep your software up to date; subscribe to bugtraq or your distro's security bulletin or both.
On the other hand, HTTP is great when you want to give lots of people unlimited ANONYMOUS access to something. I'm sure there is a way to throttle bandwidth, but can you do it on a class by class basis? In proftpd it's a simple "RateReadBPS xxx" and I'm set.
As always, choose the tool that fits _your_ purpose, not the one that everyone says is "best"; they both have good and bad qualities. And http can be just as secure/insecure as any other protocol.
The time of day is 29:33.
If you want the real answer, it's ftp. All the way. Unless we're talking Gig-E. The specs for 10BaseT and 100BaseT will only allow protocols such as ftp to even get close the the supposed throughput.
I'm sure others can add details to this.
Though I'm tempted to say for a less skilled downloading community you should use http and the reverse for ftp. But now I'm thinking that doesn't quite fit since one can always wget a http file (or ftp file for that matter) and one can use ftp though a web browser.
Eh, my vote is https, more secure, and maybe more manly since it's not port 80? Of course moving out a directory structure is more natural for me than to click on pretty little icons.
Therefore I conclude use ftp (or sftp) if you got users the like command prompts. Otherwise just go for the one whose logs are more fun for you to read.
If you're only offering files to a group of users who you can give passwords to, you could even use SCP. (Secure copy...uses sshd on the server side)
It all depends on the application. I only use SCP to move files around if I have the choice, just because I like better security if I can have it.
But if you want to offer files to the public, I'd recommend offering both FTP and HTTP so people can use the most convenient.
I would consider an HTTP/1.1 server a better choice than an FTP server. HTTP works with firewalls better. It can support "resumable" downloads, by requesting byte ranges, although I'm not sure if there's a client that can do this for file transfers. I don't understand how it would be slower than FTP -- it's just a byte stream.
Of course, I can't "mget" files from a webserver using my "http" command-line client, but I can give multiple URLs to wget to accomplish the same thing.
but use scp. SCP is part of the SSH suite.
as a matter of fact, I mention it in my beginners walkthrough of ssh on my website. I'd suggest checking it out if you got the time.
and yes, it's probably got incorrect information, but it's sole purpose is to teach my future father-in-law how to use ssh.
Looking for Book Reviews? Check out Literary Escapism.
Who are your users? If you are supplying 1-6mb pdf's of recipes for housewifes you probably don't want to use ftp...it will just create confusion and complicate the user experience. If, on the other hand, you are sharing custom software then you can guess that your users will be more experienced with computers and comfortable using ftp.
Do you need a website upgrade?
where'd that "Software" topic icon come from?
Security-wise, HTTP is a big win over FTP if only because it makes your port-filtering easier - "allow to 80" is simpler and less likely to cause unintended holes than all the things you need to do to support FTP active and passive connections. Certain FTP server software has a reputation as having more security holes than IIS, but there are FTP servers out there that are as secure as Apache.
1-6 MB? Use HTTP. It's probably more convenient for everybody, for instance there's no login procedure/delay for the end user. With files no bigger than that, people won't have the need to resume interrupted downloads either (the biggest advantage of FTP is the ability to do that when downloading a CD image :).
There are two things to consider:
* protocol
* application
Neither ftp nor http are `full of holes'.
It very much depends which specific server you are using. Now, IIS is full of holes, and some ftp servers have been full of holes in the past, and still are... and there has been holes in apache as well.
As far as the functionality goes, ftp indeed provides some very useful functionality for file transfers: the ability to resume an interrupted transfer. Most servers and most clients support it nowadays and that is really VERY useful.
ftp is slightly more difficult to set up when firewalls are in effect, because of the two actual tcp channels used (data+control), which means the firewall needs to know about ftp (or it can get very unpleasant: active/passive protocols tricks will get you through one firewall, not through two of them).
Of course, if you are a complete moron, you will botch your server configuration and leave yourself wide open, or choose the wrong server, or forget to upgrade, and get security holes all over the place... doesn't matter which
protocol you use.
In reality, depending on what you want to serve, modern answers include sup, cvsup, rsync, and all kinds of other applications which leave ftp faaar behind (not even talking about http...). But then, they are less accessible and a little more 3l33t, which is maybe what you are looking for.
From my point of view (A network administrator), I provide both ftp and http servers for the same files (stick all downloads off /download or something, and set the ftp root to that). This has several benefets...
1) I've found HTTP transfers are a little faster than FTP transfers (just personally, and I can in no way prove it - it may be user error, or just the programs I'm using)
2) I've found that FTP clients are everwhere - Windows, Linux, BSD, everything I've ever installed has included a command line FTP client, but not a web browser unless I specifically remember to install one. Further more, most of the "live CDs/boot disks" that I use don't have a web browser, but do have FTP... Thus, if you're serving files that a person with out a web browser/server might need, I'd set up both...
3) FTP security is what you/your daemon makes of it. wu-ftpd has a long history of being rooted... ProFTPd dosn't. VSFtp doesn't. HTTP security is the same way... IIS has a long history of being rooted... Apache doesn't... *(Not to say that there haven't been occasional exploits for these platforms)
There is no clear "Use this" or "Use that" procedure here, it depends entirely on your situation, what you're serving, what your network setup is, etc...
-PhaseBurn Welcome to Linux country. On quiet nights, you can hear windows reboot.
Arguments 2 and b aren't arguments, they're polemicizing. Arguments 1 and a are both true (AFAIK). So, basically it comes down to a matter of taste. However, I'll point out that the speed of the protocol is unlikely to matter anywhere near as much as the speed of your server & pipe.
No. Delphi is gay.
I've set up a server where security is a main concern. As we wanted FTP, but didn't want our passwords sent in the clear, we have the following services:
1. SSH-FTP
2. SSL-FTP (which falls back to standard FTP if the FTP client doesn't support it).
There is also SSH available to those who want it.
No matter which they choose, our clients are always in their own individual chrooted environment. It is used for modifying their web sites (yes, we also have Apache 1.3x, but didn't bother with it for uploads).
It's GNU/Linux dammit!
Remember this is about your customer base not your ego or /.'s peer pressure. You want as much convenience for as many people as is possible with in your time/quality/cost constraints. So, don't be guided by your ego. Offer both if you can and if you cater to 1337 trolls then offer SFTP, FTP-SSL, gopher, and other not-popular protocols. Just make sure that the majority of your users can access the content and your ok either way.
its more common to be able to resume an aborted ftp session than http. but perhaps that depends on the fetching agent.
with wget, you can choose which protocol you want to fetch with. and wget can resume from where it left off when using ftp. I'm not sure it can do that with http (anyone know for sure?)
--
"It is now safe to switch off your computer."
FTP provides you with user authentication, and binary transfers (which should be faster as there is no encoding??) It can also be linked to via the web, so there's not too much hassle for the user...
On the other hand, if you don't need user authentication - and don't want to off load big file transfers from your web-server, you may as well just leave it as http.
I run into this dilemna constantly. I've found that HTTP is more reliable for downloads for intranet applications. When I build something in VB I try to use FTP, because I have better control of what's going on. For file uploads, of course, FTP is definitely the way to go. As for speed, I think FTP is technically faster, but HTTP throught the browser has the added bonus of showing you how much time is left on the download, making it SEEM to go faster. Basically, if the file is something that doesn't need to be secure, I'd use HTTP if speed isn't an issue (ie: intranet only or broadband connections mostly), otherwise, I'd go with FTP.
do your own homework, curry boy
If you're looking for security, look into sftp, part of the openssh package. It uses the same encryption as SSH, is secure, yadayada. The only drawback is that windoze users have to get the sftp client to connect to an sftp server. Our school is considering adding sftp to the student fileserver so that we can access files from home without risk of attack.
I'm the Devil the Windows users warned you about.
Anyone who says that HTTP is slower and less reliable than FTP probably hasn't done any benchmarking. Based on my experience, HTTP is definitely more reliable if only because it tends to go through firewalls easier then the two-connection FTP protocol.
Both FTP and HTTP stream data across a TCP socket -- I can't see that streaming it over port 20 versus 80 is going to make any difference.
FTP was designed to be able to do all these neat things back when the internet didn't have so many security issues. Most of these features are either not used or explicitly disabled these days... The fact that the FTP server uses a different port means that firewall have to understand and properly be configured for this. HTTP sends the data back in response to the initial connection, so it tends to be easier to get through firewalls.
If you're concerned about looking like a "wimp", then you should offer both and let people pick what they prefer. Or... Stop worrying about what people these people think and figure out what YOU think is best.
The people who would call you a "wimp" probably aren't worth worrying about.
Sean
My business is serving files to customers. I use FTP but have a few comments:
- Security is not an issue if you use the right ftp server. Check freshmeat to find the right one. Don't use WUFTPD or similar unless you have some reason to do so.
- I believe FTP is still the protocol of choice for reliably serving files.
- It's easy to implement authentication with ftp.
That said, I'm adding http support because a lot of customers can't access ftp from work because of firewalls and a lot of home users have firewalls installed (some don't even know it) and ftp causes support problems.
My recommendation is to support both protocols. I'm inclined to make ftp the default and use http as a user option because I "like to do things right" despite resulting customer problems. You might want to opt for http default.
Anoymous, download-only FTP is not too high of a risk. You do run into firewall problems - depending on your server firewall and your clients' firewall you may block both passive and active ftp connections. For download-only you can minimize exploits by keeping up to date on a reputable server (kernel.org uses proftpd which, although it's had some oopsies, is otherwise highly regarded).
.... if you want to you can offer it in addition.
Certainly HTTP is a lot easier for the majority of people to handle.
You might look at something like djb's publicfile which makes a given directory available over both http and ftp in a secure fashion.
FTP uploads? Avoid if at all possible. ANONYMOUS FTP downloads
$0.02.
Slashdot? Oh, I just read it for the articles.
How about implementing a webdav solution? You can get away from clear-text passwords, users can mount them like a drive on Mac, Windows and Linux (via DAVfs, http://freshmeat.net/projects/davfs/?topic_id=143% 2C90 )
Still have some of the unreliablity of HTTP transfers and slowness. But works a lot better through firewalls (and more securely since connection tracking works better with WebDAV).
I've found Passive Mode FTP to also be more unstable than standard ftp transfers.
Reasons: You are serving files, not receiving them, and, said files are only 1 - 6 megs. That's my vote.
ftp might be somewhat faster, but in an era of broadband, I dont' see this as a real problem. The biggest problem is that for active FTP there's a whole range of ports you have to leave open, posing a security risk. if you're behind a firewall it might be downright impossible to open that up.
with http you can get security if you need it (https/authentication) and deal with only a single port which is almost guaranteed to be open not only on your end but at the client's end as well.
Go to http://www.kazaalite.com if you haven't already. Download & install it. Then offer the files on Kazaa.
Which one to choose? Why, its trivial!
In the future, I would want to not be isolated from my friends in the Space Station.
It seems to me that the biggest security problem with FTP is that it transmits passwords in the clear. Anonymous FTP in a chroot jail should be reasonably secure. I'd be interested to hear contrary opinions.
I don't know half of you half as well as I should like, and I like less than half of you half as well as you deserve. BB
Try not. Do or do not, there is no try.
-- Dr. Spock, stardate 2822-3.
Because of the lower latency. All the logging in/handshaking stuff is not needed. You just tell the server "gimme file" and you get it. No "hello, I am like anonymous and my pwd is foo@foo.foo".. ;-)
You mention FTP is full of security holes. This is wrong, just use a recent ftp server and you won't have any security issues. The same applies with HTTP, lots of bugs were discovered in HTTP servers as well, worms even made good use of them ;-) So ftp is not "less secure" than HTTP!
HTTP is nice and simple for the novice users to get stuff. And FTP is nice for other things.
Really, this question should be answered by you. You don't need our help, your the one who has to support it.
Being called a dork on Slashdot must be like being called the retard in special ed.
It's not really important enough to stress yourself over - do both and move on. You don't tell us much about the _type_ of files you will be serving up, though, which is about the only thing I think that might make much of a difference. If, for example, it's a bunch of Windows files that the majority of your user base will probably be accessing via a browser then the ftp access might never/rarely be used. If the files are more likely to be accessed via *nix users it's more likely that the ftp access would get a bigger workout (I have lots of systems where I've never left the cmd line and appreciate a simple ftp to dl some needed files). These are broad generalizations, of course, but points to what I think the more compelling reason for one vs. the other (if there can be only one! (TM)) - what would your users prefer?
"The bigger the lie, the more they believe." - Det. Bunk
If I were ...
1.) concerned with security and everyone getting files has an account with me, sftp only.
2.) had a bunch of computer illiterates who only know how to surf the web, http only.
3.) allowing a bunch of people who don't necessarily have an account with me, ftp and http.
That said, a combination of all three should be sufficient for just about anybody who wants/needs the files. I use ftp (sftp when it's available) to transfer information between my office and home computers. Of course, if it's a big project with several people working on it, why not use a CVS repository?
As protocols goes, FTP is the king for "worst protocol of ... mankind". Really. Anyone who has ever had to (even tried to) implement an FTP client or, even worse, an FTP server knows what I'm talking about. The protocol is "specified" as if to help create security and connection problems.
HTTP might not be the best of protocols either, but at least it's an easy, well understood, and (to me most importantly) single-connection protocol. If you got a connection to a server, you do have a connection to a server. Not so with FTP where there are more than one channel of connection to attend to (and that's not counting the differences between e.g. PASV and PORT mode!).
As for efficiency, HTTP beats FTP any time. Hands down.
If I were you, I'd probably set it up using HTTP.
One nice feature in many ftp clients that doesn't exist for your web browser is to download all files which match a certain mask, or all files in a directory.
prompt
mget *.tgz
or users might have nice ftp clients that have transfer queues so they can Q up several directories, hit go and come back later.
If you expect people to transfer more than about 7 files at once I would definitely setup that FTP, which can be run securely enough with chroot.
Note that HTTP 1.1 can resume. As long as the user is downloading w/ a download manager that has a clue, that's not a problem.
The efficiency differences will be debated forever... the common wisdom is that FTP is more efficient, but there is also evidence to the contrary. That isn't the point.
To me, this is a problem of authentication. If you want EVERYONE to have these files, why not just use the HTTP server? If you're targeting a select few people, then why not use the built-in authentication mechanisms of FTP?
Yes I know there are authentication mechanisms for HTTP, but they're arguably harder to implement than setting up an FTP server.
Are your clients only using web browsers to retrieve these files? I'll get flamed for this, but web browsers were not designed for FTP, and thus are klunky at it. HTTP wins there again.
Don't worry about it. Just use HTTP and let the FTP bigots flame away.
My experience with FTP/HTTP is due to FTP's authentication it responds slower and is harder to configure, whereas HTTP has near instant authentication & is more easily multithreaded on downloads.
Of course FTP is the better protocole, but now people use HTTP for everything so you might as well go with the trend.
Je t'aime Stéphanie
Isn't the whole point of your server to share files? So what's wrong with looking like a wimp or a dinosaur? Just use the protocol that works!
Use HTTP. No, use FTP. Well, wait a minute, HTTP is better. On second thought, though, FTP is cooler. No, I like HTTP. Nah, just use FTP.......
Lemme guess... MP3s?
"Consider the lillies of the goddamn field."
I've found some firewalls/gateways (esp. doing NAT) still have issues doing outbound FTP via NAT properly, even with the client in passive mode.
However, FTP can be (fairly) easily resumed if the connection is lost, which can be important if a larger file is being downloaded.
In general, I've found HTTP file transfer to be both reliable and problem-free firewall-wise, but YMMV.
FTP has generally better support for transferring plain-text. By downloading a document using the ASCII protocol, FTP automatically fixes up newline conventions to match your native OS. HTTP doesn't do that and requires user agents to respect most every ASCII-based newline convention. Unless your user agent is really intelligent in that regard and fixes up newlines automagically, you'll probably end up with text using newlines incompatible with your environment.
This is one area that I feel HTTP was deliberately weak on. I can't see any reason why HTTP couldn't require any text/* media type to be sent in a canonical format, with "network" newlines. If every Internet transport protocol did this properly, there would never be a situation where you have a file on your computer using the wrong type of newline.
FTP supports file listing. Many HTTP servers support auto-indexing, but it really isn't the same.
It's quite a bit easier to get a bunch of files from an FTP server in many cases. "wget ftp://some.server.tld/some/directory/*.ext" can't be replicated over HTTP without some scripting against the index.
Now, YOU have to decide if this is a good thing or a bad thing for your application.
-Peter
Come on people, use the Z-Modem protocol. It can resume transmission on a file transfer where HTTP or FTP can not. The only way a FTP or HTTP can resume transmission is with the GetRight tool.
:) HA HA!
I remember in my days of BBSes with X and Y Modem, and then when Z-Modem showed up we all couldn't be happier. When some idiot in the house picked up the phone and disconnected you from hours and hours of downloading the latest Liesure Suite Larry, I just reconnected and started to resume my downloads (but only if I had enough credit, then I might have to upload some crap).
I'd suggest UUCP.
-Kris
If you plan on only sharing files that are listed on your website you may as well go with HTTP. The Reason Being that you have less to take care of, managing and FTP Server Alongside of a HTTP Server is a Pain in the Ass:
*Two separate sets of permissions that need to be in sync
*Two Protocols that could possibly be blocked by your ISP (They don't like FTP much)
*Browser Integrated FTP clients suck and it's a pain in the ass to have to copy and paste addresses into another client for the user
*Usually Extra Licensing Costs if your using a M$ Platform
*Most net users have download managers that are based on HTTP and that can resume (less bandwidth for you).
*FTP clients are mysterious "hacker" type software to average Joe.
Go with HTTP, it makes life easier and shows that your and your clients/users time is worth more to you than a little extra overhead.
However, real men write a protocol that works over DNS TXT records using the CHAOS protocol. I'm actually working on this!!!
As in Take A Damn Shower. And go check out some night life. Jesus, you're using perceived social status as a criteria for choosing a fucking protocol. Who gives a rollicking rat's ass what people are going to think about your choice in protocols? Go worry about whether you should tell her before or let her find out the hard way that you're almost done. I suddenly feel vastly less nerdy for not ever having worried about looking like a dinosaur or a wimp because of my friggin file server's protocol.
Oh, and real men use FTP.
Shawn
Because you gotta bitch
IE will resume a d/l too, providing the temp file is still in cache
Id love to see who these "experts" are. FTP FASTER than HTTP? Um .. Ok
The answer is quite obviously that HTTP is the better choice for you.
FTP is known for security holes. FTP is often misconfigured. FTP uses two ports and commonly has problems with firewalls, stateful or not.
HTTP you wont have to deal with most of the headaches.
Why do we have all these new ask slashdot question that sounds like a tech with a years experience is asking how to do his job?
I vote for a new section, "How do I do my job" with a dollar bill as the logo.
Neck_of_the_Woods
#/usr/local/surf/glassy/overhead
vi is better than both
If I remember clearly, FTP is prefered over HTTP for downloading larger files for various reasons. First of all, HTTP is stateless, meaning it will accept your connection, pump out whatever data it returns and doesn't do much more really. FTP isn't stateless, meaning you connect to it, it responds, negotiates a transfer and actively tries to keep things going, resulting in (somewhat) slower initial connection because of more protocol overhead, but far more reliable down/uploads. Also, FTP security can be as good as you want it to be. Disallow anonymous access, set proper users accesses, chroot them into their home directory and use a decent, up-to-date server such as Proftpd. Also, you can't (easily) upload files with HTTP, though it is possible. However, FTP uploading is far more elegant and again, more reliable.
Hate me!
How does the connectionless nature of HTTP contribute to the performance of file transmission?
>>typically 1MB-6MB
since you are obviously running a MP3 server, I think you should set up a FTP server. The users who actually use and understand a FTP client will be the only ones who will upload to your FTP/MP3 server, which is why you are really doing this anyways. Sure, you want to be cool and share, but getting new MP3s without the work of searching is where it's at, from my experience.
I ran both from my computer, and setting up a FTP server like Serv-U is easy as can be, not that the HTTP version is putting a man on the moon either.
(Cleaning up the text a bit)
Well, 2 and 4 are nothing more than acephalic punditry, unworthy of our attention, which leaves 1 and 3.
The fact that HTTP doesn't use a binary connection to transfer binary files means that, yes, it is frequently slower than FTP. Especially since your listed file sizes imply that you're not offering text files for download.
While FTP doesn't have any security holes (Yay for false generalisations!), many of the readily available ftp daemons have had shaky track records in the security area.
I don't really have an answer for you, I just wanted to say acephalic. :-) Acephalic acephalic acephalic...
Teach your kids: "C++ made baby Jesus cry."
HTTP is amateur and will make you look a wimp.
FTP is a crumbling legacy protocol and will make you look a dinosaur.
These two factors should not even enter the equation. Whatever you decide to use, you should decide based on which one accomplish your goals for a given situation.
Or you can look at it this way: You will look dumb, if you decide on an inferior technical solution only to appease peoples opinions on what they might think of you, or how 'in' you are. Those people really dont matter anyway, they read Business 2.0 and Information Week and believe it is gospel. These same people believed in the 'new economy', and XML this and that, blah blah.
Do the right thing.
Please, FTP, just die.
It's two-TCP-connections approach is incredibly painful, and raises all sorts of ugly issues with firewalling, NAT etc.
If you're offering a link from a webpage, the majority of people will probably download from within a browser. Even if they don't, pretty much every OS ships with a command line tool to grab a file over http - wget, BSD's (including MacOS) fetch, etc. Tool support isn't really an issue.
One caveat is that typically a website runs off something fully fledged like Apache, which is a bit heavy for serving up large files. You might want to look at something like boa or thttpd. (Personally, I've seen performance problems on a webserver clear up by moving all the static images off Apache, onto images.sitename.com running thttpd).
HTTP is waaaay faster than FTP. P2p also use it. Shareaza p2p can even be youre default download manager.
Dump FTP. Would you consider hosting a web site over FTP? No, so why files?
HTTP "upload" is trickier than FTP upload, if that's a factor.
Computers are full of security holes. Both webservers and FTP daemons have the risk (and history) of 'sploitable holes. As to configuration-related risk, there's more inherent in the default configuration of an FTP server, since its original concept of ops is browsing a physical directory tree, but most current FTP servers can be locked down as tight as any webserver.Just can't win, can you? Only posers worry that their choice of protocol might look bad. It's not fashion, folks.If you had to support only one, pick HTTP, since I bet you're already running a webserver. And if it's really about fashion, run an exotic protocol like CORBA or SCP or something.
Welcome to the Panopticon. Used to be a prison, now it's your home.
It is possible to get approximately 80% of the theoretical maximum throughput of your pipe using a single FTP connection, whereas HTTP can hope for around 60% max for a single connection. The only thing faster than an FTP-based protocol (tftp, pftp) is a raw socket, and they rarely get better then 90%. Most schemes like pftp (parallel ftp, see this paper) are implemented to get as close to theoretical maximum throughput by having multiple data connections transfer the file. Of course you'll see the difference in performance more for large file transfers. The previous comment about HTTP being OK for small files is right on the mark...you will hardly notice a 20% gain when the transfers are only taking a few seconds.
Forget ftp or http, WebDAV is the answer you need, modern, user friendly and convenient, your users will love it, not the fastest but it is right.
WebDav gives you the advantages of http (firewall transparent) and some more (SSL, events, log processing, XML extensions). The worse thing about WebDav is that is usually harder for the admin to get it running and that it's not as universal as ftp are.
Help, is there any way to script SFTP? I know with ftp you can use the .netrc file to hide the ftp login, etc.. but I haven't found any solution to automate a secure ftp process.
I make these: http://beatseqr.com
Here's how i might do it.
:)
1. Web content on whichever web server you already use.
2. Conent "en-masse" which you would normally want to deliver over FTP should be put on a failry secure web server, if you're worried about security. Tux would do the job as a very quick and secure webserver (an LVS cluster of like, 2-3 of these, maybe, i'm no LVS expert).
OR
Use a more secure ftp server (ProFTPd).
ZERO
I use two programs to retrieve files, wget and Mozilla. Both show the download rate whether I'm fetching from HTTP or FTP.
What bugs me is when servers won't tell me the final downloaded file size -- no ETA available. I've seen both FTP and HTTP servers do it. The same goes for servers that don't support resuming or last-modified dates. They suck.
Does my bum look big in this?
Why not use use DIP or DAP instead? They are dinosaur protocols I am well aware: Dinosaur Internet Protocol and Dinosaur Access Protocol, but they have been around for millions of years. Compression is excellent and I hear they have a very fluid viscosity to them that make them permeable, yet full of energy potential. Unfortunately DIP and/or DAP packets do tend to explode when exposed to high temperature electric wiring, and they put off a foul smelling odor, but you are sure to be the L33t3zt kiddie on your block cuz U n0 the DIP and DAP codez!
All Ad hominem replies happily ignored as the sender shall be deemed to lack the faculties to comprehend the equation.
P2P?
I've written a tutorial on how you can use P2P on your website to save bandwidth, space etc. An obvious way to do this would be to run a P2P client and share the file on a simple PC & Cable Modem. This works, but it is a bit generic and un-professional. A better way to do this may be to run a P2P client such as Shareaza on a webserver. You could then control the client using some type of remote service (Terminal Services, for example).
P2P has it's advantages. Such as:
- Users who download the file also share it. This is especially useful if the client/network supports Partial File Sharing.
- When you release the file using the P2P client, you only need to upload to only a few users. Those users can then share the file using Partial File Sharing etc.
- Unlike FTP and HTTP, they aren't connecting to your webserver. Thus, it saves bandwidth for you and allows people to browse your website for actual content, not media. (Though, media is content). In addition, there is ussually "Max # of Connections" allowed to a server or FTP. Not so on P2P.
- P2P Clients have good queuing tools. At least, Shareaza does. It has a "Small Queue" and a "Large Queue". This basically allows you to have, say, 4 Upload slots for Large Files (Files that are above 10MB, for example) and one for Small Files (Under 10MB). Users who are waiting to download from you can wait in "Queue", instead of "Max users connected" on FTP.
Though, at it's core, all of the P2P I know of uses HTTP to send files etc. But the network layer helps file distribution tremendously.
For a good overview of FTP see: http://www.protocols.com/pbook/tcpip5.htm#FTP
Search first, ask questions later.
Why the heck is FTP so horrible in windows IE? Any clues? It takes forever.
-Scott
If security is an issue, FTP with TCP wrappers is the way to go. Make sure the server's OS is patched, set all the security configurations for the FTP and the wrappers, disallow anonymous login, and encrypt the files with PGP. You could also create logins for downloading, and different logins for uploading. Do copious syslogging so you can see which login ID is doing (or trying to do) what.
>> "HTTP is restricted by browsers, many of which will not support files larger than a certain size."
What browser would that be? Certainly not IE, NS/Moz or Opera.
FTP is a quirky, extensible protocol, great for uploading, downloading and sharing files, and you can do wonderful things with FTP and databases which web servers only dream about.
Rich.
libguestfs - tools for accessing and modifying virtual machine disk images
you can't shift-click files in your web browser.
If your users are sophisticated enough to run an actual FTP client, they might enjoy that feature.
Otherwise, HTTP gets through firewalls and proxies.
You're wrong, but you brought up a good point.
HTTP download managers can do a great job of recovering from timeouts and disconnects. One that I use on windows can even make six simultaneous connections to the same server and get six parts of the same file then reassemble it. It can even do that trick with sis separate mirrors of a file on six different servers. It's called lightning download. http://www.lightningdownload.com/ I use it mainly to get ISOs. I would always get 500 meg of a file then fail before I started using it
One thing I like about FTP is that most clients allow you to select multiple files and grab them all in one shot. I think that you can do this with wget over HTTP, but I don't know of any web browsers that can.
mentions of uucp, kermit, rcp/scp, but nobody has mentioned fsp. maybe i'm just being too obscure.
On-demand UUEncoded email transfers! Yeah baby!
HTTP is stateless and FTP is stateful, yes. However, the "overhead" required for a stateful connection is not going to cause a slowdown in transfer except possibly an the onset for the authentication. Stateles IP protocols don't have persistent connections, they shoot packets out and don't care about the receiving end because they assume that IP devices in the middle or the end will ensure delivery and proper resequencing. Stateful protocols keep an active connection and this is supposed to be faster. After all, ATM's big hold over, say DS3 and T1 lines, is that they open a pipeline and shoot data downstream. It is "connection oriented," like a phone call.
My suggestion would be to go with FTP if the size is anything above a few MB for administrative reasons. HTTP doesn't allow you to limit connections and bandwidth could be incidentally mashed into virtually nothing per user. You can limit not only the number of active downloads, but even who can download if you wanted.
Think of almost every Linux distribution - they use FTP. Why? Ever tried downloading something, connected at a smooth 100k/s or something and found, a few minutes later, you're pulling 2k/s? That USUALLY doesn't happen with FTP. Once you're in, you're in.
If you have tiny files, of course, logic dictates you go with HTTP. But for anything of size, do *yourself* a favor and go with FTP.
Adam
I know you are..
I don't know why, but no one has bothered to suggest something more cutting edge, like Bit Torrent.
If you provide files to normal people, use HTTP. If you provide files only dumb linux geeks would download, provide all protocols you can implement. Just like everyone of them masturbates to different kinds of pr0n, each of them has his favourite protocol for transferring files.
BitTorrent is the coolest file transfer idea around. Basically you have a central "server" that has the whole file and all the people trying to get the file download a little chunk. Then they all share the little chunk they have with every other person trying to download it. This way you have distributed bandwidth among all of your clients for uploading, and the more people getting the faster it goes.
http://bitconjurer.org/BitTorrent/
What I don't care for with FTP is the continuous setup/teardown of data connections. What is even worse with active FTP is that the client side of the data connection establishes server ports, and the server becomes the client (I'd like to be able to use plug-gw from the TIS FWTK for FTP, but this is not possible for the data connections). However, even when enabling passive FTP, the data connections are too prone to lockup. The difficulty of implementing all of this in C probably contributes to the FTP server vulnerabilities.
Still, if you want both (optionally anonymous) upload ability and access from a web browser, FTP is the only game in town.
From the network perspective, the rsh/rcp mechanism is cleaner (in that there is only one connection), but it still has the problem of either passing cleartext authentication or establishing unreasonable levels of trust with trivial authentication. In addition, with rcp syntax you must know much more about the path to a file, and there is no real "browsing."
Many say that SSH is the panacea for these problems, but sometimes I am not concerned about encryption and I just want to quickly transfer a file. The SSH man pages indicate that encryption can be disabled, but I have never been able to make this work. SCP also has never been implemented in a browser for file transfers. I should also say that I've never used sftp, because it has so little support.
Someday, we will have a good, encrypted file transfer protocol (and reliable implementations of that protocol). Sorry to say, but ftp, rcp, and scp are not it. What will this new protocol support?
Boy, I never thought that I could rant about file transfer software for so long!
As far as I can tell, only the amatur wimps using http think that ftp is a crumbling legacy protocol full of security holes.
I think FTP got a bad name security wise because of wu-ftpd, which is known for its long and glorious tradition of remote root exploits.
I have to use FTP every single day, but mostly for big data transfers. Proftpd has served me well.
"You never know when some crazed rodent with cold feet might be running loose in your pants."
-Calvin
That's pretty much the only reason I'd use FTP over HTTP. When you're doing bulk transfers of large files over a potentially lossy connection, the ability to restart is really nice. By restart, I mean finish the rest of an FTP transfer that failed in the middle.
Of course, you have to have an FTP client that supports this.
Both compliment a nice bottle of Chablis.
Well, if you really want to be on the cutting edge why don't you use P2P? That way you can even spread the load as well and not use as much bandwidth. You don't have to use gnutella or kaaza there are other systems out there that you can use. Gnutella would probably be the easiest but it's a nice way to be able to serve files.
Now if you want to serve private files I'm sure there are probably ways to do this as well. I'm not sure. But if you're doing anonymous you might as well use P2P.
Here's a start:
http://www.badblue.com/helpgnu.htm
It's a windows product but it's a start.
sri
The question does not contain enough information to form a proper answer.
Don't start with finding the solution, figure out what it is you want, what you want it to do and then find the right tool. We can not tell you which is right with almost no information about the use of it, for what and what is the average user profile etc.
HTTP and FTP can be equally insecure, but it shouldn't be much of a job to properly secure a ftp.
my sig
Use HTTP and don't make your server names unique (use round-robin DNS instead). That way broadband ISPs that use caching will only need to download your file once in a while. This is good because (a) you save money on transfer charges, and there is more bandwidth available for you, and (b) the broadband provider saves money on bandwidth AND delivers the file faster to its customers. It's win/win. There are algorythms to make ensure 'freshness' (or use a unique filename for each version if you have frequent changes.) Seriously... go HTTP only.
Gimme a break. HTTP is NO slower than using a raw socket blast of the file for large files -- unless you use transfer encoding, which you probably won't for large file transers, the only overhead beyond the raw socket method is the tiny HTTP header.
From the end-user's perspective, downloading via http is more convenient, especially if you're on a corporate network and have to go thru Squid proxy since ftp-by-browser-thru-squid just sucks.
How come you don't know this yet have a 4-digit user id on slashdot? Under what rock have you been hiding the last years?
As far as I know Google does not spider ftp archives. Therefore if you want people to be able to search for all the files offered (including inside word processing documents and PDFs) then you are better going with http.
:-)
I think that the ability for people to find the files using a search engine outweigh the other considerations if you are seeking to make the files public.
Slashdot Beta should die a painful death.
rsync is out of the question; it simply doesn't scale for more than a few clients (rsync of the portage tree for the gentoo mirror here on ftp.oregonstate.edu abuses that machine).
ftp is nice because we can limit the number of connections but that isn't fine-grained enough; if i have 150 connections available, I might have 150 dial-up users or 150 broadband users.
http will let me rate limit but it doesn't have the limited connection handling that ftp does.
I would say provide ftp & http and get your stuff hosted on a few well-connected mirrors if its fairly busy.
All PASV mode does is allow the client to initiate the data connection, rather than the server. There is still a control connection and a separate data connection.
This can be helpful when the client is NAT'ed, otherwise the client will send a PORT command with an unroutable address. Of course, if the server is NAT'ed, the reverse will happen. There are stateful NAT devices that will actually exmaine FTP control connections and rewrite PORT commands, but NAT and FTP are basically a pain in the arse to deal with. Throw some encryption in the mix (sftp) with NAT, and you'll understand why FTP is not long for this world.
Only on Slashdot could you learn that FTP uses UDP for the data connection or that PASV mode only uses a single socket!
I'm pretty sure the only difference between HTTP and FTP is the connection and handling of transfers. TCP ends up doing the actual file transfer. Write an FTP and HTTP server some time.
:)
I'm not sure if this has already been said, but I hadn't seen it yet.
:wq
It seems like only yesterday...
Sigs are bad for your health.
Each has their place.
FTP was designed for interfacing with the filesystem of a remote Unix system, with the filesystem permissions that are granted to the user you log in as. FTP lets you browse the hierarchy, including examining ownership, permission, and symlink targets; pretty much the same as what you get with 'ls -l'. Apache does file listings, but only shows file names, last modification dates, and size. This makes FTP more suitable than HTTP for remote mirroring of directory trees. This also makes it easier to "browse" what an FTP server has to offer, on a directory-by-directory basis.
With FTP, the server prints a response when a client connects. Usually, the client sends a user name, password, the 'SYST' command, and asks for the current working directory, tells the server what mode (ASCII or binary) it wants, changes to the directory with the file it needs, sends a PORT command, then finally requests the file. With HTTP, the client connects, sends a request, and the server responds. That's 8 client commands and 9 server responses with FTP, as opposed to 1/1 with HTTP. Each time a command is sent, the client has to wait for the server to respond. The latency adds up, and that means, especially on high-latency connections, FTP is slower to initiate and begin downloading than HTTP. Who said HTTP is slower?
Regarding reliability, both protocols and modern implementations of their clients and servers have features to resume a broken download from where it left off. Who says one is more reliable than the other?
HTTP is more simple than FTP. As far as I can tell, in FTP "active mode", the client sends a PORT command with an IP address and port number that is listening. In "passive mode", the server sends an IP address and port number that is listening, after the PASV command. These address/port combinations are used for the actual file transfer.
Active mode doesn't work if there is NAT between the client and the server, unless the NAT system rewrites the packets so that the IP address the server sees in the PORT command is the outside, external address of the NAT system. When an FTP server is behind NAT, passive mode cannot be used without a similar kluge; it must get an outside-world IP address to connect to from the client, which active-mode PORT does. If both client and server are behind NAT, then one of these NAT kluges must be in effect for file transfer to be possible.
This address/port nonsense could be part of the security concern with FTP. I also believe older FTP implementations allowed the client (in active mode) or server (in passive mode) to specify arbitrary address/port combinations, so that the FTP server or client could be used as a proxy in an attack. Is this still the case?
With HTTP, transfers are conducted on the same TCP connection as control is on, and therefore doesn't need to concern itself with IP addresses and ports, and the people using it have fewer NAT headaches.
Let me get this straight. You went to search the web and got conflicting, likely ill-informed, and inconclusive reports. So you went to Slashdot?
Put it on Kazaa as "Britney23489023.mpg" and just publish the filename. :)
Just use both.
# Erik
As many people have said, it depends.
...)
FTP has a great advantage in that you can request multiple files at the same time: mget instead of get. Additionally, you can use wildcards in the names, so you can select categories / directories of files with very short commands. (mget *.mp3 *.m3u
Modern browsers allow you to transfer multiple files simultaneously, but they don't queue files for you - FTP will. This may be important if connections might get dropped - the FTP transfer will complete the first file, then move on to the next. In the event of an interruption, you will have some complete files, and one partial (which you can likely resume). For multiple simultaneous transfers - from an http browser - you may have some smaller files finished, but it's likely that all larger files will be partials, and will need to be retransmitted in their entirety, since http doesn't quite support resuming a previous download.
So, if you're going to have a web page with many individual links, and you think that most people will download one or two files, http will probably suffice. If you expect people to want multiple files, or that they will want to be able to select groups of files with wildcards (tortuous with pointy-clicky things), then you should have FTP.
It's not that hard to set up both, and that's probably the best solution.
- The Sigless Wonder
You may save yourself a lot of bandwidth by using HTTP, especially if you set it up as a preferred method for the clients, since those downloads may get cached at various places.
Be sure to implement proper expires for your stuff, though, so people don't get frustrated with cached content.
... be aware the IE limits max connections to a single server to 2. So if you open up two big concurrent downloads, IE won't let you access anything else on the site until one of the downloads completes. It doesn't give a warning or anything, it just hangs there waiting for the download to terminate.... A solution is to use a different domain name for your large files to prevent the browser from failing to load your "regular" pages. Another issue with IE is that it seems to ignore the content-length header. Thus if the connection closes early for whatever reason, IE will treat the download as "complete" even if the length doesn't match content-length. This has caused me more than one corrupt download in the past.
HTTP is as reliable and fast as FTP, but shitty browsers (IE) can still cause problems. This isn't a fundamental limitation of HTTP though.
This give you the mountability of directories that ftp provides.
So we are left to be smart-assed and provide shots in the dark as to what the "best" solution is to this.
Of course, since these days most networks use switches instead of hubs, one has to wonder how much of a security risk sniffers are anyway. I guess there's always the possibility of ARP poisoning or some rogue router between the source and destination, but honestly, how likely is that? I guess it just depends on how secure you need your data to be.
You don't need to use something with encryption like HTTPS or SFTP. These are publicly available files, right? I do recommend you post md5sums for each of these files both in the download directory and on the webpage that links to each of these files.
HTTP is most likely to get cached by caching proxies. So if your target audience is likely to be behind a caching proxy (Unv campus?) and you want to save yourself some bandwidth, use HTTP on a dedicated port or box so you can control concurrent downloads. HTTP might also get you through more downloads. Then again if someone's firewall will only let them get to port 80, they're too stupid to be downloading files from me anyhow.
Personally I'd recommend FTP. It's easier to control the number of concurrent downloads, concurrent connections from a single IP, transfer speeds, and integrates well with your authenticated uploads by yourself or whatever other person will be uploading files for download. Saying that FTP is less secure than HTTP is complete BS. Both have the same level of insecurity. The most common FTP daemon (WU-FTPD) has had horiffic security holes in the past which is why no one in their right mind should use it. ProFTPd on the other hand has had very few security problems (usually long after the affected release has been surpassed by a couple new releases at least). I highly recommend you look into ProFTPd for and FTP solution. You can always offer and HTTP solution but I highly recommend you limit the concurrent number of transfer if you use HTTP.
There is one thing you cannot use http for: Specifying the filename on the client computer. The URL does not specify the file extension (a php file can output a png file, for instance), only the mime type can give a hint.
:-)
But if you want to serve a firmware file, for instance, you might use an uncommon file extension like ".fir". This filename extension cannot be served correctly via http, only via ftp. Check out how it works on a Macintosh if you don't believe it
Dybdahl.
I wrote one of the first download managers years ago that resumes interupted downloads of FTP and HTTP. From my point of view FTP is a bitch. FTP is a lot slower to actually start the download. You have to use 2 connections. One to send the commands and one to recieve the data. You do not just request the file in one command like HTTP. You must connect, set the current directory, tell the FTP server what mode to use etc., then start receiving the file on the other connection. FTP is a tool for creating new folders, deleting, uploading etc. HTTP is the way to go for downloads.
cause everyone is guaranteed to have working http acces! while ftp might not work under some circumstances!
Thank you Mr. AC and well said. Someone please mod this to something that doesn't suck to help eliminate the blind confusion that is Slashdot trying to figure out what FTP is.
Saying Java is nice because it works on all OS's is like saying that anal sex is nice because it works on all genders.
I serve 1-6MB files to two clients all month, averaging around 100GB per month. Actually, they're mp3s and the clients are just xmms boxes around the house. Ocassionally they'll be playing a song and half way through just skip to the next song, but it's usually days between occurances (every couple of gigabytes, or every couple of hundred files).
I think it's due to the streaming nature of the transfer, and I doubt it'd occur with proper transfers.
On the other hand, FTP will max out my 100MBit ethernet, while HTTP will get maybe a few megs per second. FTP is probably better on system resources, and if you have a phat pipe, probably maxes it out easier.
ZModem-1k can kick both FTP and HTTP's asses. It's fast and can reconnect. It can even send filenames! Quite frankly, I don't see why you are still using 20+ year old protocols. ZModem came to us in the 80s, way more recent than the crumbling Internet you are referring to.
Well, I have to go now. Now you can go back putting that chrome exhaust pipe on your Civic.
Better yet, if you have the budget, use iPods. Your users will thank you.
sulli
RTFJ.
Most web browsers do not support resumes for files. You need 3rd party add-ons. Resume is just a standard command in FTP.
is ncftp. It's got filename completion and all the file shell commands. I resort to it when mozilla chokes (which is most of the time) -- especially when getting large files.
To-do List: Receive telemarketing call during a tornado warning. Check.
You arses. You missed it all.
The important point of this is that large-scale networks want to contol traffic. This is what QoS is about. This is a valid goal, and easily (AND CHEAPLY) implemented when traffic regulation is handled on a per-packet basis.
Y'all can stuff gophers, ferengi and other badgers down the port 80 pipe - with the end result being that we can no longer maintain any QoS.
To make this practical for you keyboard-drooling SUPER_@)(*$WISKED high speed home xDSL/CABLE clan; imagine your XP firewall sending your credit card info out via port 80 'cause yer dumb arse let it do so, with mucho no thinkola about WHY it let that content out.
FUCKIT. let's just let ALL apps fire and request data via port 80.. firewalls and traffic control be damned. Push the problem further up the stack. Why NOT? It's *EASY* to code, right?!
Use Bit Torrent over a http server, and you get scalability, download resuming and fault tolerance almost for free. Plus, it's definitely fashionable.
All generalizations are false, including this one...
if you're running a huge file archive, do FTP. If this same archive has screen shots and descriptions and stuff, use HTTP (both). If you're lazy and only want to use one of the 2, stick to HTTP, there's more for it in the future.
File Transfer Protocol enuf said? :)
i'm not about to install a p2p programm just to download some file. and if you want me to use my bandwith to upload your file to other people, sorry, forget about it. and p2p downloads are usually much less reliable then http downloads. also, those clients are a security hazard. plus the intregrity of the downloaded files is not guaranteed.
Many ISPs use transparent HTTP proxies these days. If your files are cached by an ISPs proxy it will reduce your own bandwidth use, which is always a good thing.
As the old Perl adage states, "There's more than one way to do it"... If you consider the various options as being different levels of complexity and control, it really does depend on the likely technology that your target audience has, and their technical ability (all of which has already been mentioned).
I'd say that by far the simplest option is HTTP - I'm assuming that your server is Unix-like, so we're talking Apache, which most distros have as default (or easily installed). FTP is also easily available, but introduces security and authentication issues.
If you have control of your clients, then SFTP is a good, secure solution, but takes more setting up and introduces some bandwith issues.
You have to decide what level of service is most appropriate, and how much effort and time you have spare to put it in place.
S.
Just burn the files on CDs and distribute them to the people who want them by US mail. Sure, it's slower and maybe not secure, but you won't be flamed by the FTPers and HTTPers. (Watch out for FedExers though...)
-- ignoring AC's since... well, always --
The answer is simple ... publicfile by djb.
http://cr.yp.to/publicfile.html
You can share files via FTP or HTTP without running a FTP server or a HTTP server.
No security holes to worry about either.
Frankly, who gives a damn what the "experts" say. If someone refuses to download your file served ala HTTP because its "n0t KewL" then the problem is on the wireless end of their keyboard. This "elite" attitude is so old, so outdated, and frankly, no longer elite.
Reliablility isn't so much an issue for a file that is only 1 to 6mb. If you already have a HTTP server, and don't have an anonymous FTP server, that SHOULD figure in your decision.
You should make the file available to download in the format that serves you best and offers the easiest way for people to get it. Don't get caught in the trap of following what select 'experts' say you should do.
Tequila: It's not just for breakfast anymore!
If you think I am going to fall for that old troll then you have another thing coming! You probably think SCSI is better than IDE as well.
Correct me if I'm wrong, but FTP security holes don't matter if you're talking about non-sensitive data with non-sensitive user names and passwords. In that case, I'd go FTP, with a backup via HTTP for those who for one reason or another can't access ftp (if say only port 80 is open).
I also assume the following:
I would say that you should go with HTTP for sure. Of course, you can provide both, but there are some key reasons for using HTTP.
Easier Configuration Perhaps I'm just not that swift, but I've found that web servers (including Apache) are easier to configure. This is especially true if you have any previous web server experience. Of course, the FTP server is more complex due to its additional features that HTTP doesn't have, but assuming that (c) is true, then you won't need to mess with group access control rights and file uploads.
Speed This whole "FTP is faster" stuff is not true. HTTP does not have a lot more overhead than FTP; it may even have less overhead than FTP in certain cases. Even when it does have more overhead, it is in the order of 100-200 bytes, which is too small to care about. HTTP always uses binary transfers and just spits out the whole file on the same connection as the request. FTP needs to build a data connection for every single data transfer, which can slow things down and even occasionally introduce problems.
Easier for Users Given assumption (d), your users will be much more familiar with HTTP URLs than FTP addresses. You could just use FTP URLs and let their web browsers download the files, but then you lose the benefit of resuming partial downloads.
Simple Access Controls Though some people need to have complex user access rules, you may very well just need simple access controls. HTTP provides this (look at Apache's .htaccess file), and you can even integrate Apache's authentication routines into PAM, if you are really hard core.
There are a few main areas where FTP currently holds sway:
Partial Downloads Web browsers typically don't support partial downloads, but the fact of the matter is that the HTTP protocol does support it (see the Range header.) The next generation of web browsers may very well include this feature.
User Controls Addressed above.
File Uploads Again, HTTP does support this feature but most browsers don't support it well. Look to WebDAV in the future to provide better support.
In summary, just use HTTP unless you need complex access rules, resumption of partial download, or file uploading. It will be easier both on you and your users.
We had a client a few years ago (about 1998), that we needed to set up data transfer for.
IBMGSA were running their IT systems, and they decided that FTAM (over OSI) was the way to go, citing FTP's supposed lack of reliability, and lack of features such as resume...
They were absolutely serious when they said that too.....
We eventually managed to get the whole setup working (FTAM, OSI, X.25 over an ISDN link)...but it was the most unreliable, cryptic, complicated system I've ever had to deal with...
A couple of years later, the client ditched IBMGSA, and the company that took over their systems took one look at the FTAM / OSI mess, and threw it out and replaced it with FTP over TCP/IP, and we've never had another problem.
Advanced users are users too!
Well, I'm just a user/programmer/hobbyist and not a network expert, but I have to say I find FTP a more elegant choice. It is a little faster, it is quite reliable, it is the tool for the job (that's why they call it File Transmission Protocol). It's also supported by EVERYTHING, including all major browsers, classical ftp client, "fancy" ftp clients, wget etc. And, last but not least, you can use wildcards over FTP because it is supposed to "mimic" a real filesystem.
I usually prefer (as a user, always) to do my transfers via FTP. Hell, I'd use ZModem if it was available! (quite faster, and extra speed is always nice if you have a 33.6K modem, anyway it is not relevant but it made me remember the good old days when men where real men and modems where painfully slow).
P.
download-only policies (ie. just like HTTP)
That was also what I thought until I read section 9.6 in the HTTP/1.0 specification about the PUT method. If that method is not intended for uploading files, what is the purpose? Of course many HTTP servers does not allow you to use that method.
Do you care about the security of your wireless mouse?
False premise, false implication, false conclusion.
just do what I do and mail people newly-obseleted floppy disks
Fully licensed blockchain psychiatrist
But http is not that much more secure either. Yes apache is more secure then IIS. But it does have vulnerabilities and is listed by the fbi as one of the most dangerous apps that can be cracked as well as IIS.
I use to have two terrible modem based ISP's before I upgraded to a cable modem. I had alot of bad packets from noise on my line. The speed seems the same but I would have various crc errors sometimes when I download large files over http. I have never had the same problems with FTP. Now since I have a cable modem the problems have went away but it seems that ftp has better error checking then http but I am not an expert in the protocals. Just do your research before you pick a ftp or http daemon.
If you use a Linux/Unix box make sure you turn off inet. FreeBSD 5.0 sysinstall makes this very easy to do. Also you do not need to use the latest and greatest ftp daemon. The large ones are generally less secure. I heard that the Debian developers created a tiny but free and relativly secure ftp daemon which you may want to use. I do not remember the name and I have never used it so I can not comment but it has been mentioned on slashdot several times why debians ftp site is so secure.
http://saveie6.com/
Seems to me you should just use CVS. Isn't everything supposed to be open source these days anyways ;-)
I would say go with http.. my reason is simple.. my college blocks ftp! There are two sides of the story, first off, the complete blocking of ftp started a week ago when they switched firewalls.. they couldn't get the ftp to work, and they decided it wasn't a high priority so they didn't do anything to fix it... After yelling at the IT department for half the day I got the priority up. But besides that.. Even before, when they allowed ftp, it was only active ftp.. I was unable to go to a website, click on a file that is served from ftp:// and download it! I'd have to use flashfxp or another client to d/l files.. I was able to figure that out quickly, but what about the other 99% of the school that wants to download say shareware from download.com or a driver from somewhere.. they are denied! FTP is a superior protocol for file transfer in my opinion, but administrators don't seem to care about it as much as they do http.
Simple ftp does not implement encryption. Therefore, sensitive information (such as login info) is sent clear text across the internet for everyone and their cousin to see. ssh isn't guaranteed security either. However, ssh ftp is definately better than simple ftp.
If you're going to be using Anonymous ftp however, then the added security of ssh may not be required.
I hate to suggest it to the slashdot crowd, but it'd be much easier just to set up a NTFS/samba share. MSIE will allow users to browse it, it's faster than http, and I don't think it's actually very vulnerable to attacks. Of course Microsoft is evil, yadda yadda, but you'll get a lot of gee whiz points from MSIE users who like the fact that it's just like their computers. (and most people either use windows or know how to use samba)
'nuff said.
Why don't we post some good links to resources on setting up a secure FTP server. It not totally painless (I know the hardway) the info is out there but kind of obsure. I don't even remember what I DID anymore... I remember not knowing how to lock a user into there directory and there was some simple argument that did it, I looked all over before someone showed me, I don't remember that anymore either...
How about it? Any good tips, tricks, tutorials, on running secure ftp, and all the neat-o things to lock it down?
I'll let slashdot speak for itself:
One man tells a falsehood, a hundred repeat it as true.
Yeah. Thats correct. For a good plain text explanation look it up on the Netscape site. FTP is hell for firewalls (I wrote one). With active FTP you need active filtering on the client side, with passive ftp you need to open a whole range of high ports. If you have a cheap router (on the client side) your NAT connection limit may be reached too. As an addititional note: normally you can set which ports the passive ftp clients may connect to in the ftp server configuration. Then you only need to open these specific (>1024) ports. And to top it off: I would use HTTP. Only one connection needed, more (and probably easier to understand) configuration options, better for firewalls, resume also available (mozilla, getright, wget etc), binary stream so not too much overhead, authentication options, encryption....well that should be enough for now. Warper ps. pro ftp: filesystem: segmented downloading, searching archives, linking into filesystems, getting file information etc. etc. etc.
The reason to use sftp on publicly available files or hhtps is so that people <*cough*>Carnivore</*cough*> can't track what I'm doing online. Sure, they can tell what sites you are visiting, but they can't tell what content you're looking at, or which files you downloaded... Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads <*cough*>Verizon</*cough*> and the RIAA wants access to those records... If they don't know what you downloaded they can't (potentially unjustly) prosecute you...
But the question brings up another point: FTP is a rather cumbersome protocol for transferring files. Ok, so you have two separate connections for entering commands and for sending data. Why then can I not enter any commands while data is being transferred?? Some FTP clients simulate this capability by open multiple control connections, but it really should be built into the protocol, IMHO.
HTTP was designed for small text file transfers, and not really for bigger files (although it does usually handle them pretty well).
I propose an "FTP2" protocol. This protocol will have the following features over and above the original FTP protocol:
Control connection is fully functional even when data is being transferred
Multiple data connections are allowed for each control connection (configurable by server operator)
Ability to queue files for download
Separate server-wide limits on control and data connections (for example, so I can log in and look around even if all data connections are used up. If I try to download something, it is queued until a data connection frees up. This puts an end to hammering on anon-ftp servers.)
Modular authentication -- ability to support authentication methods other than clear-text passwords
Support for encrypted connections
No server connecting to the client to establish a data connection, eliminating most firewall and NAT problems
MIME-type support, rather than clunky BIN/ASCII modes.
Yeah. Thats correct. For a good plain text explanation look it up on the Netscape site. FTP is hell for firewalls (I wrote one). With active FTP you need active filtering on the client side, with passive ftp you need to open a whole range of high ports. If you have a cheap router (on the client side) your NAT connection limit may be reached too.
As an addititional note: normally you can set which ports the passive ftp clients may connect to in the ftp server configuration. Then you only need to open these specific (>1024) ports.
And to top it off: I would use HTTP. Only one connection needed, more (and probably easier to understand) configuration options, better for firewalls, resume also available (mozilla, getright, wget etc), binary stream so not too much overhead, authentication options, encryption....well that should be enough for now.
Warper
ps. pro ftp: filesystem: segmented downloading, searching archives, linking into filesystems, getting file information etc. etc. etc.
Topic says it all.
:)
wget -r -l 0 -H http://slashdot.org/
Go ahead. Download the entire web. FTP was never this easy.
Gee, I should really patent some of this stuff.
On a global scale, HTTP and FTP are significantly more efficient than P2P. In most cases, the little P2P pirate boxes create more noise pinging eachother than the size of the files they transfer. The problem with FTP and HTTP is, of course, that the publisher has to pay for the bandwidth. P2P allows publishers to externalize costs. In most cases the different P2P servers are stuck in places that simple burn the commons. They consume resources on the common grounds...for example the bandwidth on a local cable. The main thing P2P does is steal bandwidth from telephone companies and businesses, but that's okay. The cost of P2P is lower for publishers because, someone else is paying for it. The technology itself is less efficient than either HTTP or FTP.
I'm not suggesting this is the answer to your problem, I haven't seen any GUI-based fsp clients (or a unix cli one for ages for that matter).
But it did seem very useful for what it was designed to do, serve out large files to anonymous outsiders.
If you're sending PDF recipes to housewives, DO use FTP. Ever seen what Acrobat does to Apache?
Do daemons dream of electric sleep()?
Just raise the taxes on crack.
Use publicfile by Dan Bernstein. A fast and secure ftp/www server. Since you are only serving files this is the perfect solution.
Imagine a time when you want to download DeCSS for your linux boxen from a foreign server, but someone is logging your downloads Verizon and the RIAA wants access to those records
Why in Hell would the Recording Industry Association of America care who has software to break the DVD Content Scrambling System? They market music on CDs, not movies on DVDs.
Or did you mean to say MPAA?
Yeah, yeah, it's just a careless error, but things like this make us all sound like a bunch of idiots. If we can't even keep track of which evil corporate organization goes with which issue, then how is anyone supposed to take us seriously?
David Gould
main(i){putchar(340056100>>(i-1)*5&31|!!(i<6)<< 6)&&main(++i);}
Being in IT for a large Fortune 500 company that sells an operating system among other things (no, not Microsoft), I can share some of my expereinces with you. So take it for what it is worth.
Our FTP servers run both HTTP and FTP providing the same content in the same directory structure. There are five servers that transfer an average of 1-2 TB (terabyte) per month each, so they are fairly busy. On a busy month each server can go as high as 7 TB of data transferred. File sizes range from 1 KB to to whole CD-ROM and DVD-ROM images. I think the single largest file is 3 GB.
The logs show a trend of HTTP becoming more popular for the last several years and not stopping. It is currently at 70% of all downloads from the "FTP" servers via HTTP. While the remaining 30% is via FTP. Six years ago (I lost the logs from before this time, they are on a backup tape but I am way too lazy to get that data), it was completely reversed. 75% of downloads were via FTP and 25% were via HTTP. 90% of all transfers are done with a web browser as opposed to an FTP client or wget or something.
One thing we learned was that many system administrators will download via FTP from the command line directly from the FTP server, especially during a crisis they are trying to resolve. They do this from the system itself and not a workstation. The reasons for this are a bit of a mystery. Feedback has shown that we should never get rid of this or we might be assassinated by our customers. We thought about it once and put out feelers.
I would say if you don't need to deal with incoming files and you file size is not too large then stick with HTTP. Anything over about 10 MB should go to the FTP server. An FTP server can be more complicated. It seems like the vulnerabilities in FTP daemons has died down in the past year or so. Also, fronting an FTP server with a Layer 4 switch was a lot more tricky because of all the ports involved. If you want people to mirror you then go with FTP or rsync for private mirroring. In reading the feedback, most power users seem to prefer FTP, perhaps because that is what they are used to. Also, depending on the amount of traffic you might need to consider gigabit ethernet.
The core dumps being uploaded are getting to be huge. Some of those systems have a lot of memory!
It's not HTTP's fault that Microshit can't implement a browser that correctly interprets the "Content-Disposition" MIME header.
Do daemons dream of electric sleep()?
It'd be nice if Phoenix and Mozilla would acquire that ability. For some reason the developers' stated position is that it won't happen anytime soon, but one can always vote for the bug anyway.
FTP is much better.
Wow - that's quite a statement to make (for someone spreading quite so much Fear Uncertainty and Doubt)!
HTTP is restricted by browsers, many of which will not support files larger than a certain size
This is not going be be an issue for anyone using a modern web browser (such as Internet Explorer or Mozilla). To suggest this a a potential problem really is beyond misleading.
Sacrificing a bit of speed and capabilities such as resume might be made up for with ease of use..
HTTP is much faster you are sacrificing speed when using FTP (which should be immediately obvious). Using FTP transfer requires additional overhead, greater than that of HTTP, more traffic equals slower download. It also equals greater CPU overhead on your server, which is dramatically noticeable on servers taking many hits.
HTTP also offers support for resume - I'm astounded that so many posters do not know this. Many browsers, as well as CLI tools such as 'wget' and 'curl' both support HTTP resume (and of course Apache supports it too).
So, by your own admission, it's easier, and as a point of fact, it's also faster and supports resumable downloads.
Add to this that not running an FTP server unless you have do reduces your exposure to exploits and that maintaining an FTP server will require more administration then FTP becomes a very hard sell for anything by large downloads (beyond the size the poster has indicated they will be serving).
There was a time when network connection speeds were so slow that I'd only download files over FTP, as that was the only reliable way to download files, but that was over 6 years ago.
FTP is the [F]ile [T]ransfer [P]rotocol.
period.
...according to Andrew Tridgell, who says by that time that Samba will be long, long gone.
The problem with using HTTP for large file downloads is that, in most cases, it's cheaper ressource-wise to span multiple FTP simultaneous connections than HTTP connections. Of course, this only becomes a real problem if you have more than a few hundred virtual hosts on a single box. So save your httpd processes, and use FTP for large files.
I say go with HTTP. I've been serving large files (anywhere from 10 MB to half a gigabyte) from Apache with no problems. The only reason you would need FTP is for file uploads, and even that you can do using HTTP (with multipart POST requests or PUT requests, which, unfortunately, most browsers don't seem to support).
:-) It's usually easier to click a link to a file than log into an FTP server and navigate to the right directory, remember to turn binary mode on, and download the file.
Just about everyone with a omputer'll be able to use HTTP or HTTPS just fine.
As other have pointed out, HTTP does seem to be more web-browser friendly. When I have the option, I always go with the HTTP link. Half the time my browser doesn't seem to be able to access the FTP server, although I can access it using a regular FTP client.
The HTTP protool is a lot simpler (write a client in 10-20 lines of Perl simple), and less likely to be a problem for people behind firewalls.
And you can do '1337 PHP scripting on yer web server
Duct tape, XML, democracy: Not doing the job? Use more.
With the small size of your files, ease should be considered. Mainly for the person(s) who is going to maintain the files on the server. If they are not Guru's, select the method that allow the maintainer to keep the files available and up to date with minimal "hand" updating to worry about. Not being able to easily maintaining the site and keeping the files current is often the downfall of a server. User's will be willing to use whatever reasonable method specified if they know that they can get what they need/want. If ease of access by random users is required, remember most browsers will also support starting up an ftp client when specified by a link.
Starting multiple TCP connections for a single file download can be advantageous, because of congested network paths.
If there are 500 TCP downloads ocurring, each download will theoretically get 1/500th the bandwidth.
Therefore, by opening multiple TCP connections, you will increase the amount of bandwidth for your transfer, at a cost to everyone else using the connection. This is because you've effectively doubled the size of your receive window (one for each connection), causing the host you are downloading from to stuff that many more packets down the pipe.
The problem is, when everyone does it, it completely negates any advantage to using this method. It also leads to packet loss, since you have that many more TCP connections (each with its own receive window) fighting for pieces of the pie.
Whenever I see a list of FTP mirrors with one HTTP version, the HTTP version is faster and more reliable 9 times out of 10.
I suspect that's because 99% of people are downloading from one of the FTP servers.
I put to you that would be more logical to suspect it's because HTTP is faster than FTP as a transfer protocol. It generates less traffic (and uses less CPU overhead) which means downloads end quicker.
Additionally the CPU overhead generated by FTP connections also causes many sites to limit the number of users who can connect, which often results in 'busy sessions', something much rarer with HTTP (as HTTP servers typically have very high thresholds for the number of concurrent connections they will support). The overhead on a server of a user downloading a file over FTP is much greater than that of a user downloading the same file over HTTP.
Although FTP is of course theoretically more reliable than HTTP, in practice, because of 'Server busy: Too many users' messages combined with the speed and reliability of modern connections (which in turn makes HTTP more reliable) mean the the reverse is often the case from a user perspective - which is what I think the poster is getting at.
This may be partly due to poor FTP server configuration defaults and/or poor administration, but they cannot shoulder all the blame.
The potential lack of reliability with HTTP is a very minor issue these days, and the extra overhead of integrity checking files in addition to relying on TCP is just not warranted for all but the largest of files.
This doesn't make FTP completely redundant, but it does make it make it redundant when your files are small and your users are on fast, reliable connections (though the value of 'fast' varies in relation to the size of the file, even 33 kbps is 'fast' compared to the speed of connections that proliferated when the File Transfer Protocol was developed).
I did some tests a few moths ago, HTTP sends about 2% less bytes than FTP.
Its because FTP sends data in bigger packets.
Less packets means less TCP/IP overheads.
I think people should mod this up until someone from FreeBSD confirms if this is true or not.
HTTP is amateur and will make you look a wimp.
FTP is a crumbling legacy protocol and will make you look a dinosaur.
Why care so much what others think? Efficiency beats out looks any day.
Reading a lot of these posts, I find many, many people saying how difficult FTP is to get set up and working. It's really not that hard. My server (see .sig) took a total of a few hours to set up. The hard part was finding people to help me troubleshoot, since I couldn't very well do it from inside my network.
I'm using Pure-FTPd, and I recommend it if you're using a platform it'll run on. To get through a firewall, just set it to know your external IP address (with -P), set it to use a particular range of ports for passive connections (with -p) and set your router to route those ports to the server. Easy.
I found the meaning of life the other day, but I had write-only access.
I agree with your post - though I'd add to that - in saying the less over head required by HTTP makes it even faster that FTP.
(And also that HTTP traffic generates less serverload than FTP).
Dan J. Bernstein has written a fantastic, lightweight server that will serve files via either or both FTP and HTTP depending on how the client connects.
If you want to serve files to the public, this is the most secure way to do so. If you need to provide the files to only certain logins, use something else. If not, you can run this on very lightweight hardware and if it's the only server running, you won't get hacked. Period.
If you want speed, but to not look like a dinosaur, use CVS... then when people can't connect, then treat them like they are idiots.
But FTP is faster.
Has a poor implementation of FTP. FTP may wreck havoc on IE, crashing every window. Yes, my friend, the standard is long broken, but it was not broken on its own. Using a client FTP program like SmartFTP or any variant will yield amazingly better results than IE. Then again, you could just use Mozilla/KHTML/Opera (anything but IE) and have no trouble.
The end user has these problems. That is why. It is the fault of a single monopolistic application--made by Microsoft, enough said.
SAFT & rsyc - why aren't these implemented into browsers and NNTP clients? Freaking ridiculous that we have these excellent methods available (for YEARS - they both predate the web, in fact), but noone knows about them as they've not been mainstreamed into appropriate places. *sigh*
And if you want to do a distributed thing, check out 'BitTorrent' - seems very interesting (though new and unproved).
I see too many FUD replies here:
:-)
1) HTTP doesn't support resumed downloading.
- That's ridiculous. It has since HTTP/1.1 years ago. In fact, it can even do things like request bytes 70,000 - 80,000, then 90,763 - 96,450, etc.
2) HTTP doesn't support security/authentication
- Ridiculous. HTTP has an open-ended model for authentication and security, many of which are secure and standardized. If you REALLY need security, use HTTPS.
3) HTTP doesn't support uploading
- HTTP/1.1 has had this for a while. Netscape 4.7, Mozilla 1.1, and IE 4+ support this. I must admit though, it sucks.
Several people have pointed out the real differences:
1) FTP doesn't like firewalls
- Passive FTP fixes this, but it has quirks and limitations.
2) FTP supports directory listing, renaming, uploading, changing of permissions, etc.
- This is what FTP is for
- This can be done in HTTP, but requires serious work
- If the scope creeps, shell access would be better.
What is the nature of your audience? Are they going to grab one file at a time, or pick a bunch of files? HTTP is better for quick grabbing one thing. FTP is nice if you want to see a list of things and queue them up. Yeah, you CAN queue stuff with those crappy HTTP managers, but I haven't come across one I like yet.
What is your primary audiences connection? FTP is easier to resume from. HTTP requires more shitty download spamming managers. (or wget, but lets be realistic here) If most of the users might be on broadband, then who cares if the HTTP connection drops and they have to reget the file? But if they are dialup, then having to redownload sucks.
For relatively small files, when authentication isn't an issue, go with HTTP.
I personally would do both, just because FTP gives you so much more control.
-- Having a Creationist Museum is like having an Atheist place of worship
Would use sftp in a SSH tunnel over IPv6 encapsulated in IPv4 over X-25. Don't forget to encrypt, obviosuly with an unknown russian algorithm, not only the files but also the instructions on how to download'em. If you have some spare time, instead of sftp you can develop your own [undocumented] protocol.
This way you won't look like a wimp. Jaws will drop, you'll be recognized as a uber BOFH and your peers will respect you.
I've seen a lot of posts saying "use HTTP because you don't have problems with firewalls". This attitude pisses me off. If you don't want firewalls to interfere with your traffic, then dyke them out of your network. If you can't, then you're probably at work and shouldn't be sharing files over the open 'net anyway. Understand that firewalls are there to protect you, and that for each time you curse them, 1000 hackers are denied access to your kiddie pr0n and stolen credit card numbers. Also understand that port 21 is blocked for a reason, and port 80 should be too except for direct connections to your web site. If you're on an ISP that blocks port 80, and you still want to share your ripped MP3s, get a business account. Don't whine about how life is unfair.
On a related note, a big "what the fuck are you thinking" goes out to people who design/run non-HTTP services over port 80, or even port 443 (you know who you are). Did you people bother to read the OSI layer 4 spec? If all your services are going to run over port 80 to avoid firewalls, why the hell bother with port numbers in the first place? Build your own protocol, fine, but release an RFC and get your own assigned port number for it for fuck's sake. That's their entire purpose in life, after all.
[/rant]
This post expresses my opinion, not that of my employer. And yes, IAAL.
Covalent's Apache FTP module
I am not sure if there is really a purpose to this and am just throwing ideas around. But you could host an FTP on an internal network and route it through an HTTP CGI script. You can also use encryption over the HTTP connection to make it secure without resorting to SFTP. I just happen to use this method myself since SFTP requires special clients and is a bitch to setup...just some food for thought
scp is your friend. Learn how to use it, and it will handle all of your (non-anonymous) file transfers. It is a beautiful thing.
Using your sig line to advertise for friends is lame.
...I prefer CPIP rather than cumbersome TCP/IP applications like HTTP or FTP. If it was good enough for Noah, then it's good enough for me.
this sig is a highly rehearsed improvisation
Ok, I've seen several posts now regarding the seemingly-dead FSP protocol, which I'd never heard of until today. Someone care to explain this?
"Quoting famous computer scientists out of context is the root of all evil (or at least most of it) in programming." - K
cpio
Secure FTP
Dolemite
Save the World! Use a Quote!
The problem with uploading to Gnutella1 [compared to Gnutella2] is that the other people downloading the file have a harder time getting it
Then how does one distribute programs designed for platforms other than Microsoft Windows using the Gnutella2 protocol, which (last time I checked) only Windows-platform Gnutella clients support, and which Windows users are unlikely to download and share?
Will I retire or break 10K?
Use the hotline-compatible hxd. Supports resume and a bunch of other stuff. Like a bbs.
Just use IPv6. By tacking on a "sixxs.org" on the end of the URL, you will achieve technological elite status. The best part is that since most people aren't on that backbone, we can't verify if you're telling the truth or not.
You can run a HTTP and FTP on the same port using an app called JavaSwitch. Doing this can actually make FTP a little more secure.
Let's say you use JavaSwitch and have it accepting HTTP and FTP traffic. You can set JavaSwitch up to run on port 80 (the "standard" port for webservers). Now when someone scans your ports all they will see is port 80 open and assume that all you are running is a webserver when in fact you are running a webserver and a ftp server. This still doesn't help the fact that FTP is a clear-text protocol and shouldn't be used anyway. Stick with SSH.
You can get JavaSwitch here.
A clever person solves a problem. A wise person avoids it. -- Einstein
If you're anticipating a large number of downloads, especially in a short period of time, you should really use Bittorrent, although it might not be necesary with such small files, but you'd save money on bandwidth.
I run a ftp for similar file-sizes (1-6 meg) using a Windows 2000 Pro box (yeah i know i should stick to my preachings about the wonders of linux but im not 100% with my abilities to lock down linux yet) and im using Bulletproof FTP server which is hella cheap but has every feature you can need and is very secure. I highly recommend it. It handles beautifully.
"Self-destruction might be the answer" --Tyler Durden
VSFTPD is the best ftp server out there today, ok maybe Tux 2.0 is as good. VSFTPD has never been broken into and RedHat, and a number of other high profile FTP sites use it on their distribution servers. It's also damned fast. Homepage
Right, because HTTP *isn't* plain text
Were that I say, pancakes?
What rsync needs is a port assignment (in addition to its existing 873 for clear connections) for SSL/TLS. Alternatively a "STARTTLS" feature added to the rsync protocol could do the same job. What this gets us is the ability to create a secure SSL service where passwords and data are reasonably confidential, while not having to worry about adding users to the host system itself (which is required for either rsync over ssh, or for scp as an alternative) which exposes a risk of accidentally making them shell users (not everyone wants to do that).
now we need to go OSS in diesel cars
To prove it, I built this:
p :/ /www.jonandnic.com/pictures&dir=d:/pictures&name=p ictures&thumbs=1
http://www.jonandnic.com/explorer2.99/?path=htt
I happen to have anonymous access via SFTP. I also allow anonymous SSH too. It pretty much consists of making an 'anonymous' user w/o a password... BTW, if there is a way to find my IP address from /., please don't try it. I don't have the bw to take a /.ing...
Luke-Jr
Try running the FTP server behind one firewall with NAT while the client is behind its own firewall with NAT, and both machines have the same private address. Really confusing at best.
now we need to go OSS in diesel cars
sftp incurs a terrible CPU overhead, especially if many people are going to be downloading at once. I doubt most web servers could concurrently handle a few dozen 3des encrypted sftp connections without slowing throughput, and if you're hosting files, thats the last thing you want. FTP is supposedly more bandwidth-efficient (though I've never seen proof), but I can still get 400k/sec downstream over http, so I don't think it's a huge problem. I'd just use http for the ease of setup. Securing a public-access ftpd is a true pain.
You don't have to sit there waiting and waiting
for that 500 megabyte ISO image to transfer.
HTTP isn't clear text?
x t/html;q=0.9,text/plain;q=0.8,video/x-mng,image/pn g,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1
A ccept-Language: en-us, en;q=0.50
This is a request from a web browser.
GET / HTTP/1.1
Host: 127.0.0.1:16
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.2b) Gecko/20020923 Phoenix/0.1
Accept: text/xml,application/xml,application/xhtml+xml,te
Accept-Encoding: gzip, deflate, compress;q=0.9
Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
Keep-Alive: 300
Connection: keep-alive
Seems plain enough to me.
A clever person solves a problem. A wise person avoids it. -- Einstein
Hmm.. There's no reason why you couldn't just perform the download via HTTP if your target customer or audience has the bandwidth.
If you're concerned about the customer losing connectivity, you could always just break the file up into small chunks for download.
Both FTP and HTTP, depending on clients and servers, supports continuation-downloading(ie, recovering from where you left off.) So if one connection flakes out, you can always recover.
The long and short of it?
HTTP(S) or FTP(s/FTP), it doesn't matter. With both, you can restrict access and/or grant access to specific files. You can recover from some point or offer split files. With HTTP, you can have the information piped through a back-end piece of code so that you can monitor who does what when, but something similar can be done with FTP as well.
If you've already got a webserver going and secured, you're probably better off offering the files via HTTP. Nothing unprofessional about it.
Fact is, if your target audience is the general public, you'll fare better with HTTP. Why? Because while most people who are online will probably have access to a web browser and know how to download HTTP files, they may not have an FTP client or necessarily know how to use their browser as an FTP client.
Me? I offer photos and docs online via HTTP. It's easier and I can control the visual aspect of the "download area" to create a more professional look than that of the "industrial" FTP interface.
Nothing wrong with FTP or sFTP. I use them at work and at home. But then again, when I find a site which offers HTTP or FTP? I pick HTTP to download first. It's only when I'm still looking for something that I go with FTP... which, by the way, becomes difficult to navigate on some clients. A nicely generated webpage featuring the item via HTTP would be much more elegant.
Just my two cents.
Winged Power Photography
while FTP firewalling will give you prematurely grey hair (and if it doesn't, then you're not doing it right).
Ahahahahahahahhahahh... now that was FUNNY. Very true!
personally I like http downloads over ftp. I'm on dialup. I like to know how long its going to take to download a file. I don't get that info with ftp. as for http being slow? never noticed. but I would if I couldn't tell how long it was taking.
-
Most of the time, if you're offering files for download, a great portion of your user base will be at the college student level, and most universities bottleneck your ability to download via ftp thorugh standard ports, limiting your ftp download speed. I live in a dorm and because of the bottlenecked ftp I have to download via ftp to get it done reasonably, which is sad, because I have to download my linux iso's via http as well. Something to keep in mind if your user base will be student centric.
Choose wisely you must...
If it ain't broke, don't fix it.
This sig no verb.
He's talking authentication schemes, and HTTP has many more options. HTTP Digest authentication allows non-cleatext passwords. There is no such option with FTP, all passwords are cleartext.
And though it's technically HTTPS not HTTP, at least there is a common implementation of HTTP over SSL out there. Everything encrypted, and you can even skip passwords entiredly and do authentication using certificates if you want to go through the trouble. FTP over SSL isn't all that common.
Well there's always the option of FTP over SSH2. I'm sure you can find a Java applet that will do the SSH2 and make the tunnel needed for secure FTP.
Now when we talk about Java, there's another possibility. Some sites (cr@ck) use a Java downloader. It doesn't mean that the Java applet that downloads the file uses HTTP or FTP, it can be some sort of propriety protocol (or you can combine the best of both worlds.)
One way is to have the applet on a SSL'ed (https) page and it does some decrypting as it downloads a pre-encrypted file from your FTP. Or the person can just download the encrypted file directly and use the applet on the secured page to decrypt it. There's ALWAYS a way to have your cake and eat it all by yourself, too.
If its tech folk, ftp would be perferable... if it's non-tech folk, http makes things easier for them.... But this is from a guy who misses gopher, so take it for what its worth
- Do you have bandwidth issues? If you are serving files to many people, FTP servers allow maximum concurrent users, which can be useful. I know you can do this with HTTP, but it's difficult to segment the downloading >1Mb files traffic from the normal site traffic. A separate service also allows you to use all the Quality of Service stuff in the 2.4 kernel nicely.
- Do you have a large array of files that the user might want to download, such that using an FTP client to ctrl+select multiple files is the right answer compared to having your users click on twenty links and have to cope with twenty dialog boxes?
- Do your users need to be able to upload files to you? This can be done with HTTP, but you'll need some PHP processing or similar on the server, it doesn't support resuming, and it won't work through many company firewalls, and therefore isn't a good option. HTTP uploading it particularly hopeless for large files, as it provides no user-feedback.
However, you should NOT use FTP if you answer no to either of these:- Are you running some flavour of unix? There just aren't any robust Windows FTP servers. Yes, I'm prepared for the flame war about this.
:) - Can you be bothered to keep your FTPd patched? ProFTPd and WU-FTPd are both frequent appearers on bugtraq. You need to stay on top of the patches, or you will be 0wn3d.
Simple, see?I work for an organization that performs hosting for a website that hosts both large and small files (avg 1-5 MB, max ~200MB). We support download via HTTP and FTP both. As a general rule, the website directs users toward using HTTP to download files, and by far (about 10 or 20 to 1) we see users grabbing files via HTTP.
A note however. If users want to download large numbers of small(ish) files, and you don't care about authentication, FTP becomes the far more preferable option for our users. The command set for FTP just makes it quite a bit easier to use wildcards to specify a large number of files.
Of course, if security is important, go for HTTPS or SFTP. Just remember that Joe Windows never heard of SFTP.
As always YMMV.
In my case, I run an Apache server on my file server box because it gives me full 100% throughput at Ethernet speeds. That's 10+ megaBYTES/sec. The average anime episode downloads in 10-15 seconds, and when friends come over, I don't have to worry about them having the right kind of client when they want to leech.
Where FTP becomes useful is when you want people to upload to you.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
And an FTP or HTTP server (running chrooted, just in case, and owning none of its own files) in user-mode (after the connect) and with the authentication and other fruit stripped out is dead easy to make effectively unbreakable.
It wouldn't be hard to fake up an encryptionless scp server either, and don't forget Gopher!
Got time? Spend some of it coding or testing
...is get yourself some better pr0n! Jesus man, those better be picture files and not movies!
If all you have is a hammer, everything looks like a nail.
Any serious network security professional who's not just faking it considers the FTP protocol an *evil* *abomination*. This protocol MUST DIE! Someone please publish an RFC that "officially" deprecates it.
Note I'm talking *protocol*, not any particular client or server implementation. The protocol is fundamentally flawed.
Well im afrade that you have to realise that linux isn ot he only os about. Windows has had resuming of both ftp and HTTP download for years now with programs like GETRIGHT and Flash GET. So resuming is not an issue for 80% of the Internet population. HTTP is userly faster then ftp and has no problem with firewalls. Dont limit your views to *nix based systems or you become as bast as windows users. Yes i know there is bad grammer, spelling and what not in this reply im dislexic and proud --- Andy---
I prefer downloading via HTTP. It's usually faster for moderate sized files. I run both an HTTP and FTP server, but i rarely use the FTP server for anything but uploads. HTTP's just easier to deal with.
Who's the user? If it's WIndows lusers, then by all means use http. It makes their life easier. But if the users are *NIX, Mac or other, then use ftp. These folks have a clue and can deal with it.
Warning! The above is sarcasm. But it is so apropos. Paraphrased quote from a conversation I had with a Windows luser:
"I need to find a good FTP program so I can download this software I found. Which do you think is better, ProFTP2K for $25, or should I spend more for EZDownloadFTPPlus at $50?"
A Government Is a Body of People, Usually Notably Ungoverned
All I use is secure FTP. Seems fine to me and no DUNS number necessary.
As far as ease of use:
1) Get a better computer
2) Get Fugu
3) Get a clue. End-Users aren't that difficult, you just need to be able to communicate.
Yeah? Go fish! Does require version 3.1 though.
Got time? Spend some of it coding or testing
After hosting an HTTP file transfer area for some time for my company, we decided to move to an FTP setup that was a bit more sophisticated.
So far, it's been a failure for two reasons:
1. IE blows as an FTP client, and users aren't comfortable dropping into the (somewhat crappy) DOS FTP client.
2. Firewall setups at the fortune 500 companies that we deal with normally seem to keep FTP access off-site restricted.
Why are you letting these clowns ruin our country?
I just set up WeebleFM http://sourceforge.net/projects/weeblefm/
It's a PHP front end to FTP. My FTP ports are only open to the loopback interface. Users get the usablity of a clean web interface, and I get to have encrypted password-controlled FTP on a box that only has port 80 open to the internet.
WeebleFM uses mcrypt to encrypt traffic (and I'm pretty sure I could get it to work over https).
Using standard unix permissions, a careful directory schem, and vsftpd's chroot capabilities, I can have an internet filesharing arrangement with blind drop boxes, a group accessible directory and any number of world readable directories.
Some ISP's (and companies if the user is behind a corporate firewall) provide transparent caching proxies for HTTP transfers, saving you the cost of actually serving their copy of the file.
I just put all of my public files on a big smb share. Nothing beets M$ for secure file sharing. Just don't put none of them underscores in you'r domain names. Those broken legacy unics domain controllers can't read them. Boy... That had me stumped fer a while. Thank god for the helpful M$ community, or I'd still be try'n to email all them big files. -U6H!
That's not true--they can tell roughly what the size of files you're looking at, and if you're browsing, which files were triggered-downloaded--for example images embedded in web pages.
All this information makes it trivial to find out which pages you downloaded, when, and how long you visited.
The only thing it doesn't do so well is listen in on data you send. But consider--if they're listening to the tcp channel to the remote site then they can listen to outgoing tcp connectivity as well.--and correlate what information came in, and what information went back out.
You think a single encrypted channel buys you any privacy? Get real.
I say screw both - use SSH instead!
The1Genius - Littera Scripta Manet
FTP implementations frequently use a fixed, small window size. HTTP on the other hand will honor the system limit, almost always larger even without tuning.
t mls _tcp tune.html/ faq.html
Dramatically simplified, it means that the connection can send a lot more packets without hearing back from the far end, enabling the connection to reach higher speeds (imagine a phone call where you had to say 'okay' after every word the other person said. Now imagine only having to say it after every sentence. Much faster.)
The tiny window size of (most crappy legacy implementations of) FTP starts to affect download speed at just 25ms latency, and has a huge effect over 50ms.
A properly tuned system with HTTP can make a single high-latency transfer hundreds or even thousands of times faster than FTP.
Relevant links:
http://www.psc.edu/networking/perf_tune.h
http://www.nlanr.net/NLANRPackets/v1.3/window
http://dast.nlanr.net/Projects/Autobuf
sar.casm \'sa:r-.kaz-*m\ \sa:r-'kas-tik\ \-ti-k(*-)le-\ n [F sarcasme, fr.
LL sarcasmos, fr. Gk sarkasmos, fr. sarka]zein to tear flesh, bite the lips
in rage, sneer, fr. sark-, sarx flesh; akin to Av thwar*s to cut 1: a
cutting, hostile, or contemptuous remark : GIBE 2: the use of caustic or
ironic language - sar.cas.tic aj - sar.cas.ti.cal.ly av
Were that I say, pancakes?
Who the hell authenticates when serving up files?
Geez, anonymous read access for all. However,
one could always use sftp.
Were that I say, pancakes?
In Unix, you can background this operation fairly simply...
This will start downloading your file in the background, will autoresume if something happens, waiting 15 seconds between retries, and never give up until the file is completed or the source no longer has it. You'll get an email when it completes one way or another.
Help us build a better map!
I'm an independent web developer who is unhappily dependent on ftp. The main problem is that none of the companies that host my clients' web sites offer secure ftp. More often than not, I inherit web sites that are already hosted somewhere, and the client is not interested in switching providers.
I have recently partnered with a hosting company that can support secure ftp, and I'm working on getting that going. However, there's no way I can convince all (or even most) of my clients to change host providers at a higher cost, just because the ftp is secure. I suppose I could try to strongarm them into it, but, well, I'd kind of like to keep my clients.
It seems to me that if web hosting companies would start encouraging sftp, then us web developers could start phasing out this ugly ftp stuff. That should (hopefully) get the ball rolling.
Let's start a grassroots campaign! "Nerds Against FTP" (NAFTP) Soon we'll start seeing green "FTP Free" banners everywhere!
-SB
"Molest me not with this pocket calculator stuff."
- Deep Thought
Polite rant:
I have never been a spelling fascist but this is to much. Have you ever heard of spell check? You are insulting everyone on slashdot. Typo here and there is excusable but "inventing" your own orthography is plain rude.
Honest rant:
Fuck you!
I'm not trying to suggest that I wasn't that clueless at one time, just that maybe this isn't a /. caliber question.
Oh well, kudos to the poster for being concerned enough about security to ask around. I'll just shut up now.
Oh yeah; FTP.
"A language that doesn't affect the way you think about programming, is not worth knowing" - Alan Perlis
So I should abandon encryption and just be free to let whomever listen to traffic coming from my box? If the the images are locally referenced, they'll be transferred securely as well (talking https here). And modern browsers keep connections open for reuse, so that only thing that can be gleaned is when a spike happens, which means that you are loading content from the same site, but not how big it is, since a web page is a bundle of different elements that need to be retrieved.
Also, if you think that just knowing when data packets are sent and received tells you the contents of the data, then why hasn't SSH been broken yet? With those assumptions, it should be easy to use a statistical model of people's keystrokes and break into a session. Realistically? Not possible.
You've got to start somewhere, especially if you're going to get the world to transfer to using encryption everywhere. The mail server I use for private mail among friends has no unsecure channel going out of it. All of it's protocols are protected with SSL/TLS, including SMTP inbound and outbound. And besides, the argument I was making was for using encryption to connect to a public mirror site. Since the majority of mirrors have HUGE numbers of different files, and the amount of control data sent is never constant, knowing how much data is sent in either direction will NOT be a guarantee that you know what was transferred.
AND, I was talking about legal prosecution (or persecution, depending on how you see it), and a document that says you downloaded approximately enough bytes to be a suspect program isn't enough to stand up in court...
and this isn't even a cliff post.
.000002 cents just because I feel like typing tonight.
To the question at hand I have to thrown in my
HTTP, good for files small enough to be downloaded in seconds. FTP anonymous access, good for files that take minutes or more.
People who don't use this technique are the exception not the rule. Unfortunately people exist who refuse to look beyond the face of getting the job done. In areas were outcomes are not always rendered under ideal controlled conditions this leads to unpredictable outcomes. People who think simply either deal with this or allow their servers to propagate worms like nimbda, codered, or slammer.
That being said your best bet is to run a chrooted ftp server with anonymous access. Use this server for large files and your http server for webpages and images.
You want bittorrent
l
http://www.bitconjurer.org/BitTorrent/index.htm
It makes it so a few people start downloading, and they in turn upload what they have to others, and it just kind of "spiderwebs" out, reducing the strain on the original host.
I wish huge projects (distros, mozilla, etc) started using this. It would make everything SO fast.
I'm not the guy that coded it, just a happy user.
Wow. That's a strange twist. No, Mr. Pot, I am not black.
He created a cockamamie setup involving external daemons and strange rulesets and workarounds. This is the amateurish stuff that household cowboys (most of whom post to Slasdhdot, it seems) hack up and call "done".
My NAT'ed FTP servers have one or two firewall rules and a proper configuration and they work. Give me a 500MHz machine and I'll set up an FTP server (compiled from source) behind NAT inside of an hour. Properly.
BD Phone Home!
Shameless plug. Like you weren't expecting it.
ftp> get mybigfile.iso
/* walk away from PC. Come back 2 hours later */
/* Doh!!!!! */
file transfer started
ASCII data transfer of mybigfile.iso successfully completed.
I know it's not openSSH, but they have a very nice UNIX product, and the M$ Windows product comes with SFTP (both with icons and the applications are self-explanatory)
FTP, in my opinion is antiquated.
In the uploads arena, I could transfer any file faster on rsync or scp, or even cvs -z5. And thats over 56k.
FTP isn't so secure, it's pratically text on port 23 (wonder why the warez sites still use it, aren't they trying to avoid goverment agencies? and M$ and all that crap?).
HTTP is faster for smaller files as FTP eates up a whole lot of time doing the connection procedure.
The biggest advantage of XML is that software does not have to be changed for a different XML data format - they all use XML, the standard bracketing syntax. I know most major browsers support an XML view mode, so they don't need to be upgraded to download by XML.
I'm sure that there are people out there saying that XML is inefficient, but that's simply not the case. We can use special XML commands that allow us to include large blocks of binary data -- at the expense of portability to 14-bit computers, of course. In total, an XML download should only have about 30kb of metadata added to it. Author, guid, PGP signature, original source, license, and all that good stuff.
We can hope for the day when the need for binary transfers will be over, since everyone will be using XML files.
I worked on the DEW Line in the early eighties. The USAF installed a crypto system that had been compromised. We were not allowed to send any classified traffic over it. One reason we thought of for this was to give the Russians a hobby breaking the keys and decrypting our purely administrative traffic. The more likely reason was that USAF was completely insane.
Use Bulletproof FTP, if you're running under Windows. I'm sure other download managers allow it too, but BPFTP seems to not include crapware. That, and it's a decent FTP client.
--grendel drago
Laws do not persuade just because they threaten. --Seneca
If you're talking about the human engineering aspect of this discussion only, then I have no disagreement with you. However, FTP is just as technically feasible over SSL, since SSL works at a lower level on the network stack than FTP.
Furthermore, there are good FTP clients that have SSL support. For example, CuteFTP supports FTP over SSL (and has a very user-friendly interface, for the clueless end user).
There are a good number of servers supporting FTP over SSL. ServU and Sambar are some of the windows servers. Just do a google see what else there is.
All the time - even four years ago when I was relying on HTTP's resume to work around a cable modem which would hang connections after 50KB transfered (don't ask), it didn't require anything unusual.
Amen to you. Way to many people thinking they set things up correctly simply cause it works. A sledghammer and a bulldozer both work to demolish a house. Choose the correct one and you'll save yourself a lot of time.
HTTP is definitely faster, because there are fewer security checks, and there's no reverse connection to make. As with most things, it's not a matter of what's empirically better, but what's more appropriate for which task. If you need a quick download and you don't think any data is going to be distorted beyond TCP's ability to sort through, HTTP is your man. When integrity counts, FTP or some other protocol with more stringent checks would be the choice.
I, for one, welcome our new Antichrist overlord.
Like most /. types, I'm frequently pulling down files from all sorts of different servers. I'll pick HTTP any time I have the choice as my experience has been that FTP is far more likely to be slow and unreliable.
I'd prefer to admin Apache rather than any FTP server I've used, simply because it's more flexible.
There's also a technical problem with FTP - the split control / data feature. Firewalls rarely break HTTP; FTP is far more likely to get complaints from users. Most of the servers are also rather low quality, too - small HTTP transfers almost always complete before an FTP server finishes connecting, let alone transfering a file, and I've run into a surprising number which don't allow you to resume transfers.
Seems to be a lot of comments about firewalls and FTP from people who obviously don't work with them. Remember there are three basic types of firewall technology: packet filters, proxies, and stateful inspection.
Packet filtering alone is always a problem because you have to open up all of the high ports.
Proxy firewalls and FTP (active or passive) are a no brainer as long as either feature has been enabled. Remember that proxies "watch" the conversation so it will manage the connection if it's data coming back to the client on port 20, and will recognize the 'pasv' command in the command channel.
Stateful Inspection firewalls include proxying code for the major protocols ie FTP, HTTP, Telnet, etc. So you are covered here as well.
If you are having problems using FTP through a firewall then you are probably:
-Are being blocked intentionally
-Have a lazy security admin who hasn't updated the firewall in five years
-Have a stupid router jockey "securing" the network with router ACLs (packet filters).
As long as you are using a major firewall release like Checkpoint, PIX, Netscreen, IPTables, etc, that is up to date there will not be an issue getting FTP to work.
Ettercap is your friend here. :)
i have some stuff you can smoke that will mellow you out.
in the meantime, he wasn't just talking about getting FTP working with NAT, he was talking about getting FTP through ssh tunnelling working with NAT.
Shame on Google.
Thank you.
It seems everyone talks about DOWNLOADING. How about UPLOADING? How fast and wonderful is HTTP for that?
End of story.
Ben "You have your mind on computers, it seems."
what's wonderful about ssh is that it provides essentially a security layer. the entire ftp connection is tunneled through ssh. same goes for any software that needs to connect to another computer. it's a brilliant way to secure a connection without having to reimplement all the encryption
BSD is for people who love UNIX. Linux is for those who hate Microsoft.
times like this I want Gopher back.
in the meantime, he wasn't just talking about getting FTP working with NAT, he was talking about getting FTP through ssh tunnelling working with NAT. hm..like its that hard? seriously, it took no more than 15 minuttes getting working my SFTP server. Either you're clueless or just plain stupid, prolly the latter....
I have never thought of FTP as a crumbling legacy protocol full of security holes. Each protocol has its flaws, security flaws or not. HTTP isn't free from security holes.
They are two different protocols designed for two different purposes. If you plan on transfering large amounts of data, and want it to be more reliable, I would most definetely recomment FTP. FTP is made for transferring files, and that is what it is best at doing.
If you are not concearned with data integrity and plan on transferring decent sized files, I would wish you the best of luck with HTTP.
As far as security holes go? I have no idea, but I imagine both protocols have had their fair share of security issues.
For more reliable transfer of larger files, I would really recommend FTP.
---
Mike
I'm going to kick the next person that I see with their karma rating in their sig.
You might consider setting up a BitTorrent tracker on your website, and making the files available that way. It's not any worse than website bandwidth, and since you're also leveraging the outbound bandwidth of sites that download the file, it could be faster.
Editor Emeritus and Senior Writer, TeleRead.org
It's like FTP, but it's SECURE!
FTP:
No standard directory listing
Designed for human readable command lines not machine readability
Bad error recovery, many differing standards on restart
Password sent in plain text, not even concealed
ASCII mode! Need I say more?
HTTP: none of the above
Serve out anonymous FTP through public file (http://cr.yp.to/publicfile.html). Then there aren't any security holes.
Really. The security holes in sendmail can be fixed by installing qmail. The security holes in BIND can be fixed by installing djbdns. The security holes in WuFTP (and most others) can be fixed by installing publicfile. There are also other good programs out there as well.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
Yep, I really like SFTP. It works the way I'm accustomed to working with FTP when I'm at a command line. It's secure.
I'm not sure why more people haven't mentioned SFTP. It really is great.
--Richard
What a load of BS!
/etc/hosts.allow file. That way if you're not in the proper IRC channel (where I can talk to you in case you're fucking up) you won't be able to connect to the FTP. One more thing not to worry about when securing the box.
When an admin tries to block multiple connections from the same client, it's not because he doesn't want this client to get his files in 1/6th the time. He wants Joe Sixpack to get the files without squeezing some poor bastard on a 28k8 (yes, they still exist) off of the system.
Plus, how often does this multiple connection crap actually yield a performance boost? Getting 2 connections normally doesn't give more than a few kb/s and each subsequent one will just give you less speed, if any.
I've had a stupid mfer trying to get no less than 1000 simultaneous connections off of my FTP!
Going back to the original question, I feel there's more control over what's happening when you're working with FTP, which is particularly handy when you've got a small, localized FTP running. I've made an IRC bot that will put the IPs of the various people in the channel into the
Yes, yes, I know that this only works at connection time, and a user is free to leave IRC again once the connection has been made. Still, it's better than nothing (and for HTTP, it clearly is just that: Nothing).
My 0.02
With the proper, stateful, FW there's no need to open any high ports. The FW's in Linux and *BSD can have all (high) ports closed and recognize when they need to open _one_ port to _one_ client for a short period of time.
Peder
ftp includes mget. So I can download all the pictures with:
mget *.jpg
Without messing around clicking and saving lots of things. Sometimes a command line rocks.
Andrew
If you're providing downloads that may reach a decent volume, I would recommend a caching appliance. A CacheFlow server (they were bought out by BlueCoat systems) or something similar should do the trick. These can cache HTTP and FTP inbound requests, and instead of having the load on your servers, these things will serve up your files instead. A good cache appliance can serve up files a lot faster/more-efficient than a web server, and you won't bog down your box with processes.
These caching appliances can also broadcast multimedia streams (Real or WinMedia) with the correct licenses.
I would do some research on some CacheFlow boxes, which, by the way, are selling on the cheap at eBay. These were the $25,000 (circa 1998-2000) priced appliances that you can now pick up for, sometimes, in the hundreds of very low thousands.
Serve up over P2P protocols, and tell people what filename to look for. If the file becomes very popular, you'll only need to serve up the page giving directions on how to download it :)
(yes, yes, I know, requiring people to use spyware-laden software is not nice, but it's a good idea in an ideal world)
Stop the brainwash
Instead of banning download managers that can do segmented downloading, why not just limit the number of connexions from a given IP? That solves both the segmenting and the "tons of files at once" problems.
I can make a dozen or more connexions to your FTP server with nothing more exotic than Netscape. Why pick on download managers when they use the same number of connexions? (BTW, Getright says right in its configuration that "some servers regard segmenting as rude" and recommends against it.) Better to limit connexions to x-many per IP address, and let the user spend them any way they wish.
BTW, if you do limit connexions, please remember that it usually takes one for browsing the site (using Netscape or whatever) PLUS one for the download manager to fetch the file. Otherwise the user who was looking with a browser has to leave the server, then wait for the browser connexion to close (which can take a while) then finally paste the link into the DLManager. So a limit of two connexions from a given IP is a nice practical minimum, and surely not a hard load for anything outside of home servers operating over dialup.
PS. I love FTP's convenience, and I always try to be extra-polite to small servers (and not rude to big ones). I do use Getright, and have segmenting disabled (which BTW is the default).
~REZ~ #43301. Who'd fake being me anyway?
Because it hurts people behind NAT
What do you want to transfer ?
In another view, I'm really fed up with the current 'anything over HTTP' approach, there are even VPN over HTTP...
#include "coucou.h"
Why and how is that?
I see connexion-per-IP limits fairly often, tho mostly on small or private servers -- which of course are the ones more adversely-affected by connexion hogging. Whereas a server with a large user limit really isn't going to be all THAT crippled by a few people using several connexions apiece.
~REZ~ #43301. Who'd fake being me anyway?
HTTP is OK, but try downloading that file from the command line. Even Windows has a command-line ftp-client.
Also, FTP has the useful "mget"-command to download multiple files. Try that with HTTP.
Finally, many FTP-clients can download whole directory-trees. I have yet to see this for HTTP, too.
Synopsis: If you have a specific problem, use a specific tool. The generic tool will be able to do everything - badly. A set of specific tools will serve you well in all situations.
I highly recommend using the Berkeley:
for secure ftp transfers. The SafeTP client & server code runs well on Unix / Linux as well as those MS-based boxen.
Our ISP has been successfully using SafeTP for several years. Their windoz users transparently use SafeTP with any Windoz FTP application.
Unix folks get that good ol' command line tool. Other Unix interfaces exist for HP-UX, Solaris, Linux, DEC OSF, FreeBSD, OpenBSD, Irix, AIX, etc. SafeTP works across firewalls as well.
With cryptographic protection and authentication, SafeTP is a nice ftp solution.
chongo (was here)
I've been running at about 90% of my theoretical speed (7.2k/8k) using both http and ftp for big files. I assume the "real" max speed is higher than they tell me though, has managed to do about 7.5k with both multipart ftp and multipart http. Never found any difference between the two though.
Kjella
Live today, because you never know what tomorrow brings
Stand on your rooftop with a CD burner and frisbee CDs to anyone who calls to initiate a transfer!
The ONLY way to serve files!
The main strengths of HTTP over FTP for file transfers are:
The other differences one sees are due to server design issues. I.e. most FTP servers are large and spawn a process per connection, which makes FTP sessions much slower than HTTP sessions. But if you want to use FTP, there are very fast FTP servers out there.
Overall, in today's world, it does not make sense to use FTP unless you have a requirement from your users. For public access to files, use HTTP or something more modern, such as rsynch, or a P2P network.
As usual, you should answer such questions by thinking about your target users and asking yourself what they are likely to be most comfortable using. Chances are it's their main tool, the web browser.
My blog
Grrrrr it's CONNECTION not CONNEXION
And the people shall be oppressed, every one by another, and every one by his neighbour Isaiah 3:5
Pros: FTP-like filesystem-style handling, while still using HTTP.
"Most sites have less than 10Mbits of outgoing bandwidth, which Apache can fill using only a low end Pentium-based webserver."
The rsync algorithm docs say that [...] the much more expensive strong checksum must be calculated for each block where the weak checksum matches.
So the strong checksum (MD5) must be cached for a lot of blocks (typical application: files are nearly identical), and block size is usually 500 - 1000 bytes. Is 1/30th the size of the original (500 bytes vs. 128 bits) acceptable for the checksum cache?
For small stuff, avoiding that extra data-connection-setup overhead is a real winner.
"Little does he know, but there is no 'I' in 'Idiot'!"
If you have a limit of 2 connections per IP, then if on a network of 100 machines behind a NAT gateway, three persons want to download something from that FTP server, at the very least one (if not two) of them is screwed: He has to wait fot the *other* people to having finished their download before he can start his.
"Because other people get it wrong" is no reason to get it wrong, too.
I used to run a server which distributed ~3TB/month. Initially I served these files via proftpd, but it soon became apparent that ftp daemons are far too bulky for high-volume serving.
:)
Enter apache. On the same hardware which keeled under around 30-50 ftp sessions, I could handle over 400 concurrent http sessions, with plenty of ram left over for the vital cacheing
All most people would want to do with FTP can be done with HTTP.
There are hardly any practical legitimate reasons for an FTP server to shove data to some other place rather than the FTP client and other dangerous combinations FTP allows.
Uploads? Uploads had better be disabled on FTP by default. Uploads are disabled in HTTP by default, but you can configure your site to accept uploads.
Mass downloads of files? Just standardise. As is you can already download entire websites. With HTML (not javascript) the links are there. If you want files only you just have to standardise on a program readable way to figure out which links are files to download.
Resuming of downloads - works on most webservers, works on most ftp servers.
And http allows more - better compatibility with firewalls and proxies. Can be made to support SSL easily - most popular clients support SSL.
---
FTP is a bad protocol. The PORT and PASV thing is stupid. Most protocols which specify network layer data (IP address, port) in the content layer for programs to use, are bad designs.
How would PASV or PORT look with IPv6? With HTTP, you don't have to change the protocol much if at all.
FTP traffic is given lower priority than HTTP traffic in a large number of packet shaping / DiffServ type routing algorithms.
These algorithms are based on the assumption that HTTP traffic consists of fairly short bursts and not long sustained transfers which is typically what FTP traffic looks like. Based on these assumptions, these routers give lower priority to FTP traffic than they do to HTTP.
This does not mean that you should serve large files off HTTP since it'll be "faster". Au contraire, it means that you should be fair to others and serve them over FTP, so that the routers can do the correct packet shaping even if it means a slight speed hit to you.
Think of people downloading huge files off your web server and screwing up your warcraft (/quake/whatever) game.
some webservers already allow you to resume broken http downloads. some web browsers are too brain damaged to do it right. in this case use a download manager or copy the URL into as a wget parameter and let it handle it.
What ? Me, worry ?
alot of companys only have one external ip.
that'd mean only one user in the entire company can use the server - bad.
Plenty of people are behind proxies and NAT-like technology, that make a potentially large number of users seem to come from the same IP-address.
Blocking users based on the number of connections from their IP-address is not an option.
/pah
Give me a 500MHz machine and I'll set up an FTP server (compiled from source) behind NAT inside of an hour.
Give me a 486 and about three seconds and I'll set up a HTTP fileserver behind NAT.
Checkout http://cr.yp.to/publicfile.html
I'm participating in a developer course now. The place where it takes place is very closed since all contact with the
outside World is through a Microsoft ISA Server. That means that I can't even check mail, use IRC or Remote Desktop, or
download from FTP sites. Therefore I'd recommend using HTTP links, because if I can reach the web page with the download
link, I'm also sure that I can download.
FTP has lots of design flaws and not to mention implementation problems, and so does HTTP. Check out GSTP, http://sf.net/projects/gstp/. It's TLS-based, and feature-rich.
Don't use https or sftp for big files, it's very resource demanding, unless you have a hardware accelerator.
Why not implement both http and ftp, it's not much effort, just assign the same document root to the (virtual) hosts on both ftp and http.
I don't think there is any big security issues, when just talking about downloading anyways.
...was available in Netscape 4; under windows, just open an FTP url, then drag a file on top of the browser. Under UNIX, after opening an ftp url, an "upload file" option appears in the file menu.
I don't know if Mozilla et al still support this.
It's pure genius! Transfer files via ICMP echo requests! Add a special byte in the ICMP header or data to make the target recognize it as a file transfer and not spam you with echo replies.
Why not implement FTP over ICMP? It will be the best ever!
IMO FTP is a more efficient protocol than HTTP, both in terms of negotiating the connection and also in terms of actually shoving the data down the wire.
:-( so, as with the Betamax vs VHS debate, popularity weighs in heavier than technical merit.
There are plenty of secure ftp servers out there. FreeBSD's native one, and ProFTPD to name but two.
Unfortunately, as I discovered when I had my ADSL line connected almost two years ago, my provider put a higher QoS value on port 80 than port 21
1) Design your own propriatary protocol.
2) Implement it as a primary function of a custom operating system.
3) Leverage this wedge of necessary technology into a monopoly on hardware and software.
4) Profit!
I use Windows... like a two dollar wh.. why don't I just go ahead and not finish that sentence.
Stick with ftp.yoursite.com and www.yoursite.com, do it right, yeah its a lot of work but dont be lazy. Dont assume people have a browser or wget...
I think not.
90% of proxy systems out there don't support HTTP very well. Some don't at all.
FTP: Isn't guaranteed to work from behind a NAT firewall. Usually works, esp. with modern NAT firewalls, but isn't guaranteed. HTTP is. Also, the FTP support for Microsoft proxy servers (Found on many corporate firewalls) isn't that hot.
If you want maximum compatibility, even in the weirdest of network situations, HTTP is the way to go.
And as far as HTTP being slower/faster than FTP - 90% of the time I've been to a site that offered an option of HTTP and FTP downloads, HTTP was significantly faster and was often encouraged by the site because it uses less server resources.
retrorocket.o not found, launch anyway?
once your head is blown off?
No body wants your shitty little files anyway, stick 'em in the mail.
Uuuuhhh, I thought most http connections were not keep alive. So that means your page showing the directory listing (file choice) isn't connected...
Having clicked on a file to download, the http connection is interrupted by the download manager and it then takes over using as many connections as it is setup for.
Your browser just stays on it's listing page. If your download manager has maxed out the allowable number of connections, any link you click on for that site should fail...
At least thats the way I figure it works and has worked for me using a download manager.
Oriental Hero "I want to live in a city where the Police don't shoot you" Jean Charles de Menezes
HTTP is without question easier to use than FTP. Especially in a firewall environment, HTTP will be MORE reliable than FTP for downloads. If you are concerned about security, use SSL for transferring files.
My 2 Cents
It really depends on you and your target user.
Target user:
Http works great for the point and click crowd but can fail badly for the scripting or Linux/Unix command line crowd. FTP is a better choice for the knowledgable crowd but sometimes too complex for the end luser. FTP with a link on a web page is frequently a good compromise.
You:
If you use remote hosting; are not willing to track, chase down, and install ftp patches along with the httpd patches; do not clearly understand how to secure an ftp server; or have other admin skill issues then ftp is probably not for you.
Nooooooo. Too many application already have bad security built in. Tunnel. Use a seperate well tested secure tunnel.
Joe Batt Solid Design
Does he have nice stuff? Can you post an address?
-FF
SQUEAK, the Death of Rats explained.
Last time I tried this on Apache it took like forever. I think this was because the file was encoded, sent and then decoded. The rate was about 1 MB per minute on a 100 Mb LAN.
Limiting the number of connections from a single IP address sounds like a great solution, until you discover that 300,000 IBM employees are only allowed to download your software one person at a time because they all go through the same huge firewall gateway in Colorado.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
What about Kermit with CRC-32 checking and amazing packet length.
Come on people get with the late eighties.
-bucktug
I had a flame... but she had a fire.
One of the limitations of FTP and HTTP for file transfer is their inability to fully use available bandwidth. It's not much of an issue at slow (100Mbps) connections, but just as an experiment, try transfering a few Gigs over GigE. Note the bitrate. Now initiate another connection at the same time, and note the total bitrate. Aha! Parallel streams are better! There are specialized programs for bulk data transfer: pftp bbftp bbcp tsunami gridftp to name a few, but they don't do Windows. Bummer!
I suspect he was talking about the protocol, not just a particular client, so he's using Lynx to talk to http servers - not ftp.
I suspect that was also the point of the original post - not to discuss whether a particular http/ftp client is better than another - because any number of clients can be written, displaying stuff, and interacting with you in any way you want really. The way lynx, wget, curl, fetch or any number of ftp clients interact with the user is irrelevant.
No - what's more intersting is how do they stack up as protocols for tansferring bytes in bulk? I have a feeling that http does just as good a job these days, but this is just based on casual observance of file transfers using both mehtods.
FTP is a better protocol than HTTP if you're intending to use it for transferring of files, especially directory hierarchies. Most people who complain about ftp have three gripes: It's not always friendly to firewalls (but that's a firewall problem, not ftp's fault), it's not secure (that's only partially ftp's fault.. it's really the implementation of the server that's mostly to blame), or that IE doesn't handle it well (yeah, it's a Microsoft product... you expect it to work?)
FTP almost always provides faster streaming (at least it seems that way to me), it's trivial to limit bandwidth and number of connections without impacting your web site (if you serve via http, you can only do this by setting up virtual hosts and mucking with alot of configuration files).
FTP supports continuation, which is more difficult to do in HTTP -- although not impossible.
Modern FTP servers support on-the-fly tar and compression of directories when a recursive retrieval is asked for, and usually the client can choose if they want to unpack it as it comes in, or just leave it as an archive. You could do this in apache, but I've not seen it done yet (in fact, I've not seen recursive directory retrieval in apache).
There ARE other protocols out there (FXP I seem to recall), but none in popular use. SSH also provides a way to get files using SCP or SFTP, but there aren't many good clients in the windoze world.
Now, having said all this.. nothing stops you from providing both services off the same directory.
And that's the level of insightful commentary I expect from someone posting at -1...
Sure, you could write a HTTP client that acts like an FTP client (ls [pattern], mget[pattern], mput [pattern]), but to make it work reliably across all servers, the server would have to have a somewhat-standardized FTP-like LIST command or page, since it would be hard to do globbing when the index page is hard to parse. Actually, you could do a folder-view interface like IE does for FTP, but that interface would suck for lynx.
So I claim that to get the convenience of FTP using HTTP, you'd have to add server features, and write clients to use those features. Of course most people never use globbing, so they might as well use plain HTTP.
The ocean parts and the meteors come down
Laid out in amber, baby.
Actually, "connexion" is a Britishism. (Which I've found I'm less likely to typo due to better "finger flow".) I also support saving the baby U's by preserving colour and honour. :)
~REZ~ #43301. Who'd fake being me anyway?
But how often do you see 100 people behind NAT all wanting to download from the same server, where that server is "small" enough that number of connexions matters??
~REZ~ #43301. Who'd fake being me anyway?
They aren't the only ones who are insane, although the fact that they also have all that firepower is a little worrying.
I remember going to a particularly paranoid seminar on security, and being told about a group of European missionaries who were exchanging pgp-encrypted emails in some African country, and the secret police asked them why they were doing this, which proved how important it is to encrypt your messages. I thought it proved what a dumb idea it is: anyone screening messages is going to pick out the encrypted ones as potentially interesting, and, even if they can't break the encryption, they can always break your door down, pull out your fingernails or just confiscate your computer.
Sending plain text emails, substituting sensitive words where necessary, strikes me as a far better way to go. Although most of my emails would send even the most conscientious spy to sleep...
Virtually serving coffee
Has anyone seen those P2P download buttons on some gaming sites ? They let you download large (100s of MBs) game demos from multiple locations, just like Kazaa and others. That's the best way. Somebody mark this as off-topic !
I connect to a lot of small servers that only allow ONE connexion per IP. They *always* complain if I have a view open in Netscape, AND try to fetch a file with Getright. And they'll continue to complain for up to 15 minutes after I switch NS to some other site. But they will let GR in instantly if I briefly close NS.
When I check with Netstat, it'll claim NS still has a connexion too, so it's not just the server's opinion.
~REZ~ #43301. Who'd fake being me anyway?
Of course, if you do limit the number of connections from a single IP, you may be limiting seperate clients who happen to be coming from the same proxy. This is particularly common with large ISP's, eg AOL.
<*cough*>Carnivore</*cough*>
HEY! Everybody seems all concerned that I'm looking at their stuff. I resent that! I'm not a bad guy, really. Why me?
Do you _honestly_ think that I care where you go on the web, dmayle? Well, I don't. Hmph.
I don't limit connections at all on any of the machines I maintain. However, I block badly behaving robots in a similar manner to those that block the download manager. Limiting connections by IP has other issues to consider such as multiple people browsing from the same NATd network etc.
;) Most people don't seem to care one way or the other.
Not all download managers are upfront about the issues of segmenting if I remember correctly, but it's nice that getright at least disables it by default.
It's also nice to hear that you make an effort to be polite to small servers
Nooooooo. This is for public access servers to download files. You can't just set up a tunnel for every Joe Downloader that comes along and wants the files. SSL/TLS will do the job just fine.
now we need to go OSS in diesel cars
It sounds like your objective is to make files available for download by the public. That being the case, your best solution is to provide both methods and let the person downloading the file determine which method is better for his/her/its needs.
Some will prefer ftp because it's faster. Others, especially those behind overly-restrictive firewalls, will find that http is a more usable alternative.
Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?
But then you wouldn't be showing off your mad-k00l h4xorz skillz. It's not about getting the job done it's about how eleet you do it.
I have a quite simple reason for why I like FTP over HTTP.
I do not have to sift through an endless stream of advertizing and links to get to my file. Its all structured and catagorized to what I want.
I am confused here, I stated that FTP works better thru a proxy, because the password login issue with http is confusing...password and ID to get out, password and ID to login to the site for download. You then say "90% of proxy systems out there don't support HTTP very well. Some don't at all." which seems like an agreement then you state "If you want maximum compatibility, even in the weirdest of network situations, HTTP is the way to go." which seems to directly contradict your previous statement. I work from behind a large corporate firewall, unix of course, and using IE and http downloading becomes very tricky when you have 2 id's needed for a download, especially when they are not the same. FTP out of the browser is even wierder when logins are needed at both ends.
errr....umm...*whooosh* *whoosh* Is this thing on ?
Oops. s/HTTP/FTP/ at the beginning of my post. Most proxies I've dealt with didn't deal with FTP well, not HTTP. Big typo there.
As to proxies: For non-authenticating proxies and access to public files, there are a total of 0 logins to get a file. FTP requires at the very least an anonymous loging. (Although most browsers handle this automagically.)
For authenticating proxies (Microsoft proxy servers using NTLM for example), the user usually only has to deal with authentication with the proxy once (And in most cases, they do so in order to even log in to their machine).
Aforementioned MS proxies don't do FTP very well. A company I previously worked for used to have a non-MS proxy that only supported HTTP. NOTHING ELSE.
retrorocket.o not found, launch anyway?
hm..like its that hard? seriously, it took no more than 15 minuttes getting working my SFTP server. Either you're clueless or just plain stupid, prolly the latter....
sFTP has nothing to do with ftp, appearances aside. sFTP doesn't need to be "set up", its been built in to every OpenSSH install I've ever done, because its an extension of ssh (so it took you 15 minutes how...?). What the original AC rant (of mine) was about was actually using FTP over ssh. sFTP is a better solution, and its what I personally use, but the (human) client insisted on being able to use their site production tool's builtin ftp client to update their site. Hence, a real ftpd was required. I wanted to protect their passwords from being sent cleartext, and the best way to do it while still maintaining compatibility with integrated ftp clients was to use ssh tunnels. Most of the users are on OS X where theres an openssh installed, and a the windows users can use putty's ptunnel (or install cygwin and openssh). Its possible this also could have been done with stunnel on both ends, though I think the same problems (ftp uses network and transport layer info like ip addresses and port numbers, at the application layer) still would have applied.
He created a cockamamie setup involving external daemons and strange rulesets and workarounds. This is the amateurish stuff that household cowboys (most of whom post to Slasdhdot, it seems) hack up and call "done".
My NAT'ed FTP servers have one or two firewall rules and a proper configuration and they work. Give me a 500MHz machine and I'll set up an FTP server (compiled from source) behind NAT inside of an hour. Properly.
Hi I'm the original poster your replying about. I didn't use "external daemons" (Where did you get that idea?). My setup also worked straightaway with a couple NAT rules and a proper configuration. But it didn't work with ssh tunnels, and I doubt yours would either, unless you've specifically configured it that way (with additional firewall rules). The ultimate setup was to have it *only* work over ssh, so as to not allow users to ever send their password in cleartext (so the nat gateway doesn't forward port 21, but it does forward the high range of ports the data connections from the client will need to come in on).
Judging from your comments, I'd guess you've never setup an ftpd behind nat for use with ssh tunnels. Your comments are the amateurish stuff that household cowboys post on slashdot to feel better about themselves. And I see you didn't have the sense to [X] Post Anonymously when spewing your drivel out over the internet, Stewart Honsberger. Do you feel better about yourself now? Please, try and connect to one of your ftp servers over an ssh tunnel before replying again, so you can see for yourself what I mean.
It's been done. I believe you're thinking of lftp. Fabulous little program.
"Patriotism is your conviction that this country is superior to all other countries because you were born in it." -- GBS
No, I don't hide behind cowardly anonymizers, thank you.
Already done. What's the problem?
BTW - if the thread was about serving public, anonymous files; why were you digressing and talking about SSH tunneling? Did you want to feel better about yourself and your meagre accomplishments, or did you just not understand the question?
Don't bother responding if you're going to hide behind Slashdot. I'm not that terribly interested in what you have to say.
BD Phone Home!
Shameless plug. Like you weren't expecting it.
The other day I was copying a large file (Mozilla 1.3b source, repacked as .bz2 to save space) from a remote machine. The machine I was copying it to locally crashed (likely due to bad power; LINE-R is mandated), so I just symlinked it into my remote ~/public_html/ directory and used wget to finish the job.
I still cringe every time I have to do wget -c; I have bad memories from extremely poor Win32 download restarters. Invariably the download would be corrupted. I have never had a problem with wget.
And I've noticed personally that FTP connections tend to be more stable and faster once they get going (although it does take longer to establish the connection). I don't know if this is because of something in the protocol, or just because it seems more FTP sites than HTTP sites implement a strict max-users limit, but I tend to prefer FTP for downloads when available.
Usually I just use Mozilla to find the link on the page that says "current version here" or whatever and then wget that.
TO BUY A NEW CAR WOULD MAKE YOU SEXUALLY ATTRACTIVE.
1) HTTP is great for downloads but can anyone tell me how to UPLOAD to an HTTP server?
2) Use Passive tranfers (eliminates NAT issues on the client side)
3) GET A REAL FIREWALL/ROUTER (a good firewall/router will handle the ftp protocol even with NAT)
That being said, I don't know about having multiple FTP sites behind a firewall...I don't know of any hardware firewall that can handle that, but in any case HTTP is still useless for uploads.
Cry Muppet! and let slip the observations of the-bleeding-obvious!
LART LART LART
For the benefit of you (and other semiliterate posters who can't grok something unless it's explained in small words) I think you'd benefit from reading this again:
Additionally the CPU overhead generated by FTP connections also causes many sites to limit the number of users who can connect, which often results in 'busy sessions', something much rarer with HTTP (as HTTP servers typically have very high thresholds for the number of concurrent connections they will support). The overhead on a server of a user downloading a file over FTP is much greater than that of a user downloading the same file over HTTP.
Once you actually understand how parse that, let's skip on a bit (I don't want to repeat my entire post) and read this:
This may be partly due to poor FTP server configuration defaults and/or poor administration, but they cannot shoulder all the blame.
So clearly I do understand the difference between the configuration of a given server and the protocol.
Now read this (and note the emphasis, if your having too much trouble understanding all of it at once):
Although FTP is of course theoretically more reliable than HTTP, in practice, because of 'Server busy: Too many users' messages combined with the speed and reliability of modern connections (which in turn makes HTTP more reliable) mean the reverse is often the case from a user perspective - which is what I think the poster is getting at.
Now did you not understand any of that? Logically I would not be able to give such commentary and in such level of detail without knowing the elementally basics which you've so inelegantly attempted to express.
Please try and grasp this basic point:
The most important thing you DON'T seem to be getting is:
The FTP protocol requires both more bandwidth and a notably more complex server than HTTP does.
Which means, ipso facto:
That allowing someone to download via FTP uses MORE bandwidth and MORE CPU time than using HTTP.
At a maintainable given speed, it is possible to serve more files on a server via HTTP that via FTP. Additionally the HTTP downloads will end slightly quicker (most notable with small downloads around the size the article poster indicated they would be serving) because of the lower overhead.
And finally (quote, NOT from me):
Just wait 'til you get an HTTP site that gets maxed on traffic - HTTP server too busy messages really suck... sometimes you don't even get the connection.
I've managed sites that take millions of hits a day and servers with tens of thousands of individual web sites.
It tells you what the contents of the data is because you were talking about web-traffic. If it's real web-traffic, encryption buys you nothing: the attacker gets a copy of all the files and can correlate how much data you download to the most likely places you're currentyl visiting. It doesn't matter that your browser has keep-alive. The spikes themselves are the important parts--timing is everything.
If you're interesting in finding out why simple encryption is basically meaningless in applications like you describe (and I'm not talking about SSH sessions here--I never said anything about SSH--I'm talking about the situation you originally outlined) then take a look at the implementation of the cypherpunk remailers.
The more advanced ones send out junk data at regular intervals and then once this pattern is established, they can replace the junk data with real encrypted data provided the actual traffic patterns remain the same.
But your scenario--encrypted web traffic--was pretty nebulous to begin with. Perhaps if you were a bit more detailed, we could avoid further (apparent) misunderstanding.
As for your SMTP remailer, that's still insecure. If you ever implemented a mixmaster remailer you'd know this.
Let's say the police one day come knocking at your door. Let's say they've been monitoring your "secret" smtp server for a while and have recorded every byte of (encrypted) data along with precise timestamps that's ever gone through your server. The encrypted data itself might be meaningless--but consider what happens when they raid you and your friends' computers. Now they have data they can correlate.
And how do you know what will stand up in court? Are you a lawyer? Do you have case law experience? If they're monitoring the channel, they have your IP address. What makes you think you'll be "lost in the noise" of all the other transfers and the various control data passed back and forth? Sure it's not a guarantee of what you're transferring, but even if you do directory listings--that can (surprisingly) narrow things down to the point where it becomes obvious what you are downloading: especially if you do it by hand.
After all, in that case even just the timing of the control data can be a dead giveaway.
You're handing off tonnes of information for an attacker to base conclusions on, and your idea of encrypting such communications isn't as realistic as you think.
Two good books for you to read through are "Applied Cryptography" by Bruce Schneier, and "Cryptonomicon", by Neal Stephenson. The latter is dead boring until the end, but it does go through the motions with almost clinical attention to the concept of information warfare.
anyway, if your running a *nix variant try pure-ftpd, it is secure, flexible and easy to use.
---- Put Sig here:
You, sir, should post more often with your insightful and informative comments. I miss the comments from you, evil_spork, and all the other sporks who never post anymore. Have a good day, sir.