Slashdot Mirror


Using Single Apache SSL/Non-SSL in Production?

tck1000 asks: "I currently maintain some legacy webservers, running Apache 1.3.x on Linux, on x86 hardware. Two separate daemons are used. One to serve SSL vhosts, and one to server non-SSL vhosts. Each of these servers also is compiled with PHP, mod_perl, and JServe, and also works with a Tomcat servlet engines. In the process of planning an upgrade path, I've thought about using a single daemon to serve both the SSL and Non-SSL vhosts. Is this a good idea?"

"These webservers serve about 4 million hits a day across all the vhosts. I'm worried about memory usage if every httpd process has to load mod_ssl, as well as everything else they load.

I've been searching for comparisons between running 2 daemons (and the associated effort in maintaining/upgrading/patching), vs. running a single daemon (with any added overhead it entails).

I've found a lot of examples of how to do it, but not much on the why's.

Comments, Opinions, Ideas, Links?"

37 comments

  1. Slashdot/Sourceforge/etc... by krow · · Score: 4, Informative

    This is what Slashdot does (and sourceforge too for that matter), they use one Apache host to serve content and use pound to do the SSL.
    Good luck!

    --
    You can't grep a dead tree.
    1. Re:Slashdot/Sourceforge/etc... by Graelin · · Score: 2, Interesting

      Pound is awesome but it has a few very serious problems:

      1) Memory leaks - in our environment pound leaked memory like a siv. I had a cron job that restarted it daily it was so bad. In Pound's defense, this may have been a leak in the openssl libs.

      2) Compatibility - Older versions of pound had very serious problems with some browsers. We had many clients that should not access the secure website. You had to upgrade to the pound-current tarball to fix it, which probably introduced the memory leak mentioned above. :)

      3) It is slow - I hadn't noticed how slow it was until we replaced it.

      4) Pound does not scale well when used as a load balancer. We couldn't get it to handle more than 100 concurrent connections. This may have been a hardware limitation (was not a dual proc, had only 1 gig of ram.)

      We replaced pound with an Intel SSL accelerator and haven't looked back. It rocks.

  2. No, it doesn't by mirabilos · · Score: 2, Insightful

    Honestly, this is the first time I've heard that
    someone _does_ use two different servers for
    http and https.
    If you really want to increase security, use the
    new chroot facilities.

    --
    My Karma isn't excellent, damn it! (And /. still does not get UTF-8 right in 2012. Wow.)
    1. Re:No, it doesn't by sylencer · · Score: 3, Insightful
      If you really want to increase security, use the new chroot facilities.

      I don't understand what you are saying here. People use https to prevent others from sniffing their traffic, e.g. for credit card numbers or other data that should be kept secret, like passwords. Chroot environments are used for a completely different purpose: To keep the impact on your whole system as little as possible when (not if!) a security flaw in the daemon is discovered and thus an attacker can execute arbitrary code on your machine.

    2. Re:No, it doesn't by mirabilos · · Score: 1

      I was more thinking about securing the daemon, not
      the content exchanged.

      Another one: use propolice (the stack protector)
      if you can.
      http://www.research.ibm.com/trl/projects/security/ ssp/

      --
      My Karma isn't excellent, damn it! (And /. still does not get UTF-8 right in 2012. Wow.)
    3. Re:No, it doesn't by stevey · · Score: 1

      SSP is good, but not commonly available in distributions.

      I've made packages for Debian stable/unstable which are available from my security pages.

      More feedback is always appreciated - as it stands I use them, but I've no idea about others!

    4. Re:No, it doesn't by mirabilos · · Score: 1

      Right; on the other hand, it still has issues,
      because of which some programmes (mozilla, wineX)
      still are compiled with -fno-stack-protector.

      In OpenBSD since release 3.3, propolice is enabled,
      as in MirBSD (http://mirbsd.de/). The latest MirBSD
      development snapshot, available in 1-2 days under
      https://mirbsd.bsdadvocacy.org:8890/current /
      (while writing this, I'm building it), has gcc 3
      with propolice, W^X (ie, memory pages are either
      writable or executable), NXSTACK and NXHEAP
      protection mechanisms, and the default apache,
      suitable for HTTP and HTTPS, comes chrooted.
      Most of that is thanks to the efforts of the
      OpenBSD guys, but in contrast to them, our apache
      httpd also has support for IPv6!

      shameless plug: we'd be glad if people tested it.

      -

      On the propolice page I referenced above, there
      are instructions on how to build FreeBSD and
      GNU/Linux with this protection - if you like
      building from source. This is especially inte-
      resting for servers - many exploits are hindered
      to work at all, or at least just crash the server
      (DoS) instead of function (remote hole).

      --
      My Karma isn't excellent, damn it! (And /. still does not get UTF-8 right in 2012. Wow.)
  3. The Concern is... by Fubar420 · · Score: 1

    Heres the bottom line. SSL is used to secure things that you dont want people to see, CC#'s, etc.

    If your SSL machine is the same as your www host, then if the www host (a more likely target for random attacks) is compromised, the SSL is worthless, since they can replace your cert, access protected data etc, under the same permissions of the www daemon.

    SO, if your SSL daemon is handling data that sensibly should not be on the most obvious target for first attack, then no, its not a good idea.

    If on the other hand, your ssl daemon is simply mirroring the plaintext in an SSL form, then for all intents and purposes, it doesn't matter

    --
    -- (appended to the end of comments you post, 120 chars)
    1. Re:The Concern is... by Covener · · Score: 1

      If your SSL machine is the same as your www host, then if the www host (a more likely target for random attacks) is compromised, the SSL is worthless, since they can replace your cert, access protected data etc, under the same permissions of the www daemon.

      Replacing the certificate? Wouldn't just about every browser throw a fit at that? The SSL isn't being used to protect any kind of persistent data on the filesystem.

  4. Link by jpkunst · · Score: 4, Informative

    Comments, Opinions, Ideas, Links?

    Recipe 7.4: Serving a Portion of Your Site via SSL from O'Reilly's Apache Cookbook ?

    JP

  5. Might consider more than that... by iwadasn · · Score: 2, Interesting

    First of all, I am a bit of a web admin. I write software all day, run my own website on my machine, and many of my past jobs were writing asp (shudder) and cgi of various sorts.

    Basically I've always found that people get into trouble when they run webservers with excessive complexity. On a modern webserver, serving just about anything, the actual performance of the webserver software doesn't mean much, all the time is taken with DB interaction and custom code (cgi, jsp, etc..). I would therefore suggest that you try to simplify a bit.

    I've tried jboss and tomcat and don't have any complaints. They work very well, and (at least from what I've heard) are very secure. No buffer overflows to be found, for obvious reasons. In addition, the portabiliity is nice.

    Because of this you might consider trying to run tomcat without apache. The SSL in tomcat works fine, and it's easy enough to set it up so that it will serve both secure and non secure pages. This simplifies everything by placing the entire webserver (except for the cgi) in a single process. I've never really tried the cgi through tomcat, but I'm told it works so you might want to try it at some point. Not sure about PHP, and of course do a lot of testing and investigation before rolling this out into production (as if that even needed to be said).

    If this works for you, you've eliminated all the OS/architecturre dependencies at once. In addition it'll help by cutting out one piece of software (apache), and might help security.

    1. Re:Might consider more than that... by Homology · · Score: 3, Informative

      With four million hits a day it does makes little sense to use Tomcat even for static content. Apache serves static pages faster than Tomcat and with less resources.

    2. Re:Might consider more than that... by lewp · · Score: 1

      On the one hand you're right: When generating a dynamic page your code and database accesses are going to take the vast majority of the time.

      At the same time, how many images does an average page of yours use? How many stylesheets? How many external JavaScript files? Your code may only be running on 5% of the requests to your server.

      Static content simply needs to be blasted out to the user as quickly as possible. There's not much sense in using a 12MB Apache process loaded down with mod_perl, PHP, and god knows what else to do this. Similarly I wouldn't be wasting a precious Tomcat thread on this kind of grunt work.

      If you're going to be running a small site for you, your friends, or maybe even a small community, then using one server to do all your work is fine. If you ever want to run a big site, though, you need to make sure you're using the right tool for each job.

      --
      Game... blouses.
    3. Re:Might consider more than that... by iwadasn · · Score: 1

      Good point. It probably makes a lot of sense to have a separate apache server to blast out images and style sheets.

      I was, however, under the impression that the vast majority of the cost of a website is paying for bandwidth, so it doesn't make a whole lot of difference (cost wise) if the webserver runs half as fast. In fact, if you can save a $80,000/year admin by consolidating all your servers into ten identical boxes each serving everything, then perhaps it's cost effective, even if you have to pony up an additional $40,000 for hardware up front.

      Anyway, it's surely not the answer for everyone, but probably something to consider. At work we're currently in the opposite end of the regime (and at my last job too). We're moving code from VB to C# and Java, so we're consolidating all of the disparate systems running god only knows what custom hack onto a few boxes running java and C#. We save both ways, more standard config means less to admin, more efficiency means less to pay for. If we were moving from native to Java the equation might be a little different, but it might still be worth it.

      I do hear that tomcat is 1/5 the speed of apache serving static content, so if you're overloading your hardware already and static content is a big part of the mix, it's not for you.

    4. Re:Might consider more than that... by lewp · · Score: 1

      Well, hardware is cheap compared to people, that's for damn sure. At the same time, splitting your servers into static and dynamic servers is almost trivial.

      Half the servers run one httpd.conf, half the servers run another. Or half the servers run Apache, half the servers run your favorite app server. J2EE servers are enough of a pain in the ass to deal with that I'd rather get rid of half of them for simple Apache/thttpd/publicfile/mathopd/etc static content servers anyway, even if it is another piece of software to manage.

      If your admin can't do this without extra staff, he's either lying or he needs to be fired :).

      The homogeneous configuration idea is a good one; get a lot of identical hardware and load it all with the same OS if you can get away with it. But while it means that all your boxes are the same, if you take it too far it also means that a lot of your boxes are more complex than they have to be.

      --
      Game... blouses.
    5. Re:Might consider more than that... by Tet · · Score: 1
      I've tried jboss and tomcat and don't have any complaints. They work very well, and (at least from what I've heard) are very secure.

      YMMV, but we've found exactly the opposite. We've had 3 separate security problems through using tomcat, two of which caused "session leakage", i.e., displaying one customer's session information to another. As a finacial services site, we just can't afford that sort of exposure. Yes, it only shows up under high load, but the 4 million or so hits a day that we get is enough to provoke it. Consequently we're moving to Resin (rather than ditching servlets altogether, which would be my preferred option, but whatever).

      Because of this you might consider trying to run tomcat without apache.

      Tried it. It's slow and inflexible. Stick with Apache.

      --
      "The invisible and the non-existent look very much alike." -- Delos B. McKown
    6. Re:Might consider more than that... by iwadasn · · Score: 1

      Out of curiosity, did you report/fix the bugs?

    7. Re:Might consider more than that... by Tet · · Score: 1
      Out of curiosity, did you report/fix the bugs?

      Yes. Two had already been fixed by others (one was in a newer release, one wasn't but had a separate patch available). The third we fixed ourselves.

      --
      "The invisible and the non-existent look very much alike." -- Delos B. McKown
  6. A very simple question with a complex answer? by shfted! · · Score: 1

    Why?

    --
    He who laughs last is stuck in a time dilation bubble.
    1. Re:A very simple question with a complex answer? by krow · · Score: 1

      Each additional virtual host requires additional memory (especially with Slash where memory is tied to the virtual host directly). So, you want as few virtual hosts as possible.

      --
      You can't grep a dead tree.
  7. At this point... by SanityInAnarchy · · Score: 1

    You have to do quite a bit to isolate them. You must at least prevent against simple local-user DOS like a forkbomb (remember that it is a C one-liner), and you must isolate the apache users. If you access any of the same content via SSL as you do with straigh HTTP, you have to put them in the same group, and be very paranoid about other things.

    SSL itself isn't secure enough for me -- I have to trust VeriSign. So there are better ways of storing really sensitive information.

    At this point, we are mainly guarding against local attacks. If the software does that for me in an unobtrusive way (dropping permissions, chroot, etc.), then that's ok. But I can't be bothered to patch my kernel and such against every possible scenario, and no one should -- you find the sweet spot between security and convenience.

    And it's damn inconvenient to have to buy more memory because you _might_ have your HTTP apache server hacked before your SSL one is.

    --
    Don't thank God, thank a doctor!
    1. Re:At this point... by petard · · Score: 1
      SSL itself isn't secure enough for me -- I have to trust VeriSign. So there are better ways of storing really sensitive information.
      This is wrong on two levels:
      1. Verisign has nothing to do with the security of data transmitted using SSL, and the only thing you could ever trust them for is vetting the identity of the people whose certificate requests they sign. They never control your private keys, and you can operate an SSL site that has nothing whatsoever to do with verisign. Just set up your own CA.
      2. SSL is not used for "storing really sensitive information". You use SSL to encrypt data in transit, not for storage.
      --
      .sig: file not found
    2. Re:At this point... by SanityInAnarchy · · Score: 1

      #1: Verisign can fake that, though. They can create their own private key and replace yours with it temporarily. Either way, I have to trust them quite a lot.

      As for setting up my own CA, there's the problem of people having to trust me initially. I actually prefer that to verisign (it's free), but there could be a better solution in general -- like an encrypted network filesystem.

      #2: You're right, of course -- my bad, it is transit. And that is what this person was suggesting: "SSL is used to secure things that you dont want people to see, CC#'s, etc." I would _not_ trust that to SSL with verisign. I might trust it to SSL with my own CA, on a network filesystem as above, but at what point do you draw the line between 'ssl' and 'not ssl'? ssh uses libssl, yet people rarely call it "ssl rsh" or "ssl telnet" or something like that -- telnet over ssl is an entirely different animal.

      --
      Don't thank God, thank a doctor!
    3. Re:At this point... by petard · · Score: 1

      #1: Verisign can fake that, though. They can create their own private key and replace yours with it temporarily. Either way, I have to trust them quite a lot.

      This is only a trust issue for SSL clients, not for an ssl server.


      at what point do you draw the line between 'ssl' and 'not ssl'?

      SSL is a specific protocol, not a library. It has been standardized by the IETF in RFC 2246. If the protocol described in that document is implemented, it may properly be described as SSL or TLS. SSH, of course, does not implement that protocol although many of its crypto operations are the same.

      --
      .sig: file not found
    4. Re:At this point... by SanityInAnarchy · · Score: 1

      True, they can't break my server just because I use SSL. But consider this:

      1.) I put critical documents on an SSL site.
      2.) VeriSign swaps out my private key for their own.
      3.) User requests secure page from my site, giving username and password.
      4.) VeriSign intercepts the request, proxies it to my server (a man-in-the-middle attack), and keeps a copy of the file downloaded.
      5.) VeriSign now has a full copy of whatever file the user was downloading.

      So if I care at all about the clients, or if I am in fact trying to distribute certain files securely, then yes, I personally have to trust VeriSign more than I currently trust my little brother, who is more mature than most corporations I know.

      --
      Don't thank God, thank a doctor!
  8. You're Doing Fine by osewa77 · · Score: 1

    If you don't gain any additional benefit from using two apache instances, and you have a hang on the configuration, then what you currently have is just perfect. The difference is not significant on a server with lots of memory (512mb+) as is typically used in web hosting. As usual, more specific information about your servers would have been helpful!

  9. Some advice... by Graelin · · Score: 1

    First, the biggest problem with having so many virtual hosts is the file handle utilization. This is mostly caused by having seperate logs for each host (a custom access log and an error log.) After 300 hosts you'll start adding ulimit lines to your apache startup scripts. And that's ugly. You can prevent this one by defining no log entries on your vhost configs and using a global config. Write a script to parse out the logs for each domain at the end of the day or whatever.

    Have you considered an SSL accelerator? They really do work. All HTTPS requests are handled on that box making the requests themselves normal HTTP on your webserver. No need for SSL support on your web server AT ALL. I cannot tell you how handy these things are. I'm sure you can find one that can fill your needs. Ours has handled 500 concurrent connections before.

    Another thing you might consider, although I'm not sure how it would work in your situation, do not define any VirtualHost entries at all. A global mod_perl URL translator handler could do all of that work for you.

    Let's say, for example, that you host www.example.com. Your URI Translator will look at the requested domain name and set the requested filename to /home/users/example.com/www.example.com/html (or whatever you're using.)

    If that works for you, you'll never have to restart apache again. Instead, just adding users to the directory structure will make the websites available.

    1. Re:Some advice... by Peter+Cooper · · Score: 2, Interesting

      Regarding your latter idea, wouldn't that screw things up when it comes to running CGI scripts, and the like? After all, every single thing would be running as the apache process, rather than in a chrooted environment, as many secure systems do. I may, of course, be missing the point.

    2. Re:Some advice... by Graelin · · Score: 1

      Actually, it wouldn't. The URL Translator only maps URLs to system filenames. Basically, it just tells Apache where the CGI is and does not impact how Apache will execute the CGI. The script will still execute in a seperate process.

  10. wow by man_ls · · Score: 2, Interesting

    You sir, are in the exact same situation I was in just two days ago.

    My conclusion:

    Moving to one process to serve all requests, SSL and non-SSL, is more of a hassal than it's worth.

    SSL and Non-SSL on one process requires IP-based vhosting. If you're using vhosts you probably don't have multiple IPs per server. Thus you must use name-based vhosts, and SSL gets confused.

    I gave up and installed two processes and am very happy with that solution.

    Apache+OpenSSL 2.0.48/Win32 on Windows Server 2003.

    1. Re:wow by dave420 · · Score: 1

      SLL and non-SSL on one process doesn't require IP-based vhosting - you can quite happily use port-based vhosting. *:80 for http, and *:443 for SSL...

    2. Re:wow by docubot · · Score: 1

      This is not true. Each individual SSL hostname must have it's own IP address. However it is possible to run HTTP and HTTPS on the same IP address. As a matter of fact, I have about 1400 HTTP vhosts and 7 HTTPS vhosts running in one Apache process on 7 IP adresses.

      Yes, it is possible. Yes, I highly recommend it.

      Relevant parts:

      ### Section 2: 'Main' server configuration
      Port 80
      ## SSL Support
      <IfDefine SSL>
      Listen 80
      Listen 443
      </IfDefine>

      ### Section 3: Virtual Hosts
      NameVirtualHost 192.168.0.2
      NameVirtualHost 192.168.0.3

      <VirtualHost 192.168.0.2:80>
      ServerAdmin webmaster@domain.com
      DocumentRoot "/home/www/htdocs/path/to/domain.com"
      ServerName abc123printing.com
      ServerAlias abc123printing.com www.abc123printing.com
      </VirtualHost>

      <VirtualHost 192.168.0.3:443>
      DocumentRoot "/home/www/htdocs/path/to/domain.com"
      ServerName secure.yourdomain.com
      ErrorLog /weblogs/error_log
      SSLEngine on
      SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSL v2:+EXP:+eNULL
      SSLCertificateFile /path/to/certs/secure.yourdomain.com.crt
      SSLCerti ficateKeyFile /path/to/keys/secure.yourdomain.com.key
      <Files ~ "\.(cgi|shtml|phtml|php3?)$">
      hSSLOptions +StdEnvVars
      </Files>
      <Directory "/home/www/cgi-bin">
      SSLOptions +StdEnvVars
      </Directory>
      SetEnvIf User-Agent ".*MSIE.*" \
      nokeepalive ssl-unclean-shutdown \
      downgrade-1.0 force-response-1.0
      CustomLog /weblogs/ssl/ssl_access_log sslcombined
      </VirtualHost>
      </IfDefine>

      <VirtualHost 192.168.0.3:80>
      ServerName secure.yourdomain.com
      RedirectMatch (.*)$ https://secure.yourdomain.com$1
      </VirtualHost>

  11. idiot by Anonymous Coward · · Score: 0
    Help stamp out slashdot trolls--send $1 to GNAA PO Box 69 Key West, FL 32269-069

    I sent a dollar to that address and it came back undeliverable. Jerk.

  12. Not an Apache specific answer... by jea6 · · Score: 1

    One reason to keep these running separately is that it becomes easier to dedicate hardware specifically to the SSL portions of your hosting infrastructure, should you choose to go that route.

    Another consideration is whether Apache limits your use of a cert:key pair to each instance. We have always used (Netscape Enterprise|iPlanet Enterprise|Sun ONE). Earlier versions would limit you to 1 instance per port listened OR 1 instance per SSL cert. Eventually, we had up to 30 instances of the httpd running because there were 30 different SSL certs.

    Good luck.

    (There should be an 'Ask Slashdot: Revisited' where people follow-up on any suggestions received from 'Ask Slashdot'.)

    --

    sarchasm: The gulf between the author of sarcastic wit and the person who doesn't get it.
  13. Actually it's less overhead... by mengel · · Score: 1
    You can do multiple-port and/or multiple IP based virtual hosts just fine with one apache instance and virtual servers. We're doing about 70 at my site.

    The only really ugly case is trying to do named virtual host SSL; which would work fine if you could get the browsers to agree on how they verify certificates:

    • For IE, put multiple CN entries in your certificate.
    • For older Netscapes, make the *first* CN be a wildcard matching the respective hosts
    • For newer Netscape/Mozilla builds, who knows?

    But if you're already running the SSL servers under apache, you just move those vhosts into your other configuration and just run one server. You may have to specify the SSL options in each <VirtualHost> section, rather than configuration -wide though. You just have fewer total httpd's running, and that will be less overhead.

    --
    - "History shows again and again how nature points out the folly of men" -- Blue Oyster Cult, 'Godzilla'
  14. Buy SSL hardware! by Anonymous Coward · · Score: 0

    SSL takes up, on average, about 80% of the CPU on systems that process SSL themselves. This means your CPU is spending more time on encryption than actual requests.

    Just go buy some SSL hardware. You can get SSL network cards fairly cheap, or you can spring for the external equipment. These free up the CPU so they can do actual work.

    $.02