Slashdot Mirror
Sites Rejecting Apache 2?
An anonymous reader writes "Vnunet
reports on the low adoption of Apache 2 has caused its producers to advocate freezing development of the open-source Web server until makers of add-in software catch up. Almost six months after the launch of Apache 2, less than one percent of sites use it, due to a lack of suitable third-party modules." I'm not sure where they are getting the freezing Apache development part, more talk about forking for 2.1 right now on the httpd mailing list. The article does have it right though that until there is a reason to upgrade and the modules are in place that adoption is not going to happen. While the cores of both Perl and PHP are thread-safe, the third-party modules are not. This renders one the larger reasons to use Apache 2.0, the threaded http support, useless for applications using either of these application layers. It comes down to the question of whether the third-party module writers are better off supporting what is used or what is new.
Like MySQL, GD, ImageMagick, etc.
In PHP at least, they are a very important part of site writing.
I respond to your sigs
PHP, mod_perl, any of the Java servlet modules. These are are third-party (basically the server doesn't ship with them even if they are other ASF projects). Anyone running anything other then a flat HTML site needs at least one of these or something similar.
You can't grep a dead tree.
The build system in Apache 2, while being vastly improved over the Apache 1 build system, is rather complicated, and has lead to a number of packagers simply not bothering, or having a hell of a time packaging it.
There is no RedHat or Debian packages of Apache 2.0 (offical as in from RedHat or Debian, and part of their stable distribution). There are a few Debian people who are packaging Apache 2.0 (namely Thom May, who is the current package bunny...err...maintainer *grin*), but last I heard they were having a horrible time getting it working, and it's still only in unstable (sid), and hasn't made it to testing (sarge).
If it gets into RedHat and Debian's stable distributions, chances are it'll make a higher percentage mark on site usage. Till then, I don't think things are going to change much.
Isn't mod_deflate similar in function to mod_gzip? I have not tried it yet, but it seems to play the same role.
PHP support seems to be somewhat stable on apache2 using the prefork mpm. The threaded mpm's don't work on FreeBSD, so I didn't really have a choice.
The preformance seems to be pretty good after I removed the unneeded modules. --Matt
when distros will start shipping 2.0 as standard,
everyone will "just use" it. Of course there would
be some rejection rate, of stubborn people. 1.3
development would stop and everyone would slowly roll over to 2.0.
pro 2.0:
- threaded stuff is blindingly fast. most systems threads are faster then processes
- other new technologies, like layered content filtering are great for developers of hight traffic sites.
pro 1.3:
very very many people using apache use linux. Linux threads are almost same performance as processes. Due to kernel limitation, you can stack only so many threads per process.Plus threaded model does not account for stability. One NULL pointer dereference and you're gone. Apache2.0 of course uses bundles of threads. so you still have multiprocess model kicking around.
Expect 2.0 gain popularity on systems like Sun, BSD and Win32 where processes handling is relatively expesive. Threads are dirt cheap.
As everything, things take time. Just like well brewed beer.
cheers.
Because it's multi-threaded. There are a bunch of strings attached when you thread stuff. For example, thread children all operate in the same memory space (as opposed to the pre-forking Apache 1.x, where each child process had it's own memory space)... that alone has a HUGE impact on how modules must be coded. In order to maintain backwards compatibility, a hybrid pre-fork / thread server setup would have to be constructed.
On a side note, I'd have to disagree with the CTO of Aperio Technologies, Solaris also gets a serious performance improvement with Apache 2, albeit not as good as Windows, but still decent.
<Amanda`> I just went out to the parking lot in my bathrobe to exchange warez CDs.
Apache 2 is in Mandrake contribs (not really supported nor officially maintained), so if you buy the 9.0 ProSuite, it will be available. I am hearing talk from Mandrake that Apache 2 will be the default web server in Mandrake 9.1.
The RH beta includes Apache 2.0 by default. Expect market share to rise when the new RH ships.
OK, OK, I'm waiting for our 3rd party ad serving software vendor to stabilize their module for our HTML servers too. But, for static (read: gifs and jpgs) we use V2 in threaded mode. Its VERY stable, VERY fast. We do 70-100MB sustained with 4 dual P3 graphics/static servers backing our HTML boxes, each taking 3000 connections a pop. Just give it a little time for adoption, this is great code! Great work, Ryan and crew!
Apache 1.x has a big problem when it comes to dynamic/updating data in shared hosting environments: security, or lack thereof.
All php, mod_perl, (and pretty much anything except suexec cgi) based pages are run as the same uid/gid as the apache server. Everything your scripts have read/write access to, so does everyone else on the same machine.
So, for instance, if your database passwords are in a php script, or a file that a your php script reads, the webserver must have read access to that data in order for it to work. Since everyone else's scripts also run with the webserver uid/gid, they also have read access to your database username/password info, and can therefore connect to your database, and do all the damage they want.
To address this problem, Apache 2 has the perchild MPM which allows a virtual host to have it's own process fork, uid/gid, and thread pool. Unfortunately, the perchild MPM is not presently stable.
With that being unstable, and php and mod_perl also being "experimental", Apache 2 doesn't really offer an advantage over 1.3 yet. ...But don't be so certain that Apache 1.x "ain't broken".
Let's clear up a few things. Yes, PHP support has been somewhat slow in coming, but the main reason is that there is very little motivation for us to rush to support it. This is because most of us really don't see the advantage of 2.0 yet. The threaded mpms don't work at all on FreeBSD due to bugs in the FreeBSD kernel threading code. These are fixed in FreeBSD's CVS, but are not in any released version as far as I know. Also, as was mentioned, PHP itself is threadsafe, for the parts that count anyway, but what about the 100-150 different libraries that PHP can link against? We know some of these are not threadsafe. We also think we know that a number of them are threadsafe. The rest, who knows. Do you want to be the first to discover that a certain library is not threadsafe? Thread safety issues don't tend to show up until you start banging at the server with production-level load. And the errors can be quite subtle and random in nature. These are not PHP libraries we are talking about. These are things like libgd, freetype, libc, libm, libcrypt, libnsf.
Of course, if you run the non-threaded pre-fork mpm, it should be ok. But really, what is the point then? That's why PHP support has been slow going. We develop stuff because we need it ourselves for something. Right now spending a lot of energy on supporting Apache 2 seems somewhat futile. What we need here is a concentrated effort on the part of many different projects to pool their knowledge and generally improve the thread safetyness of all common libraries. I have written a summary and started this work here:
Thread Safety Issues
I would very much appreciate comments and additions to this. I don't think Apache 2.0 is dead in the water, it just needs better overall infrastructure in terms of non-buggy kernels and a push to make all libraries threadsafe before it can really become a viable solution for sites needing dynamic content.
Or, alternatively, we might start pushing the FastCGI architecture more to separate the Apache process-model from the PHP one.
You are quite offbase here. The API change is a minor thing. It's the process model and the fact that everything you link into Apache now has to be threadsafe. Even if the API was perfectly backward compatible you wouldn't suddenly have rock-solid support for any old Apache 1.3 modules because the process model is completely different now.
Red Hat's (null) 8.0 beta 3 has 2.0.40. You can probably take the SRPM for it and rebuild it on RH 7.x. I haven't tried it but it should work.
I agree it will get a LOT more use once the Linux and BSD distros start shipping it by default, and once PHP and mod_perl are solidified for it. The Red Hat beta includes both, so they should be about ready.
The most powerful features of Apache based sites aren't features of Apache but of 3rd party modules. PHP, mod_perl, mod_dav, mod_throttle and even Microsoft Frontpage modules contribute significantly to the appeal of apache. There is an excellant Report on Apache Module Popularity by SecuritySpace.com. In considering this report, you should notice the month over month growth in the usage of modulees which have not yet been ported to Apache 2. The developers of these modules will most likely respond to customer demands for support of apache 2, which is dependant of the Apache Software Foundation's ability to convince customers of the benefits of upgrading to Apache 2. In this respect the marketing of Open Source Software mimics the marketing of treditional commercial software. Let's hope they don't adome the strategy of some commercial software vendors by simply refusing to provide security fixes or updates to Apache 1.3.x when needed.This would certainly outrage Apache users, but in the case of Open Source would have the secondary effect of promoting forking of the codebase. On the bright side customers do have a recourse in the case of Open Source, where, they're left twisting in the wind in the case of commercial products.
--CTH
--Got Lists? | Top 95 Star Wars Line
You incriment the left-most number in the release number. So 1.3 is not expected to be compatible with 2.0, and Linux kernel 2.4 is not expected to maintain backward compatibility with 1.0 ;) This makes things much easier to maintain and see at a glance.
Now as to why they did it, Apache 1.3 is great. I love it, but it is not as cross-platform as it pretends to be (it does not perform well on Windows) and it really is not built for speed. If you need these things, you need multithreading, a better abstraction model so you are not assumign POSIX compatibility (and hence emulating it on Windows) etc. This means you break the compatibility. Pure and simple, but in the end, you get a better product.
Think of Apache 2.0 as Apache-- Next Gen. Not yet supported but when it does, it will be more competitive than 1.3.x because it has a better architecture.
LedgerSMB: Open source Accounting/ERP
You are better off using the open_basedir restriction instead of safe_mode for this. Set the open_basedir for each virtual host to that virtual hosts DocumentRoot and then PHP scripts will only be able to open files under that dir.
Of course, both open_basedir and safe_mode are crappy solutions to a problem that needs to be solved higher up. Like with the Apache2 perchild MPM, but that is a long way from being production quality on a couple of different levels.
Not to put salt in open wounds, but in IIS, which uses threads, they use a concept build in Windows: apartments. You have single threaded apartments (STA) and multi-threaded apartments (MTA). The webserver itself uses threads for handling requests and when a certain library is called/opened by the code, that library takes care of in which apartmentstyle the code is ran: in an STA or in an MTA. VB6 com objects f.e. can't run in an MTA, so they are run in an STA. This is controlled by windows (as a configparam of the com object). So here you see a combination of both worlds: multi-threaded and safe where it has to be, without the hassle of forcing the developer to write threadsafe code when the code itself isn't multi-threaded, but the environment is.
Of course, there are some issues: when you let the code executed by the request of user A create an object in an STA and move that into a container which can hold both STA's and MTA's, and let code executed by the request of user B access that user A's STA object, you get thread unsafety and possible crap.
However: the OS's functionality offers the option to do it threadsafe and still have multi-threading in full effect. Perhaps a thing to look at for the thread/process guys in the Linux kernel team.
(It has been a long time, but afaik, a simple fork() is not forking off a complete new process, but a childprocess which runs as a thread inside the mother process, or am I mistaken? (if not: why then the threadsafetly crap NOW, because a fork() will result in the same issues)
Never underestimate the relief of true separation of Religion and State.
Actually it is the other way around. Linux has the smallest process creation and process switching overhead of any Unix with virtual memory. It is simply not possible for threads to be all that much faster than that. Apache 2 is optimizing something that simply was not all that expensive on Linux in the first place.
Finally! A year of moderation! Ready for 2019?
cLive
-- Trinity in high heels carrying a whip: The donimatrix - there is no spoonerism
Why not just use MaxRequestsPerChild?
This way you can knock off each Apache child one by one after a given period of use without having to restart Apache completely.
Top Most Bizarre/Disturbing Error Messages
FreeBSD's current threading is implemented in user space, although work is under way to move it into the kernel, that works is being done *ONLY* for SMP scaling and quantum utilization efficiencies.
As it stands, it is fully compliant with the POSIX threads standard.
If it is not working for Apache, it is because Apache is not a POSIX compliant threads client implementation.
From looking at the code, we can see this is the case, with the Apache code having an assumption of kernel threads, which you are not permitted by the POSIX standard to assume.
Although I have not yet verified it, an examination of the code *seems* to indicate that it has "the Netscape problem", which is an assumption about scheduling coming back to a given thread in a group of threads after involuntary preemption by the kernel when the process quantum has expired.
In older versions of Netscape, this displayed as a bug in the Java GIF rendering code, which was not thread reentrant, in that if you used a Java application as a web UI, and moved the mouse before all the pictures were loaded, the browser would crash. After I explained this, Netscape corrected their assumption, and the problem went away.
Ignorance of the requirements for writing threaded applications which will work on all POSIX compliant threads implementations is no excuse, nor is it a valid reason for blaming the host OS, unless you make it known what your requirements are, above and beyond the standard contract offered by POSIX, and that you are stricter than an application written to the POSIX interface, without such additional assumptions.
You will find that you have these same problems on MacOS 9 (NOT FreeBSD-derived), MaxOS X (uses Mach threads), Mach, Plan 9, VxWorks, OpenVMS, etc..
You will find you do NOT have these problems on systems with implied contracts above and beyond those provided by the POSIX standard: Solaris, UnixWare, Windows, and Linux. You may have *other* problems in Windows, related to implied contracts over virtual address space issues (see other posting).
-- Terry
The build process has been slowed down and, IMO, gone entirely broken. Previously I ran the configure script, which took a minute or so, compiled and installed. It worked.
Now a run a monstruous ./configure, which calls itself recursively and takes about ten minutes to complete, at which time any and all warnings have scrolled well past the top of the window. It does not report easy mistakes such as trying to make "so" a shared module until it is almost finished. And the libraries are not linked against the modules properly, so attempting to use a static libssl or libm is not possible.
An upgrade from 1.3.x to 1.3.x+1 took about half an hour. An upgrade from 1.3.x to 2.0.x has taken me the better part of two days, including reinstalling openssl shared so that mod_ssl works at all, for no immediate gain.
I can understand that people do not make the switch.
Threading a server can significantly increase performance. That is why many if not all commercial web servers are threaded (including IPlanet/NES and IIS).
Threaded programming is more difficult than non-threaded programming (just like mod-perl programming is more difficult that plain perl programming). Usually, it is because globals are used. Web servers are typically easier to thread (because each transaction doesn't usually interfere with others).
A single threaded server takes one request at a time, processes it, and then takes another request. The way Apache got around this was to have multiple processes, each which could take requests.
The problem is one of scale. While it is possible to have 1000 people simultaniously hit your web site at the same instance, it is unlikely that you will have 1000 processes running to take their requests. So some users have to wait. But is is possible to have a small number of processes with 1000 threads available to take requests.
Threads reduce memory useage. For example, each process has to load the code for the executable into memory, which multithreaded processes share. Also, if there is server file caching, mutiple threads can share the cache, but multiple processes can't.
Also threads can make more efficient use of resources. Lets say your application connects to a database on the back end (which is probably multithreaded, by the way). Lets also suppose that some transactions take longer than others. The first problem in a non-threaded application that each process has to have its own database connections. They cannot be shared between processes. Also, each process has to first wait for the tcp connection, then wait for the database to respond, then wait for the data to be sent out. While they are waiting, they cannot process other requests. The problem is that all the processes could block on the database doing long connections, while other requests that might not even require database connections wait. In a threaded model (with enough threads), many transactions can be started, while only the ones that actually have to do database connections block on the database.
Finally, threaded programs are more efficient in a multi-processor enviornment. These days, more and more servers have more than one processor. Because each thread can run on a separate processor, you can more efficiently use the hardware.
Threading is the way of the future. That is why Java caught on on the server side. Because it supports threading in the language (something that C or C++ don't do). The Apache writters were looking towards the future, not at the past.
I think the main process simply passes the socket descriptor for the new connection to the virtual host process. Passing descriptors isn't terribly efficient, but it only happens on connection, and certainly more efficient than piping data the way you describe. I'm pretty sure the Apache 2.0 design is efficient and scalable.
If you set things up correctly, then most of that memory is actually shared between the processes. If you read the mod_perl guide (http://perl.apache.org/docs/1.0/guide/performance .html#Sharing_Memory) it explains all of this stuff for you...
Cees
The basic problem is thread group affinity.
Basically, the promise of threads is that you will not be paying the equivalent of a full process context switch overhead, because your VM and other process-specific things will not have to change when context switching from one thread in a process and another thread in a process.
On a machine that has 1001 processes, and you are the 1 process, and you have five threads in your thread group (process), You basically have a 4 out of 1004 chance of one of your threads being picked as the next thing to get a quantum, when one of your threads makes a blocking call, so that it's no longer runnable.
What that means is that you have just reneged on the promise of lower context switch overhead, if you run thread #1, then run "cron", and then run thread #2.
So you have to play favorites, and say "I know "cron" has been waiting a long time, but I just blocked processing on thread #1, and thread #2 is runnable, so I'm going to preferrentially run thread #2, because it lets me avoid the VM switch, and the TLB shootdown, and the other overhead of a full process context switch, and therefore lets me keep my promise about threads being lower overhead than processes".
Any time you play favorites, you starve your non-favorites; just like a Robin or Sparrow with a Cuckoo's Egg in its nest.
So then you have to add all sorts of arcane accounting and other crap to avoid the starvation of other processes, and your scheduler becomes very, very complicated.
Compare this with Scheduler Activations, or an async call gate, where you give a quantum to a process -- and the quantum belongs to that process. In this case, your process runs until either there are no more threads to be run, or until its quantum is used up.
Things are actually more complicated than even this; for example, you want a threaded program to compete as multiple processes for quantum, or you are encouraging people to write programs that fork multiple children, instead of threads, in order to allocate themselves more quantum. On the other hand, you want to set some upper bound on the amount of unfair competition a single unpriviledged program can engage in, relative to other processes on the system.
If you attack thread group affinity as a scheduler problem, the amount of complexity you introduce is substantial, and there will always be corner cases.
There's actually been a huge amount of research on this; check the NEC CS search engine for "scheduling" and "load balancing" and "parallel".
-- Terry