Slashdot Mirror

The best way of dealing with complexity is to simplify and limit features to the essentials. JavaScript, graphics, fancy layout, etc. all can present an bottomless hole for time and bandwidth.

Another thing to realize is that the relational database you are using is probably one of your biggest bottlenecks. Relational databases are slow. MySQL is actually one of the faster ones, but that's because it doesn't make a lot of the guarantees or provide a lot of the features that a "real" relational database provides. You can get a lot of performance from your relational database by tuning it, but ultimately, the architecture and functionality itself present a limit. If speed is of the essence, consider using dbm, plain files, or memory mapped files (Apache has several "databases" internally, and that's its approach.)

For a single person project, Java is probably not the best implementation language. It really shines for multiprogrammer projects. You may find that Python, PHP, or Ruby are better choices. Perl and Tcl are also widely used, but they are also the oldest of the scripting languages and have a lot of rough edges and clunkiness. The performance of all of them is excellent when used as server plug-ins. Perl and PHP have by far the largest libraries and toolsets.

Some of the packages I haven't used but that look interesting are the following. Enhydra takes a much better approach to dynamic HTML than most other packages, I think. Erlang / Eddie address scalability, reliability, and clustering in a really clean way (but you have to learn a new language). Zope I think has a lot of good ideas, but I'm not sure I'd use it for a large server.

Most load balancing solutions use some kind of network routing hacks. That's efficient, but can be a pain to configure. However, there is now a load balancing module for Apache that works as a proxy; that's probably also worth looking into.

The Coda file system is a free, next-generation AFS; while I'm not sure Coda is mature enough yet to be used in heavy-duty applications, systems like it help with scalability by replicating files automatically and have been used on some really large web sites.

I would stay away from commercial "application server platforms". They often are mostly repackaged open source software, and they are expensive and complex, and while they claim to be general, they are also usually created with fairly specific commercial applications in mind ("shopping cart", etc.). I think those kinds of packages are worthwhile for corporate developers that work with large teams of programmers and need consistency, documentation, training, and support.

So, to recap, this kind of stuff is still a lot of work and it seems like you are ahead actually. But there are a lot of proven open source tools available that have been used on really big projects. There are also a lot of open source developments coming around that may make this kind of project a easier.

In an n-tier environment (web server, app server, database server) all the application server really does is:

• maintain a persistent pool of connections to the DB;
• maintain state in a stateless protocol;
• help separate application logic from display logic
• perhaps assist in transaction load-balancing

I have worked with a half-dozen commercial application servers and the all work equally well for anything I've built. Since there are 2 excellent open-source app servers, Zope and Enhydra, I see no reason to spend a dime on a commercial product unless of course their proprietary RAD tools are something you can't live without (trust me: you can live without them). Still, none of them beat AOLServer pointing to an Oracle8i DB set up for transactions.

The only thing you really need to remember in transaction processing is that the network is the bottleneck.

• If you can shorten the network path, do it.
• If you can design your app so you send the fewest number of bits across the network, do it.
• If you can use the filesystem apache is sitting on to serve "dynamic" content, do it.
• Make everything you can a static file, even if you have to generate it from a database overnight.
• And cache everything you can.

The best two systems I have seen for separating content from presentation are both based on java.
Enhydra and Webmacro. You may want to check them out and both are open source.

Enhydra (the OpenSource Java Application server) has similar issues.

So, they've decided to mix and match.

Somebody should add Enhydra to the list (I would, but I don't remember my login information for the JavaLobby). Enhydra is a very rockin' application server written in Java. It's open source too, which is always a plus.

For now, I'm running a startup outta my home - but it's only in the devlopment phase. Some things I've budgeted for this one when I get some $$$ (in 3 weeks!):

E-com server: P3-450(X2), 256MB, RAID, >= RedHat6, Enhydra, IBM JDK 118, Apache

Db Server:Postgres (free+transactions), same hardware (ditto for Auction Db server)

Auction server: same hardware, Apache, EveryAuction

All running on a 100mbps switch. When the need arises, we'll move the hardware off to Qwest or somethin. All the above can be purchased for under $20k! Our whole goal for the business - software wise - is free and open source.

We've come to the obvious conclusion that there's no need to run anything M$.

Using this standard, you can write the logic of your app and use the app server's tools to deploy it, and not spend time worrying about partial failures, network synchronization, how fine grained access control is managed, and all the other problems associated with distributed programming. And it doesn't mean you have to be a java fanatic -- the latest ejb spec isn't even based on rmi, it aims for iiop compliance instead.

A couple of open source ejb app servers to consider:
EJBoss
JOnAS

Also take a look at the ejb-friendly xml work at Enhydra and of course, the world's greatest servlet engine, Apache JServ.

I suspect that which application server you want depends a lot on the application you want to build. I've had a look at Zope, and found it a bit of a pain because the actual development of the web site was too web-oriented. (I want to keep everything in files and keep these under revision control.) As well, it wasn't really optimised for the programming side of building applications.

What works much better for me is Enhydra, which is very good in terms of developing Java code, and lets you leave plain stuff (html files, images) just as it is.

However, as I said, I suspect which will be better for you depends a lot on what you're trying to build.

cjs

Actually, you can use Apache JServ as the servlet runner to operate Enhydra under. If used in this configuration, Enhydra offers extensions and additions to what you would normally have with plain servlets. You might want to look at this for some info on why a few people have chosen to stick with Enhydra. BTW, if you don't have JServ (or a web server for that matter) you can run Enhydra apps directly with the built in web server functionality.

Locomotive
Protomatter
Enhydra