Building Large Scale e-Commerce Systems?

Fross asks: "I am Technical Architect at a reasonably large e-commerce firm, which naturally has a very large and active Website, with tens of thousands of registered users, and millions of page hits a month. The time has come to reimplement the old architecture from the ground up, and I'm looking for other (preferably Open Source, or Open Source-friendly) solutions that will give a faster, more scalable, more reliable, easily maintainable solution than the current implementation (which is running under ASP, on IIS of course). I have a fair amount of experience with Perl, some with PHP, but have never used them for something of this size. Does anyone have experience of deploying something this large with these technologies, and can give some valued information on how these solutions cope?"

Comments

Try J2EE on IBM's WebSphere with Oracle or DB2

[Anonymous Coward]

Although my employee contract explicitly forbids me from mentioning any details about our arquitecture, I can give you a few pointers if what you're after is high-transaction, low-latency, 24x7, and high-availability. As surprising as it may sound, the most flexible solution we've found is Java-related. We had 4 teams each with 5 people trying out 2 different approaches per team (total of 10 different options). The main technologies (in no particular grouping or order) we looked at were Apache/PHP, Microsoft's ASP, Alaire's ColdFussion, Oracle, IBM, and Microsoft databases, iPlanet and WebSphere's J2EE (Java 2 Enterprise Edition) servers, XML, Linux, Solaris, Windows 2000. The approach I'd recommend is a cluster of one of these: Linux, Solaris, IBM Mainframes (you'd be surprise at how their latest generations handle web traffic and transactions), both for the application servers and the database servers. The beauty of J2EE and one of the strongest reasons to select it was that we could trully mix and match different OSs, databases, hardware, etc with virtually no problems, and our internal benchmarks show that JSP (Java Server Pages) are superior to Microsoft's ASP or Alaire's ColdFussion. Also, under heavy load for high-transaction computations we found out (and this we double-checked for it was a complete surprise) that Java code performed just about the same as C++ compiled code. We assume this has to do with the the latest HotSpot dynamic compilation technology from SUN. If you can afford it, I'd say put 2-4 people testing this stuff out. If you're organization is sufficiently large you can get full versions from all of these products for free for a decent trial period. Above anything, I recommend you keep and open mind and be objective by doing your own tests. This simple vision is what's causing us to transition to a Java-based arquitecture. It's light years ahead of when we first tried it a couple of years ago (when we dismissed it for mission-critical operations), and it will certainly shield you from the constant stream of OS versions, DB engines, vendors, etc. Saile

One option

[djweis]

My employer is using the ars Digita software. We haven't had any problems with it, and if you plan on doing more than just ecommerce, it's got a huge amount of functionality, all integrated together. You can save a ton of time.

Try Contacting a Few Companies

[scotpurl]

I recently noticed that http://www.insight.com started using PHP, which seemed downright forward-thinking for a vendor to do.

Most folks like to blab about what they've done, so try emailing a few folks, or see what old classmates or fishing buddies the VP's in your company know elsewhere. That'll open doors quickly.

The only thing of real substance I can offer is obvious, but I'll offer it anyway (just to get the ball rolling). Try to modularize everything, and make it generic. Perl, mod-Perl, and PHP all work on NT or *nix, which helps divorce you from a single-vendor situation. The servers can then have any OS or any web server. You could even do a mix in a server farm.

You'll likely need to do something tricky for to bypass some performance bottleneck, but try to keep those tricks simple, with the intent of eventually finding some non-tricky method of getting around that bottleneck.

Re:Large scale systems

[PhiRatE]

My apologies, our current system has scaled up to a million hits/day, not a month :)

Many thanks

[Fross]

...for all your suggestions. got a few ideas to chase up now. :)

fross

Large scale systems

[PhiRatE]

Yep, been there, done that. Our particular solution revolved around a cluster of linux servers fronted by a linux/iptables layer-4 router and various custom stuff to perform rapid failover. The cluster nodes themselves used Apache with PHP4, with a database backend and rsync to distribute content to the cluster nodes from a distribution box.

In order to reduce the work, you could use a cisco localdirector or equivalent layer-4 switch instead of the iptables stuff, the rest should basically fit together almost out of the box.

This solution scales very well and handles millions of hits/month. Recommended additions are squid running as an accelerator, since it can absorb a significant number of hits that apache would otherwise have to deal with. However the addition of the Tux in-kernel web stuff on the cluster nodes would probably be an even more preferable solution.

Also recommend dual-cpu machines, they do really well with apache.