Slashdot Mirror


E-commerce with mod_perl and Apache

rob_99 writes: "Cool new mod_perl article at perl.com documents building a large scale ecommerce solution w/ mod_perl & apache!" Pretty cool stuff - it's kind of funny to think how ephemeral their work turned out to be.

77 of 174 comments (clear)

  1. Case Study: eToys.com by Alien54 · · Score: 2, Offtopic
    They didn't go under, but I remember wishing they would, given the controversy with EToy

    Interesting to since what was nappening behind the scenes away from the marketroids.

    --
    "It is a greater offense to steal men's labor, than their clothes"
    1. Re:Case Study: eToys.com by Alien54 · · Score: 2
      they filed for Chapter 11 protection,[...]Does that count as going under? ;)

      Sorta

      So is this the best example to use for a successful implementation of mod_perl?

      --
      "It is a greater offense to steal men's labor, than their clothes"
  2. difference between mod mod_perl? by Anonymous Coward · · Score: 3, Funny

    17 mod 5 = 2
    17 mod_perl 5 = ?

  3. What the heck is ephemeral?? by vanguard · · Score: 4, Funny

    EPHEMERAL
    Adjective
    1. Lasting for a markedly brief time: "There remain some truths too ephemeral to be captured in the cold pages of a court transcript" (Irving R. Kaufman)

    2. Living or lasting only for a day, as certain plants or insects do.NOUNA markedly short-lived thing.

    ETYMOLOGY From Greek ephemeros, ep-, epi-, epi-, + hemera, day

    Just in case your vocabularly sometimes leaves you wondering (as mine does).

    --
    That which does not kill me only makes me whinier
    1. Re:What the heck is ephemeral?? by BJH · · Score: 3, Funny

      From the context, I think he meant "ubiquitous"...

    2. Re:What the heck is ephemeral?? by s390 · · Score: 2

      From the context, I think he meant "ubiquitous"...

      No, he meant short-lived. The company (etoys.com) mentioned in the article is tits-up.

  4. mod_perl is definatly ready by Brontosaurus+Jim · · Score: 3, Informative

    I remember my first expierance with mod_perl. I was working for a small development company. That's not the key part

    The key part is that I was in charge of the perl backend, and it was really lagging. Loaded mod_perl up and *bam*, we had a fast enough system. I never did a serious benchmark, but it was like 100% improvment

    Thus all I can say is good job guys! This doesn't surprise me at all, and it's good to see recognition get passed around in the correct quantity for once :)

  5. I'm just glad they didn't try to convince me... by dstone · · Score: 3, Offtopic

    ...that they used MySQL.

  6. Re:PERL - the "Write-Only" language... by jc42 · · Score: 4, Interesting

    No; I haven't had that problem at all. Perl is a
    good language for writing understandable code.

    I was recently contacted by some people about a
    web site full of cgi scripts that I wrote several
    years ago. They sorta apoligized for not getting
    back in touch, explaining that the code was so easy
    to understand and modify that they just did it and
    didn't need to contact me. But now they had some
    more sophisticated needs that they weren't sure they
    could program, so they thought they'd call back an
    expert.

    So maybe the lesson here is that I should learn to
    write less readable code. Then people won't be able
    to modify it themselves, and they'll have to hire
    me to make trivial changes.

    --
    Those who do study history are doomed to stand helplessly by while everyone else repeats it.
  7. Servlet container maybe... but application server? by mke · · Score: 4, Interesting
    Perl certainly does a lot, and it is probably overlooked unfairly for a lot of web applications. As the authors illustrate, it makes a damn fine servlet container.

    But an application server it is not. Container managed persistance, transactional support, message queues, naming and lookup services? Integration with existing business objects and processes? These aren't trivial in Perl, but they are the core functionality of app servers. They pulled off clustering for eToys, but it was hardly an out of the box solution.

    If all you want is a servlet container, don't spend the money on an application server. Tomcat works great and iPlanet bundles one with their web server.

  8. Perl + e-Commerce in the field by Adam+Wiggins · · Score: 5, Insightful

    Having built numerous e-commerce sites and even worked on a commercial shopping cart written entirely in Perl, I can't say that I think it's the best choice. It's great if the job is small; it's quick to code, is installed on almost all hosting providers, and so forth. But for large projects the code becomes very difficult to manage, mainly because getting people to write clean Perl code is difficult.

    There's a second downside: lack of good SSL networking. SSL is critical for credit card processing. I ported the GPL version of TCLink for Perl, and we ran into a lot of problems due to lack of a standard SSL library. For the first version of the client we ended up distributing a patched version of Net::SSLeay which added all of the certificate authentication functions we needed (and are missing from the standard install of Net::SSLeay). Later this became such a headache for the various platforms we were trying to support that we just coded a .xs version of the Perl client which wrapped around our C library. This has proven to be much easier to maintain.

    Long story short: sure, Perl's great. Is it the best choice for e-commerce? I'm not so sure.

    1. Re:Perl + e-Commerce in the field by JSkills · · Score: 2, Informative

      Well, unfortunately I have to disagree with Perl not being ecommerce suitable. Your issues likely stemmed from your choice of SSL library. We went with Apache, mod_perl, and Raven SSL in our Apache build. The ecommerce provider we use, Linkpoint, actually provides a .pm file as an API to their payment gateway. It was embarassingly easy to set up.

      We also use HTML::Mason - the best mod_perl add on period.

      Granted, we're not handling the volume of transactions as an eToys.com, but we use an extremely similar hardware configuration and have been running successfully without issue for quite some time.

      I cannot recommend Raven's SSL and HTML::Mason enough.

      Jusy my 2 cents ...

    2. Re:Perl + e-Commerce in the field by schulzdogg · · Score: 2
      We also use HTML::Mason - the best mod_perl add on period.



      Do a lot of people use mason? When the subject of embedding code in HTML comes up the only thing people talk about is PHP. I've used mason for personal stuff, but at work it was decided that PHP was the choice. I'd like to hear about anybody using mason in any serious way.

      For the curious Mason vs PHP: I havn't been really impressed with PHP. It does what it advertises but I havn't seen anything that it can do that perl can't do. I also havn't seen it do anything better than perl, with the exception of it's treatment of arrays and hash's which is a great way of unifying the two concepts. One PHP negative is that it doesn't have the extensive library support that perl does. It has pear, but it doesn't seem very developed. I guess if you're learning a new language, why waste your time on php when you could learn perl and use mason, and when you were done you'd have transferable perl knowledge?

    3. Re:Perl + e-Commerce in the field by Electrum · · Score: 2, Insightful

      Getting people to write clean code is a management issue. You can write clean code in any language, and you can write bad code. Perl has a reputation for encouraging obfuscated code, but if you look at the entries for IOCCC, you might think the same thing about C. The company I work for uses PHP for large ecommerce projects, and I can assure you that PHP doesn't force people to write clean code. If people are going to write bad code, I'd rather it be in a safe language like Perl or PHP, rather than C or C++. Of course, I'd rather not work with people at all that don't have the discipline to write good code :)

  9. Re:Let me get this straight... by jjeff · · Score: 2, Insightful
    The worst terrorist attack in recorded history occurred last month, and now we're involved in a WAR and you people have the gall to be discussing building a large scale ecommerce solution w/ mod_perl & apache???? My *god*, people, GET SOME PRIORITIES!

    So in your opinion, nobody is allowed to do anything while "The souls of the victims are watching"?
    How are people supposed to move on from this tradgedy if they cant do anything to take their minds off it?

    Anyway to get ON TOPIC, perl is a great language to write this sort of thing in. I prefer using Embedded Perl myself (i wonder how that would perform as an ecommerce solution.).
    but perl IMHO is the most powerful and versatile CGI language around, java creates too much of a load, and C requires you to recompile after altering any of the code.

    ok im rambling now..

    --
    when everything is working perfectly.. BREAK SOMETHING before something else FUCKS up!
  10. Wow. by Fixer · · Score: 3, Interesting
    These guys, while not having it exactly made, had resources I'm envious of.. To wit,
    Not all of the team had significant experience with object-oriented Perl, so we brought in Randal Schwartz and Damian Conway to do training sessions with us. We created a set of coding standards, drafted a design and built our system.
    Say what? You brought in who? Gee.. And I had to learn it all on my lonesome..

    But I gotta say, I would have loved to have been on that team for that. Though I would have rankled at the suggestion of needing "help" with object oriented programming, it would have been worth it to meet those two.

    --
    "Avast! Prepare for the rodgering!" THWACK! "Arrr.. me nards.."
  11. Why use mod_perl ... by realdpk · · Score: 2, Insightful

    It seems like the primary (note /.'ers, I said PRIMARY, not your little pet project :) ) use for mod_perl is to avoid the overhead associated with loading modules.

    It seems like the better choice would be to avoid using modules altogether. Or using another language like C or PHP that has stuff built-in. It's amazing how much CPU time loading a simple perl module takes.

    1. Re:Why use mod_perl ... by fanatic · · Score: 5, Insightful

      It seems like the primary ... use for mod_perl is to avoid the overhead associated with loading modules.

      Even if you don't use modules (which would be silly, roughly comparable to not using sub-routines), you still incurr the penalty of loading and compiling your scripts each time they run if you don't use mod_perl. Also, mod_perl gives you amazing control over how the requests are interpreted and responded to. But even if it it didn't, a 5 or 6 to 1 improvement in requests/second (which is what I saw just by using mod_perl to run the script, with little of no optimization) is nothing to sneeze at.

      --
      "that's not encryption - it's a new perl script that I'm working on..." - from some Matrix parody
  12. Perl has a strong nice -- more than you think by spRed · · Score: 5, Interesting

    C++ guy who did industrial OO perl for two years --

    [quick disclaimer, feel free to use ruby or php for a substitute for perl below]

    Perl is very good for a much larger range of projects than you think. [with a HUGE deference to C++/OO people, see note at bottom]. Very few sites rely on pure computational power. Most sites only need a small compenent to be fast, and that you can implement in the language of you choice with perl bindings. The majority of your site is glue -- and this is what perl does well.

    I worked for a small team (5 developers) that wrote 200k lines of perl in a year and a half. That represents a far larger body of code if it had been written in another language. That isn't to say you lose control by using perl. You get to ignore the details, and in the bargain get a perfect language (and some extra time) to write regression tests.

    All in all, unless you are a games site that cares about milliseconds, you can get a page out the door in under .15 seconds with perl. The images loading (even on broadband) will be the gating factor on pageloads.

    -spRed

    the deference to OO folk, you can teach people who only know C++/Java perl, you can't teach people who only know perl the others. The reason why is that real CS educations are portable. Perl [and other interpreted languages] are looked down upon by CS folk - so only non-CS folk will persue perl as a primary language. Perl is, however, extremely useful for whatever you are doing (95% of you). Give it a chance, it will save you alot of time in the long run.

    --
    .sig Karma out the wazoo, better to spend points elsewhere if this is above 2 or below 0
    1. Re:Perl has a strong nice -- more than you think by Ian+Bicking · · Score: 4, Insightful
      Perl [and other interpreted languages] are looked down upon by CS folk
      Don't be silly -- CS folk like interpreted languages. Good, wholesome languages like Scheme, Smalltalk, Prolog... sure they don't like Perl, but don't blame it on the fact it is interpreted. Blame it on the fact that Perl is the antithesis of formalism.
    2. Re:Perl has a strong nice -- more than you think by NonSequor · · Score: 3, Informative

      Actually, Scheme can be compiled. There are a number of implementations that just interpret it and relatively few that actually compile (Bigloo, MIT Scheme are the free ones that I can think of).

      --
      My only political goal is to see to it that no political party achieves its goals.
    3. Re:Perl has a strong nice -- more than you think by Ian+Bicking · · Score: 2
      Well, there's a vague distinction between truly compiled languages and interpreted languages. Almost every language being written right now uses byte compiling -- straight interpretation is very uncommon these days. Compiling to C or machine code is just another step in that direction -- but the distinction is more quantitative (degree of compilation) than qualitative.

      All the more reason intellegent computer scientists won't be biased against Interpreted languages.

    4. Re:Perl has a strong nice -- more than you think by prwood · · Score: 2, Insightful

      [quote]so only non-CS folk will persue perl as a primary language.[/quote]

      I earned a Bachelor of Science in Computer Science, which I received in May after completing four years of computer science coursework in a CS department that was focused on the OO development model. We did most of our advanced development in C++, so I got plenty of exposure to that. I learned it well enough to get satisfactory marks (mind you, I didn't get perfect marks, but I at least showed that I could use what I was learning).

      Now? I am working at a large e-commerce company, developing website/database functionality in Perl. Perl is my primary language. In fact, including me, five out of the six developers in my department earned their BSCS at the same college. We all program in Perl. We also write a lot of modules to encapsulate functionality for our site, and write/use them in an OO manner. Some of us are not as crazy about OO as others, but we all use it. We have certainly never 'looked down upon' Perl.

  13. Re:PERL - the "Write-Only" language... by chromatic · · Score: 5, Informative
    I've had that problem with PERL, but then I discovered its predecessor, Perl. It's a much nicer language -- it has warnings, an optional pedantic compiler mode, lexical variables, embedded comment support, a debugger, copious documentation, numerous libraries, and active user communities. Best of all, it's not limited to CGI!

    If you've had a bad experience with PERL, give Perl a try!

  14. Re:I love Perl. by Furry+Ice · · Score: 3, Informative

    Fully compiled? Servlets? C#? Apparently you've never heard about this thing called "byte-code". It's definitely not fully compiled. Perl doesn't have to be spaghetti, either. Read the article. If you know how to use Perl 5's OO features and ensure that you (and everyone you devlop with) always use them, you can stay on top of things. Not to mention the advantage you get from using a higher-level language.

  15. Re:OO Maintainable Perl.. by chromatic · · Score: 5, Insightful
    What are you talking about?

    Object Oriented Perl is extremely powerful and flexible. It's possible to do exotic things like multiple dispatch, it *does* support true encapsulation, and you can change inheritance hierarchies at run time. I've seen people write first-class objects and classes (think Self), and I've implemented mixins having seen them in Ruby. You can use all sorts of design patterns, have compile time member checking, and even modify objects as the code is running. There are even good reasons to use multiple inheritance (Perl's not the kind of language that thinks it's smarter than you.)

    If you mean "member data tends to be class based instead of object based", then you're right. That's a true limitation of OO Perl. It would also be nice to have a refactoring browser like Smalltalk has.

    As for readability, I can't read anything written with Kanji characters. That doesn't mean I'll ever claim it's "unmaintainable compared to English". (To be fair, complex data structure dereferencing *can be* hard to read in Perl.)

  16. Serious Scalability by rnicey · · Score: 4, Interesting

    I part own an e-commerce company that's partly built with Apache/mod_perl/MySQL. We crank millions of dollars through those servers every week (porn still pays, what can you say ;-). The compile time of a script that pulls in libraries with 200k+ lines of code without mod_perl is several seconds. With, it's < 1sec. True we have/need 4GB RAM/server, but it does allow us to scale to whatever we want.

    I often read articles saying perl/mysql can't handle enterprise solutions. Well, it can, does and is doing so in our case. Like any tool, it's as effective as the people wielding it.
    We also have a huge codebase in C/C++, Java, Python and a few others. There's even a few NT boxes thrown in for dealing with some obscure hardware. I'm not particularly bias one way or the other, I just need the results.

    By the by, I do have one solution for ensuring my programmers don't take sloppy shortcuts and write obviously quick and dirty code. It's the pink slip method. I find it works equally well in any language ;-)

    Robert Nice
    WebsiteBilling.com Inc.

    1. Re:Serious Scalability by jallen02 · · Score: 2, Interesting

      I wish more practical posts like this got moderated up.

      While I think all software does indeed have limitations I think developers tend to impose artificial limitations.

      I don't like mySQL all that much but I will be the first to admit I have been a part of a team that forced it to scale. Uhm MySQL doesnt scale? Look at what your posting to! Slashdot isnt even a massive web cluster or anything and it handles the load quite nicely. We used PHP, SQUID, and mySQL in combination to make the system scale absolutely through the roof.

      It is all about the developers in most cases!

      I also don't like Perl and would not choose to implement in it but I find the claims that its impossible to write clean code in it a little far reaching. I have seen Java, C, PHP, and Perl written well. I have also seen code in each language an indecipherable jumble of spaghetti code (even in Java and even worse in PHP)

      So anyway...use technology that gets the job done and keeps you in the green.. stop fretting about the small stuff (and its all small stuff ;)

      Jeremy

  17. Why? by drodver · · Score: 2, Interesting

    Where is the advantage of this over a simple round-robin scheme? With round-robin each server will at least get the same number of sessions to start. With your scheme it'd be possible to for a couple of machines to be bogged down while the rest are relatively idle. Sure that might be unlikely to happen but I don't understand why you would even give it the chance.

    1. Re:Why? by jallen02 · · Score: 2, Interesting

      Sticky sessions defeat the purpose of a clustered load-balanced system.... You can end up with a disproportionate number of users on one system bringing it down and lessening their experience.

      We have an entire centralized server that does nothing but serve session data. From the ground up the server and database is optimized for serving session data.

      Works quite nicely :)

      Jeremy

    2. Re:Why? by perrin_harkins · · Score: 2, Informative

      We also had a central server, but with a local cache on each machine to improve performance. Although it is possible for one machine to get more than its fair share of sessions, in practice it didn't happen. The load-balancer did a good job of sending new sessions to the least busy machine.

  18. Re:Scalability Myths by chromatic · · Score: 2, Informative

    Because Perl threads are experimental until 5.8 is released?

  19. MVC + XSLT + Python by 1010011010 · · Score: 2

    Great article.

    I'm looking at setting up a similar type of system using postgres, apache, mod_python, and XSLT. Has anyone else ventured down a similar path?

    --
    Napster-to-go says "Fill and refill your compatible MP3 player", which is a lie. It's not MP3. It's WMA with DRM.
    1. Re:MVC + XSLT + Python by john@iastate.edu · · Score: 2
      Their 3-level architecture is exactly what I've used any number of times. That and templating HTML is the key, IMO.

      I personally, like to use C and DB files for most things (I don't need Oracles firepower, complexity, or price :)

      My templates look something like this:

      ${include head.html}
      <form method=post action="${path_url}">
      $?{messages}${messages}<hr>$.
      $?{gt ${v_count} 0}<table border=1>
      <tr><th>Mark</th><th>Volum e Class</th></tr>
      $@{for i 0 lt ${v_count}}<tr>
      <td>${radio volclass ${v_class[${i}]}}</td>
      <td>${v_class[${i}]}</td>
      </tr>$.
      </table>
      <input type=submit name=action value="View/Edit Properties of Marked Class">
      <hr>$.
      $.${include buttons.html}
      </form>
      ${include tail.html}

      --
      Shut up, be happy. The conveniences you demanded are now mandatory. -- Jello Biafra
    2. Re:MVC + XSLT + Python by Ian+Bicking · · Score: 2
      Instead of straight Python with mod_python, you may wish to consider Webware or SkunkWeb, which are both application servers for Python (it ain't all Zope!)

      I use Webware myself, and quite like it. Similar to Java Servlets, without the verbose language. I haven't used SkunkWeb myself, but it looks reasonably mature (it's at v3, where Webware hasn't quite reached 1.0 yet... but that doesn't necessarily mean much).

  20. Re:I love Perl. by Ian+Bicking · · Score: 3, Interesting
    I can't say I agree that JSP, ASP, PHP, or any of the *SP family of languages are good design. They encourage exactly the opposite of MVC, they encourage you to mix HTML and (non-display related) programming code.

    This leads to a total mess, and even when the mess is kept under control it's usually with lame templating techniques like having a standard header and footer, which are initially expedient but not very flexible. "Initially expedient" is exactly what the *SP languages achieve, but at the price of later ease of development.

    Of course, you can use any of those languages with a <% at the top of the file, and %> at the bottom, with an entirely seperate templating solution, but that kind of renders the whole point of those implementations moot. In my own experience with PHP, my code has started as normal, mixed-HTML/PHP code, and eventually ended up as just one chunk of PHP code.

  21. mod_layout is your friend by belg4mit · · Score: 2, Insightful

    http://software.tangent.org/projects.pl?view=mod_l ayout

    --
    Were that I say, pancakes?
  22. Re:Servlet container maybe... but application serv by Ian+Bicking · · Score: 5, Insightful

    If you're problem is even moderately interesting, there will be no out-of-the-box solutions. That Open Source might require more implementation time may be true, but the "turn-key" fantasy is most always a lie.

  23. eToys went down like a lead zepplin by Anonymous Coward · · Score: 2, Interesting

    I used to work for eToys.

    In November, eToys had over well over 1000 employees. By March, they had fired all but a 10-20 employees (all management), and most of those people are gone by now. Few of these 1000 employees received their full severance package because eToys had no money (This was a violation of the contract, but suing was a lengthy and tedious option).

    Between December of 1999 and December of 2000, eToys burned though several *HUNDRED* million dollars in funding.

    KB Toys bought the eToys merchandise (toys), the eToys brand, some of the computer hardware (but not the intellectual property needed to maintain it, e.g. the engineers and administrators), and I think the warehouses (One of the warehouses was a $20-million high tech warehouse).

    That counts as going under.

    Their only successful subsidiary, Babycenter.com, lives on. But they don't use mod_perl (It's an Apache + java application server shop).

  24. Mod_Perl vs. XML issues resolved? by Xunker · · Score: 2

    Have the Issues with expat, etc, that mod_perl has been sorted out? XML is vital to "e-commerece" infrastructure these days, but the last time I tried using the perl XML modules with mod_perl (2 months ago) I was greeted with a stunning array a segfaults, and I was told it was a known issue (something to do with expat, IIRC). I did look in the usual places and can't find any mention of these issues being fixed, unless I'm not looking in the right place?
    .

    --
    Hilary Rosen's speech was about her love of money and her desire to roll around naked in a pile of money.
    1. Re:Mod_Perl vs. XML issues resolved? by Animats · · Score: 3, Flamebait
      Perl, interestingly, isn't very good for processing structured text files efficiently. The usual state-machine parser (get next character, get type of character, fan out on type) is inefficient in Perl, because "get next character" from input is slow. The obvious approach results in shifting the entire input string by one character for each character removed. You can't subscript a Perl string character by character. (This is a religious issue for Perl zealots.)

      As a result, Perl parsers for HTML, XML, and SGML typically have the tokenizer written in C. "expat" is the low level part of an XML parser written in C. So Perl programs that parse HTML, XML, or SGML typically have a C component.

      That C component has to go into the Apache server (and run with server privileges, with all that implies, like a potential for buffer overflow attacks) for mod_perl programs to use it.

      It's an annoying limitation of Perl.

    2. Re:Mod_Perl vs. XML issues resolved? by chromatic · · Score: 3, Informative

      It's fixed in Apache 1.3.22. You could also disable the EXPAT rule when compiling Apache.

    3. Re:Mod_Perl vs. XML issues resolved? by emoon · · Score: 2, Informative

      Yeah, it's called Apache 1.3.22. :)

      Seriously, it wasn't a problem with mod_perl, but the way Apache would be built.

      It's been fixed in the latest rev of Apache.

    4. Re:Mod_Perl vs. XML issues resolved? by Matts · · Score: 5, Informative

      That C component has to go into the Apache server (and run with server privileges, with all that implies, like a potential for buffer overflow attacks) for mod_perl programs to use it.

      Complete and utter BS.

      What are "server privileges" ? Everything in apache runs under the nobody user (or configurable).

      What potential for buffer overflow attacks? Show me one.

      The component does *not* have to go into the Apache server for mod_perl programs to use it. Not even slightly. If that were the case, DBI would be unusable from mod_perl, wouldn't it? No-siree, the C component (expat) was included in Apache to support mod_dav. They modified expat slightly (and called it expat-lite), and thus we had symbol conflicts, and the wrong code being executed, and BOOM. So now Apache 1.3.22 uses the DSO expat, same as XML::Parser does, and the problems are gone.

      Someone please mod me up or him down. Facts can be checked in the mod_perl and mod_perl-dev list archives.

      --

      Matt. Want XML + Apache + Stylesheets? Get AxKit.
    5. Re:Mod_Perl vs. XML issues resolved? by scrutty · · Score: 4, Informative
      You can't subscript a Perl string character by character

      Whats wrong with this ?


      @chars = split //, $string


      Admittedly the split function can't work on individual characters as it needs an IFS but substr seems to work ok for me.

      What about this

      $ perl -e ' map{ print $_, "\n"} split // ,"qwerty" '
      q
      w
      e
      r
      t
      y




      Or have I misunderstood ?

      --
      -- Oh Well
  25. Re:PERL - the "Write-Only" language... by perydell · · Score: 2, Informative

    As for stripslashes and addslashes you should look into magic quotes with PHP.

    As for too many print statements you can do <?=$variablename?> to print more concisely.

  26. eToys had over 100 people coding HTML by Anonymous Coward · · Score: 3, Informative

    eToys had over 100 people coding plain vanilla HTML... is that a good example of how mod_perl can be used as an ecommerce app server?

    mod_perl definately has it's uses, but eToys is a bad example. Remember, eToys went out of business, in part because they spent way too much money on hardware and had way too many employees doing simple jobs.

    You would think a large site like eToys would make good use of templates and would reuse programming objects all over the place. This wasn't the case for eToys at all.

    eToys initially felt it was too difficult to use mod_perl to create templates or reuse objects on their site. They almost never reused elements in their pages, and they had very few templates. There was hardly anything there you could call an 'application server', and most of their pages were not dynamic (outside of store item entry).

    Instead, they had over 100 HTML people to write the HTML for just about every single page on their site. It wasn't a very large site either. When I say 'HTML', I don't mean 'they wrote DHTML or PHP or java or perl. They had 100 people writing plain vanilla *HTML*.

    Towards the end, eToys hired a few smart folks who knew the capabilities of mod_perl. They were on their way to creating a templated dynamic site. In the middle of this new effort, they went bankrupt.

    (I used to work for eToys. Posting as AC to protect me from Toby Lenk's lawyers).

    1. Re:eToys had over 100 people coding HTML by Anonymous Coward · · Score: 2, Informative

      What were you smoking? Perhaps the HTML coders you are referring to were the folks that wrote descriptions, reviews, etc. The same writers that were English majors.

      I was in the tech suite and never saw "100 coding HTML". Lots of Oracle DB staff, Perl programmers, a few admins and some misc employess. In fact, during the all hands tech meeting (when the laptops got stolen) I don't think there were 100 people in the ballroom.

  27. Re:I love Perl. by DNAGuy · · Score: 2, Informative

    I can't say I agree that JSP, ASP, PHP, or any of the *SP family of languages are good design. They encourage exactly the opposite of MVC, they encourage you to mix HTML and (non-display related) programming code.

    This is why n-tier apps were invented. Javabeans, COM components, whatever. Just because you can put logic in your pages doesn't mean you should. Gives you the option to run threads asynchronously and/or via message queuing and increase your performance considerably.

    Remarkably, in my experience building high performance (not necessarily highly scalable) sites, I've learned (from pain and suffering and/or other coders) that building those strings to push outo the buffer is often very expensive. You wouldn't think...

    The other lesson I learned is the value of profiling tools. Particularly tools to show you what is going on at your database and a sniffer to show you what's going over the wire under load. I found problems that I would never have noticed without these tools.

    --

    BRENT ROCKWOOD, EST'd 1975

  28. Why Oracle? by Ogerman · · Score: 2

    So they used mostly open source software, but not for the database engine. Is MySQL really that bad wrt. scalability? (Or has that changed with newer versions?) And what about PostgresQL? If it is the case that there are no viable open source solutions for highly scalable databases, why have there been no development efforts to produce more powerful database engines?

    This is just a question. Use your mod points on good answers. (-:

    I don't need no steen'kin karma!

    1. Re:Why Oracle? by pease1 · · Score: 2, Insightful
      I worked on a site that used mod_perl/html::templates/MySQL/Oracle. Worked very nicely, we did a lot with just a few people. Originally, we used MySQL and the developers complained very mightly and loudly when the management weenies decided to switch to Oracle.

      Why? Business and partnering. Not technical. When the CEO was in meetings with possible partners and said "MySQL" when asked about our database, he got wrinkled noses and confused looks (mys..what?) in response. When he said "but we are migrating to Oracle," the result were ohhhss and hhhhhaaaa. Sigh.

      Once the technical team got into Oracle, they liked it and started to use various features that MySQL didn't support (at the time). I can remember doing rollbacks more than once while handjamming SQL code before a demo.

      Nonetheless, I think we all remain MySQL fans.

    2. Re:Why Oracle? by perrin_harkins · · Score: 2, Interesting

      At that time, neither MySQL or PostgreSQL could handle the load from our traffic. Oracle also provided things we needed like transactions, replication, message queueing, foreign key constraints, etc. Yes, PostgreSQL had some of those at the time, but it had poor performance. Some of these things have changed since then, and it woudl be interesting to see how big PostgreSQL could scale these days.

  29. Re:I love Perl. by Starky · · Score: 4, Insightful
    Spaghetti code comes from a spaghetti mind.


    Using Perl as opposed to, say, Java or C++, is like using Emacs as opposed to Notepad.


    You can learn Notepad lickety split. You'll get alot more done if you know neither and have to get something done in a single day.


    But over the long haul, you'd be far better off using Emacs.


    Perl (and Python) are the same way: If you turn a bunch of inexperienced coders loose with no oversight and no structure, you'll get crap. But if you have a team of experienced Perl programmers, they will beat the pants off of any team of, say, C++ developers in terms of development time any day of the week.


    Not to detract from C++. I am simply saying that the languages are used for different things, and that poor code follows from poor coding standards and practices, inexperience, and unorganized minds.

    So if you can't write Perl without seeing spaghetti, I would suggest you have either some learning to do or that you need more discipline in your coding practices.

    --
    -- My choice of computing platform is a symbol of my individuality and belief in personal freedom.
  30. You're asking the wrong question by Starky · · Score: 2, Insightful
    You should instead ask, "Why wasn't everything written in C++?"


    They wrote a small piece in C++ because it was the right tool for the job. They wrote the rest in Perl because it was the right tool for the job.


    If the whole thing was written in C++, they would probably still be coding the basic application.


    When your needs are to get an application to market quickly, there are few languages that can compete with Perl.


    If you question the scalability of Perl, I would challenge you to write a complex web application in any language that would handle 200,000 sessions and 2.5 million page views per hour.

    --
    -- My choice of computing platform is a symbol of my individuality and belief in personal freedom.
  31. I don't think thats what he meant by stuce · · Score: 5, Informative

    TrustCommerce offers a drop in .pm module as well for people writing perl front ends to their web sites. The Raven SSL package allows apache to serve https, not make ssl connections from a perl program. The trouble with SSL in perl is not serving https+mod_perl, its in writing those easy drop in .pm modules for connecting to the eCommerce gateways. Lucky for all you eCommerce site developers, we gateway programmers get to worry about that, not you.

    I don't think Adam's comment was meant to detract from the power of something like Mason and mod_perl for developing web site front ends. It was just that of all the API's we have open source libraries for (C, Perl, J2SE, J2EE, Python, PHP and ColdFusion), the Perl module was by far the hardest to develop as the existing tools lack some important features.

  32. Re:OO Maintainable Perl.. by asackett · · Score: 2, Insightful
    ... and I don't think anyone would argue against the well known write-only feature of perl code.

    Maintainable yes...

    Which is it? Write-only, or maintainable? My guess is maintainable, because I have never once heard one with a solid understanding of regular expressions state that perl is "write only".

    --

    Warning: This signature may offend some viewers.

  33. Re:I love Perl. by SCHecklerX · · Score: 3, Interesting
    I can't say I agree that JSP, ASP, PHP, or any of the *SP family of languages are good design. They encourage exactly the opposite of MVC, they encourage you to mix HTML and (non-display related) programming code.

    This leads to a total mess, and even when the mess is kept under control it's usually with lame templating techniques like having a standard header and footer, which are initially expedient but not very flexible. "Initially expedient" is exactly what the *SP languages achieve, but at the price of later ease of development.

    Every time I see a post like this, I have to ask...what do you think of 'normal' gui applications? ie, GTK or even raw X11 Protocol stuff? You certainly don't separate your interface from the code driving it there, so what is your point with embedded scripting? Seriously.

    On a good team, embedded stuff is a joy. You can have your layout artists do their mockup, and then you just insert the code into their mockup to make it all work. The layout guys can still use their gui publishing tools, and you just have to link the code to the forms. Dangerous with idiots, but very effective when the page layout guys are smart enough to keep their hands off anything between the [($|-|+|#]'s

  34. You should be safe then by Wee · · Score: 5, Insightful
    The reason why is that real CS educations are portable. Perl [and other interpreted languages] are looked down upon by CS folk - so only non-CS folk will persue perl as a primary language.

    Like the subject says, you should be safe in your career. After all, you know The One True Programming Language, right? Perl isn't a toy, though, is it? I mean, if it's functionally equivalent with Ruby or PHP in all respects then it must be worth something, right? (Just don't let those interpreted guys into the Compiled Officer's Club... they might tell wild stories about getting lots of work done really quickly or spread notions about non-compiled languages being the right tool for the job...)

    Seriously (or maybe not), perhaps you could check your OO bigotry at Dr. Dobbs door when you come slumming it in Scripting Land? I know some of us started using Perl before a "real" language like C++ or Java, but that doesn't make us unable to learn that real language, does it? Does it mean that we won't be able to get real work done? Are the sites we work on less useful to you?

    Why would you look down on someome who has started out with Perl? I started out in the computing world using BASIC on my VIC-20. I didn't have the luxury of Java or C++. Do you think I can still learn? Am I worthy? Can I evolve to your level? Will I ever be able to look down my OO nose at mere scripters like you do? Can I be effete?

    Ok, sorry to bait you. I use perl as much as I use shell scripts. I no longer use BASIC or Pascal and I'm glad. Quick and dirty GUI apps in C and C++ are a pipe dream -- Java works very well for that (especially when your app has to very quickly work on everything from BSD to WinNT). But I hate to break it to you: I have no CS eductation. The only CS course I ever took I wound up teaching. Should I resign from my job? Am I doing my company a disservice by "trying" to write OO Java apps for them since I don't have a "real CS education"? Are the OO perl apps I've written even OO enough to get me in the club? Are my C apps more portable than my lack of education? Are my Java apps?

    You should realize that your "real" CS education bought you knowledge, but not at the expense of anyone else's abilities. Software Engineering is not a zero-sum game; we can all play without dimishing the accomplishments of others. But if you need to feel important, then I guess your post is like therapy for you or something...

    Life is too short to be a snob.

    -B

    --

    Ash and Hickory, straight-grained and true, make excellent bludgeons, dandy for the cudgeling of vegetarians.

  35. If only.. by bLanark · · Score: 2, Interesting

    Well, this shows that Apache and mod_perl have the speed, performance, configurability, etc, to get the job done. Why do we see so many Microsoft/ASP web sites?

    I think that it's due to marketing. If the open source movement paid for advertising, publicity, etc, then a lot more people would consider the open source alternative, but they opt for the large company with marketing, and reps, and so on....

    --
    Note to ACs: I won't mod you up, even if you are being funny or insightful. So take a chance! It's not real life!
    1. Re:If only.. by Lazy+Jones · · Score: 4, Insightful
      Well, this shows that Apache and mod_perl have the speed, performance, configurability, etc, to get the job done. Why do we see so many Microsoft/ASP web sites?

      Because just like Apache/Perl or JSP/Servlets, PHP etc. ASP can also be an adequate environment for large-scale solutions if someone with enough experience is using it. If you throw enough brains and money on a problem set, you're bound to get a good answer, no matter which technology is used (unless it has some fundamental deficiencies, of course).

      As far as the article is concerned - I doubt that more than the top 1% of experienced Perl programmers could build anything like that...

      --
      "I love my job, but I hate talking to people like you" (Freddie Mercury)
  36. Re:PERL - the "Write-Only" language... by tzanger · · Score: 3, Interesting

    My personal favorite bit is the way you can suddenly create a new instance variable on a whim, just by assigning a value to $this->varName, halfway down a random function.

    This, and I would think to say that all your piss-poor examples of why Perl is bad are cured with one statement:

    • use strict;

    Stop spreading FUD. I've seen and worked with many large, OO and non-OO Perl scripts which are a joy to maintain. You can get bad code in any language. Perl lets you shoot yourself in the foot if you want to. My personal preference is not to hire these types of programmers instead of complaining that the language is too flexible.

  37. How many 'nodes'? by mgkimsal2 · · Score: 2

    I'm a bit late here, but I can't see how many 'nodes' they have. 7000 - 20000 orders per hour, or whatever other stats they throw up, just isn't impressive unless I know how many machines (and their power) handle that. 5-10? Probably impressive. 100? Probably not. Anyone got any more info?

  38. Re:Servlet container maybe... but application serv by frankie · · Score: 2

    If your problem is even moderately interesting, there will be no out-of-the-box solutions.

    I hope that by now, e-commerce should not be an interesting problem at all. It's a standard business practice that ought to have simple (and secure) OotB solutions.

  39. Re:Servlet container maybe... but application serv by The+Pim · · Score: 3, Insightful
    But an application server it is not.

    Ok, I'm going to try you on this. Reason is, I've written fairly serious server code in Java, and didn't use many "application server" facilities, because I didn't see the advantage.

    Container managed persistance

    Far as I can tell, this saves you writing some trivial SQL statements. Plus, as soon as you have any interesting data (ie, not just one row in a table) or performance needs (this is backed up by benchmarks by an experienced app-server user), container manager persistence is impractical anyway, so you have to learn how to do it yourself.

    transactional support

    Unless you have some exotic need, a transactional data store is the beginning and the end of the solution.

    message queues

    Easily implemented over an SQL database (I wrote some pseudo-code, but it's too hard to format in slashdot).

    naming and lookup services

    I know Java has some facilities for this, but what do they do that's not easy with LDAP or similar?

    Integration with existing business objects and processes?

    You'd be surprised how many of these are already in Perl. :-) That aside, most of this means accessing an SQL database. I mean, sure, Java will integrate better with a Java shop and other Java software, if that's important.

    They pulled off clustering

    Pretty easy with a remote data store.

    --

    The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
  40. Re:Servlet container maybe... but application serv by perrin_harkins · · Score: 2, Informative
    But an application server it is not. Container managed persistance, transactional support, message queues, naming and lookup services?

    I think you're over-stating the necessity of these things. Container managed persistence is often a way to replace one short SQL statement with 3 long XML files. Message queues are trivial in a good database (Oracle). Naming and lookup services are need if you're using EJBs, but we didn't need EJBs.

    They pulled off clustering for eToys, but it was hardly an out of the box solution.

    How do you define out of the box? We didn't do much work beyond the custom programming that was needed for our application. The open source components filled in our anfrastructure needs very nicely. It wasn't a "packaged solution", but it was easier than many commercial packages I've had to use in the past.

  41. Re:Load balancing (breaks with NAT) by cloudmaster · · Score: 2

    So, what happens when you have a request coming from behind an iptables firewall (or something else that does this) using "iptables -A POSTROUTING -s $INTERNAL -j SNAT --to 10.1.1.1-50" (assuming I didn't mangle the syntax) and thus have requests that rotate through the 10.1.1.1 to 10.1.1.50 IP range? This is fairly common, and breaks things that depend on the remote IP for session tracking.

  42. Re:I love Perl. by tshak · · Score: 2

    Compiled means this (in this context): Things are not processed (interpreted) from the "top-down".
    A variable exists whethoer you declared it on line 1 or line 500. There are many benefits to this. I'm talking from a developers perspective - byte-code gives me the same end result (essentially).

    --

    There is no longer anything that can be done with computers that is nontrivial and clearly legal. -- Paul Phillips
  43. Re:Not really enterprise architecture by perrin_harkins · · Score: 4, Informative
    Separating code into MVC layers is a good start, but the real issue is the interfaces between M and V and between V and C. The article really made it sound like these components were coupled pretty tightly.

    Not true. There was a clean separation. The model objects didn't have any HTML or display code in them. The templates (views) had no control flow code, only formatting. The controller knew how to pick a view, but not what was in it.

    What if eToys had survived the dot-com implosion, and bought or were bought by another company? How well would this system plug into (or be plugged into) any special-needs components of the new system? That's the real acid test of maintainability in the large.

    We had well-defined and documented APIs for all of the components. The hard part in integrating with another system would have been the differences in core functionality, not the interfaces.

    Maybe these standards exist for perl, and I haven't used it enough in an enterprise context to stumble across them. However, you can't help but stumble across the J2EE stuff when you work with Java. Are these standards perfect? No. Are they complete? No. But they're standards, and that's what makes pluggable components possible.

    Only to the extent that you can move your J2EE app from one server to another. Every J2EE app uses the basic components (EJB, JSP, servlets, etc.) in its own way. You certainly can't just connect up your warehouse management code to someone else's shipping code without a thorough agreement on the interfaces. Expecting anything more than that is just dreaming.

  44. Re:PERL - the "Write-Only" language... by tzanger · · Score: 2

    Come on, if you're going to flame me, at least make sense. I replied to a post about PHP, gave some PHP code examples as to why PHP is bad, and you've come back with that Perl users' mainstay: use strict;

    I did read your post; I noticed that you had one reference to PHP in there, and I also took into account the subject line and came to the conclusion that you were knocking both Perl and PHP, as Perl has some of the same problems (creating variables out of thin air and silent conversion between incompatible types mainly) as PHP. It was a thought-out flame. Honest.

  45. Re:OO Maintainable Perl.. by FatHogByTheAss · · Score: 3, Insightful
    You go ... uhh.. girl!

    Except for this...

    To be fair, complex data structure dereferencing *can be* hard to read in Perl.

    Yes, but(tm)... In the context of OO, you should never have to dereference a complex data structure. You shouldn't have to know a dang thing about the structure.

    Regardless, I've found people squawking about the "write only" feature of Perl all share a common feature amongst themselves. None of them write Perl.

    Go figgure.

    --

    --
    You sure got a purty mouth...

  46. Re:I love Perl. by SCHecklerX · · Score: 3, Interesting

    I just want to add to my previous comment. You can embed things, but still keep the code pretty separate. Just write a bunch of modules, and then everything you want to do could just be (in embedded perl, for example) [+ &niftyfunction(@stuff) +] I think using embedded scripting in this fashion is even MORE readable than your separate code/layout way of thinking b/c you don't have a bunch of "print" crap getting in the way of what you are writing, and it also allows your layout guys to do the layout...much less work for you, the programmer, and everything is very maintainable, imagine that!

    Perl is highly flexible and can be very modular if you code it properly. My web page isn't a great example, but I do use one module on it :) http://freefall.homeip.net/about/ Real projects I've been a part of, however, have some very nicely abstracted OO routines that allow programmers and layout folks to work together very efficiently using CVS.

  47. Re:Scalability Myths by Anonymous Coward · · Score: 2, Insightful

    I wrote the C++ daemon. The question here is not "Perl is scalable" or "C++ is scalable". The question is "is the site scalable"? Use the correct tool for the job. Without boring you with too many details, the search engine kept two large in-memory indices. A perl script (best tool for the job) extracted the search data from the database, built the indices, wrote to a file. The C++ server read the file (in about 7 seconds) in was serving away. Why C++? It was easier (for me) because I had more C++ experience than Perl and I prefered the ability to share the index in a threaded environment. I would have been much harder to write the database extraction and Index build in C++, and much harder (for me) to write the daemon in Perl. The right tool for the right programmer for the right job. An aside: I have just finished my first Java/JSP project. While the java solution was nice, I found the Perl solution easier because of the nicer syntax for hashes and Template Toolkit is awesome.

  48. Re:Servlet container maybe... but application serv by Ian+Bicking · · Score: 4, Insightful
    Unfortunately, unless you are selling goods in a very straight-forward way, e-commerce also includes content management, which is not a Solved Problem.

    If you are just selling products in a plain way, you can get simple enough e-commerce. But if you want to provide good e-commerce that ties in with other content, again there are no out-of-the-box solutions. Also, you have to be happy with a out-of-the-box appearance, where you get to control a few graphics and colors on the page, but the essential layout is fixed. It's usually (but not always) pretty easy to change layout -- but that's still not really out-of-the-box.

  49. Interchange by danpbrowning · · Score: 2

    Interchange (ic.redhat.com) is much better. :-)

    --
    Daniel
  50. Perl vs. get char by Animats · · Score: 2
    "s///" ...
    That's only useful if the "s///" operator controls an inner loop. If you have a program where the high-speed loop is the outer loop, like a parser, that trick doesn't help.

    Go read the comments for HTML::Parser before the C version.

  51. Re:I love Perl. by vanza · · Score: 3, Interesting

    I can't say I agree that JSP, ASP, PHP, or any of the *SP family of languages are good design. They encourage exactly the opposite of MVC, they encourage you to mix HTML and (non-display related) programming code.

    As it has been pointed out, at least with regards to JSP, "allows" does not mean "encourages". And talking about MVC, you should take a look at one of the many "MVC"-like implementations for Servlet development (be it using JSP or not).

    --
    Marcelo Vanzin
  52. Not impressive, if those numbers are right. by mgkimsal2 · · Score: 2

    This still doesn't seem that impressive. I mean, the SIZE of it is, but it doesn't really 'prove' Perl (or anything else) is 'scaleable'. The design approach is scaleable, with enough money.

    If they really were doing 2 million page views per hour, that's about 600 page views per second. Across even JUST the 40 'dynamic page builder' servers, that's 15 pages/second. If you include the reported 20 static page servers, you get 600/60 = 10 pages per second. Certainly nowhere near taxing on a dual P450, ime.

    What this really points out is that people need to spend more time engineering solutions rather than buying 'scaleable enterprise fill-in-the-blank' for hundreds of thousands of dollars. Again, just mho. :)