Which CGI Language For Which Purpose?

zpengo asks: "The blessing and curse of CGI is that there are dozens, if not hundreds, of ways to accomplish any given task. All languages have certain strengths and weaknesses, and there is no "best" CGI scripting language. I'm wondering what the community has learned about CGI languages through their experiences. Which languages go best with which tasks? (Also, are there any less-known languages that work well for particular circumstances?)"

Comments

Like Perl and Java Servlets--Love PHP

[Anonymous Coward]

I've used Perl before--it works well for highly functional pages where you may need to do tons of regular expressions. It doesn't scale very well though, even with mod_perl. And the object model isn't quite as elegant as in other languages. Java servlets scale well, but Java is such a clunky language for text processing that I only use it for middleware applications (usually database related). JSP has the same problems. PHP has, IMHO, the best combination of a great object model, great scaleability, and great templating. You can write elegant code and still not get pissed off when your designers request yet another interface change (It's so easy!) AC

Re:Like Perl and Java Servlets--Love PHP

[dlc]

• [Perl] doesn't scale very well though, even with mod_perl.

This is false. Apache with mod_perl runs at the same speed as Apache -- the combined speed of your computer, network connection, disk, etc. Yes, you can write slow mod_perl stuff (if you use it as glorified CGI), but mod_perl itself has no such limitations.

mod_perl scales very well -- Boston.com serves a few million pages a day using Apache/mod_perl. It's all about how you write your Perl.

But, I will give you this much -- if you run CGI scripts under mod_perl, they do not run nearly as fast as they would if you wrote mod_perl handlers, though they still run much faster than CGI.

How can you dislike the JSP way of doing things, but like PHP? They use an identical model. PHP suffers from many of the same problems as CGI (logic and HTML combined in the same place, etc), but the speed issue is definitely not one of them. However, PHP, even well-written PHP. will never be as fast as a well-written mod_perl handler, for the simple reason that mod_perl handlers are compiled once and live in memory for the life of the Apache server (parent) process. There is no file to read every time it gets called. Plus, PHP (and CGI, and JSP, and servlets) have the limitation that they can only produce output. You want to modify how the URI gets translated into a filename? You can't do that in PHP. You want to customize the authentication process? Nope, you can't do that in JSP. You want to modify the environment for all the other handlers? Not is ASP, you can't. For anything other that generating output, you have to use either C or mod_perl. And the beauty of the whole thing is -- yup, you guessed it, it runs at memory speeds -- no filesystem accesses (unless you write them in specifically), no scripts to stat, no logic/HTML combination to parse.

You are right about Perl's object model, though.

darren

Cthulhu for President!

Re:Like Perl and Java Servlets--Love PHP

[maraist]

Sorry, but this isn't my experience. Perl is interpreted with very few optimizations. There are a good number of important calculations and operations that just do not work quickly in perl no matter what you do. FFT calculation, and graphic generation, for example are not going to go fast no matter what. Obviously, these types of modules should be found in C-compiled perl-module forms since they are so high profile ( though freeking gif image support has resently been discontinued ). But you should also be able to find these C front-ends in most any scripted / CGI-friendly language.

All Apache-mod_perl does is remove the over-head. Well, there is very little overhead to perl to begin with. I've run all sorts of comparisons between java and perl, and for small - medium sized scripts, compilation is barely noticable. The slowest parts tend to be in database reconnections, forking off seperate processes to do simple operations like file grepping, finding, etc.

You get past all of that with PHP and java-servlets though, so everyone is really on a level playing field. PHP even makes use of perl reg-exs ( though I'm not sure if they use the same optimized engine, even though they have compatible syntax ). It really comes down to the advantage your language has to your type of solution. For example, I have created a very powerful web templating system in perl which takes HTML and imbeds them into other HTML files all the while emulating both static-HTML and perl-CGI ( really just a glorified extension to Apache::Registry and Fast::CGI ), but I found that the same algorithm would run incredibly slowly in a java-servlet world because of the un-optimized string parsing in Java. But if I redesign the java-system so that it uses a truely optimized 'parser' then I can get better performance than even in perl.

I don't have numbers to back me up. But assuming a multi-CPU configuration, ( perl-threads still being considered beta ) a well designed java-solution will outperform a mod_perl solution ( especially since java-servlets can transparently span multiple computers, either by resource pooling, or through RMI segmentation / pipelining of the logic ). You can achieve this sort of distribution in perl ( I've successfully tried ), but it is non-trivial and not a logical attribute of the language.
Thankfully though, mod_perl gives you the advantage of running multiple independant, concurrent perl-proceses in each Apache-worker. Thus you can make use of multiple CPUs at the process level. The only problem is that you have heavy context switching, as opposed to the light-weight context switching of a threaded model ( such as java-servlets ). You really notice the difference in a thread-optimized OS such as Solaris and associated [uSPARC] hardware ( where the CPU can thread-swap between instructions ). At this point, one file goes out to the DB while one reads from CGI parameters while two or three fight for raw computing horsepower.
Another issue with process segmented workers is the redundant usage of memory and thus the propensity to Virtual memory thrashing. You, of course, always want to put more than enough memory in a server, but you also would like to cache as much of the web site as possible. If you were to scale to 50 workers, your memory contension would be well beyond that of a CPU-cachable range.
Apache will soon be multi-threaded, and perl is making serious strides to make it's threading both safe and portable. 5.6 has a nice multiple interpreter model which works nicely for global variable consistency ( it's main purpose is to emulate forks on win32 machines ). This combination could bring perl almost back up to par, in terms of scalability, as java-servlets. But it is yet to be seen.
For small-scale operations ( Apache-Front-end, HTML-rendering, Business-logic, database-back-end ), where the business-logic is minimal, you can use mod_perl with apache and have the DB on another computer and you have plenty of scalability. To my understanding this is exactly slashdot. It is when business-logic becomes comparible in resource requirements to the other elements that perl really falls short. It's hard to farm out CPU cycles to other machines or CPU's. And if your logic serializes access to data ( via DB row/table locks ) then the ability to finish business logic computation really starts to stand out like a soar thumb.
As we move back into the world of server-based computation ( java-applets and client side web-apps never really took off ), this sort of massive business-logic will become increasingly important. Serious apps are still done in C ( or C++ if you're into extending your code ), but more and more are done in java in order to give you platform flexibility. Perl just doesnt show up in the enterprise model, or even the real service model beyond the simple form entry services like slashdot.
I'm not saying you can't do magical things with perl ( I've already proven to my company that you can. TCP, forks, and threads are your friends ), but the technology does not easily afford you to do this. Java was designed with all of this in mind, they just don't always achieve their goals completely.
I hope enough of you read this and find value in my experiences. I am not an authority, just a power user.
-Michael

Separate code and layout

[Chouser]

I've said it before, and I'll say it again: Do everything you can to separate the program logic (the code) and the HTML layout.

Why:

• The people who originally build, or later fix, the HTML may very well be different than the people who work with the code. The skill sets are almost completely different, but if you let the two comingle, you will require the maintainers be proficient in both skill sets.
• If the look of your site changes but the code and the HTML are entangled, it can be very easy to break program logic while trying to make visual changes.
• If you ever hope to support multiple natural languages, you're going to want your program logic to reside in completely different files from any of HTML or visible text.
• Clean application design and clean web design are almost completely incompatible. If you try to find a compromise between them that lets you alternate between code and HTML, you will get a result that is bad design for both targets.
• Future content formats (such as XML) might be nearly trivial if you can keep the layout (currently HTML) seperate from the code.

How:

• Avoid those seductive solutions that encourage embedding code in HTML. This includes PHP, EmbPerl, ASP, and others (Zope?). Since this question is about CGI, you're already on the right track to avoiding these.
• If you must use one of the above, work very hard to separate the code anyway. Put as much of your code into separate files as you can, and then just "#include" (or equivalent) the appropriate HTML file.
• Use technologies that encourage this separation. My current favorite is HTML::Template, a perl module. This in itself may be a good enough reason to recommend perl as a first choice for most CGIs.

So in conclusion, I would recommend perl along with HTML::Template. This also allows for future migration to mod_perl instead of CGI, giving you a performance upgrade path. Although this is obviously not the best solution in all cases, it very often is the best, and is very rarely the worst.

--Chouser

CGI Languages

[msuzio]

My comments on languages and their usefulness in Web programming (I have to make this short, but I've been doing this for a long time now, so email me if anyone has questions and wants my humble opinion)

Perl
Awesome language. Great library support (stellar, really) and very easy to learn. Stick to good programming practices, avoid the dark side, and you can do a lot with it.
For anything beyond trivial apps, mod_perl or a similar accelerator is definately needed.
Object oriented programming is fairly nice; it allows OOP without being strictly wedded to it, and it has very powerful introspective and adaptive features (closures, ability to interrograte and modify the symbol table so you can actually *add* methods dynamically as the code runs. Wow. Allows multiple inheritence if you really need it).
Database programming via DBI/DBD is very well supported.

Java
Another good tool. A very nice OO syntax, strict exceptions to insure a robust system, and an architecture for using it with a Web server (Servlets). Add to that JDBC (which is well supported, and not hard to use or learn). Also add introspection and built-in RMI.
I haven't used Enterprise Java Beans at all yet, but I like the concept. Implementations seem fairly weak and non-performant so far, however.
I would prefer Java over Perl in cases where I needed to have a distributed system that might talk to other systems I didn't code or have ability to modify. In Java, CORBA and RMI are pretty easy to use, and EJB support will probably be well worth having in the somewhat near future.
Java is a lot more inherently performant than Perl, although the difference between jserv and mod_perl is not all that huge in our tests.

Bad points of Java: the compile cycle bites once you're used to Perl (edit and run). It also is a tad too strict (again, once you're used to a more free-wheeling approach in Perl).

Tcl
Phil at photo.net loves this. He uses it in AOLServer, which handles embedded Tcl (somewhat like mod_perl?). It has the same sort of benefits as Perl (no compile cycle, loose syntax and ability to do things more than one way, several libraries), but I personally never liked Tcl all that much. I wouldn't rule it out, though, check out photo.net.

C/C++
As usual, use this for maximum speed. Despite any claims people might make, Java is even in the best of cases not as fast as C. However, C is not going to be the fastest language to develop in, and since speed to market and ability to adapt are often key in Web programming, I tend to not consider C or C++ a good choice. The lack of a lot of external support or common specs hurts C and C++ -- I'm not aware of a server-independent API for "embedded" C code (like Servlets are for Java, for instance -- code that runs with the Web server persistantly or on some other persistant server). C could be great for implementing a separate server process that front-end scripts connect to from the Web server. Imagine a mod_perl front-end that connects to a server written in C -- this could be pretty fast and have a lot of flexibility.

Just throwing ideas out. Maybe this answers some part of your question??

Oh yeah, speed of text processing!

[msuzio]

Duh. Forgot some good points.

Perl is great for two other reasons: text processing in general is great (the language was built to do that as it's first design goal!), and regular expression support is first-class.

Given that most Web programming is pure text-processing work, Perl has a big leg up there. Java lacks simple text processing (it can do it, but the code is more verbose and harder to write), and C, although fast, has little text-processing built in to the standard library (other than scanf, strtok, and other functions I'd love to forget ever having learned ).

So yeah, another win for Perl. Text processing is easy to code, and fairly well optimized, given that it's the core of the language. I also find DBI/DBD easier than JDBC!

So, to reiterate, my preference is:
Perl, Java, C, C++

But choose wisely! Pick something you feel competant to program in, something that addresses your needs of speed and complexity, and something that is maintainable!

As an aside, I skipped all those "code in page" paradigms: ASP, PHP, JSP. I just cannot pursue those sorts of paths, because in all my experience, the need to totally separate presentation and code is pretty strong.

"Hey, you got code in my HTML!"
"Hey, you got HTML in my code!"

Not a nice mix :-).
(As an alternative, we use a templating language with XHTML + XML that our code can "fill in" values in and branch around conditional text.
Home grown, but very effectively for our purposes)

Python or Perl

[TimeHorse]

Well, to preface my bias, I've no great love for Perl and am a Python-head tried and true.

Now, what you choose SHOULD be based on your server. Although in this case I'm of the Linux mind, I do see MS IIS or whatever as a valid if somewhat more buggy variant on the web-server landscape. Apache is still the best server out there if you want real POWER over you system. So, the first question you have to ask yourself is what platform you will be using. In the Win32 world, you have a lot more choice than most realise because there is not only ASP, as simple Microsoft Scripting language (though I personally abhor it) and the Microsoft Internet Classes for both MFC and COM. One thing you have to watch out for though is depending on the client browser's platform. Relying too much on Win32 could completely alienate Macintosh users, never mind *NIX. However, there exist Win32 ports for both Python and Perl so whichever you choose you should be safe in developing for the target server platform. I personally give extra points to Mark Hammond's WinPython UI. Also note that Zope has been ported to Win32 in case you decide to go the Python-Zope route. More on Zope later.

The beauty of CGI is that it only runs on one or a small number of machines so you have complete knowledge of the architecture on which your CGI will run. Such is not the case for UNIX Shell Script [sh, bash, or *csh], and except for the smallest of Web applications, Perl is typically just as good. Such is not the case with JavaScript or VB Script or even the state of the Java VM of whoever is your client. One thing I will say though is if you're sure you'll have a small client base, you can safely rely on your clients all running the latest version by forcing them to get it from Sun periodically. However, if the client base is large, this is prohibitive. Not only will you get more complaints from people regarding the frequent updates, but also you'll have a greater chance of introducing bugs because the users are not properly updating. The alternative is to stick with the AWT 1.0 distributed with most browsers and suffer all those nasty bugs with a lost of nasty kludges.

Anyway, so I've brought up Java, and now let me relate it to CGI. Basically, the question is, who plays better with Java: Python or Perl. I think given JPython alone Python can take that match, so if your client base is small, or you can suffer all the kludgy bugs of the nasty Java VM distributed with most web browsers, and you want to distribute some of the work over the clients using Java, Python is the way to go.

Next I want to discuss text processing. Text processing tends to be a big part of web development and thus having a good parser is a major benefit. In this contest, Perl wins hands-down. There is no better language for text parsing I know of, and not even Snowball can out-perform Perl in this respect. However, Python 1.6, now in Beta, is supposed to introduce UNICODE Regular Expression matching narrowing the gap between the two languages quite a bit.

Now for size, each language can serve a different need entirely. For small-scale applications, Perl is easy to hack and quickly get your programme out. However, once you know Python, you can code at comparable speed. The Python advantage comes out in the large-scale project. Since Python is a strictly-syntaxed language as opposed to the more free-form Perl, it is typically much easier to understand for the new programmer or developer. It is much easier to debug Python code than it is Perl. However, a skilled programmer in either discipline will code the same application in near comparable time, the training a Python coder to use Perl or a Perl coder to use Python is usually pretty easy to do, though the time to train in a small application may not make it worth it. As a rule of thumb, the larger the project, the more of an advantage you'll get out of Python. Also the more hands you need to develop a project, the easier it will be to co-operate with Python, as the code is easier for an outsider to understand.

Now both Python and Perl have a byte-code and many many libraries, so there is little advantage with either, though perhaps because of the longer life I think we can give reusability over to Perl. After all, more people have been developing Perl than any other CGI language over time. The same goes for third party support, such as connecting to a database. One advantage I think Python has though is the ease with which one can convert Python code to ANSI C++, my personal language of choice. With the Python / C++ combination, one can develop prototypes quite easily and quickly in Python, then slowly move each component over to C++ for speed. [Yes, you could move to ANSI C too, but why would you want to when good C++ code can be just as fast and is a lot easier to understand.] To be sure, you can do the same thing with Perl but the languages [Perl and C++] don't convert as well and the interface, as will most things Perl, is rather kludgy.

Now, there is one issue with web development that neither Python nor Perl do address well, and that is remote development. Typically to do remote development, you have to either SSH / Telnet to the server and do your work there, or more typically run a web server locally and synchronise files from time to time as needed for each version. However, with a tool like Zope, you can do all of your development remotely without having to use ftp or telnet. Plus, Zope is built from Python so you can extend it with packages and external methods written in Python quite easily. The downside to Zope is that for package development it is very tedious to use. The problem with Zope is that for every change you make to a package structure, you need to restart Zope for it to recognise the changes. This can be tedious in the extreme. Another possibility is Microsoft Visual InterDev, which relies heavily on Microsoft components and architectures. I don't recommend it for anything but the most Win32 of circumstances. There is also Web Objects, which can make development very simple but the price / performance ration is usually out of the range of all but the most corporate of servers. However, none of these give the true Web feel of a good CGI script and there is no substitute for the usefulness of Python or Perl.

In conclusion, I'd say you could use either language for just about any web application, but certainly for pre-existing libraries Perl is your best bet, and for long-term development you should go for Python. Whatever you choose, I wish you luck and hope that I can in even a small way have been some help.

Be Seeing You,

Jeffrey.

The list

[angelo]

If you are running IIS: ASP
If you are looking for something easy: Python or PHP
If you are looking for speed: PHP4/Zend Optimizer
If you are looking for flexible: Perl
If you are looking for good XML compatibility: Perl/Java applets/PHP+expat
If you just want to make headers/footers for a common theme: SSI/XSSI

Now, I don't know about most webmasters, but the three I see most often are Perl, PHP, and ASP. I have to admit, ASP is great for connecting to databases, as long as you grab someone else's code. PHP has built-in ODBC connections, and Perl has modules for it.

Perl seems to have the most free scripts written for it, followed by PHP. ASP seems to show a strong commercial following, which is not a surprise.

If you know C, PHP is easy, and Perl is as well. PHP is a bit more strict than Perl. Perl has more than one way to do things down to an art.

ASP is OK, but if you have a choice, stick to plain HTML. We have a phrase when it comes to ASP in our office. The phrase is "what were they thinking?" and it applies to anything from the choice of VBasic, to Management's deciscion to implement it in the first place.

I prefer PHP, for its flexibility, it's portability (moreso under version 4) and its ease of use.

Ask Slashdot

[DonkPunch]

Dear Slashdot, what "Ask Slashdot" question would be most likely to generate a flamewar?

1. What editor should I use?
2. Which window manager should I use?
3. What license should I release my code under?
4. Which programming language should I use for _____?
5. Which Linux distro should I use?
6. Linux or GNU/Linux?
7. Big-endian or little-endian?
8. Buy or lease?
9. Boxers or briefs?
10. Blonde, brunette, or redhead?
11. Paper or plastic?
12. Ginger or Mary Ann? (Ladies: Professor or Gilligan?)
13. Tastes great or less filling?
14. How many licks does it take to get to the Tootsie Roll center of a Tootsie Pop?
15. Star Trek, ST:TNG, Deep Space Nine, Babylon 5, or...?
16. Shaken or stirred?
17. Color or monochrome?
18. Frames or no frames?
19. Country or western?
20. Walk or Don't Walk?
21. Dr. Drew, Dr. Ruth, Dr. Laura, or Dr. Suess?
22. Joystick, mouse, or keyboard?
23. Cooperative mode or Deathmatch?
24. POP3 or IMAP?
25. Hex, Octal, or binary?
26. WHAAZZZzzzzzUUUUUUuuuuP?
27. Troll or Flamebait?

(Sorry, I got a little carried away.)

PHP - how to avoid mixing code and HTML.

[at-b]

Why, oh why?

For some bizarre reason, people in this thread keep insisting that the best choices for web scripting languages are those that don't mix the actual code (aka the 'logic') and HTML code.

And of course, they have a point. Unfortunately, they then argue that PHP is a poor choice because you have to mix PHP logic and HTML code in pages, (usually ending in .php3 or .php or whatever), which leads to obfuscated code, poor maintenance, and difficulty in separating the coding bit from the designing bit. And indeed, that's a bad thing, since your average interface designer will understand very little about those odd DB calls in the page, and will want to concentrate on getting his DHTML to display right.

So, PHP is a bad choice, right? Wrong.

Most people here seem to be completely oblivious to the fact that PHP3/4 can use the FastTemplate.class, which is (for Perl people out there, just another module) used for just that purposes - creating templates that don't have any PHP code in them. Instead, they have XML-like PHP calls in all the right places, and nothing else.

Example: Let's say Slashdot was coded in PHP, using Fast Templates. I haven't looked at the Slashcode lately, but I'm guessing that at the top of the page, it references your currently-used login name, and displays it. If Slashdot were coded in PHP, it could either have

get_current_loginname() ?>


or whatever. PHP code, right? Ugly. Don't want to mix code and HTML. So instead, we're using Fast Templates.

{DisplayCurrentUsername}

And that's pretty much it. You can predefine dynamic rows, blocks, whatever, and play around with variables inside them, but you simply use { } to denote a dynamic Fast Template call, and that's it. The entire code side is then handled by another fast template code file, call it index.php or whatever you want, where the defined { } variables are defined and coded in PHP. Which means full separation of code and HTML. When your designers start messing with the page, all they have to do is to remember to insert very short, simple calls like the ones above wherever they want to have them. They can even use font tags or any other mean of design to change the { } output in any way they wish, so the entire design part is up to them, and the coders don't have to worry about Netscape4x backwardscompatibility, etc.

And btw - Fast Templates really are fast. With the upcoming PHP4 Zend optimizer, tests have shown 100%+ speed increases over ASP, and it's all free, and well-documented. PHP has great database interaction modules, and for a site like Slashdot (or, to use an example that actually uses PHP, Freshmeat) there is no faster solution that PHP+mySQL. Many of the sites out there that have filenames ending in .tpl are in fact PHP template sites. No obfuscated code, and clear separation of form and content - isn't that what modern web coding practices should be about?

But of course, if you do choose that path, you will have to explain to your boss why that critical project came in way under budget, and much faster than expected. Won't that be just horrible? :-P

Alex T-B

Re:CGI is the most improperly-used term on earth.

[gothic]

It defines the INTERFACE between the web server and the 'CGI application' (meaning, an application using the CGI interface).

I don't mean to sound sarcastic, but the 'CGI Application' is a script, therefor it's a 'CGI Script' ...Make sense? You write scripts in languages that implement CGI, and you write it for the sole purpose of being on a webpage, and going through CGI. Therefore, it basicly becomes a CGI Script in name.

It's kinda like 'Coffee cup' ...The cup isn't made out of coffee, but that's what the name *technically* implies. It holds coffee, so it should only be called a 'cup containing coffee' .. Never a 'coffee cup', or a 'cup of coffee'. Welcome to the American language.. =]

PHP

[Phrogman]

If you are building webpages I think the best tool is undoubtedly PHP. It has all the power of perl (with the exception of regexes which are not as evolved) and it is far faster to develop in, and probably to learn as well.

It supports multiple different databases, including mysql, postgresql, as well as any ODBC compliant db.

Version 4, due to be released this month I believe (its currently available as RC1) offers an optimizer (available at Zend.com), and will come with a compiler as well.

It easily integrates directly into your HTML code. It is multiplatform, and its free.

Its main advantage is the ease with which web applications or websites can be developed. I am currently working on the software for the backend support of several websites, and using PHP I am able to develop this application very rapidly. Far faster than I might be able to do with Perl.

Obviously for those who already know Perl, they will be able to develop just as effectively in that scripting language anything that can be done in PHP. But for the new user seeking to learn a scripting language, PHP cannot be beat.

Sort of

[skew]

I hope you're not saying that CGI script is an invalid term. It is a script that conforms to the CGI protocol, not a script that implements the CGI protocol.

However, you're right; this question is clearly misstated. There is one clear favorite language for developing CGI scripts: Perl. You can write CGI scripts in virtually any language (that was the goal when defining the interface), but few people do. Not with CGI.

However, there are a wide variety of languages for web development, including Java (servlets and JSP), VisualBasic (ASP), PHP, ColdFusion's CFML, C/C++ (via NSAPI, Apache module), etc. Few people use these languages for CGI; each of these languages has its own respective interfaces for communicating with the server instead. (Incidentally, so does Perl: mod_perl or PerlEx.)

For more info:

• Original CGI spec
• CGI RFC Project
• O'Reilly's new Programming CGI (2nd Edition) to be published this summer (*cough* ;-)

- Scott Guelich

I recommend VBScript!

[dmccarty]

For serious code-freaks like me, I really recommend VBScript. It has powerful features, like being able to integrate your operating system with your Outlok address book. I was recently able to write a powerful script that sends itself to all of your friends, thus proving the power of the Windows Scripting Host! (Unfortunately, this powerful script also sent itself to all of my friends' friends, and that started to get crazy. Then these security companies started calling me, and now the NBI in Manila is cooperating with the FBI? Puta'ng ina!)

Oh, where was I?--oh yeah, back to how powerful this scripting tool is. It's really powerful, and I seriously recommend it! (In fact, my boss said that when I come back to work after my 10 years in the "big house" it will be even more powerful than it is today.)

--