Slashdot Mirror
What Are Good Web Coding Practices?
brink asks: "Recently it's seemed as if, due to larger and faster computers, lean and mean code isn't such an issue. However, it occurs to me that there's an overlooked area where there's little or no mention of the importance of efficient code: Web sites. This isn't so much in terms of HTML (I know that webmonkey had an article about efficient tag layout), it's more in terms of Perl, PHP, or whatever. It seems to me that the fact that your code has the potential to be executed in thousands of instances concurrently doesn't impress on anyone the necessity for uberefficient programming. Dynamically generated pages and DB querries are good examples, or just plain memory usage of your code. So my question is as follows: are there any good works to read which focus on programming for the Web, as in avenues the programming medium affords? Have I just been oblivious and missed them?"
Philip and Alex's Guide to Web Publishing is a great book, but it's not a good reference if you're looking for nitty-gritty coding knowhow. It seeks to avoid that stuff, since you should be able to think about building web sites on a higher level, where the particular language doesn't matter.
However, if you'd like to be a web coding ninja, do problem sets 1-5 on this site. If you don't have the exact environment in which to complete them (AOLserver, Oracle, CyberCash account), improvise. After completing those, you should be able to tackle anything, and you'll know how to code a great site.
Well, I guess they get away with it because, simply put, Java works on the server-side. Time to post the standard response to this sort of thing...if minimal waste of cycles is _all_ that mattered, we'd all be coding in assembly, or be wiring the code into hardware. We don't, so obviously Sun 'gets away with this' because they offer a fine product that panders to such banal needs as "maintainability", "time to market", and "reusability".
And, BTW, there is a difference between Java being notorious for being slow (which it is) and Java being actually notoriously slow (which it isn't).
Back to the topic, as far as website coding is concerned, most of the best practices having to do with any complex software system also apply to website development. If you're doing object-oriented development, I especially recommend practices such as:
1) Learning and utilizing established Design Patterns where appropriate. Don't forget that they've become "established Design Patterns" because over time, they have demonstrated their value again and again. There are plenty of books on the topic, the most famous (and IMO useful) of which is Design Patterns: Elements of Reusable Object-Oriented Software
2) Refactoring. Essentially, this is going through code that has already been written and aggressively removing duplication. This results in a cleaner system implementation that is far easier to identify bottlenecks in and tune. There is a good book on that by Fowler that is just called Refactoring.
3) Where possible, avoid "new" in favor of "proven" technologies. "New" always implies risk. Credit is often overly awarded to those who "invent" or use "novel" solutions to problems when other, time tested solutions might have done the job better.
4) Try to build a clean separation between your data, your logic, and your presentation. This will greatly help with code maintainability.
It's important to note that not all of these solutions will have an immediate, positive impact on the performance of your website. In fact, in many cases it is just the obvious. However, if rigorously implement these solutions, you are likely to build a website that is easy to optimize, and that scales very well with more hardware.
"UNIX" is never having to say you're sorry.
Two things: first, Perl isn't really slow; when you run it, most of the time is spent running Sarathy's heavily optimised code to handle things like hash lookups and regexes, not your punny little userland code ;) It only _seems_ slow because of the overhead imposed by having to spawn a new interpreter and recompile the script into bytecode every time the script is run. But most of the problem is easily fixed by mod_perl.
:) Instead, perlcc embeds an entire copy of the Perl interpreter, plus a shitload of unnecessary modules, along with your program in bytecode-compiled form. The result is bloated, buggy, ugly and generally not much faster than just running your script. So it's not of much use, but if you really want to do it, you get to.
Second, yes, the perlcc utility is able to compile a Perl script... but not by translating it to native code. No, that would be just too easy...
(P.S.: Cool username... I've always preferred post-Syd Floyd myself, though.)
To the editors: your English is as bad as your Perl. Please go back to grade school.
"Undegreed" is the most worthless term I've ever witnessed the application of in reference to the ability of an individual to write quality code.
I'm not going to purvey any content in this post other than the degree to which I am dumbfounded by this assertion: VERY.
Wow, Bob. You're astonishing.
"Why use HTML when javascript, cascading style sheets, java, cookies, imagemaps, and shockwave will do?"
Why do web developers feel the need to use every technology available on a website when they usually didn't need anything more complicated then plain-ole HTML? I dono how many times I've seen implemented in javascript the meta refresh tag!
Add on to the fact that most sites have become unnavigatable by text browsers and their users (such as the blind). My old boss used to have a saying: "A picture is worth a thousand words, unless it's a picture of a word".
All this 'rich content' that is arguably unnecessary and excessive means that the web is just that much slower, needing even more bandwidth to get anything done.
And my last gripe is about modifying HTML that was created in a HTML editor. Did any of the programmers who coded up these editors ever hear of wordwrap?? Every time I edit a file I have to re-format the whole thing to make it inteligable.
Yes, I'm griping. You can think of me as the crotchety old man who sits on the porch "Yeah, I remember the good old days, back when people edited HTML with 'vi' and we didn't have all these fancy-pants 'wizards' to generate webpages; you could search the web then and find the information you were looking for instead of a bunch of infomercial-like websites with too much animation hawking things. Why it must have been waaaay back in 1995, yeah, Yahoo was still a stanford tilde account back then."
-- Greg
Slashdot, would a spell-checker for posting be too much to ask? It's not rocket science!
A lot of people have said that the bottlenecks tend to be generating output, making database connections, and the like, but that is often not true. Generally, the bottleneck is the network connection. The way I often think of pages is in terms of the connection speed of the server. I have done work for people with slow connections from the server to their backbone, and in cases like this, connection time becomes less important than network speed. So I send the header, send some content for the top of the page, and then make my database connections, and do my calculations.
Even though it's often frowned upon, I like to use a lot of includes that are generated by the client whenever possible. That way, the multiple hits to the server (i.e., more network traffic) can make a slower server seem faster. When it comes down to it, when poeple say they want the web to be faster, they mean a fast experience, not a fast server.
Just something to keep in mind.
darren
Cthulhu for President!
(darren)
There is a classic CS quote that says that a program spends 90% of its time in 10% of its code. Make this code run twice as fast and you will nearly double the speed of your program. Optimise everything else and you won't see any difference at all.
The important thing is to write clean, readable, code. Do not be overly concerned about going to great lengths to improve speed during development. As soon as you have a working system, build a set of automated testing tools and generate some benchmarks. Determine where the bottlenecks are and try to fix them. If you don't know what is really slowing down the system, you will waste many hours optimising code that has no need to be optimized!
This is a trivial operation when using a language such as C with no threads and/or accesses to remote resources. (Although despite this fact, I still see people trying to write optimized spagetti code and never touch a profiler.) Things get a little more difficult when you start talking about database-driven web pages since there are so many more pieces to analyze.
In addition to analyzing the execution time of your script, you must also look at the execution time of database queries. Unfortunately, it is not always possible to profile individual queries, and some databases such as Oracle are highly tunable (i.e. you can tune the system to acount for perhaps poor database design.) When dealing with databases, you are often disk-bound as well, so applying the same profiling techniques to your hardware will often provide good results.
When your database and web server are on the same machine, just using ps or top will give you a good idea of where your cpu time is going. And don't forget to make sure that you aren't swapping. If you are swapping, optimise for memory use or buy more ram. If you've maxed out the RAM on your machine, consider splitting the application onto multiple machines. Sometimes it is cheaper to solve a problem by throwing more hardware at it, but this should only be considered as a last resort.
BTW, I'm assuming you've already done the obvious and are taking advantage of the best technologies such as mod_perl or Java servlets. Forking several copies of Perl just to generate one web page just isn't terribly smart...
-p.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web.
YIKES big guy back yer arse up please. You have got a chip on your shoulder about undegreed programmers.
Ten years ago, that attitude would have worked.
Let me tell you a little about myself.
I am 19 years old and I DO consider myself a web developer and a pretty damn good one.
No I do not know everything about computer science. No I dont know a WHOLE lot about math and the nitty gritty details of algorithms and all that. I DO make it something I work very hard to understand. I strive to improve my code and my level of professionalism ALWAYS. I study algorithms, and programming a LOT.
I try and give clients or my work VERY debugged code with no errors, make it look professional and make it work in an intuitive and effecient manner for them.
Effciency of my code is ALWAYS a concern to me and I try my best to make optimized code. I apply what I know about algorithms and data structures to make my code as maintainable as possible. I dress in casual business clothing and I in NO way claim to know it all. And if imasked if I can do something and I cannot I say NO I can't do that right now, I am capable but I would have to learn how. That is my business philosophy and its gotten me a lot of work and me and the clients have been very pleased with the results.
In your mind I am a 'kid' but I am a professional and I attempt to also incorporate ethics into my code use and my business practices and manners.
And something people do NOT like to admit.. Some people my age are VERY capable of fulfilling senior web development positions No I do not match some god forsaken profile for education nor experience.
The sadder part is I have met people with CIS, and CS degrees who are just absolutely clueless. They needa be beaten to DEATH with the cluestick. It causes me great pain because I know they make a lot more than me and here I am training them. Yeah I know a degree says something about a person and that they were abble to attain it.. But open your mind a little.
There are some of us that are professional and professionals.
So I am 19 with two years of 'paid' experience. Does that make me less of a professional if I admit my limitations and am constantly expanding my knowledge and limitations to beyond even my expectations?
I was stuck into a huge business APP and I worked my ass off.. but I groked it and was a productive member of the team. *shrugs* Your comment was very crude and not well thought of.
Yes I understand your viewpoint but you can say the same about ANYONE who claims to be a programmer.
Jeremy
Phillip Greenspuns work is highly recommended: Go check it out!
The problem is that most dynamic web pages (Active Server Pages,php3,Cold Fusion,perl,whatever) are produced with scripting languages that pale in speed and functionality when you compare them with a compiled piece of executable code or a highly optimized SQL stored procedure.
Also, code is just plain ugly in ASP or those other scripting languages. You don't want to cram a lot of logic in there, because it's hard to maintain
So, the solution is to remove as much functionality as possible from your server-side scripting language. Put all of your buisness logic and CPU-intensive operations in SQL stored procedures, compiled code, or some other equivalent.
In industry buzzword-speak that's 3-tiered architecture. (Tier1: ASP,php,etc. Tier2: SQL stored procedures,compile code Tier3: the database
The end result is that your web site is :)
a. easier to maintain and
b. you're reaping the performance benefits as well
OtakuBooty.com: Smart, funny, sexy nerds.
Don't optimize a single thing until you find there is a problem! Instead, write clean readable code that you can work on later if you need to.
I've spent the last couple months fixing and replacing a bunch of web code written by a guy who thought performance was everything. You have to get it working first. His code does the wrong thing incredibly fast. Bloody Fucking Useless (TM). I'll spare you the details of his N-Tier Network with Application Servers and crap. I've rewritten half of it in plain old (slow) ASP, and I'm getting much better performance and an easier time fixing bugs.
Once it works, evaluate whether you have a performance problem. If you don't, then leave the damned thing alone.
If you do, start profiling. It's not wasting time where you think it is, guaranteed. Ask the super dawgs of optimization (the John Carmacks, Michael Ahbrashs etc...) how many times they just "knew" where the problem was only to be totally surprized once they broke out and profiled it.
Anyone complaining about how Perl, Python, ASP, PHP or whatever is so much slower than C/C++ is really missing the boat. Web pages spit text out the front door, and talk to files or database through the back door. There is seldom an algorithm to be seen, and most the of processor time is spent doing something other than interpreting your script.
It's a lot easier to optimize a query, add an index, put more memory in the server, disable the screen saver, whatever than it is to rewrite everything in C++ using ODBC.
Once you do rewrite it in C++, you're back to where you started. It's still too slow, and now you need to restructure the query, add memory, turn off the screen saver or whatever...
One other thing, don't believe the hype. Everything latest and greatest is not necessarily fastest and best. Microsoft get's paid when you replace your old perfectly good stuff with whatever they've just released. No one is ever going to convince me that ADO is faster than ODBC when ADO uses ODBC to do its work. What the hell is MTS good for? It's a solution to a problem I haven't seen.
bob you sound bitter. some kid steal your job?
last time i was in college, i missed the html degree signup sheet....damn.
"This is where god would go if he wanted to get off blow!"
In most cases the easiest thing to do is simply buy more hardware and replicate. This is easy to do if you're serving up static (or even dymamic) web pages that don't need to share any state across multiple web servers. This is feasible because in many situations the bottleneck of the site is the web server and dynamic page generation itself, not access to the back-end database. For really large sites, a replicated/clustered database is going to be needed, not just a single machine running MySQL.
The Application Server industry has started to provide some solutions for building scalable web sites -- for example, IBM WebSphere, BEA WebLogic, and other systems provide a platform for building web-based applications, which generally consist as a middleware layer between an HTML/presentation 'front end' (i.e., a web server) and a database 'back end'. These middleware systems support replication and clustering to increase efficiency. Replicating the front-end web servers is easy, and many products support this. You can even buy fancy network switches which load-balance HTTP requests across multiple web servers. Replicating the back-end database is much harder, but that's how companies like Oracle earn their money.
Not surprisingly nearly all heavily-loaded Web sites have to do a huge amount of work to get all of this working right. One thing you might ask is whether the process of building a scalable Web site could be made simpler. That's the goal of the research project I work on at UC Berkeley, called Ninja.
Matt Welsh, mdw@cs.berkeley.edu
There are books written on how to optimize code in general:
-Code Complete by Steve McConnell, Microsoft Press
-Writing Efficient Programs by Jon Bentley, Prentice Hall
-Flooring It: The Optimization Challenge by Michael Abrash, from PC Techniques 2, no. 6 (February/March 1992)
But code tuning can have varying results under different languages and environments. A lot of people seem to think that code optimization is unstable and difficult to maintain.
There is sometimes a difference between "efficient code" and "good code". Code may be efficient, yet hard to read and maintain.
To know what techniques are the best, a lot of experimentation is in order. It is also a good idea to become part of a community of experienced web developers, and learn from their experiences.
The greatest boost in performance I have seen in my web experience has come from using the mod_perl module for Apache, not necessarily because of anything I did in the code itself.
In dynamic sites the bottleneck is generating the content (running servlets, accessing databases, etc.), so it makes sense to optimize there. Unless your pages is /really/ complex, or the browser /really/ stupid, rendering time is negigable. And in any case, you can't do much about it even if you /do/ have the best code. Pure HTML just isn't that much of an issue. Moving into XML and XSL, and DOM manipulations, though, client-side performance may become more of an issue.
It's 10 PM. Do you know if you're un-American?
I've always like Phillip's Greenspun's ideas on web development. In a nutshell, he preaches reliable and efficient code/tools. He can be a bit biased but mostly his ideas are good (according to me).
His book is available online for free.
That which does not kill me only makes me whinier
That's so true.
My old company assumed that just because someone was a "web designer" and maybe knew a little javascript, they were qualified to do programming on complex web sites (e-commerce, etc.) They're not anymore qualified to do database work than any old artistically inept programmer is qualified to design a new logo for a company.
Companies need to wake up and realize that it takes two types of people to build the web: "real" programmers, and graphic designers. Of course some sites only need one or the other, and some people are talented enough to fall into both categories... but don't assume that Joe the Graphic Designer can code a linked list or a SQL statement...
OtakuBooty.com: Smart, funny, sexy nerds.
As far as HTML on a page, I found that you should optimize for Rendering Time, which isn't always the smallest page, esp. when it comes to tables.
For server side programming, unless every page is generated dynamically, you should program it the easiest way possible and throw some CacheFlows in front of it.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web.
--- Speaking only for myself,
However, some specific things I can think of are:
Seperate out HTML/text and code as much as possible. Ideally, there should be no, or almost no, HTML in the code. This makes life easier for the HTML designers, and also means you get bugged less to alter the code to alter the output.
Make it as simple/easy/reliable/quick/painless as possible to stop/start any server processes. I've worked on a project where we had a perl process, but stopping/restarting had some unpleasant side-effects, meaning we had to restrict doing such an operation to the quietest hours.
Make sure you have a good and accurate test environment. Having some tools to help you debug the running site itself is very usefull - retesting doing a whole load of operations by hand is very labourious.
Test and code for both availability and scaleability. You want good resiliant code with minimum service downtime (which means using reliable OSs and development enviroments - don't just choose the latest).
Make sure you can say "no" to the management. You *will* get PHBs trying to force new features on you when the current ones aren't even working right yet. Make sure your code is flexible though as it'll be changing/evolving a lot anyway. Also make sure you don't get stuck doing fire-fighting - ie stamping out so many little bugs that you can't fix the fundamental problems. (this is the kinda problem M$ has with it's code development...)
Personally, I wouldn't say that C, C++, Perl and PHP promote good programming practices. Java's certainly much better here. I'm not sure about other development environments. (don't worry about server side Java's speed as on a good server, it'll be close to C/C++. I'm not kidding.)
Most scripting languages like Perl or Python need to be compiled into bytecode, and then executed. The overhead of compiling the program every time it is executed can be quite a burden, especially for applications like CGI programs where the program is started very frequently but runs only for a short time.
/usr/bin/perl for each hit. Again, this becomes significant once you start getting lots of hits.
Even if you can cache the compiled bytecode (as Python does), there is still the overhead of starting
The best thing you can do is use something like Apache's mod_perl which avoids the overhead of starting a new perl process for each page, and also the overhead of compiling the program each time. Similar features exist for all popular scripting languages and Web servers.
Once you've done that, you may find that there are other startup costs you can factor out. For example, does each hit make a new connection to the database? Depending on your application you may be able to get away with some sort of local cache, either on disk or cached in memory in some way.
Finally, you can make your site faster to _use_ and save the user's time if you make your pages appear gradually. If you're churning out a long, complex page, try to make it appear as it is computed, so the user can start reading immediately. This is partly a matter of choosing layout that web browsers can render incrementally, and partly of how you write your program. (Slashdot's comment display, at least in Minimalist Mode, is very bad at this. In Netscape 4.x you have to wait for the whole 100Kbyte or so before seeing any of it.)
-- Ed Avis ed@membled.com
Yes, it is more expensive. But, many companies are racing to be the first for survival, and they have been blessed with enough VC money, so that it is irrelevant that it takes hundreds of thousands of dollars to pick up the performance. Time to market is much more important, and having programmers spend time performance tuning takes away from that.
One clear example here is Java vs. C++. Sure, Java is much slower than C++; but developing something in Java takes orders of magnitude less time than in C++. So, it is better to throw money at hardware to cure the performance problem, but will get you to market sooner, than code in C++, which results in faster code, but takes much longer.
Of course it's not quite this simple. You can't just throw hardware and money at every problem. Obviously your application can't be so slow that no one will use it. And key performance problems are first at the high architecture level, which are mostly language and platform independent. You will always have to spend time performance tuning at some point. But initially---even long term---time to market is more important.
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
I went through so many hairstyles, fashions, music trends, the occasional piercing and tattoo that my family started referring to me as Rodman a number of years ago. However, I've been writing code for 16 years now. And I'm good. I'm good enough that now I also get to see the streams of applicants coming in and get to decide whom to hire and how to put together the teams.
Get this straight:
If I get a group of "green-hair"s in, I'll hire the ones that can code, work together, and who won't steal from the company.
If I get the GH's and some poindexter's, I'll hir the ones that can code, work together, and who won't steal from the company.
If I get a group of poindexter's, I'll hire the ones that can code, work together, and who won't steal from the company.
Part of the problem in hiring is knowing whether or not the prospective (green haired or not) can code, work with others, and won't steal from the company. Your credentials, I'm afraid to say, are what get you in the door. [ If you know someone who can recommend you, that can get you in the door as well, but I really consider that a social credential more than a side-stepping of the system. ] If you've never done anything productive, you could very well be the best programmer in the world, but you've never worked on anything. This means (to me) that you probably don't know how to work with people, or how to pursue joint code development, or what deadlines are, or why tasks are prioritized, etc.
A degree, certification, or award, is an indication that you have completed some structured program designed to increase or test your assimilation of a body of knowledge. Are there people with the proper certifications who aren't qualified to work in this industry? Certainly. Are there people with the proper certifications who are able to work? Certainly. Are there people without those certifications who can do the work? Certainly.
So, certifications are worthless then. Bullshit. Like I said before, they are an indication of completion of a structured program. Given two applicants for one position whom I judge to be of equal capability, willing to work for the same pay, I'm generally going to hire the one with more certification -- primarily because I know that person is more likely to be able to work under deadlines and prioritized sets of goals.
Similarly, given a set of qualified applicants (can do the work, can work together, won't rob us blind, have the same relative qualifications) and too few positions, I will often favor the applicants who put the most effort into applying: well written resume/cover letter, well-dressed, punctual, polite, aggressive (no there's no contradiction there), excited, motivated, etc. These people are more likely to work hard, stay with the company, act on their own initiative, and so on.
The perceived tightness of the job market (and, actually, it's not quite as tight as has been made out) has brought to the average technophile an attitude that not only are they in high demand, but also that their intrinsic coolness, which derives from often dubious technical ability, is a substitute for proof of ability. Only one component of employability is technical competence, the other more important ones include social maturity, dependability, commitment, responsibility, ambition, and respect. Some will certainly whine that technical ability is the most important, but in this day (contrary to what many of you seem to believe) that's simple to find. If that's all you have, you lose out, because the next person in the door has the others too.
"Cause there's 40 different shades of black, so many fortresses and ways to attack, so why you complainin'?"
Check out Philip and Alex's guide to web publishing. This the book you were looking for?
o/~ Join us now and share the software
Some techniques that are essential to web development are:
Often, the speed of the web is, in reality, how fast the pages appear to be. This is the most important thing to remember when designing pages that are supposed to be fast.
darren
Cthulhu for President!
(darren)
I don't know of any books, but here's what I do...
I work for a fairly large software company and we use a Linux/Apache/MySql system to dynamically track and record our production. This database/machine is getting hammered on the backend (data importing) and on the front end (our web/perl interface).
How do we tweak this datatabase to run 5 different programms and still output 1.5 million transactions a day on a 200Mhz machine with 32Mb of ram?
Efficient code!
Nothing different than you would do with your C code, just use smart programming. With perl, use the 'strict' module. Always destroy database handles as soon as you can, don't try to do complex sorts in your code, that's something the database is capable of.
Static if you can...If there is a page on your site that doesn't change very often (ie. it chagnes less than half the amount of times that it is accessed per day) generate it with a server daemon, and then let users access it has a regular html file. This saves CPU time and decreases latency of accesses.
A lot of the web programming starts with good database design. There are countless books about this. If your databse is optimized so that there are no full table scans and relationships are tightly bound, then writing efficient code is fairly simple.
Overall, just keep it simple and efficient.
I am a professional PHP developer and I personally put quite a bit of thought into the code as I write it. I think a very good set of articles which help with the problem you mentioned, at least from one angle, are the Cached Dynamic Module articles at php builder.
Don't moderate this as a Troll or a Flame, I write in Perl most of the time myself. But I've recoded some stuff and tested it, and blew the doors off.