Slashdot Mirror
What Are Good Web Coding Practices?
brink asks: "Recently it's seemed as if, due to larger and faster computers, lean and mean code isn't such an issue. However, it occurs to me that there's an overlooked area where there's little or no mention of the importance of efficient code: Web sites. This isn't so much in terms of HTML (I know that webmonkey had an article about efficient tag layout), it's more in terms of Perl, PHP, or whatever. It seems to me that the fact that your code has the potential to be executed in thousands of instances concurrently doesn't impress on anyone the necessity for uberefficient programming. Dynamically generated pages and DB querries are good examples, or just plain memory usage of your code. So my question is as follows: are there any good works to read which focus on programming for the Web, as in avenues the programming medium affords? Have I just been oblivious and missed them?"
Sorry for Enligh.
Degrees are meaningless. The best web designers I know, none of them have degrees. The people with degrees I know are idiots who practically need someones help to tie their own shoelaces.
Degrees just prove you can recite stuff parrot fashion and remember it for exams. I've yet to meet one person with a degree who actually seemed to have a superior amount of ability compared to anybody else I know.
Degreeless kids... Idiot.
- Turn buffering off whenever possible (e.g., $|++ in perl).
Actually, this may make the response time a bit faster for the user, but the script will run slower because it has to do more system calls instead of filling buffers.--
I support anonymous posting.
I think the problem is that there's a line of what's advanced and what's not? I consider normalizing a database schema to be trivial. It's in one of the early chapters of the first every database book I read, and that was an application guide, not a Database Theory book. I would fully trust that anybody who could solve a differential equation could be taught quickly how to normalize a database, without a college class on it.
As for programming style. You're absolutely correct again, good style is important, design is CRUCIAL, and hacking doesn't work for maintainable code.
Please understand, I don't think that getting a degree is a waste of time at all. I'm in a dreaded middle cube, and if I go around every cube near mine, I'm surrounded by two Princeton PhDs, an MIT Masters, an RPI bachelors, an RPI masters, a CMU masters and I'm not sure of the last person's education. Then there's me. A guy who didn't quite finish college because life got in the way, and suddenly it was more important to be self-sufficient than it was to be degreed. It's amazing the decisions you make when you find out a parent is terminally ill. In my case, my decision was that I'd sacrifice the degree, and get a job to pay to finish my education sometime in the future, thus freeing up extra money to take care of my mom.
Education is useful, degrees are not. Too many people confuse the two.
----------------------------
sphix42 links to the '3rd Incarnation' of the Cached Dynamic Modules. To check out what was written previously, so you can get a background on whats goin on:
#1 Cached Dynamic Modules
#2 Another Look: Cached Dynamic Modules
#3 Cached Dynamic Modules pt 3 (sphix42's link)
-----
If Bill Gates had a nickel for every time Windows crashed...
Granted I'm not part of the goth culture but all of my friends who are are invariably jet-black haired, not green.
How can you say "elegant code follows where memory is pinched"?!?!?! Do you remember working with overlay files? What about writing self-modifying code to get a little more functionality out of an assembly program? How about grouping your functions by where they sit in memory rather than what they did for the program so you could use relative jumps and save a few bytes per branch? How about using video memory to store data?
*shudder*
And this is all from when I started coding, in 1987 or so. I don't remember when memory was really pinched, from the 8-bit days - things were way worse then. Imagine using the RAM on your floppy microcontroller for space. Not exactly what I would call elegant. More like an absolute hack-fest.
Palms have always had relatively large amount of memory for special purpose devices, ESPECIALLY when you factor in the huge amount of ROM they carry, allowing for lots of functionality built into system calls. A good design, mind you, but not having memory has never led to elegant design.
Old Man Shane
The concept of "SQL stored procedures" intrigues me. Do you mean server-side scripting functions that are stored in an SQL database and fetched and run on demand?
If so, do you know of a good reference or set of examples detailing this sort of behavior?
Static if you can...If there is a page on your site that doesn't change very often (ie. it chagnes less than half the amount of times that it is accessed per day) generate it with a server daemon, and then let users access it has a regular html file. This saves CPU time and decreases latency of accesses.
It isn't always possible, but it is good. One thing I do is crontab up the active part, then have them write to files that are SSI'd in. #include file is faster than #exec cmd, especially when the cmd has to deal with net hits itself.
And Mr "Undegreed" up above would get along just fine with Phillip "I'm a godlike creature from MIT" Greenspun ;-) I like his writing, and he knows his stuff, but if you read a lot of it in one sitting you end up wanting to go out and beat up smug ivy leaguers.
OTJoke: "How do you know when you are in the same room as a Harvard man?" "He'll tell you."
"don't fall into the fallacy of believing that Perl can solve social problems. Maybe Perl 6 can, but that's a ways off"
Even though it seems cool to use gobs of server-side java and huge application servers, a simple method like straight php or aolserver with low overhead will help out.
You mean, includes? sure!
Isn't that what the Slashdot effect is for?
I see even classic Slashdot is now pretty much unusable on dial up anymore.
I thought that the middle tier was more of a help than a hinderance by giving you a central place to store the business rules and manage transactions across dispersed databases and servers.
Speed, sure, but the bigger the application the less likely that speed will be the primary driver ...
Why is that, exactly?
It seems to be a common point that people are making, but no one has explained why it is that they are faster (vs. ODBC or something).
Jordan
It depends I seen lets take your example
You have incresed your speed by 5 seconds
5 sec x 7200 hits = 36000 sec
36000 sec = 600 min
600 min = 10 hr
now lets talk milliseconds
15 ms you made it go 10 ms
5 ms x 7,200,000 hits = 36,000,00 ms
36,000,000 ms = 36,000 sec
36000 sec = 600 min
600 min = 10 hr
If you web site gets alot of hits then
it is worth it else don't bother and waste
the time
http://theotherside.com/dvd/
Supposedly, a mod_python is in the works.
-- Thrakkerzog
>Since when do you need to have multiple server architectures run the same code?
It's very nice that I, as a developer, can have an entire copy of the Java-powered website running on my Linux box, and can then move those source files and compiled class file out onto the E10000 for deployment with no headache.
Yes, Perl does the same thing, as do many other platforms. Which only helps to make the point that "multiple server architectures" sometimes actually do need to "run the same code".
>Java IS slow
Java IS _fast enough_ to be usable on the server, where the hardware costs for an enterprise-level web application are negligible. And it carries with it other advantages (such as excellent documentation...there are more O'Reillys for Java than for any other programming language)
"UNIX" is never having to say you're sorry.
(I replaced nearly 80,000 lines of Perl with only 20,000 of Java)
You make some good points, but this one seems suspicious. I have done dynamic web sites in Java, Perl, PHP, ASP, etc., and have found that it generally takes a lot more Java to get the job done.
...Is worth a look.
Went to a talk of Philip Greenspun's there a
while back, where he told a story of a horribly-
written database backend, using the C interface
to Oracle 5, and how the people involved just
threw hardware at the problem until amazon.com
was usable. Said story has an obvious moral.
Personally, if I want speed I use persistent
CGI (process creation is a big overhead on VMS)
and a compiled language like C, C++ or, more
usually, VMS Pascal (don't laugh). But then, I
don't have amazon's money.
K.
-
-- Proud descendant of semi-nomadic cattle-herders.
I agree with most of what you say, that most new technologies make pages more and more limited as far as who can view them, but this is not the case with CSS. A properly coded page that relies on CSS is actually better in Lynx, for instance, than an equivalence HTML page. Why? Well because you can avoid unnecessary usage of tags like in your document that you would use in straight HTML. Instead, you can create a new subclass of other tags which will give you the desired appearance but which will be treated appropriately by browsers that cannot produce the desired style. This is only with CSS 1. When CSS 2 support comes along, look out. It allows you to get rid of the biggest joke in HTML layout techniques, tables. Instead of dealing with putting every piece of content into a table cell, you define the layout in an intelligent way with a style sheet, just like professional publishing programs do. Similarly, when a browser comes along that cannot present it in the desired manner, the content is still readable, even if the browser doesn't understand CSS. Check it out, it is the way we'll be publishing on the web here real soon.
History has the relation to truth that theology has to religion-i.e. none to speak of. - Lazarus Long
And you pay attention to how PHP handles global variables, which is non-standard.
--hunter
RateVegas.com - Vegas Reviews
Your argument about Java being slow has already been rendered mute.
Java on the SERVER is different than the Java app or applet you were running... Also, have you heard of Hotspot? It actually ends up helping quite a lot, especially in server apps where the thing is running for weeks...
Did you notice that EJB is the STANDARD for building really big sites (E-Trade, part of Amazon, part of E-Bay, etc...).
--hunter
RateVegas.com - Vegas Reviews
Zend has two different optimization technologies.
One is the Zend optimizer. It's sort of like Sun's HotSpot. It's an inline optimizing compiler...
The Zend Compiler is something different. It's written to allow people to distribute PHP scripts without the source (ie it generates a closed intermediary format for PHP).
--hunter
RateVegas.com - Vegas Reviews
Maybe you don't need a degree in web design or perl programming, but you could sure use an english class or two.
http://overwhelmed.org
Your apparent hatred for "undegreed kids" and profound love for "proper credentials" is pretty much unfounded. I'm 24 and manage the IT department at a smallish company. I have no degree.
While many of the people who work for me were busily attending keggers at the frat house and taking basket-weaving for the cheap credit, I was completely immersed in an IT environment. Years of practical experience and both hands-on and classroom training with the Army have given me an understanding of the flow of an IT department that you'd be hard pressed to find in a course.
Many, many college grads actually have their heads on straight; but to indicate that a diploma is a pre-requisite for any level of intelligence, professionalism, or ability to perform on the job is sheer foolishness. I think somebody overpaid for the college education and is feeling bad about their lot in life now.
-Jer
This is slightly off-topic. Can PHP put code outside the HTML page, and then the HTML page only has PHP tags that make function calls to the PHP code outside it? This can be done in JSP (JavaServer Pages), so you don't have to put all your logic on the same file as your HTML.
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
Impressively wrong, for a variety of reasons.
Garbage collection being the most obvious, and the largest.
But, with a Netscape Enterprise server (Solaris) running multi-threaded, we were able to make shared read access to gdbm files fast and threadsafe in a very natural way in C. Is perl threading good enough now, and does mod_perl handle multithreaded applications?
Well, I haven't been doing full-time web development work for a few months now (thank Kibo!), but I've heard some very nice things about Perl 5.6's thread support (although, according to the official pages, it's still experimental and subject to change at any time). However, it seems to me that if it comes to a point where you definitely feel the need to have your system itself make use of "userland" multithreading, then you're still better off using C/C++ and a nice little threading library such as MIT's. (Of course, I'm not sure as to how the Netscape server works re. threads... if it handles multithreading automatically for you in an NSAPI app, then I suppose that it should do the same for Perl running on it, no? Then again, I'm just guessing.)
By the way, do any of the "other" popular scripting languages - Python, PHP, Rebol, whatever - support threading?
To the editors: your English is as bad as your Perl. Please go back to grade school.
Find yourself one of these people.
And then punch them in the nose for inventing COBOL.
Conscience is the inner voice which warns us that someone may be looking.
Conscience is the inner voice which warns us that someone may be looking.
-- H. L. Mencken
If it's only one large Perl script to speed up, have a look at mod_fastcgi.
In my application it was *much* faster than mod_perl and the RAM usage is well defined. With mod_perl the server went swapping to death.
Isn't the standard 3 tier solutions more like
Teir 1: client(Browser, etc), Tier 2: business rules (scripting language), Tier 3: data stores (Relational DB of your choice).
I'm sure the client is considered a tier, so if one makes a disctintion between stored procedures and the database system that is executing the queries then you're looking at a 4 tier system.
I think this is a horrible book. The author is extremely opinionated and frequently off base. His assertion that AOLServer will cure all the world's problems is ridiculous. In fact, the entire book seems to be an advertisement for AOLServer. The book is filled with ludicrous gems like "no one uses Java for server-side scripting". Take with a heavy dose of salt...
I concur. The crux of the problem is that web based applications are just that, applications like any computer application. So all the good program development and design principles apply.
A good developer can produce good, useable and efficient applications regardless of the medium i.e. client / server, web based - intra / extranet. The main problem comes when developers aren't exposed to good practices, techniques and methodologies. These can either be taught or learnt from experience. This is also an ongoing task.
I don't consider formal education a sign of a good developer but it does mean they have been exposed to good practices. The really good university educated people I've met or worked with have consistently blown me away.
I've met people from both sides of the draw and either can be as good. The most difficult thing is if a developer ceases to learn / strive either through ego (I'm shithot!) or through laziness / ineptidude.
The main danger is when you get yahoo cowboys who come in bang out a mess that may do the job now but can't scale or be easily upgraded. All too often management don't understand this or understand the difference in philosophy.
Web sites and applications are not a separate entity to general applications, they are one and the same. The skills required are slightly different but the core knowledge / principles are the same.
'Nuff said.
/* Andrew Fong - rogue programmer */
Although writing efficient code is important, I think that keeping the code in good style and organized and in general easy to maintain is much more important than efficient code.
Write maintainable code, then tune your code to be effiecient. I've seen people make the most convoluted code just to save a couple milliseconds. Write good code then find where the biggest slowdowns are and make them more efficient.
Eric Anderson
Whatever you do, do not mix all code and logic in a single file the way traditional ASP and PHP is done, it is nice to start with but when it comes to redesigning or recoding it can be an absolute nightmare.
...in fact msot serious hacking is done by UNIX, and UNIX based systems such as Linux or C++...
--
--
My sometimes helpful blog
'Even if you make it as unoptimized and cruddy as possible, it still takes negligible processor power. '
I'm afraid I have to take exception to this one -- when you are designing a site to support many thousands of people, all served dynamic pages, CPU time most definitely becomes an issue.
You should get to know your language; which constructs take more time to access, which loops are most efficient, etc.
Also factor in any component/object performance considerations. Even if you are calling compiled code, there will be ways to make it do more work in less time.
In short, don't ignore performance just because 'its the web!'. If your site only has 5 people on at any one time, then yeah, don't worry about it. But standard adages apply to web programming just like any other kind -- know what you are trying to do, know the requirements, and know the tools you are working with.
This is a great "perspective" book, but if you're looking for pragmatic advice this is not it. He has a lot of information on his favorite setup (AOLServer/HP/Tcl) but has little to say about anything else, besides, "This sucks, that sucks, you're an idiot if you do that..." etc.
If you're gonna buy this book, buy it for the photos and the excellent writing.
To be honest, I've never seen a book on efficient web programming. For better or worse, web engineering is all about rapid development and deployment. If it takes you twice as long to make something, someone else will beat you to it. If you look at the way some of the truly massive sites (like eBay) are put together, it's staggering that they can support 100 users, much less millions, but with money to burn it just hasn't mattered much.
Go pick up a copy of Advanced Programming in the Unix Environment and Unix Network Programing, V1 by Stevens and write your own perfect solution, or run mod_perl and be aware that each of your Apache children are sucking down 30+ MB of RAM. You decide. :)
universities are a hell of a lot more than 'reading the right book'. the feedback you get from a class full of your peers, some of who may be smarter than you, plus the feedback from professors is a *lot* more than you can get in the "real world" where someone with your skillset or better may be miles away from you. universities tend to concentrate the best of the best and put them within talking distance of each other.
Which, of course, is exactly what database vendors would like.
If you don't mind that, stored procs are great. But database independence can be A Good Thing [tm], for example, when the university gets a big fat check from Sun Microsystems, and suddenly you're looking at converting everything to Oracle 8.
Anyway, something to think about. You can also encapsulate and hide data, etc. etc. etc. if you keep your data access methods separate so that in case you decide you want to pull user data from an LDAP server instead of the DB, you don't have to change every place you coded that.
Ahhh...
Thanks for clearing that up. I don't have any experience with mod_perl (besides knowing that it's an module and it does perl) so I didn't know that it cached scripts like that.
PHP can be run as a CGI program, but is much more often used as an apache module, just like mod_perl. In that case, what's the benefit of perl over php?
Also, php4 uses the zend engine, which can produce pseudo-compiled programs from php pages, and you can even get a beta version of the zend optimizer, which makes them run even faster. Before complining about the efficiency of php (3), please try out version 4. RC1 is available at http://www.php.net/
>Seperate out HTML/text and code as much as possible
:), it _is_ possible to separate code and html. There's a php module called FastTemplate (I believe there's a version written in perl too) which allows you to do just that, by creating template pages with placeholders that a php script can fill in with dynamic data. It's works wonderfully to separate the work of the coders and the designers, which is something that certainly should be done.
If you're using php (which I strongly recommend
I don't have a link for you, just search for it on google, or CPAN, or and decent php user site. (I think I first read about it on phpbuilder.)
> Cross platform. Yes, on the server, the cross platform promise of Java works.
Since when do you need to have multiple server architectures run the same code?
> Integration with other languages and platforms, such as CORBA, Microsoft COM, etc.
PHP does that. Perl does that (I think.) Cold Fusion does that.
Java IS slow. I'm on a AMD K6-2 350 and a Java Hello World took about 3 seconds to respond to a "destroy" request. Most programs take around 1 second. A slightly larger program is slow enough to be unusable.
o/~ All God's children shall be free in Pirates of the Caribbean, when we reach that Magic Kingdom in the sky... o/~
Just commenting on one piece of your comment. Is it so wise to always destroy database handles as soon as you can? Yes, they take up memory that could be used for processing another request. However, the act of opening a database connection is both intensive and fairly uncontrollable(you can't just re-write the database driver on a whim to get better performance, unless you're project is a hell of a lot better funded than most in the software industry are). It seems to me like keeping a pool of persistant database connections wouldn't be such a bad idea. Indeed, it also seems to me that this is the direction that the technology is headed. What do you think?
Usually when I try to do a script based web site, I separate the presentation from the script as much as possible. I have a flat a HTML document that has special tags that my script searchs for. When the script finds them, she inserts the dynamic content there. This is very easy to do in perl. This way someone else can design the presentation of the web site, and I don't have to worry about him/her breaking my code. Isn't that the proper practice for any other type of programming?
_ _________________
This obviously carries a penalty on the server side, but what do you think represents a better savings for your company? Saving on one server or saving on one web developer?
I wish this question came a month later so that I can plug a general (free (beer and freedom)) implementation of this type of thing that I am working on right now.
___________________________________________
___ This sig is in boldface to emphasize its importance!
yes, I can't spell
effeciency=efficiency
It's 10 PM. Do you know if you're un-American?
> In industry buzzword-speak that's
> 3-tieredarchitecture. (Tier1: ASP,php,
> etc. Tier2: SQL stored procedures,
> compile code Tier3: the database
What you say is not entirely correct. In "industry buzzword-speak", Tier 1 is the client (Webbrowser), Tier 2 is the application server (IIS/ASP, Apache/Jserv, Apache/PHP, etc.) and Tier 3 is the database. The whole point of this is to decouple the application logic from the client on one side and from the data store on the other hand.
If you use stored database procedures (Oracle/PL/SQL, SQL Server/TSQL), then you put your business logic on the third tier, which makes the whole thing less scalable and also dilutes your three tier model.
It's almost impossible to have a baseless snobbish opinion of the General Theory of Relativity.
Of course, Microsoft gives a useful web load testing tool away for free... I'm running it now, in fact.
Anybody want to plug perl/php/apache equivalents of note?
"Consider yourself a member of a virtual corporation with Mr. Torvalds as your Chief Executive Officer." - Linux Advocac
I beg to differ. One could go on and on about how we have fast, fast processors today. But Windows is a junky program with no regard to any efficiency. Many people, especially with Linux, still have 486s and old Pentiums. Others have an abundance of people connected to their systems and need to squeeze every last bit out of the systems in order to serve the demand. If everyone says that the computers we have are so fast we can make bad, inefficient code. Then everyone will make bad, inefficient code and require people to get even faster computers which they DON'T REALLY NEED. And this is because people write inefficient code with no regard to speed or efficiency.
Chris Hagar
"The price of freedom is eternal vigilance." - Thomas Jefferson
sorry, more precisely what I meant to say is that I use one database handle in a script. What I destroy is my "dbresult" handles (created with a $dbh->prepare($statement) ) type statement. for a large website, I think persistant connections are the way of the furure though. good call.
So are there any widely used libraries that make C coding CGI easier?
You might want to take a look at cgilib.
Use Java Servlets:
- Security risks from buffer overflows elminiated.
- Just about all web servers support them now.
- More understandable and maintainable. Sorry. I like perl as much as the rest guy, but sometimes it's like reading chinese.
- "synchronized" keyword helps with re-entrancy.
- Use a native JDBC driver, if you can. The speed is very good and the code makes sense.
- Phun stuff like servlet chaining and JSP's.
- BTW: Make sure you use templates for your pages and a tool that supports them. You should never have to edit a web resource in more than one place.
I've just had very good results with this combination. Your mileage may vary.
"I'm an old-fashioned type of guy. I worship the Sun and Moon as gods. And fear them."
You do have a genuine need for multiple server architectures run the same code, when your Internet start-up starts working as a 5-person company with a handful of clients and all you can afford is a bunch of Linux x86 rackmount servers on the co-lo facility; and grows to a 120-person company with 400 clients, and x86 doesn't cut it anymore; as it happened to us.
Had we used some other language, we probably wouldn't be able to move to Solaris on SPARC easily. When our Sun Enterprise boxes are here, all we will have to do is to move our servlets. With Java servlets, you can go all the way from your puny AMD box to RS/6000s. When you're not a college kid designing a Web page for fun; and you're building a mission-critical Web application for a business; this really matters. The last thing you want to do is rewrite your application because ASP does not run on the new Solaris box you're getting.(To nitpickers: Yes, I do know that Cobalt bought ChiliSoft, and ASP does run on Solaris. It's just not the same thing unless you can port your accompanying COM objects to Solaris as easily)
We have thousands of lines of code working for a mission critical application, all written in Java. It's very fast and responsive, and a lot of very big, new sites are using servlets.
PHP and ColdFusion really do not stand up against Java servlets. These are server-side scripting tools for minor applications and Web designers. I don't mean to say that it's impossible to create big, extensive sites with these tools, since there obviously are a lot of such implementations. With PHP/CF you get the convenience of integrating the business logic and presentation logic in the same layer for easy maintenance (which is something you can not do with Java servlets unless you're using some kind of template engine like WebMacro, FreeMarker or an application server infrastructure like WebSphere, Enhydra or WebLogic); but you don't get the power of using Java, and a bunch of very well designed class libraries, RMI, JSSE, JavaSpaces, etc.
Java is really the best server-side Web development tool out there for big applications, when used as part of a well-designed application server platform. When we started out there weren't any application servers, so we ended up writing a lot of stuff for database connection pooling, template engines etc., but now this stuff is available. I recommend you throw away your prejudice, and give a serious consideration to Java servlets.
--
BluetoothCentral.com
A site for everything Bluetooth. Coming soon.
Zigbee Central: A Zigbee weblog
I've done this. One word. Don't.
He is speaking of a database concept of precompiling a batch of queries in SQL so they skip the input/parsing/optimization phase when called, which is a good idea for any complex, time consuming, or often called query.
Yea, but he's right.
Typically, from my experience, tight, memory and processor friendly code is easier to maintain and contains fewer bugs because the developer THOUGHT before implementing.
The "new school" of thought is that time to market is of utmost priority, which causes poor decisions to be made, like reliance on scripting languages and poor implementation. I have worked on multiple projects that required a ground-up rewrite because of performance and maintainability issues because rapid time to market was the only consideration.
The ironic thing about this is that the issues brought about by this philosophy almost invariably leads to a longer time to market because the bug bashing phases gets extended out so long.
Lessons to be learned:
1) ALWAYS treat performance and memory usage as prime concerns. This forces you to think algorythms through and avoids rewrites and broken code.
2) Generally it is better to implement in a tight, compiled language. These languages are exponentially more efficient and generally no harder (many times easier) to develop in then their scripted cousins (compare C++ to VBScript.. C++ is 6 months to learn, a year to master, VBScript is 12 hours to learn and a lifetime to master. Discussions about VBScript programming are invariably about how the underlying implementation works and how to opimize and overcome issues, whereas C++ programmers know what their compiler does and spend their time making their algorythms better.)
3) Keep it simple, stupid. If you can eliminate a component, do it, be it interpreter, server, datatbase, OS, GUI, Code, anything. The more peices a project has, the more things can go wrong and the less you can do about it. This is what I hate about the COM model when overused (as it often is). Sometimes a function should just be a function and a text file should just be a text file. Script, XML, multi-tier, component object models, these all have their place, but don't be afraid to not include them when they don't!
I was development lead on a Y2K support project for a major software company. We handed the project off to another division after version 1.0. When I was transfering my knowledge I was asked questions like:
Q) What database engine and model do you use?
A) tab-delimited text file
Q) What what is your COM Model?
A) It's just a single executable... no object model.
Q) Why didn't you use COM/DB?
A) Why would I?
Turns out this team had written a similar app. Mine took about 15 seconds to do it's job and consisted of a single executable 200K in size... VERY old-school. Theirs took about 2 hours and contained about 10M of binaries.. Mine was developed and supported by a single dev and a single tester. Theirs, a team of 10 devs and 5 testers. Theirs got scrapped.
Old school rocks. You can put out more code in less time. I can put out less code in less time that does the same thing faster.
The master programmer adds features by deleting code. Until you grok how this is done, you are adding to the problem, not the solution.
hehe.. you sudder, I get homesick feelings..
musing on finding ram in a system, I once:
1) Added nice "eye-candy" to a program... a kaleideoscopic display up in the corner that was truly hypnotising to watch. What it actually was was my datastructures being manipulated in video ram, disguised as nice eye-candy.
2) Told the printer to go off-line and used the 16K readable buffer that it had by querying the parrallell port.
Those were the days *sigh*
I was not saying don't to n-tier.
I was saying "don't store scripts in a database to be pulled and used dynamically at run time"
I'm working on a database chuck full of VBScript and Java Script, with a script interpreter component loaded onto a dumb middle tier with all the business logic stored in the database as scripts!
His misconception of an SP sounded much like this system, and my recommendation is: DON'T TRY THIS AT HOME! It's SICK AND WRONG!
Business logic goes in code. not a DB.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web.
Is that some resentment I hear?
And why do you assume your idea of "proper credentials" is the same as anyone else's? It's obvious you're an engineer and not a project manager. Most business-minded folk don't care about earrings, the color of someone's hair, or even their programming style. All they care about is getting the job done. And there's not a lot of people to choose from, so it's a matter of priorities. And guess what? Programming style is not high on that list of priorities.
re XHTML: you're hyping this. Anyone who writes HTML which will have a browser re-render the layout several times will probably write it in the same way in XHTML. No difference there. Pages written in XHTML will not be better than pages written in HTML. They are better prepared for future browsers and business applications, that's true.
loz
In IIS 5 this is now the default.
Loz
Loz
Slashdot's comment display, at least in Minimalist Mode, is very bad at this. In Netscape 4.x you have to wait for the whole 100Kbyte or so before seeing any of it.
This is actually netscape's fault. It has some kind of unwillingness to move things on the page as the page loads, so it often finds itself stuck waiting to know how wide a table has to be or something.
Trees can't go dancing
So do them a big favor
Pretend dancing stinks!
Yeah, I'm a bit spoiled by perl as well. Perl tends to make CGI VERY easy. I learned to do Perl/CGI stuff at the same time, and it all seems much more natural than do it in C. Just slap in CGI.pm and away you go with easy-made forms. And Perl's string handling makes parsing out data quite easy. Of course, it's bit sluggish, but then again most of my scripts are small and run on a pretty fast server, so I guess I'm double spoiled. :) And you have to watch what modules you use, as some can be slow (like DateManip).
Then again, not all CGI HAS to be fast. I would use C/C++ for the user end stuff, so you can impress your client with fast searches, etc. But for other things, like just parsing logs and non-time critial events, a niced-down plain perl script is just fine, otherwise you might waste vaulible time not optimizing the good stuff.
I heard that it's possible that perl scripts CAN be compiled, but the last I heard of that was that the support was rather buggy. Has it got any better? Yeah, I should be checking it myself, but I'm still learning this stuff and want to hear from someone who did something heavyduty with it.
Some things may be done client-side. Of course, you can't rely on XML browers like Mozilla's navigator. But with an XML database, you can offer XML as an alternative so that they who use an XML browser have those advantages like speed and client-side sorting of downloaded data. In the future, the XML version can be standard. For searching performance, you may want to use a "classical" database too, but that certainly is XML's only problem.
I have some university education. If I ever finish my thesis, I'll even have a degree....
I have taught myself a great deal of computer science knowledge, and am a very competent programmer. I am well-paid for my knowledge and abilities. I've been working with computers for over 23 years.
I have NEVER taken a single day of CS theory. I've taken many CS theory texts and read them, implemented their algorythms, etc, all without formal education.
I think that CS theory is, without doubt, very important to a software developer. I don't think our educational system gives one heave about whether they are actually teaching anything to anyone, and so, a formal introduction to a subject doesn't really mean anything. A person MAY learn something about CS theory, or they MAY NOT. The piece of paper indicates ability to write tests, not know CS theory.
Given all the above, and my own experience, I can state clearly that the piece of paper gives no guarantees. At least with engineers, if you don't know what you are doing, you are legally liable. I know of no software developer (who isn't an engineer) who would admit the same responsibilities. And so, no guarantees.
Only what you can actually accomplish indicates what you've learned. The degree doesn't indicate actual knowledge, just that you've at least had exposure. Its sort of like a disease - exposure may mean you've been infected - it doesn't guarantee it.
A comment overheard in a corn field `If you have better ideas, lets hear them. I am all ears.'
I don't know what university you went to, but most of my third year classes still have 75 students or more. Getting feedback from a professor is more like a foot race to see who can get in line outside his door first after class.
Most of the more experienced engineers I've worked with tell me that a lot of the math and advanced theory they learned in school hasn't been used since then.
It's true that "universities tend to concentrate the best of the best and put them within talking distance of each other", but the brightess minds tend to be anti-social.
In a work environment, anti-social behaviour isn't "awarded" like it is in school (where there is less emphasis on teamwork). In real life, a person cannot succeed unless they can communicate in order to teach and learn from their peers.
----- rL
I dislike the implication that many here make that those of us that sought, and obtained degrees (such as myself) wasted our time
Deciding whether or not it's a waste of time is up to you. I am still in school. I know that I have a lot to learn and school is one of the best places to learn, especially if you don't have a nice connection in the biz.
That is a bad thing
Yes, your points are valid. Maybe I should have been more specific, but what I meant was some of the advanced and/or unrelated things that you learn at school. This can't be helped since schools have no way of knowing where all of their students are going to work. A pure work environment can help a learner focus on one area instead of learning little bits of everything or having to take and arts class or something (which some may view as a waste of time).
----- rL
I think you made the right choice.
A lot of people go to school because they think that they will learn things they can't possibly learn in the real world, and that's just ignorant. Many bright minds have skipped school altogether reasoning it is a large waste of time.
I think what it comes down to is EXPERIENCE. If a 15-year old kid with green hair has been working on web scripting/coding for 5 years, then he has experience. Some kids these days take to web programming like flies on honey.
The above posters views on "advanced" topics is also shakey. What makes you think that a bright teenager can't read the same books you did in college?? A lot of bright kids are making a lot of headway these days as long as they aren't stiffled by biz guys who think that just because they are kids, they can't possibly know how to program. IMHO, there are more older adults that think they know more than they do getting jobs just because they are older, not because they are more qualified than teenagers and/or young people.
----- rL
I'll admit from the start that where I work, we are completely Microsoft. I should also mention that I work at a university, and the IT powers that be have managed to get some good deals on software I guess. I don't want to get into a platform war though, I'm just trying to offer some suggestions on how to improve site efficency. We are running NT4 on all of our servers, MS SQL server, etc. Pretty simple basic server setup overall. All of the stuff that we do is done using JScript in Active Server pages. We were usually doing most of our SQL in our scripts. We had heard of Stored Procedures, but never really looked into them all that much. Once we took the time to sit down and write a few, we've found that our site speed and efficency has increased a LOT. There were a lot of SQL statements that we used a lot, and we had modularized them to function calls, but you just can't beat letting the SQL server do the majority of the work. Try to pass of work to the server that is designed to do it - it may take a little extra work and planning, but it is really worth it, especially if you are doing a lot of heavy database accessing.
I think you can do this on Apache with "module chaining" configuration. At least, you can do this using Apache::GzipChain (sp?) module for all mod_perl output.
I think, in general you're correct. i used to work in a very large company (big red circle for a logo, used to be the phone company...), and the overwhelming trend was that people with more formal education had better programing skills. there was also a very real trend that those with more experience had better skills. there were exceptions to both, of cource, but the trend existed.
what i think is pretty key here is the difference between education, certification, and vocational training.
regardless of where it's obtained, be it in school, or work, or whatever, it's education that makes someone a good programer (or a good _whatever_, for that matter). training is clearly a part of that education (you'll make a lousy C programmer if you don't the syntax), but it's not the whole story.
certification (like degrees), tends to indicate a certain level of training, but really says very little about overall education. like Roundeye says, things like working in a group, understanding of deadlines and priorities, things like that, are equally important to being more than a coder, but a programer, an architect.
certifications may not be the whole story, but they're better than nothing. if you've been in the "real world" for a decade, and you've got good references, but no degree, great. you're still a viable prospect. but if you've got no degree, you've never worked on anything, but can code me a C compiler in 15 minutes... sorry, better luck next time.
i've run hiring of various IT departments for people in my former company, and i only saw one aplicant (out of several dozen) worth looking at who didn't have a degree. i didn't see any who didn't have solid work experience.
myself, i've got no degree. i'm good at what i do, but i know i'd benefit from certain types of formal education. there are things you learn in a good program that do more than increase your knowledge: they change the way your mind works.
a guy i used to work with tought a 12 week cource in fortran. after restructuring the cource, he was producing better fortran programers than the cource had ever previously seen. his restructing consisted of this: 10 weeks learning concepts in pascal, then the last two in fortran. the vocational training of learning to fortran syntax was the easy part; learning how to build and design programs was the hard part.
i consider it something of a tragedy that most University CompSci/CompEng programs are becomeing glorified VoTec cources. any department that teaches intro programing (or related) using Java or C++ is a good example. OS cources teaching from Linux is another danger sign.
the industry has no shortage of coders; they're a dime a dozen. what we're at a serious shortage of are program architects. and it's rare to learn that mucking around in your basement.
i speak for myself and those who like what i say.
> This is actually netscape's fault. ;)
IE5 do the same. Lynx is the best.
"Video bona proboque; deteriora sequor." -- Ovid
as a rather intelligent 16 year old, i completely agree with you. the people that talk down to people that are younger then they are are hypocrits.
/. have complained that MCSEs dont know anything? yeah thats what i thought
how many of you who are complaining about teenagers also shared experiences over in the hellmouth discussion? you probably bitched about the stereotypes people put on you...why are you doing it to us?
oh and just because you have a nice looking degree that you can put on your resume doesnt mean you know anything. it just means you paid some money to sit through some classes to have someone else tell you that you know something. the MCSE is a degree, right? how many people on
experience and a portfolio is the only way to truely find out if someone knows what they are doing. i have developed a few web sites in my time (and i am only 16!) my current project for school is called My.AlWood (AlWood High School is my high school) It will let students and teachers check via the web their schedules at school. it displays assignments and extracurriculars. it runs on linux with apache 1.3.9 and PHP4. the database is postgresql. the source code is there for people to look at and may eventually be GNU-licensed. i am not going to post the site URL here b/c the server is an old Macintosh. if anyone wants to know, you can figure it out from the DNS records for the holt.henry.il.us domain or you can just email me and ask!
give the younger set a chance, guys. we really arent so bad.
holt
...
${YEAR+1} is going to be the year of Linux on the desktop!
Simple rules are to avoid using http sessions as data structures, use object pools as much as possible (not just for database connections), try and keep things as stateless as you can possibly get away with, pay attention to the hardware on which your web app will be deployed (for instance, if your db host dwarfs your middle tier by ten orders of magnitude it might not make sense to cache SQL results in the middle tier after all), and try to keep the presentation layer clean and relatively code-free for the designers (who will otherwise cost you tons of time in mistakes and inane questions). There are other more specific tips, such as refer to values via references to existing data structures rather than instantiating new Strings to hold such values.
A decent Java book on concurrent design is: Concurrent Programming in Java: Design Principles and Patterns.
Dude...Gauss *REALLY* wouldn't appreciate having his name mentioned so close to that twit xenophile's. Not to mention, lgas never lived in Philly. *shrug*
Many of the good books have already been mentioned in other posts...
One thing to keep in mind: optimize as you go (use common sense). Here are a few examples:
A lot of little performance hit eventually adds up to a bigger performance problem. Granted, there will be large performance problem, but sloppy programming should not add to the real problems.
The important thing here is this -- Don't overcomplicate your webpages, honestly, as DHTML is cool, how much of a _REAL_ requriement do you have of your page? Is it a form processor, a dynamic frames page, what? Chances are, if you keep it simple, you wont have a problem, and that's the real optimization of web-code. While not true nessiarly in terms of C/C++ programs, it remains true in web development. Lots of people make "Web Optimizers", all they do is basically check for redundant operations, if your fully experienced in your language, you should already be aware of these redundant operations, and already be avoiding them.
--- 'dex
Is there any equivalent of the Perl: print "HERE" line 1 line 2 line 3 HERE in PHP?
I mean't:
print <<"HERE"
1
2
3
HERE
in PHP?
A few helpful sites:
Online gaming for motivated, sportsmanlike players: www.steelmaelstrom.org.
Online gaming for motivated, sportsmanlike players: www.steelmaelstrom.org.
How about Visual FoxPro?
There's a great framework called WebConnection from West Wind Technologies. Except for the little dll (written in C++) that handles the interaction between the webserver and your app, the whole framework is written in FoxPro (and the source code is included).
It's written specifically for performance and stability. There's a very active support board, and features are constantly being added. The site has a lot of information and documentation, and a downloadable demo and sample code.
WebConnection has support for XML, DHTML, email, FTP, SQL and stored procedures, PDF generation, session management, client side apps with HTTP data transfer, remote COM calls, and on and on.
Sorry for the blantant ad plug here, but I work with this every day, and have for the last couple of years. It really works and it works well. The performance and reliability are great.
If you are working with databases and the web, (and you are a database programmer), you really need to check out it's features. Of course, as with any programming, you still have to write efficient code.
One of the issues that affects this is that individuals that are self-starters, that have learned how to code because they wanted to, and have done so on their own, can code better than an individual that didn't decide until his junior year in college to be a comp sci major. So the guys walking out of college are going to be using old AS/400s, NT platforms and the like, while the "undegreed" kids are the ones that have hacked on linux since the 1.0 kernels were released.
Who would you like on your team? I'd rather have the individual that doesn't have a pretty piece of paper from some prestigous college that his parents paid for him to party through.
I was going to post on how to design high-capacity web sites as we do here at MP3.com, but got involved in this instead. Feh.
-ted
Of course along with this, it is also essential that you don't build up your server-side scripts from scratch, patching things along as you go. All these functions need to be a smart as possible and must get the most out of each process as possible. Remember, a lot of times it takes longer for the scripts to connect to the db than it does for it to retrieve your data, so one possible tip is to combine your queries into a few broad executions. So along with the db, have an overall structure for your code.
I think something that is very helpful that I've learned is that not everything needs to be done by the server-side scripts included in that page. Let's say for instance that I would like to return the number of files on a system, and this has to be added to the file counts of three other systems. Now if this were to be executed when the page is requested, it would take awhile because the script would have to make the db query right then and there and count all the specified types of files and then do it three more times (so in total you may be seraching through the entire db 4 times). What would be better is if the situation allows for periodic data collection. If the data doesn't change that much, then it would save a lot of time to write a small java program that resides on the server and every 30 min or so performs the db queries that your page would. It would write it out to a file and so when you serve the page up, all you have to do is read that file to include the info and you're done. This makes a lot of sense because not only are you saving the user time but you are saving your database from choking because if 5 people want to access the same information, the info will be the same, so why compute it 5 times over?
So essentially I think the most important things to remember are to optimize the db, optimize your scripts, and pool your data.
somebody had to post the most obvious idea: yeah, code everything in c.
shit, why don't you write your own web server and rdms while you're at it.
look, if you're amazon, you have the cash to fill a room with nameless drones and demand everything in c.
an optimal solution for a particular web may require multiple languages ("for some stuff"), but most of the code for building pages should be interpreted, otherwise making changes (fixing mistakes) will take too much time.
the majority of people should use a string processing oriented, interpreted language for creating dynamic pages, and they should not waste any time "proving" compiled c is faster. the earth revolves around the sun.
I'm also undegreed. I'm 19 and I've taken one college class (History of Cinema). I've used computers my whole life but I've only done Web Development for about 9 months. I could go for the computer science degree but it would be a waste of time. My boss once handed me a ASP book and said, "build me a shopping cart". It took 3 weeks of very fusturating work to pull it off but now I can design custom shopping carts. Along with database driven e-commererce sites, extranets, database driven product catalogs. Work experance beats school experiance. If I took computer courses I'd know all about bitwise operators and recursive functions, but I couldn't take a relational database class until my juinor year. The part Matt Damon says in Good Will Hunting that takes place in the Havard bar put's it best:
"You're going to wake up 20 years from now and realize you spent 150 grand on an education you could have gotten for 1.50 in late charges at your local library."
Or something like that
Hollow words will burn and hollow men will burn.
www.primedata.org/psx.htm written in Visual C++ by Erik Aronesty the only bad thing i can say about it is that it's not available for Linux... yet. the docs suck email and maybe i'll fix them
One clear example here is Java vs. C++. Sure, Java is much slower than C++; but developing something in Java takes orders of magnitude less time than in C++. You know, people say this all the time but I've never seen any evidence to support it. The only scientific study I've ever seen on the subject concluded that development in Java is no faster than development in C/C++ (which is what I've found to be the case in practice as well); it's quite a nice study of a number of issues. 40 programmers were given a problem, some implemented the solution in C, some in C++, and some in Java. Average run times for the different languages (with statistical spreads and best/worst times) were calculated, along with memory use metrics and other neat stuff. Postscript version at: http://wwwipd.ira.uka.de/~prechelt/Biblio/jccpp_ca cm1999.ps.gz PDF version (tougher to read, because it's ps2pdf output) at: http://wwwipd.ira.uka.de/~prechelt/Biblio/jccpp_ca cm1999.pdf Sumner
-- rage, rage against the dying of the light
I converted the page by hand but then realized I already had a program that would have done it automatically. Try out this awesome web development program by Evrsoft (only for Windows). Go to Tools | Tidy HTML Quick Tools | Convert to XML. Instant XHTML.
Um, that doesn't seem simple to me. How exactly would you implement this simply? How do you check if the database has changed without querying the database again?
A man, a plan, a canal -- Panama!
Ok, you've struck a nerve like Madonna used to strike a pose. I'm 31, been an I.T. professional for ten years and have no degree. I dropped out of college in my senior year primarily out of frustration with the entire process/institution. Having worked full time as a programmer and attending night school to finish my remaining credits, I began to see the lack of reality behind the instruction. The problems presented in class were either demonstrations of some obscure theory of computation having no connection to the issues I faced out there in the "real world" or were so cookie-cutter simple that they added nothing to my practical knowledge base. Having faced no small amount of discrimination during the search for my first job for being non-degreed, I am still sensitive to, and deeply offended by, those who choose to view a person's technical competence and "worth" to the company first by the presence of a checkmark next to the Degree box on the employment application. People without a college degree (especially back in the "old" days of the early 90's) have had to work harder and develop themselves to a much higher degree than their degreed counterparts just to maintain parity and garner consideration by the old guard that to continue to uphold this no degree=no brains generalization, especially during this tight tech labor market, is the real sign of incompetence.
Oh for christ's sake. Undegreed kids, formal knowledge. That's ridiculous, there are so many things wrong with your post that I'll just have to focus on one:
A degree has nothing to do with someone's ability to program. Nothing in college prepares you for working in the outside world. Your whining about 'undegreed kids' getting jobs is pure elitism. Now as far as I'm concerned there is nothing wrong with elitism as long as it's based on acheivement and ability. Not on money, which is at the root of your complaint. You're upset because these kids have circumvented the system by not paying ridiculous amounts of money to a bullshit institution that trains them in no way for real life.
Home computers have provided people with the ability to learn a trade, and be damn good at it, from the comfort of their own homes. All for the price of a computer. Hell, as prevalent as the internet is today it's no longer necessary to lay out money for books to learn to program, it's all out there for free.
I think you were born after your time. You sound like you'd be happier programming in your suit and tie at IBM in the 50's, among all the other batch trained automatons.
I'm the big fish in the big pond bitch.
Hey, now. I AM a green-hair, but I have certifications (CNA, Network+, some for web shit), 3years doing professional web design, 1 year PERL programming, and 1 year system administration experience. Its not like all the green hairs just got on the web yesterday. We (the G-H's) have been here for a long time, and will continue to do so. I know since every 17 yr old wants to dye his shit green, we kinda get stereotyped, but some of us have earned our geekdom the hard way. Slackware all the way baby....
./skank
That is physical three-tier.
Logical three-tier is far more relevant:
1. Presentation
2. Logic
3. Data
For example:
1. Browser + JSP
2. EJB
3. Oracle
Or:
1. Browser + ASP
2. COM and stored procs
3. SQL Server
Just as important to consider is the environment that you are creating for:
The code and your approach to that code will change dependent on your target audience. I have worked at companies that only use Microsoft products and built VBScript ASP pages for their company intranet because it tied in smoothly with SQL Server, and turned around and used the exact same model in Perl for another client to use. The form looks the same, but the hardware and audience is different.
On a seperate note, ASP is very cool because IIS will let you code your server-side scripts in VBScript, Perl, or Javascript, plus you can incorporate COM objects from C/C++, VB, Java, and Python....it's one Microsoft object that actually has some promise!
The most important thing about code is style: it should be easy to read, easy to change, and easy to debug. This is true of any code, not just web code.
Effeciency--or the so called `constants' in big-oh notation--should always be delt with last. To help in the battle against slow code, there are optimising compilers (such as php4's zend engine, and the standard C compliers), and profiles to help identify bottle necks. But if your code isn't clean, you're going to have a hell of a time doing any optimising.
Here ends the first lesson..syntax like that is discouraged. Since php code is embedded in the html, you use html where it's appropriate:
<?
echo "
<b> this is html </b>
";
?>
is uglier than:
<b> this is html </b>
so why use it?
If you mod me down the terrorists will have won
While mostly theoretical, Knuth's "The art of Programming" is a great place to start once you've gone through the other newbie books. Click here to find the boxed set at B&N
Of course, most people would probably prefer to get the individual volumes, or check them out of the library.
"Live Free or Die." Don't like it? Then keep out of the USA
Your boss _actually_ bought the book? Ooh you lucky, lucky git! My boss whacked me over the head with a big stick, and then told me to build a corporate web site. During my lunch hour. I also had to buy my own books, but at least I get to keep them. Some people have all the luck ;8)
-- Cisk for the Cisk God
There's a few good points there. In theory people _should_ be hired on their ability, however from experience I've found that this is hardly ever the case. Most of my colleagues were hired because they were either cheap, or they were cheap. Part of the cheapness comes from a lack of certification, and because we do not have certification there is no incentive for the company to have us trained and certified. A certified employee is not viable because he then becomes, less cheap, and more attractive to competitors. Eventually the certified employee will leave for a better paid job, taking all of his experience with him. Strangely enough, the few people we do get in who have a computer-type degree are usually hopelessly inadequate when it comes to using computers, making coffee and generally getting the job done. And they get paid peanuts anyway.
-- Cisk for the Cisk God
Excellent. Concise, and to the point.
Thanks.
Thanks, but that's C++.
I specifically asked for a C library.
So are there any widely used libraries that make C coding CGI easier?
I know that at one point ncsa came with binary utilities for cgi scripts. but I dont remember any C libraries for the same task.
Methinks a library in the class of CGI.pm, could seriously replace perl for people who know what they are doing.
When making a website effecient you must FIRST design it to get a lot of the clutter out of a website. The more clutter = the more crap = the more complexity.
KISS - Keep it simple Stupid. That principle will help in the long run. If you limit your design you can limit the ineffecienies of coding.
Design should be about 80% of the website's time to get up. If you can make the interface easy to the user, it will be easy for a programmer to build and UNDERSTAND what he needs to do. Then it is all up to basic coding tenets.
I can program myself out of a Hello World Contest!!
> Yes, it is more expensive.
Actually, you may find that "expensive" hardware is quite cheap when compared to the value of programmers' time. As you say, it can be the difference between getting something out the door faster (or at all!).
Working in an adequate, high level, fst-to-develop-in language (pick one you like) lets you leverage the single most expensive resource of all: competent, skilled labor.
With the salary and job security benefits to ourselves, I'm a little surprised more people aren't aware of and more vocal about the fact that we (meaning the skilled folks mentioned above) REAALY ARE the single most valuable asset a tech company has. Everyone else is replacable.
Of course there is a downside: your startup may fail due to inability to find good people.
To cirle back to the argument above -- this implies choosing tools that maximize the utility of developers' time. Stable platforms, sane OS's and languages, good tools, and high level languages for most tasks -- choose what to code in C/C++ carefully!
Madness takes its toll. Exact change please.
One clear example here is Java vs. C++. Sure, Java is much slower than C++; but developing something in Java takes orders of magnitude less time than in C++. So, it is better to throw money at hardware to cure the performance problem, but will get you to market sooner, than code in C++, which results in faster code, but takes much longer.
Can Java servlets be compiled into native code like Java apps? If so, they'll be as fast as any C++, C++++, etc. module.
Will I retire or break 10K?
The parsing is usually not much of a problem, but when a query is executed several times each minute or even second, the performance-difference starts to show.
Also, stored procedures have a huge advantage when moving lots of data: imagine you want to store a picture of 50kb in a database. In SQL, you'd have to convert the 50kb of binary data to a string, making it twice as big and then sending it to the database-server, which needs to convert it back. This takes lots of time. With a stored procedure you can just tell the database-server: "Hey I got 50kb of stuff coming for stored proc #3" and that's that :)
Much of the time, it's less expensive to throw hardware at it, not more. Moore's Law does wonders for the speed of your application, but it doesn't apply to the productivity of your programmers.
- read the tuning docs for your servlet engine, and try different engines too. - Make sure you are using the JIT
- Make sure you have enough memory (-mx and -ms optoins)
- PROFILE YOUR CODE using any number of freeware or commercial profilers (not sure why you want destroy to go fast, when a page hit is going to one of the get methods)
- make it multithreaded, They're serialized by default I believe.
- manage your cookies.
- use the newest JVM.
When you've done that, then look at the system wholisticly (sp?) fire a few hundred hits at it. watch what happens in terms of cpu and memory. Are you sure Apache is tuned? are you sure there is enough memory? are you sure you've allowed the JVM to use enough memory?
But in Oracle you get most of that benefit just by using the same sql statements , and counting for variations with bind variables.
Oracle recognizes a statement that was run before, and pulls up a pre-compiled version out of it's cache (shared_pool)
In both cases you're talking 10ms on an unloaded system. But at high levels of concurrency, 100s of statements per second, the cached way is much faster. Under high load cached statements avoid cpu overhead which is scarse, but more importantly do not have to wait for a semaphore (shared_pool_latch) to get into the critical region of the oracle kernel to do the compile and update the cache.
as for the rest of the pro-java rhetoric, I agree. I think java is a great way to go, but you often need scalable hardware.
Like any platform, you have to learn java and how it plugs into an environment before you can say it sucks performance wise.
The direction of most modern programming languages is to hide the details and abstract the developer from the underlying data, hardware. Who writes ANY assembly code anymore? Not too many people.
Computer Science, Software Engineering, whatever you call it, is the science (art?) of managing compromises. There is a tradeoff in each and every decision made in the development process.
The abstraction of programming languages is simply trading speed, memory usage, bloat, etc for a major boost in the development time.
Take Win32 programming. Would you rather your app in VB or in native Win32 calls. Both do the same. One is smaller, leaner, but took 10 times as long to write and is prone to 100 times the amount of bugs and ends up costing the company 1000 times the dollars to maintain.
In the end, the choice of programming language, methodology, is going to depend on your circumstances.
In my opinion, software engineers need to understand everything about a computer and how it works, not just how to make the web page or java applet show up. The fundamentals of Computer Science (Data Structures, file stuctures, operating system design, compiler design, assembly language programming, database design) is the most important part of any education.
Knowing the newest, latest, and greatest JavaC++ASP language is not. If you know how languages are developed, how they work, the logic behind it, and most importantly, how it can benefit your life, there will never be a programming language that you can't pick up with a bit of work.
Good point--the net connection is usually a throughput bottleneck, but not necessarily a latency bottleneck.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
"Middle tier" is just a fancy word for library. Of course you should separate the underlying logic from the Web glue, but you don't need a buzzword to do it. Further, the languages used in the first and second tiers are irrelevant to the logical separation.
The evaluation of an action as 'practical' . . . depends on what it is that one wishes to practice.
Perl stuff basically screams under mod_perl. The drawback to mod_perl is it increases the RAM footprint of the httpd child process dramatically, to about 5MB per process. When spilling all your images, etc. to a 28.8K visitor, this becomes an issue... A clever guy has written some articles on perlmonth, When Perl Met Apache that talks about using a Proxy to enable mod_perl to do its thing, then return to a ready state quickly, and let the proxy take care of the transmission of content.
cat
I have done quite a bit of messing around with asp, and found that iis has a hard time rendering the code in the correct order (a good example is setting a connection object to nothing and getting an error because iis has already discarded it).
One of the most important observations I have made is that if you isolate discrete chuncks of code in functions, the out of order execution stops.
Simply isolating the connection object in a function like
set myconn = getConnection()
increases the system responsiveness by a factor of 4, and keeps your server from getting those annoying dropouts where the asp pages don't respond. This is where the cpu is somewhere around %50, and you can't get it higher, even with more hits.
Try it yourself, I recommend the 'Microsoft Web Application Stress Tool' which you can download for free from their site.
Always remember to release your objects! Good luck!
love is just extroverted narcissism
We have actually started prototyping the middle tier componenets as WSC (windows scripting host) activex components, and porting them to a compiled language. This way you don't have to change the method of instanciation in the asp code, and you don't have to mess around with unloading the components from IIS when you make changes.
love is just extroverted narcissism
I have worked on a small e-commerce site that gets 3 million hits per month. The front page is loaded dynamically and actually hits the database 3 times! There server is always quite responsive. If the site grows and the resources are getting used up, I will change the front page database to a text file, which will load much faster. If that is n't enough, I will optimize the product query page. And if that isn't enough......
My point is that it is really most important to make your code flexible and readable. As you monitor the site in the real world, you will get a better idea of the optimizations that you need, and can optimize your code as needed.
love is just extroverted narcissism
PHP 4 has here docs but it uses <<< Besides the extra <, syntax is the same Example <<< END_TAG lots of 'junk' and "junk" and junk END_TAG
I disagree.
Although I don't know about ASP, PHP4 compiles the scripts before executing them. Zend has been working on a compiler for php4 and although this doesn't solve the problem today, it will help a lot in the future.
Stored procedures are good if you like to write your applications in propriatary code. They are fast, but, I've found, are very hard to maintain.
It's no harder to maintain business logic in a PHP or ASP script than it is a C script. It's just code.
There's cgicc, which you can get here
"Do you think there are answers to everything here? Is that true in the place you come from?" - Agia
I think it should also be noted that just because you have a degree doesn't mean you can code... The best web designers are the ones who know how to write their own code and make it cleaner than a person who creates a page on MS frontpage and tweaks it.
Kris
botboy60@hotmail.com
Nerdnetwork.net
unrelated.
We are talking about good practices and you are mentioning ASP. That itself is an oxymoron.
--
If Microsoft is the solution, I want my problems back
yes Perl does have that overhead, but all you need to do it throw in mod_perl and bam...perl caching can be enabled. Dont use CGI, put everything in modules. This way you don't have to execute a CGI everytime, but rather called the cached modules.
And while you're at it, take a look at Gereld Richters Embperl over on perl.apache.org Using HTML::Embperl is great. It makes so many things so much easier. Tables are done for you dynamically, and since it all gets cached, the site can perform optimally
"Time is long and life is short, so begin to live while you still can." -EV
Getting to my real point...Program in an object orientated fashion. It makes your life much easier. Every program should be built of layers of other smaller files (modules, classes, call it what you will). By doing this you give yourself the ability to work with small files with very specific purposes and because of this you have the ability to create more and more modules (perl) based on the earlier core versions. The Core module all your programs can use (have one that takes care of all your DBI calls). Then to customize your program just write another module (perhaps based on an existing one, or a brandnew one). I don't do much other programming for the web outside of Perl and JavaScript, but I do know that everything can be programmed in an object orientated fashion which makes everyone's lives easier.
"Time is long and life is short, so begin to live while you still can." -EV
Yeah, you're right. You could definitely consider the client to be tier of its own.
A lot of people also consider the SQL stored procedures to be part of the data store tier, but I think they're part of the middle tier. Feel free to roll your own interpretation. :P
OtakuBooty.com: Smart, funny, sexy nerds.
Good question. OK, I've only worked with MS SQL server here (forgive my sins, please), but I think most SQL databases have this functionality...
The basic way to work with data on a SQL server is through a single SELECT, INSERT, DELETE, UPDATE, or some other statement. A stored procedure groups a (potentially) large number of SQL commands together and stores them on the SQL server. Basically a little program written in the SQL server's native language.
Why is this useful?
Like I say, all my experience is with MS SQL server. I'd to hear more about how stored procedures work w/ MySQL and other GPL databases, because I'd planning on moving into Linux development work.... hopefully soon. :)
OtakuBooty.com: Smart, funny, sexy nerds.
Well, we're talking about how to great around a bottleneck here, and the bottleneck is dynamically generated pages. I don't think the answer to solving a bottleneck is to rely more heavily on the bottleneck itself!!!
If a PHP script has to be re-compiled each time it's needed, that's why it's a bottleneck. What's faster, compiling a PHP script 1,000,000 times for 1,000,000 website visitors, or compiling a shared library or stored procedure just once for all 1,000,000 visitors? :)
Then again... does PHP4 compile a script only once into actual executable code, or does it compile it each time it's needed (requested by a web visitor)? If it only needs to compile it once, that rocks, and it it nullifies a lot of the arguments for a middle tier between PHP and your database. And it's a great argument against ASP because ASP is either interpreted or re-compiled at run time (close to the same thing really)
OtakuBooty.com: Smart, funny, sexy nerds.
Yeah. Middle tier==shared library. Buzzwords suck. Leave it to marketing fools to come up with a new buzzword to make it sound like they just invented something that's been around since the beginning of time....
OtakuBooty.com: Smart, funny, sexy nerds.
What's wrong with n-tiered architecture? Aside from the fact that it's a buzzword and buzzwords suck... it's a way of structuring your app so that it's maintainable. In any good code, you should centralize your logic whenever possible...right?
No matter what you call it, the goal is to move logic and CPU-intensive tasks OUT of the scripting language that generates your web pages, and INTO some sort of code that is fast,centralized,and maintainable. Take it from someone who's built a lot of dynamic sites the wrong way and the good way. :P
OtakuBooty.com: Smart, funny, sexy nerds.
dirty code gets passed around a lot,, in all of the different web dev circles. ( html, js, php, asp etc etc ) this is because everyone shares code especially with JS.. and copy-pastes. that's why a lot of people will all use a roundabout hack in their code.
<3x, kevin
Others may flame me -- sure it's not as good as it used to be -- but it's still not a bad resource;
http://hotwired.lycos.com/webmonkey
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
I've read a lot of ASP tutorials and resources and I have yet to see 'the right way'. Maybe 'the right way' doesn't exist.
Some people initiate/open/close/trash database connections on every page. Other examples show creating a db connection at the session level and just leaving them open as recordsets are pulled throughout the session. Obviously cuts down on the connections, but also abandons connections to just timeout. Still other examples open one connection at the application level. This would seem to me to seriously hinder the db performance though.
I've personally been frustrated with the song and dance I have to go through every time I want to pull some db info in ASP. Maybe it's because I came from ColdFusion. Maybe I'm a lazy bastard. But I'd rather have an efficient program and do some extra typing/cut-and-pasting so I'd like to hear what some more experience ASP developers do. Peace. Sway
Peace. Sway
Peace. Sway
I think one of the main reasons that efficiency isn't much considered in doing server-side programming is that most of the time, these scripts that run only actually execute for a few seconds. It doesn't seem to be worth the time spent making the code memory/CPU/time efficient if it only runs for 15 seconds. Sure, you might be able to shave off some of that time, so now it runs at 10 seconds instead of 15, but was it really worth the 10 hours you spent doing it?
Perhaps the biggest issue when hacking code for client side execution is to keep it as simple and as legal as possible. It may be less efficient, but it is more likely to perform across all platform in a much more consistent manner.
I like the b-tree idea, but what do you do when you are working with multiple tables? For instance, I've got something like 35 tables going for this particular database. How do you handle joins and such?
You've intrigued me enough to check out PHP, as its something I haven't tried yet. So far I code in C++, Perl, VB(ASP mostly), and Java. Seems to me the best way to go for what you are talking about would start with standard data structures that are in STL. STL leaves some things to be desired however, there are no hash tables or b-trees, so you still have to build your own from lower level components or from scratch.
Maybe an "improved" STL would be the ideal open source project you are looking for? I'd be interested in that as well.
Email me if you'd like to continue this discussion. jeverist@nospam.page1book.com Also you can probably find my website now.
No, Thursday's out. How about never - is never good for you?
Which is the reason for components. You can move most of your logic to C++ components(best results) VB, Java, or something else. My experience is with COM in this arena, so I can't speak for other component architectures. The components are then called from the web pages via ASP or whatever platform you are using.
Stored procedures compile the SQL itself. Its not the same thing as having a C library. You have precompiled SQL that you can pass parameters to.
I thought Oracle did have a C library, so I'll have to check your statement. It would be kind of dumb if they didn't since Microsoft has come up with a C library and fast access to Oracle through OLEDB(sort of an improved ODBC)
If you are looking for access to data structures directly, you probably should roll your own. You're not likely to come up with something that works better than good old SQL for data access any time soon.
No, Thursday's out. How about never - is never good for you?
One thing I haven't seen mentioned here though is that plain-vanilla SQL is a standard(ANSI SQL 92) and therefore portable between vendor implementations. I personally try to avoid vendor-specific SQL whenever possible for this reason. Going from a cross-platform SQL statement to a vendor-specific stored procedure is something that takes 2 or 3 lines of code in the SP itself. So I wouldn't discount coding in SQL because its vendor-specific.
As far as using data structures rather than a full-fledged RDBMS, I've had success using the C++ standard template library(STL) to create data structures like maps and lists where using a DB would be overkill. But in a solution that required multiple tables, I just don't see how you could realistically improve on SQL.
No, Thursday's out. How about never - is never good for you?
Besides slashdot, which we all (should) know runs on apache/mod_perl, there is the Internet Movie Database. So here we have two major sites that run on perl, and those are just a couple that I can think of immediately. I'm sure there are more.
No, Thursday's out. How about never - is never good for you?
Always comment your code..
Recently it's seemed as if, due to larger and faster computers, lean and mean code isn't such an issue. I gotta say first off, that inefficient code should not be tolerated in ANYTHING. With faster PCs, more RAM available, and more HDD space, apps are becoming more and more bloated, and definitely not efficient. What happened to the days of building an engine that would tweak the he11 out of a program's code to get the absolute best out of junky slow hardware? If we were to continue working like this, we'd have some blazing fast apps out there that would beat the pants off of anything else out there right now. Unfortunately, everyone's now just racing to get their products out first, and then issuing patches that fix bugs, not even the other problems such as inefficiency. are there any good works to read which focus on programming for the Web, as in avenues the programming medium affords? Have I just been oblivious and missed them?" Unfortunately, I mostly wanted to rant about the first paragraph, and don't know of any really good books that'll help you program efficiently for the web content, but I'm sure it's out there, and of course you can always look into getting books on efficient programming and look at them similarily for web. When you find one, post it on Slashdot! I want to know more! TZ
TeamZERO -=[You can be smart as Einstein; But without passion for life, you suck toast.]=-
I say, screw commenting code.
Writing CGIs isn't the equivalent of blueprinting designs. It's an art - like painting. Did Michaelangelo put little hash signs all over the Sistine and stencil in "I used a #5 brush here" or "This paint was made from pig fat"?
So, maybe people will want to review your code because they're bored or want to make themselves seem useful, but if they're actually worth anything as programmers, they'll be able to figure out how you did it.
"Beware he who would deny you access to information, for in his heart he deems himself your master."
iCab is a German-made, mac-only, pay-for-it browser (a strange combination to be sure). The real feature of this browser, though, is it's "rate-HTML" function... a little happy/sad face in the top right corner that gives a detailed (and depressingly long) list of HTML "errors and warnings". If you thought you were writing lean mark-up (or though Dreamweaver was doing a good enough job with "just a little tweaking") this browser will scare the pants off of you.
BTW, the "preview" version is still free... and it doesn't do javascript.
2 1337 4 u!
Java is notoriously slow
Java is slower than compiled languages (C, assembler) but nobody writes C code to do CGI much anymore. The risk of buffer overflows and pointer chaos is too great. Perl became a great tool for CGI development because it does things that C doesn't do, like bounds checking an so on, minimizing the risk of remote exploits via buffer overflows.
It's bad enough on the client side in an applet but it seems like a tremendous waste of cycles to use it on the server side
I agree that client side Java (I'm referring mainly to applets) is a total disaster. We can blame Nutscrape for a lot of this.
However, Java is not a total waste on the server.
There are various things a developer must keep in mind when writing any kind of software; performance is one of them. Other things that are always on my mind: memory requirements, simplicity/readability, time to completion, ...
My point is Java may not be the absolute fastest thing on the market, but for web development it is ideal. Perhaps Python has advantages over Java that I am not aware; many Python programmers I know on efNet encourage me to ditch Java in favor of it. When running a JVM on Solaris, I have been impressed with the results.
Actual story: I wrote a Servlet/JSP application that took a poll, tallied the results, and allowed administrators to view the results of. It's used by Texaco on their intranet.
The first day it was live it recorded 13,000 completed surveys, each survey about 50 questions in length. I made careful usage of threads and shared resources. In the end, I had a stable program that never crashed throughout it's usage on their intranet.
-- Happy Java Man
As for dynamic content, with high-volume sites I'm sure efficiency is a big issue. I understand the big guys like Yahoo have gone to great lengths to optimize their code - if they hadn't, I don't think any amount of hardware would handle the load. But in my own experience with fairly low-volume sites, the bigger problem is in producing clean, readable, debuggable, and reusable code. That's not always an easy task in PHP or ASP, and I've seen a number of big-name sites (who somehow neglected to fix the ::$DATA bug :) with truly horrible coding. Why is it we seem to be ignoring the lessons of 30 years of programming when we're building these sites? (Don't even get me started about web programming security practices...)
It's just that I use Windows 98 (not by choice). On this thing, everything is slow and horribly buggy. So because of that, I guess I became some sort of "clean freak" who has to make everything as fast as possible.
When I finally have the money to buy my own box, I will be using something much better (i.e. anything but OS/2). Then, I guess I won't have to worry about speed any more. (=
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
Sorry about missing that tag. I may be pretty good with HTML, but even I screw up...
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
Well, it wouldnt be a button at all. i would probably just use a normal link... using SSL...
or something...
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
That the HTML is clean, and that the scripts work without any unwanted side effects. Compiled stuff does help speed, but I prefer scripts, because it's much easier to modify. It's faster for development, anyways. It's edit and check for scripts, instead of edit, compile, check for compiled stuff.
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
XHTML, as far as I know, is the HTML 4.x DTDs redone in XML. I have heard that there is to be no HTML 5, that XHTML 2 will cover it. So, I guess it's high time to start learning XHTML...
Anyways, with XHTML, isn't it possible to make extensions to it that any XHTML ready browser can handle?
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
So how do we know what part of the program is the 10% that should be optimized?
In my opinion, it should not be just one part, but all of the program. Optimizations almost invariably bring an improvment in speed, whether we can notice it or not. Imagine running 50 instances of an unoptimized program (or 50 different unoptimized programs). Now, imagine that they are partially optimized. Some speed increase, maybe a second or so. Complete optimization? Maybe five or six seconds.
That's the goal. And even though that's not much time lost, imagine the amount of time wasted a 9-5 day by an office of 100 people. That's a lot of lost productivity.
So if anyone will complain about optimizing, they should go the whole way, and not be halfhearted by only doing 10%.
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
That actually seems like a really smart and good idea. Anyone can use this on their own sites without any problems (legally or otherwise) or is it protected by you or someone else? Because I think this is a good idea for my site(s) too.
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
WYSIWYG editors still control what the markup is going to be. It's still better just doing the work in something like Emacs or another text editor.
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
I still believe that completely optimizing the code is better. The more time shaved, the faster it runs... And even if you can't detect any increase in speed, it's still faster.
----------
Is this sig off topic?
Chris 'coldacid' Charabaruk Meldstar Entertainment
here's your medal.
Where are we going and why am I in this handbasket?
And, while perl is great for getting something out quickly, if you are working on an application consisting of dozens of interlinked components/pages/packages, perl is not easier to manage than C. 10x faster is a lot.
What percentage of websites are coded in perl? What percentage are coded in C? And you call C the obvious idea? :)
Perl 5.6 threads, BTW, are clearly documented down in the source for perl as not ready for production code yet, and due to change significantly.
In my own experience a year and a half ago, this was exactly not true. Yes, perl does run surprisingly fast, and yes, for many applications it's perfect, and yes, when we installed mod_perl it got faster, probably from eliminating the forking and recompilation...
But, with a Netscape Enterprise server (Solaris) running multi-threaded, we were able to make shared read access to gdbm files fast and threadsafe in a very natural way in C. Is perl threading good enough now, and does mod_perl handle multithreaded applications?
does it pull in Posix.pm? cuz you can hear the machine *sigh* when that sucker loads, and all so i can call pwd without forking a shell :)
You like CGI.pm? I really couldn't get used it it. I like a paradigm where the form is spit out by and the action processed by the same CGI (so the form variables appear in one place in the source). CGI.pm is downright buggy when used like that.
If I'm doing something that's going to run intensively and I don't want CGI to spit out the form, I follow the same source paradigm anyway, but throw together a "makefile" that will generate .html form "object" files and automatically patch up all the links.
If that is the only thing you are going to use a php tag for, don't use it.
It's the amount of HTML vs PHP that determines what tags I use... Some cases I use extra php tags's, in others I just echo the html/head/body tags.
- - -
- - -
I would connect, but I think IRC is for #friendless_losers.
Do it all directly in C, or better yet, assembler.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web. I guess this really took a lot of thought to write. Maybe we should hire the people with blatantly moronic stereotypes? I have undegreed and/or people with various degrees (Political Science, Nuclear Engineering, an Army Ranger) working with me and we produce great code. Please think before you write.
reverse the domain to email me
The Microsoft tool is pretty Mickey Mouse in comparison to the stuff I mentioned. They think 100 threads per client machine is a lot, where the other tools generally run 1000 threads per client box. The Microsoft tool works fine if you're running a small site, but if you're Lycos, Yahoo, or Slashdot, you need serious tools.
This is why the company I work for sells big expensive software for testing websites. For a mere (insert insane sum here) you can get Compuware, Mercury, or a few other companies to tell you what will happen when a few thousand users try to access your system all at once and how to fix it.
Hey there, I'm an undegreed kid. No green hair or tongue stud, but I don't have a degree. I've got 30 credits at RPI, and I have 25 at Penn (not Penn State). I've worked a couple years in the real world with amazing programmers. You know the reason I don't have my degree? it'd waste my time. No, this isn't my arrogance. I arranged for tuition reimbursement and sat down with two coworkers who had spent a lot of time teaching me advanced programming topics and they determined that I already knew and was practicing the things I'd learn in further undergrad education. So yes, I could get a degree, and have the highest credentials. I could buy a suit and pretend to be well groomed. But would it make me a better asset to a company? not really, no.
----------------------------
Some people are confused about stored procedures. You can write stored procedures in C, C++ and other languages. Oracle also supports stored procedures in Java, it even has its own embedded JVM that optimizes everything.
If you are doing lots of updates and such it is easier to do them with stored procedures in the database because the database can cache and optimize and doesn't require the latency and overhead of the network layer.
You also can centralize your business logic this way.
However, there are a few things one can do to improve performance:
sigs are a waste of space
I agree with the above. In some benchmarking I did on the "life" algorithm, using exactly the same method to solve it (as best as the languages allowed), Java was 200-400x faster than Perl, depending on what JVM I was using. It was even faster than C sometimes (just compiled with -O though, but to be fair, I didn't use any optimisations for either...)
The point is well taken, but it's also pretty extreme. I'm just saying that bloatware follows more easily when there are copious resources to waste. When there's not enough memory to do the job properly, bad hacks become the norm. But in the middle, there is a sweet spot where there's enough resources (memory, speed, UI) to do the job properly, but not so much that the system is too big (physically, as in the case of the palm) or overpowered for the task at hand. Balance is a good thing!
I think not...(*poof*)
Another area where elegant coding still matters *a lot* is on handheld computers and devices such as cellphones and net appliances. Palm is doing well because they started fresh on a new platform, and the platform (nice form factor and sw, but limited memory and cpu) encourages compact, elegant code. Whatever you think of the various Win32 platform interfaces, it's pretty clear that the UI on WinCE devices is a limiting factor, and the underlying Win32 legacy does not encourage compact or particularly efficient coding.
Can you imagine what a Palm device could do with the power they're throwing at WinCE these days? 32mb Ram? 150MHz? Of course, the downside of Palm's current advantage is that they're becoming complacent and slow. I'd love to see Palm/Handspring maintain their business focus on that sweet spot where the form & function is more than adequate, but doesn't encourage programmers to take the attitude of "aw, hell, we've got cpu to burn!" But I fear they will not. The more things change, the more they stay the same.
Jon
I think not...(*poof*)
Yeah, Netscape 4.x is a bit stupid, but it doesn't help that Slashdot sticks things in a table for no reason at all.
-- Ed Avis ed@membled.com
How about security?
If you program in Perl, do you know what -T means?
If you program in C, have you carefully considered buffer overflows?
etc?
Regards,
Ben
My usual seat in the cluetrain is at A HREF="http://pub4.ezboard.com/biwethey.ht
Unless you're doing something really wicked, I doubt optimization will get you much. Just document what your functions do in some kind of header file and try to make it readable (don't use perl *cough*). Your code may be brilliant, but you want to know what you wrote after you sober up, right? :)
yah, good point.
One could also write all their cgi proggies in a compiled language, and avoid the whole interpreter overhead.
Just be sure to wear the gold uniform when you beam down -- you know what happens when you wear the red one.
Oh, I love my company and will be the first to defend it. I have total access to the owner, and nobody who isn't really good at what s/he does survives very long.
The downside is that with a loose structure comes heavy responsibility. But with that comes the opportunity to do a great job and earn the warmth and affection of the employees, which makes it a really fantastic environment if you can cut it in the first place.
Of course if you can't, it's a real nightmare. The BOFH wasn't fired; he quit because his stress level was way too high. Saved us the bother of firing him.
D
----
Most databases have indexes, which can be thought of as pre-computed sorts. These are used to both retrieve and sort data.
If your SQL database uses indexes effectively, it should be many times faster than perl because much of the sorting has been pre-done for you.
Since I'm writing a response to this already, a few thoughts:
I don't doubt that Oracle is a fine database product, but it's also ruinously expensive and has sluggish connection speed. With mySQL, I can open and close the database in each script I run without significant performance penalty; with Oracle, I'd have to change this and most likely make my code more complex and therefore more bug-prone.
I use C for my CGI scripts primarily because most of them are exceedingly complex, and I feel better using a "real" programming language with low function calling overheads and a syntax that's clean and easy for me to understand later on. I may be paying a performance penalty for keeping things in CGI, but I feel the extra reliability created by the self-contained nature of the programs is worth it. In addition, because I know this stuff cold (thanks to doing it for years), I can crank stuff out extremely fast. If you're starting out, though, you're probably better off using mod_perl or equivalent, unless you have multi-thousand line CGI programs like I do.
D
----
I try to use "lite" JavaScript, sticking to core functions that are pretty consistent between platforms. For instance, sometimes it's handy to open new windows for things like comment screens or email messages; this can be done effectively with very tiny JavaScript programs that have a significant positive effect on usability. For instance, on the web site I've developed for my employer, both employees and the public can "Press the red button" at any time if they have questions or concerns. The red button is far more effective in a new window, since users can check out their problems in the other window while writing the message.
I'm sorry to say I've never heard of XHTML before - care to enlighten me?
D
----
Our company hired a fellow as our network support guy who had ample credentials and was determined to get even more.
Turns out his entire life goal was to get more credentials. When we discovered that they hid a surprising lack of knowledge and a deep-set hatred of users, his days were numbered.
I'm not convinced at all that finishing a structured program (which this guy loved) is good preparation for a business environment - at least not at our mid-sized, free-wheeling, vaguely unstructured company.
D
----
I agree with most of your post, but the above statement is interesting...
For there to be a 19 year old with 7 years working on Apache, she must have started as a 12 year old with 0 experience with Apache. Such a person was definately not a web developer at that time. It's an extreme example, but that may be where the original poster's bitterness about "undegreed" web developers comes from.
200x is a hell of a speed improvement. Java and Perl (when done right) are both JIT compiled languages with similar language complexity. I fail to see precisely how switching similar code from Perl to Java could alter your speed by 200 times. Are you sure you did not alter the order of algorithm complexity? That is the only thing I could see causing such a radical difference in speed.
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
I've done web sites in Perl. Any language can be abused, but for large systems, it is much, much easier to write hideous spaghetti code in Perl than it is in Java. And I hardly call Perl's object oriented "features" (i.e. hacks) appropriate for larger systems, especially when compared to Java, which may not be perfect, but it's certainly a big improvement.
I love Perl, and think it's the greatest thing since sliced bread. But right before a big marathon, I'm going to need a heck of a lot more than sliced bread...
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
- Much, much easier and robust than C or C++.
- Cross platform. Yes, on the server, the cross platform promise of Java works.
- Integration with other languages and platforms, such as CORBA, Microsoft COM, etc.
The Java performance problem is usually not such a big deal on the server, and is addressed by throwing more hardware at it. For well-funded companies racing against Internet time, the above advantages make the extra money thrown at hardware a no-brainer decision.Note that I'm not comparing Java to PHP or Perl, because the category of problems each is suited for is different. For large enterprise systems, C, C++, and Java are your choices. For smaller, script-driven type sites, PHP and Perl are more appropriate.
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
I thought it was just because the code was poorly written. Have you looked at it? :-)
"Cause there's 40 different shades of black, so many fortresses and ways to attack, so why you complainin'?"
Netscape 4 is horribly slow at rendering nested tables. It's not the web designer's fault that HTML was never designed to do what we expect of it today (and nested tables are one way to get a lot of stuff to lay out right). Netscape 3 is much faster, so it's clear that the problem is in the browser. But Netscape 4 has always sucked from day 1. How well Mozilla will do remains to be seen (it's horribly slow now, but I'm told that's because so many debug checks are still in there).
now we need to go OSS in diesel cars
sigh, the old perl vs java debat again (zillionth time?).
"Since when do you need to have multiple server architectures run the same code?"
No vendor lock in. Ability to move your stuff to bigger platform should you need more horsepower. A
"PHP does that. Perl does that (I think.) Cold Fusion does that."
So, I bet you can list a number of other languages that can do that as well.
"Java IS slow. I'm on a AMD K6-2 350 and a Java Hello World took about 3 seconds to respond to a "destroy" request. Most programs take around 1 second. A slightly larger program is slow enough to be unusable."
So, buy some decent hardware, get the right JVM (and OS) and make sure you have enough memory (128 should be enough). Since we are talking webservers here there really is no excuse for crappy hardware and outdated software. Probably development/maintenance cost is much higher than the cost of the needed hardware for a website of decent size.
I worked with Java years ago on 66 Mhz HP-UX machines. I once managed to load and run the corel office demo on it. That was slow (and ugly). But hey, we are talking about a really crappy port of a 1.02 jvm here. I'm currently running JDK 1.3 Beta on my computer at work (PII350, 192 MB, winnt) and rarely have performance problems running such programs as TogetherJ, Forte and JBuilder.
Jilles
Ever wondered why slashdot has been looking the same for years? It's probably not because the owners don't want to change it but because their perl code base grew too ugly to change it radically.
Perl is good for shortlived projects (most websites change every few months). Since it is possible to produce code fast in perl it is ideal for the fast paced internet world where time to market is everything.
In an enterprise situation websites are complex, need to be reliable and contain lots of stuff that really does not change very much over time.
For such systems, you don't want to use ducktape type of languages like Perl. You want to use a language and development environment that helps you address these issues.
Jilles
I think you might have missed a buzzword or two in your post.
No... wait... I can't think of any other buzzwords... maybe you did use them all. You must be a manager.
I recently wrote a replacement for an online retirement plan management system that handles over 12,000 transactions an hour.
I replaced nearly 80,000 lines of Perl with only 20,000 of Java, and even when you include the database time in the total, the Java solution is an order of magnitude faster under no load. Under load, it's 40 times faster and still scaling linearly when Perl has started to cause a cascade overload of the server.
When we don't have to hit the database, our servlets are over 200 times faster than the same page built by our Perl CGIs.
While you could theoretically write something faster in C++, you'd have to go to all the trouble of implementing the equivalent of a servlet engine yourself to start competing for speed with Java's lightweight process model. With Java, you can just start writing your app.
So what if its 5x slower. CPUs and Memory is cheap, web developers are not. I would rather have a slightly slower site in a month than a lightning fast site in 6 months that slowly leaks memory because someone forgot to unallocate a buffer somewhere. Perl (VB, Python etc) are muchmuch faster to develop, easier to debug and less likely to have memory holes than a C/C++ module. For most applications the server is waiting for a database far more than it is crunching data in the module, and for the sort of processing most web apps do there is not much benefit from using C (its not like many people are calculating Pi to n places on a website, its mostly text processing and database accessing).
...in fact msot serious hacking is done by UNIX, and UNIX based systems such as Linux or C++...
--
--
My sometimes helpful blog
Graphic designer for logos/appearance/layout
HTML monkey for layout production
User interface designer (NOT the same as graphic design)
Backend programmer (CGI/PERL)
Database designer (MIGHT be same person as above)
Evil overlord/overseer/webmaster (to keep the VISION of the sight clear and crack the whip on the rest)
Skippy
"False modesty is the refuge of the incompetent." - The Stainless Steel Rat
XHTML? Some combo of XML and HTML?
Its all a system of tradeoffs. The advantage of using a scripting language is rapid development and debugging at the cost of the richer functional libraries and preformance of a compiled language. Another advantage for project managers is that they can use junior programmers for the first tier and spare the attention of senior level developers on the second and third tier.
This makes planning essential. If you identify what pieces will be written on which tier, your life will be easy or a living hell. Most techniques in web application programming are found in articles. Even if you're programming in PHP, look at ASP websites as they'll often have extremely helpful article on using components, how to structure your database, caching tricks, and other information which is platform independent.
Dr. Fardook drfardook@evilconspiracy.com
(flipped through replies but didn't find too much on this)
/. doesn't do static page generation - people's user profiles comprise a fairly small set of look-and-feel options. Apart from that "page generated for Mr. Foo Bar by a marauding flock of random geese" thang, there are only so many combinations for the rest of the page, and generating them in one shot would make /. pages *so* much quicker to load. Likewise, regenerate only the required pages on every update (article or comment). You don't lose any customizability that way, and gain hugely on response time and webserver throughput.
As far as possible, stick to occasional generation of static web pages, rather than generating them on the fly with mod_perl or mod_php or CGIs or whatever. With clever design, I think a huge number of websites can be converted to a statically generatable version, escalating performance in a big way. You can run the html generator (which could be just CGI >foo.html) using either a cron job, or run "make" whenever you update any content.
As an example, I've always wondered why
Read his first paragraph again:
You could optimise the other 90% - but why? Programs tend to do alot of looping over a very small percentage of their total code. Most of the statements in your code will get executed only once, but a small chunk of it will get executed 100s or possibly even 1000s of times.
If you can figure out where most of the time is being spent in your code, and optimise that, you'll be much better off than simply optimising some random parts. If you can somehow shave 50% of the speed from a part of the program which is only taking 10% of the total time, you've saved 5%. If you can shave a mere 20% from the part taking 90%, then you save 18%.
It's not a matter of being half-hearted... it's a matter of being intelligent. Why optimise and possibly obfuscate most of your code, when doing it to a carefully chosen section is so much more effective?
-Spiv.
In my experience, and most will agree, you should always write code to execute as effeciently as possible. I doubt Rob, years ago, when first designing Slashdot had effeciency at the top of his list. He couldn't forsee the millions of hits the website would take. As a web contractor, I've learned the hard way to always comment code, and use objects/classes. Most of us when doing contract work, never think about who/what is going to maintain it after we leave/job is done. I often look back at old projects and wonder "What the hell was I thinking", or maybe even question what the program does, even though I wrote it. I've encountered the slashdot effect a few times on our server, and wished I had written the code better. Just my 2 cents
As with everything else, a degree is what you make of it. While I was getting my degree, I saw ALOT people coast through, taking the courses that required the least amount of work. Not coincidently, those courses were the ones that provided the least benefit to the students, but they were the ones with the highest attendence.
So I don't think that anyone can say with absolute conviction that a coder with a degree is better than one without - simply because there are a lot of slackers and guys who see that CS is going to pay them $40k/yr out of college.
On the other hand, I believe that taking the courses required for a degree is important. Computer science theory, although boring in general, is almost a necessity to achieve a thorough understanding of what you're doing. With the background in theory, you can quickly and easily apply techniques in any language, because you know why the techniques exist. I think you'll find that one common attribute of the authors of the most common computer languages - higher education.
To say that you don't need formal training is arrogant and short sighted.
Some people have stated that owning a degree shows that you make a good parrot - being able to recite word for word what was shoved down your throat. I would suggest that the lack of a formal education makes it very difficult to truly learn and utilize differing techniques. I think that you'll that alot of people without the benefit of a degree fall into the trap of seeking out someone else's solution to a problem they have, but never fully understanding the solution. If they don't understand the solution, they attempt to apply that solution everywhere they can, because that solution is "in their toolbox". These people expand their abilities by acquiring more and more code from other people, filling their toolbox but never fully understanding these new tools.
With the formal training and the expose to the theory, a person is more able to develop their own solutions and to fully understand someone else's solutions. This, I believe, would make that person a better programmer.
PHP 4.0 and the latest version of ASP (don't use it don't know the version #) are both compiled to bytecode (very much like Java). JSP is *parsed* by the servlet is been run under, translated to a plain Java file and then compiled to Java bytecode (maybe newer JSP engined compile directly to bytecode, I am not sure, this is the case with JServ 1.1 + GnuJSP).
Now, stored SPL procedures (which is not the same thing as SQL --SQL is the root of most SPL languages, but pretty much each vendor has their own) are also compiled to bytecode and stored within the DB structure.
So far, if we're comparing the two schemes, we've gained nothing in comparison: both language genres compile to some machine-independent bytecode that's supposedly slower than assembly and faster than interpreted languages.
Under this layer is where things get interesting: how does SQL/SPL access the data structure? through a proprietary, vendor specific library. How do PHP/ASP/Java? thru SQL. That's where the performance difference comes in.
What would you rather code in? SQL or PHP? Java or some vendor specific stored procedure language? (which not only locks you in, it probably has less features and is not as robust). What would kick some major ass, would be to circumvent SQL and access the data structures *directly* from PHP/Java. SQL is not the end-all, be-all of data access. It's just entrenched, and the lowest common denominator (you can always figure a way to do anything in SQL).
The other layer you're missing is the data-structure itself: the DB vendor (Oracle in our case) doesn't know what you're gonna end up doing with the DB --thus, they lock you in to certain type of data structure (and no, not all DB data structures are created equal). That's really the reason we now have really fast-read RDBMSs like MySQL that suck for transactions, cool OOP-like ORDBMS like Postgres that suck for reads, and a multitude of commercial DBs that each have their niche.
But the end-developer (the PHP author here) *does* know what each data structure will end up doing. So, if you can have a common API from PHP that can access several types of data structures *directly* (NOT thru SQL), and you could specify what you want (fast read, object-store, transactions), all the advantages of a proprietary RDBMS go away. That's where OSS could kick ass: write the structures and the libraries to read them and use a common, rich language to access them *directly*.
We're not really disagreeing; C++ will always be faster on the server side. But the server side is controllable by the developer/deployer: you can always add more muscle to your servers. Development time, flexibility and robustness are more important.
engineers never lie; we just approximate the truth.
Errr... Stored procedures in DBs are basically compiled to bytecode that has faster access to the DB structure (i.e. probably a C library).
;-)...
Now, contrast this to a modern scripting language (PHP4, ASP) or Java --a high level language (not unlike a SPL) that also compiles to bytecode. The performance difference here is access to the data structure: in all the languages above (ASP, PHP, Java) the way the DB is accessed (i.e. the speed of the JDBC, ODBC driver or PHP module) is what determines the speed of access to the DB.
In other words, there really isn't much of a difference between using an SPL in a DB or an external scripting language that has fast access to the DB data --e.g. the PHP module for MySQL. I think what confuses you here is that Oracle has *never* given a C library (or anything similar) for access to the DB structure, forcing everybody to use their proprietary SPL.
IMHO, the future lies with highly developed, open and popular languages (PHP and Java being my favorites here) that have C-level access to data structures *directly*. I.e. you could potentially get rid of SQL (or make SQL another language with access to the C library) and, more importantly you could have different types of data structures in the same application: e.g. a very fast read-only DB file (think MySQL) and a robust object-store DB (think Postgres) that look and feel the same to the end developer (the PHP dude). If anybody wants to start an OSS project like that, count me in
engineers never lie; we just approximate the truth.
My employer does a lot of data-mining at a very low level: we have one of the most complex read *and* write data-mining applications out there, and yes we do a lot in SPLs, and SQL and C++ and even our own in-house languages. I am confident (in fact, I know) that you can considerably out-perform SQL if you write "to the metal", in this case the data-structure itself.
Knowing this, and the problem I laid out before (that the DB developer doesn't know what the application developer needs) I have personally started to question the need for an RDBMS, particularly in the case of Web applications.
Let's face it, a full RDBMS for a data-store of a few TBs makes perfect sense, but for a website of a few pages (even a few hundred pages) it's overkill, especially, as a lot of posts have said so far, since that's where your bottleneck is in dynamic sites. So why are we running full RDBMS, or even low-powered clones of such (i.e. MySQL)?
I am willing to bet performance would go wayyy up, if you could have a relational data structure directly available from PHP (a basic B* tree, for example). Why?
One factor: PHP does caching; but it can't cache an *external* data source, such as a DB table, because it can't be sure that it was not modified by someone else; but, if the datasource was *internal* to PHP, caching of records, queries or whole tables would be simple: less disk/data access = more speed.
If you are not familiar with the wonders of PHP4 (not 3.x), check out php.net (and the best online manual in existence) and the now private company behind their engine, Zend. The Zend folks have recently got together with the MySQL folks, to further integrate the two platforms; there is a chance they will end up with a direct access scheme --and that's the main reason I am betting on PHP...
engineers never lie; we just approximate the truth.
HTML files with PHP are read and evaled every time they are called. With mod_perl handlers, the scripts are compiled when Apache starts up and then never gets recompiled until you restart the server. That is the difference. The same thing goes for ASP scripts, and other embedded scripting languages (this actually includes things like ePerl, Embperl, and other embedded Perls). So, PHP is fast (no spawning a new interpreter), but the pages need to be read every time they are called.
I believe that PHP4 will have the option to compile HTML/PHP pages, so this will not continue to be an issue.
darren
Cthulhu for President!
(darren)
I'm only very familiar with Microsoft SQL server,
(if you have to use this product I'm sorry for you, but sometimes you have to learn about it anyay)
I'm not certain of the support in other sql servers (although I'm sure major servers such as Oracle include server stored procedures).
Essentially what happens in Microsoft SQL server
is that you can create certain scripts in T-sql (transact-sql) which can be run as queries or within triggers (scripts that execute on insert, update, or delete).
The cool thing about stored procedures is when you're using ODBC, which can be incredibly slow, especially with large recordsets. You can move large updates (things like updating the dates on all records in a query) onto the server, where its caching and local data makes it MUCH faster.
Doug
Venn ist das nurnstuck git und Slotermeyer? Ya! Beigerhund das oder die Flipperwaldt gersput!
Your (php,mod_perl,compiled CGI) code can be beautiful, elegant, optimized, efficient, easy to understand, and simple to edit/extend, but it makes little or no difference whatsoever if 100K still has to be delivered to the client, interpreted, and displayed. I think the quantification of 'good web code' should be dominated by "bang per byte the client has to deal with" rather than more traditional programming standards. Subjective, I know, but I think given the current constraints of the medium more attention needs to be paid to what happens on the client end (end-user available bandwidth id not increasing as fast as processing power does {not Moore's law!}).
I am quite civilized, and I should be brought a beer immediately. -- Bruce Sterling
Umm, for sql server, look at msdn... Oracle calls 'em something similar. I'm sure you can find stuff on other sites. Sorry, my SQL experience is limited to those two (and very little of the latter.) I know personally you can do a lot with TSQL on m$ offering. Plus, even a select query is good in a SP if you use it all the time, 'cause the execution plan is pre-compiled and the overhead is lower. Plus, if you want to change the damn query, you just change it once. SP's and the like (I think just about any decent RDBMS will have 'em) get you a lot of the benefits of 3-tier (offload CPU/Disk access off web server, logic only in one place) without the complexity.
---
DO NOT DISTURB THE SE
The problem isn't the new technologies, it's people who use those technologies without following the standards. If you read and follow the standards put out by the W3C, you'll find that the standards are very much geared towards the creation of webpages that are accessible to all clients, whether they be text only or graphical.
<sigh>
Maintainability.
6 months from now, can the author undertsand the code? Can the other programmers? Can the project manager? Well enough to be able to make an informed decision whether it can be moved forward in a certain direction?
Maintainability. It's not just about writing comments, it's about clean design, a sense of what separation, encapsulation, and inheritance truly is for. It isn't just to debug better, and certainly not to make faster code (but code faster, maybe).
Maybe your code is perfect now. But it won't be 6 months from now. That's what my degree taught me, and boy am I grateful.
It's why I dare call myself a software engineer. Maybe I am a lousy engineer, but at least I understand the priorities involved in that word.
Please, insist on hiring only the most credentialed, and best groomed crap out there.
t
- bloat
- poor FORMATTING
- and
- other
simply mythY M M V
XML causes global warming.
You seem to be saying the second. But only the first is true. There are better ways of optimizing non-SQL logic, than to bog down your database server with it.
I apply what I know about algorithms and data structures to make my code as maintainable as possible.
I hate to be crass.. Did you read what I wrote dude? Ive been doing this for two years.. Ive learned a lot the hardway about maintaining my code. Let alone someone else who did not READ my post moderated you up. Ive spent countless hours doing stupid things like commenting and making variable names make sense and code layout make sense..
And I dare call myself a programmer...... Did you see that part of my post?? Seriosuly..Yes I combined a couple of thoughts since algorithms do not apply to code maintenance its more common sense. But yes it is something I strive for since *I* have and do support contract work on occasion. Which means going back over my code. Like I said ive learned the hardway. Im glad you got to learn about it in college.. I learned about it working.
- Optimize your queries. Don't use more queries than you absolutely need to. For instance, a lot of people show the latest n items and also show the latest date the page was altered. Instead of querying twice, just use the date on the most recently dated item in the recordset the first query got you (assuming it returns the adddate.)
- Use cache. When someone requests your page, simply check if the database changed since the previous hit - if not, then show him cache you generated.
- Use your database. I've seen loads of sites using very advanced database-software that supports stuff like stored procedures, triggers and views, but the developer only uses it for raw SQL-querying. Stored procedures for instance do wonders for performance.
So the best thing to do first when building a high-performance database-driven website is contemplate and set up a datamodel. It's essential (get some books)CGI? and Perl?
.com/javaworld/topicalindex/jw-ti-ejb.html Or here: http://www.cs.rutgers.edu/~shklar/isas98/
Not if you want to handle millions of transactions, you won't.
Distributed N-tier architecture, scalable and capable of handling serious loads. Good load balancing in software and hardware. Distributed databases, LDAP for user reg's, cache for the data. Batch processes that run when there is not too much traffic to update and clean your cache. Transaction management, distributed transaction management on different systems.
Resource management, memory, RAID systems, caching, HD space.
If you want to be multiplatformed and n-tiered, go here: http://divya.com/pubs/ejb/ and here: http://www.javaworld
And all this is just a beginning of understanding. You actually need a good Software Architect(s) in order to optimize your performance under heavy loads, good Interface Designer(s) to optimize the user interface, good graphics designer(s) to be efficient and good looking, coder(s) who do your *ml work
here is what I do for living, just click on 'Product' there.
You can't handle the truth.
Of course it should be noted that some degreed professionals also have green hair and/or studs through their tongues.
One of the most important aspects of any optimization process is profiling. Unfortunately profiling is also one of the hardest tasks to accomplish when it comes to web development. If you can't identify where your bottlenecks are or where the most time is spent, you can't optimize very well.
The fact that it's fairly common these days to find that a web request lands on a load balancer which forwards it to a webserver which may dynamically generate a page which makes a call to an application server which may execute a stored procedure by going through some sort of middleware database connector to a RDBMS on the backend means that there are a lot of different things that have the potential to be optimized.
I think some of the most important advice to keep in mind is that generally the optimization technique which yields the best results are architectural redesigns. If your architecture is inherently slow or imposes inherent limits on your performance, it doesn't matter how much you tweak the individual components.
Once you have an optimal architecture, each component in the architecture needs to be examined discretely. For example, there are a lot of optimizations that can be done to perl code that don't apply to Java at all, and vice-versa. Once you get to this stage, read every article, buy every book, etc. that you can regarding your specific technologies.
Our company is developing search technology using Java and by following the java-specific advice in the 'Java 2 Performance and Idiom Guide' we were able to reduce the time it took to do an average search from 10-30 seconds down to 100 to 300 milliseconds.
/** If you see this comment twice, its because I was a dumb ass and accidently posted a reply in the wrong thread the first time, doh! **/
.PNG .JPG. .)
a l
I too am a professional PHP developer. I have found that planing ahead is the best way to write good code.
I always try to build small functions, that can be re-used and generate clean and quick code.
However, as a coder, you need to be aware what the design team is doing. I fortunately do a lot of my own design, and
am aware of how large file sizes and extraneous "extras" like javascript buttons, popups, etc waste resources.
Unfortuately the majority of designers I have worked with have been paper publisher, who have only recently started
producing layouts of the web.
I am contantly telling them to avoid things like background gradients, and lots of images fading from one to another.
Sure it looks good, but doesn't lend itself well to scaling.
On some projects it is frustrating to have to deal with browser incompatibilities, resulting in extraneous code, to use
one block for Netscape, and another block of code for I.e., and maybe a third for Lynx. This scenerio doesn't happen
too often, as planning inadvance helps alot, as does design for the lowest common denomiator (I usually target a 3.0
browser at 640x480).
Sometimes you're lucky and the project is for a local intranet. Those are the best because you can design for the
standard browers (usually a I.e 40 or 5) and can rely on a fast connection.
My list of things to keep in mind when coding for the web:
1. target audience, (what do we expect the clients to have)
2. Do I need a database to store my dynamic content, where is that machine at in relation to the web host? What
can I do to cut down on database transactions?
3. How many of the viual elements can be reused. I try to break down the site into sections, a header section,
footer, navigation, body, etc. the more these peices can be reused the faster, your site will be, and the easier it
will be to maintain.
4. Graphics effientcies? Where can I subsitude background colors in tables for an image. If there is an
animated image, can I chop it up so that the animated portion is as small as possible, and store the image layout in a
table? (.GIFs are bad, and animations are annoying, but sometimes the client insists) What format is the smallest
or
5. Maintainability. This includes this liek a meanful directory structure, consistent capitalization, comments,
etc. While it is important to do things the right way up front to save time in the long run. I have found that when i
start off quick and dirty, the project stays that way. You almost never go back and add comments or clean the code once
it works, you only will do so if there is a problem. So I feel it is important to address as many issues as you can
upfront. Plan for easy maintainability. If your logic is easy to follow, it will be easy to maitain (and therefore
there will be less temptation to put inefficient band-aids in place.)
6. Testing. Make sure you test your site and application on as many browsers and at as many resolutions as
possible. And even if you have a DSL or CAble modem, get a dial-up account to test the access. (A ppp link over a null
modem with the speed turned down also will work for this. .
For references I use a pretty standard HTML syntax book, an Older version of Javascript Unleashed (sams), I also highly
recommend the MySQL mSQL by Reese and King (O'Reilly) and The PHP on-line anontated manual (http://php.net/ docs.php3).
When I get stuck, or what advice I always go to the mailling list archives at
http://www.progressive-comp.com/Lists?I=php-gener
-MS2k
I too am a professional PHP developer. I have found that planing ahead is the best way to write good code.
.PNG or .JPG. .)
a l
I always try to build small functions, that can be re-used and generate clean and quick code.
However, as a coder, you need to be aware what the design team is doing. I fortunately do a lot of my own design, and am aware of how large file sizes and extraneous "extras" like javascript buttons, popups, etc waste resources. Unfortuately the majority of designers I have worked with have been paper publisher, who have only recently started producing layouts of the web.
I am contantly telling them to avoid things like background gradients, and lots of images fading from one to another. Sure it looks good, but doesn't lend itself well to scaling.
On some projects it is frustrating to have to deal with browser incompatibilities, resulting in extraneous code, to use one block for Netscape, and another block of code for I.e., and maybe a third for Lynx. This scenerio doesn't happen too often, as planning inadvance helps alot, as does design for the lowest common denomiator (I usually target a 3.0 browser at 640x480).
Sometimes you're lucky and the project is for a local intranet. Those are the best because you can design for the standard browers (usually a I.e 40 or 5) and can rely on a fast connection.
My list of things to keep in mind when coding for the web:
1. target audience, (what do we expect the clients to have)
2. Do I need a database to store my dynamic content, where is that machine at in relation to the web host? What can I do to cut down on database transactions?
3. How many of the viual elements can be reused. I try to break down the site into sections, a header section, footer, navigation, body, etc. the more these peices can be reused the faster, your site will be, and the easier it will be to maintain.
4. Graphics effientcies? Where can I subsitude background colors in tables for an image. If there is an animated image, can I chop it up so that the animated portion is as small as possible, and store the image layout in a table? (.GIFs are bad, and animations are annoying, but sometimes the client insists) What format is the smallest
5. Maintainability. This includes this liek a meanful directory structure, consistent capitalization, comments, etc. While it is important to do things the right way up front to save time in the long run. I have found that when i start off quick and dirty, the project stays that way. You almost never go back and add comments or clean the code once it works, you only will do so if there is a problem. So I feel it is important to address as many issues as you can upfront. Plan for easy maintainability. If your logic is easy to follow, it will be easy to maitain (and therefore there will be less temptation to put inefficient band-aids in place.)
6. Testing. Make sure you test your site and application on as many browsers and at as many resolutions as possible. And even if you have a DSL or CAble modem, get a dial-up account to test the access. (A ppp link over a null modem with the speed turned down also will work for this. .
For references I use a pretty standard HTML syntax book, an Older version of Javascript Unleashed (sams), I also highly recommend the MySQL mSQL by Reese and King (O'Reilly) and The PHP on-line anontated manual (http://php.net/ docs.php3). When I get stuck, or what advice I always go to the mailling list archives at http://www.progressive-comp.com/Lists?I=php-gener
-MS2k
Being a rather intellegent 14 year old, I'm amazed at how many people on slashdot are actually bashing the younger generation for being idiots, and how a stereotypical bad web designer is gothic with green hair and a tongue piercing.
I know and enjoy reading quite a bit of literature and writing as well, so in sixth and seventh grade I got the wonderful opportunity to visit a young writers/readers convention. You would be amazed at the number of teens that are gothic or do have green hair. If you can actually see through the outside and the green hair you realize that some people are incredibly smart. On the flip side, some kids that look like idiots have talents that would amaze any intellectual person.
If you go through life thinking that everyone younger than you is stupid, you need to rethink your ideas. Be it generation X, the baby boomers, whoever was a teen when you were maturing, remember that someone though the same thing as you. Do you like that feeling? How many people who cruise slashdot did amazing things as kids/teens, whether it had anything to do with the internet/computers or not?
How many of you were harassed or beat up at school because you looked different? Reread the hellmouth threads. I know several goths and it has nothing to do with psychotic devil worshiping. It's about being different.
I built my own site, using no my fat brain. Go ahead, www.nerdnetwork.net, look at all the meta tags, you won't find a meta name=creator m$ frontpage, or anything else.
Don't let a stereotype get the best of you.
Kris
botboy60@hotmail.com
Nerdnetwork.net
If your bottleneck is the scripting language, i suggest getting something besides a 286 for your webserver. Although that Cray that's running your RDBMS must be pretty sweet :)
That all depends where the degree comes from. If the school had a good CS program, you will end up with someone who should be able to pick just about anything up and learn it quickly because they have a good background in theory. If the school had a bad CS program, they learned how to program well in one language on one platform and will be unable to learn anything else since they do not understand the underlying theory.
Perhaps I'm biased because I think my own CS degree has been extremely helpful in my career. I have worked in a research job as well as a development job and been successful in each.
However, last time I checked there were 3 IT jobs unfilled for every filled position. In some areas the ratio is 6:1.
A smart company IMHO would try to recognize actual talent than be swayed by that degree, or certification for that matter. By your logic, a 19 year old kid who might have spent the last 7 years working on Apache would be unqualified to be a web developer!
Luckily, there are many companies who don't use this retarded logic when hiring.
No, Thursday's out. How about never - is never good for you?
Java is notoriously slow. It's bad enough on the client side in an applet but it seems like a tremendous waste of cycles to use it on the server side. Granted (afaik) each instantiation of a servlet doesn't bring with it another VM process but the performance hit is still there. How does Sun get away with this?
These are just a few things I have observed that may help.
Devolver's Homepage... more fun than a box of crackerjacks.
What's more, the rendering task is distributed, while the generation is centralized at the web server (and associated applications). Thus, the effect of code inefficiency is magnified on the server side.
Optimization of server-side web code isn't really that different from the general problems of parallel-process and database-access optimization. There's nothing magical about the fact that the front end is a web server rather than (say) a 3270 or the like. All the usual problems are there -- resource shared vs. exclusive access, race conditions, process model tuning, and so forth -- and all the usual solutions apply.
Many companies go astray when they fail to realize that a web app is just a traditional app with a particular front end. This leads them to believe that the rules of good enterprise software development don't apply, and 99% of the time disaster ensues, either immediately, or when they attempt to scale the system up or add/change features.
When all you have is a hammer, everything looks like a skull.
Philip and Alex's Guide to Web Publishing is a great book, but it's not a good reference if you're looking for nitty-gritty coding knowhow. It seeks to avoid that stuff, since you should be able to think about building web sites on a higher level, where the particular language doesn't matter.
However, if you'd like to be a web coding ninja, do problem sets 1-5 on this site. If you don't have the exact environment in which to complete them (AOLserver, Oracle, CyberCash account), improvise. After completing those, you should be able to tackle anything, and you'll know how to code a great site.
Two things: first, Perl isn't really slow; when you run it, most of the time is spent running Sarathy's heavily optimised code to handle things like hash lookups and regexes, not your punny little userland code ;) It only _seems_ slow because of the overhead imposed by having to spawn a new interpreter and recompile the script into bytecode every time the script is run. But most of the problem is easily fixed by mod_perl.
:) Instead, perlcc embeds an entire copy of the Perl interpreter, plus a shitload of unnecessary modules, along with your program in bytecode-compiled form. The result is bloated, buggy, ugly and generally not much faster than just running your script. So it's not of much use, but if you really want to do it, you get to.
Second, yes, the perlcc utility is able to compile a Perl script... but not by translating it to native code. No, that would be just too easy...
(P.S.: Cool username... I've always preferred post-Syd Floyd myself, though.)
To the editors: your English is as bad as your Perl. Please go back to grade school.
"Undegreed" is the most worthless term I've ever witnessed the application of in reference to the ability of an individual to write quality code.
I'm not going to purvey any content in this post other than the degree to which I am dumbfounded by this assertion: VERY.
Wow, Bob. You're astonishing.
"Why use HTML when javascript, cascading style sheets, java, cookies, imagemaps, and shockwave will do?"
Why do web developers feel the need to use every technology available on a website when they usually didn't need anything more complicated then plain-ole HTML? I dono how many times I've seen implemented in javascript the meta refresh tag!
Add on to the fact that most sites have become unnavigatable by text browsers and their users (such as the blind). My old boss used to have a saying: "A picture is worth a thousand words, unless it's a picture of a word".
All this 'rich content' that is arguably unnecessary and excessive means that the web is just that much slower, needing even more bandwidth to get anything done.
And my last gripe is about modifying HTML that was created in a HTML editor. Did any of the programmers who coded up these editors ever hear of wordwrap?? Every time I edit a file I have to re-format the whole thing to make it inteligable.
Yes, I'm griping. You can think of me as the crotchety old man who sits on the porch "Yeah, I remember the good old days, back when people edited HTML with 'vi' and we didn't have all these fancy-pants 'wizards' to generate webpages; you could search the web then and find the information you were looking for instead of a bunch of infomercial-like websites with too much animation hawking things. Why it must have been waaaay back in 1995, yeah, Yahoo was still a stanford tilde account back then."
-- Greg
Slashdot, would a spell-checker for posting be too much to ask? It's not rocket science!
A lot of people have said that the bottlenecks tend to be generating output, making database connections, and the like, but that is often not true. Generally, the bottleneck is the network connection. The way I often think of pages is in terms of the connection speed of the server. I have done work for people with slow connections from the server to their backbone, and in cases like this, connection time becomes less important than network speed. So I send the header, send some content for the top of the page, and then make my database connections, and do my calculations.
Even though it's often frowned upon, I like to use a lot of includes that are generated by the client whenever possible. That way, the multiple hits to the server (i.e., more network traffic) can make a slower server seem faster. When it comes down to it, when poeple say they want the web to be faster, they mean a fast experience, not a fast server.
Just something to keep in mind.
darren
Cthulhu for President!
(darren)
There is a classic CS quote that says that a program spends 90% of its time in 10% of its code. Make this code run twice as fast and you will nearly double the speed of your program. Optimise everything else and you won't see any difference at all.
The important thing is to write clean, readable, code. Do not be overly concerned about going to great lengths to improve speed during development. As soon as you have a working system, build a set of automated testing tools and generate some benchmarks. Determine where the bottlenecks are and try to fix them. If you don't know what is really slowing down the system, you will waste many hours optimising code that has no need to be optimized!
This is a trivial operation when using a language such as C with no threads and/or accesses to remote resources. (Although despite this fact, I still see people trying to write optimized spagetti code and never touch a profiler.) Things get a little more difficult when you start talking about database-driven web pages since there are so many more pieces to analyze.
In addition to analyzing the execution time of your script, you must also look at the execution time of database queries. Unfortunately, it is not always possible to profile individual queries, and some databases such as Oracle are highly tunable (i.e. you can tune the system to acount for perhaps poor database design.) When dealing with databases, you are often disk-bound as well, so applying the same profiling techniques to your hardware will often provide good results.
When your database and web server are on the same machine, just using ps or top will give you a good idea of where your cpu time is going. And don't forget to make sure that you aren't swapping. If you are swapping, optimise for memory use or buy more ram. If you've maxed out the RAM on your machine, consider splitting the application onto multiple machines. Sometimes it is cheaper to solve a problem by throwing more hardware at it, but this should only be considered as a last resort.
BTW, I'm assuming you've already done the obvious and are taking advantage of the best technologies such as mod_perl or Java servlets. Forking several copies of Perl just to generate one web page just isn't terribly smart...
-p.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web.
YIKES big guy back yer arse up please. You have got a chip on your shoulder about undegreed programmers.
Ten years ago, that attitude would have worked.
Let me tell you a little about myself.
I am 19 years old and I DO consider myself a web developer and a pretty damn good one.
No I do not know everything about computer science. No I dont know a WHOLE lot about math and the nitty gritty details of algorithms and all that. I DO make it something I work very hard to understand. I strive to improve my code and my level of professionalism ALWAYS. I study algorithms, and programming a LOT.
I try and give clients or my work VERY debugged code with no errors, make it look professional and make it work in an intuitive and effecient manner for them.
Effciency of my code is ALWAYS a concern to me and I try my best to make optimized code. I apply what I know about algorithms and data structures to make my code as maintainable as possible. I dress in casual business clothing and I in NO way claim to know it all. And if imasked if I can do something and I cannot I say NO I can't do that right now, I am capable but I would have to learn how. That is my business philosophy and its gotten me a lot of work and me and the clients have been very pleased with the results.
In your mind I am a 'kid' but I am a professional and I attempt to also incorporate ethics into my code use and my business practices and manners.
And something people do NOT like to admit.. Some people my age are VERY capable of fulfilling senior web development positions No I do not match some god forsaken profile for education nor experience.
The sadder part is I have met people with CIS, and CS degrees who are just absolutely clueless. They needa be beaten to DEATH with the cluestick. It causes me great pain because I know they make a lot more than me and here I am training them. Yeah I know a degree says something about a person and that they were abble to attain it.. But open your mind a little.
There are some of us that are professional and professionals.
So I am 19 with two years of 'paid' experience. Does that make me less of a professional if I admit my limitations and am constantly expanding my knowledge and limitations to beyond even my expectations?
I was stuck into a huge business APP and I worked my ass off.. but I groked it and was a productive member of the team. *shrugs* Your comment was very crude and not well thought of.
Yes I understand your viewpoint but you can say the same about ANYONE who claims to be a programmer.
Jeremy
Phillip Greenspuns work is highly recommended: Go check it out!
The problem is that most dynamic web pages (Active Server Pages,php3,Cold Fusion,perl,whatever) are produced with scripting languages that pale in speed and functionality when you compare them with a compiled piece of executable code or a highly optimized SQL stored procedure.
Also, code is just plain ugly in ASP or those other scripting languages. You don't want to cram a lot of logic in there, because it's hard to maintain
So, the solution is to remove as much functionality as possible from your server-side scripting language. Put all of your buisness logic and CPU-intensive operations in SQL stored procedures, compiled code, or some other equivalent.
In industry buzzword-speak that's 3-tiered architecture. (Tier1: ASP,php,etc. Tier2: SQL stored procedures,compile code Tier3: the database
The end result is that your web site is :)
a. easier to maintain and
b. you're reaping the performance benefits as well
OtakuBooty.com: Smart, funny, sexy nerds.
Don't optimize a single thing until you find there is a problem! Instead, write clean readable code that you can work on later if you need to.
I've spent the last couple months fixing and replacing a bunch of web code written by a guy who thought performance was everything. You have to get it working first. His code does the wrong thing incredibly fast. Bloody Fucking Useless (TM). I'll spare you the details of his N-Tier Network with Application Servers and crap. I've rewritten half of it in plain old (slow) ASP, and I'm getting much better performance and an easier time fixing bugs.
Once it works, evaluate whether you have a performance problem. If you don't, then leave the damned thing alone.
If you do, start profiling. It's not wasting time where you think it is, guaranteed. Ask the super dawgs of optimization (the John Carmacks, Michael Ahbrashs etc...) how many times they just "knew" where the problem was only to be totally surprized once they broke out and profiled it.
Anyone complaining about how Perl, Python, ASP, PHP or whatever is so much slower than C/C++ is really missing the boat. Web pages spit text out the front door, and talk to files or database through the back door. There is seldom an algorithm to be seen, and most the of processor time is spent doing something other than interpreting your script.
It's a lot easier to optimize a query, add an index, put more memory in the server, disable the screen saver, whatever than it is to rewrite everything in C++ using ODBC.
Once you do rewrite it in C++, you're back to where you started. It's still too slow, and now you need to restructure the query, add memory, turn off the screen saver or whatever...
One other thing, don't believe the hype. Everything latest and greatest is not necessarily fastest and best. Microsoft get's paid when you replace your old perfectly good stuff with whatever they've just released. No one is ever going to convince me that ADO is faster than ODBC when ADO uses ODBC to do its work. What the hell is MTS good for? It's a solution to a problem I haven't seen.
bob you sound bitter. some kid steal your job?
last time i was in college, i missed the html degree signup sheet....damn.
"This is where god would go if he wanted to get off blow!"
In most cases the easiest thing to do is simply buy more hardware and replicate. This is easy to do if you're serving up static (or even dymamic) web pages that don't need to share any state across multiple web servers. This is feasible because in many situations the bottleneck of the site is the web server and dynamic page generation itself, not access to the back-end database. For really large sites, a replicated/clustered database is going to be needed, not just a single machine running MySQL.
The Application Server industry has started to provide some solutions for building scalable web sites -- for example, IBM WebSphere, BEA WebLogic, and other systems provide a platform for building web-based applications, which generally consist as a middleware layer between an HTML/presentation 'front end' (i.e., a web server) and a database 'back end'. These middleware systems support replication and clustering to increase efficiency. Replicating the front-end web servers is easy, and many products support this. You can even buy fancy network switches which load-balance HTTP requests across multiple web servers. Replicating the back-end database is much harder, but that's how companies like Oracle earn their money.
Not surprisingly nearly all heavily-loaded Web sites have to do a huge amount of work to get all of this working right. One thing you might ask is whether the process of building a scalable Web site could be made simpler. That's the goal of the research project I work on at UC Berkeley, called Ninja.
Matt Welsh, mdw@cs.berkeley.edu
There are books written on how to optimize code in general:
-Code Complete by Steve McConnell, Microsoft Press
-Writing Efficient Programs by Jon Bentley, Prentice Hall
-Flooring It: The Optimization Challenge by Michael Abrash, from PC Techniques 2, no. 6 (February/March 1992)
But code tuning can have varying results under different languages and environments. A lot of people seem to think that code optimization is unstable and difficult to maintain.
There is sometimes a difference between "efficient code" and "good code". Code may be efficient, yet hard to read and maintain.
To know what techniques are the best, a lot of experimentation is in order. It is also a good idea to become part of a community of experienced web developers, and learn from their experiences.
The greatest boost in performance I have seen in my web experience has come from using the mod_perl module for Apache, not necessarily because of anything I did in the code itself.
In dynamic sites the bottleneck is generating the content (running servlets, accessing databases, etc.), so it makes sense to optimize there. Unless your pages is /really/ complex, or the browser /really/ stupid, rendering time is negigable. And in any case, you can't do much about it even if you /do/ have the best code. Pure HTML just isn't that much of an issue. Moving into XML and XSL, and DOM manipulations, though, client-side performance may become more of an issue.
It's 10 PM. Do you know if you're un-American?
I've always like Phillip's Greenspun's ideas on web development. In a nutshell, he preaches reliable and efficient code/tools. He can be a bit biased but mostly his ideas are good (according to me).
His book is available online for free.
That which does not kill me only makes me whinier
That's so true.
My old company assumed that just because someone was a "web designer" and maybe knew a little javascript, they were qualified to do programming on complex web sites (e-commerce, etc.) They're not anymore qualified to do database work than any old artistically inept programmer is qualified to design a new logo for a company.
Companies need to wake up and realize that it takes two types of people to build the web: "real" programmers, and graphic designers. Of course some sites only need one or the other, and some people are talented enough to fall into both categories... but don't assume that Joe the Graphic Designer can code a linked list or a SQL statement...
OtakuBooty.com: Smart, funny, sexy nerds.
As far as HTML on a page, I found that you should optimize for Rendering Time, which isn't always the smallest page, esp. when it comes to tables.
For server side programming, unless every page is generated dynamically, you should program it the easiest way possible and throw some CacheFlows in front of it.
Of course, since most of the people selling themselves as "web programmers" are undegreed kids with little formal knowledge, there's a lot of crap floating around. Until employers value experienced professionals with the proper credentials (instead of thinking that every kid with green hair and a stud through his tounge is a web expert), there's little hope for a standard of quality on the web.
--- Speaking only for myself,
However, some specific things I can think of are:
Seperate out HTML/text and code as much as possible. Ideally, there should be no, or almost no, HTML in the code. This makes life easier for the HTML designers, and also means you get bugged less to alter the code to alter the output.
Make it as simple/easy/reliable/quick/painless as possible to stop/start any server processes. I've worked on a project where we had a perl process, but stopping/restarting had some unpleasant side-effects, meaning we had to restrict doing such an operation to the quietest hours.
Make sure you have a good and accurate test environment. Having some tools to help you debug the running site itself is very usefull - retesting doing a whole load of operations by hand is very labourious.
Test and code for both availability and scaleability. You want good resiliant code with minimum service downtime (which means using reliable OSs and development enviroments - don't just choose the latest).
Make sure you can say "no" to the management. You *will* get PHBs trying to force new features on you when the current ones aren't even working right yet. Make sure your code is flexible though as it'll be changing/evolving a lot anyway. Also make sure you don't get stuck doing fire-fighting - ie stamping out so many little bugs that you can't fix the fundamental problems. (this is the kinda problem M$ has with it's code development...)
Personally, I wouldn't say that C, C++, Perl and PHP promote good programming practices. Java's certainly much better here. I'm not sure about other development environments. (don't worry about server side Java's speed as on a good server, it'll be close to C/C++. I'm not kidding.)
Most scripting languages like Perl or Python need to be compiled into bytecode, and then executed. The overhead of compiling the program every time it is executed can be quite a burden, especially for applications like CGI programs where the program is started very frequently but runs only for a short time.
/usr/bin/perl for each hit. Again, this becomes significant once you start getting lots of hits.
Even if you can cache the compiled bytecode (as Python does), there is still the overhead of starting
The best thing you can do is use something like Apache's mod_perl which avoids the overhead of starting a new perl process for each page, and also the overhead of compiling the program each time. Similar features exist for all popular scripting languages and Web servers.
Once you've done that, you may find that there are other startup costs you can factor out. For example, does each hit make a new connection to the database? Depending on your application you may be able to get away with some sort of local cache, either on disk or cached in memory in some way.
Finally, you can make your site faster to _use_ and save the user's time if you make your pages appear gradually. If you're churning out a long, complex page, try to make it appear as it is computed, so the user can start reading immediately. This is partly a matter of choosing layout that web browsers can render incrementally, and partly of how you write your program. (Slashdot's comment display, at least in Minimalist Mode, is very bad at this. In Netscape 4.x you have to wait for the whole 100Kbyte or so before seeing any of it.)
-- Ed Avis ed@membled.com
Yes, it is more expensive. But, many companies are racing to be the first for survival, and they have been blessed with enough VC money, so that it is irrelevant that it takes hundreds of thousands of dollars to pick up the performance. Time to market is much more important, and having programmers spend time performance tuning takes away from that.
One clear example here is Java vs. C++. Sure, Java is much slower than C++; but developing something in Java takes orders of magnitude less time than in C++. So, it is better to throw money at hardware to cure the performance problem, but will get you to market sooner, than code in C++, which results in faster code, but takes much longer.
Of course it's not quite this simple. You can't just throw hardware and money at every problem. Obviously your application can't be so slow that no one will use it. And key performance problems are first at the high architecture level, which are mostly language and platform independent. You will always have to spend time performance tuning at some point. But initially---even long term---time to market is more important.
----------
In a real emergency, we would have all fled in terror, and you would not have been notified.
I went through so many hairstyles, fashions, music trends, the occasional piercing and tattoo that my family started referring to me as Rodman a number of years ago. However, I've been writing code for 16 years now. And I'm good. I'm good enough that now I also get to see the streams of applicants coming in and get to decide whom to hire and how to put together the teams.
Get this straight:
If I get a group of "green-hair"s in, I'll hire the ones that can code, work together, and who won't steal from the company.
If I get the GH's and some poindexter's, I'll hir the ones that can code, work together, and who won't steal from the company.
If I get a group of poindexter's, I'll hire the ones that can code, work together, and who won't steal from the company.
Part of the problem in hiring is knowing whether or not the prospective (green haired or not) can code, work with others, and won't steal from the company. Your credentials, I'm afraid to say, are what get you in the door. [ If you know someone who can recommend you, that can get you in the door as well, but I really consider that a social credential more than a side-stepping of the system. ] If you've never done anything productive, you could very well be the best programmer in the world, but you've never worked on anything. This means (to me) that you probably don't know how to work with people, or how to pursue joint code development, or what deadlines are, or why tasks are prioritized, etc.
A degree, certification, or award, is an indication that you have completed some structured program designed to increase or test your assimilation of a body of knowledge. Are there people with the proper certifications who aren't qualified to work in this industry? Certainly. Are there people with the proper certifications who are able to work? Certainly. Are there people without those certifications who can do the work? Certainly.
So, certifications are worthless then. Bullshit. Like I said before, they are an indication of completion of a structured program. Given two applicants for one position whom I judge to be of equal capability, willing to work for the same pay, I'm generally going to hire the one with more certification -- primarily because I know that person is more likely to be able to work under deadlines and prioritized sets of goals.
Similarly, given a set of qualified applicants (can do the work, can work together, won't rob us blind, have the same relative qualifications) and too few positions, I will often favor the applicants who put the most effort into applying: well written resume/cover letter, well-dressed, punctual, polite, aggressive (no there's no contradiction there), excited, motivated, etc. These people are more likely to work hard, stay with the company, act on their own initiative, and so on.
The perceived tightness of the job market (and, actually, it's not quite as tight as has been made out) has brought to the average technophile an attitude that not only are they in high demand, but also that their intrinsic coolness, which derives from often dubious technical ability, is a substitute for proof of ability. Only one component of employability is technical competence, the other more important ones include social maturity, dependability, commitment, responsibility, ambition, and respect. Some will certainly whine that technical ability is the most important, but in this day (contrary to what many of you seem to believe) that's simple to find. If that's all you have, you lose out, because the next person in the door has the others too.
"Cause there's 40 different shades of black, so many fortresses and ways to attack, so why you complainin'?"
Check out Philip and Alex's guide to web publishing. This the book you were looking for?
o/~ Join us now and share the software
Some techniques that are essential to web development are:
Often, the speed of the web is, in reality, how fast the pages appear to be. This is the most important thing to remember when designing pages that are supposed to be fast.
darren
Cthulhu for President!
(darren)
I don't know of any books, but here's what I do...
I work for a fairly large software company and we use a Linux/Apache/MySql system to dynamically track and record our production. This database/machine is getting hammered on the backend (data importing) and on the front end (our web/perl interface).
How do we tweak this datatabase to run 5 different programms and still output 1.5 million transactions a day on a 200Mhz machine with 32Mb of ram?
Efficient code!
Nothing different than you would do with your C code, just use smart programming. With perl, use the 'strict' module. Always destroy database handles as soon as you can, don't try to do complex sorts in your code, that's something the database is capable of.
Static if you can...If there is a page on your site that doesn't change very often (ie. it chagnes less than half the amount of times that it is accessed per day) generate it with a server daemon, and then let users access it has a regular html file. This saves CPU time and decreases latency of accesses.
A lot of the web programming starts with good database design. There are countless books about this. If your databse is optimized so that there are no full table scans and relationships are tightly bound, then writing efficient code is fairly simple.
Overall, just keep it simple and efficient.
I am a professional PHP developer and I personally put quite a bit of thought into the code as I write it. I think a very good set of articles which help with the problem you mentioned, at least from one angle, are the Cached Dynamic Module articles at php builder.
Don't moderate this as a Troll or a Flame, I write in Perl most of the time myself. But I've recoded some stuff and tested it, and blew the doors off.