One of the most important aspects of any optimization process is profiling. Unfortunately profiling is also one of the hardest tasks to accomplish when it comes to web development. If you can't identify where your bottlenecks are or where the most time is spent, you can't optimize very well.
The fact that it's fairly common these days to find that a web request lands on a load balancer which forwards it to a webserver which may dynamically generate a page which makes a call to an application server which may execute a stored procedure by going through some sort of middleware database connector to a RDBMS on the backend means that there are a lot of different things that have the potential to be optimized.
I think some of the most important advice to keep in mind is that generally the optimization technique which yields the best results are architectural redesigns. If your architecture is inherently slow or imposes inherent limits on your performance, it doesn't matter how much you tweak the individual components.
Once you have an optimal architecture, each component in the architecture needs to be examined discretely. For example, there are a lot of optimizations that can be done to perl code that don't apply to Java at all, and vice-versa. Once you get to this stage, read every article, buy every book, etc. that you can regarding your specific technologies.
Our company is developing search technology using Java and by following the java-specific advice in the 'Java 2 Performance and Idiom Guide' we were able to reduce the time it took to do an average search from 10-30 seconds down to 100 to 300 milliseconds.
Of course it's up to the Slashdot staff to decide what they consider off topic, but under the moniker "News for Nerds / Stuff that Matters" I would say that there are any number of valid arguments for why this is not off-topic.
Probably the most important one in my personal opinion is the fact that a vast majority of geeks and slashdot readers probably work in the tech sector and a vast majority of those probably took a hefty option plan as an alternative to a higher salary. Probably more so than the average American, the geeks reading slashdot have probably been clutching their wallets and doubling over in pain recently.
Aside from that, another obvious connection is the fact that tech stocks are all the buzz in the market these days, and many people are blaming Microsoft or at least the Microsoft ruling for the dive. Especially when you consider that all the Linux and other non-Microsofty organizations have taken big hits too, I think there is plenty for geeks to have opinions about on this topic.
Several people have mentioned, in this discussion and others, that it would be great to have a touch screen for it. I think some people have already figured out how to do this. This would be great, but really not that useful unless the device had a battery (rechargable of course) that let you unplug it and walk around with it. Of course then it would be nice to have a case to close over the screen to protect it while you were in mobile mode.
Another suggestion would be to turn a full 360 from the current position and offer cash & prices for the most innovative uses of IOPeners... then the company will have a team of eager hackers out their doing work for them (hooking up batteries and touch screen overlays) and creating altnerate business models and revenue streams for them (for free).
Is it actually a fact that Netpliance sells the machines at a loss? Some co-workers and I tried to estimate the cost of manufacture of one of these things (I have one sitting on my desk at work) and the number we came up with was about $85 or less. Now obviously this wouldn't be a great margin, but it's not a loss either. It would be nice to see some actual per-component cost numbers as well as manufacturing costs.
Re:Telnet is the only solution.
on
SSH v. SRP
·
· Score: 1
Not to mention that these days, unless you have a massive investment in security, if a hacker gets the login/password of a non-root account, that is pretty much equivalent of having root on just about any version of UNIX. In my experience it is very hard to get into a well-protected machine from the outside, but once you're on, there's always a way to get root that can be found in 5 or 10 minutes of poking around. If your machine is pretty tightly locked down from the outside, and you always use SSH or some similar sniffer-proof method of logging in, you're in pretty good shape. If on the other hand, you use telnet or some other insecure form of logging in, you might as well not have passwords on any accounts including roots. It's probably also worth mentioning at this point that having your host on a switched network make some people think they are impervious to sniffing, but if you are going over a WAN to the switched media, it makes no difference, and if your switch has a monitor port and the switch is not well protected you're just as vulernable. Might as well write your passwords on yellow stickies and put them on the top of your monitor.
It's a matter of choosing stability over performance. If your application is not mission critical, by all means, use RC1 with the JIT. If it is, using RC1 instead of the actual release would be ill advised at best. For example, try creating more than 600,000 objects at one time in RC4.
I assume you are kidding when you say that you have to look things up in the Java API if you want to do anything useful. How is this any different than any other programming language? You have to use man pages or API specs for anything in C if you do not already know how say the socket libraries work or whatever. If you can memorize it for C, you can memorize it for Java, and probably more easily since Java is better planned, more well thought out and intuitive. Not only that but the use of javadoc on all of the actual java class libraries means there is pretty damn good documentation for the whole language all organized conviently and easily in one place, and a lot easier to use than man pages. Also you compare it to C/C++/Perl/etc. Hasn't the software engineering profession been around long enough that people should realize that different tools are better for different jobs? Don't use a hammer to dig a hole when a shovel is available and all that.
Why not use a loop device (/dev/loop0) to immitate a CDROM so you don't need to burn CDs and waste time/blank CDs? Better yet, why not just write your own Beam-it like client that takes a CDDB hash and turns it into a Beam-it hash. I bet they might even be the same thing. Better yet, why not just buy music and support the artists you like?
gcc, g++, Linux. The difference between C and C++ is whether it's in a class or not. If you compile the C program with g++ you naturally get the same results.
First is a reply to various people's assertation that whether or not JIT can speed things up, you pay an initial cost for it:
This is a technically true statement, however it is misleading. There is obviously a cost for compiling code at runtime, however the question that should be asked is does this cost outway the benefit of the optimization that it provides?
If you start with the assumption that the JIT can provide code, that _once_it_is_compiled_ runs faster than that which an average human coder writes, then it is simple to realize that in some situations, the JIT compilation will kill you, and in others it will
The first case would be if you were to write a program like the unix command 'date' in Java. The time it takes the compiler to compile this code is always (I'm guessing) going to be slower than it is to just yank the date out of the hardware and spew it to the screen. Plus after the Java program compiles, it still has to make calls to native code to yank the date out of the hardware and spew it to the screen, so it's doomed to be slower than just about anything but the worst handcoded assembler or even C.
However, the second case is that you are writing a major application; perhaps a web application. The application starts up, compiles all the classes, you lose maybe 2 or 3 seconds to this process if you're running a huge amount of code on a crappy processor. However, the application runs for 200 days, constantly handling mortgage applications, or doing text indexing, or selling t-shirts or whatever. During this time, the code that took 2-3 seconds to optimize is executed millions and millions of times. It's obvious that you would benefit greatly from a JIT compiler in this case (assuming you accept the base assumption that the JIT can created better code).
So, IMHO, there's not much more to say about whether the JIT causes more of a performance hit than it provides so I think everyone can safely go back to discussing whether or not it can really outcode a human. I'll save my two cents on that for another day.
In the meantime the second thing I'd like to add is a set of numbers I've come up with while working on a paper about whether or not Java is a viable platform for systems with high performance requirements. I plan to release the paper soon and when I do I'll try to post about it here, or you can contact me by e-mail if you're specifically interested. In the meantime here are some example numbers I've seen trying to compare Java, C++ and C performance.
Test 1 - Integer Divisions Average time in C, w/ -O3: 00:00.76 Average time in 1.2 JVM with JIT: 00.02.30 Average time in C: 00:08.18 Average time in 1.1 JVM with no JIT: 00:50.97
Test 1 - Integer Divisions on a different (slower) machine w/ C++: Average time in C: 00:01.2x Average time in C++ (no optimizations): 00:15.3x Average time in C++ (w/ optimizations): 00:15.3x Average time in Kaffe 1.1 VM w/JIT: 00:18.7x Average time in Sun Java 2 JVM w/ JIT: 00:18.7x
Test 2 - Integer Divisions in Multiple Threads Average time in C w/ -O3: 00:07.52 Average time in 1.2 JVM with JIT: 00.18.74 Average time in C: 01:13.04 Average time in 1.1 JVM w/ no JIT: 08:30.00
I'll leave it to you to deduce what you like from these numbers, but the first thing that surprised me was that the optimized Java code consistantly stomps all over the unoptimized C code. After thinking about it for a while this makes sense, but how often do people remember to throw in the -O3? Sure, on commercial software that's going to be burned on CD and shipped to millions of people, they probably remember... but in a Makefile in an Open Source project? Or even in smaller commercial shops? How many average C programmers are even aware that it can make a 10x difference in speed? If you are using a JVM with JIT compiling, you will always be running optimized. Another interesting thing is that I do not have a machine I can run Sun's HotSpot compiler on to compare it, which I will be doing prior to finishing the paper. Also at the rate things are accelerating it seems obvious to me that Java performance will catch and probably surpass C in relatively short order. It's just about already overtaken C++, which people said was 10 years away just a year ago.
Slashdot Mirror
One of the most important aspects of any optimization process is profiling. Unfortunately profiling is also one of the hardest tasks to accomplish when it comes to web development. If you can't identify where your bottlenecks are or where the most time is spent, you can't optimize very well.
The fact that it's fairly common these days to find that a web request lands on a load balancer which forwards it to a webserver which may dynamically generate a page which makes a call to an application server which may execute a stored procedure by going through some sort of middleware database connector to a RDBMS on the backend means that there are a lot of different things that have the potential to be optimized.
I think some of the most important advice to keep in mind is that generally the optimization technique which yields the best results are architectural redesigns. If your architecture is inherently slow or imposes inherent limits on your performance, it doesn't matter how much you tweak the individual components.
Once you have an optimal architecture, each component in the architecture needs to be examined discretely. For example, there are a lot of optimizations that can be done to perl code that don't apply to Java at all, and vice-versa. Once you get to this stage, read every article, buy every book, etc. that you can regarding your specific technologies.
Our company is developing search technology using Java and by following the java-specific advice in the 'Java 2 Performance and Idiom Guide' we were able to reduce the time it took to do an average search from 10-30 seconds down to 100 to 300 milliseconds.
Of course it should be noted that some degreed professionals also have green hair and/or studs through their tongues.
Of course it's up to the Slashdot staff to decide what they consider off topic, but under the moniker "News for Nerds / Stuff that Matters" I would say that there are any number of valid arguments for why this is not off-topic.
Probably the most important one in my personal opinion is the fact that a vast majority of geeks and slashdot readers probably work in the tech sector and a vast majority of those probably took a hefty option plan as an alternative to a higher salary. Probably more so than the average American, the geeks reading slashdot have probably been clutching their wallets and doubling over in pain recently.
Aside from that, another obvious connection is the fact that tech stocks are all the buzz in the market these days, and many people are blaming Microsoft or at least the Microsoft ruling for the dive. Especially when you consider that all the Linux and other non-Microsofty organizations have taken big hits too, I think there is plenty for geeks to have opinions about on this topic.
Several people have mentioned, in this discussion and others, that it would be great to have a touch screen for it. I think some people have already figured out how to do this. This would be great, but really not that useful unless the device had a battery (rechargable of course) that let you unplug it and walk around with it. Of course then it would be nice to have a case to close over the screen to protect it while you were in mobile mode.
Another suggestion would be to turn a full 360 from the current position and offer cash & prices for the most innovative uses of IOPeners... then the company will have a team of eager hackers out their doing work for them (hooking up batteries and touch screen overlays) and creating altnerate business models and revenue streams for them (for free).
Actually the "HOME" key on the keyboard (the one on the top right) is the Escape key. You don't need to swap out the keyboard to hack it.
Is it actually a fact that Netpliance sells the machines at a loss? Some co-workers and I tried to estimate the cost of manufacture of one of these things (I have one sitting on my desk at work) and the number we came up with was about $85 or less. Now obviously this wouldn't be a great margin, but it's not a loss either. It would be nice to see some actual per-component cost numbers as well as manufacturing costs.
Not to mention that these days, unless you have a massive investment in security, if a hacker gets the login/password of a non-root account, that is pretty much equivalent of having root on just about any version of UNIX. In my experience it is very hard to get into a well-protected machine from the outside, but once you're on, there's always a way to get root that can be found in 5 or 10 minutes of poking around. If your machine is pretty tightly locked down from the outside, and you always use SSH or some similar sniffer-proof method of logging in, you're in pretty good shape. If on the other hand, you use telnet or some other insecure form of logging in, you might as well not have passwords on any accounts including roots. It's probably also worth mentioning at this point that having your host on a switched network make some people think they are impervious to sniffing, but if you are going over a WAN to the switched media, it makes no difference, and if your switch has a monitor port and the switch is not well protected you're just as vulernable. Might as well write your passwords on yellow stickies and put them on the top of your monitor.
It's a matter of choosing stability over performance. If your application is not mission critical, by all means, use RC1 with the JIT. If it is, using RC1 instead of the actual release would be ill advised at best. For example, try creating more than 600,000 objects at one time in RC4.
I assume you are kidding when you say that you have to look things up in the Java API if you want to do anything useful. How is this any different than any other programming language? You have to use man pages or API specs for anything in C if you do not already know how say the socket libraries work or whatever. If you can memorize it for C, you can memorize it for Java, and probably more easily since Java is better planned, more well thought out and intuitive. Not only that but the use of javadoc on all of the actual java class libraries means there is pretty damn good documentation for the whole language all organized conviently and easily in one place, and a lot easier to use than man pages. Also you compare it to C/C++/Perl/etc. Hasn't the software engineering profession been around long enough that people should realize that different tools are better for different jobs? Don't use a hammer to dig a hole when a shovel is available and all that.
Why not use a loop device (/dev/loop0) to immitate a CDROM so you don't need to burn CDs and waste time/blank CDs? Better yet, why not just write your own Beam-it like client that takes a CDDB hash and turns it into a Beam-it hash. I bet they might even be the same thing. Better yet, why not just buy music and support the artists you like?
gcc, g++, Linux. The difference between C and C++ is whether it's in a class or not. If you compile the C program with g++ you naturally get the same results.
I have two items to contribute.
First is a reply to various people's assertation that whether or not JIT can speed things up, you pay an initial cost for it:
This is a technically true statement, however it is misleading. There is obviously a cost for compiling code at runtime, however the question that should be asked is does this cost outway the benefit of the optimization that it provides?
If you start with the assumption that the JIT can provide code, that _once_it_is_compiled_ runs faster than that which an average human coder writes, then it is simple to realize that in some situations, the JIT compilation will kill you, and in others it will
The first case would be if you were to write a program like the unix command 'date' in Java. The time it takes the compiler to compile this code is always (I'm guessing) going to be slower than it is to just yank the date out of the hardware and spew it to the screen. Plus after the Java program compiles, it still has to make calls to native code to yank the date out of the hardware and spew it to the screen, so it's doomed to be slower than just about anything but the worst handcoded assembler or even C.
However, the second case is that you are writing a major application; perhaps a web application. The application starts up, compiles all the classes, you lose maybe 2 or 3 seconds to this process if you're running a huge amount of code on a crappy processor. However, the application runs for 200 days, constantly handling mortgage applications, or doing text indexing, or selling t-shirts or whatever. During this time, the code that took 2-3 seconds to optimize is executed millions and millions of times. It's obvious that you would benefit greatly from a JIT compiler in this case (assuming you accept the base assumption that the JIT can created better code).
So, IMHO, there's not much more to say about whether the JIT causes more of a performance hit than it provides so I think everyone can safely go back to discussing whether or not it can really outcode a human. I'll save my two cents on that for another day.
In the meantime the second thing I'd like to add is a set of numbers I've come up with while working on a paper about whether or not Java is a viable platform for systems with high performance requirements. I plan to release the paper soon and when I do I'll try to post about it here, or you can contact me by e-mail if you're specifically interested. In the meantime here are some example numbers I've seen trying to compare Java, C++ and C performance.
Test 1 - Integer Divisions
Average time in C, w/ -O3: 00:00.76
Average time in 1.2 JVM with JIT: 00.02.30
Average time in C: 00:08.18
Average time in 1.1 JVM with no JIT: 00:50.97
Test 1 - Integer Divisions on a different (slower) machine w/ C++:
Average time in C: 00:01.2x
Average time in C++ (no optimizations): 00:15.3x
Average time in C++ (w/ optimizations): 00:15.3x
Average time in Kaffe 1.1 VM w/JIT: 00:18.7x
Average time in Sun Java 2 JVM w/ JIT: 00:18.7x
Test 2 - Integer Divisions in Multiple Threads
Average time in C w/ -O3: 00:07.52
Average time in 1.2 JVM with JIT: 00.18.74
Average time in C: 01:13.04
Average time in 1.1 JVM w/ no JIT: 08:30.00
I'll leave it to you to deduce what you like from these numbers, but the first thing that surprised me was that the optimized Java code consistantly stomps all over the unoptimized C code. After thinking about it for a while this makes sense, but how often do people remember to throw in the -O3? Sure, on commercial software that's going to be burned on CD and shipped to millions of people, they probably remember... but in a Makefile in an Open Source project? Or even in smaller commercial shops? How many average C programmers are even aware that it can make a 10x difference in speed? If you are using a JVM with JIT compiling, you will always be running optimized. Another interesting thing is that I do not have a machine I can run Sun's HotSpot compiler on to compare it, which I will be doing prior to finishing the paper. Also at the rate things are accelerating it seems obvious to me that Java performance will catch and probably surpass C in relatively short order. It's just about already overtaken C++, which people said was 10 years away just a year ago.