It's great that Linux's evolution towards user friendliness includes tools outside the base of casual users.
A better manual would have come in handy when I was trying to use the filter option to isolate packets.
Who knows, maybe support for SSL will come next? I'd love to be able to snoop and decipher ssl data on the fly (If I had access to the private key of the enciphered stream).
While I tend to disagree and believe in porting rather than writing, I have to admit that you've made a very compelling argument and commend you on your eloquence -- very well stated.
Solaris is designed for the enterprise machine with higher end machines with heavy load. Solaris really begins to shine when you have lots of people pounding on it.
Looks like their site has already felt the Slashdot effect.
I've been surprised that we haven't seen more site spoofing, but that's one of the issues SSL is supposed to take care of.
I wonder if they could get a PayPai certificate from verisign or if Verisign would recognize this blatant attempt at fraud and refuse to issue them a certificate?
The one thing I've kind of worried about is what would happen if someone hacked into one of the DNS repositories like register.com and doled out their address in lieu of PayPal.com.
Again, SSL would come to the rescue, but only if you pay attention to the security warnings as they come across as they would have to forge a certificate and it wouldn't be from one of the certificate authorities accepted by defaults in most browsers.
There are plenty of ways to block unwanted sites from accessing any given site ranging from the heavy handed firewall access list to software solutions such as robot exclusion and browser inspection.
If this becomes a larger issue later on then I feel that the issues are best addressed with technology rather than legislation.
I hate seeing matters such as this brought to the court system. A system filled with people making decisions about technology they may or may not fully understand, after all, they're judges and lawyers not Linux kernel hackers and systems administrators.
I feel this way in spite of the fact that Yodlee is a personal threat to my site. I work for a company providing account information on the internet. We have worked hard and spent millions of dollars to provide this service and secure our site from hackers only to find that our users are willingly giving their login credentials to Yodlee.
Yodlee has become a repository housing many of our users login ids and passwords. Is it secure? Hell if I know. They have some legalistic tripe at signup intended to shield them from responsibility. If a user's account gets compromised though, you can bet that they'll come after us rather than Yodlee. I never even knew this was going on until I was investigating some strange accesses to our site which suggested an ill behaved robot.
I guess my point is that while I understand the temptation for litigation in these issues, however, technology is much better equipped to handle these problems.
The value of a brand name is measured by the deceptively simple formula:
Vpc = ns * csp / nc
Where Vpc is the value per character, ns is the number of stocks issued, csp is the current stock price and nc is the number of characters in the name of the brand.
For coke,
Vpc = 8,730,000 * 42 / 8 = 45,832,500
(Note that I'm using the brand name Coca-Cola in this calculation, and am not counting the hyphen)
For Microsoft,
Vpc = 5,600,000,000 * 78 / 9 = 48,533,333,333.
Contrary to the article, the Microsoft brand name is already much more valuable. I can only assume that the author must have been using the the Coke/Microsoft Corporation comparison.
All joking aside, they probably use market research in conjunction with statistical methods such as confidence intervals to form one huge SWAG. I wouldn't take it too seriously.
These Militant Linux Geeks are a major issue today. While helpful to the development of Linux because they are often programmers who like to go in and hack around in the code, they are also potentially a threat to the OS. These are not people who use Linux because it is better; these are people who use Linux because they like the elitism that Linux gives them. They prefer to understand things that others don't.
There are a few of these around. I suspect that a large percentage of them are former Amiga owners (Which holds a special place in my heart as a truly magnificent machine). I have always been amused to see these folk arguing "My OS is harder to use than your OS so I'm superior so nyaaah".
I've also never quite understood the BSD/Linux and Linux distribution controversies. It really doesn't matter to me which distribution of Linux or whether BSD ends up taking over the largest marketshare as long as some Linux derivative is out there competing effectively with Microsoft.
To me, it's all unix. I tend to select my distributions based on who has the biggest marketshare not because I favor one over the other, but because I want to support a move for consolidation among the distributions and because I haven't seen enough difference between the distributions to really care (I've used RedHat, Mandrake and Slackware). I use my system to do real work and from my viewpoint each distribution enables me to do it remarkably well.
On the other hand, I have a windows partition because
I sometimes like to play games which aren't available to linux
My wife likes to surf the net and hasn't quite got the hang of the linux command line.
Linux doesn't have visio *yet*.
If my wife screws up the computer, at least the damage is contained within the windows partition which doesn't really contain anything valuable anyhow (except perhaps my previous high scores:-))
The Linux mantra nowdays consists of "World Domination", but the Linux militants seem to want to keep it within the fold of the hardcore unix geeks. Sorry guys, you can't have your cake and eat it too. World domination comes hand in hand with user friendliness.
Really this is only half a story without a copy of their contract on hand.
Having said that, from my experience, disatisfaction with sites done by consulting companies is usually the result of one of the following:
Poor specifications and too little direction from the project manager within the employing company. Which result in too many decisions being made on the part of the consulting company.
Design by committee. Too much direction from too many members of the employing company, each trying to enact small, often insignificant changes to the site, driving up the effort level of developing the site.
Poor costing on the part of the consulting company on a fixed rate project. The consulting company pulls out once it reaches a certain cost threshold *oops, we're not making as much money as we anticipated*.
The only real question is did they meet the terms of their contract? If not, they should be given the opportunity to do so, or have their fees pro-rated.
The site they put together looked pretty good to me, albeit, I had to turn javascript on in order to use it.
GAs can be used to gain intuition towards better, deterministic solutions.
Undoubtably, there are other techniques that produce more deterministic results (at least with respect to ascending closest hill), that's why people combine GA with other search mechanisms. They all have good and bad points. Proper combination of disparate search algorithms can lead to better solutions in that the algorithms complement each other, making up for the weaknesses inherent in their counterpart.
I think the fact that GAs mirror nature does give GA more press coverage. People are fascinated with nature, and modeling nature's basic operations inside of a computer is more interesting (and readable) than a treatise on alpha beta pruning.
I was wondering the same thing. I've seldom been able to achieve anything of value with populations less than 50, though that could be due to flaws in my implementation.
Perhaps the evaluation of an individual's fitness is expensive, perhaps taking hours or days of computational time on a supercomputer to achieve. The article seemed to hint at this.
Larger populations mean greater possibility of mutation, and less probability of plateauing at a local extrema.
Perhaps he plans to run this many times, finding different extrema each time... Great question though.
The wonderful thing about GAs is the fact that they can be used to locate interdependent relations.
If the cost and time isn't prohibitive, I'd input as much into the search as you can. Let the GA tell you what's important. Mathematicians sometimes use GA to attain notions of relations between things. The GA doesn't give them the big solution, so much as little solutions which hint at a bigger one.
GAs are, however, only as good as the fitness function used to evaluate the solutions. If your fuel efficiency evaluation is flawed, then your results will be as well. The point being that if your evaluation function doesn't take the ENTIRE car into account in it's evaluation, then the extra GA input will be superfluous.
Surprisingly, I have heard little on the Beowulf front concerning Genetic algorithms. If you think about it, Genetic algorithms and Genetic programming are perfect candidates for some massivly parallel programming which is made a heck of alot more affordable through use of Beowulf clusters.
The neat thing about Genetic programming is that you could treat each Beowulf node as it's own little Galapagos island. Seed each node with a genetic population (which could vary in size depending on the capability of the node), then every so often, take the fittest of the various populations and use a random wheel to reseed each of the beowulf nodes (or islands if you like to think metaphorically).
You could even vary the fitness models of each node, creating different environments on each node/island. The wonderful thing about Genetic programming is the infinite possibilities.
Repeat this process until something interesting evolves.
That sounds pretty interesting. I have a similar story. I've dabbled in GA/GP as well. I was working on a statistical sorting algorithm which sorted numbers based on where a number within a set "probably" fell, in order to create smaller partitioned subsorts. As a result, I found that I needed an equation for the CDF (Cumulative Density Function).
If you've ever taken statistics, that's the long table in the end of the book. I never understood why they didn't supply an equation instead of a big long table until I spent 3 days trying to find the equation with analytical calculus. It turns out that it's impossible to solve via calculus.
I found a crude approximation, but it was too long and involved too much calculation to be useful in my sorting algorithm. Enter genetic algorithms.
I coded up a GP symbolic regression and plugged in all the x and f(x)'s in the table in my statistics book for evaluation in a least squares fit fitness model.
I ran it over and over until it spit out something that deviated by.001 over all the supplied points!
It was a huge equation, however once I got through simplifying all of the fancy 0's and fancy 1's, it transformed into the startlingly simplistic equation of e^x/(e^x + 1) if memory serves me.
I continued to run the GP, and got other equations which deviated even less, however they were more cumbersome and I was looking for something quick for my sorting algorithm.
Interestingly enough, nature has a slight push towards simplicity in that smaller DNA strands replicate quicker and may more quickly exhaust the resources available in competition with other DNA strands. This was demonstrated in an experiment which produced what is known as the "Spiegelman monster", in which a scientist put a small DNA strand in a test tube full of nucleotides, effectively removing competition from the environment. When the scientist later analyzed the contents of the test tube, he found that the tube was dominated by the smallest strand capable of self replication and this strand was dubbed "Spiegelman's monster".
I've tried to incorporate this slight but present push towards simplicity into my fitness functions in GA/GP programming by giving ever so slightly higher ratings to simpler solutions. This also helps the computer out as it may calculate the results on simple solutions more quickly than complex ones.
Anyway, kudos on your inventive application of Genetic algorithms. It struck me as being similar to an genetic search for a fractal compression.
I would guess that your evaluation algorithm involved dictionary lookups and the gibberish to english ratio, but would love to hear further detail.
That's exactly what I was going to point out. You can't avoid being billed, one way or another. Figuring out new ways around the system, like ip phones, will undoubtably cause the phone companies and isps to figure out new ways to bill you for the service. You may, however, save money while they react.
This kind of reminds me of the 80's when the phone companies were trying to lobby to charge modem users higher fees.
In the meantime, people are using a technology (voice over ip) less suited for it's purpose than it's predecessor. This traffic will be driving up isp costs since more traffic will be driving up the bandwidth requirements.
I don't know much about the IP phones, but chances are they are bound to a specific port or there is some characteristic about the IP packet which will allow such traffic to be monitored, calculated, and billed for.
In large scale software development projects, it has been my experience that typically, noone has full "mastery" of the code. You have individuals who have a full mastery of aspects of the code, and if you are lucky, the group as a whole will have mastery of the code base. Unlike you, the statement that they were working on mastery of the code actually gives me confidence in the code base. It tells me 2 things. 1) I am dealing with honest folk. A known problem is much easier to deal with than one which gets shoved under the rug by a team of marketing analysts worrying about how the admissions will affect their stock prices. Too often, vendors work hard to deny obvious defects in their applications. Honesty is refreshing. 2) Their gaining confidence as a team, and more ambitious releases will ensue. I just hope that their developing code "mastery" that is redundant across individuals as they will undoubtably suffer attrition as time goes on. - Pat
Thanks for all the information. Like you, I don't place too much stock in benchmarks, especially ones existing prior to product implementations. Volano specs also demonstrated a performance difference, though it should be noted that Volano server produced a thread dump when running versus IBM JDK 1.3. The closest test I could run to a pure 1.2.2 w/inprise jit versus IBM 1.3 was to have the server run 1.2.2 w/ inprise with varied clients. IBM still produced superior numbers though not as pronounced as the previous test. It was about 25% faster (sorry I don't have the actual data handy).
At my work, we're using java (quite successfully) in a system which generates over a million legacy system accesses a day, formatting the results through use JSP as well as DCL (a small perl-like language written in java). The system does millions of dollars of business a day in a mission critical atmosphere. Java delivers great performance for us and minimizes development time.
I downloaded and retested with Inprise's javacomp JIT and my marks effectively doubled from 444 to 823 using CaffeineMark 3.0 as my benchmark. Looks like my initial test didn't incorporate a JIT, but still, the IBM benchmark results were far superior, especially on the logic section of the test which scored 495,997, more than an order of magnitude larger than any of the other tests in the CaffeineMark benchmark. As far as I can tell, my benchmark was legit.
I just ran a CaffeineMark on my PC versus the jdk1.2.2 supplied by SUN versus the new jdk1.3 supplied by IBM. SUN's caffeinemark was 444, while the IBM JVM scored a whopping 6,309. This is an amazing performance difference.
Slashdot Mirror
A better manual would have come in handy when I was trying to use the filter option to isolate packets.
Who knows, maybe support for SSL will come next? I'd love to be able to snoop and decipher ssl data on the fly (If I had access to the private key of the enciphered stream).
While I tend to disagree and believe in porting rather than writing, I have to admit that you've made a very compelling argument and commend you on your eloquence -- very well stated.
Looks like their site has already felt the Slashdot effect.
I wonder if they could get a PayPai certificate from verisign or if Verisign would recognize this blatant attempt at fraud and refuse to issue them a certificate?
The one thing I've kind of worried about is what would happen if someone hacked into one of the DNS repositories like register.com and doled out their address in lieu of PayPal.com.
Again, SSL would come to the rescue, but only if you pay attention to the security warnings as they come across as they would have to forge a certificate and it wouldn't be from one of the certificate authorities accepted by defaults in most browsers.
Has anyone heard of stuff like happening?
If this becomes a larger issue later on then I feel that the issues are best addressed with technology rather than legislation.
I hate seeing matters such as this brought to the court system. A system filled with people making decisions about technology they may or may not fully understand, after all, they're judges and lawyers not Linux kernel hackers and systems administrators.
I feel this way in spite of the fact that Yodlee is a personal threat to my site. I work for a company providing account information on the internet. We have worked hard and spent millions of dollars to provide this service and secure our site from hackers only to find that our users are willingly giving their login credentials to Yodlee.
Yodlee has become a repository housing many of our users login ids and passwords. Is it secure? Hell if I know. They have some legalistic tripe at signup intended to shield them from responsibility. If a user's account gets compromised though, you can bet that they'll come after us rather than Yodlee. I never even knew this was going on until I was investigating some strange accesses to our site which suggested an ill behaved robot.
I guess my point is that while I understand the temptation for litigation in these issues, however, technology is much better equipped to handle these problems.
Vpc = ns * csp / nc
Where Vpc is the value per character, ns is the number of stocks issued, csp is the current stock price and nc is the number of characters in the name of the brand.
For coke,
Vpc = 8,730,000 * 42 / 8 = 45,832,500
(Note that I'm using the brand name Coca-Cola in this calculation, and am not counting the hyphen)
For Microsoft,
Vpc = 5,600,000,000 * 78 / 9 = 48,533,333,333.
Contrary to the article, the Microsoft brand name is already much more valuable. I can only assume that the author must have been using the the Coke/Microsoft Corporation comparison.
All joking aside, they probably use market research in conjunction with statistical methods such as confidence intervals to form one huge SWAG. I wouldn't take it too seriously.
There are a few of these around. I suspect that a large percentage of them are former Amiga owners (Which holds a special place in my heart as a truly magnificent machine). I have always been amused to see these folk arguing "My OS is harder to use than your OS so I'm superior so nyaaah".
I've also never quite understood the BSD/Linux and Linux distribution controversies. It really doesn't matter to me which distribution of Linux or whether BSD ends up taking over the largest marketshare as long as some Linux derivative is out there competing effectively with Microsoft.
To me, it's all unix. I tend to select my distributions based on who has the biggest marketshare not because I favor one over the other, but because I want to support a move for consolidation among the distributions and because I haven't seen enough difference between the distributions to really care (I've used RedHat, Mandrake and Slackware). I use my system to do real work and from my viewpoint each distribution enables me to do it remarkably well.
On the other hand, I have a windows partition because
The Linux mantra nowdays consists of "World Domination", but the Linux militants seem to want to keep it within the fold of the hardcore unix geeks. Sorry guys, you can't have your cake and eat it too. World domination comes hand in hand with user friendliness.
Having said that, from my experience, disatisfaction with sites done by consulting companies is usually the result of one of the following:
The only real question is did they meet the terms of their contract? If not, they should be given the opportunity to do so, or have their fees pro-rated.
The site they put together looked pretty good to me, albeit, I had to turn javascript on in order to use it.
Thanks for the link. I think I'll use that as a reference when I'm evangelizing Linux in the scientific community :-)
Undoubtably, there are other techniques that produce more deterministic results (at least with respect to ascending closest hill), that's why people combine GA with other search mechanisms. They all have good and bad points. Proper combination of disparate search algorithms can lead to better solutions in that the algorithms complement each other, making up for the weaknesses inherent in their counterpart.
I think the fact that GAs mirror nature does give GA more press coverage. People are fascinated with nature, and modeling nature's basic operations inside of a computer is more interesting (and readable) than a treatise on alpha beta pruning.
Perhaps the evaluation of an individual's fitness is expensive, perhaps taking hours or days of computational time on a supercomputer to achieve. The article seemed to hint at this.
Larger populations mean greater possibility of mutation, and less probability of plateauing at a local extrema.
Perhaps he plans to run this many times, finding different extrema each time... Great question though.
It's still an infinite search. It's not the number, but the nature of the parameters that matters most.
If the cost and time isn't prohibitive, I'd input as much into the search as you can. Let the GA tell you what's important. Mathematicians sometimes use GA to attain notions of relations between things. The GA doesn't give them the big solution, so much as little solutions which hint at a bigger one.
GAs are, however, only as good as the fitness function used to evaluate the solutions. If your fuel efficiency evaluation is flawed, then your results will be as well. The point being that if your evaluation function doesn't take the ENTIRE car into account in it's evaluation, then the extra GA input will be superfluous.
The neat thing about Genetic programming is that you could treat each Beowulf node as it's own little Galapagos island. Seed each node with a genetic population (which could vary in size depending on the capability of the node), then every so often, take the fittest of the various populations and use a random wheel to reseed each of the beowulf nodes (or islands if you like to think metaphorically).
You could even vary the fitness models of each node, creating different environments on each node/island. The wonderful thing about Genetic programming is the infinite possibilities.
Repeat this process until something interesting evolves.
If you've ever taken statistics, that's the long table in the end of the book. I never understood why they didn't supply an equation instead of a big long table until I spent 3 days trying to find the equation with analytical calculus. It turns out that it's impossible to solve via calculus.
I found a crude approximation, but it was too long and involved too much calculation to be useful in my sorting algorithm. Enter genetic algorithms.
I coded up a GP symbolic regression and plugged in all the x and f(x)'s in the table in my statistics book for evaluation in a least squares fit fitness model.
I ran it over and over until it spit out something that deviated by .001 over all the supplied points!
It was a huge equation, however once I got through simplifying all of the fancy 0's and fancy 1's, it transformed into the startlingly simplistic equation of e^x/(e^x + 1) if memory serves me.
I continued to run the GP, and got other equations which deviated even less, however they were more cumbersome and I was looking for something quick for my sorting algorithm.
Interestingly enough, nature has a slight push towards simplicity in that smaller DNA strands replicate quicker and may more quickly exhaust the resources available in competition with other DNA strands. This was demonstrated in an experiment which produced what is known as the "Spiegelman monster", in which a scientist put a small DNA strand in a test tube full of nucleotides, effectively removing competition from the environment. When the scientist later analyzed the contents of the test tube, he found that the tube was dominated by the smallest strand capable of self replication and this strand was dubbed "Spiegelman's monster".
I've tried to incorporate this slight but present push towards simplicity into my fitness functions in GA/GP programming by giving ever so slightly higher ratings to simpler solutions. This also helps the computer out as it may calculate the results on simple solutions more quickly than complex ones.
Anyway, kudos on your inventive application of Genetic algorithms. It struck me as being similar to an genetic search for a fractal compression.
I would guess that your evaluation algorithm involved dictionary lookups and the gibberish to english ratio, but would love to hear further detail.
90 support personnel, 4 managers, 8 developers (of which I am one), 2 million customers growing about 200K a month.
This kind of reminds me of the 80's when the phone companies were trying to lobby to charge modem users higher fees.
In the meantime, people are using a technology (voice over ip) less suited for it's purpose than it's predecessor. This traffic will be driving up isp costs since more traffic will be driving up the bandwidth requirements.
I don't know much about the IP phones, but chances are they are bound to a specific port or there is some characteristic about the IP packet which will allow such traffic to be monitored, calculated, and billed for.
In large scale software development projects, it has been my experience that typically, noone has full "mastery" of the code. You have individuals who have a full mastery of aspects of the code, and if you are lucky, the group as a whole will have mastery of the code base. Unlike you, the statement that they were working on mastery of the code actually gives me confidence in the code base. It tells me 2 things. 1) I am dealing with honest folk. A known problem is much easier to deal with than one which gets shoved under the rug by a team of marketing analysts worrying about how the admissions will affect their stock prices. Too often, vendors work hard to deny obvious defects in their applications. Honesty is refreshing. 2) Their gaining confidence as a team, and more ambitious releases will ensue. I just hope that their developing code "mastery" that is redundant across individuals as they will undoubtably suffer attrition as time goes on. - Pat
Thanks for all the information. Like you, I don't place too much stock in benchmarks, especially ones existing prior to product implementations. Volano specs also demonstrated a performance difference, though it should be noted that Volano server produced a thread dump when running versus IBM JDK 1.3. The closest test I could run to a pure 1.2.2 w/inprise jit versus IBM 1.3 was to have the server run 1.2.2 w/ inprise with varied clients. IBM still produced superior numbers though not as pronounced as the previous test. It was about 25% faster (sorry I don't have the actual data handy).
At my work, we're using java (quite successfully) in a system which generates over a million legacy system accesses a day, formatting the results through use JSP as well as DCL (a small perl-like language written in java). The system does millions of dollars of business a day in a mission critical atmosphere. Java delivers great performance for us and minimizes development time.
I downloaded and retested with Inprise's javacomp JIT and my marks effectively doubled from 444 to 823 using CaffeineMark 3.0 as my benchmark. Looks like my initial test didn't incorporate a JIT, but still, the IBM benchmark results were far superior, especially on the logic section of the test which scored 495,997, more than an order of magnitude larger than any of the other tests in the CaffeineMark benchmark. As far as I can tell, my benchmark was legit.
I just ran a CaffeineMark on my PC versus the jdk1.2.2 supplied by SUN versus the new jdk1.3 supplied by IBM. SUN's caffeinemark was 444, while the IBM JVM scored a whopping 6,309. This is an amazing performance difference.