Slashdot Mirror
The Gradual Public Awareness of the Might of Algorithms
Soylent Mauve writes "The trend toward data- and algorithm-driven tuning of business operations has gotten a lot of attention recently — check out the recent articles in the New York Times and the Economist. It looks like computer scientists, especially those with machine learning training, are getting their day in the sun. From the NYT piece: 'It was the Internet that stripped the word of its innocence. Algorithms, as closely guarded as state secrets, buy and sell stocks and mortgage-backed securities, sometimes with a dispassionate zeal that crashes markets. Algorithms promise to find the news that fits you, and even your perfect mate. You can't visit Amazon without being confronted with a list of books and other products that the Great Algoritmi recommends. Its intuitions, of course, are just calculations -- given enough time they could be carried out with stones. But when so much data is processed so rapidly, the effect is oracular and almost opaque.'"
and often hilarious or silly. People already trust computers too much.
I just (a few minutes ago) found this free PDF book about algorithms (written for the undergrad-level student). It's pretty good: http://beust.com/algorithms.pdf
Said one computer scientist getting his day in the sun:
"I'm melting, I'm melting!"
Math is a really really powerful tool.
Whereas algorithms are instantly aware of their own prowess.
Is management starting to wonder (again) whether a computer can really do a better job making the important decisions? But can it yet? There is so much data that needs to be acquired in order to return a meaningful answer.
Some of the most powerful organizations are probably making deals to combine as many databases as possible. Interesting to see (if they would let us see) if that will give them the answers they're looking for. As data acquisition becomes more accurate and less expensive, there will be less privacy but more creative computer output, a trade-off in the value of personal information leading to the possible marginalization of humanity.
Know your pads. One time pad: good for cryptography. Two timing pad: where to take your mistress.
But seriously, a food recipe is an algorithm for all general purposes. All these people are saying is that the machine learning algorithms and match peoples' personalities and buy stock are too complicated for the average Joe Programmer Wannabe and look more or less like a black box. (which if they employ neural networks, instead of say SVN, they are actually black boxes even for the author who wrote it...).
Algorithms, as closely guarded as state secrets
Under the Bush administration they are state secrets. Anyone who doesn't think their online activities pass through some great filter looking for whatever threat du jour that George's paranoia deems a menace need to think twice.
Right to privacy is only a memory. A memory getting more and more faint every day.
Sounds like the title of a song by Jean-Luc Ponty or Tangerine Dream.
Sheesh! Someone needs to spend some time with a dictionary.
If only we could have a gradual (or sudden) awareness of the power of heuristics and modeling ...
Am I part of the core demographic for Swedish Fish?
Here, this may come in handy:
http://www.buttafly.com/bush/index.php
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
You might as well argue about public awareness of the power of "recipes" or "formulas", because that is all algorithms are. Businesses were run by algorithm long before the advent of the computer.
This is just silly. Someone is not comparing apples to oranges, but calling apples oranges. That does not make them so.
maybe not as beautiful as 'clasic' ones, but algorithms indeed. Something like shapes, you know, 'clasic' algorithms (ie: sort) are somewhat like circles (simple formulaes) but real objects (ie: leafs) are extremely complex formulaes only approximated by fractals and with a lot of 'heuristics' in it.
What's in a sig?
Great Algoritmi
That's Muhammad ibn Ms al-Khwrizm to you, pal.
This is one of the reason why getting a CS degree is important, despite what the ignorant masses say in the IT industry. Sure writing lame CRUD applications will satisfy your average customer's needs, but sophisticated algorithms are what provide value beyond a simple shopping cart.
If you're still entrenched in the thought that a CS degree "isn't needed for what I do," then let me propose a somewhat common problem. Suppose your client wants the built in reporting in your web application to minimize the amount of noise introduced by users who forget their password and create a new account rather than resetting it. It's up to you to write code to detect these duplicate accounts. How do you begin doing this beyond simple string comparisons?
As you've demonstrated, the "oracular" part is badly mistaken.
Amazon almost NEVER guesses something I'd buy.
If I buy a new DVD, I am instantly bombarded with ads for EVERY new DVD. I buy the new Terry Pratchett book and I'm bombarded with EVERY book by him or co-authored by him or licensed by him or whatever. I don't want derivatives.
I picked up the "V" comic book (graphic novel) and now I'm bombarded with every comic book they have.
As relates to your post, you can't be the only techo neo-pagan out there. But they just cannot fit you to that group, can they? Although it should be very, very easy to do so.
Algorythms? What is that, music that hurts or something?
They call me the wookie man, I guess that's what I am
Don't get caught up in the hype here. Algorithms are nothing special on their own. These articles are trying to make them look important, like inventions or physical objects, to further pump up the notion of software patents. It's not algorithms that are evil in GWB's great internet filters, it's the machinery that's been built on top of an otherwise dumb network and free internet that's evil.
Without algorithms, there can be no computing but there's nothing really special about any one in particular. Algorithms are just instructions, and there are many ways of achieving the same result. Algorithms can stand alone or be combined into programs that do things users want. The net result is just another set of instructions that can be considered a larger algorithm. Without modern computing equipment, most of these instructions are useless. Like the article say, "try doing this at home." No problem, if you have a computer but a real pain if you only have pen and paper. Medical imaging devices take advantage of mathematics that was little more than a curiosity when it was first published in 1917. The inventors of the device reinvented the math without knowing it some forty years later but it was not until the 1980s that the devices became practical due to the lower cost of computing.
This article is pumping up the value and utility of business methods. Common sense is a valuable thing, but it's not always an invention and business methods never are.
DMCA, Hollings, Palladium. What might have sounded like paranoia is now common sense.
Drums that raise our awareness of global warming.
A post a day keeps productivity at bay.
that the Method of Moments was being applied to economic data. I always thought it was an EM simulation tool, but the theory is generally applicable.
Yeah, I'm just waiting for "algorithms" to replace "on the ground" as the next overused buzzword in the media. "We asked General Petraeus about his algorithm for winning the war in Iraq" "Algorithmically, Bob, it seems the Steelers are unbeatable for this year's Superbowl" "That's right, Jane, it looks like mid-length skirts are the algorithm for fashion success this year" It's gonna be great!
Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clarke, "Profiles of The Future", 1961 (Clarke's third law)
just how close are we to having this statement be 'real' to a large majority of people on this planet? discounting any second or third world countries, how many people in first world countries would consider the 'oracular' nature of an algorithm to be 'magical'?
the education system spread throughout the world is creating an over and under society incapable of distinguishing high technology from magical sources. yep, this can only bode well for the future of humanity.
'i pray to you lord skynet, pls water my crops on the back 40!!!11 here is a sacrifice to your computations.....'
Over the years I've never missed the degree I never got, but I have caught people with CS degrees doing bonehead things nobody who ever tried to do animation on a Commodore 64 would be dumb enough to try. Not knowing about floating point rounding errors comes up all the time, and is especially nasty when the pennies stop adding up right in the business math. One person I know at a large manufacturing concern insists that you should look for people with computer engineering degrees, because they are at least taught how the machine works. CS people aren't, and many of them have never written a program in a lower-level language than Javascript or stored data without the assistance of a DB engine.
Brackets contain world's first nanosig, highly magnified:[.]
It's hard sometimes to explain to a person why algorithms are so important. I thought I'd try and list the top 5 informatics/comp sci. algorithms that *really* help peoples day to day lives. Reply with more! 5. Stero image reconstruction: Now just for cute things like Hawk-eye in tennis and cricket, soon to read road signs and stop you from hitting pedestrians. Also a cornerstone of robotics. 4. Quicksort and other sorting algorithms. It would take decades to sort even a city phone book without O(n log n) sorting. 3. Cryptography/Cryptoanalysis: Lets you shop online, and breaking crypto saved hundreds of thousands of lives in WWII. 2. Pagerank and hashing: Even the unitiated can learn the world's pooled knowledge. Once access to all opinions and all knowledge is free (in every sense of the word), true civilisation might be achievable. 1. Fast fourier transform and friends: We rely on this for most of our communication, and image/movie/audio compression. Machine learing algorithms are important, but they are not important enough to make this list, yet.
What is this world coming to when algorithms can get away with murder ?
This is somewhat ironic, since both the internet and the algorithm were invented by the same person. :-)
This is completely idiotic. All logic works based on algorithms, whether it's in your head or in a computer. Only a monkey or someone with an agenda would write such article. "The Gradual Public Awareness of the Might of Algorithms" eh???????????????????
"There is so much data that needs to be acquired in order to return a meaningful answer."
The key to a good answer is a great question.
This is one of the reason why getting a CS degree is important, despite what the ignorant masses say in the IT industry. Sure writing lame CRUD applications will satisfy your average customer's needs, but sophisticated algorithms are what provide value beyond a simple shopping cart.
Yes, but what is the volume of demand? Google may need one AI expert for say every 200 "regular" programmers. Everybody wants to be King, but there is only one thrown.
Table-ized A.I.
I was going to reply to the GP with the same idea, but your explanation is so much better. Essentially, heuristics just answer a (hopefully) similar problem, not the one at hand.
To do list for Windows
I'm getting my day in the sun? What did I do wrong? I'm sorry! I won't do it again, please let me back inside! I'm getting a tan. Help!
EvilCON - Made Famous by
A heuristic is still a type of algorithm, which is simply a recipe for achieving a result. You seem to be stating that that an algorithm can only be a particular KIND of recipe, which is simply incorrect. The definition of "heuristics" is limited to particular types of procedures, but the definition of "algorithm" is not.
Further, several posters here are simply incorrect about something else. Contrary to what has been stated several times, neither algorithms or heuristics are "guaranteed" to produce correct results. Only correctly-designed algorithms and heuristics produce correct results. There are many examples of bad algorithms and heuristics that have been used in the past (and no doubt there are some in use now), but that does not make them any less "algorithms" or "heuristics". It simply makes them examples of poorly-designed algorithms and heuristics.
In TFA they give an example of an algorithm to validate a credit card number. The funny thing is that steep 2 is not needed at all, you would get the same result without it.
My favorite recommendations are when I am comparing different versions of the same cd for example. I sometimes see that "Customers Who Bought This Item Also Bought" the same item, but the the hard back version, deluxe edition, etc. This can perhaps be attributed to the fact that such differen editions appear later and that some people buy both versions, but it's still dubious.
Or when looking at this cd by a band called Goose, Amazon says that "Other customers suggested these items:", followed by items such as "Favourite Christmas Recipes (Favourite Recipes)"...
Firehed - Unfortunately, thanks to medical breakthroughs, common sense is not as common as it once was.
I applied for an NSF fellowship last year when applying to grad. school. One of the reviewers essentially torpedoed my otherwise well-received application with a 2/5 on "broader impacts" and the following one-line comment:
The reviewer was a particularly clueless example, but it illustrates that even people within the field still judge algorithms of dubious use. It would have probably been much nicer for my own study if this article came out last year, in any case.
(Of course, they're really talking about using people to guide ML in general, which is quite a bit different from the theoretical study of algorithms, but I'll count on the public not to notice the distinction).
The good news is that I'll probably have no trouble getting a job in the field after I finish my doctorate.
I'm a graduate student in CS right now. One of the things I'm researching is stochastic approximation heuristics. Without any argument, these are algorithms. They have to be algorithms, or else the Church-Turing Thesis doesn't apply and we wouldn't be able to have computers do them at all.
An algorithm is, broadly speaking, a terminating sequence of deterministic steps that effectively derives outputs from provided inputs. But don't believe me--after all, I'm just a random guy on Slashdot. But maybe Cormen, Leiserson, Rivest and Stein's Introduction to Algorithms should be believed:
Don Knuth has an equivalent definition of algorithm in The Art of Computer Programming. He makes explicit a couple of details which are implicit in the CLRS definition, but other than that they're interchangeable. Knuth talks about the effectiveness of algorithms, in that an algorithm must uphold the promises the programmer makes about it.
So now that we've got a decent definition of "algorithm", one that's approved by five of the brightest lights in computer science, let's look at simulated annealing. This is a stochastic (random) heuristic approximation process. You say it's not an algorithm, because sometimes it'll give barkingly wrong answers. I say it is. So let's look at our definition of algorithm, and see whether it is or not.
It's well-defined, in that every step of the process has mathematical clarity and precision. It's deterministic, in that if I feed it the exact same inputs (including initializing the pseudorandom number generator to the same seed value), I get the exact same outputs. It will always terminate, thanks to a counter that limits the annealing process to a couple of million operations. And finally, it is effective, in that it upholds the promises I, the programmer, make about the outputs.
According to your reasoning, it fails on the effectiveness criteria. It's not an algorithm because it doesn't solve NP-COMPLETE problems, it simply approximates them. But that's a straw man argument: I never claimed it solved NP-COMPLETE problems, therefore the effectiveness of the algorithm is not determined by whether it solves NP-COMPLETE problems.
No joke: "people that also bought XXX also bought clean underwear"
I guess I'm lucky in the "clean" crowd.
Therefore, just as you state, an heuristic is (can be) an example of an algorithm, but an algorithm is not necessarily a heuristic. Q.E.D.
You can't have it both ways.
"People that bought this random hentai also bought dirty underwear."
Have you been touched by his noodly appendage?
The science of agriculture speaks the language of numbers.
Al ... Gore ... Rhythms
"Math is a really really powerful tool."
Do I use the "1" or the "7" to cut through a log?
Here is one you have probably never heard about. Did you? http://critticall.com/ArtificialSort.html
Reminds me of this strip.
Also, wasn't there an incident where Mr. Knuth went to meet someone (some bigshot at an institute) and he got up to shake his hands saying, "Mr. Knuth! Its a pleasure to meet you. I have read all your books!" and Donald Knuth shook hands, smiled and said "You are lying".
Can't remember where I read that but maybe somone else can cite a relevant source.
Politicians and Pedophiles: Two groups of exploitive bastards who are most dangerous when they're thinking of children.
I don't think Amazon et al are too concerned with rigorous Big O notation and nonrandomized input optimization.
Maybe I am splitting hairs, but I suspect these folks are occaisonally *applying* Cormen, Leiserson, Rivest, and Stein rather than collaborating with them.
I am very small, utmostly microscopic.
The point of the art in NYTimes was that computer systems [in which we embed algorithms] have turned out, after enough years of cost reducing hardware, speeding up communications and harnessing that power to consumer-friendly uses, to be more potent and capable of symbiotic intelligence than even Turing might have expected. By themselves, the computers are only peer-level players in human activities when you see them in Sci-Fi.
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
Hookers will ring your door bell, carrying a 6-pack and a tinfoil hat. They were automagically delivered by Amazon, because it just knew that you wanted a hooker with a cool 6-pack right now. And the tinfoil hat because it also knew you were going to freak out when you knew that you are beign precognited with such accuracy.