3,860,567 = Number of 20 year olds (2000 census rough estimate based on 1/5th of 20-24 year olds)
27% = Percent of population over 25 with a bachelor's degree (2000 census)
25% = Percent of students taking the most popular/useful classes (estimate)
50% = Percent of these students using the most popular textbook (estimate)
5 = Years a textbook edition remains in print (estimate)
6% = Risk free rate of return (estimate)
$100 = Average textbook price (estimate)
20% = McGraw hill net margin (per www.fool.com)
The textbook company would sell 131,259 textbooks per year, for a net profit of $2,625,186 annually. Given the 5 year life span and 6% risk free rate, the textbook company would be willing to sell a textbook with the above expected sales for no less than $11 million. This means we could purchase roughly 9 of the most popular textbooks for $100 million. May be off by a fair margin, but it's clearly not going to be near 100 textbooks. Seems like there are much better uses of the money.
I'm trying to glean what some of the practical implications could be of this discovery.
It seems to me at this Ricci Flow differential equation could be quite useful practically. For example, in pattern recognition, if a computer could build a 3d model of an object using multiple vantage points, then simplify the object to one of the handfull of object types described by Perlman using the Ricci Flow, then this simple catagorization might help in the identification of complex objects (e.g. a donut really is a donut, even if it's been heavily frosted).
Do you know if Perlman's technique for handling the singularities will help with the numerical implementation of this process? Or are these issues numerically simple to solve - but only challenging to solve in proof?
I too am in data mining, but I use EMACS to write all of my code, and my case sensitive programs all prefer lower case. So caps-lock has been re-mapped using keytweak to CTRL. The only problem is when I use someone elses computer I usually end up over-writing anything I try to copy with C while turning on caps-lock.
There were rumors that supplies of the X360 were artificially constrained, obstensibly to create hype and/or to punish uncooperative retailers. I doubt that Nintendo will follow suit, given they are trying to appeal to the masses, and are creating a lot of hype driven by product innovation.
That is certainly part of it. The other part is the network effect, and the large demand side increasing returns ball that is started rolling by the low price point.
As an MBA student having just studied the rise of the video game industry in the late 80s, lead by Nintendo, this article makes perfect strategic sense for Nintendo. The industry then was catagorized by "Demand Side Increasing Returns." Rather than the typical increasing returns created on the supply side by economies of scale or scope, demand side increasing returns occur through network effects. When there is a large installed user or support base for a product in a category such as this, there is a large incentive for consumers new consumers to purchase from the established player rather than from new entrants. Just like what Microsoft has done with the OS (not implying Nintendo is like Microsoft, but their strategies are similar in this regard).
This demand side increasing returns effect is only magnified by the current environment. By significantly innovating in the category and undercutting the competition on up-front costs, Nintento is set to build a substantial user base very quickly. This user base will feed upon itself with the free internet access, as this will draw many players who will have to purchase a Wii to play with other players online. Nintendo wins in the end because their licensing agreements for game development and production are very favorable, giving them a substantial fraction (20% or more historically) of game sales. They will quickly recover any loss or marginal profit on providing a cheap console with free network access through game consumption. And consumers won't complain because we are quite happy to spend $400 on games spread out over several years when we only feel it in small chunks as we periodically purchase new games.
I'd buy Nintento stock if it hadn't allready skyrocketed.
I'm a statistician and data mining consultant, and i've implemented models based on millions of records generating consulting fees in the high hundreds of thousands of dollars. I thus have a strong understanding of the data, modeling and project management aspects of data mining ventures.
I believe there are several fundamental factors required to make a data-mining project succesful:
1) A mathematically precise definition of what it is to be modeled (the response) as in the probability of purchasing product x rather than "profit"
2) Multiple sources of data linked together: demographic, financial, transactional, etc...
3) A set of "variables" from 2) that have a strong, intuitive relationship to the response in 1)
4) A reasonably sophisticated statistical algorithm designed to weed out the significant relationships between 1) and 3)
5) An organizational culture willing and capable of institutionalizing the model into a decision making process
In my experience, these projects end up failing because of problems with one of the above critical steps. Roughly 50% of the time 5) will hold back a project. This isn't what a consultant, statistician or DBA will tell you, because more often than not they don't stick around long enough to see it through to the bottom line. Step 1) and 4) are basically table stakes. If you can't very precisely define your objective (and thus that which you would like to predict, the response) you aren't going anywhere. Similarly, if there is a significant flaw in your analytical method (e.g. overfitting with neural networks) then you'll produce rubbish.
Steps 2) and 3) really determine the upper bound for how succesfull the project can be. Often, there is a fundamental driver of the value in question that is not accounted for in the analysis. For example, in modeling auto-insurance claim frequency, it is very difficult to obtain number of miles driven, and yet it is the most significant factor impacting the occurrence of claims. In the end, the quality of the model hinges on linking together high-quality data sources that are often transactional and deriving from them variables that would dramatically suprise you if they weren't predictive of the response in question.
For a publicly traded company paying something can be better than having to expense something or reducing your revenues. The primary quantity of interest is net income, which is accrued revenues - expenses and is not cash-flows. They could very well have decided to expense the whole amount this year, which would have hurt their earnings regardless of when they paid it. However, as with most class-action lawsuits, the settlement terms require that the claimants pursue the plaintiff for their reimbursement - hence the credits. The fact that these will be deducted from revenues in the periods in which they are redeemed implies that Google either expects a large amount of these not to go unused, or is doing some fancy accounting to smooth out their earnings.
While I enjoyed this film, I think that the major reason it receives poor reviews from popular culture critics is the poor character acting. Scenes designed to build emotional rapport between the main characters came of stiltingly, leaving me wondering if the director was so engrossed in the costumes and special effects that they forgot the actors. Charlize Theron is such a brilliant actress (Monster) that I couldn't understand how amatuerishly some of the dialog was delivered. The scene between Aeon and her sister in the market comes particularly to mind.
That being said, I thought the plot of the movie was fantastic, and some of the acting lived up to it. But there's no way a film this far out is going to be accepted by the mass consumers unless they can very closely relate to the characters, as evidenced by the success of Serenity.
Will there come a day where the study of hypnosis, or other forms of cognitive suggestion, is abused by firms for marketing? Perhaps some allready are. What kind of privacy law would restrict this?
"The biggest benefit is knowing that we're giving jobs to American workers, versus a foreign country,"
This isn't sufficient motivation for US firms to rural-source, and neither are local accents or convenient time zones. The reason the programmer makes $100 in NYC is that they need to be there physically, to interface with a broader team, client, management, etc... If a job can be sourced to someone in a small town in America, 99% of the time it can be sourced to someone in India, for pennies on the dollar.
I was a competitive quake3 player, in the clan stickmen, and used to play with fatal1ty before he won his first major tournament. I would travel to Kansas City and LAN with him for a few days, and what most people don't realize is that he is truly obsessive about playing. He would put in 15 hour days 7 days a week, exericise to keep in good physical shape, and study the game, and more importantly, it's players, incessantly. Any time I had an advantage over him in a 1v1 map, we would play it over and over and over again untill he had learned all of my patterns.
Few people appreciate how much passion he has for winning.
This is only the first step, a new technology referred to as telemeatics will soon replace any need to track your cell phone. Our cars are full of computers, just waiting to be wirelessly connected. Soon, such services will be able to:
- Detect vehicular crashes, alerting both the nearest emergency services, your family and your insurance company.
- Monitor your vehicle, suggesting maintenance.
- Allow your Insurance company to track the number of miles driven, your average speed, percentage of quick stops, etc...
- Learn about the road conditions from the vehicles in front of you.
And these are just some of the initial easy to reach applications.
Rather than spending $5 to download a movie that is going to 'expire' in 24 hours, what about a constant per-minute rate for an on-demand, live, un-recordable broadcast? Suppose that the average film is 100 minutes long, then a $.05 per minute rate would achieve the same effect as a $5 rental, but if I decide to stop watching halfway through, i've only spent $2.50, and I can shell out the $2.50 a week later when I want to pick the film back up.
A local version of what I've watched of the film could be stored and encrypted on my computer, to accomplish the p2p scalability. Best of all, if I decide the film sucks, I can stop watching it and save myself some cash.
One of the major problems with Amazon is that there is little to no incentive for me to rate a product or provide any feedback, unless I want to itch my altruistic 'benefit the shopping masses' bug, or i have some axe to grind. However, i use the Netflix rating system extensively, because they use my ratings to provide feedback on what new movies I might like, and the system actually works. How can Amazon incentivize people to tag??
What with gmail being copyrighted in the UK, perhaps this is designed to prevent others from frivously patenting google's technology and then suing them in an attempt to get paid off?
You are exactly right, and in a modern society with millions of people interacting, we can leverage the available information to optimize interactions. Credit-scoring is precisely that. The next time you go and buy a home, marvel at the fact that you can even get a loan. A 100 years ago a bank would have no clue as to how trusworthy an individual you were, and without a significant amount of capital or collateral, you couldn't obtain financing. This was a foundation of the seperation of classes. Without credit, it's extroadinarly hard to ride out the major fluctuations in cash flow experienced by people near the poverty line. The wealthy can lose 50% of their income and still put food on the table, but do that to someone makeing $20k a year and they are completely margenalized.
I feel strongly that this is an issue that the Slashdot readership over-reacts to out of paranoia. Few large companies and governments are really "out to get us," but just like us they do screw things up here or there. After all, they are only human.
I am a data mining consultant for insurance companies, and I specialize in part in assembling external sources of data for use in predictive modeling. I have seen the other side of how useful this data can be for individual companies, marketplaces and society as a whole.
For example, consider personal credit for automobile insurance - one of Choicepoint's major markets. It's not suprising that a person with a credit score in the 500s, who hasn't paid a bill on time since college is going to be a bad insurance risk. If you're totally irresponsible with your finances, you probably haven't got the rest of your life in order, and there's a higher chance reckless behavior or fraud. Regardless, statistical modeling can show that these people can be orders of magnitude more expensive to insure - and that's based on analysis of data sets with millions of insureds.
With credit-scoring, those drivers can be identified and charged a premium commensurate with their risk. Without it, you me and every other responsible driver has to subsidize their behavior. Not only isn't that fair, but it gives the person no incentive to improve their behavior. Credit-scores are only one of many variables used in these models, and overall, their use in pricing has dramatically lowered the cost of insurance, and the frequency of accidents.
Certainly, I'm concerned about the security breaches that have occurred at these vendors, and the quality of the data is often a serious problem. But I hope the regulation of the industry doesn't throw the baby out with the bath-water.
I am a consultant with one of the big four audit firms, and each week I submit billable hours by state and city. Last year I spent 150 nights in hotels, and for any state where I worked over 40 hours I had to pay income tax. Aside from inflating my H&R block bill, I think the system is fair. I get a tax credit in NY (my home state) for the hours I don't work here, and pay those taxes (often at lower tax levels) in the states I travel to.
I wouldn't say that it's devoid of relevance to anything. When comparing neural networks to "simpler" modeling methodologies such as generalized linear or generalized additive models, the distinction is very meaningful.
Regardless, you haven't addressed my primary point that the limiting capacity is the construction of meaningful and comprehensive training data. There are three primary constraints on developing these models:
The data used in training.
The structure of the model.
The method used to train the model.
I think there are tons of smart people out there capable of developing model structures and training methods to emulate and generalize with data given to them (e.g. neural nets, svm's, genetic algorithms, random forests, etc...). But who can construct a set of data that could support an "intelligent" classifier? I don't think we are on the right planet yet, let alone the right ball park.
It is fairly easy to show (see Bishop 1995) that a simple two layer neural network can scale to reproduce arbitrarily complex but smooth functions to any degree of required accuracy, and that a three layer neural network can extend this capapility to functions with discontinuities. While mathematically this is a tantalizing prospect, and only begins to cover the work that has been done to extend the capabilities of neural networks and other machine learning algorithms (such as support vector machines), there remains a fundamental problem. In order for these networks to effectively learn, they must be presented with a tremendous number of high quality and meaningful sequences of input and output.
For example, in text recognition, hundreds of thousands of hand written characters are painstakingly hand labeled with their correct letters and used as a learning database on which the algorithm is trained. The algorithm will then accurately reproduce the correct categorization for a suprisingly high number of the training examples, and any new examples drawn from the same population. But given new examples written in a different script or style, the classifier will fail to generalize
How can we hope to create a training database that is comprehensive enough to cover a topic that, when learned, would demonstrate intelligence? And fundamentally, aren't we just creating a really good mimic?
The tricky thing is how to fix this. As a data mining consultant I organize the purchase of hundreds of thousands of dollars worth of sensitive data for our clients. And in all of these purchases, I have never seen a vendor proactively validate that my team, or my client's were bona fide.
One would hope that these vendors check that our companies are statutory entities and that our e-mail and mailing addresses are associated with these entities, but these can all be spoofed or are difficult to verify. We often sign legal agreements, but that doesn't matter much if I'm a thief.
Fundamentally, the onus of identification should fall on the government. A system of encrypted public keys used to electronically transfer data between verified entities could prevent this from being a problem.
Regardless, the type of data that Choicepoint sells (predominantly personal credit) is used throughout the banking and insurance industries. If companies are restricted from using the information, it will become meaningless, and markets will regress to less efficient states and we will all lose in the end.
Slashdot Mirror
Consider the economics of it:
3,860,567 = Number of 20 year olds (2000 census rough estimate based on 1/5th of 20-24 year olds)
27% = Percent of population over 25 with a bachelor's degree (2000 census)
25% = Percent of students taking the most popular/useful classes (estimate)
50% = Percent of these students using the most popular textbook (estimate)
5 = Years a textbook edition remains in print (estimate)
6% = Risk free rate of return (estimate)
$100 = Average textbook price (estimate)
20% = McGraw hill net margin (per www.fool.com)
The textbook company would sell 131,259 textbooks per year, for a net profit of $2,625,186 annually. Given the 5 year life span and 6% risk free rate, the textbook company would be willing to sell a textbook with the above expected sales for no less than $11 million. This means we could purchase roughly 9 of the most popular textbooks for $100 million. May be off by a fair margin, but it's clearly not going to be near 100 textbooks. Seems like there are much better uses of the money.
I'm trying to glean what some of the practical implications could be of this discovery.
It seems to me at this Ricci Flow differential equation could be quite useful practically. For example, in pattern recognition, if a computer could build a 3d model of an object using multiple vantage points, then simplify the object to one of the handfull of object types described by Perlman using the Ricci Flow, then this simple catagorization might help in the identification of complex objects (e.g. a donut really is a donut, even if it's been heavily frosted).
Do you know if Perlman's technique for handling the singularities will help with the numerical implementation of this process? Or are these issues numerically simple to solve - but only challenging to solve in proof?
I too am in data mining, but I use EMACS to write all of my code, and my case sensitive programs all prefer lower case. So caps-lock has been re-mapped using keytweak to CTRL. The only problem is when I use someone elses computer I usually end up over-writing anything I try to copy with C while turning on caps-lock.
There were rumors that supplies of the X360 were artificially constrained, obstensibly to create hype and/or to punish uncooperative retailers. I doubt that Nintendo will follow suit, given they are trying to appeal to the masses, and are creating a lot of hype driven by product innovation.
That is certainly part of it. The other part is the network effect, and the large demand side increasing returns ball that is started rolling by the low price point.
As an MBA student having just studied the rise of the video game industry in the late 80s, lead by Nintendo, this article makes perfect strategic sense for Nintendo. The industry then was catagorized by "Demand Side Increasing Returns." Rather than the typical increasing returns created on the supply side by economies of scale or scope, demand side increasing returns occur through network effects. When there is a large installed user or support base for a product in a category such as this, there is a large incentive for consumers new consumers to purchase from the established player rather than from new entrants. Just like what Microsoft has done with the OS (not implying Nintendo is like Microsoft, but their strategies are similar in this regard).
This demand side increasing returns effect is only magnified by the current environment. By significantly innovating in the category and undercutting the competition on up-front costs, Nintento is set to build a substantial user base very quickly. This user base will feed upon itself with the free internet access, as this will draw many players who will have to purchase a Wii to play with other players online. Nintendo wins in the end because their licensing agreements for game development and production are very favorable, giving them a substantial fraction (20% or more historically) of game sales. They will quickly recover any loss or marginal profit on providing a cheap console with free network access through game consumption. And consumers won't complain because we are quite happy to spend $400 on games spread out over several years when we only feel it in small chunks as we periodically purchase new games.
I'd buy Nintento stock if it hadn't allready skyrocketed.
They should add a W... I can't believe any Japanese company would name a product one letter short of WWII.
That's like giving a DEA agent in Columbia a "bulletproof" vest.
I'm a statistician and data mining consultant, and i've implemented models based on millions of records generating consulting fees in the high hundreds of thousands of dollars. I thus have a strong understanding of the data, modeling and project management aspects of data mining ventures.
I believe there are several fundamental factors required to make a data-mining project succesful:
1) A mathematically precise definition of what it is to be modeled (the response) as in the probability of purchasing product x rather than "profit"
2) Multiple sources of data linked together: demographic, financial, transactional, etc...
3) A set of "variables" from 2) that have a strong, intuitive relationship to the response in 1)
4) A reasonably sophisticated statistical algorithm designed to weed out the significant relationships between 1) and 3)
5) An organizational culture willing and capable of institutionalizing the model into a decision making process
In my experience, these projects end up failing because of problems with one of the above critical steps. Roughly 50% of the time 5) will hold back a project. This isn't what a consultant, statistician or DBA will tell you, because more often than not they don't stick around long enough to see it through to the bottom line. Step 1) and 4) are basically table stakes. If you can't very precisely define your objective (and thus that which you would like to predict, the response) you aren't going anywhere. Similarly, if there is a significant flaw in your analytical method (e.g. overfitting with neural networks) then you'll produce rubbish.
Steps 2) and 3) really determine the upper bound for how succesfull the project can be. Often, there is a fundamental driver of the value in question that is not accounted for in the analysis. For example, in modeling auto-insurance claim frequency, it is very difficult to obtain number of miles driven, and yet it is the most significant factor impacting the occurrence of claims. In the end, the quality of the model hinges on linking together high-quality data sources that are often transactional and deriving from them variables that would dramatically suprise you if they weren't predictive of the response in question.
That's my 2 cents...
For a publicly traded company paying something can be better than having to expense something or reducing your revenues. The primary quantity of interest is net income, which is accrued revenues - expenses and is not cash-flows. They could very well have decided to expense the whole amount this year, which would have hurt their earnings regardless of when they paid it. However, as with most class-action lawsuits, the settlement terms require that the claimants pursue the plaintiff for their reimbursement - hence the credits. The fact that these will be deducted from revenues in the periods in which they are redeemed implies that Google either expects a large amount of these not to go unused, or is doing some fancy accounting to smooth out their earnings.
While I enjoyed this film, I think that the major reason it receives poor reviews from popular culture critics is the poor character acting. Scenes designed to build emotional rapport between the main characters came of stiltingly, leaving me wondering if the director was so engrossed in the costumes and special effects that they forgot the actors. Charlize Theron is such a brilliant actress (Monster) that I couldn't understand how amatuerishly some of the dialog was delivered. The scene between Aeon and her sister in the market comes particularly to mind.
That being said, I thought the plot of the movie was fantastic, and some of the acting lived up to it. But there's no way a film this far out is going to be accepted by the mass consumers unless they can very closely relate to the characters, as evidenced by the success of Serenity.
Will there come a day where the study of hypnosis, or other forms of cognitive suggestion, is abused by firms for marketing? Perhaps some allready are. What kind of privacy law would restrict this?
This isn't sufficient motivation for US firms to rural-source, and neither are local accents or convenient time zones. The reason the programmer makes $100 in NYC is that they need to be there physically, to interface with a broader team, client, management, etc... If a job can be sourced to someone in a small town in America, 99% of the time it can be sourced to someone in India, for pennies on the dollar.
I was a competitive quake3 player, in the clan stickmen, and used to play with fatal1ty before he won his first major tournament. I would travel to Kansas City and LAN with him for a few days, and what most people don't realize is that he is truly obsessive about playing. He would put in 15 hour days 7 days a week, exericise to keep in good physical shape, and study the game, and more importantly, it's players, incessantly. Any time I had an advantage over him in a 1v1 map, we would play it over and over and over again untill he had learned all of my patterns.
Few people appreciate how much passion he has for winning.
Sounds like Google may be ready to go starbucks.
This is only the first step, a new technology referred to as telemeatics will soon replace any need to track your cell phone. Our cars are full of computers, just waiting to be wirelessly connected. Soon, such services will be able to:
- Detect vehicular crashes, alerting both the nearest emergency services, your family and your insurance company.
- Monitor your vehicle, suggesting maintenance.
- Allow your Insurance company to track the number of miles driven, your average speed, percentage of quick stops, etc...
- Learn about the road conditions from the vehicles in front of you.
And these are just some of the initial easy to reach applications.
Rather than spending $5 to download a movie that is going to 'expire' in 24 hours, what about a constant per-minute rate for an on-demand, live, un-recordable broadcast? Suppose that the average film is 100 minutes long, then a $.05 per minute rate would achieve the same effect as a $5 rental, but if I decide to stop watching halfway through, i've only spent $2.50, and I can shell out the $2.50 a week later when I want to pick the film back up.
A local version of what I've watched of the film could be stored and encrypted on my computer, to accomplish the p2p scalability. Best of all, if I decide the film sucks, I can stop watching it and save myself some cash.
One of the major problems with Amazon is that there is little to no incentive for me to rate a product or provide any feedback, unless I want to itch my altruistic 'benefit the shopping masses' bug, or i have some axe to grind. However, i use the Netflix rating system extensively, because they use my ratings to provide feedback on what new movies I might like, and the system actually works. How can Amazon incentivize people to tag??
What with gmail being copyrighted in the UK, perhaps this is designed to prevent others from frivously patenting google's technology and then suing them in an attempt to get paid off?
You are exactly right, and in a modern society with millions of people interacting, we can leverage the available information to optimize interactions. Credit-scoring is precisely that. The next time you go and buy a home, marvel at the fact that you can even get a loan. A 100 years ago a bank would have no clue as to how trusworthy an individual you were, and without a significant amount of capital or collateral, you couldn't obtain financing. This was a foundation of the seperation of classes. Without credit, it's extroadinarly hard to ride out the major fluctuations in cash flow experienced by people near the poverty line. The wealthy can lose 50% of their income and still put food on the table, but do that to someone makeing $20k a year and they are completely margenalized.
I feel strongly that this is an issue that the Slashdot readership over-reacts to out of paranoia. Few large companies and governments are really "out to get us," but just like us they do screw things up here or there. After all, they are only human.
I am a data mining consultant for insurance companies, and I specialize in part in assembling external sources of data for use in predictive modeling. I have seen the other side of how useful this data can be for individual companies, marketplaces and society as a whole.
For example, consider personal credit for automobile insurance - one of Choicepoint's major markets. It's not suprising that a person with a credit score in the 500s, who hasn't paid a bill on time since college is going to be a bad insurance risk. If you're totally irresponsible with your finances, you probably haven't got the rest of your life in order, and there's a higher chance reckless behavior or fraud. Regardless, statistical modeling can show that these people can be orders of magnitude more expensive to insure - and that's based on analysis of data sets with millions of insureds.
With credit-scoring, those drivers can be identified and charged a premium commensurate with their risk. Without it, you me and every other responsible driver has to subsidize their behavior. Not only isn't that fair, but it gives the person no incentive to improve their behavior. Credit-scores are only one of many variables used in these models, and overall, their use in pricing has dramatically lowered the cost of insurance, and the frequency of accidents.
Certainly, I'm concerned about the security breaches that have occurred at these vendors, and the quality of the data is often a serious problem. But I hope the regulation of the industry doesn't throw the baby out with the bath-water.
I am a consultant with one of the big four audit firms, and each week I submit billable hours by state and city. Last year I spent 150 nights in hotels, and for any state where I worked over 40 hours I had to pay income tax. Aside from inflating my H&R block bill, I think the system is fair. I get a tax credit in NY (my home state) for the hours I don't work here, and pay those taxes (often at lower tax levels) in the states I travel to.
Regardless, you haven't addressed my primary point that the limiting capacity is the construction of meaningful and comprehensive training data. There are three primary constraints on developing these models:
The data used in training.
The structure of the model.
The method used to train the model.
I think there are tons of smart people out there capable of developing model structures and training methods to emulate and generalize with data given to them (e.g. neural nets, svm's, genetic algorithms, random forests, etc...). But who can construct a set of data that could support an "intelligent" classifier? I don't think we are on the right planet yet, let alone the right ball park.
It is fairly easy to show (see Bishop 1995) that a simple two layer neural network can scale to reproduce arbitrarily complex but smooth functions to any degree of required accuracy, and that a three layer neural network can extend this capapility to functions with discontinuities. While mathematically this is a tantalizing prospect, and only begins to cover the work that has been done to extend the capabilities of neural networks and other machine learning algorithms (such as support vector machines), there remains a fundamental problem. In order for these networks to effectively learn, they must be presented with a tremendous number of high quality and meaningful sequences of input and output.
For example, in text recognition, hundreds of thousands of hand written characters are painstakingly hand labeled with their correct letters and used as a learning database on which the algorithm is trained. The algorithm will then accurately reproduce the correct categorization for a suprisingly high number of the training examples, and any new examples drawn from the same population. But given new examples written in a different script or style, the classifier will fail to generalize
How can we hope to create a training database that is comprehensive enough to cover a topic that, when learned, would demonstrate intelligence? And fundamentally, aren't we just creating a really good mimic?
The tricky thing is how to fix this. As a data mining consultant I organize the purchase of hundreds of thousands of dollars worth of sensitive data for our clients. And in all of these purchases, I have never seen a vendor proactively validate that my team, or my client's were bona fide.
One would hope that these vendors check that our companies are statutory entities and that our e-mail and mailing addresses are associated with these entities, but these can all be spoofed or are difficult to verify. We often sign legal agreements, but that doesn't matter much if I'm a thief.
Fundamentally, the onus of identification should fall on the government. A system of encrypted public keys used to electronically transfer data between verified entities could prevent this from being a problem.
Regardless, the type of data that Choicepoint sells (predominantly personal credit) is used throughout the banking and insurance industries. If companies are restricted from using the information, it will become meaningless, and markets will regress to less efficient states and we will all lose in the end.