- whack-jobs (inability to police religious fundamentalist groups, like "Free Market Fundamentalists" who will sabotage the project because it offends their faith).
The solution to free market whack jobs -- really just private sector rent-seekers -- isn't public choice rent-seeking; it is to collect and then redistribute all economic rent -- that cannot be allocated to their true source as positive externalities (PE's like public domain technologies) -- equally to all segments of society. Anything else creates positive feedback loops that cut out positive sum innovators from the system.
Non-innovative rent seekers are the problem, whether in the public or private sectors. When I say "the problem" I mean it -- they could quite possibly fry the biosphere.
Interestingly it was Gerard O'Neill who argued in the 1970's for solar power satellites constructed from lunar material and, as part of that argument predicted the industrialization of China would lead to increased CO2 emissions from coal burning that would mandate radical restructuring of global energy technology. It may be too late now to pursue nonterrestrial material SPS since the baby boomer generation, raised and educated to pioneer space from childhood, was denied that opportunity by --- well that is the question of the millennium if not the epoch isn't it? There are almost as many answers to that question as there are religions.
The proximate cause was that despite there being an obvious direction in place subsequent to the space race (remember the Apollo program?) that could have been followed through to space industrialization -- the launch service industry did not enjoy the same protection from government competition that the satellite industry enjoyed:
* (c) Private enterprise; access; competition
In order to facilitate this development and to provide for the widest possible participation by private enterprise, United States participation in the global system shall be in the form of a private corporation, subject to appropriate governmental regulation. It is the intent of Congress that all authorized users shall have nondiscriminatory access to the system; that maximum competition be maintained in the provision of equipment and services utilized by the system; that the corporation created under this chapter be so organized and operated as to maintain and strengthen competition in the provision of communications services to the public; and that the activities of the corporation created under this chapter and of the persons or companies participating in the ownership of the corporation shall be consistent with the Federal antitrust laws.
The fact that Malthusian paradigm didn't precisely follow the Club of Rome's "Limits to Growth" model doesn't change the reality of the Malthusian paradigm given a fundamentally limited biosphere undergoing its largest extinction event in 60 million years. The Club of Rome merely added academic fashion to the urgency of the Malthusian situation still facing the biosphere. The 1970s was the right time to start the drive for space industrialization based on a private launch service industry. It didn't happen, the pioneering culture that founded the US is being replaced by government policy with less pioneering cultures and now we're all facing some increasingly obvious difficulties -- not just pioneer American stock -- and not just humans.
The cost of getting silicon into space from the lunar surface would be orders of magnitude less than launching from earth due not only to the much shallower gravity well but also due to the absence of atmosphere.
No beanstalk needed.
At worst a Dyneema Rotovator might be needed but probably not even that.
First, the bulk of the materials are manufactured in space from lunar raw material transported to orbital facilities so you don't need to land those facilities on the lunar surface, and you don't have to worry about g-loading the raw materials you are sending to the orbital facilities.
Second, you don't manufacture everything in space -- only bulky materials like solar cells, reflectors, structural members and perhaps klystrons. Only residual materials (raw and manufactured) are of terrestria
Matt Mahoney has communicated his concern to me that the 1.3 bits per character entropy measured by Shannon is likely a smaller number with the enwik8 corpus due to regularities from embedded markup. He has already compressed enwik9 (1,000,000,000 bytes) to less than 1.3 bits per character and his analysis shows that this is largely due to a large section of data tables present in that larger sample -- which entails a large amount of embedded markup. While the entropy of enwik8 is unlikely to be as low as enwik9, this difference does evidence the lower entropy of embedded markup.
Until Shannon type experiments, involving humans doing next character predictions of enwik8, are performed, the bounds of enwik8's entropy range must remain unknown but is likely lower than 0.6 to 1.3. As such an experiment would be expensive, it is going to be difficult to say with any simple bpc measure when the Hutter Prize is breaching the threshold of AI. What the Hutter Prizes bpc metric gives us, however, is a clear measure of progress.
My apologies to the other members of the Hutter Prize Committee and the/. community for this error.
PS: Another area of concern raised by Mahoney is that enwik8, at 10e8 characters, is only as much verbal information as a 2 or 3 year old has encountered so although it is sufficient to demonstrate AI capabilities well beyond the current state of the art, his preference is for a much larger contest with fewer resource restrictions focusing on the 10e9 character enwik9 which is more likely to produce a the AI equivalent of an adult with encyclopedic knowledge.
What could be clearer than that prediction is relevant to AI? ML gives predictions and EM gives decisions based on predictions. Maximum compression of observations gives optimal models for prediction as you yourself have asserted, however old-hat this understanding of compression is. It all seems so straight forward if you accept, as you say you do, the relationship between compression and optimal prediction. I don't understand what your issue with the Hutter Prize is.
The Hutter Prize submissions consist of compressed corpus plus decompressor. It does not specify the method used, quite deliberately. In practice, it will probably be a combination of human insight augmented by computation that will produce the compressions. In the early stages, it should not be surprising that computer programs without a lot of a priori knowledge are squeezing out redundancy.
Clearly being able to predict the next symbol in a stream coming from a corpus of human knowledge is relevant to AI and such prediction is the core technique of compression.
As for "optimality", you aren't thinking about things at the right level. If you have an unknown environment the fact that it is unknown means you are making educated guesses as to maximum likelihoods, or "expectation maximization". No -- of course those guesses are very unlikely (approaching zero) to be correct 100% of the time.
So am I take it you disagree with Mahoney's statement:
He gives a formal proof, but it basically says that the only possible distribution of the infinite set of programs (or strings) with nonzero probability is one which favors shorter programs over longer ones. Given any string of length n with probability p > 0, there are an infinite set of strings longer than n, but only a finite number of these can have probability higher than p.
No, you have misunderstood what I mean when I say:
What constructs/concepts did it reify in order to achieve better compression of human knowledge and how much did they contribute?
The "it" to which the above sentence refers is not the compression program, but rather the program that outputs the corpus: the decompression program plus the compressed representation. In principle, this decompression program and compressed representation can be constructed entirely by scholars/epistemologists with no computers assisting them except during the testing of the decompression process.
Look up Kolmogorov Complexity. It's been proven that the general task of optimal compression is not just "hard", it is uncomputable. Think of it like this:
Take a random bit string of some reasonably large length -- say 1 million bits. Answer the question: What is the shortest program that can output that bit string?
It's been proven you can't compute that optimal program as the output of another program.
Human text is not the same as a random bit string, however much we might joke about it.
The value of the Hutter Prize is the epistemological implications of the compressed human knowledge: What constructs/concepts did it reify in order to achieve better compression of human knowledge and how much did they contribute? That tells us a lot about the human conversation we can't get any other way.
The value of the Hutter Prize isn't in the use of Hutter's theory to build AI. The value of the Hutter Prize is in the use of compression ratio to provide a figure of merit for AI that is very good. That value has been proven in various ways -- mathematically and practically.
Hutter's* AIXI, http://www.idsia.ch/~marcus/ai/paixi.htm makes another argument for the connection between compression and AI that is more general than the Turing test. He proves that the optimal behavior of an agent (an interactive system that receives a reward signal from an unknown environment) is to guess that the environement is most likely computed by the shortest possible program that is consistent with the behavior observed so far. In other words, the most likely outcome for any experiment is the one with the simplest explanation, where "simplest" means the smallest program that could model what you currently know about the universe.
He gives a formal proof, but it basically says that the only possible distribution of the infinite set of programs (or strings) with nonzero probability is one which favors shorter programs over longer ones. Given any string of length n with probability p > 0, there are an infinite set of strings longer than n, but only a finite number of these can have probability higher than p.
non-connectionist previous attempts (the stuff that came from the functionalists) has come up pretty short - and will continue to do so even if scaled up massively.
paq8hp12 uses a neural network, ie: it has a connectionist component.
That's also why most abusers commit suicide after they break and kill their ex's. They're too scared to go to prison.
Actuallly this is quite insightful except for one thing:
She doesn't actually cop to the fact that the "alpha of state", for which she works, is a wimp who is so chickenshit that "he" has to resort to black and Hispanic prison gangs to meet out "his" worst punishments on spouse-abusing white guys, and that "he" can't actually mate with the women who look to "him" for protection.
Past studies have documented the prevalence of black on white sexual aggression in prison.(213) These findings are further confirmed by Human Rights Watch's own research. Overall, our correspondence and interviews with white, black, and Hispanic inmates convince us that white inmates are disproportionately targeted for abuse. Although many whites reported being raped by white inmates, black on white abuse appears to be more common. To a much lesser extent, non-Hispanic whites also reported being victimized by Hispanic inmates.
Other than sexual abuse of white inmates by African Americans, and, less frequently, Hispanics, interracial and interethnic sexual abuse appears to be much less common than sexual abuse committed by persons of one race or ethnicity against members of that same group. In other words, African Americans typically face sexual abuse at the hands of other African Americans, and Hispanics at the hands of other Hispanics. Some inmates told Human Rights Watch that this pattern reflected an inmate rule, one that was strictly enforced: "only a black can turn out [rape] a black, and only a chicano can turn out a chicano."(215) Breaking this rule by sexually abusing someone of another race or ethnicity, with the exception of a white inmate, could lead to racial or ethnic unrest, as other members of the victim's group would retaliate against the perpetrator's group. A Texas inmate explained, for example: "The Mexicans--indeed all latinos, nobody outside their race can 'check' one without permission from the town that, that person is from. If a black dude were to check a mexican w/out such permission & the mexican stays down & fights back, a riot will take place."(216)
The causes of black on white sexual abuse in prison have been much analyzed. Some commentators have attributed it to the norms of a violent black subculture, the result of social conditioning that encourages aggressiveness and the use of force.(217) Others have viewed it as a form of revenge for white dominance of blacks in outside society.(218) Viewing rape as a hate crime rather than one primarily motivated by sexual urges, they believe that sexually abused white inmates are essentially convenient surrogates for whites generally. Elaborating on this theory, one commentator surmised that "[i]n raping a white inmate, the black aggressor may in some measure be assaulting the white guard on the catwalk."(219)
Some inmates, both black and white, told Human Rights Watch that whites were generally perceived as weaker and thus more vulnerable to sexual abuse. An African American prisoner, describing the situation of incarcerated whites, said:
When individuals come to prison, they know that the first thing that they will have to do is fight. Now there are individuals that are from a certain race that the majority of them are not physically equip to fight. So they are the majority that are force to engage in sexual acts.(220)
Another African American inmate, while generally agreeing with the idea of whites as easy victims, gave a more politically-oriented explanation for the problem of black on white sexual abuse:
Before I continue, let me explain that I consider myself to be speaking from mainly a black perspective. The reason I say that is not to be racist, but to emphasize that on the ma
Humans run at about 100 watts. Solar flux PEAK is about 1000 watts per square meter during peak sun, or about 100 watts average. In theory, you can feed a person on one square meter of solar flux. So maybe, just maybe, you can pull off these "vertical gardens" in urban areas if you carpet the tops of all the buildings with gardens. Stacking greenhouses doesn't really help. Now, here's the rub: Where are you going to get energy conversion efficiency that high?
Wired's money is on the car of the future coming from NASA
When did NASA issue their IPO???
Seriously, when I read things like this, after what the "Space transportation system" has turned out to be, I can't help but see images of the Wired editorial office populated by characters from the movie "Idiocracy".
The way spectrum is currently utilized, it is like land. Although it doesn't have to be this way, reality dictates that until proper technologies for spectrum utilization are put into place, that spectrum be treated like land:
The users should rent it from the government that is enforcing their property rights over this natural resource.
This is a principle called "economic rent".
Milton Friedman has declared such taxation the "least distorting" kind of tax.
The way to set the rental agreement is to determine the liquidation value of the "land", and then charge a rent on it equal to the interest rate on short term US treasury instruments.
As with any rental agreement there would be other terms but the basic idea is that such resources enjoy liquidation value changes that are primarily a result of the economic environment -- meaning economic externalities drive the liquidation value -- and allocation of externalities is a social function.
Maybe if physicists started with a proper theory of time they'd figure out the reason our formalisms are so prone to over-serialize algorithmic descriptions. But then asking physicists for consilience with other disciplines is basically like asking a gang of tweakers to reflect on the bloody holes they're itching in their heads.
As for computer scientists... well... I really don't know that there is much going on of any value outside of Kolmogorov Complexity. Maybe something will come out of that that will resolve the issue while the physicists are trepanning themselves into theoretic epilepsy.
Slashdot Mirror
The solution to free market whack jobs -- really just private sector rent-seekers -- isn't public choice rent-seeking; it is to collect and then redistribute all economic rent -- that cannot be allocated to their true source as positive externalities (PE's like public domain technologies) -- equally to all segments of society. Anything else creates positive feedback loops that cut out positive sum innovators from the system.
At the end of my political activism that's what I had settled on as the solution and then wrote a white paper analyzing the consequences of the policy at the macro and microeconomic levels.
Non-innovative rent seekers are the problem, whether in the public or private sectors. When I say "the problem" I mean it -- they could quite possibly fry the biosphere.
The proximate cause was that despite there being an obvious direction in place subsequent to the space race (remember the Apollo program?) that could have been followed through to space industrialization -- the launch service industry did not enjoy the same protection from government competition that the satellite industry enjoyed:
It wasn't until 1990, when a coalition of grassroots groups across the country lobbied hard for 3 years, that similar legislation got passed for launch services.
The fact that Malthusian paradigm didn't precisely follow the Club of Rome's "Limits to Growth" model doesn't change the reality of the Malthusian paradigm given a fundamentally limited biosphere undergoing its largest extinction event in 60 million years. The Club of Rome merely added academic fashion to the urgency of the Malthusian situation still facing the biosphere. The 1970s was the right time to start the drive for space industrialization based on a private launch service industry. It didn't happen, the pioneering culture that founded the US is being replaced by government policy with less pioneering cultures and now we're all facing some increasingly obvious difficulties -- not just pioneer American stock -- and not just humans.
The cost of getting silicon into space from the lunar surface would be orders of magnitude less than launching from earth due not only to the much shallower gravity well but also due to the absence of atmosphere.
No beanstalk needed.
At worst a Dyneema Rotovator might be needed but probably not even that.
First, the bulk of the materials are manufactured in space from lunar raw material transported to orbital facilities so you don't need to land those facilities on the lunar surface, and you don't have to worry about g-loading the raw materials you are sending to the orbital facilities.
Second, you don't manufacture everything in space -- only bulky materials like solar cells, reflectors, structural members and perhaps klystrons. Only residual materials (raw and manufactured) are of terrestria
Until Shannon type experiments, involving humans doing next character predictions of enwik8, are performed, the bounds of enwik8's entropy range must remain unknown but is likely lower than 0.6 to 1.3. As such an experiment would be expensive, it is going to be difficult to say with any simple bpc measure when the Hutter Prize is breaching the threshold of AI. What the Hutter Prizes bpc metric gives us, however, is a clear measure of progress.
My apologies to the other members of the Hutter Prize Committee and the /. community for this error.
PS: Another area of concern raised by Mahoney is that enwik8, at 10e8 characters, is only as much verbal information as a 2 or 3 year old has encountered so although it is sufficient to demonstrate AI capabilities well beyond the current state of the art, his preference is for a much larger contest with fewer resource restrictions focusing on the 10e9 character enwik9 which is more likely to produce a the AI equivalent of an adult with encyclopedic knowledge.
What could be clearer than that prediction is relevant to AI? ML gives predictions and EM gives decisions based on predictions. Maximum compression of observations gives optimal models for prediction as you yourself have asserted, however old-hat this understanding of compression is. It all seems so straight forward if you accept, as you say you do, the relationship between compression and optimal prediction. I don't understand what your issue with the Hutter Prize is.
The Hutter Prize submissions consist of compressed corpus plus decompressor. It does not specify the method used, quite deliberately. In practice, it will probably be a combination of human insight augmented by computation that will produce the compressions. In the early stages, it should not be surprising that computer programs without a lot of a priori knowledge are squeezing out redundancy.
As for "optimality", you aren't thinking about things at the right level. If you have an unknown environment the fact that it is unknown means you are making educated guesses as to maximum likelihoods, or "expectation maximization". No -- of course those guesses are very unlikely (approaching zero) to be correct 100% of the time.
Thanks for correcting that error of mine.
Are you saying NLP model perplexity has never proven to be very useful in evaluating AI systems? Really? How are you measuring utility?
Take a random bit string of some reasonably large length -- say 1 million bits. Answer the question: What is the shortest program that can output that bit string?
It's been proven you can't compute that optimal program as the output of another program.
Human text is not the same as a random bit string, however much we might joke about it.
The value of the Hutter Prize is the epistemological implications of the compressed human knowledge: What constructs/concepts did it reify in order to achieve better compression of human knowledge and how much did they contribute? That tells us a lot about the human conversation we can't get any other way.
The value of the Hutter Prize isn't in the use of Hutter's theory to build AI. The value of the Hutter Prize is in the use of compression ratio to provide a figure of merit for AI that is very good. That value has been proven in various ways -- mathematically and practically.
Connectionist models are models. Any model needs to be interpreted to be understood.
PAQ8HP12 -7 enwik8.paq8hp12 enwik8
and moving the enwik8 archive to the parent directory:
./PAQ8HP12 -7 enwik8.paq8hp12
. ..
../enwik8/enwik8
JamesBowery@oldatlantis ~/hutter/paq8hp12
$ time
100000000 enwik8: extracted
16381959 -> 100000000 (1.3106 bpc) in 22398.08 sec (4.465 KB/sec), 941315 Kb
real 373m19.379s
user 0m0.031s
sys 0m0.030s
JamesBowery@oldatlantis ~/hutter/paq8hp12
$ ls -al
total 114216
drwxrwxrwx+ 2 JamesBowery None 0 Jul 1 00:43
drwxrwxrwx+ 6 JamesBowery None 0 Jun 30 14:59
-rwxrwxrwx 1 JamesBowery None 99696 May 14 09:16 PAQ8HP12.EXE
-rwxrwxrwx 1 JamesBowery None 100000000 Jul 1 06:54 enwik8
-rwxrwxrwx 1 JamesBowery None 16381959 Jun 30 21:09 enwik8.paq8hp12
-rwxrwxrwx 1 JamesBowery None 465211 Jul 1 00:43 temp_HKCC_dict1.dic
JamesBowery@oldatlantis ~/hutter/paq8hp12
$ diff enwik8
JamesBowery@oldatlantis ~/hutter/paq8hp12
$
Try this.
Actually, the size of the program (decompressor) binary is 99,696 bytes, and it is the binary size that is included in the prize calculation.
paq8hp12 uses a neural network, ie: it has a connectionist component.
Actuallly this is quite insightful except for one thing:
She doesn't actually cop to the fact that the "alpha of state", for which she works, is a wimp who is so chickenshit that "he" has to resort to black and Hispanic prison gangs to meet out "his" worst punishments on spouse-abusing white guys, and that "he" can't actually mate with the women who look to "him" for protection.
From Human Rights Watch's "No Escape: Male Rape in Prison":
brought to you by Carl's Jr.
Humans run at about 100 watts. Solar flux PEAK is about 1000 watts per square meter during peak sun, or about 100 watts average. In theory, you can feed a person on one square meter of solar flux. So maybe, just maybe, you can pull off these "vertical gardens" in urban areas if you carpet the tops of all the buildings with gardens. Stacking greenhouses doesn't really help. Now, here's the rub: Where are you going to get energy conversion efficiency that high?
When did NASA issue their IPO???
Seriously, when I read things like this, after what the "Space transportation system" has turned out to be, I can't help but see images of the Wired editorial office populated by characters from the movie "Idiocracy".
The users should rent it from the government that is enforcing their property rights over this natural resource.
This is a principle called "economic rent".
Milton Friedman has declared such taxation the "least distorting" kind of tax.
The way to set the rental agreement is to determine the liquidation value of the "land", and then charge a rent on it equal to the interest rate on short term US treasury instruments.
As with any rental agreement there would be other terms but the basic idea is that such resources enjoy liquidation value changes that are primarily a result of the economic environment -- meaning economic externalities drive the liquidation value -- and allocation of externalities is a social function.
As for computer scientists... well... I really don't know that there is much going on of any value outside of Kolmogorov Complexity. Maybe something will come out of that that will resolve the issue while the physicists are trepanning themselves into theoretic epilepsy.