> Slashdot does have biases. But when one of those hot-button topics comes up, I'll see a bunch of +5's on the minority side, as well.
Unfortunately there does seem to be a lot of "political" moderation going on on Slashdot, but we still get the phenomenon you observe because - contrary to popular opinion - there is not monolithic group-think block on Slashdot. There is a wide variety of opinions on technical, social, and political issues, and the supporters of each position are entitled to post or moderate just like all the others.
There may be biases arising from having one viewpoint outnumbered by another, but there doesn't seem to be a groupthink-style consensus on any topic. Hell, a large fraction of Slashdot replies are posts claiming that their parent poster was an idiot.
I once had a teacher who rode as an auxiliary law enforcement officer in the evening (volunteer work, IIRC), and he said that most domestic violence calls involved women beating up their men. Often as not, a true-to-comic-form frying pan on the head when them man comes in drunk.
> From the President's State of the Union Speech January 28, 2003. "Tonight I'm proposing $1.2 billion in research funding so that America can lead the world in developing clean, hydrogen-powered automobiles."
And how much unbudgeted money did he spend on the Near East this year?
> I can see how this could be done as a really big simultaneous equation where the coefficients are dummy values for verb/noun etc with a parallel set of equation based on the grammar rules (combination), then variance analysis to eliminate the typos. Pretty elementary.
If you think it's pretty elementary you should write it up and publish it, since doing so would make your name an instant household word in fields ranging from philology to computer science, and probably also harvest you a fine crop of honorary PhDs and cushy job offers.
The problem of inducing grammars from examples has been intensely studied, and about all we know about it is that it's hard. (For example, we have a theorem showing that it's impossible to learn an arbitrary Context Free Grammar from any finite number of example strings.) The way children learn their language's grammar is so baffling that intelligent people have seriously supposed that you're born with a grammar processor in your brain that already knows how to process any possible natural language grammar and a support module that helps you determine which grammar to use by inducing a small set of switch settings from the examples you hear.
For that matter, after several decades of research we are just now getting to the point where computers can reliably parse natural language sentences even when we already know the language, its grammar, and have a lookup dictionary for all the words in the language. Automatically determining the parse of a sentence where both the grammar and vocabulary are completely unknown is a phenomenally more difficult problem; I suspect it's impossible even in principle.
As to your suggestion, I'm curious how you're going to solve a system of simultaneous equations when the data you are working with doesn't actually express any equalities. (Stop for a minute and think how you eliminate variables in a system of simultaneous equations.) For that matter, I'm not even sure what your representations are supposed to be.
It almost sounds like you're wanting to try all possible combinations for the part of speech for each word, but the combinatorial explosion would eat your lunch (n^m solutions, for n parts of speech and m words, even assuming no words can play multiple roles). Perhaps worse, even if you could enumerate all the possibilities you wouldn't be able to tell which one was correct. Since you don't know the grammar in advance, and since natural language grammars can be remarkably different from each other, you simply wouldn't have any way of knowing which part-of-speech mappings resulted in grammatical sentences and which didn't.
And if by some chance you did guess the correct grammar, you still wouldn't have a clue what the words meant.
> then variance analysis to eliminate the typos
If that's possible for unknown languages, it should be easily applicable to known languages. Are you suggesting a methodology for automated proofreading, that would catch typos in manuscripts? Could it be embedded in the slashcode, to automagically correct the typos in our posts? This technology alone would make you rich, even without all the other stuff you would need for interpreting unknown languages.
> But when we have 100s of pages of internal consistency it is possible to seperate nouns and verbs, subjects and objects, and the grammar that links these up.
How? Can you give an example of when this has ever been done?
> But, a volume of self consistent language (even a made up one) of over a hundred pages of text with accompanying pictures should fall to statistical and linguistic analysis.
I doubt it. How many possible mappings are there between strings of characters and meanings? And even with plausible interpretations of the pictures (e.g., a herbarium), the number of things that might be said in that context is for all purposes unbounded:
xyz =?= "this soothes the throbbing toe"
xyz =?= "this is very poisonous" xyz =?= "this grows only in Ys" xyz =?= "I learned this from my grandmother"...
Surely it will never be deciphered if it is in an unknown language.
> Champolion cracked the Rosetta stone with much much less.
Actually, he had the benefit of a parallel text.
In the absence of a parallel text, this will only be decyphered the way Linear B was: after a rigorous analysis of the patterns in the text, and a much tighter context (essentially lists of <picture,name,number> tuples), it was noticed that some very obvious translations ("man" and "woman", or such) fit the inflectional pattern of a language historically spoken in the region where the texts were found, and that simple mapping could be extended to other obvious <picture,name> pairs without introducing inconsistencies.
I suppose it's possible that something similar could be done with the manuscript, but IMO only if there are some clearly labeled images that give tight enough a context to guess the specific word being used. And then some luck, because somebody has to recognize some language-specific patterns (such as the Greek masculine/feminine inflectional suffixes). And of course, more luck in what language it happens to be: Linear B might never have been deciphered if Greek didn't use gender-based patterns in its noun declensions.
If it happens to be written in some unknown language, IMO it will never be deciphered.
> In fact, the letters appearing at the beginnings and ends of words should be a random sampling from the frequency distribution of letters in the whole text; this may be the easiest metric to check.
Actually, the distribution of initial letters might be preserved, or at least mostly preserved. If the source text is written so that lines always begin with a new word, and the grille is always aligned with the start of a line, then what you read out of the grille will preserve the frequencies of word-initial letters. But if you read more than one "word" out of the grille before moving it, you will get a mixture of the true word-initial distribution plus the distribution of all the letters in the document. And if you don't always align the grille to the start of a line, all bets are off.
Off hand, I don't see any way that the distribution of word-final letters would be preserved. The first thing I would do to detect a hoax is compare that distribution to the distribution of all the letters in the document. If they are the same, then I would suspect the use of a grille or some other randomizer.
Those who read the article can take note of an interesting challenge: though Rugg has shown that it is possible to generate a high quality hoax using a Cardan grille, proving it to be a hoax may require producing a character grid that will actually generate large portions of the text. My question is, could that be done with a genetic algorithm, and are any Slashdotters up to the task?
Also, a few comments about formal analysis. Notice that if you took some arbitrary text, typeset it in a fixed-width font to force the characters into columns, and then skimmed it with a grille in order to generate a new text, you would automatically preserve such basic statistics as character frequency, including spaces and also punctuation if you used them in your grid. (Depending on how you applied the grille, you could actually be generating a simple permutation of the original text.) However, you would disrupt all the within-word correlations.
For example, in compound words derived from Latin there is a familiar pattern where ad C* ==> aCC* (where C is some arbitrary consonant), but that pattern would be completely obscured if the characters were read off a diagonal grille as shown in the photograph. You would still get the increased frequency for C, but not the common aCC pattern.
More subtly, there are some well known universals of syllable structure in natural languages, but those would be scrambled just as the aCC would be. You would have the right proportions of consonants and vowels, but not a realistic distribution within words.
Likewise, prefixes and suffixes would be scrambled. If it is a hoax generated by a Cardan grille, it should not have prefix/suffix patterns that occur commonly in many languages. (Ditto for suffixal inflections.) In fact, the letters appearing at the beginnings and ends of words should be a random sampling from the frequency distribution of letters in the whole text; this may be the easiest metric to check.
Also, by using spaces as characters in your grid you'd get the right proportion of spaces, and therefore the right average word length, but you would obscure any patterns in word length. Someone has already linked to studies of the word lengths in the manuscripts, but those assumed that the distribution of Latin word lengths word lengths would be preserved. However, only the average would be preserved. I suspect the distribution would be converted to a gaussian. Anyone got time for the experiment? (Notice that you may generate extra spaces with the grille, depending on how you use it. For example, what do you do when your grille starts running off the bottom of the page in your source text? Or, if your grille has 10 windows, do you transcribe to the first space and then move the grille, or do you transcribe everything in the grille and insert a "virtual" space for position 11? It looks to me like you might be able to generate the document's actual "word" lengths from Latin, given only some very basic assumptions.)
> "Total Recall" (1990). Miserable adaptation of a clever idea by PKD. I won't describe the movie, which you've probably already seen, but I'll describe the original story which you probably haven't read... (warning:spoilers - skip to the end of the paragraph)
Don't worry; the plot synopsis is so twisted it will erase your mind by the time you get to the end of it.
Slashdot Mirror
> Slashdot does have biases. But when one of those hot-button topics comes up, I'll see a bunch of +5's on the minority side, as well.
Unfortunately there does seem to be a lot of "political" moderation going on on Slashdot, but we still get the phenomenon you observe because - contrary to popular opinion - there is not monolithic group-think block on Slashdot. There is a wide variety of opinions on technical, social, and political issues, and the supporters of each position are entitled to post or moderate just like all the others.
There may be biases arising from having one viewpoint outnumbered by another, but there doesn't seem to be a groupthink-style consensus on any topic. Hell, a large fraction of Slashdot replies are posts claiming that their parent poster was an idiot.
> Ive been beaten shitless by a woman before
I once had a teacher who rode as an auxiliary law enforcement officer in the evening (volunteer work, IIRC), and he said that most domestic violence calls involved women beating up their men. Often as not, a true-to-comic-form frying pan on the head when them man comes in drunk.
> "fashial"?
Yes, didn't you know Fashism was named after Hitler's moustache?
> Other than that, I agree that a manned flight would have a higher success rate than a robotic one due to real-time correctional ability.
Can astronauts convert between metric and common units in their heads?
Isn't that what they call tatoos on your...
> It's just a New Year Worm - nothing much different
Happy New Worm!
> Seems like the worm must be "human-activated", a user must manually click the link received through MSN to download the worm
Oh, no problem then.
> Pearl Harbor was "infamy" because it was an act of war [embargo is economic, and therefore 'civil'] done before war was formally declared.
> Among other things, it was a war crime.
Too bad the USA didn't declare war on Iraq...
> From the President's State of the Union Speech January 28, 2003.
"Tonight I'm proposing $1.2 billion in research funding so that America can lead the world in developing clean, hydrogen-powered automobiles."
And how much unbudgeted money did he spend on the Near East this year?
> Your post just proves trolls aren't doing much tonight.
It's Trollish New Year tonight.
> Did they finally admit to knowledge of 007's actions?
Yeah, but it was just a long list of the girls he bonked. He never made time to do any actual spying.
> In 2002, I researched the COSMIC background
Yeah, lots of people do that in college... Usually with the help of LSD and stuff.
> I can see how this could be done as a really big simultaneous equation where the coefficients are dummy values for verb/noun etc with a parallel set of equation based on the grammar rules (combination), then variance analysis to eliminate the typos. Pretty elementary.
If you think it's pretty elementary you should write it up and publish it, since doing so would make your name an instant household word in fields ranging from philology to computer science, and probably also harvest you a fine crop of honorary PhDs and cushy job offers.
The problem of inducing grammars from examples has been intensely studied, and about all we know about it is that it's hard. (For example, we have a theorem showing that it's impossible to learn an arbitrary Context Free Grammar from any finite number of example strings.) The way children learn their language's grammar is so baffling that intelligent people have seriously supposed that you're born with a grammar processor in your brain that already knows how to process any possible natural language grammar and a support module that helps you determine which grammar to use by inducing a small set of switch settings from the examples you hear.
For that matter, after several decades of research we are just now getting to the point where computers can reliably parse natural language sentences even when we already know the language, its grammar, and have a lookup dictionary for all the words in the language. Automatically determining the parse of a sentence where both the grammar and vocabulary are completely unknown is a phenomenally more difficult problem; I suspect it's impossible even in principle.
As to your suggestion, I'm curious how you're going to solve a system of simultaneous equations when the data you are working with doesn't actually express any equalities. (Stop for a minute and think how you eliminate variables in a system of simultaneous equations.) For that matter, I'm not even sure what your representations are supposed to be.
It almost sounds like you're wanting to try all possible combinations for the part of speech for each word, but the combinatorial explosion would eat your lunch (n^m solutions, for n parts of speech and m words, even assuming no words can play multiple roles). Perhaps worse, even if you could enumerate all the possibilities you wouldn't be able to tell which one was correct. Since you don't know the grammar in advance, and since natural language grammars can be remarkably different from each other, you simply wouldn't have any way of knowing which part-of-speech mappings resulted in grammatical sentences and which didn't.
And if by some chance you did guess the correct grammar, you still wouldn't have a clue what the words meant.
> then variance analysis to eliminate the typos
If that's possible for unknown languages, it should be easily applicable to known languages. Are you suggesting a methodology for automated proofreading, that would catch typos in manuscripts? Could it be embedded in the slashcode, to automagically correct the typos in our posts? This technology alone would make you rich, even without all the other stuff you would need for interpreting unknown languages.
> But when we have 100s of pages of internal consistency it is possible to seperate nouns and verbs, subjects and objects, and the grammar that links these up.
How? Can you give an example of when this has ever been done?
> > contains pictures of unrecognizable flowers, naked nymphs
> obviously a troll
Fortunately the article didn't show the goatse illustration.
> Are you my boss, 'cause you sound like him...
PHB is easy to translate, since 99 times out of 100 anything they say translates to "Hurry up!".
Surely it will never be deciphered if it is in an unknown language.> But, a volume of self consistent language (even a made up one) of over a hundred pages of text with accompanying pictures should fall to statistical and linguistic analysis.
I doubt it. How many possible mappings are there between strings of characters and meanings? And even with plausible interpretations of the pictures (e.g., a herbarium), the number of things that might be said in that context is for all purposes unbounded:
> Champolion cracked the Rosetta stone with much much less.
Actually, he had the benefit of a parallel text.
In the absence of a parallel text, this will only be decyphered the way Linear B was: after a rigorous analysis of the patterns in the text, and a much tighter context (essentially lists of <picture,name,number> tuples), it was noticed that some very obvious translations ("man" and "woman", or such) fit the inflectional pattern of a language historically spoken in the region where the texts were found, and that simple mapping could be extended to other obvious <picture,name> pairs without introducing inconsistencies.
I suppose it's possible that something similar could be done with the manuscript, but IMO only if there are some clearly labeled images that give tight enough a context to guess the specific word being used. And then some luck, because somebody has to recognize some language-specific patterns (such as the Greek masculine/feminine inflectional suffixes). And of course, more luck in what language it happens to be: Linear B might never have been deciphered if Greek didn't use gender-based patterns in its noun declensions.
If it happens to be written in some unknown language, IMO it will never be deciphered.
> In fact, the letters appearing at the beginnings and ends of words should be a random sampling from the frequency distribution of letters in the whole text; this may be the easiest metric to check.
Actually, the distribution of initial letters might be preserved, or at least mostly preserved. If the source text is written so that lines always begin with a new word, and the grille is always aligned with the start of a line, then what you read out of the grille will preserve the frequencies of word-initial letters. But if you read more than one "word" out of the grille before moving it, you will get a mixture of the true word-initial distribution plus the distribution of all the letters in the document. And if you don't always align the grille to the start of a line, all bets are off.
Off hand, I don't see any way that the distribution of word-final letters would be preserved. The first thing I would do to detect a hoax is compare that distribution to the distribution of all the letters in the document. If they are the same, then I would suspect the use of a grille or some other randomizer.
Those who read the article can take note of an interesting challenge: though Rugg has shown that it is possible to generate a high quality hoax using a Cardan grille, proving it to be a hoax may require producing a character grid that will actually generate large portions of the text. My question is, could that be done with a genetic algorithm, and are any Slashdotters up to the task?
Also, a few comments about formal analysis. Notice that if you took some arbitrary text, typeset it in a fixed-width font to force the characters into columns, and then skimmed it with a grille in order to generate a new text, you would automatically preserve such basic statistics as character frequency, including spaces and also punctuation if you used them in your grid. (Depending on how you applied the grille, you could actually be generating a simple permutation of the original text.) However, you would disrupt all the within-word correlations.
For example, in compound words derived from Latin there is a familiar pattern where ad C* ==> aCC* (where C is some arbitrary consonant), but that pattern would be completely obscured if the characters were read off a diagonal grille as shown in the photograph. You would still get the increased frequency for C, but not the common aCC pattern.
More subtly, there are some well known universals of syllable structure in natural languages, but those would be scrambled just as the aCC would be. You would have the right proportions of consonants and vowels, but not a realistic distribution within words.
Likewise, prefixes and suffixes would be scrambled. If it is a hoax generated by a Cardan grille, it should not have prefix/suffix patterns that occur commonly in many languages. (Ditto for suffixal inflections.) In fact, the letters appearing at the beginnings and ends of words should be a random sampling from the frequency distribution of letters in the whole text; this may be the easiest metric to check.
Also, by using spaces as characters in your grid you'd get the right proportion of spaces, and therefore the right average word length, but you would obscure any patterns in word length. Someone has already linked to studies of the word lengths in the manuscripts, but those assumed that the distribution of Latin word lengths word lengths would be preserved. However, only the average would be preserved. I suspect the distribution would be converted to a gaussian. Anyone got time for the experiment? (Notice that you may generate extra spaces with the grille, depending on how you use it. For example, what do you do when your grille starts running off the bottom of the page in your source text? Or, if your grille has 10 windows, do you transcribe to the first space and then move the grille, or do you transcribe everything in the grille and insert a "virtual" space for position 11? It looks to me like you might be able to generate the document's actual "word" lengths from Latin, given only some very basic assumptions.)
> before the atomic clock the Earth was always on time.
Maybe we should set it 5 minutes fast, to allow a little extra time.
> "Total Recall" (1990). Miserable adaptation of a clever idea by PKD. I won't describe the movie, which you've probably already seen, but I'll describe the original story which you probably haven't read
Don't worry; the plot synopsis is so twisted it will erase your mind by the time you get to the end of it.
> Unless I'm misinformed about that point, it seems like you would need an enormous audit trail on the database.
They do track it, by subscriber ID if a subscriber is logged in when a change is made, otherwise by IP address.
For each article you can pull up a history, both as diffs and as full versions of the article. Pretty slick.
> Till they do that they will not get any money from me.
Thanks. When they're back up, I'll mention you in the article on 'Scrooge'.
> This is the beginning of a growth period for Microsoft that is on a whole different scale than the last one.
No, I don't want to buy your MSFT.