Student Uncovers US Military Secrets

karthik_r085 writes "According to The Register, An Irish graduate student has uncovered words blacked-out of declassified US military documents using nothing more than a dictionary and text analysis software. Claire Whelan, a computer science student at Dublin City University was given the problems by her PhD supervisor as a diversion. David Naccache, a cryptographer with Gemplus, challenged her to discover the words missing from two documents: one was a memo to George Bush, and another concerned military modifications to civilian helicopters."

Comments

obvious solution

[Anonymous Coward]

Obviously, the next step the government will take is to require all documents be written in fixed-width fonts. Either that or they will require that all documents be converted into fixed-width before they are released for FFIA inquiries.

Don't see how this is a big threat.

Re:obvious solution

[zhenlin]

Variable width fonts makes this easier. Or not.
'iiii' probably has the same width as 'MM' in some variable width fonts.

On the other hand, fixed width fonts allows calculation of the exact amount of letters to fit in.

In any case, the 'official' font of the US Government was Courier New 12 for quite some time.

Re:obvious solution

[Feanturi]

Wouldn't fixed-width just make it easier to figure out how many letters were in the missing words?

Re:obvious solution

[Lord Ender]

Yes. But knowing 5 letters are in a word doesn't narrow it down nearly as much as knowing the word is 46 pixels long.

Re:obvious solution

[nacturation]

If you're talking about encrypted text, then your point is very valid. However, for English words you can get a much better result by using a dictionary to limit the number of words that fit the pattern.

How many 5 letter words are there in the English language? According to /usr/share/dict/words, there's 9987 words, from aalii to zymin. Compare that to how many combinations of letters add up to 60 pixels? If the letter "i" is 4 pixels -- 3 pixels for the letter, one pixel space after it -- then you *could* guess that the word is "iiiiiiiiiiiiiii". In fact, there's a hell of a lot more possibilities doing it the pixel way, but you can reduce this down by using a dictionary. "iiiiiiiiiiiiiii" isn't in the dictionary. You can also reject outright words that have impossible letter combinations. Three of any letter in a row can be rejected, Q followed by X can be rejected, etc. The rest you do a dictionary lookup to see if they exist.

It'd be an interesting exercise to perform. Luckily for the researcher, the word preceding the blacked out word was "an", which implies that the next word starts with a vowel. So that narrowed it down to only 7 potential words based on pixel length and dictionary lookup, and the one that seemed to work best was Egyptian. However, if all you knew was that it was an 8 letter word beginning with a vowel... you'd be looking at 6089 possibilities (again, according to /usr/share/dict/words and grep).

No real Secrets were harmed...

[LostCluster]

The student didn't actually solve for any real US secrets, because the documents she was using were already declassified. However, as an academic exercise this demonstrates that there's still information being conveyed in the typical black-out way of "redacting" certain words from documents.

And, since the information was known, we're sure that she did come up with the correct solutions.

This information isn't even blacked out!

[zogger]

On the contrary, the Project for A New American Century group, a coupla dozen high ranking neocons, CLEARLY outlined what they were going to do once they got in power. It's all on their publically available website. Some of it is in PDF downloads, but it's there. They planned to invade basically the oil producing nations of the middle east, and some others. They got in power, in charge,and wow, they invaded. They also said they needed a "pearl harbor" like event in advance to justify the invasion, and get the US people all enthused around it, and golly gee mother of all coincidences, that event occurred..

I mean, it's real, it's there, you can see the names, the documents, it's written clearly, and the mass controlled media won't hardly ever mention it. I've seen very brief mentions at the best. I have yet to meet anyone in meatspace who has ever heard of them or their documents though. Wonder why that is? And I know it's been posted on slashdot several times, by various people, as well as on literally thousands of other forums and blogs. Radio talk show hosts all over have been clued in, but only a small handful even bother to acknowledge it, let alone come to the obvious conclusions looking at it. Journalists by the thousands have been clued in, yet there's a severe lack of coverage by most of the big names out there.

No I don't blame democrats, or republicans, I blame the US people in general for being so unbelievably stupid and naieve and un-caring for this disaster. We are a nation of sports and entertainment addicts more than anything else. No one gives a crap. They are taught from the time they are toddlers to NOT give a crap. They are taught to parrot one of two party lines that are always essentially complete lies, and to be happy with that, and to never go further than to keep corralled into one of those two parties and to swallow down the 6 o clock news pablum. So they do it, brainwashing since being able to understand human speech is quite effective apparently. They simply refuse to learn from history,and they refuse to acknowledge reality, and that's why we generation after generation keep getting hosed. You are force fed you are either a liberal-democrat, or a conservative-republican and that is SUCH A LOAD OF CRAP. I am so amazed people keep falling into that trap.

oh well...

Re:Ingenious...

[Tony-A]

All you need to do is use a fixed-width font, and then all the decrypter would be able to find out is how many letters in the given word

"In January, the State Department required that its documents use a more modern font, Times New Roman, instead of Courier"

That my friends is what is know as "progress".

Solutions . . .

[Dausha]

Well, there are two solutions to this method of cracking. The first is never release classified documents. However, this does not work well in a free and open society.

Nowdays, most, if not all, classified documents are created electronically. Perhaps the source document should be kept in an archive. When it is declassified, they just delete the text needed to lower the classification, or maybe replace the text with a few '#' to show were text was missing (but never a one-for-one character replacement). Then the released document is a little harder to crack.

Re:Ingenious...

[wideBlueSkies]

Why do they have to release the original documents with original sections blocked out?

Why not just release retyped docs with placeholders for blocked out sections.

For instance:

Original:
It seems that the president wishes us to bomb the hell out of iraq. He's pissed off that saddam wanted to kill his daddy. also there's the issues of controlling the oil flow, and protecting israel. god forbid anyone thinks that the israilies are the biggest part of the problem out there.

Released with blocks:
It seems that XXXXXXXXXXXXX wishes us to bomb the hell out of iraq. He's XXXXXXXXXX that saddam wanted to XXXXXXXXXXXXXX. also there's the XXXXXXXXXXXXXXXXXXXXX the XXXXXXXX, and XXXXXXXXXXXXXXXXX. god forbid XXXXXX thinks that XXXXXXXXXXXXX are the XXXXXXXXXXXX XX XXX XXXXXXXXXXXXXXXXX.

My proposal:
It seems that |CLASSIFIED| wishes us to bomb the hell out of iraq. He's |CLASSIFIED| that saddam wanted to |CLASSIFIED|. also there's the |CLASSIFIED| the |CLASSIFIED|, and |CLASSIFIED|. god forbid |CLASSIFIED| thinks that |CLASSIFIED| are the |CLASSIFIED| |CLASSIFIED| |CLASSIFIED| |CLASSIFIED|.

I think this simple step would go a long way towards soving the problem. The process could probably even be automated somewhat by using some type of OCR software on the original blocked out documents.

Is there some law against this? Like that TPTB have to release the original doc?

wbs.

Then maybe I'm stupid too

[The Tyro]

but I've been looking at that site, and haven't found too much alarming stuff, speaking as a life-long, well-traveled american. Their principles are clearly stated... do you think other nations don't have groups strategically plotting their future course? The difference in America is that you can go on a website, see the players, and read what they have to say. My cursory perusal didn't turn up much objectionable material. Can you point me to some specific papers and/or citations? I'm genuinely curious.

I also never found a position paper advocating a conquest of the middle east and theft of their oil. For pity sake, americans want to buy the mideast's oil, not seize it (if the US military seriously wanted to take it, there'd be little to stop them... but that's not how americans see themselves on the world stage).

One can attempt to argue whether American prominence is good for the world... but I would challenge you to put forth a better choice (China? Russia? Iran?). It's the nature of world affairs for the dominant powers to emerge... I would also submit to you that "the United Nations" is not an appropriate alternative... the UN's lack of action has resulted in much pain and suffering around the world, and their ludicrous committee appointments (Sudan and Cuba in the human rights group, for example) bring the credibility of that body into serious doubt.

The US is the "big boy" on the block, and an easy target for derision... but on the whole I'd consider the US a force for good in the world... our track record in confronting various evils, and settling/winning various wars and conflicts speaks for itself.

Granted, whether we have the political will to make Iraq work out remains to be seen. We certainly have the physical ability, but unfortunately that's not america's achilles heel... it's politics.

Re:Then maybe I'm stupid too

[demachina]

"The US is the "big boy" on the block, and an easy target for derision... but on the whole I'd consider the US a force for good in the world... our track record in confronting various evils, and settling/winning various wars and conflicts speaks for itself."

Keep telling yourself that. Its simply not true. Chat with the average person in Iran, Guatemala, Argentina, Chile, Haiti, Dominican Republic, the Phillipines.

The Phillipines endured a brutal, genocidal occupation by the U.S. from the end of the Spanish American war up to World War II. After World War II the U.S. backs the massively corrupt rule of the Marcos regime. I'll post this same link I post everytime an American says how good they've been to the world:

http://www.isp.nwu.edu/~fprefect/politics/timeli ne .html

Some parts of it are overdone and a stretch but it has all the names and dates for all the misery the U.S. has inflicted on the world in the last century which you can corroborate easily if you choose to not believe this source.

In 1953 Iranian Nationalists gained power at a time when the British were looting 88% of Iran's oil revenues. The Iranians demanded a more equitable deal and offered the British 25%. Blockeds and boycotts ensued. The British ran crying to the U.S. and Truman. Truman ignored them. When Eisenhower took power it happened the Dulles brothers, head of the CIA and Secretary of State were lawyers form Anglo-American oil. The Dulles brothers used the CIA to topple the Iranian government and installed the Shah of Iran, who was every bit as despotic as Saddam was as far as the secret police, torture and disappearing people went. Rather than giving the British their oil contracts back they were given to, you guessed it, American oil companies. The reason the U.S. Embassy in Tehran was seized was revenge for all the misery the U.S. inflicted on Iran under the Shah.

This also points out that the U.S. has in fact been using its military and intelligence power to win control of oil fields for American companies since World War II at least. The Japanese attacked Pearl Harbor because the U.S. was embargoing Japan's access to U.S. controlled oil fields in Indonesia.

To date control of oil fields has been primarily for the economic benefit of the seven sisters(the big oil companies formed from the break up of Standard Oil though there are a lot less than seven now thanks to mergers). They have immense influence in U.S. politics, especially on the Republican's. George H.W. Bush's main career before politics was at Zapata Oil which built off shore oil rigs and ships to do contract drilling for the big oil companies and many foreign governments. Its widely suspected Zapata was also a CIA front, since there ships tended to be parked just offshore of every hotspot in the world. Zapata is also a key factor in the closeness of the Bush family to the royal families in Kuwait and Saudi Arabia. Halliburton's oil operations closely resemble those of Zapata.

In the future as oil reserves start to run out strategic control of the oil fields will determine the economic winners and losers of this century at least until somebody comes to their senses and starts investing billions in developing alternative energy sources instead of fighting over the current fossil fuel sources.

China's oil consumption in particular is exploding at double digit annual percentage growth and its a contibutor to the current tight oil market. The Neocons are in fact looking ahead to when the day there isn't enough oil to meet demand. When that day comes they will look pretty smart when they have the U.S. military sitting in the middle of all the old oil fields in the Middle East and all the new ones in Central Asia. When that day comes some people will get the oil their economies need and some nations will go dark.

Re:Then maybe I'm stupid too

[demachina]

"Let's see, the U.S. freed the Phillipines from a Spanish tyranny and rather than subsuming the islands, the U.S. eventually set them off on their own."

The U.S. did subsume the Phillipines for about 90 years. It wasn't until 1986 when Marcos was toppled and 1992 when the U.S. removed its huge military bases that it achieved something resembling real sovereignty.

Apparently you've never read the history of the initial American occupation of the Phillipines. The U.S.replaced Spanish tyranny with American tyranny.

U.S. Brig. Gen. Jacob H. Smith: "I want no prisoners. I wish you to kill and burn, the more you kill and burn the better you will please me. I want all persons killed who are capable of bearing arms in actual hostilities against the United States."
Major Littleton W. T. Waller: How young?
Smith: Ten years and up.
--Exchange on October 1901, quote from the testimony at Smith's court martial by the New York Evening Journal (May 5, 1902). General Smith, a veteran of the Wounded Knee massacre, was popularly known as "Hell Roaring Jake" or "Howling Wilderness".

The civilian causalties as the U.S. fought the Phillipine insurgency was most probably in the hundreds of thousands.

"Most of the mideast prior to the actions of western oil companies were vaste wastelands traversed by ignorant nomads. The western oil companies discovered the oil, gave it value, and it then was stolen by the disgusting murderers that call themselves governments in the mideast."

Are you American, British or Israeli. Thank you for once again proving what an arrogant, imperialistic, bunch Westerners are. Some of the "murders that call themselves governments" are close friends of the Bush family and the best of allies of the U.S., the Saudi royal family, the Emir of Kuwait, etc. Either your respect the sovereignty of nations or you don't. If you think a western company can enter a country and take all its resources with little or no compensation to the country which owns the resources you are a blatant imperialist.

Mossadegh. The head of Iran the U.S. overthrew was Time "Man of the Year", fairly progressive, anti-communist and Truman wouldn't even consider overthrowing him, the Dulles brothers on the other hand could care less when there was a chance to seize control of Iran's oil for U.S. oil companies.

Not sure how well you are versed in history but Iraq sits on top of the cradle of civilization. There were great civilizations there when your Western ancestors were living in caves or sod huts and running around in animal skins.

The number system you use today, though possibly Indian in origin, was introduced to the West by Arabs. They have had rich civilizations, great empires, and some of the world's best scholars. There have been periods when Arab culture was far more advanced than Europe's.

Many of the misfortune's of the Arab world can be traced to military interventions from the West, including the Romans, the Crusades, British imperialism and now U.S. imperialism.

Re:Then maybe I'm stupid too

[hey!]

The US is the "big boy" on the block, and an easy target for derision... but on the whole I'd consider the US a force for good in the world... our track record in confronting various evils, and settling/winning various wars and conflicts speaks for itself.

When G W Bush was a candidate, his way of speaking used to set my teet on edge: kind of a rote, up and down sing-sing reading of the teleprompter. He doesn't do that since 9/11: he speaks with the true conviction of a man who is convinced he has a personal mission.

I liked the old way better. The new Bush is frightening.

I would love it if the US were a force for good in the world. But I dont' believe that as a nation we know enough about the rest of the world to decide what is good for them. Freedom; yes. Democracy: yes. I strongly beleive these are good for any people of any culture. However, we don't really understand people in a place like Iraq to effectively promote these ideas. We don't even have national memory of the way these ideas were used in the past as self righteous fig leaf for ruthless exercises in the application of power. Rhetoric that is inspiring to us only reminds them of bitter disappointments in the West going back to Sykes-Picot. The more stirring a project sounds to us, the more it will incite fear and revulsion on the people we plan to impose it upon.

So, I'm very disturbed by any kind of messianic program to drag the unenlightened into accepting our values. What is even worse than telling seductive lies is being seduced by them yourself. In the first case you discredit yourself. In the latter case you discredit yourself and your ideals.

I'd much prefer a policy which frankly pursued our national interests, but tried to do it in a modestly ethical way.

Re:Then maybe I'm stupid too

[demachina]

"They are now characterized by illiteracy, poverty, religious fanatism and government oppression. Western societies are characterized by high rates of literacy, greater wealth, more even distibution of wealth, rule of law, and rapid technological advance, a high degree of social mobility and individual social and economic freedom. As a result Arabs have become jealous and emittered, despiratley recounting ancient greatness to preserve a lingering pride in their failed civilizations."

Are you suggesting the U.S. is entering its decline?

I hate to break it to you but "illiteracy, poverty, religious fanaticism and government oppression" could pretty easily be used to describe the trend in the U.S. today though it certainly hasn't reached epidemic proportions in all categories yet. I'm pretty nervous with the the extent to which fundamentalist Christianity has inserted itself into the Bush administration. Everyone has their right to religious preference but they should be leaving it at the home and in the church when they enter government. The Founding Father emphasized the separation of church and state because many of them were well aware of religious persecution in Europe at the time.

As for you ramblings about first mover advantage I don't really see the point. All civilizations rise and fall. So will American and Western European civilization, fall that is. You seem pretty eager to condemn China to the dustbin of greatness but all indications are that today they are a juggernaut that will pass the U.S., E.U. and Japan in economic supremacy, at least, and in the not to distant future.

"From those facts you nonsensically conclude that Romans are partly accountable for Arab misfortune and the relative ascenadancy of the west. Europeans were ultimately defeated in the crusades by Arabs."

On this point I conceede and punt. Arab history is so complex and poorly understood by this Westerner I'll have to admit I have no clue how they reached the nadir they did in the early 20th century.

Re:WMD!!

[Anonymous Coward]

Oh, boo hoo. Just because a couple of idiots from South Armagh decide to drive a truck filled with fertiliser into Canary Wharf doesn't mean that they had *any* support from people in the Republic. Northern Irish terrorism is a particularly insular phenomenon which by and large has no greater connection with people in the Republic than it does with those in the mainland UK.

Take a representative sample of Irish people and ask them whether they want stronger economic and social ties with the United Kingdom or a reunited Ireland, and I can guarantee that over two-thirds will say they want better ties to the UK.

The idiots in the North are no more representative of Irish culture or political identity than those in the UK's BNP.

Re:WMD!!

[igbrown]

"I suggest the people who modded this comment funny go and look at what the Irish did to London in 1993."

I am not sure that the IRA == the Irish, any more than Al Qaeda == the Muslims.

Re:WMD!!

[Angus Prune]

And maybe they should look at what the English have done to the Irish for hundreds of years.

History cannot be examined in isolation.

Re:WMD!!

[Speare]

How sad is it when you read:

• Re: WMD!! (Score: 5, Funny)
  Next stop for her: Guantanemo Bay...

The government has already proven it will detain people just for what they know, without criminal charge, without provocation, without family access, without legal representation, without regard for international criticism, without regard for international laws and norms, without safeguards for personal safety, without justification or oversight by the courts.

I doubt the G goons will be sweeping up this particular researcher, but what small and subtle distinction really lies between this case and others? What shred of humanity protects her from the inhumanity of the Bush/Rumsfeld/Ashcroft three-ring circus? Oh, she has red hair and freckles? Alrighty then.

Completely Unimpressive

[bbagnall]

The title of this article sounds impressive, but the results are wishy-washy. It can only narrow down one missing word to maybe half a dozen possibilities. Who is to say the word is not North Korea instead of South Korea? And since most blackouts are several words long, it is not useful at all.

Reasons for Iraq invasion and who is behind it?

[kbahey]

One can enumerate the reasons for Iraq's invasion as follows:

• Establishing a precedence for preepmtive war. Now America has bybassed the UN, and global opposition to this unilateral action. If the will to build an empire arises, then it will be done without any regard to what the rest of the world think or say. You can read the following articles too:

  • The True Rationale? It's a Decade Old by James Mann, March 7, 2004

  • PBS had a good program before Iraq was invaded called the War Behind Closed Doors. You can watch the entire program in 30-60 minutes intervals:

  • Specially interesting is this page in the Project for the New American Century Statement of Principles where you can see who signed this document. Interesting to note that all of them are either now in the Pentagon (Rumsfeld, Wolfowitz, Feith, ...etc.), or are aids to Cheney (Libby, Abrams, ...etc.)

  • An overview of who is who in the neocons circle of power.

• Securing cheap oil. That is obvious. Bush's family history in oil makes that an easy one to figure.

• Complete Dad's job. The personal desire of G.W. Bush to continue where his father has left, to finish the job, and do better.

• The Israel Factor. Read the Israel connection, and how Zionism influences US foreign policy. If you take a look at the players in the PNAC above, and you will find them all staunch Zionists, whether Jews or Christians.

• Construction Contracts. The Infrastructure contracts for US corporations to rebuild Iraq is a lucrative business. Of course the Halliburton link has been reported several times (Cheney used to be its manager or director). The defence spending, plus the contracts should fuel the US economy for a while, or that is what they thought would happen.

The planning to invade Iraq was done before September 11, 2001 attacks, as ex-secretary Paul O'Neill has revealed

As many would notice, Bush is not running the show. Bush is the ideal front for such an operation. He thinks he is doing the right thing, and that God has to do something with it. You can see this PBS program The Jesus Factor.

There are two factions grappling for Bush's attention. The moderate pragmatics (Powell, Armitage), and the extremist ideologue (Cheney, his subordinates, Rumsfeld, his subordinates). Powell's position is almost identical to Shimon Peres when he was the Foreign Minister in the Sharon government, a rational pragmatic dove amid the ideologue extremist hawks.

What is funny and sad at the same time, is that the US Foreign policy is now crafted by the Pentagon and the Vice President in accordance with neocon think tanks like the PNAC. No role whatsover is given to the Department of State (where it should really belong), and Powell is merely a messenger (go tell the UN we are doing so and so, try to sell it diplomatically, ...etc.). No wonder Powell has said that he will not seek a second term even if Bush gets reelected (and repeated it a few weeks ago). Not nice thing being in his shoes I guess.

I would not go as far as to say that they intentionally planned and executed the September 11 thing. But the neocons sure did exp