Slashdot Mirror
Student Uncovers US Military Secrets
karthik_r085 writes "According to The Register, An Irish graduate student has uncovered words blacked-out of declassified US military documents using nothing more than a dictionary and text analysis software. Claire Whelan, a computer science student at Dublin City University was given the problems by her PhD supervisor as a diversion. David Naccache, a cryptographer with Gemplus, challenged her to discover the words missing from two documents: one was a memo to George Bush, and another concerned military modifications to civilian helicopters."
Oh OH, i can see it coming already, text analysis and dictionary software declared as Weapons of Mass Destruction! That, and Ireland is going to become the next member of the "axis of evil"
This is why I don't work for an intelligence agency. On the other hand, I'm still probably better qualified than people who think blacking out a few words in a document strips them of contextual information...
Ph-nglui mglw'nafh Gates M'dna wgah'nagl fhtagn.
...said
"Please please please let the army attack Iraq"
Apparently the word that was blacked out was please.
Pretty funny, but luckily she's from Ireland. If an American did this they'd probably receive a visit from some intelligence goons in short order.
Can gov stop her research on National Security grounds.
An example of the program in use.
G.W Bush is the ____________ of the United States of America.
After the program
G.W Bush is the idiot of the United States of America.
Obviously, the next step the government will take is to require all documents be written in fixed-width fonts. Either that or they will require that all documents be converted into fixed-width before they are released for FFIA inquiries.
Don't see how this is a big threat.
The Monde (famous French newspaper) published an article on the story a few days ago. An English translation can be found here.
The student didn't actually solve for any real US secrets, because the documents she was using were already declassified. However, as an academic exercise this demonstrates that there's still information being conveyed in the typical black-out way of "redacting" certain words from documents.
And, since the information was known, we're sure that she did come up with the correct solutions.
The other way to get around this problem would be to do the blackouts against a digital version of the document, so that the words are all replaced with blocks of equal size without revealing any information about how long the oriignal words were.
Damn, with our nation in the state it is today? She'd be goddamn lucky to get ONLY a visit. Sad but true :(
This is a classic example of security through obscurity.. And how it fails miserably.
This statement is false.
Perhaps the US government should start using text message lingo in their memos.
"An Egyptian Islamic Jihad (EIJ) operative told an Egyptian srvic @ d sAm tym dat bn l@n wz plnin 2 exploit d operatives acces 2 d us 2 mount a terrorist strike"*
Could make decoding sensitive documents much more difficult and at the same time provide jobs for teenage cryptologists.
*lingo courtesy of transl8it.com
Change the length of the blacked out portion to some standard generic length to avoid disclosing the word length? Then you could only use context.
Or if you wanted to be really sneaky, randomize the length of the blacked out box, to spur wild goose chases.
So...can they now tell us how REALLY killed Kennedy?
What did the documents reveal??
I think you'd be surprized how much irrelevant 'intelligence' ends up classified. Often, it's stuff which is already public (although not always general) knowledge but which the administration wants to deny.
A lot of ass-covering, basically.
But it gets even stranger. For instance the case of the de-classified CIA documents relating to the Pinochet dictatorship in Chile. (Whups, now there's a piece of flamebait..)
Anyway, a bunch of these documents have been re-classified by the current administration, apparently to hide such disturbing secrets like what Señor Pinochet's favorite drink was. (Scotch)
Before they release it they should convert the blacked out parts to 1337 speak...
Everybody dies frustrated and sad and that is beautiful
Nuculer.
Dictionary-based approaches seem to miss this one for some reason.
If people knew how easy it was to "break" simple means of protection, we'd see far less in the media.
If you film a person in backlight, his face will be dark when you see him on tv. Cranking up the contrast (in most cases, just the contrast on the tv will do) shows the face clearly.
If you pixelize the face of a person, he's not recognizeable. But unless he stands completely still, his movements will give enough info to calculate the originating pixels after a couple of minutes.
If you apply a standard mixer filter to a persons voice, it sounds dark and unrecognizeable... Until you run the reverse algorithm.
If you black out sentences with a marker, it's often just a question of holding the paper up agains the light to read it.
I never understood this behaviour anyway. Why show a person on TV that obvoiously not want to be recoznized (however carefully concealed by the production)?
As for documents - I'm pretty sure most documents are available electronically. Why not just delete the stuff you don't want people to see?
I personally think she discovered it while her computer was infected with the Irish Virus
MEMORANDUM TO ALL EMPLOYEES OF THE DHS AND US INTELLIGENCE SERVICES:
So as to counter the terrorists' latest methods for conducting espionage against our great nation, all official documents will now be composed in a combination of Wingdings 3 and MS Comic Sans.
Sincerely, The Management
But one thing to consider is whether redaction software does this at present. I don't believe that there is any such software. This story is informative in that it reveals that current redaction techniques may be inadequate.
www.timcoleman.com is a total waste of your time. Never go there.
As if that would work, only 1 in 10 RTFA to begin with.
quote of memo to bush from the article:
"An Egyptian Islamic Jihad (EIJ) operative told an XXXXXXXX service at the same time that Bin Ladin was planning to exploit the operative's access to the US to mount a terrorist strike."
and from the article itself:
"This eliminated all but seven words: Ugandan, Ukrainian, Egyptian, uninvited, incursive, indebted and unofficial. Naccache plumped for Egyptian, in this case."
AH-HAH!
so an egyptian operative told an *egyptian* service....
man this is some tricky work! uncovering covert secrets for sure!
seriously though the technique is pretty awesome
.. already out there now. Like I'd like to see a lot of the Black Vault's thousands of documents translated, just for one interesting example, one of many. woo hoo this is cool!
A fixed-width font (like courier) uses the same width for all characters. A document printed in a fixed-width font would make the process easier, because you would know with certainty how many letters fit into a black box.
If you read the article, the seven words that were found to be a possible fit range from seven to ten characters, implying that the document was printed in a variable width font.
What if the blacked out word is not in the dictionary? Most of these blacked out things are very likely names or places, things that could not be so easily brainstormed or listed.
...would have been if the censored bits were revealed by running the document through the spelling and grammar check in Word!
Student Uncovers Military Secrets
What's wrong with that headline? She is a PhD student, she was able to deduce what properly lay under the black marks, and the uncovered material was classified, probably at a fairly high level.
they won't know to mod this down
The threat -- purported to come from a then- and still-unknown group calling itself the "Group of the Martyr Ebenezer Scrooge" -- was contained in a classified compilation of intelligence on possible terrorist attacks produced by the CIA (news - web sites) in late 1974, according to the documents.
"A new organization of uncertain makeup using the name 'Group of the Martyr Ebenezer Scrooge' plans to sabotage the annual courier flight of the Government of the North Pole," the CIA said in its December 17, 1974 "Weekly Situation Report on International Terrorism."
"Prime Minister and Chief Courier S. Claus has been notified and security precautions are being coordinated worldwide by the CCCT working group," it said, identifying the night of December 24-25, 1974 as the date for the planned "sabotage."
It was not clear whether the CIA had learned of a prank threat to Santa, or if the analysts compiling the report had inserted it as a holiday joke.
Along with the threat to Santa, the situation report included deadly serious incidents and warnings including intelligence detailing potential terrorist attacks in the Middle East, possible bombs at the British embassy in Buenos Aires and a plane hijacking.
Despite the dubious nature of the threat to Father Christmas, the CIA blacked out all references to it when the situation report was declassified in 1999, according to the documents.
The documents, which include the original report as well as the redacted version released to the public in 1999, were released by the National Security Archive as part of its campaign against the "overclassification" of government files.
"The CIAs secret Santa leads the archives lengthy compilation of declassified documents that illustrate the arbitrary and capricious decision making that all too often characterizes the US governments national security secrecy system," it said in a statement.
The National Security Archive, part of the George Washington University in the US capital, is a private research group that seeks the declassification of government documents through the Freedom of Information Act for historical purposes.
The Cheese Stands Alone.
On the contrary, the Project for A New American Century group, a coupla dozen high ranking neocons, CLEARLY outlined what they were going to do once they got in power. It's all on their publically available website. Some of it is in PDF downloads, but it's there. They planned to invade basically the oil producing nations of the middle east, and some others. They got in power, in charge,and wow, they invaded. They also said they needed a "pearl harbor" like event in advance to justify the invasion, and get the US people all enthused around it, and golly gee mother of all coincidences, that event occurred..
I mean, it's real, it's there, you can see the names, the documents, it's written clearly, and the mass controlled media won't hardly ever mention it. I've seen very brief mentions at the best. I have yet to meet anyone in meatspace who has ever heard of them or their documents though. Wonder why that is? And I know it's been posted on slashdot several times, by various people, as well as on literally thousands of other forums and blogs. Radio talk show hosts all over have been clued in, but only a small handful even bother to acknowledge it, let alone come to the obvious conclusions looking at it. Journalists by the thousands have been clued in, yet there's a severe lack of coverage by most of the big names out there.
No I don't blame democrats, or republicans, I blame the US people in general for being so unbelievably stupid and naieve and un-caring for this disaster. We are a nation of sports and entertainment addicts more than anything else. No one gives a crap. They are taught from the time they are toddlers to NOT give a crap. They are taught to parrot one of two party lines that are always essentially complete lies, and to be happy with that, and to never go further than to keep corralled into one of those two parties and to swallow down the 6 o clock news pablum. So they do it, brainwashing since being able to understand human speech is quite effective apparently. They simply refuse to learn from history,and they refuse to acknowledge reality, and that's why we generation after generation keep getting hosed. You are force fed you are either a liberal-democrat, or a conservative-republican and that is SUCH A LOAD OF CRAP. I am so amazed people keep falling into that trap.
oh well...
She didn't get these US Military secrects off of a BDSM site with pictures of women dragging men around on a leash did she?
Ooops, never mind
Steve
Like the US 'harboured' Timothy McVeigh, or for that matter the September 11 terrorists? The fact that there are terrorists in the country does not mean that they are 'harboured'. I'd like to see *recent* complaints from the British authorities or any other source that the Irish government is actively 'harbouring' terrorists, or not doing all it can against them, thank you very much.
If your definition does not require government support or acquiescence, but you are just pointing out that there are terrorist suspects living openly in Ireland, well we have these things called evidence and due process which in this country at least are required before people can be locked up (less so in the US I believe since the Patriot Act, Guatanamo Bay, etc.) Unfortunately there is not always sufficient evidence to obtain convictions against such suspects.
Besides, by that definition there would be *far* more terrorists being 'harboured' in Northern Ireland, which is British terrority last time I looked. Ipso facto, the British government is harbouring terrorists that kill its very own citizens. Sheesh.
I don' think that would work. Didn't SCO already try this already?
$cat
One way to solve this problem, of course, is to develop a font that constantly varies the size and type so that your document ends up looking like a ransom that's been clipped and pasted from a newspaper.
One nice thing about being paranoid, you're never bored.
___
It's the end of my comment as I know it and I feel fine.
Well, there are two solutions to this method of cracking. The first is never release classified documents. However, this does not work well in a free and open society.
Nowdays, most, if not all, classified documents are created electronically. Perhaps the source document should be kept in an archive. When it is declassified, they just delete the text needed to lower the classification, or maybe replace the text with a few '#' to show were text was missing (but never a one-for-one character replacement). Then the released document is a little harder to crack.
What those who want activist courts fear is rule by the people.
Has anyone any photos of this geek girl? Yes, I tried Google images, but I don't think she looks like a puppy.
8 of 13 people found this answer helpful. Did you?
The Memo Went like this:
URGENT: MSG from GEORGE W. BUSH
TO: JOINT CHIEFS OF STAFF
1. ATTACK IRAQ
2. ____???____
3. PROFIT!!!
Claire has finally revealed the second step!
Read the article to find out.
Sorry, but in at least one of the cited examples, the methodology used requires an assumption that is false.
The proposed method depends on the calculated length of the missing word(s).
I believe that the "memo to George Bush" is the now infamous PDB of 8/6/03 (it was released in a PDF format). In this, the actual letters in the missing words were changed to nonsense characters (including non-alphanumeric symbiols) before the black box was drawn in. So the spaced taken up by the "redactions" have nothing to do (except by chance) with the length of the original words.
Sorry. Try again.
but I've been looking at that site, and haven't found too much alarming stuff, speaking as a life-long, well-traveled american. Their principles are clearly stated... do you think other nations don't have groups strategically plotting their future course? The difference in America is that you can go on a website, see the players, and read what they have to say. My cursory perusal didn't turn up much objectionable material. Can you point me to some specific papers and/or citations? I'm genuinely curious.
I also never found a position paper advocating a conquest of the middle east and theft of their oil. For pity sake, americans want to buy the mideast's oil, not seize it (if the US military seriously wanted to take it, there'd be little to stop them... but that's not how americans see themselves on the world stage).
One can attempt to argue whether American prominence is good for the world... but I would challenge you to put forth a better choice (China? Russia? Iran?). It's the nature of world affairs for the dominant powers to emerge... I would also submit to you that "the United Nations" is not an appropriate alternative... the UN's lack of action has resulted in much pain and suffering around the world, and their ludicrous committee appointments (Sudan and Cuba in the human rights group, for example) bring the credibility of that body into serious doubt.
The US is the "big boy" on the block, and an easy target for derision... but on the whole I'd consider the US a force for good in the world... our track record in confronting various evils, and settling/winning various wars and conflicts speaks for itself.
Granted, whether we have the political will to make Iraq work out remains to be seen. We certainly have the physical ability, but unfortunately that's not america's achilles heel... it's politics.
Even if a man chops off your hand with a sword, you still have two nice, sharp bones to stick in his eyes.
Its ok we can solve this by arresting the student and banning any software that does this. Just like we solved the Iraqi abuse problem by taking their cameras away, and how we solved the Berg murder by making sure no news outlet would publish or link to the video, and how we solved the terrorists hi-jacking planes and crashing them problem with iris and finger scanning, (so now they can still get on the plane, but when they've crashed it we will know who did it and not to let them on next time). Or maybe its more like how the CD copy-protection system being defeated by the shift-key problem was defeated by threatening the student under the DMCA! or could it be how the drug problem was totally solved by throwing half the population in jail? [insert something about DRM solving everything and letting governments send sensitive documents in full without having to worry about someone reading the bad words] great, so i guess we can bomb for peace and fuck for virginity after all :)
This comment does not represent the views or opinions of the user.
What I'm talking about here is information which is already public.
The info in the case-in-point, for example:
The Egyptian government cooperates with American intelligence agencies in monitoring and combating militant islamists.
This is a well-estabished fact. There are both former officials and former islamists who have publicly testified to that. Everybody knows.
But what is the point here? It's not to keep that info away from terrorists for sure, they know already.
The title of this article sounds impressive, but the results are wishy-washy. It can only narrow down one missing word to maybe half a dozen possibilities. Who is to say the word is not North Korea instead of South Korea? And since most blackouts are several words long, it is not useful at all.
Also, considering the political state of Northern Ireland (which is part of Great Britain, etc.) last year, most of these terrorist groups have just become glorified drug gangs, since a major way they go their money was through the drug cartel, now with a cease fire, there is more infighting and killing between the protestant militant groups over drug issues, than between the catholic and protestant groups.
Are those documents redacted in the final photocopy, or are they redacted by hand (very expensive, but they're spending our money, after all)?
There are two simple solutions that go beyond and below high technology.
Unless they crank down the brightness as far as possible, most photocopiers put down a varying amount of toner to paper. A cloth soaked in, say, spirit solvents, when wiped across the page, will expose part, if not all, of the text. Similarly, this can be done with most magic marker inks.
Just because you can mod me down, doesn't mean you're right. Shoes for industry!
One can enumerate the reasons for Iraq's invasion as follows:
Establishing a precedence for preepmtive war. Now America has bybassed the UN, and global opposition to this unilateral action. If the will to build an empire arises, then it will be done without any regard to what the rest of the world think or say. You can read the following articles too:
The True Rationale? It's a Decade Old by James Mann, March 7, 2004
PBS had a good program before Iraq was invaded called the War Behind Closed Doors. You can watch the entire program in 30-60 minutes intervals:
Specially interesting is this page in the Project for the New American Century Statement of Principles where you can see who signed this document. Interesting to note that all of them are either now in the Pentagon (Rumsfeld, Wolfowitz, Feith, ...etc.), or are
aids to Cheney (Libby, Abrams, ...etc.)
An overview of who is who in the neocons circle of power.
Securing cheap oil. That is obvious. Bush's family history in oil makes that an easy one to figure.
Complete Dad's job. The personal desire of G.W. Bush to continue where his father has left, to finish the job, and do better.
The Israel Factor. Read the Israel connection, and how Zionism influences US foreign policy. If you take a look at the players in the PNAC above, and you will find them all staunch Zionists, whether Jews or Christians.
Construction Contracts. The Infrastructure contracts for US corporations to rebuild Iraq is a lucrative business. Of course the Halliburton link has been reported several times (Cheney used to be its manager or director). The defence spending, plus the contracts should fuel the US economy for a while, or that is what they thought would happen.
The planning to invade Iraq was done before September 11, 2001 attacks, as ex-secretary Paul O'Neill has revealed
As many would notice, Bush is not running the show. Bush is the ideal front for such an operation. He thinks he is doing the right thing, and that God has to do something with it. You can see this PBS program The Jesus Factor.
There are two factions grappling for Bush's attention. The moderate pragmatics (Powell, Armitage), and the extremist ideologue (Cheney, his subordinates, Rumsfeld, his subordinates). Powell's position is almost identical to Shimon Peres when he was the Foreign Minister in the Sharon government, a rational pragmatic dove amid the ideologue extremist hawks.
What is funny and sad at the same time, is that the US Foreign policy is now crafted by the Pentagon and the Vice President in accordance with neocon think tanks like the PNAC. No role whatsover is given to the Department of State (where it should really belong), and Powell is merely a messenger (go tell the UN we are doing so and so, try to sell it diplomatically, ...etc.). No wonder Powell has said that he will not seek a second term even if Bush gets reelected (and repeated it a few weeks ago). Not nice thing being in his shoes I guess.
I would not go as far as to say that they intentionally planned and executed the September 11 thing. But the neocons sure did exp
2bits.com, Inc: Drupal, WordPress, and LAMP performance tuning.
Fairly easy.
Guys at CIA were having some fun around Christmas time. When it came time to release the documents, they decided they'd rather not look like they were assing around on government time, so redacted it.
That Jesus Christ guy is getting some terrible lag... it took him 3 days to respawn! -NJ CoolBreeze
During the reign of Pinochet, writer Ariel Dorfman used to convey the extent of the official censorship of his articles by incorporating the censored sections as blacked-out text and photos, with the understanding that people could fill in the blanks for themselves based on the surrounding text, knowing where the blanks were.
What's left out is as significant as what is included.
... especially the famous 19-minute expletive.
Good thing Ned Flanders wasn't around.
in the US, 50% don't even vote, and in the popular vote it was an almost even split, so only 25% of eligible adult voters elected bush.
Of course, we have an electoral college, they actually decide. It varies state to state how they do that though.
I support "no professional politicians". Single terms for every elected office. No more than 10 years total government service or involvement, and no pensions whatsoever. Medical and survivors bereavement benefits for veterans, or people in current employ only. Limit campaign contributions to 100$, from individuals only, no soft money from corporations, no lobbying gifts, trips, speakers fees, etc. Let's call bribes "bribes" and finalize outlawing them. It's not a free speech issue.
And stuff like that there, along those lines.
Basically, turn government back to being a peoples government, and not a separate class of connected washington insiders.
How to get people to vote? Easiest way,not my idea, but, I heard this before and it's really cool, make income tax deadline day be the day before the main elections, instead of april 15th. You'll get much higher voter participation, no doubt about it.. We could also declare voting day a national holiday, so no one has to choose between going to work or voting, and make it a full 24 hour vote period. It SHOULD be patriotic to vote, and it SHOULD make a difference. Maybe if we even had runoffs instead of just any number majority wins, call it you need 2/3rds to win, like a supermajority in congress, it would help, and having a ranking system on the candidates, with a zero being a legitmate number. There's lots of possibilities.
People are still "afraid" to vote third parties or independents, they got brainwashed into that "lookout! you'll 'waste' your vote" meme, and you hear party activists from both the democrats and the republicans saying that. Don't "waste" your vote. Phooie. They just want to keep a lock on the process, like it's written someplace that we can only have democrats or republicans, forever and ever, like it's the law or something. To me, a vote not cast is the only wasted vote. I've been voting for decades, hardly ever got my guy in, so what, I voted who I wanted, not who I thought would win, and I won't vote against someone either.
Besides that, don't know. Whenever I hear people in meatspace bitching about somethin in politics/government, I question them, it's tricky but effective.. usually they get embarrased quickly, they can't hardly name any persons in politics, don't really know what's going on, but sure can rattle off their "teams" roster and the latest scores, or how their favorite band is doing, craplike that. I then nail them on some current events or names, get blank stares, and I go "How do you come to your opinion if you aren't aware of the issues or names?" Along those lines. Basically, I shame them, politely, show them they really don't know what's going on, then point them in a few directions in case they decide they SHOULD find out more and get active. Probably not effective, but dang if I'm going to stand there and nod my head "yes" to blather from people who really have nothing more than the most meager 15 second sound bite awarness of what's going on, then they think I should value their opinions or something. I just can't do that anymore, lost my patience long ago with that.
I can appreciate someone I really disagree with, AS LONG AS they have at least done some reading and research into a topic. Those people can learn from you, and you can learn from them, it's important to be honest and open to new data and be able to adjust your opinion, I certainly have over the years, I'm not static except from the POV of always trying to find out the real truth in a matter. I DETEST being lied to.. But the "don't bother me with any more facts, my mind is made up" crowd are nuts, better to just shine them folks on, it's a waste of calories dealing with them on anything more than the most trivial matters. doesn't mean they can't be nice people in all other aspects, but if it's anything important having to do with politics and directions and government, if their minds are locked, they are lost, they drank the kool aid, poisoned themselves. Just move on then.
Just 346 words remained on the list at this stage. The next stage is to involve the brain of the researcher. This eliminated all but seven words: Ugandan, Ukrainian, Egyptian, uninvited, incursive, indebted and unofficial. Naccache plumped for Egyptian, in this case.
Hmmm. So of the 346 words listed in the dictionary, the researcher had to guess? To a knowledgable attacker (eg. an analyst at the CIA who specializes in Egyptian terrorists) this step might be trivial, but in that case, they probably knew the answer anyway. On what grounds did the researcher choose Egyptian? Most likely because of the phrase "Egyptian Islamic Jihad". That is pure guesswork. It could have been an important point that the operative talked to Syrian intelligence or got captured by Mossad. The researcher simply didn't know.
The more general point is that algorithms cannot create information. Just like in image enhancement, if the data isn't there, in order to put it there you need other information to go on, and that is a non-algorithmic process (inference based on data), precisely because it is not guaranteed to be right.
Posters recognized by their sig,