Slashdot Mirror
U.S. Gov To Spider Internet
HopeSeekr of xMule writes "Perhaps as one of the first high profile uses of Alexa's WebSearch Platform, the U.S. government plans to search, link and reference every news site, blog and email on the Internet, using sophisticated AI codenamed ADVISE to do the correlations. Unlike traditional dataveilance like Echelon, ADVISE aims to find terrorists before they strike and even deduce their motivations in wanting to commit their crimes. Part of the breakthrough is a way for humans to view data as 3D holographic images with tech recently used at the Superbowl."
This won't help dealing with the terrorists at all.
What if they communicate via
- plain old websites/ftps
- internet storage servers, irc, etc?
- instant messangers
- VoIP
- decentralised networks?
Lets not forget that they can
- obsfucate.. simplest method would be typing stuff into a CAPCHA-like image. OCR has no chance...
- use slang
- encrypt!
It will end up as an intrusion to the privacy of ordinary people unaware of this and/or private communications among companies.
Its called Skynet. But it is looking for terorists...like Sarah Conner.
Let's see how well it works.
Sorry slashdot.
Proponents of this initiative boast that other data mining systems, such as Starlight, have already proven their worth in the fight against terrorism. However, given the fact that the current administration knew full well that Osama bin Laden intended to use hijacked airliners as missiles in a terrorist strike, but chose not to act, and that the CIA managed to uncover this information without a wholesale violation of the privacy of American citizens, I really can't see the justification here.
Why exactly does the Bush administration need such vast amounts of information to conduct their 'war on terror'? And why were they unable to use the perfectly good intel they did possess to thwart the worst terrorist attack ever on American soil?
One thing's for sure...it doesn't really matter whether the people OK this initiative or not, as Dubya & Company have amply demonstrated a complete contempt for the law of the land.
____
~ |rip/\/\aster /\/\onkey
IP ranges and user agent please.
Also, does it obey robots.txt?
www.terrorists.evil
User-agent: US-govt
Disallow: /
I wonder how long it will be before this system is used for political and/or selfish purposes?
George Orwell would be writing non-fiction if he were alive today.
He who knows best knows how little he knows. - Thomas Jefferson
I don't suppose this is going to honor the rules in my robots.txt.
Insert Generic Sig Here:
Unlike traditional dataveilance like Echelon, ADVISE aims to find terrorists before they strike and even deduce their motivations in wanting to commit their crimes.
"Hmm... ADVISE seems to think the terrorists are fed up with the 'nazi-like spy regime,' and are planning to use undead monsters to attack its servers.
Also, the terrorists want more boobies."
This was a good use of a few billion dollars to Haliburton.
The ______ Agenda
This just looks like the security people are getting desprate and trying to cast a wider net. The secret wiretaps used on citizens was a wide net that seems to have had poor results.
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
What a way to deal with resource depletion!
spoonerize "magic trackpad"
Excuse me?
If what he says is true, then it's possible that the technology has been used to protect our lives. Our freedoms are a different matter. Which of the two you consider to be the more important is a pretty strong indicator of whether you're a free country or a police state.
Real Daleks don't climb stairs - they level the building.
are they going to monitor e-mail?
Blogs and news sites are things we publish to the world and are easy to spider. Emails are private communications. In order to monitor them you have to either intercept them in transit or search records on private servers. Even if the email is available via webmail, you have to gain unauthorized access in a way that is generally considered trespass.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I'm pretty sure it comes bundled with IE on windows XP, but since it collects data about sites you visit (although it can't be traced to you), many programs regard it as a form of spyware, I know Spybot S&D does.
It goes beyond George Orwell's dystopian vision wherein a person can be punished for merely expressing sentiments that some AI may view as a likely vector for future anti-Establishment rhetoric; e.g. pre-crime Thought Crime.
By specifically targetting blogs (as email is already heavily trolled) who they're really going after are anti-Establishment political activists who won't be silenced. E.g. people like myself, HopeSeekr of xMule, who make distributed tools to prevent this fascism from ever *totally* clamping down on freedom of speech/expression.
Since the 380 Milliion dollar concentration camps capable of holding a million plus people are already being built, the only question is when will you be prompted to act (even as little as developing a program for open systems such as xMule, which is designed for the BSDs and Linux)? When the stormtroopers demand your papers? When your sister's head meets the butt of a soldier's gun? When you are shot protecting her? When?
The questions aren't if and when, they're now how bad and will your loved ones survive.
Promote freedom; fight fascism.
Comment removed based on user account deletion
Why is it that it is always the US government that seems to have been up to all this stuff since WW2 and increasingly even after the Cold War? I thought you were supposed to be the people from the land of the free and whatnot, really suspicious of government intrusion into people's lives, et cetera. Considering that a lot of you are always willing to disparage the Europeans for their love-affair with government, I certainly wish a lot of you would just take the log out of your own eye first... it's your government, despite all the rhetoric, that is horribly control-mongering at home and eager to support whatever right-wing dictator abroad, while ours concentrate more on making sure that kids with cancer don't die in the name of economic efficiency should they be unfortunate enough to be born to parents of financially limited means.
Go ahead, mod me troll/flamebait... at least I won't post this AC.
I want to play Free Market with a drowning Libertarian.
Someone should have told them that 24 is not a reality show.
Jack Bauer : Chloe, I'm sending you a picture. Can you datamine for him?
Chloe O'Brian: Sure. send it to my screen.
Computer: Blip...blip...blip.
Chloe O'Brian: Jack - it's the well known terrorist named...
I can't wait to see history books in about 100 years or so. Bin Laden's going to be up there with Sun Tzu and General Meade for the title of "greatest strategist ever."
Singlehandedly causing the West to self-destruct is no small potatoes.
Oh, this 911 wtc pretext incident is only the latest of a long history of the America elite using/allowing/manufacturing "pretext incidents" in order to start wars and grab power. See this page on HOW TO START A WAR.
However, I think this War On Terror has opened the elite up to the future possibility, should there ever be an anti-elite grassroots political movement, that our current laws might be used against the elite in order to try them for treason. Historically, treason could only be used if someone worked for/aided a foreign govt which was an American enemy.
Obviously, the War on Terror is not a war against a foreign govt.
Thus, we can start a War On The Elite. They are really, of course, the real enemy of all Americans. Always have been, always will be. That realization is what seperates Europeans from Americans, at least in part. They realize it is TOP against BOTTOM. We do not.
So try the elite in court for treason. We now have the legal precedent. Perhaps.
Who are the elite? Higg level politicians, CEOs of megacorps, prominent leaders large think tanks and nonprofit foundations, rich people, lobbyists, etc.
eat shiat and bark at the moon
...from the seriousness of this.
Modern times have led us into an age which reflects a lot of our worst fictional nightmares and we are allowing it to happen because we are accepting it because there is a "cmon, that was just a book/movie/joke. it won't *really* be like that" type of attitude.
The fact is that this sort of "total information awareness" nonsense is absolute power, and absolute power corrupts absolutely. Again, not a cute "quote" written for posterity, but a cold hard fact.
I believe that crime is a necessary catalyst for change, and that many things that were illegal in the past are now no longer illegal because society has recognised that these "crimes" were overblown, and that the thinking of the time would have labelled every person a criminal. Today the vast majority of people are labelled criminals by one group or another.
The point of all this is that a "Total Information Awareness" or a "Pre-emptive criminalisation" or even an instant criminalisation in the case of security cameras etc. lead us to a situation where our society is made up of criminals, 100% policing is necessary, and zero social change can ever occur.
Rich Gentlemen Hide - The Existential Comic
I seem to provide this quotation quite frequently these days. It was said by Lord Hoffman, sitting as a British Law Lord, in their ruling on the UK detention-without-trial fiasco a few months back:
And, unlike the rest of us, the Law Lords sitting in that case presumably did have access to any classified information they required. It's very convenient that the government can always tell us how its draconian policies are protecting us from imminent doom (but they can't tell us how for security reasons). That argument is rather less powerful when its critics include people on the inside who would be well aware of the full facts.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
As any web server can choose the page to display to any given client, how exactly does the system work out what is real and what is not?
For that matter, exactly how do they expect to access password or IP protected sites?
Could we can it already? Or is there still some moron out there who believes that bullcrap?
Sorry for the language, people, but I feel insulted. Just how DUMB do they think I am?
Terrorists don't use the net. At least not if they're halfway smart, and hell, they are! They ain't some dumb, mindless bomber drones (ok, some are, but look at the US soldiers... same way 'round, just with rifles). The key heads are very bright individuals, they know what they're doing. They know logistics, they know psychology, they know how to build a network right around your feet without you noticing.
Do they use the 'net? Let's assume they do, ok? Let's for just a moment assume they do.
First of all, they WILL NOT use the net for anything but the minimally necessary form of communication. They won't blog, they won't chat, they won't spend time in a bboard, all they do is MAYBE sending some data from A to B. And it won't be much data.
This data will be encrypted by best state-of-the-art encryption.
A good deal of this data will be plain false, and it will be false in a way that they can discern whether the feds were sniffing. Simply for testing their communication channel for being tapped and their key for being broken.
If you consider, all this incredible effort just 'cause some oil countries dared to think 'bout taking Euros instead of Dollars for their crud... it's amazing what some old hydrocarbones can move and shake in this world.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
You should check out my article Judicial Tyranny Killed America in 1803 over at my blog: Incendiary.ws. ALso, spread the word.
Promote freedom; fight fascism.
"So try the elite in court for treason. We now have the legal precedent. Perhaps."
First of all, how do you try a class of people in a court? 1 at a time?
Second, even if we assume that its possible, how do you plan to win?
Your only chance is revolution. Good Luck, becuase most people arent on your side.
Let me give you a little hint -- its easier to move from the "bottom" to the "top" than it is to war against them.
If I had something I wanted to move over the internet, without anybody being able to read it, I would use a one-time pad or some other nearly-as-secure encryption. It's so easy to do.
This program will only catch the foolhardy, and will could be used for nefarious purposes against (mostly) law-abiding American citizens.
So it is a bad idea.
Remember, as Americans, we have the right, and duty, to inform our congress-critters and other representatives when we think the government is heading the wrong way. Send a fax to your Senators and Representatives today. Fax their local office and their Washington office.
there are 3 kinds of people:
* those who can count
* those who can't
Whew! I'm glad their developing their own algorithm. If they'd license something from say, google, then I'd be worried they'd actually find something.
It's good to see our tax dollars are going to work replicating of all things - a SEARCH ENGINE. Is this because google wouldn't turn over their search results to the govt?
Well the cat's outta the bag now! The fact that we're even talking about this means that somewhere a terrorist is smiling. Good job liberal Christian Science Monitor! You're officially on notice!
</em>
If anything, it is the Americans' trait of fearing their government more than the foreign enemies, that is to blame... The latter fear has increased substantially in recent years, hence the public's acceptance of the administration's eavesdroping antics.
Your attempts to whip the former fear up, on the other hand, are so far fruitless, because, although the government has not become much better, it has not become much worse either... I'll take the unauthorized eavesdropping on terrorist suspects over the authorized raid on the child abuse suspects any day.
What "wholesale violation of the privacy"? The article talks about harvesting web-sites. No more invasive, than what Google and other search engines do for a living... Carnivore or the Clipper chip — yes, that could've been threatening...In Soviet Washington the swamp drains you.
But by placing your content on the public Internet, it's in "plain sight". There's no warrant required to look at it.
Xenon, where's my money? -Borno
Okay, it won't obey robots.txt. But what will the spider present itself as so we can lock it out? Or, even better, what are the sure signs that it's really Google or Yahoo or MS snarfing up my sites? Because I don't really care if other spiders get don't ahold of anything - close to 100% of legitimate searchers come through the big three engines. Should be possible to configure and script it so that anything but the spiders we approve of don't come up with much. If there are more than so many requests per minute, for more than so many pages - or it it goes to honeypot pages that aren't what the real public is interested in - lock the suckers out or feed them garbage. They'll find an Internet filled with hagiographies of the Bush family.
"with their freedom lost all virtue lose" - Milton
>>
Damn...I knew I picked the wrong week to order The Complete Idiot's Guide to Defeating the Great Satan. But the price was just so good.
I think the inclusion of email is what gives this the swarmy, big brother overtones. We've also have ample evidence that the Bush administration can't be trusted. The combination of Bush political flaks with no regard for privacy or the law and large amounts of personal data is what makes it scary to me.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
But of course, from the "brains" who are behind bolstering the costly, debt-exploding military-industrial complex (for fighting unjustified elective wars, no less), we are now seeing the formation of yet another unneeded program to scrape the web, with American tax dollars^W^W^Wproceeds of treasury bond sales to China (interest paid for by our children/grandchildren).
On top of this, we have a regime with widely demonstrated incompetence and/or willful negligence deciding to build a program like this. They couldn't even deal with the plain-language warnings they received regarding al-Qaeda's plans to hit tall buildings with jet planes. What I'm driving at is they can collect all the data in the world, and they have no ability to understand it or act on it, at least as long as His Lordship, King George is in power.
Steve Magruder, Metro Foodist
I can just see this now, using the BOINC client you can aide homeland security in detecting and catching those evil doers. It will send a portion of those emails (public mailing lists), websites, blogs, news groups, etc to any patriotic American to process and possibly flag the internet for a black list of words and certain contexts. Just wait and see.
Turn based strategy game that runs over XMPP. Phalanx
Do I get called a terrorist if I say I FUCKING HATE BUSH for abrogating the 4th amendment to the bill of rights?
i on.billofrights.html
"Amendment IV
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
http://www.law.cornell.edu/constitution/constitut
Wouldn't I be a coward if I didn't hate him for this breach of our fundamental rights? I assure you if the British had such a system for sifting all communications for treasonous intent we would still be the British commonwealth of the Americas. After all some of the original American REOVLUTIONARIES (can you say violent overthrow of the "legitimate" British government) communicated through committees of correspondence:
"In an era before modern communications, news was generally disseminated in hand-written letters that were carried aboard ships or by couriers on horseback. Those means were employed by the critics of British imperial policy in America to spread their interpretations of current events.
Special committees of correspondence were formed by the colonial assemblies and various lesser arms of local government. The committees were responsible for taking the sense of their parent body on a particular issue, committing it to a written form and then dispatching that view to other similar groups. Many correspondents were members of the colonial assemblies and also were active in the secret Sons of Liberty organizations."
http://www.u-s-history.com/pages/h675.html
Can you say secret terrorist organiztion boys and girls I knew you could. Of course the British had a right to monitor their public communications (letters), right? Afterall if they were doing nothing wrong...
Tired of all the isms, don't exploit people as an employer, or a government, mmmmK?
I'm sorry, but government agents had all the pieces that pointed to what happened on 9/11. And yet they were not able to put the pieces together until what? A year later? So, even if this spidering works as stated by the government, isn't there a 0% probability that they still won't be able to actually USE that data to help deter anything? And I agree with one of the other posts here... I doubt they will respect robots.txt. If they did, then all the terrorists would do it set that up on their web server... What the government needs to do is clamp down on how the terrorists get their MONEY. If the 9/11 hijackers were cut off from the big Oil baron money coming from Al Qaida even three months before 9/11, they would not have had the ability to buy airline tickets and perform the terrorism... Instead of listening in to my phone calls to my grandmother, I think the government should scrutinize EVERY single monetary transaction that is initiated from outside the US into the US. That seems alot easier and more effective than spidering the web for some obfuscated terror information written in Farsi code.
This is not something "we" need to be willing to do! My civil liberties are NOT YOURS TO GIVE AWAY! I'm terrified that a CS prof at Stanford thinks that it's no big deal that the US wants to spy on its own citizens and deprive us of our rights under the 4th and 5th amendments. (Yes, the 5th ammendment too, since US Citizens have been held on US soil without being charged with a crime, and thus deprived of due process of law.)
How can any educated person think this loss of privacy is "no big deal"? I'm at a loss for words.
What we need is a Web site to aggregate any known information about this project, especially the IP ranges from which it is operating. They can't spider anything if they don't get past the firewall.
The power of monolithic government can only be opposed by the organized efforts of informed citizens. The Internet makes it easier for us to be spied upon, but it also makes it easier for us to know who is doing the spying--and stop them.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
> The U.S. government plans to search, link and reference
> every news site, blog and email on the Internet, using
> sophisticated AI codenamed ADVISE to do the correlations.
> Unlike traditional dataveilance like Echelon, ADVISE aims
> to find terrorists before they strike and even deduce their
> motivations in wanting to commit their crimes.
Seventeen minutes later, Spynet became self-aware, and induced a nuclear exchange, destroying ANYONE NOT WEARING LIKE SIX MILLION SUNBLOCK! Have you ever had anything growing inside you? Do you know what it's like to create something? Wait, Statue of Liberty? That was our world! You maniacs! You blew it up! Damn you! Damn you all to Hellllllllllll!!!!!!!!!!111!!111!11oneone!!one
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Secondly, Democrats in the late 50's and early 60's were extremely divided over civil rights legislation. Many Democratic senators from southern states were strongly opposed to it, and even Eisenhower and LBJ weakened the first attempt (the Civil Rights act of 1957) to the point that it was practically useless.
I don't think it's controversial so much as plain false. Can you back this up with some factual data? The FBI 2004 Hate Crime Statistics indicate that about 63% of reported hate crimes with known offenders are committed by whites. Does this mean that hate crime laws are applied disproportionately against whites, or simply that more whites are committing hate crimes? Back up your assertion that the laws are applied in a "bigoted fashion". Anyone who has watched the antics of the Bush Administration over the past five years would think twice about making this statement. Try to get into a Bush "town meeting" if you're a registered Democrat. Try to get federal funding for scientific research that contradicts Bush's theological views. Try to stay out of jail for telling your patrons at the library that the government was snooping through your records. Try to keep from being blacklisted by Karl Rove if you are a Republican that doesn't toe the party line on the warrantless wiretaps issue.Try to tell Mr. Bush that you are neither with him nor with the terrorists and see what he says.