CIA Invests In Firm That Datamines Social Networks

An anonymous reader writes with this excerpt from Wired: "In-Q-Tel, the investment arm of the CIA and the wider intelligence community, is putting cash into Visible Technologies, a software firm that specializes in monitoring social media. It's part of a larger movement within the spy services to get better at using 'open source intelligence' — information that's publicly available... Visible Technologies crawls over half a million web 2.0 sites a day, scraping more than a million posts and conversations taking place on blogs, online forums, Flickr, YouTube, Twitter and Amazon. (It doesn't touch closed social networks, like Facebook, at the moment.) Customers get customized, real-time feeds of what's being said on these sites, based on a series of keywords. 'That's kind of the basic step — get in and monitor,' says company senior vice president Blake Cahill. Then Visible 'scores' each post, labeling it as positive or negative, mixed or neutral. It examines how influential a conversation or an author is. ('Trying to determine who really matters,' as Cahill puts it.) Finally, Visible gives users a chance to tag posts, forward them to colleagues and allow them to response through a web interface."Apropos: Another anonymous reader points out an article making the point that users don't even realize how much private information they're sharing over these services.

Comments

Can somebody tell me

[mrdoogee]

Why a US government agency needs an "investment arm?"

Re:Can somebody tell me

[AliasMarlowe]

Why a US government agency needs an "investment arm?"

Just copying the Brits. They've been referring to many kinds of government spending as "investment" for years now - even chunks of the welfare system. The debasement of the English language proceeds apace, on both sides of the Atlantic...

Re:Can somebody tell me

[Tubal-Cain]

If you've got your own little money tree you aren't as tied to budgets set by someone else.

Re:Can somebody tell me

[mayko]

What they really need is an "investment brain."

Re:Can somebody tell me

[sopssa]

Visible Technologies crawls over half a million web 2.0 sites a day, scraping more than a million posts and conversations taking place on blogs, online forums, Flickr, YouTube, Twitter and Amazon.

Well, hello there!

(their "Visible Technologies" highlights must be flashing with this slashdot story)

Re:Can somebody tell me

[Interoperable]

Just in case Visible Technologies crawls /. looking for it's own name: Fuck Off

Re:Can somebody tell me

[Thanshin]

Just in case Visible Technologies crawls /. looking for it's own name: Fuck Off

Salutations from a common SLASHDOT.ORG entity,

Do you mind if I ask you a question?

How influential are you among the other entities of SLASHDOT.ORG.

Thank you.

Re:Can somebody tell me

[TheCarp]

You may also wonder why they needed to illegally . Or perhaps you might wonder why they would dose "their own" citizens with LSD

I think Zack De La Rocha, The Last Emperor & KRS-ONE said it best in their track "CIA"
"Need I say the C.I.A. be criminals in action"

But given that the same song said that "President Clinton should delete them", I guess it wasn't as popular as it could have been :) and sadly, since 9/11 they are actually percieved to have a job again. A front job is always a very good thing for a criminal. Nothing like an air of legitimacy to hide criminal minds.

-Steve

Re:Can somebody tell me

[LordKazan]

that statement is neither necessarily true nor necessarily false - corporations and the government are bureaucracies. Sometimes one is better, sometimes the other is.

For example the National Weather Service kicks the living crap out of every private company trying to do the same thing. They pay well, the recruit the best and brightest, they are managed by professionals with experience doing what their underlings do [something you often only can DREAM of in the corporate world or the government world].

Medicare is another example - it's operating overhead is 4%. The operating overhead of private "insurance" (sorry, it's fraud, not insurance anymore) is a whopping 30% MINIMUM.

On the other hand there are some things private industry IS better at doing, and the government quite often contracts out to these people - construction comes to mind, software development, etc.

The government, when run by skilled people, tends to be much better at private industry than doing things that are "natural monopolies" (police, fire, roads, water, etc) or things the profit-motive would harm [like insurance].

Re:Can somebody tell me

[megamerican]

What you don't understand is that part of the CIA has ALWAYS had an investment arm, even before the CIA and OSS existed. The CIA was born out of the private intelligence networks already well established by Wall Street, hence why so many of the early CIA was filled and run by Ivy League schools and Yale's Skull and Bones crowd.

The funny thing is Facebook has long since been implicated as being funded indirectly by In-Q-Tel.

The second round of funding into Facebook ($US12.7 million) came from venture capital firm Accel Partners. Its manager James Breyer was formerly chairman of the National Venture Capital Association, and served on the board with Gilman Louie, CEO of In-Q-Tel, a venture capital firm established by the Central Intelligence Agency in 1999. One of the company's key areas of expertise are in "data mining technologies".

Since 1947 the CIA and other intelligence activities have been more and more privatized. They have always used front companies. Search for the Northwoods Documents, which were authored in the late 1950's.

Many have argued that E.O 12333 privatized a lot of intelligence work. Read Confessions of an Economic Hitman if you want to know one reason why they do this.

This is really only news to people who don't pay attention.

Re:Can somebody tell me

[sopssa]

Psst.. Visible Technologies, please do something about the Anonymous Coward bastard.. he's such a troll in every freaking thread.

Re:Can somebody tell me

[jamstar7]

Why a US government agency needs an "investment arm?"

To help fund off the books black ops projects, of course. Can't exactly go before the House Budget Committee and request multiple millions for bribe money to be used on foreign dictators, now, can you? And to provide plausible deniability, like 'Air America' back during the Vietnam days.

Re:Can somebody tell me

[jamstar7]

But given that the same song said that "President Clinton should delete them", I guess it wasn't as popular as it could have been :) and sadly, since 9/11 they are actually percieved to have a job again.

Strange. Kennedy fired Dulles & his Number Two, then wrote a couple executive orders breaking the CIA into a thousand pieces to be swallowed up by the various military intelligence services. His body was still cooling off when LBJ rescinded those orders and ended up starting the Vietnam War.

The Russian Federation's experiences with ex-spooks (former KGB officers) tells us that if you fire them, they'll just go underground as criminals. Russian Mafia, anybody?

Re:Can somebody tell me

[inviolet]

Medicare is another example - it's operating overhead is 4%. The operating overhead of private "insurance" (sorry, it's fraud, not insurance anymore) is a whopping 30% MINIMUM.

Probably not a good example to use in illustrating your point. Dealing with Medicare billing is such a gigantic heartache that doctors' offices who do so, and they are a small minority, will have to hire at least one specialized clerk just for that purpose. In this sense, Medicare is shifting its overhead onto its customers. Whereas private insurance is required, by competition, to be reasonably easy for all parties to deal with.

Regading your "fraud!" quip, I think the problem lies in our mistaken belief that "health insurance" = "health plan", and the subsequent chaotic conversion of the industry from the former to the latter. I would prefer to have health insurance, which is much cheaper than a "free zyrtec!" must-carry monstrosity whose sole purpose is to shift the cost of unhealthy people onto healthy people.

Re:Can somebody tell me

[Anonymous Coward]

I sometimes wonder about Google too...

I also wonder about the trillions the Federal Reserve refuses to talk about. Where did the money go?

Re:Can somebody tell me

[j. andrew rogers]

Why a US government agency needs an "investment arm?"

Investment vehicles like In-Q-Tel are not redundant with conventional venture capital and were created to fill some clear funding gaps in the existing technology venture markets.

First and foremost, they tend to invest in ventures with technologies that are sufficiently advanced or unusual that normal VCs will promptly ignore the venture. This came out of a realization that really advanced computer science and hardware technologies that the agencies needed were being routinely ignored in the traditional venture markets because VCs don't understand technologies that don't play buzzword bingo or which don't follow the herd. Investment vehicles like In-Q-Tel have a much stronger long-term technology vision and in-depth technical competency than traditional VC firms, which can be beneficial if you are building a startup based on serious geekery.

Second, they provide an inside track into organizations that would otherwise be very difficult for a startup with no ties to the defense industry to sell into. There are big customers of advanced computer and software technologies in the defense organizations, but getting a product in front of the right people is no easy task if you are an outsider.

Re:Can somebody tell me

[causality]

The Russian Federation's experiences with ex-spooks (former KGB officers) tells us that if you fire them, they'll just go underground as criminals. Russian Mafia, anybody?

So we should learn something from the ronin, perhaps. Nothing like history repeating itself...

Re:Can somebody tell me

[Vahokif]

We wouldn't be here talking about this if it wasn't for DARPA.

Re:Can somebody tell me

[tomhath]

As others have already pointed out, Medicare is a terrible example. When you compare overhead per patient served, private insurance beats Medicare handily. And with the profit motive removed there's relatively little incentive to reduce fraud, as reflected in the enormous amount of outright fraud and unnecessary work billed to Medicare.

Government is better at providing services that can't easily be charged to those who use them (CIA and military are good examples). As soon as it makes sense to directly charge for something like a parking lot or private security the private sector takes over.

Re:Can somebody tell me

[dwiget001]

>

Sorry, but the total monies owed in benefits for Social Security and Medicaid/care combined, as of about three weeks ago, all unfunded, is/was 65 TRILLION DOLLARS (Source: GAO).

We are adding one to two TRILLION to this figure every year. And, per GAO circa June 2008, even if we confiscated everyone's income (100 percent of it) from here on out, we wouldn't even have enough money to may the interest on the money owed in benefits for these programs. The bank is broke, and we are just printing money (causing the dollar to devalue severely) like it was going out of style (and, it pretty much is going out of style the way things are going, due to its devaluing).

Yeah, this is a combination of Social Security and Medicaid/Medicare, but we cannot afford to continue *not* funding these programs and just keep paying the benefits out of newly printed money all the time. Drastic measures are needed *right now* to stop the U.S. Government caused draining of the U.S. economy in regards to these (and probably other) programs. Benefits need to be cut drastically, or these programs just need to be wound down and eliminated completely. Yeah, it's gonna suck for a bunch of people, but not doing so is gonna suck for a whole lot more people (every U.S. citizen).

Re:Can somebody tell me

[ObsessiveMathsFreak]

Why a US government agency needs an "investment arm?"

Because US intelligence agencies are probably 10 to 15 years behind in terms of their data gathering and data mining abilities.

Let me put it to you this way. Would a company like Google, with the amount of data it has and the way it uses it, have been allowed to exist during the cold war? Not a chance. At the very least, Google would have extremely close connections to the establishment and it would be far more likely that it would have found itself coralled, curtailed or shut down by now. In the days when intelligence services were actually intelligent, the porous nature of the internet combined with the archival properties of computers would have been recognised as a powerful combination.

I would wager that top men in the CIA and indeed FBI can barely use email, and simply cannot fathom the web. Under their watch, Microsoft, Yahoo, Ad agencies, banks, marketers and above all the mighty Google have been allowed to amass compile and index a wealth of information on the populace that would have put the Stazi to shame, and they have not one whit of an idea what is contained there. If Google were an intelligence agency, it would be the most powerful one on earth. If the CIA was a search company, they would be outperformed by undergraduate CS students.

This speaks to a wider issue of the relevance of the CIA in the current world. It has proven itself incompetent at dealing with terrorists, and modern industrial spies. It cannot even match the capabilities of companies in its own country. The only real foe it ever faced, the USSR, is 20 years in the grave. The organisation is bloated and incompetent, good for the odd coup in a banana republic, but little else.

Right now, it is trying to find a role for itself in the new world. This initiative says to me that they intend to be the vast indexers and archiver that Google is, trolling people's data and storing it for later use and of course misuse. I'm seeing a future, more J. Edgar Hoover like CIA, armed with 20 years of secrets, scandals and embarrassing connections, making its presence and influence felt in the political, economic and social spheres.

The CIA should have been downsized after the wall fell. That mistake is going to end up costing the USA in the long term.

Re:Can somebody tell me

[mcgrew]

How influential are you among the other entities of SLASHDOT.ORG.

What difference does it make? His response was his, and is just as valid if nobody else agrees as it is if everybody else agrees. And I'd like to add my "fuck off" to these evil dickweeds as well.

Re:Can somebody tell me

[geekoid]

People ahve been saying that since' it was started, and yet they continue to be wrong.

Less alarmist and pundit sites for you, more reading the papers by the people who actually study it.

Re:Can somebody tell me

[geekoid]

And EVERY OTHER CLERK specializes in dealing with the private insurance agencies.

Oddly, the health bill is just that, health insurance. We all pay and we all get benefits. Not a whole lot of people in the medical industry would be working any different then they are now.

Re:Can somebody tell me

[geekoid]

Another example is the VA. consistently rated as one of the best medical institutions in the world.

The 'Government' does 10;s of thousands of project successfully every year. Sadly, the media only reports failures and perceived failure, and the government is open. Unlike private companies that don't need to disclose their failures.

Re:Can somebody tell me

[demachina]

Because in case your haven't noticed the U.S. government has turned in to a gigantic corporation, an extremely corrupt and incompetent corporation at that. There isn't anything resembling a government "of the people" in Washington any more and both parties are equally to blame. By any definition the U.S. has moved in to the realm of "state capitalism", and again both Democrats and Republicans are equally responsible, its been happening for a while but the last couple years it became a fait accompli as breakneck speed. "State Capitalism" is also known as "Fascism".

It a unique corporation in that it can garner revenue by:

A - Extracting it by force from it customer base in the form of taxes. For a new first if the current health reform bill passes they will force you to spend 10K a year on insurance from a private corporation or face punitive taxes and still have no insurance.

B - Issuing debt in the form of T Bills which has been a bottomless reservoir of cash as long as the dollar is the global reserve currency and China and Japan keep buying them. This will soon come to an end. When the dollar is phased out as the reserve current by the rest of the world because they are fed up with American mismanagement the U.S. will have to live within its means and it will be a cataclysm. The ONLY thing keeping the U.S. out of bankruptcy is the dollar being the reserve currency and China pegging to the dollar by buying U.S. debt. The U.S. current accounts deficit (the combination of trade deficits and Federal deficits which must be up around $2 trillion this year) would be completely unsustainable otherwise. If the dollar wasn't the reserve currency the U.S. would be in the hands of the IMF now, getting "austerity measures" and IMF loans.

C - The Fed can just print money which they've been doing at a ferocious pace lately which is why the dollar is tanking and gold is soaring. If you have any money sitting in a bank account a large percentage of it has disappeared in the last year without most people realizing it. As proof America is now on the same plane as Zimbabwe the Fed has been printing dollars and immediately turning around and buying U.S. debt (i.e. T-Bills). When you are printing money to buy your own debt you know your economy is doomed and your country is pretty much a criminal enterprise on par with Robert Mugabe.

You also know your country is a criminal enterprise when a former Goldman Sachs CEO, while at Goldman Sachs got the law changed to allow 30-1 leverage, and then as Treasury Secretary infused Goldman Sachs with something like $70 billion in free money at tax payers expense with no strings attached and NO debate when 30-1 leverage nearly destroyed them. It would, no doubt, have devastated our economy if Goldman Sachs had failed which it would have if AIG had failed. But, it would have been worth it to A) get rid of the cancer that is Goldman Sachs on our economy and to B) maintain the crucial concept of "moral hazard" without which free market Capitalism doesn't work.

In the system we have now Goldman Sachs is now both an FDIC insured bank and a high flying, gambling investment bank. It is getting billions in Federal guarantees and billions of dollars at 0% from the Fed, FREE MONEY, which they are gambling on stocks and bonds, making a killing and driving the current market bubble. They now KNOW if their risky investments crash again they are too big to fail and the taxpayers will just bail them out again. It is head they win, tails we all lose. When you no longer have moral hazard you no longer have capitalism, you have "state capitalism".

Re:Can somebody tell me

[dargaud]

So we should learn something from the ronin, perhaps. Nothing like history repeating itself...

You mean like let them seppuku themselves ?!? I'll vote for that and watch the video with enthusiasm.

Re:Can somebody tell me

[dwiget001]

I do even better than that, I read the information from the source, the GAO and other direct government information sources, not papers, not non-government web sites, etc.

Looks like you should do the same.

Re:Can somebody tell me

[brkello]

Uh, no it isn't. Medicare covers a large number of people and you are trying to tell people that only a small number of doctor's offices deal with Medicare? Who the heck does your fact checking, Glenn Beck?

Any insurance policy is all about shifting the cost of unhealthy people on to healthy people. You have no idea if you are going to get a catastrophic illness. Even if you eat right, are young, etc, is not guarantee. A friend of mine just died from leukemia at 31. Are you trying to say you don't want a drug plan...that you want to only pay for a catastrophic illness and you take the rest of the cost. Well, then do that. Government run full coverage health care would probably still be cheaper than if you only got catastrophic insurance from a private for-profit company. And enjoy paying $100+ for doctor's visits.

Re:Can somebody tell me

[will_die]

At least get the liberal talking points correct it is 3% for Medicare and 20-25% for private insurances, some places use 12% for private but the 2-3% for Medicare is constant.
According to the CBO for 2001-2005 Medicare admin costs were 24% higher on average per person then private insurances.
Also Medicare does not have to worry about little things like taxes or even fraud. Regina Herzlinger, Harvard Business School, has a study that shows that 10-20% of private insurance admin overhead goes into fraud prevention Medicare does not care since it is not thier money. Also Dartmouth researchers have shown that around a third of Medicare's spending goes to medical procedures that are not needed.
So if you are going to allow yourself to be used aleast be correct and use that 3% figure. Also the insurance that is used by federal employees, and that the Democrates are not giving up for themselves is private insurance.

Re:Can somebody tell me

[TheCarp]

CIA spooks being disembowled? I might masturbate while watching that.

!Anonymous.

[Tubal-Cain]

An anonymous reader writes with this excerpt...

Anonymous to us, maybe...

Re:!Anonymous.

[Thanshin]

No, no. He's truly anonymous.

Even to himself.

For all he knows, he could be you.

Re:!Anonymous.

[snspdaarf]

"The other day, upon the stair, I met a man, who wasn't there...."

some people are stupid...

[cryoman23]

on sites like twitter u just don't go and tell/fill in personal information... and if its mandatory scrap the site

Re:some people are stupid...

[gnick]

Exactly right. If you're posting for the world to see, even if you're using an alias, you'd better be flying the straight and narrow and be cautious enough to avoid posting anything that would allow identity theft.

Still, this kind of intrusion infuriates me to the point that I'm going to log in to my Amazon account (cleverly disguised user name surfergrrrl123), buy a bunch of peroxide and acetone, build myself a heckuva bomb in the garage of 1313 Mockingbird Ln (an abandoned house - not my address - Ha - Catch me now!), and then drive the 13.6 miles from my house to the CIA building in Birmingham to blow it up.

That way, those less cautious than I am will remain safe.

Re:some people are stupid...

[megamerican]

And people on facebook thought i was joking when I said my interests are: "Giving away personal data to the NSA."

Re:some people are stupid...

[The Archon V2.0]

build myself a heckuva bomb in the garage of 1313 Mockingbird Ln (an abandoned house - not my address - Ha - Catch me now!)

Yeah, well wait until you're sitting there building it and a guy who looks like Frankenstein's Monster shows up and throws a tantrum. You'll run away so fast it looks like a recording of you was sped up!

Re:some people are stupid...

[citylivin]

I believe these kinds of softwares, or others like them, will be able to fingerprint you by writing style alone. Alot of people make common grammatical mistakes or typos over and over. For instance, in the previous sentence and generally, i say "alot" instead of a lot. I also do not capitalize certain words. I am sure if you entered all of slashdot into some software programmes, you could very easily determine who my sock puppets are, purely based on writing style alone. Then of course there is the content, when most people will think nothing of mentioning the city or region that they are from. Even their industry. I am sure there are softwares out there which are already trying to gather this kind of data. Hell even i can do it if someone uses an uncommon phrase in more than one post.

positive or negative, mixed or neutral based on..?

[Smegly]

> "...labeling it as positive or negative, mixed or neutral." Positive or negative based on who's point of view? How would they rate something like this: "The last president sucked big time - and he's a stooge for oil barons!"

forget privacy, it's a waste of money

[Red Flayer]

Then Visible 'scores' each post, labeling it as positive or negative, mixed or neutral. It examines how influential a conversation or an author is. ('Trying to determine who really matters,' as Cahill puts it.)

Seems like a redundant effort. Why not just check the author's karma on slashdot?

Surely my high slashdot karma means I'm one of the most influential people on the internet... right? Right?

Re:forget privacy, it's a waste of money

[AJWM]

Surely my high slashdot karma means I'm one of the most influential people on the internet... right?

Well, it would, but your user number has too many digits.

Re:forget privacy, it's a waste of money

[sopssa]

So by implication you are saying that I am unimportant?! I think that if you look at Slashdot history you will find that I, the Anonymous Coward, have had many more posts than you have.

If I ever see you, Anonymous Coward, in the street, I'm gonna hit you in the face for all the crap quality posts on Slashdot. You better change your name now, because you are EXPOSED!

Re:forget privacy, it's a waste of money

[dkleinsc]

By that standard, CmdrTaco is more influential than Bruce Perens or Wil Wheaton.

Re:forget privacy, it's a waste of money

[gnick]

Yes, AC, but classifying you as 'positive', 'negative', or 'neutral' has baffled even our most proficient data mining experts here in Langley. That's why we had to contract out. One of my direct reports came to me the other day near tears:

"Sir, we just can't figure him out! One day he's writing insightful commentary with informative links correcting somebody who had made a simple mistake. The next day he was making harmless snarky jokes. And this morning he posted a long list of instructions on... On... On 'owning a nigger' sir. Either Anonymous Coward is completely insane, or facing a massive conspiracy."

Re:forget privacy, it's a waste of money

[bill_mcgonigle]

Well, it would, but your user number has too many digits.

Eh, you can't really blame him - some of us held out for a long time, thinking the Internet would always be anonymous. But then they made it so you didn't have to preview if you were logged in...

Re:forget privacy, it's a waste of money

[zoloto]

Considering he's created this site to foster tech-specific talk over 10 years ago instead of releasing press releases or blog with anecdotal chatter...

Re:forget privacy, it's a waste of money

[mcgrew]

So CmdrTaco is the most influential man on the planet? We're doomed!

In-Q-Tel?

[mujadaddy]

I have about a hundred of their Box Car Willie records!

...and they probably have about a thousand of mine! /rimshot

Your Rights Online

[SloppySevenths]

I'm curious if they poll slashdot comments for articles in the "Your Rights Online" category.

Re:positive or negative, mixed or neutral based on

[sopssa]

Don't you worry about the labeling - as soon as you post something that has keywords like "terrorism" you will be^H^H^H^H^HCARRIER LOST

Re:positive or negative, mixed or neutral based on

[vegiVamp]

Terrorism is bad.
Terrorism is bad.

Hmm.

TERRORISM is bad.

Nope, must've been your connection. Doesn't seem like there's anything wrong with terrorism afteCARRIER LOST

Why is this considered an YRO issue?

[Drummergeek0]

This is data that people freely post to be read by all anyway. All this seems to do is aggregate it. If you post it in a public forum, you shouldn't care who uses it or how. Unless the sites being scraped have policies against said scraping, who cares? I see it as a very valuable tool for sales departments.

Besides, I am sure the signal to noise ratio for this system is incredibly low, so one has to wonder how much usable information is retrieved.

The only problem I have with this is that my tax dollars are going to fund it.

Re:Why is this considered an YRO issue?

[MBC1977]

"This is data that people freely post to be read by all anyway. All this seems to do is aggregate it. If you post it in a public forum, you shouldn't care who uses it or how."

Because individualized personality profiles can be built of off seemingly innocuous data.

Re:Why is this considered an YRO issue?

[idontgno]

Reading publicly-posted comments is not a problem. At least, not to me. (I do know some thickies that are shocked, SHOCKED, that someone besides their BFFs can read their social networking crap.) Anyways, sure, public posting is public. Even lolcat knows that.

But agencies of state power reading, aggregating, correlating, and scoring... drawing secret conclusions based on hidden agendas and closed criteria... that's disturbing. Shades of J. Edgar Hoover's secret file cabinet and COINTELPRO and the basement of Stasi HQ.

This sounds naive, but on principle this should be opt-in only. If this were for marketing purposes, it certainly would be. But for stuff which actually matters (life, liberty, et al.), it's beginning to look like non-participation is the only opt-out. And the chilling effect is as effective as any active anti-dissent measure.

Re:Why is this considered an YRO issue?

[Drummergeek0]

Because individualized personality profiles can be built of off seemingly innocuous data.

Sure they can, but unless you are posting information you don't want people to know or are trolling, why do you care. The same goes for talking to people in person. If you make speeches or comments regularly at a town hall meeting or a similar function, eventually the people in attendance will begin to know how you think and what your personality is. And that information will get to the people/organizations that the statements are about. I think something like this will help weed out the more clever trolls, and help make useful information more influential. Unless the CIA starts rolling out death/silencing squads I don't see any issue.

Like I said I think the only real issue is that my tax dollars are going to fund it.

Re:Why is this considered an YRO issue?

[Drummergeek0]

This sounds naive, but on principle this should be opt-in only. If this were for marketing purposes, it certainly would be. But for stuff which actually matters (life, liberty, et al.), it's beginning to look like non-participation is the only opt-out. And the chilling effect is as effective as any active anti-dissent measure.

How is that different from the real world (as in not online)? Why should there be any difference between online participation and real world participation. If you disagree with something or have suggestions on how to better run something, how is this different than participating by writing and editorial, attending a protest/picket line, or simply attending an organizational meeting with similar interests?

Re:Why is this considered an YRO issue?

[Drummergeek0]

Remember ear damage is not generally not recoverable.

So it is recoverable?

You just proved my point though. I do not want to hide my information and I freely make it available. While my posts and other information could be used for nefarious purposes, it doesn't need to be aggregated for that, at the same time if my data is used to positive purposes such as keeping an eye on what a community is thinking to make better decisions, I am all for it. If you don't want your information used in either way don't participate.

My opinion is that your opinion is useless if you don't want to be attached to it. I fully believe that what I post is correct and wouldn't post it otherwise, so I have no problem with you knowing who I am. That is why my email address is next to user name.

Re:Why is this considered an YRO issue?

[causality]

This is data that people freely post to be read by all anyway. All this seems to do is aggregate it. If you post it in a public forum, you shouldn't care who uses it or how. Unless the sites being scraped have policies against said scraping, who cares? I see it as a very valuable tool for sales departments.

Besides, I am sure the signal to noise ratio for this system is incredibly low, so one has to wonder how much usable information is retrieved.

The only problem I have with this is that my tax dollars are going to fund it.

I'll explain that with a hypothetical analogy. There's nothing wrong with a person who can see your house from the public street. You knew it was a public road before you built a house near it, after all. However, you might find it a bit unsettling if the same van is always parked on that road and its occupant is always watching your house day and night. You might find it downright alarming if you noticed that he was videotaping your premises and taking notes about your daily activities. You might wonder what he plans to do with that information. You might be unable to come up with any good or desirable uses, but able to see a ton of abuse potential for it. But by your logic above, that should be okay because you had no expectation of privacy for anything you make visible from a public street, right?

In meatspace we do tend to draw a line between someone who happens to drive on that road and happens to glance at your premises, and someone who acts like a malicious stalker. There's a very good reason for that. The reason is not dictated by the special needs of meatspace; it is not the result of the law of gravity or the law of magnetism. No, the reason is rooted in sound principle. Principle is an abstract thing that applies equally to the streets and the Internet. I realize it's trendy for officials and such to act like we've never faced any of these questions before merely because a computer is involved, but it's not necessary.

The moment your creative output is collected, tagged, and studied, you become an object of study. It's a rather demeaning status when it's done for no good reason and occurs against your will, and by people who frankly don't give a damn about you. I see one major use of this system and it's not a good one.

In a truly representative government, the government changes over time to meet the changing needs of the people. The nature of that change depends on the people themselves and in this way it's a natural change, not an engineered one. Predicting it, for example to capitalize on it, always has some element of chance. This is a "problem" for people who think they should be holding the reins.

So they come up with systems like this one. Now they can quantify things like political influence and find out, with fine precision, where it comes from and who possesses it. What would have taken a massive propaganda effort in the past can now be done with just a little "push" at just the right place. Do some of you ever wonder where the restrictions came from that prohibit the CIA from spying on Americans? Do you imagine they are a product of chance? This is, after all, a method of circumventing those restrictions.

Re:Why is this considered an YRO issue?

[MBC1977]

Ok, lets look at this from another perspective: once a individual personalized profile is built, one can tell how you think and act - your pattern of behavior, if you will. Taken one step futher, (admittedly this is a hypothetical scenerio, but one we seem to be fast approaching)... government (local, state, federal - you choose) sanctioned "behavior and thought police".

Hmmm Joe Schmoe based upon his profile may be a threat / menace to society, lets "reeducate" him. Technology developed like this is both scary and unethical. (While that may sound naive to some, I personally don't believe the government should have that detailed knowledge).

Re:Why is this considered an YRO issue?

[Drummergeek0]

The only problem I see with that analogy is that you are saying it is someone watching everything I do, and only me. While monitoring the blogs can lead to that, I would see this as a van that drives through my neighborhood everyday, taken pictures of the houses. While still a little unsettling, all they are really going to see is what I put out for them to see. I may have a yard sign that supports a candidate or says some religious/political statement, but everything else about me is kept inside. If they want to know more about me in particular they would have to narrow it down to me as an individual and then investigate me further, multiply that by the millions of users online and that would be a logistical nightmare.

While it is easy for them to see who lives in a particular house based on the address, it is a completely different scenario to know that all around, and still have any meaningful information. The amount of manpower it would take to tie a real 'meatspace' individual to their cyberspace id is pretty high and would take a considerable amount of time to do it for everyone they monitor.

While this may seem like a precursor the "1985" big brother scenario, I just don't see it going that route. I think this will more than anything be another form of metrics. Similar to reading the minutes from organizational meetings and newspaper editorials, as well as phone/street polls.

Most of these blogs will not freely give up their logs and user information, and any other method of tracking will be guess work at best. Multiply that amount of time by the amount of users in cyberspace on all of these different blogs, the government would need an entirely new division to handle it and it still would take years to have anything even remotely useful.

Re:Why is this considered an YRO issue?

[mcgrew]

Unless the sites being scraped have policies against said scraping

Who cares what your policy is? You put your site on the public internet, I'll use it any damned way I see fit.

Re:Why is this considered an YRO issue?

[Drummergeek0]

Just seems a little too "1985" to me, I just don't see the government having the kind of manpower needed to do something to that extent. Just look at /., they had to start having the members of the user base police themselves because of the amount of trolling, and that was just to clean up the comments. Imagine if they wanted to tie every user to a flesh and blood person (with SSN not email). Now multiply that by the sheer amount of public forums out there, it would take a HUGE organization just to handle that, and even then it would takes years if not decades to have any even remotely usable information.

Re:Why is this considered an YRO issue?

[mcgrew]

Sure they can, but unless you are posting information you don't want people to know or are trolling, why do you care.

Trolls care? Wow, you learn something new every day! And here I though they were just asshats...

Re:Why is this considered an YRO issue?

[causality]

The only problem I see with that analogy is that you are saying it is someone watching everything I do, and only me. While monitoring the blogs can lead to that, I would see this as a van that drives through my neighborhood everyday, taken pictures of the houses. While still a little unsettling, all they are really going to see is what I put out for them to see.

So surveillence is only bad when it's personal? I can't get behind that. There is no principle in it, there is only the consideration of whether you alone will have to bear the burden of it. While it may make you feel more equal to know that everyone else is being treated the same shitty way that you are, with no regard for their privacy, it's still no excuse to treat people that way.

To me, the idea that the CIA can personally spy on you and only you is bothersome. The idea that it's just as easy for them to spy on many people at once is even worse. It's not an improvement, not if you don't wish to see intelligence agencies engaging in domestic spying and data mining. If you love freedom and understand a thing or two about how it is compromised, then there are two concerns here. One is that such a system can target individuals.

The other concern is that it indicates a government agency that is able and willing to overstep its bounds. Your comfort that they are not personally targetting you does not help with this one. These are not people who respect the limitations under which they are expected to work or the liberties those are designed to safeguard. Rather, they view those limitations as obstacles or challenges, to be disposed of as soon as possible and by any means necessary. Any excuse will do; "safety" is a popular one.

While it is easy for them to see who lives in a particular house based on the address, it is a completely different scenario to know that all around, and still have any meaningful information. The amount of manpower it would take to tie a real 'meatspace' individual to their cyberspace id is pretty high and would take a considerable amount of time to do it for everyone they monitor.

Yes, it would take a lot of manpower. So they are doing their best to automate the process, to offload as much of that effort onto sophisticated machines as they possibly can. Such as the ones this firm is using. In other words, the CIA seems to agree with you and is obviously doing everything it can about the "problem" of how difficult it is to spy on everyone.

What, did you think they would just give up?

Re:Why is this considered an YRO issue?

[Drummergeek0]

My point was that by spying on everyone, they are not really spying on anyone. Back to the home analogy, they get pictures of the neighborhoods to see what kind of people live in that neighborhood. Even more so, this really isn't even like monitoring your house, more like monitoring a meeting hall, they want to know what was said during a public meeting. Would you be worried if you went to a political party meeting and a member of the CIA (or any other government agency) was there? I for one would be happy that someone from the government might be listening to what was being said because that opens up the opportunity for it to be heard by the powers that be. And I doubt that the CIA is interested in anything but political information. Could this information be abused, of course it could, but then again, every ability the government has can be abused. I just personally don't see that much risk to them knowing what my political views are.

Yes, it would take a lot of manpower. So they are doing their best to automate the process, to offload as much of that effort onto sophisticated machines as they possibly can. Such as the ones this firm is using. In other words, the CIA seems to agree with you and is obviously doing everything it can about the "problem" of how difficult it is to spy on everyone.

What, did you think they would just give up?

Automating the process isn't as easy as it sounds. To truly identify someone you need their IP address, and hope that the IP address is there personal one, not from a internet cafe or hot spot. You might get lucky based on an email address, if the email address is both valid and has the personal information available. But to achieve this you need to be able to get user records from the companies that have those addresses, as well as the logs from the sites that have the postings. As shown on previous /. articles, sites don't tend to give up that personal information freely. And with there being not standard of user names on all sites, there is no easy way to tie users together. My ID, DrummerGeek0, ties to me here, but does not in other places, and I know that some of my user names on other sites are tied to different people because I couldn't register them. Add on to that the millions of troll/spam accounts out there. As I said before, this would be a logistical nightmare that couldn't just be automated.

The "sophisticated machines" you are speaking of just scrape a site and parse the contents. It looks for certain keywords and follows logic to determine the tone of the statement. That is nothing compared to tracing out who someone is.

And the idea that they are monitoring what you do, they are monitoring what you say in public forums that are available to everyone, if this truly worries you, keep silent and don't post, but this is no different than all other forms of recorded public communication. You give up the right to privacy when you post, you wave a flag saying that YOU have an opinion, and YOU want it to be heard. I don't see a problem in the CIA, or anyone else(government, commercial, or individual) deciding to listen.

I am not in support of big government, or the government knowing everything I do, but if I post a political statement, I am usually hoping it is listened to. Otherwise, why state it? That seems to be the scare here too, is that if I disagree with this administration they will silence/censor/kill me. If you are seriously concerned about that put on a tin foil hat and stay offline, otherwise look at this for what it is, a way to monitor the pulse and sentiment of the American people another way, online. This may lead to change in policies, it may lead to astroturfing, or it may lead to nothing, just another piece of information the CIA has that they do nothing with.

Re:Why is this considered an YRO issue?

[causality]

The "sophisticated machines" you are speaking of just scrape a site and parse the contents. It looks for certain keywords and follows logic to determine the tone of the statement. That is nothing compared to tracing out who someone is.

Do you know why it would sort out specific keywords? Let's say they are interested in information that only a tiny percentage of the population would talk about. That could be anything from non-mainstream political ideologies all the way up to communication among terrorist groups or worse. The keywords let them focus their manpower on the tiny percentage they are after. They then have the resources to find out who these people are.

Over time, I would expect this system to become more and more accurate. In fact I wouldn't be the least bit surprised if they have multiple systems: highly classified, extremely expensive systems with capabilities that far exceed the scope of this project, and projects like this for their "light work" that they are willing to let the public know about.

The problem is not one of capability or determination. The problem is one of trust. Do you trust this unaccountable agency to only go after things like terrorist cells and foreign threats to our security? Or does it cross your mind that any one group with this much power might abuse that power and that this is particularly a concern when the public isn't even allowed to know about everything they do, let alone exercise oversight over it?

Re:Why is this considered an YRO issue?

[Drummergeek0]

Of course there is that threat, but if they had that kind of technology, why on Earth would they be investing in a private company to do something like this? The CIA may not have public oversight in the way that we would like, but at the same time in true matters of national security, thank god they don't. There are certain things that just don't need to be known by all. If this was some nefarious plot, I think they would probably be more under wraps, at least about who did the investing.

I am pretty sure they will use this as it was designed, as a marketing tool. They will use this power, as you said, to better sell what they want to the American people.

What this company does is pretty simple, given the manpower, the CIA could easily create something on there own, the fact that they are using a commercial entity tells me that they don't have the highly sophisticated systems you spoke for this operation.

I know it is easy to think of the CIA, FBI, and other top secret government agencies as nefarious entities with their own agenda, but when it comes down to it, they are other American people just like us. They have agendas, sure, but not all are nefarious, sometimes a cigar is just a cigar. We have no reason to believe that there are bad plans being made to silence/censor/reeducate the American people.

And as far as the terrorist search goes, my hats off to them if they find that kind of information on PUBLIC forums. But I can tell you, that the red tape they will have to go through to get information on a "hunch" is more than it is worth because in matters of user privacy, cases still have to go through some department of the judicial system. In a way, there is your oversight because for the CIA to arrest, search, or seize property of an American citizen, they would still need a warrant which requires a judge to sign off, for now at least.

Re:Why is this considered an YRO issue?

[causality]

Of course there is that threat, but if they had that kind of technology, why on Earth would they be investing in a private company to do something like this?

I can think of two immediate reasons. One, to provoke the very question that you asked. In other words, to encourage people to underestimate them, which is good strategy. George W. Bush followed a similar strategy, appearing to be a bumbling fool most of the time when in public yet ruthlessly and quite cleverly arranging things behind the scenes.

Two, for the same reason that you wouldn't hire an 18-wheeler to haul ten pounds of cargo. A regular passenger automobile will do the job. I would be shocked and amazed if someone could prove beyond all doubt that the CIA does not have some highly classified, extremely sophisticated machines. I would be a little less shocked to learn that they use those highly classified systems to do work that is quite ordinary and can easily be done with commodity hardware available to anyone. So if they know better than to swat at a fly with a sledgehammer, that wouldn't surprise me, but it doesn't mean they have no sledgehammers.

There are certain things that just don't need to be known by all. If this was some nefarious plot, I think they would probably be more under wraps, at least about who did the investing.

I never claimed that this was a nefarious plot, only that it's a step in the wrong direction. I can think of many uses for such fine-grained monitoring of influence and public opinion, and none of them are good or desirable. Quite simply, knowledge is power.

About secrecy, I'm with JFK on this one. To quote him:

The very word 'secrecy' is repugnant in a free and open society. And we are, as a people, inherently and historically opposed to secret societies, to secret oaths, and to secret proceedings. We decided long ago that the dangers of excessive and unwarranted concealment of pertinent facts far outweigh the dangers which are cited to justify it. Even today, there is little value in opposing the threat of a closed society by imitating its arbitrary restrictions. Even today, there is little value in ensuring the survival of our nation if our traditions do not survive with it.

I know it is easy to think of the CIA, FBI, and other top secret government agencies as nefarious entities with their own agenda, but when it comes down to it, they are other American people just like us. They have agendas, sure, but not all are nefarious, sometimes a cigar is just a cigar. We have no reason to believe that there are bad plans being made to silence/censor/reeducate the American people.

No, but it wouldn't take a very sophisticated fool to understand that the American people would be quite vulnerable to something like "re-education" if it were implemented. It's not like we believe in timeless principles anymore, and it's not like we love freedom so much that we're willing to suffer inconvenience for its sake anymore. It's not like we are a tough-minded people who think critically and understand the importance of questioning authority, and demand real answers to those questions. Perhaps one of my ancestors remembers an America like that, but I've never had the privilege of seeing it.

A re-education campaign would be frighteningly successful. Therefore, the stakes are very high. When the stakes are very high, small/unlikely risks aren't so small anymore. All I would like to see is this: that the government has no such capabilities. I like that better than knowing that they have such capabilities and then trying to trust them not to abuse them.

Re:Why is this considered an YRO issue?

[Drummergeek0]

I can agree with just about everything you said, particularly about having the over powerful machines used for a simple task. I guess my point of view is based on the time I spent employed by the US Government (US Air Force to be exact). As a fellow veteran once told me:

"If you spend anytime employed by the US Government, you will be amazed that this country is as powerful as it is."

Meaning that with the kind of resources these organizations have or potentially have at their disposal, their mismanagement prevents them from utilizing it's full potential. They may or may not have future plans for reeducation, and while the American people at this point would probably not even realize it was happening, I can't see them being able to pull something like it off.

I also try to not read to far in to most seemingly innocuous actions taken by our government for two reasons:
1. I still have hope for our government that while certain individuals are dangerous and can't be trusted, due to our constitution and the checks and balances in place that they wouldn't/couldn't get away with something as scary as a re-education campaign.
2. I don't have enough confidence in the few that would attempt such a thing to actually be able to pull it off.

Also, as far as JFK's statement about secrecy. I understand the sentiment, but based on the mentality of the American people, as well as the media, I feel that if all information (especially that of the CIA such as under cover agents and terrorist watch lists) was available it would put more lives in danger than it would help. Imagine if after 9/11 (I hesitate to even mention 9/11, but it is the only event I could think of that really gets my point across) people were able to know who the CIA thinks might also be terrorists. I feel that many innocent people would probably lose their lives, in more ways than just death (family, friends, jobs, etc) due to the sentiment at the time. And, lets face it, even if the government was completely transparent, people would still think they are hiding something.

My sentiment may seem a bit naive, but while I may be wrong about some, if not all, of my thoughts, I would rather trust (with open eyes) until I was given a reason not too than be critical and distrustful of everything. Many things I read about send up immediate red flags, but this one didn't. I just feel that there are too many other variables required for this to be a threat.

I will admit, though, that my trust level has only gone up since Bush left office, if I had read something like this a few years ago I would be expecting the reeducation to begin within the week.

Motivation?

[cosm]

How much are they really going to get from Web 2.0? Where the best party is on frat row? What Joe Blow's opinion is on policy x vs. policy y? Grandma's photo of Fluffy? I would imagine those truly interested in acts of Federal Offense would avoid large, preexisting cross-linked networks like this. If anything, motivation is more towards being a Surveillance State, or to catch some technologically ignorant people doing really bad things.

Re:Motivation?

[Nadaka]

There are three kinds of people who would use social networking sites to "do very bad things".

1: idiots, there are a lot of them.
2: those seeking to hide in the crowd.
3: those seeking to take advantage of 1.

Re:Motivation?

[localman57]

How much are they really going to get from Web 2.0?

You'd be shocked. There's still this attitude by lots of people that what happens on the internet stays on the internet. Our local probation department routinely violates people based on facebook photos of them:
-In places they've been tresspassed from
-Consuming alcohol (if it's a condition of probation)
-Pointing guns at each other
-Being around children (sex offenders)
-Driving (Habitual Traffic Offenders)

Of course, the photos could be old, or (theoretically) doctored. However, like any other evidence, they have to be put before a judge, who determines if they are likely to be incriminating. (Probation violations require a judge's decision in our state, as opposed to parole violations, which do not).

Re:Motivation?

[zoloto]

blending in is key factor #3

Re:Motivation?

[Zordak]

Our local probation department routinely violates people based on facebook photos

Seems like a harsh penalty for running afoul of probation limits.

Re:Motivation?

[demachina]

Mining peoples social networks is almost certainly extremely useful if you can find a person of interest. You can immediately identify their close associates.

I'm kind of doubting many serious terrorist or criminals actually use Twitter but I wager all the Iranians who used facebook and twitter to protest a rigged election immediately had their social networks scoured by the Iranian Revolutionary Guards. If you are trying to unroll an underground network if you find people that put their social network online must be priceless.

To put it another way I doubt mining social networks is very useful to combat crime or terrorism but it is probably an exceptional tool for political oppression and to stifle dissent.

That's kind of the basic step -- get in and monito

[negrace]

>> That's kind of the basic step -- get in and monitor, 1. Get in and monitor, 2. ??? 3. Profit!!!

Information...

[stoicfaux]

Information wants to be free...

Re:Information...

[mcgrew]

Information wants to be bought and paid for. See how dumb that cliche is?

However, when information isn't free, neither are you.

I feel sorry for the crawler

[Interoperable]

The crawler is going to get seriously depressed if it crawls YouTube conversations.

Re:I feel sorry for the crawler

[localman57]

I agree. If ever there was a computer likely to become sentient and decide that the human race should not continue, it's this one...

Re:I feel sorry for the crawler

[jamstar7]

The crawler is going to get seriously depressed if it crawls YouTube conversations.

I'm just wondering what they'll do when it hits 4chan. Do they block that, or do they send all that to the FBI & let them wade thru the pedobear posts?

Galois Lattices and Social Networks

[Anonymous Coward]

Courtesy of Google: galois lattices and social networks.

Yours In Ashgabat,
Philboyd Studge

Here's why

[NoYob]

1. To promote technologies that will add to the CIA's arsenal.
2. To buy into companies that allow them to circumnavigate Constitutional provisions against spying on American citizens.

For example, the second one, the CIA loves companies like this one and the credit bureaus because they can legally collect information on private citizens. Then the CIA "buys" the information from them and they can go to Congress and say, "Nope! We are NOT spying on Americans." - at least that's the answer to the Congressmen that aren't afraid to appear to be "weak on terrorism" or afraid to be lambasted by ignorant talk show hosts.

Re:Here's why

[DutchUncle]

So my follow-on question is, Why does everyone think it's OK for private companies answerable to no one (or the highest bidder) to be collecting this information in the first place? Well, yes, I suppose most people in this thread don't think so, but all of the normal people out there seem to be perfectly happy with the idea.

Re:Here's why

[causality]

So my follow-on question is, Why does everyone think it's OK for private companies answerable to no one (or the highest bidder) to be collecting this information in the first place? Well, yes, I suppose most people in this thread don't think so, but all of the normal people out there seem to be perfectly happy with the idea.

Because they don't view the Bill of Rights as sound and enlightened principles to be honored wherever possible that happened to be enshrined in the Constitution. They view them as rules like any other. Then they note that either the rules don't apply to those private companies or they would be difficult to enforce, and for them, that's that. It's a mentality that is all about what is allowed or what can be gotten away with, rather than what is right or wrong.

I do have a more immediate question. If an average citizen hires a person to do something illegal, both the person and the one he hired can be charged with a crime. If it's illegal for the CIA to gather data on American citizens, why is it suddenly legal when they do the same thing by proxy? Why wouldn't both they and the company they hired be prosecuted for this?

Re:Here's why

[causality]

[Now, the issue of whether it's illegal from the CIA to use the information because of the -CIA's- restrictions on domestic spying is another issue, though I could theorize that using publicly available information is not spying... but certainly that's debateable.]

I have an idea. Hopefully we can agree that when the CIA looks at your social network pages, they can correlate information and find facts and draw conclusions that would never occur to the average citizen who visits your page. Why not regard that as a type of search? They do, after all, work very hard to have such capabilities. It's not a stretch of the imagination to suggest that their capabilities are likely to be above and beyond what the individual actually intended to make publically available. Since this is a US government entity gathering data that ultimately came from US citizens, then why not regard this search as any other and have all of the rules apply that would normally apply to any police search?

Re:Here's why

[Dogtanian]

And if you posted 'private' information on the public internet, guess what, it's no longer private, so no complaining about it's use

FWIW, don't complain when information you posted "anonymously" is identified with you in the not so distant future:-

Even if the websites were unwilling to share account info with each other, I suspect that one could write a screen-scraper for information and posts on the most popular sites, and group all the public info associated with a particular account anyway- which is probably enough.

From this post describing why "anonymous" accounts give a dangerous false sense of security.

FWIW, this wasn't the first time I'd posted that, I spotted the danger some time before, as would anyone who'd even heard of 'data mining' have done if they'd applied even a small amount of thought to the issue.

Today's "safe", "anonymous" and "unrelated" data is tomorrow's personally-associated, tied-together overview of everything you've said. Even if you stopped today, you've probably put enough existing data out there to say a lot about you.

Re:Here's why

[dave562]

People go along with the program because they enjoy the "benefits" provided by the program. In the case of organizations like ChoicePoint and the credit bureaus, people like having access to credit. People like being able to spend money that they don't have. Just look at Congress and the rest of America. I read something the other day that said the debt load of America is over 100% (it was around 120%, down from 130%+ a few years ago).

People are willing to give up their rights so that they can have access to credit. Our entire system is a system of credit. Credit is created from debt. Banks "borrow" money from the Federal Reserve and that borrowed money is then lent out in the form of credit. In order to take part in the system, people need to build a "credit history". The system is rigged against people who want to use cash or live within their means. Any money saved is destroyed by inflation.

In short, the American people are debt slaves. We are all slaves to the system. Of course the travesty is that we are one of the most resource rich countries on the planet. We could feed all of our people and still have food left over to ship overseas. Yet we have people starving, we have people suffering from obesity and all sorts of health problems related to poor diet.

Now we're seeing that America is going to experience a "slow recovery" while countries like China and India are picking up steam. That isn't accident. It isn't some sad twist of fate, or bad luck on our part. The third world is easier to exploit. There are billions of people in China who don't understand the evils of credit. Americans are tapped out. We can't service our debt so the financial system is abandoning us. In reality, Americans aren't any smarter than Indians or Chinese or Africans or anyone else. We're all humans who have been blessed by good fortune with working minds. The only thing that separated America from the rest of the world was our manufacturing base, and our financial system. The manufacturing base is gone and the financial system is leaving with it.

Sorry for the tirade.

Re:Here's why

[ojustgiveitup]

Pardon me, but this is public data. The internet is public. Anything posted on the public internet is public. So *my* follow-on question is, Why does everyone thinks it's OK to bitch about things they have posted in the most public forum in human history becoming...well, public? How can this be considered spying? If you write something on your blog or in a comment on someone else's blog or on twitter or on youtube or *wherever on the internet* it *will* be public - can we please move past thinking otherwise?

Re:Here's why

[DutchUncle]

I agree completely, if you're talking about the original topic of scraping social network sites. This branch has digressed into specifically credit information (or similar) that is *not* posted publicly. Credit reporting bureaus buy information from banks about your personal transactions, and from other companies about your personal payments. That's the kind of thing one used to hire a private detective to find out on the sly; now companies are selling it as a matter of course. I forget which state was selling information that they were requiring on the driver's license application; you have to PAY to get a license and entrust the government with your information, and they turn around and sell it to advertisers. (I wouldn't mind if the government were charging advertisers a fee to add something to the envelope they're going to mail me anyway; I *do* mind if they're handing information *out*.)

Re:Here's why

[ojustgiveitup]

Fair enough! I was only referring to the original topic. (I don't like the credit racket any more than anyone else).

Re:Here's why

[brkello]

My follow on question to you would be why do you think it is wrong for private companies to collect information that people put on the Internet by their own free will?

Re:Here's why

[thejynxed]

They can aggregate data across sites on individuals using such metrics as average word count per post, average grammatical error count, specific common grammatical errors, what type of topics these individuals tend to post to, which words appear most frequently per poster, euphemisms and other linguistic identifiers, etc. and pretty reliably narrow it down to who is who. Tie this in across different media formats, various databases, frequent flier programs, etc and you come up with a veritable gold mine of intel.

Your name saying Anonymous Coward doesn't seem to mean as much these days when such analysis is used.

Re:Here's why

[mrmeval]

If google searching is any indication they're losing data about what I said on usenet in the 90s. I should have documented this loss more accurately but I'd say a good 30 percent is missing. They had all of it from when they bought the archives so I don't know why some of it is missing. Unless some special effort is made to protect this data I don't see it as lasting more than 10-20 years.

Re:Here's why

[Dogtanian]

A lot of the information on the web/net prior to the late-90s is possibly lost for good, though I suspect that a good deal of Usenet is still out there- not that I necessarily approve of this, since I'm not convinced that it was seen- or intended by most people- to be more than ephemeral at the time.

The more recent stuff? There are probably enough caches and stuff around that it will reappear sooner or later if someone wants it to. Even if the originators don't.

Datamining Social Media

[Anonymous Coward]

There are a TON of companies that are trying to datamine social media for a variety of reasons- I'm posting anonymously because I work for a company that makes one of these products.

What is interesting is companies that make consumer products all want these tools to be able to track the companies interaction with the consumer- these companies are specifically replying back to specific posters in order to stop the spread of what they call "misinformation", but in actuality is just anything where the company is painted in a bad light. Let me be clear: Corporate America wants to control everything that is said online, and the tools to do it are starting to show up. Companies are starting to employ people whose soul job is to look at social media and respond to negative comments.

I predict not far in the future there is going to be a push for owners of social media sites to have some control over who can index their content.

Re:Datamining Social Media

[AdamInParadise]

I like to use my blog to rant about unusable products and deceptive practices. Once I got a call from someone working for a large online retailer regarding a post where I labelled one of their practices as a "fraud". Technically it wasn't because the issue was not settled by a court (but another similar company was condemned for a very similar practice). He was very business-like but a bit pushy, so I googled his name. Turns out that he's basically in charge of responding to all the online criticism aimed at his company (a busy job). However, he really seemed to have to power to solve the issues faced by the complainers (misdeliveries, lateness...) so I guess that makes him "one of the good guys." Of course the very existence of such a position shows that the company is not doing a perfect job at handling customer complaints through standard channels (unlike, say, Amazon). Nevertheless, he was very upfront upon the fact that his job was to maintain the online reputation of his firm. If you do not complain online, well, he's not going to help you and you will be stuck with your problem.

Now, of course, I would totally buy from this retailer again because if I had an problem, I could since call this guy (his number is everywhere) and have it solved quickly.

I guess that my point is that it's hard to tell whether your technology will help us or enslave us.

Re:Datamining Social Media

[astar]

I have thought a bit about control of information, not just corporate control.

Google is developing a firefox plug-in for adding additional information to established web sites by viewers. "Helpful comments". Well, that is google.

I seem to recall this was done in a less restricted way perhaps five years ago by someone, it went through some courts and was considered legal, basically because it was a user choice to install the plug-in. Probably got the story from slashdot.

Writing a fire-fox plug-in is not in my skill set, but I favor this idea. For requirements, I would start with

1) user can chose his hosting server
2) anonymous
3) user moderated, perhaps with a slashdot model, since that is all I am really familiar with.

The plug-in would on request show a random comment, with a bias on how recent the comment was and moderation state.

Might actually become widely used. I am sure the Obama people would like to post anonymous comments on the fox web site. :)

Running a popular server might even be profitable.

I would be interested in comments on this idea, particularly requirements comments.

Re:Datamining Social Media

[garcia]

Let me be clear: Corporate America wants to control everything that is said online, and the tools to do it are starting to show up. Companies are starting to employ people whose soul job is to look at social media and respond to negative comments.

You're right, they are responding to negative comments. In fact, I'm impressed with the responses I have received from my ISP (Charter) and SAS. I posted some pretty pissed off comments about Charter last week when my connection dropped (I have business class and I expect it to remain up) and they not only responded to me on Twitter but they also called me to ensure my connection was back up in a timely fashion (I was up before any of my neighbors with residential connections). SAS once contacted me (actually one of their VPs did) via e-mail following a Twitter post expressing frustration about "proc gplot". They wanted to make sure that I got it "to do my bidding".

So while I am sure that what you are saying is true, I have not yet seen corporate America do anything with social media except stop negative comments the CORRECT way--by ensuring the customer is happy. If that sort of customer to company interaction continues in the way it has been, I would say that their attempts are amazingly useful. If they somehow want to change the course of those discussions forcibly or by spamming with positive, I am sure the backlash--especially with those of us that enjoy the way it currently works--will be phenomenal.

Re:Datamining Social Media

[Tobor the Eighth Man]

You're right that most companies are using it to monitor customer satisfaction and do market research, not in any negative sense but to see how people honestly feel. However, the issue is one of bad apples - in the classical, correct sense of one bad apple spoiling the barrel, I should specify, not the dismissive "just a few bad apples."

Every time a company gets spotted promulgating fake reviews or comments, they're essentially poisoning the well. As clever as we are (or as clever as we think we are), there's really no way to ascertain whether a comment or review is an honest response or well-written PR flak. And exactly because of such backlash, the market selectively ensures that marketers get better at it - it's the companies that pull it off successfully and without ever getting noticed that are going to stay in business, because if people catch on then the gig is up and the promotion probably failed from a ROI point of view.

I'm not saying we should be in an uproar that people are data-mining social media or that we should be suspicious of any and all positive comments, but it's certainly a reason to be skeptical and keep an eye on developments like this.

Re:Datamining Social Media

[mcgrew]

these companies are specifically replying back to specific posters in order to stop the spread of what they call "misinformation"

You see it at slashdot all the time. It's called "shilling". And they do worse here -- they downmod anyone who badmouths their company.

Re:Datamining Social Media

[plasmacutter]

Let me be clear: Corporate America wants to control everything that is said online, and the tools to do it are starting to show up. Companies are starting to employ people whose soul job is to look at social media and respond to negative comments.

You're right, they are responding to negative comments. In fact, I'm impressed with the responses I have received from my ISP (Charter) and SAS. I posted some pretty pissed off comments about Charter last week when my connection dropped (I have business class and I expect it to remain up) and they not only responded to me on Twitter but they also called me to ensure my connection was back up in a timely fashion (I was up before any of my neighbors with residential connections). SAS once contacted me (actually one of their VPs did) via e-mail following a Twitter post expressing frustration about "proc gplot". They wanted to make sure that I got it "to do my bidding".

So while I am sure that what you are saying is true, I have not yet seen corporate America do anything with social media except stop negative comments the CORRECT way--by ensuring the customer is happy. If that sort of customer to company interaction continues in the way it has been, I would say that their attempts are amazingly useful. If they somehow want to change the course of those discussions forcibly or by spamming with positive, I am sure the backlash--especially with those of us that enjoy the way it currently works--will be phenomenal.

and for every one of these there's a microsoft or GoP astroturfer/astro-modder on slashdot poisoning the well.

Just look, for instance, at the dramatic change in the way microsoft related titles are handled on /. for instance. Blatant, extra-fluffy astroturf immediately makes it to +5, anything remotely negative gets subjected to a massive war both against deliberately disingenuous replies and as the real mod community fights with sock puppet accounts to keep the post relevant.

Re:positive or negative, mixed or neutral based on

[gnick]

How would they rate something like this: "The last president sucked big time - and he's a stooge for oil barons!"

They wouldn't rate it at all. As soon as their filters hit "The last president sucked...", the signal to noise ratio will fall to zero and they'll abandon the Tweet.

Troubling technology

[mollog]

What troubles me about this is not the security applications, although there is risk there, too, but the political, persuasive abuse. Innocent sites like Slashdot will be 'turfed' to move public opinion and public perception.

I'll guess that this is already going on.

Re:Troubling technology

[betterunixthanunix]

This has been going on for a while; social networking websites, news websites (with comments), aggregators like /., and blogs are routinely astroturfed.

Re:Troubling technology

[TarPitt]

This is already going on.

Some companies make big money via Astroturfing:

Has Netvocates visited your blog recently

Many bloggers are starting to notice some new referrals from a company called “NetVocates” (mine showed up as coming from arrca.netvocates.com to be specific).

I recently invited a guest blogger (who writes under the pseudonym D. Sirmize) to share his political opinions on my blog. I began to get hits (55 to date) from NetVocates a couple days after his first political post. It would seem that whatever they’re currently looking at is based on opinions of a political nature.

More from http://wordsnotfists.blogspot.com/2006/06/another-inconvenient-truth-netvocates.html and of course you can look up Netvocates' own Web site, where they are strikingly open about their PR efforts

Re:Troubling technology

[numbski]

No it won't! The CIA is here to serve the American people. We^H^HThey would never do something so devious as to astroturf sites like Slashdot!

Re:Troubling technology

[jujuchef]

I couldn't agree more!
Also, the question begs to be asked: What about all those sub-groups that are morally-ethically detested like say k3dd1e pr3n, or murder? How do these groups or 'hits' flag up? If this type of system had 'hits' crop up with these areas, then what happens? Should that just go on un-noticed because company charters have no legal requirement to have ethics/morals that maintain society? Is it any different than witnessing something like the above in person and ignoring it? Franklin's quote is dated when considering our current society.
On the other hand... like the parent says, how far down does that rabbit-hole go when politics, corruption, and social manipulation come in to it. Terrorist or Freedom Fighter? Depends on the viewpoint. Justified or Illegal.. etcetera.
Lastly, is this kind of data going to be of any use? Schnier has a good article about false-positives and the sheer amount of datamining being a mathmatical mountain. Just because we know that subGroupA is slagging off companyB - is that enough to change somebody's actions? Who decides that is a good idea? does subGroupC get special upper-class opinion status?

Re:positive or negative, mixed or neutral based on

[Sebilrazen]

I disagree. Those whom we would label terrorist are the least likely to label what they are doing terrorism. They likely consider it duty or calling. If you want to use generalizations, clichés and stereotypes they may view it as a jihad.

I imagine a post would go something like:

<Deity/> himself will lead them, for they will be doing His work. There will be absolution and remission of sins for all who die in the service of <Deity/>. Here they are poor and miserable sinners; there they will be rich and happy. Let none hesitate; they must march next summer. <Deity/> wills it!

YouTube

[rainmaestro]

Damn, I feel sorry for whoever gets stuck analyzing the YouTube data. One massive 40-hour-a-week rickroll.

Re:interesting

[megamerican]

Bin Laden was, though, a product of a monumental miscalculation by western security agencies. Throughout the 80s he was armed by the CIA and funded by the Saudis to wage jihad against the Russian occupation of Afghanistan. Al-Qaida, literally "the database", was originally the computer file of the thousands of mujahideen who were recruited and trained with help from the CIA to defeat the Russians.

Robin Cook in the Guardian.

It doesn't touch closed social networks . . .

[PolygamousRanchKid]

(It doesn't touch closed social networks, like Facebook, at the moment.)

More like, they're not admitting touching them . . . at the moment.

Re:Our duty as citizens

[localman57]

I don't see why we just let them exist and operate.

Because we have bigger fish to fry.

Data mine this.

[0100010001010011]

Bomb. Obama. Whitehouse. CIA. FBI. Conspiracy. Ruby ridge. Muslim. Jihad. Osama. Israel. Arlington Road. Homeland Security. 747. 777. Pilot lessons. Explode. 9/11. Pentagon.

Think of it like Carlin's "7 words you can't say on TV".

Re:Data mine this.

[stonewallred]

Your post lacked several words such as: suitcase nuke uranium plutonium dirty bomb assassinate Flight Airplane Airport Tickets passports improvised diesel fuel oil fertilizer nitrogen detonators cipher decode infiltrate

Re:Data mine this.

[Anonymous Coward]

Attention FBI, I am staging a coup in about a year or so to take over the presidency.

sincerely, Sarah Palin.

Re:Data mine this.

[dave562]

No no no, you messed it all up. THESE are the keywords you're looking for.

SPP, NAU, North American Union, amero, dollar, fraud, SEC, insurrection, revolt, revolution, bloomberg, goldman sachs, G20

Re:Data mine this.

[MRe_nl]

"The PRESIDENT has the RED NUCLEAR MAILBOMB and an RPG, so he's going to meet with the SMALL POX CRYPTO INFILTRATION team and the SUBVERSIVES from WHITE YANKEE, then ASSASINATE the SECRET SERVICE CLAYMORE MUNITIONS after lunch."

http://echelonspoofer.com/

Domestic spying?

[codepigeon]

I thought the CIA wasn't allowed to do domestic intelligence?

Re:Domestic spying?

[sexybomber]

I thought the CIA wasn't allowed to do domestic intelligence?

They're not, but do you think that's going to be a serious impediment to them doing so anyway? First off, they're going to be trying really hard to keep their intelligence gathering a secret, so you probably won't know that they're doing it in the first place. Secondly, even if you did find out about it, what are you going to do? Sue? They'll claim state-secrets privilege within a couple minutes of you filing your complaint. Now you can't do discovery, and there goes your case.

Point being, "allowed to" is a complete non-issue here. They're going to do what they want, when they want, and there's not a damn thing you can do about it.

Re:Domestic spying?

[TheLink]

The CIA isn't. Some private company is doing it :).

If that's not good enough, I'm sure they can always make some vaguely legal request to the private company to ask another private company/organisation and so on to do the dirty work.

The benefits of outsourcing.

That's why I find it hilarious when the fanatics keep saying small government will be better than big government.

If you really think a small government that outsources all the dirty work to private corporations will be better, you're a fool.

The real problem is quality not quantity. Poor regulation, by the regulators AND by the voters.

Most people don't seem to realize that. I suppose the problem there again is quality and not quantity either... But quantity wins in democracies - and still the ignorant wonder why those in power refuse to educate properly the people who keep voting them back in...

Re:Domestic spying?

[geekoid]

A) this has nothing to do with domestic intelligence in general.
B) Their are circumstance where it is allowed
C) They do keep to their legal jurisdiction pretty well.
D) Other agency's don't like it when soneone staps withing their legal bounds.
E) People ahve sued the CIA successfully.

There are legal protection in place that are adhered tom pretty well.
Homeland security was crated to get around those protections.
That's the agency that needs to be shut down if you are concerned with rights.

Yes, the CIA isn't angels, yes that occasional have gone beyond the bounds, no they are not perfect. All things considered, they do a pretty good job.

Visible moderation

[zapakh]

+1 Influential

Re:Visible moderation

[Tubal-Cain]

If I hadn't already posted in this thread I'd give you a +1 Funny.

!YRO

[jDeepbeep]

Hey! Don't give them any more wood for the fire! :p

Re:positive or negative, mixed or neutral based on

[zoloto]

You do realize with this statement alone you could be considered a subversive act. --just sayin

(now let me check that Post Anonymously button)

Re:positive or negative, mixed or neutral based on

[jamstar7]

I imagine a post would go something like:

[Deity/] himself will lead them, for they will be doing His work. There will be absolution and remission of sins for all who die in the service of [Deity/]. Here they are poor and miserable sinners; there they will be rich and happy. Let none hesitate; they must march next summer. [Deity/] wills it!

And for 100 extra points, which Catholic pope of the 1100s said that to whip up support for a Crusade? Fanaticism isn't restricted to Islam, you know...

Halfasec, there's a knock on my do..

Which country would not do this?

[Anonymous Coward]

Cuba?

Venezuela?

China?

Germany, whose secret police is legendary? (wikipedia: "the case was thrown out in 2003 after it was discovered that a number of the NPD's inner circle were in fact undercover agents or informants of the German secret services, like the federal Bundesamt für Verfassungsschutz.")

The UK?

The EU as a whole?

Face it - surveillance fits the goal whatever your goals are, as long as those goals aren't SOLELY AND NOTHING BUT "no surveillance"

Re:interesting

[TheLink]

The CIA, outsourcing dirty work since 1947.

The CIA: Preserving your freedoms

[ThatsNotPudding]

by eliminating them.

Public information

[geekoid]

is just that, public. This means even the CIA can use it.

Re:Public information

[thehostiles]

but will they improve it, emulate the code and make their own, more powreful version? purchase the servers? I'd be willing to bet an organ that they've got more in mind than just paying someone else to do this

Obviously

[mollog]

Yes, I know that organizations are 'astroturfing'. That is why I used the term 'turf'. That's been going on for quite some time.

What's new and different is governmental use of automated tools. Would it not be fair to assume that secret government agencies, already enjoying unconstitutional immunity, would use these tools to effectively destroy groups who, for example, seek to put limits on the powers of secret government agencies?

And would it not be smart to assume that these tools will be used by politically motivated groups to shout down those brave souls who attempt to stand up for rights of individuals?

We already have media networks (Fox) pushing political agendas. Tools like this will surely be used to push those narrow agendas at the expense of free speech.

Re:What no backdoor into Windows ??

[hey]

> I'm sure they can datamine beyond any privacy settings.

Probably. But I wonder how.

Re:interesting

[demachina]

It isn't "literally "the database"", is "the base", "the basis" or "the foundation". Maybe some CIA guy with a sense of humor morphed it in to "the database" but this looks like just a pretty flawed translation or fabrication.

From Wikipedia.... "The name comes from the Arabic noun q'idah, which means foundation or basis and can also refer to a military base. The initial al- is the Arabic definite article the, hence the base."

Re:interesting

[eihab]

Al-Qaida, literally "the database", was originally...

That is incorrect. The Arabic word "Al-Qaida" means "The Base". To make that word into database, well you need to add "data" and then it would be "Qaidat Al Bayanat" (ka-edit al ba-yan-at).

To my knowledge, no one refers to a database as "base" in slang or formal Arabic.

Re:interesting

[value_added]

That is incorrect. The Arabic word "Al-Qaida" means "The Base". To make that word into database, well you need to add "data" and then it would be "Qaidat Al Bayanat" (ka-edit al ba-yan-at).?i>

I'm left wondering how and whether "All your base belong to us" translates?

isn't that cute, but its WRONG!!

[plasmacutter]

Then Visible 'scores' each post, labeling it as positive or negative, mixed or neutral. It examines how influential a conversation or an author is. ('Trying to determine who really matters,' as Cahill puts it.)

the correct answer is, C - none of the above!.. they're all on social networks so none of them matter.

Re:positive or negative, mixed or neutral based on

[Ghubi]

"I approve of ______" is a positive statement.
"I disapprove of ______" is a negative statement.
"______ is awesome" is a positive statement.
"______ sucks" is a negative statement.

No point of view is required.

So the facebook conspiracy

[TheReal_sabret00the]

does this put that to rest?

Insurance Shifts Costs Too

[weston]

I would prefer to have health insurance, which is much cheaper than a "free zyrtec!" must-carry monstrosity whose sole purpose is to shift the cost of unhealthy people onto healthy people.

Um.... if you're trying to get at might be the problems with first-dollar coverage, I think that's a point I can agree on, but there's no such thing as insurance that doesn't shift the costs of unhealthy people onto healthy people. That's what any risk pool does: the people who don't end up needing it end up subsidizing the people who do. People join anyway because most of us don't know which one we're going to be.

Dear CIA,

[Boldizar]

Dear CIA, You’ll note that the “anarchist” in my political views line under Info is prefaced by “absurdist.” I do believe in “fighting” the State and the Corporation and the Institution and the Boy Scouts and the Neighbourhood Mothers Association, and generally honouring Thomas Jefferson and Ortega y Gasset when he railed against any collective entity as being soulless, he was one of the Founders, wasn’t he?, or at least the blog and Facebook persona that I created for eventually marketing my novel, should it ever come out, does, (blended with the practicality of actually connecting with friends smeared throughout time and space) but only by using boulders (where was I?, how did I get to boulders?) not any sort of dishonourable weapons that could actually succeed at anything other than damaging my big toe. I am fully aware that any attempt to “fight” only strengthens you, the CIA, Bernie Kerik, et al., by increasing your budget and decreasing my rights, and so my fighting strategy is to be as weak, useless and nonconfrontational as possible in the hopes of weakening you. In fact, by becoming a complete corporate drone clock punching jellyfish, I hope to eventually make you fall asleep — which I shall I consider a victory. How did I get to punching jellyfish? Anyway, I apologize in advance for triggering your keywords. Please rest assured that I have no influence whatsoever. Sincerely, http://www.boldizar.com/

Re:positive or negative, mixed or neutral based on

[Nakarti]

No the keywords are almost certainly more like silent, attack, government, agent, TA^D ...