Slashdot Mirror
Northern Light Technology Makes Deal WIth C.I.A.
Llywelyn writes: "The C.I.A. has evidently written up a contract with the group Northern Light Technology to develop a search engine that can sort through the C.I.A's increasing mound of unprocessed data. Unfortunately, one of the consequences of this is that Northern Light's public search engine is fated for destruction later this month. " It's inevitable, IMHO, that some of this happen - the search engine world is overpopulated right now, and with the economic downturn, more and more companies will move to where they can survive.
I -so- want to see the list of the top search terms for that...
"pr0n"
"Where is Osama?"
"missing equipment"
"JFK"
"Castro"
...et multiple cetera...
Lets hope there is no mixup between the two sources of data.... could make it a very popular search engine...
Cruise TT
very first line of the article states it's one of their contractors -- In-Q-Tel
The One Rule Of Chess You'll Ever Need: Don't play someone who carries a kit in their bookbag.
Harry Callahan: you've got to ask yourself one question: Do I feel lucky? Well, do ya punk?
Hogsback
You think they'll let me pay a little extra to make sure my sites go to the top of the list?
I could use the hits..
Carousel is a lie!
If this is the search engine they'll be taking down. I can assure you, no one will care that its gone nor will the CIA ever accomplish anything.
.... will still have access to the site for a fee. The agreement isn't with the CIA for exclusive use.
One of the neater features of Northern Lights was the folders. I liked how they organized the info and let it flow out of your continuous clicking.
What is funny is that Northern Light is actually a better search engine than Google for specific info. Northern Light drills down on the subject only and doesn't take into consideration links to the info as Google does.
I mean, at least they didn't get google.
This sig is xenon coated, and will glow red when in the presence of aliens
Why does Northern Light have to shutter its public operation in order to handle the CIA's content? Surely there has to be a way for them to separate the CIA's stuff from their public index. Or is the announcement of the end of their public service merely coincidental to the announcement of the CIA contract, and not a condition of it?
I too feel there are too many search engines. It is extremely rare that I don't find what I'm looking for on Google, but it's not good to put all your eggs in one basket. While Google may be the best today, there needs to be competition in every market (well, almost every market).
Has anyone ever read the book CIA and the cult of intelligence ?
Anyone notice the blank line at the top of this article?
What's going on here? I smell a censor!
The cia has [deleted] civil liberties
[deleted] Former Director [delted]
without any thought
[deleted]
won't stand for this!
(please don't mod this down if you haven't read the book)
Touch everywhere, even when inappropriate.
the search engine world is overpopulated right now, and with the economic downturn, more and more companies will move to where they can survive.
:-)
according to this article posted back in August.. there should be even more search engines popping up in the future.. most of the ones on that list are still in beta mode.. personally, i like Teoma a lot
"The ones who dont do anything are always the ones who try to pull you down" -- Henry Rollins
There are other search engines out there besides google? ;)
-- Dan
I wonder if this is what's going to be the next hot career -- being able to sort and analyze data. I'm guessing the training would have to be a hybrid of sociology and comp sci in order to wade through all the stats. I'm guesing that since our ability to gain data has outstripped our ability to process it, this is the time that people are going to start trying to make up the balance?
The CIA and FBI (and others) have masses of data so large that their entire staff reading for 100 years couldnt possibly sort it all.
:)
It makes you wonder about the QUALITY of the data they are collecting,
Are they going o index their HOWTO's aka
HOW TO WIRE A CAT WITH A MIC (Seen previoulsy on slashdot)
As they teach you DAY ONE in Naval Intelligence
Everything is simply a piece of a puzzle, expect nothing monumental, no matter how small it is a detail of a bigger picture.
Now without addressing information in this manner and looking for ONE big hit , the process of intelligence gathering is broken.
YOU NEED SOMEONE to al least try to review this data before its all clumped into a selective search engine and forgotten. The CIA has horrible record retention policies in place. My bet 10:1 this is the worst possible fate of this data.
Probably just be better to auction it off to some willing buyer and hire more spooks to gather more data to auction off again, then maybe then the CIA could be considered usefull at least as a govt profit center
Sig went tro...aahemmm.....fishing........
This is a very clever direction to turn in. Given the vast amounts of data that has been collected in recent years, systems like this are going ot become required and more advanced. Perhaps what Northern Light learns while working on this project could eventually turn into the Northern Light 2 internet search engine which will be more efficient and provide better results.
Depending on the stats and reporting capabilities of the new search engine that Northern Light Technology develops, perhaps the CIA could come up with its own version of a "Ten Most Wanted" list.
Of course, this would be a bit different than that of the FBI's list, with a ranking of search strings rather than people. I would predict the top three being:
(1) where hell Osama bin Laden Cave Country
(2) bad-guys terrorists attack United States not nice people
(3) Natalie Portman hot grits petrified
Beer is proof that God loves us and wants us to be happy. -- Benjamin Franklin
A few months back I pointed out that the economic model of search engines is a bit flaky. Here's yet another carcass in the search engine war to prove the point.
Intriguingly, according to some the growth of the web is slowing. The last search engine to index the web before it reaches quiescence is the likeliest candidate for survival.
IMHO originally meant "In My Humble Opinion"
The lack of Humble Opinions on the internet has changed the use . . .
This is a plug:
Lots of companies have vast amounts of data, both text and otherwise to which they need access. The company I work for, Maxim-IT, Inc focuses on search technologies for non-structured data (like CAD files and the like).
As much as the focus tends to land on it, information gathering is not by any means the weakest link in the intelligence system. Probably we hear most about it because
a) it is glamourous (think James Bond), and
b) it often affects our civil liberties.
But the real problem with intelligence is the processing of retrieved raw information. They gather so much of the stuff it's extremely difficult to sort through it to figure out what's relevant and what's not.
That is why whenever something bad happens (like Sept 11) the intelligence community looks sloppy. In retrospect they can dig out wads of unprocessed information that would have given advance warning of the disaster. Then they take a lot of heat for missing it, even though they may not really be at fault. Sometimes it's a matter of finding a needle in a haystack.
It's a little more interesting to geeks because it's an issue of pure computer science. Processing raw data into meaningful information is computing at its best.
But developing better algorithms as a response to a national disaster is never going to be a solution that catches the public's imagination.
For those non-English speakers, "IMHO" means "In My Honest Opinion".
Non-English speakers are probably going to have more trouble with the rest of the words in that sentence...not to mention your explanation.
It seems that Northern Light may be the magic lantern they have been looking for. Perhaps they will find some illuminating facts nestled in their databanks.
"Unfortunately, one of the consequences of this is that Northern Light's public search engine is fated for destruction later this month."
According to fuckedcompany.com, they will be converting their search-engine service to a paid-only model.
Too bad. Northern Light was my primary search engine for a long while, and was my secondary recently (no need to say what #1 was.)
The really nice thing was Northern Light's categorization of hits. It was often far more useful than trusting Google's "sort by relevance"
Damn this is sad.
Trolling is a art,
I think that's "in my HUMBLE opinion," actually. And what the hell, man - the abbreviations are just part of any English language forum. And they're just as mystifying to English-speaking newbies as to anyone else.
It Is the Nature of Information to Transgress Artificial Boundaries
You make it sound like working for the CIA is some odious move of last resort. Perhaps the management and staff of Northern Light is excited about working with the intelligence agency. Perhaps they see it as a way to help their country. Perhaps the processing of terabytes of data is a thrilling prospect from a purely intellectual point of view.
The standard /. dislike of all things governmental is not necessarily mirrored through all geeks.
In-Q-Tel is a venture capital fund set up by the CIA to invest in technologies that may serve the intelligence community. This is not a CIA contract for search technology. The CIA was Verity Inc.'s first big customer and as far as I know, they're still mostly using Verity internally (I managed Verity Internet products for years).
And the article misspelled Gilman Louie's name...
Nick
This is just not true. Over the last year, more and more search engine companies are effectively consolidating - by licencing "search engine technology" from another company. The real down side of this is that the more popular (popular by the licencing...not by users) are a "pay" engine, whereby companies can move their listings higher in the rankings by paying a fee.
This has two main side effects.
One, there are a lot of search engines out there that are really the same search engine. Same query, same results.
Two, when you search with them, you're not really getting what you asked for, but what someone payed for.
I understand why companies are doing this - there isn't a really strong revenue model for search engines right now; banners don't cut it.
I suspect that soon, good search engines will just be a (hopefully) inexpensive pay site, where you pay $30 a year and can use that search engine.
/dev/psychic: No medium found
There's a lot of stuff out there on the web. So much so that the question now is to find exactly the stuff you need. It's almost as if you need an observor monitoring your actions, constructing semantic nets and offering suggestions.
Microsoft is making a stab at this using SmartTags. Of course, the intent there is not to make the web more useful, but more Microsoft.
Then there's the w3 symantic net (http://www.w3.org/2001/sw/), and finally there's the grand daddy of them all: Xanadu (http://www.xanadu.net/).
No real thoughts here, just an observation.
668: Neighbour of the Beast
You are a bad man. A very bad man.
Good thing you posted that as AC, or I'd be putting your name on the witness plate of my psionic feralimnal lypancthrocizer and thinking up some very bad things to happen to you!
"This one's better, it goes up to 11!"
Google is incapable of exact phrase searches, which makes it decidedly second-rate compared to Altavista.com.
(Sure, you can try to do phrase searching by putting all kinds of + symbols in your phrase, but even this does not work perfectly... and once you have doctored phrase A into phrase B in order to get matches to phrase A, you certainly are not doing exact phrase searching! )
Pretty stupid if you ask me. I doubt they are then intelligent enough to change the spider agent string and route the spider through various innocent proxies to disquise it is the FBI spider...
I mean, what does the FBI do? Enter in "warez" into a search engine and go out and bust heads depending on the results returned?
You know what I really thin@~.~.~.~.~.~~~..~~#~~
NO CARRIER
If you read the linked story, this doesn;t seem to have anything to do with "processing the CIA's data" at all... It's funded by the CIA's VC arm, but is going to target "large companies and government organizations". This could mean anything, but unless you're a real conspiracy theorist, it's hard to see this as something exclusively CIA-related.
Ah, computer dating -- it's like pimping, but you rarely have to use the phrase "upside your head" -- Bender
I had completely forgotten about Northern Lights until I read about their GeoSearch feature. I gave it a shot and loved the results. I know about lasso (or whatever) but didn't like it. Is there something else out there as good as Northern Lights Geosearch?
At the end of last year Kuro5hin went down. Part of the reason for this was a post in which one person detailed how easy it would be to whack a certain high ranking official in office right now. Soon after that post was made the Secret Service was banging on that guys door. The internet is not a secret place, it is not an anonymous place, and lots of people are interested in what happens on it. Don't delude yourself into thinking otherwise.
The middle mind speaks!
Anyone else remember "Snow Crash"? Where there's all that information-gathering so that "it got so there was no real difference between the CIA and the Library of Congress, so they merged and kicked out a big stock offering". Book was before the internet became big, but this sounds like the way it's going to go! :-)
Grab.
I suspect that soon, good search engines will just be a (hopefully) inexpensive pay site, where you pay $30 a year and can use that search engine.
But can you imagine all the bad possiblities if they were able to actually tie all your searches together and see WHAT YOU searched for? Sure they can do it by i.p. or cookie, but an actual account, probably verified by credit card?
On the otherhand, a search engine is a basic need to use the internet. And I'd be quite surprised if some of them didn't start heading this way, REALLY CHEAP though. Incidently I don't know exactly when google.com became my ONLY search engine, replacing altavista.com, but it happened. Probably because of the excellent results (not perfect though) and the light interface.
On another note, I get the BEST referrals from google.com to my site. I get the MOST referrals from msn.com to my site. I say BEST from google.com, because the people that find my site through them, most likely want to see my site, and end up staying. MSN's referrals are usually pretty broad topics.
Northern Light downgraded its relevance about two years ago when they weren't generating enough capital and decided to charge its visitors for "premium" articles. At that point, NL effectively ceased to be a search engine and became an information broker. Nothing's really changed.
"Folks just call him Buckethead." -- Les Claypool
NL is my second-favorite search engine (second to the mighty google), due in large part to its massive index and its ability to use complex Boolean (try that on google). When you really need to drill down and/or cut through a major noise:signal ratio, there's no better engine on the web, in my experience.
Anyone know of another decent engine with a good boolean implementation?
Nice things are nicer than nasty ones.
Does this mean that the Northernlight NASCAR series will be ended or does it mean that we'll see TOP SECRET supercars being raced? If anyone has more information please let us know.
When you're looking for information, check out The Jargon File
// abbrev.
IMHO
[from SF fandom via Usenet; abbreviation for `In
My Humble Opinion'] "IMHO, mixed-case C names should be avoided, as mistyping something in the wrong case can cause hard-to-detect errors -- and they look too Pascalish anyhow." Also seen in variant forms such as IMNSHO (In My Not-So-Humble Opinion) and IMAO (In My Arrogant Opinion).
Anomaly
PS - God loves you and longs for relationship with you. If you'd like to know more, please email me directly.
But Herr Heisenberg, how does the electron know when I'm looking?
It has already happened. There have been invisible cars on the tracks for several months now. The makes of these cars have been Daihatsu, Bricklin, Tucker, and SEAT. "Team DeLorean" is reportedly negotiating with the ever-reclusive Elvis Presley, who might be tempted onto the NASCAR track if he can be in an invisible car.
They technically do not violate rules against "seeing cars other than Big 3 on the track" since you never see them anyway.
Watch the next race and look for mysterious skidmarks, and McDonald wrappers that appear in mid-air and fall to the ground without seeming to have been thrown from a car that you see.
Reporter: You mean, the procurement officer paid $6,000,000 for grep?
Guy with face hidden: Yes.
Reporter: What happened to him when they found out?
Guy with face hidden: Well, they were going to hit him with a hammer, but they decided that would be hypocritical...
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
When we hear of the Library of Congress overthrowing a thirdworld despot, we will know that the merger is complete.
but the 6 million was paid in Flooz, not real dollars. Saw the man walking with a wheelbarrow full of 20 meg hard disks loaded with Flooz money walking down the street.
I couldn't care less about the search engine going away. I just always liked the AP News Feed they had there.
I hope it will still be available. I've tried others but not found a better one. Anyone else?
pay for it.
sulli
RTFJ.
with the CIA.
I think his name was Fidel..
haha..
gallix
"The sum of the angles of that rectangle is too monstrous to contemplate." --Commissioner Gordon
northern lights ; cia
That's a funny connection of ideas...
ePairie.com seems to have some "reliable gossip" (what is that anyway?!?) to think NL might be bought out by a Chicago based company. The "activity" might actually have something to do with the CIA deal -- or maybe a Chicago company is buying the CIA :)
http://www.eprairie.com/news/viewnews.asp?newslett erID=3261
I give up, some one get me when Elvis returns...
Actually, the CIA has helped overthrow third world despots or has helped prevent or thwart the efforts of colonial powers to attack thirdworld countries. It has nothing to do with "politically connected companies": this is a myth long ago dispelled and is only believed by a politically uninformed 1 to 5% or so of the population.
Better yet, a CIA version of the Google Zeitgeist.
Any sufficiently well-organized community is indistinguishable from Government.
second link is our old buddy goatse
said David Seuss, chief executive officer of Northern Light.
If I had the last name Seuss, I would have to go for a doctorate.
what with Google, Yahoo, and ... MSN?
FWIW, NL is second-best in my book.
Not as good as Google; better than Altavista.
Altavista used to be second best but NL overtook it.
The problem (for NL) is that second best is not good enough: I only use Google.
Of interest (???) to slashdot readers: isn't NL one of the last VMS diehards?
"Never bullshit a bullshitter" All That Jazz