Slashdot Mirror
DARPA Seeks the Holy Grail of Search Engines
coondoggie writes "The scientists at DARPA say the current methods of searching the Internet for all manner of information just won't cut it in the future. Today the agency announced a program that would aim to totally revamp Internet search and 'revolutionize the discovery, organization and presentation of search results.' Specifically, the goal of DARPA's Memex program is to develop software that will enable domain-specific indexing of public web content and domain-specific search capabilities. According to the agency the technologies developed in the program will also provide the mechanisms for content discovery, information extraction, information retrieval, user collaboration, and other areas needed to address distributed aggregation, analysis, and presentation of web content."
Like what Vannevar Bush talked about in the 1940s? That's odd: I've been assured that only space exploration can develop computers and technologies as a "spinoff". How is it possible these ideas predate Sputnik? I think it's a conspiracy.
When they can just google it?
Interesting, but does it run Beta?
Nevermind, forgot, BETA doesn't even run!
who is really commissioning and funding this little project? anyone care to guess which three letters are involved?
Gee, I can't wait to participate in an echo chamber made up exclusively of nerds that got pissed to high hell when a UI redesign was _proposed_ on a site thats totally fucking free to them and will still work the same way in the future with regards to content (gripes about content aside)... What a joy it will be:
[dream twinkles]
"The scientists at DARPA say the current methods of searching the Internet for all manner of information just won't cut it in the future" What the fuck are they thinking? If they can't just use GREP to find what they need they must be some serious federal fuckups! Get Ron Paul down here to stop this!!!!!!!!!11
[/dream twinkles]
Ah what a lovely place, indeed.
"Disable Advertising As our way of thanking you for your positive contributions to Slashdot, you are eligible to disable advertising." ORLY? As an audience member, I have not contributed in a long time.
This is my signature.
this capability exists?
FTFY
You have put a name to my pain, sir/madam.
I realized that the "Beta sucks!" chorus was bothering me, not just because it was a huge overreaction to, as you pointed out, a _proposed_ change to a _free_ site.
The thing that bothers me the most is that there seems to be almost a competition to show who can be the most obnoxious in registering their objections... from people who went through "gamer shaming" in high school, insults about nerdiness through college, and what not. It seems that these are people who have wanted to strike back at a handy target.
Like the French Bourbon dynasty when they were restored to power after Napoleon's defeat, they seem to have remembered everything, forgotten nothing, and learned nothing.
Strike while the irony is hot! -- The Freethinker
... that Snowden didn't have good enough tools available.
This probably has g00GLe scrambling, running around like a chicken with its head cut off screaming, "We are the holy grail of search engines. We are IT." HaHaHa!!
Second - we (common netizens) may welcome the sort of information availability DARPA is seeking - sort of like the scifi future where you just ask the nearest terminal whatever you want to know and magically get the answer you need - but there are lots of bad people still running around on this planet (scamsters, governments, jilted ex-lovers, religious extremists, etc.). The problem isn't the technology, the problem is our ability to handle it.
I very much suspect DARPA may be onto something. I wonder if it will be as beneficial as the WWW has been.
FBI M-O-U-S-E. next best thing to surfing along looking over your shoulder, taking notes, texting the US Marshals Service what they need to subpoena before driving over and clapping on the leg irons.
if this is supposed to be a new economy, how come they still want my old fashioned money?
The sooner you leave, the better. I'm fucking tired of these goddamn bitchfests about the Beta UI.
You're a fucking idiot. Slashdot isn't about the articles and it isn't about the fucking pathetic summaries. Slashdot is about the user-generated content. Slashdot is the comments.
Now, you think all that time spent by knowledgable members posting all that good stuff comes free? People invested their time, care and attention here. Slashdot isn't a fucking TV. Slashdot is a community.
The reason the response was negative and loud is because the beta makes it harder to follow threads. It makes it harder to follow discussion context. It makes it harder to comment within the discussion context.The beta redesign ruins the community experience. Dice is trying to turn Slashdot into another Digg or Reddit. Soon, you'll be flooded with ads you can't turn off, looking for the useful, informative and interesting comments that used to be posted here. Fuck that shit.
TL;DR: FUCK BETA!
Just how many buzzwords can we fit in here, anyhow?
Google already does domain-specific indexing, certain sites get indexed faster or deeper than others based on a number of secret rules.
For site specific search prefix your query with "site:foo.com"
"When information is power, privacy is freedom" - Jah-Wren Ryel
That should probably be Darpa's response to Google buying Boston Dynamics.
answer: google. now, where's my grant money?
Beta Rocks you hoser. eh?
Long live Beta! Beta Rocks
I love Beta. It's the best thing that has happened to slashdot since you left.
You're a fucking idiot. Slashdot isn't about the articles and it isn't about the fucking pathetic summaries. Slashdot is about the user-generated content. Slashdot is the comments.
Now, you think all that time spent by knowledgable members posting all that good stuff comes free? People invested their time, care and attention here. Slashdot isn't a fucking TV. Slashdot is a community.
The reason the response was negative and loud is because the beta makes it harder to follow threads. It makes it harder to follow discussion context. It makes it harder to comment within the discussion context.The beta redesign ruins the community experience. Dice is trying to turn Slashdot into another Digg or Reddit. Soon, you'll be flooded with ads you can't turn off, looking for the useful, informative and interesting comments that used to be posted here. Fuck that shit.
TL;DR: FUCK BETA!
Now, at least some reasoned, if seemingly over-wrought, debate.
You argue that the content, being user-provided, will go away if people are unable to see the content organized in a sensible fashion. Okay, I can buy that. I also hadn't considered that since the content is provided by users, they might have a reason to be a bit more passionate about how their thoughts are presented.
What I don't get is the need for insult to anybody who disagrees, or the call to leave Slashdot in droves even after they acknowledged that the beta was not working out and removed it.
And, again, I submit that you have validated my other point. Bitter vituperation towards the failed beta, even after it's gone, may do as much to drive away readership as the beta
Strike while the irony is hot! -- The Freethinker
So do I... One that is robust and can't be censored by anybody, ever.
“He’s not deformed, he’s just drunk!”
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, but if it's truly keeping with recent trends in other products towards removal of commonly-used features and insertion of empty glitz, as some anti-Beta whingers have asserted, I can see why there is cause for a major uproar.
We have a lot of self-professed Libertarians here, they chant the mantra claiming that if you don't like service pick a competitor. Consider Digg. That's basically the route that Digg went down, its users fled and Digg dug its own grave. There aren't so many competitors for Slashdot that its users want it too to be ruined, so they're trying to ensure that it doesn't go that route.
I don't often counsel this in the IT world, but if it ain't broke...
Just search in Google for "holy grail of search engines". Job done.
return 0; }
Now, at least some reasoned, if seemingly over-wrought, debate.
You argue that the content, being user-provided, will go away if people are unable to see the content organized in a sensible fashion. Okay, I can buy that. I also hadn't considered that since the content is provided by users, they might have a reason to be a bit more passionate about how their thoughts are presented.
What I don't get is the need for insult to anybody who disagrees, or the call to leave Slashdot in droves even after they acknowledged that the beta was not working out and removed it.
And, again, I submit that you have validated my other point. Bitter vituperation towards the failed beta, even after it's gone, may do as much to drive away readership as the beta
The layouts in Beta are evolving to look more and more like the classic site, with every passing thread that gets bombarded by classicists. So every frustration the gripers have are disappearing, except perhaps from the current lack of an "anonymous coward" checkbox to make trolling easier. For that you have to find your way all the way to the bottom of the page, click the classic link, and post anon like usual. Sounds fucking rough, doesn't it.
Slashdot is the comments.
Not anymore...
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, but if it's truly keeping with recent trends in other products towards removal of commonly-used features and insertion of empty glitz, as some anti-Beta whingers have asserted, I can see why there is cause for a major uproar.
We have a lot of self-professed Libertarians here, they chant the mantra claiming that if you don't like service pick a competitor. Consider Digg. That's basically the route that Digg went down, its users fled and Digg dug its own grave. There aren't so many competitors for Slashdot that its users want it too to be ruined, so they're trying to ensure that it doesn't go that route.
I don't often counsel this in the IT world, but if it ain't broke...
If it ain't broke, it must run at least as well as Friendster.... Right? Or maybe it runs as well as Altavista? Geocities? Need some more examples of services that failed to change at the right time? No one said "f--- this beta friendster s---, I'm going to facebook because I dont like this any more", did they? Nope, they said "well hell, 10 of my friends just joined facebook, what am I doing on friendster?" and the rest is history.
Still can be improved. Take the following 'search' which is more computational than search-y and something nothing today gets close to being able to provide an answer:
- How often has it rained on Jan 15th, in Seattle WA, over the past 50 years.
The answer being something like 40 out of 50 times it has rained on that day etc. Unless someone has literally written that phrase, or close to it, you will be stuck computing that data yourself. We have computers for this right :)
but that site is even gayer, holy shit, I didn't think something could beat out slashdot in gayness, but congrats sir , you've really outdone the gay this time, fagot
Sometimes, as you read the RFP document you can tell it's been written with a specific proposer in mind, and everybody else gets a short window to come up with an idea + jump through endless list of government hoops.
2 things:
A First, on the bitter vituperation, as one who quite unabashedly shared my thoughts about how and why the new design was/is bad: much of the speech was strongly voiced, but it was simply the truth. If Dice kills the site layout and function, I will stop coming here. That's not a threat so much as a fact that I feel dice should be privy to. I don't want them to kill the site, and I'm sure they don't want me to leave. Bluntly pointing out the lose-lose seemed to me the most direct way to communicate my feelings and affect change.
As an addendum to the above point, there is something in the culture here (and not just here) where humor and bravado is acceptable and even expected. Perhaps you have not been properly acculturated to Slashdot geek, subspecies of Internet geek. By your very presence here you should know what to expect in this regard.
B Second, Dice failed hard on this gambit. They didn't even bother to try at "reading the wind" so to speak. Hopefully they have learned from this and will at least make an attempt to understand the community before making fools of themselves in the future.
You can say that the community reaction harms the site's image, and that might even be true. On the whole though, the community doesn't care about Slashdot's image. We have this space, and it is ours, and it is good. Only Dice is concerned with image because they are trying to figure out how to grow us and monetize us. This is the ugly truth behind the whole situation, and I posit this is where where much of the "bitter" response really came from. It's creepy and sad to see a culture that you love slowly destroyed by a soulless entity that doesn't care about you or your community so much as it cares about using you and shaping you into something profitable.
In this instance, by failing to read the wind, Dice bared this truth to all of us; reminded us that this community isn't really ours so much as it is theirs. Absolutely that was a hurt to the community.
What I don't get is the need for insult to anybody who disagrees
You take the good with the bad.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
In this instance, by failing to read the wind, Dice bared this truth to all of us; reminded us that this community isn't really ours so much as it is theirs. Absolutely that was a hurt to the community.
Now that's something that I can take to the bank.
You see, I lost that aspect of it when I was seeing people posting line after line of "Fuck Beta", often anonymously, and often seeming to offer insults to anybody who felt differently.
You, on the other hand, have answered my questions without questioning my intelligence, or heaping scorn and abuse on my head for daring to have a different opinion, or generally resorting to name-calling, so I don't think you qualify for "bitter vituperation". <grin> Rather, you qualify as someone who helped me decide that this site is worth staying on. Thanks!
Strike while the irony is hot! -- The Freethinker
Freedom! Freedom! Freedom!
Sorry, what with all the negativity around here lately, I couldn't resist.
There is nothing wrong with the they way searching is done today.
This is merely a attempt to constrict the web. If you start indexing search results by TLDs, you effectively create a filter that can be handed over to the powers that be to easily channel all the "bad" hush hush content to /dev/null
The world wide web and its freedom depends on chaos.
http://www.dogpile.com/
metasearch
Insanity: doing the same thing over and over again and expecting different results. Albert Einstein
Maybe I'm wrong but... I'll bet DARPA's winning technology will be the one that redirects all search queries to the NSA. So they can tell us what to think.
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, but if it's truly keeping with recent trends in other products towards removal of commonly-used features and insertion of empty glitz, as some anti-Beta whingers have asserted, I can see why there is cause for a major uproar.
We have a lot of self-professed Libertarians here, they chant the mantra claiming that if you don't like service pick a competitor. Consider Digg. That's basically the route that Digg went down, its users fled and Digg dug its own grave. There aren't so many competitors for Slashdot that its users want it too to be ruined, so they're trying to ensure that it doesn't go that route.
I don't often counsel this in the IT world, but if it ain't broke...
If it ain't broke, it must run at least as well as Friendster.... Right? Or maybe it runs as well as Altavista? Geocities? Need some more examples of services that failed to change at the right time? No one said "f--- this beta friendster s---, I'm going to facebook because I dont like this any more", did they? Nope, they said "well hell, 10 of my friends just joined facebook, what am I doing on friendster?" and the rest is history.
There's one critical difference, though. One group lost membership by failing to adapt when competitors started sucking their members away. The other group, however, chased members away. Slashdot seems to be falling into that second category.
Google, Deepmind, etc.
"If any question why we died, Tell them because our fathers lied."
Unicorns... you forgot the Unicorns.
"If any question why we died, Tell them because our fathers lied."
Today unstructured information search findability is limited by the Shannon Limit, this is a fundamental physical limit since all pattern search engines are statistical decoders. Google does a little better than the Shannon limit by looking at which search results are selected, this a communal intelligence technique based on how we "vote" for the right result. Unfortunately this only works well for high volume searches, that's why Google work's best if you know exactly what you're looking for or you're looking for what everyone else is looking for. In commercial search Google is losing out to companies like Amazon that are investing in editorial findability enhancements using lots of work by folks on Mechanical Turk.
EBay is actually doing some of the most interesting work in findability research, that's because they have the 'everything' search problem, this is harder than the popular search problem the Google is mostly concerned with now. Google seems to have given up on technical, scientific, commercial, practical, professional, industrial, and other sorts of specialized search. This sort of information usually has a very low Shannon Limit, that's why professional search usually uses extensive manual indexing such as that provided by Westlaw.
The holy grail of machine translation is automatically extracting the exposition structure (rigor, rationale, rhetoric) from texts, but nearly no progress has been made so far despite decades of research, and here again the problem is the Shannon Limit. Presumably this is the problem that MEMEX must solve in order to succeed, but this can't be solved by machine intelligence alone,
that's a physical impossibility, so even fashionable techniques such as deep learning are out the window.
In the publishing world there are techniques that turn exposition structures into texts, this is authoring automation. This could be used to generate sample texts for some target search, and machine intelligence can be used to score matches between generated samples and search texts. In this way it might be possible to automate educated guessing of the exposition structure, and it partially gets around the Shannon limit with systematic editorial augmentation.
If they can't find the Holy Grail of search engines, I can direct them to this alternative that is just a Grail Shaped Beacon of a search tool instead...
This space unintentionally left blank.
This sounds like progmatically generated RDF(http://www.w3.org/RDF/), and could be handled via OWL(http://www.w3.org/TR/owl-features/)... Current search engines and metadata focus on content (which makes sense), but leave out context. W3C have been supporting research on Ontological networks, where semantics are collected and linked. The W3C concept is self-assigning meaning to the content. DARPA's Memex sounds like they are assigning meaning via a 3rd party. Less privacy, more meaning.
Have one without "sponsored ads".
Google has produce less useful hits over the last five years, as its advertising income skyrocketed. Search for something, and add -"photo", and you'll still get photographer. Try mens riding boots -women -womens -women's, and you'll probably get a "sponsored ad" for women's boots, with the letters "men" bolded.
This isn't even counting Target, which will claim it has anything you're looking for, but if you follow the link, oh, no, sorry... but we have !!!!!
Add the way librarians used to search, not just with AND and OR, but also "within so many words", so that men's leather boots, and men's riding boots and men's fashiondesigner wonderful boots all are returned, if you tell it men's within 3 words of boots.
A lot of the tech is already there... but ad revenue is, I mean, what we really want, right, rather than the service you're purportedly offering?
mark
My screen just came up in beta, slow to load & impossible to read comments. Tried clicking on "classic" at the bottom, it's been trashed also. I logged in today for the first time in ten years just to say Beta Sucks and I will be finding another site to read every morning with my coffee. I will be canceling my subscription...so sad but this "new" look with its intention run by greed and ego is NOT why I signed up for slashdot many years ago.
And then, let it find luckiest person in the world.