Slashdot Mirror

A couple of years ago, I went to the H2k2 conference here in New York City. I saw a fascinating talk there where I first heard the term "deep web" and some of its ramifications for national security. National security was very much on our minds at the time being only roughly a mile and a half from what we call "Ground Zero" (never liked that term).

The guy giving the speech claimed that he was a retired FBI agent and seemed to have a great deal of insight into the inner workings of national intelligence. As pointed out in the article, the speaker made the same claim that search engines only gleaned about 1% of the total information on the web. He recommended a tool called Copernic (as well as one other one that I can't remember right now) that bills itself as a "deep web" search tool. But all it appears to do is assemble the results from a bunch of other search engines. I don't recall it ever returning anything significantly "deeper" than what your average google search can yield, however.

Back to the topic of national security, he made mention that terrorist communities are thriving on the fact that only 1% of the total amount of information on the web is readily accessible. All kinds of information that would be beneficial for the NSA to know is just plain inaccessible.

He also faulted the intelligence communities for hiring "blonde haired pretty boy" college graduates, fresh out of school to analyze data in foreign languages instead of hiring local speakers. A 4.0 linguistics student will still miss out on a lot of the nuance to a conversation that a native, say Pashto, speaker will clue right into. Of course, the argument could be made that at least the "loyalties" of an American college graduate are almost guaranteed to be in the right place you can't ignore that he/she will be blind to much of the subtext of a conversation in a foreign language.

A little offtopic, but more alarmingly a point was made about the lack of digitization in the NSA of intelligence documents. Meaning that an agent will typically risk life and limb gaining access to a piece of information, who will then pass that info to a "runner" who places it in an "orange envelope" to signify its classified status. Then that same orange envelope goes into a locked filing cabinet where a good 7 or 8 times out of 10 it never sees the light of day and no attempt is made to analyze it.

But such is the challenge of the modern age. We are drowning in all of the information to produce. Vannevar Bush addressed this issue with astounding clarity right after world war II.

Quoth the Doctor:

"There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. The investigator is staggered by the findings and conclusions of thousands of other workers--conclusions which he cannot find time to grasp, much less to remember, as they appear. Yet specialization becomes increasingly necessary for progress, and the effort to bridge between disciplines is correspondingly superficial."

...and...

The difficulty seems to be, not so much that we publish unduly in view of the extent and variety of present day interests, but rather that publication has been extended far beyond our present ability to make real use of the record. The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships.

We are dealing with this problem (access to the information we produce) to a far greater extent than at any time in human history. The web, which was at one point designed and intended to be a more effective way to deal with and disseminate the oceans of data produce, has little more than square rigged ships to skim its surface.

I wasn't aware that you needed to download special software to run this Google search application.

I think the application comes into picture via the Google Toolbar and also the need to somehow organize all the Google Services & Tools. & Google has also gotten into one-click Blogging via Blogger.

In addition there are tools that visually organize the Google Search results, SearchDay - Visualizing the Web with Google - 8 January 2003

When you start having a book called Google Hacks , you know that there are a lot of HPI's (like API's but for H-Hacking), you know that there is a better way to offer access to these hacks via well organized tools. That is the form and function of the application.

Of course there are other applications like Copernic ( a longer listing here Search Tools), but I think the current applications have miniscule following. What will come from Microsoft or Google will flood the market.

My tips:

Clean up your desktop.

• If you have more than 5 files on your desktop, create a scratch folder (preferably on a RAMDISK). Create a single shortcut to that scratch folder.
• If you frequently need some of those files, create a second folder named today, frequently-used, or similar, and add a shortcut to it.
• Move frequently used application shortcuts into the quickbar. The quickbar is also a good place for applications that accept files via drag-and-drop. Drop an HTML file onto the shortcut to the currently-not-windows-standard-browser icon in the quickbar and the browser will start with the dropped file.
• Move less frequently used application shortcuts into the start menu (they probably are already there).
• Remove shortcuts from the desktop that you don't need. What was the last time you started Real Player, Quicktime or similar by doubleclicking the shortcut on the desktop instead of a media file?
• Organize your start menu. Rightclick it and choose "Explore"
• Use keyboard shortcuts for frequently used applications.

Several years ago, I found a tool called WinKey, allowing to create a huge ammount of keyboard shortcuts that do not interfer with application-specific hotkeys. Imagine a keyboard that has 80 or 100 extra buttons for applications. Weird? Useful! Just hold down the Windows key and type almost any other key to start one of your 50 most used applications.

My current shortcut mappings are:

Windows-A = ACDSee
Windows-C = cmd.exe (DOS-Box)
Windows-G = http://www.google.com/
Windows-I = Internet Explorer
Windows-N = better than Netscape: Mozilla (Windows-M is used to minimize all windows and can't be used)
Windows-Shift-N = the original Netscape 4.7 - less frequently used, so the shortcut is more complicated
Windows-O = Opera
Windows-P = Putty Menu (selfmade)
Windows-Q = Quirk for Ultraedit (Windows-U is used by usability tools and can't be used)
Windows-V = VNC viewer
Windows-W = WS_FTP
Windows-X = access the Exchange server: Mozilla Mail!

(You are not limited to letters: Numers, arrows and F-keys also work, and you can combine with Shift, Alt and Ctrl.)

And of course, I use some of the standard hardcoded shortcuts:

Windows-E = File Explorer
Windows-M = Minimize all Windows
Windows-Shift-M = undo Minimize
Windows-R = Run command
Windows-Break = Break Windows using the System Properties ;-)
Windows-F = Find files or folders

Less frequently used:

Windows-D = Show Desktop
Windows-Tab = Switch Tasks in the taskbar
Windows-F1 = Windows Help Windows-U = Utility Manager (Windows 2000) - starts Narrator and other usability tools (Winkey does not know this shortcut)

Executive summary: Click count reduction and mouse movement reduction by using short ways for frequently executed tasks. (This is very similar to what packers like winzip do. See also "poor Huffman coding" in Apocalypse 5.)

Tux2000

Endurapro 104: a buckling-spring (aka Model-M) keyboard with Windows keys and integrated TrackPoint.

Buckling-spring. There are already numerous other comments on the greatness of Model-M keyboards.

Windows keys. Okay, some of you hate them with a passion. If they bother you that much, pckeyboards sell versions without 'em. Me, I think they're useful. Use an application such as WinKey to assign keyboard shortcuts to Windows-key combinations. (Examples: Win+i to start a web browser, Win+v to open the volume control, Win+c to open the control panel, Win+down to minimize the current window, ...)

Integrated TrackPoint. Some of you hate these things too. "A normal mouse is so much better," you say. The beauty, though, is that you can use it with a normal mouse. I have both a TrackPoint keyboard and a normal mouse hooked up. If I need to do a lot of mousing, I'll use the mouse. If I want to do just a little bit, though, I can use the TrackPoint to move the mouse cursor and never have to take my hands off the keyboard. It's excellent for coding. All keyboards should be like this.

Take a look at a company called Copernic, they have been doing this type of so called better searches for years now.

Btw, on 2000 and XP (maybe 9x too), you can assign a shortcut to the command prompt, say Ctrl+Alt+S, so hitting that will get you a command prompt quickly

Or just install Cygwin and use your favourite Unix shell.


Or, do both! And get the free program WinKey so you can bind shortcuts to that windows key (yes! it IS useful!).

Win-S is one key less effort than Ctrl-Alt-S.

Damien

Doesn't Windows+M minimize all under XP?

That sort of functionality wouldn't be out of place in a modern OS.

Jon

While they (any "they) probably can probably get most of them, they probably can't get them all.

On Windows, there's Copernic 2000 What's good for Windows and on the Web?

From a business standpoint, remember that what sold us on google to begin with and why we spread the word about it was that it was unbiased and effective, i.e. likely to come up with what we are looking for whether the subjects liked it or not and regardless of anyone's value judgement on the content. We trusted that this was all automated and done by machines incapable of evaluating whether we should have access to it or not. This gave it major "cool" factor... and we are part of the group who each told a few of our friends.

While Google's technology is continously improving, it's "cool" factor is vanishing rapidly.

While altavista went into the dumper for other reasons (google was "cooler"), the speed with which google replaced altavista speaks to how our perceptions get translated to public action. Not surprising, we're usually the people our non-cyberliterate family / friends / employers ask "How do I find things on the Web?"

While I think we all respect their technology, our concern is with human agencies playing games with the search results we depend on.

If we can't depend on Google for honest results, most of us will go on to something else, and we will probably be taking a large chunk of the user community as a whole with us.

It doesn't answer the question, or provide any answer at all. The search engine summarizer, Copernic Agent has an ask question feature that actually works. I've said it before, I'll say it again. Ask Jeeves sucks! Worst search engine ever. Don't waste your time with Ask Jeeves.

• hit list clustering (e.g.: Vivissimo),
• text Summarization (e.g. Copernic Summarizer),
• translation (e.g.: Babelfish,
• Question answering (e.g.: TREC)..

Walk down the hill son, and scr*w them all!!!!

I use a program that polls all the search engines you wish... it's called Copernic

It's a cool tool because not only does it get and prioritize links from a bunch of the top search engines, it also has a mode that will go through the list that it returns and get rid of any dead links.

I'd be interested in anybody's experiences with similar software, btw.

There are a couple for windows. I do most of my searching at work. (Unfortunately, on windows machines.) I only use metaengines. For windows there is web Ferret and then there is copernic 2001. I use Copernic 2001 the most because it also let's you filter the results. You can use logical the operators "and", "or", "not", "near", "except" (and not), & "or not". It stores the in a db offline. I found this to be very useful because I can keep several queries for any amount of time.

Because the moon is already owned. (Link is in Spanish only; use the Fish or the Gist)
--

Nice list - looks much like mine. You should check out WinKey - it lets you assign any shortcut (including some window controls) to a keyboard combo with the Win95 key. They haven't updated it for years, but it works perfectly on Win2k. It's literally the first thing I install on a new box - I can't live without it. It takes ~2.5MB RAM and, in over 3 years and a few dozen machines, I've not seen it crash once.

question: is control controlled by its need to control?
answer: yes

Even worse, look at the trainslation Gist-in-Time does...this makes Zero Wing seem comprehensible.

http://translate.copernic.com:8090/?AlisUI=frames_ ex/gen_toolbar&AlisSourceLang=ja&AlisTargetLang=en &AlisUILang=en&AlisTargetURI=http%3A//www.peanuts. gr.jp/pslinux/beginnings.html

All your Linux are belong to Sony.

I make Copernic use them for me. hmmm, do they have a linux version? I am too busy to look.
Okay, so I had to check, seems they don't, but hey, it's a Canadian company ;-)

Going on means going far
Going far means returning

I often use the free (as in beer) search engine made by Copernic.

Basically they offer web searchs based on categories with each category using some of 80 different search engines to aggregate a result, with categories such as :
- programming
- tech news (searches sites such as slashdot and the register)
- games
- file search
- computer security (searches astalavista for ex)
- humour
- mp3
- e-mail addresses
- shopping
- news
- newgroups
- the Web(America),the Web(UK),the Web(Mongolian Goat Herders etc-well you get the picture)

Best of all it can filter out all the dead links before giving you its search results

It use engines such as Google,The Open Directory Project, Alta Vista, Hot Bot etc as new search engines come along they are available to use when you next update Copernic via the net.

They sell a commercial version but that differs only in the ability to cutomize more and do get rid of the annoying adverts whilst your surfing

If you want the commercial version just download the free one and get a license number from astalavista , register your software via the menu and lo and behold you have now got the Pro version.

or Copernic

I use the Copernic search engine at work,which searches a range of search engines depending
on what criteria you wish to search on

It even has a search engine for programming,and of course slashdot is amongst the computer sites they search for content!

Its not particulary elegant, but it does seem to produce good results.

Also recommended, if you have a keyboard with winkeys (I don't anymore, but I used to) is WinKey, which allows you to map programs to winkey combinations. WinKey is here.

--