Slashdot Mirror
A Search Engine For The Slower Net
Makarand writes "According to this BBC News
article researchers at MIT
are developing a search engine for people
using the web on slower net connections.
The software will e-mail queries to a central server and receive the most relevant
webpages from the search results by e-mail in a compressed form. Since the program is too big to download over a poor net connection
it will be mailed on CDs to libraries for people to borrow and install. They are also considering trying to persuade computer sellers
in developing countries to install the program on machines."
About them Modem Linkers,
ain't they kinda odd?
Goin' on the net,
with they little baud.
Look at all those Modem Linkers,
what a thing to see.
Web sites come up really slow,
get's lousy Voice/IP.
Internet at low bit rates,
what a dawgon mess.
Load a web site, take a break,
while 'pache mods compress.
How to be a Modem Linker,
don't need a ticket.
Get a local ISP,
dial up and link it.
A programmer is a machine for converting coffee into code.
Maybe we could have all webpages categorized by a number, something like 800 for science or whatever, and then we could have a filing cabinet with index cards in it. Then, people could open the filing cabinet, see a number for the page they want and then go directly to the page.
http://www.mshiltonj.com/sr/
Might be a nice way to preserve searches for later perusal. Unlike bookmarking, the returned search results are stored in an email.
This would be a good way to preserve stuff that may be the subject of removal due to court order, like xenu.net and other similar de-Googlings.
I had the same initial reaction, but after RTFA (I know, shame on me), it seems that the limitation isn't so much time, but continuous time hogging the phone line accessing Google, checking out pages, etc.
Instead, this service would package together selected results of the search, for overnight download into the PC's cache. The user can then browse through the material at their leisure without needing to use the internet connection (which is the scarce resource).
Stop by my site where I write about ERP systems & more
For those of you wondering why someone would do this, how about reading the damn article?
The program doesn't e-mail back with a mere mirror of a google / yahoo results page. It actually filters through the individual results compressing the entire page. e.g. my search turns up a CNN page and a blurb on MSNBC and I get, e-mailed to me, compressed versions of those actual sites, not just links to them.
As far the "my 28.8 modem is just fast enough" crowd -- read the article! Some of these locations the software is being developed for don't even have access to a phone line on a regular basis. And the lines they do have access to are more likely than not to be noisy as hell and not able to support a 28.8 connection.
TODAY
I am reminded of the Prepaid Legal system of doing business. You call up and ask a question, and the next day, an attorney familiar with the area you are asking about calls you back to answer your questions and advise you. So maybe this isn't all that outdated of an idea after
IN REGARD TO THE SYSTEM IN THE ARTICLE:
To have this capability back in 1973 would have been unbelievable. In 1983, to have this available to every library in the US would have been an unbelievable achievement. To have it now is so slow that I start to go google eyed even thinking about it.
BUT
This is great for countries that are 20-30 years behind in technology. It will revolutionize the search for information for areas that are not as connected as the US.
Coincidentally (?) it is also very usefult to circumvent the Great Firewall. Way to go, but it would also be nice to optionally have the cached content (ala google) e-mailed as well. That would send the last standing wall crumbling.
Code poet, espresso fiend, starter upper.
RTFA.
Why don't you just scream "HI I'M FROM 'WESTERN' CIVILIZATION AND HAVE NO IDEA HOW THESE THINGS WORK IN LESS PRIVLEGED PLACES"
Google is too slow when your school has one phone line that is used for _everything_, including net access. Not to mention the cost of using the phone anyway. This allows all the students to submit thier searches to a teacher one day, the teacher then submits the all searches with only a couple minutes of dialing up. He can retrieve the compressed results a few days later with only a minutes of dialing up. Now go read the article. Someone needs to mod that post down, hopefully the poster can redeem themselves later in the thread with something insightful.
I tried to RTFA but MIT hasn't emailed it to me yet :(.
my blog
...only webdesigners had not collaborated to turn the web into the graphics orgy it is today. I mean, have these kids coming out of graphics school even browsed the relevant w3c specifications?
News Flash !
To try out this demo, please follow these simple steps:
1. Pick up the phone and call the automated voice search system at (650) 318-0165.
2. After the prompt Say your Search Keywords, say your query to the system.
3. Click this link and a new window will open with your voice search results.
4. Say another query, and the new window with the search results will be updated with the new results.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
There are several benefits of having a TEK Client program instead of just using email. But first off, the client isn't that big -- the JAR file with the TEK classes is 125 KB. When we package it up with third-party libraries and an installer, it comes to 2 MB, and with Java included, it's 10 MB. It would be interesting to try to prune down this distribution to the minimal size -- for the prototype version, we have focussed primarily on the software's functionality.
The TEK Client program is useful because it provides a seamless interface to browsing the downloaded pages. It operates as a web proxy: users adjust their browser to talk to TEK instead of the web, and then they can view pages just as if they were connected. The URL's appear as usual in the browser's "location" toolbar, and links on the page are functional. If a URL has been downloaded before, then it is loaded out of the local cache; if it has not yet been downloaded, then the user is queried to submit a request for that URL.
The TEK Client includes a local search utility for searching the cache of downloaded pages. In this way, the user can build up a local library of information that is relevant to their community; for example, in a school setting, many searches could be satisfied using only the local cache due to overlapping interests of students.
Also, the TEK Client is useful for tracking searches. In settings where connectivity is intermittent, searches can be enqueued during the day and sent at night (or when a connection is available.) The client also provides basic user management so that multiple people can share a public installation (perhaps using a single email address, which they might not own themselves) and still keep track of their own queries.
In the future, we think there are a lot of features that could be added to the client. For instance, we could seed the client with other open-source resources, such as an atlas or encyclopedia, that could be used in conjunction with web searches. There could also be an "intelligent query builder" that helps construct Internet searches (for example, by checking spelling) before going through the time and expense of connecting and sending them off.
Many more details about TEK are available from the TEK Homepage. We are currently moving our CVS source tree to SourceForge, so if you're interested in helping to improve the software, it'd be great to hear from you!
You're right that retrieving web pages over email has already been done. A present-day service that works as you describe is www4mail, and I know people that use it regularly from low-connectivity regions.
However, the TEK system (which I'm involved in) offers several benefits over a purely email-based solution. By having a web proxy on the client side, users can use their favorite browser to view downloaded pages, complete with color and formatting, which is often absent in text-only systems. Moreover, the client keeps a local, searchable cache of all downloaded pages, and the server keeps track of which pages have been sent to avoid wasting bandwidth on duplicate content. Finally, with a web-like user interface, many users can share a single e-mail account in a public kiosk or school.
Many more details about the TEK system are available from the TEK Homepage