Slashdot Mirror
A Search Engine For The Slower Net
Makarand writes "According to this BBC News
article researchers at MIT
are developing a search engine for people
using the web on slower net connections.
The software will e-mail queries to a central server and receive the most relevant
webpages from the search results by e-mail in a compressed form. Since the program is too big to download over a poor net connection
it will be mailed on CDs to libraries for people to borrow and install. They are also considering trying to persuade computer sellers
in developing countries to install the program on machines."
About them Modem Linkers,
ain't they kinda odd?
Goin' on the net,
with they little baud.
Look at all those Modem Linkers,
what a thing to see.
Web sites come up really slow,
get's lousy Voice/IP.
Internet at low bit rates,
what a dawgon mess.
Load a web site, take a break,
while 'pache mods compress.
How to be a Modem Linker,
don't need a ticket.
Get a local ISP,
dial up and link it.
A programmer is a machine for converting coffee into code.
Maybe we could have all webpages categorized by a number, something like 800 for science or whatever, and then we could have a filing cabinet with index cards in it. Then, people could open the filing cabinet, see a number for the page they want and then go directly to the page.
...still surfing the internet with their Commodore 64s and 300 baud modems!
http://www.mshiltonj.com/sr/
Might be a nice way to preserve searches for later perusal. Unlike bookmarking, the returned search results are stored in an email.
This would be a good way to preserve stuff that may be the subject of removal due to court order, like xenu.net and other similar de-Googlings.
I had the same initial reaction, but after RTFA (I know, shame on me), it seems that the limitation isn't so much time, but continuous time hogging the phone line accessing Google, checking out pages, etc.
Instead, this service would package together selected results of the search, for overnight download into the PC's cache. The user can then browse through the material at their leisure without needing to use the internet connection (which is the scarce resource).
Stop by my site where I write about ERP systems & more
For those of you wondering why someone would do this, how about reading the damn article?
The program doesn't e-mail back with a mere mirror of a google / yahoo results page. It actually filters through the individual results compressing the entire page. e.g. my search turns up a CNN page and a blurb on MSNBC and I get, e-mailed to me, compressed versions of those actual sites, not just links to them.
As far the "my 28.8 modem is just fast enough" crowd -- read the article! Some of these locations the software is being developed for don't even have access to a phone line on a regular basis. And the lines they do have access to are more likely than not to be noisy as hell and not able to support a 28.8 connection.
TODAY
I am reminded of the Prepaid Legal system of doing business. You call up and ask a question, and the next day, an attorney familiar with the area you are asking about calls you back to answer your questions and advise you. So maybe this isn't all that outdated of an idea after
IN REGARD TO THE SYSTEM IN THE ARTICLE:
To have this capability back in 1973 would have been unbelievable. In 1983, to have this available to every library in the US would have been an unbelievable achievement. To have it now is so slow that I start to go google eyed even thinking about it.
BUT
This is great for countries that are 20-30 years behind in technology. It will revolutionize the search for information for areas that are not as connected as the US.
is not slow connections, but connections that are unreliable
Using the phone in a country like Malawi can be a real adventure. It's not like the US at all.
Coincidentally (?) it is also very usefult to circumvent the Great Firewall. Way to go, but it would also be nice to optionally have the cached content (ala google) e-mailed as well. That would send the last standing wall crumbling.
Code poet, espresso fiend, starter upper.
RTFA.
Why don't you just scream "HI I'M FROM 'WESTERN' CIVILIZATION AND HAVE NO IDEA HOW THESE THINGS WORK IN LESS PRIVLEGED PLACES"
Google is too slow when your school has one phone line that is used for _everything_, including net access. Not to mention the cost of using the phone anyway. This allows all the students to submit thier searches to a teacher one day, the teacher then submits the all searches with only a couple minutes of dialing up. He can retrieve the compressed results a few days later with only a minutes of dialing up. Now go read the article. Someone needs to mod that post down, hopefully the poster can redeem themselves later in the thread with something insightful.
And for those people with no internet connection, you can mail your search requests to MIT (Please include self-addressed stamped envelope). MIT will then process your search request within 5 business days, and mail you back the results. You can then peruse the results and marvel at the wealth at information you'd be able to find... if only you had internet access.
Shameless plug for my photos on Flickr
I tried to RTFA but MIT hasn't emailed it to me yet :(.
my blog
...only webdesigners had not collaborated to turn the web into the graphics orgy it is today. I mean, have these kids coming out of graphics school even browsed the relevant w3c specifications?
News Flash !
To try out this demo, please follow these simple steps:
1. Pick up the phone and call the automated voice search system at (650) 318-0165.
2. After the prompt Say your Search Keywords, say your query to the system.
3. Click this link and a new window will open with your voice search results.
4. Say another query, and the new window with the search results will be updated with the new results.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
They should develop a program that strips images, animation, java aplets, ActiveX components and all HTML from Web pages, leaving only the text and the links. Then send it to the users. It could be called Gopher. Or Archie.
cor... this sounds oh so familiar... anyone remember ftp by email??? History repeating itself...
Query the ftp server by email and get the directory list emailed back to you. Then you could send the command via another email which would result in the file being emailed back to you overnight ready for you to retrieve it.
And then there was "trickle" where files could be sent/refreshed to your uni's mainframe's ftp server overnight and would be there for you to play with the next morning and you would always have the most recent version of the file as they'd have been synched via trickle
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
"MIT Reinvents Archie service from the early 90's."
"Given the pace of technology, I propose we leave math to the machines and go play outside." -- Calvin
It's a shame that with the way the net is going all they will get as search results will be flash heavy sites that take 20 minutes to download on broadband, let along dial up.
.tar.gz for download and offline reading.
Where did all the sites go that you could use wget -r to grab overnight? How about the odd few that used to offer a
Content over presentation is a concept that needs to be reintroduced to the net, preferably with a stick.
Beep beep.
First, for those of you saying 'Google is fast enough even on a 14.4K' - think school with one phone line, perhaps not even available during the day. Or how about connections via satelite phone at $$/min? Suddenly you want super efficient, when you only earn 5 bucks a day.
No, google is fast enough at 300Baud. Damn, but folks are young around here.
As to what else this needs, the search engine needs to strip out all the crap before emailing a web page to you (Java, Flash, etc) - should focus on mostly text, small pictures only.
Either configure your browser or proxy to do that. Easy.
Particulary since 486's would be a common platform for people using this, so the search engine better work well on one.
Give me a break. 486's are plenty powerful enough for web browsing. Even with pictures.
You also should be able to strip out all pictures as an option to maximise text info download - remember turning off pictures in Netscape 2.x to speed up your browsing? If you need something it striped out, you should be able to query just for the bits you need later.
[sarcasm on] Really? [/sarcasm]
Also the ability to share your cache between computers would be huge if they can't have a server to do that for them. At any rate, means of transferring those precious pages you downloaded to another computer - on a floppy, unless you have local email
Give me a frickin' break. PPP over null modem serial.
This has got to be one of the worst ideas I've ever heard of. Hell, I knew of WWW via UUCP (that's email, kids) in the 90's - and that didn't require ANY "special search software."
This FAQ explains how to access most of the internet using only a standard email client.
The above document explain how to access:
FTP
ARCHIE (deprecated)
FTPSEARCH (deprecated)
GOPHER (deprecated)
VERONICA (deprecated)
JUGHEAD (deprecated)
USENET
WWW
WWW SEARCH (using standard search engine like altavista, yahoo or google)
FINGER
WHOIS
[...]
All these protocols can be accessed via email, according to the FAQ. The FAQ has been around for a long time. This explains why many (most) involved protocols are now deprecated. I used this faq in the early '90 and I don't know how it works now. At the time, it was great. The last update is 2002/04/16.
There are several benefits of having a TEK Client program instead of just using email. But first off, the client isn't that big -- the JAR file with the TEK classes is 125 KB. When we package it up with third-party libraries and an installer, it comes to 2 MB, and with Java included, it's 10 MB. It would be interesting to try to prune down this distribution to the minimal size -- for the prototype version, we have focussed primarily on the software's functionality.
The TEK Client program is useful because it provides a seamless interface to browsing the downloaded pages. It operates as a web proxy: users adjust their browser to talk to TEK instead of the web, and then they can view pages just as if they were connected. The URL's appear as usual in the browser's "location" toolbar, and links on the page are functional. If a URL has been downloaded before, then it is loaded out of the local cache; if it has not yet been downloaded, then the user is queried to submit a request for that URL.
The TEK Client includes a local search utility for searching the cache of downloaded pages. In this way, the user can build up a local library of information that is relevant to their community; for example, in a school setting, many searches could be satisfied using only the local cache due to overlapping interests of students.
Also, the TEK Client is useful for tracking searches. In settings where connectivity is intermittent, searches can be enqueued during the day and sent at night (or when a connection is available.) The client also provides basic user management so that multiple people can share a public installation (perhaps using a single email address, which they might not own themselves) and still keep track of their own queries.
In the future, we think there are a lot of features that could be added to the client. For instance, we could seed the client with other open-source resources, such as an atlas or encyclopedia, that could be used in conjunction with web searches. There could also be an "intelligent query builder" that helps construct Internet searches (for example, by checking spelling) before going through the time and expense of connecting and sending them off.
Many more details about TEK are available from the TEK Homepage. We are currently moving our CVS source tree to SourceForge, so if you're interested in helping to improve the software, it'd be great to hear from you!
You're right that retrieving web pages over email has already been done. A present-day service that works as you describe is www4mail, and I know people that use it regularly from low-connectivity regions.
However, the TEK system (which I'm involved in) offers several benefits over a purely email-based solution. By having a web proxy on the client side, users can use their favorite browser to view downloaded pages, complete with color and formatting, which is often absent in text-only systems. Moreover, the client keeps a local, searchable cache of all downloaded pages, and the server keeps track of which pages have been sent to avoid wasting bandwidth on duplicate content. Finally, with a web-like user interface, many users can share a single e-mail account in a public kiosk or school.
Many more details about the TEK system are available from the TEK Homepage