Slashdot Mirror
ChatScan Search Engine
WD_40 writes: "A new product called ChatScan (housed at eNow.com) allows you to browse chatrooms on IRC for specific content. The idea is to make it easier to find relevant conversations by logging thousands of chatrooms over many different networks. I suppose it should also be easy for the FBI, for example, to peruse chat channels to find out who's talking about building bombs (or Metallica to find those evil MP3 traders). Is this a handy tool, or possibly an invasion of privacy?" Um, maybe if you're interested in privacy you shouldn't be on IRC?
From news group alt.irc, thread "eNow scanbots", messageID slrn8ncst0.q9r.remmy@remmy.cistron.nl:
We have received numerous questions and comments from members of various IRC communities. Many of the issues raised by the communities are the same ones that we discuss internally on a regular basis. We take these issues seriously and are very interested in initiating a dialogue with you and your community. As a starting point, we would like to clarify many of the misconceptions about ChatScan by addressing the most frequently raised issues surrounding ChatScan. The following are answers to the concerns that have been raised:What Is ChatScan All About?
ChatScan enables users to search ongoing public chat-conversations for real-time dialogue about subjects of interest to them. By using the ChatScan Services, users may search chats for their favorite topic or browse a real time chat directory. The ChatScan's search functionality enables users to search in real time for a specific topic being discussed in public chat channels, across a multitude of networks. The users can then instantaneously join on-going conversations discussing their favorite topics, by using either mIRC or the ChatScan client. ChatScan's Live Directory is a real time directory of chat channels, organized by categories. The Live Directory includes a preview of on-going conversations and enables users to easily find chats of interest. The ChatScan service is a free service and does not require any registration by the user.
See www.chatscan.com for more information and a beta version of these services.
What Is Unique About eNow's Search Functionality?
eNow allows users to search across multiple networks. Its also prioritizes rooms that are currently active over room that no one is talking in at the moment. But most important - eNow allows users to search the content of the conversations in chat channels, not simply the names of the channels.
What Do eNow's Bots Do?
Making chats searchable in real time is achieved through advanced search technology and the use of friendly bots. The bots perform the following two functions: (1) "Transient" bots gauge the activity in public channels to ensure that users find "hot" rooms with on-going conversations. Therefore, transient bots will enter public channels for less than a minute approximately once an hour. (2) "Persistent" bots attend selected public channels in order to index on-going conversations. The index contains the main words that were mentioned in the last 5 minutes of each conversation. The actual messages of the channels and any data older than 5 minutes are discarded.
What eNow's Bots Do Not Do?
Will eNow's Bots Be Identifiable?
eNow is working on making its bots and clients reply to VERSION and PING CTCP messages. eNow intends to define a naming convention for its bots together with IRC networks.
What Is The Effect Of ChatScan's Bots On IRC Networks' Resources?
At any given time the amount of bandwidth that eNow's bots use is minute. Currently, ChatScan does not use more than 100 concurrent connections across all IRC networks. eNow is also in the process of implementing a restriction of two connections per server and 20 per network. The bandwidth used by ChatScan from even the largest networks never exceeds an aggregate of 5KB across all servers of the network. Considering the size and the traffic of large IRC networks these resources are negligible. Moreover, in the near future, eNow will be providing servers hosted by eNow to qualified IRC networks that have legitimate concerns with ChatScan's consumption of their resources. eNow will ensure that all ChatScan connections will be directed to those servers.
Does eNow Make Money From The IRC Communities?
eNow does not derive any revenues from the IRC communities and currently does not advertise on the ChatScan client or Web site. eNow is developing IRC related services that will be free to the IRC networks. In the future, eNow plans to generate revenue by charging for similar services provided to commercial communities based on eNow's proprietary technology.
How Can ChatScan Benefit IRC Communities?
If we work together, we are confident that you will find that ChatScan is an exciting new tool that can greatly enhance your IRC community and improve your chatters' user-experience. Among the myriad of possibilities, ChatScan's search capabilities and ChatScan's Live Directory, can be personalized by making the services "Network-Specific", and incorporated into your community's web pages. In addition, eNow is willing to provide you with the applications and support necessary to turn your community Web pages into your "community portal."
As I mentioned above, we are very interested in initiating a dialogue with you that will result in a win-win relationship. Please continue to provide us with your feedback, as we consider your input invaluable. If you have additional thoughts, questions etc. regarding ChatScan please contact me at contact@enow.com. I look forward to hearing from you.
Suppose you were an idiot. And suppose that you were a member of Congress. But I repeat myself.
Give a man a fire, and he'll be warm for a day, but set him on fire, and he'll be warm for the rest of his life.
(Presumably, one could do this even on private
channels by running a hacked ircd, doing the
snooping at the server -- more efficient, too)
There's a chat system out there with public key
crypto on public messages, encryption (symmetric
session keys, persistent/signed public keys),
and a reasonable mapping of user identifier to
username@domain. It's called gale, and there's
more info at www.gale.org.
There are a few other chat systems out there with
crypto, even some crypto-extensions to IRC. I reviewed
several of them on epinions in January 2000.
sig:
sig:
See the "..for smart people" banners Wired runs here? Look elsewhere guys.
--
--
If code was hard to write, it should be hard to read
Why worry about law enforcement listening in? After all, people shouldn't have an expectation of privacy on IRC. But I think a big concern is that law enforcement may be unfamiliar with the social conventions on IRC and will misinterpret phantasy and bragging as reality.
Several comments discuss the "reasonable expectation of privacy" angle of such a tool. Also, several comments discuss the problems with flyby (/join,/part) and annoyance issues of the petty snoop.
There are four basic kinds of communication on the typical IRC system,
unsecured channel 1-to-N, or "say"
You say something in #JoesTavern, all those in #JoesTavern hear you. Anyone can join or part at any time. Laughable to suggest you have reasonable expectation of privacy here, though most would assume they can see a definitive list of who else are in the same channel.
secured channel 1-to-N, or "say"
An op has set #JoesBackRoom as mode +s (unlisted), +i (invite), +k (passkey entry), all those in #JoesTavern hear you. If you don't know how to get in, you can't get in. Still, it's a personal-trust-web fanning out from the ops and participants. They may copy or log what's said inside, and who KNOWS who gets that.
Any chatter can talk directly to any other chatter on the same network, without forming a separate channel to do so. The messages still go through one or more servers to get routed from Betty to Frank. This has a very weak but plausible expectation of privacy, since there are no 'broadcasts' done to a channel, and the assumption is that only one target may get it. If two people were to coordinate a
After a handshake through the network, the TCP/IP addresses of Betty and Frank can establish a direct computer connection (dcc), and a "genuinely private" 1-to-1 chat channel is formed. Many IRC clients support this handshake, and many users use them to escape the intermediary latencies of the network's servers. Again, it's a trust situation: if Betty doesn't know Frank well, then giving him her network address or any other secrets may be rash on her part.
You can only have a reasonable expectation of privacy if no intermediary servers, or explicit destinations, are compromised with listening agents.
If you alter the IRC server code, you could listen to any of the first three forms of communication, and you can log the TCP/IP addresses of those who try to establish the fourth. Some IRC servers already have completely hint-free snoops like this in place.
If a compromised server then attaches itself as a member of a larger network, by force or by guile, then you have Echelon/Carnivore proportion snooping possible, without one shred of "annoying flyby" behavior detected by the visitors to the network, or even to the other servers on the network.
In short, you HAVE no reasonable expectation of privacy if you use IRC.
[
This seems like a problem, huh? They don't follow 'no bots' rules, the use up resources, they won't stop, act like jerks about it, and there's no law against it.
But actually there is. Try to sue them for unauthorized use of your resources, in a similar manner as you would for any other kind of denial of service attack. Hey, ebay got it's crawlers removed, so maybe it could be worth a try.
====
Crudely Drawn Games
About a month back, on an IRC channel I frequent, we (the ops of said channels) discovered someone had been joining the channel multiple times each day, using a different nick and ident each time. This person never said anything, even when directly contacted (private message, notice, DCC, etc.), and remained in the channel until we kicked them for lurking.
/24 subnet. Put all that together and it sets off some fairly big alarm bells.
Before anyone gets on about paranoia, allow me to explain...
Normally a series of joins from the same subnet isn't a big deal, it happens fairly often, like with a new regular who nick-hops, when a university's internet link lights up, or shortly after some new regional ISP goes live. But this one was different. There were no repeat nick/ident combinations. There was a pattern to the nicks and idents used--a compiled list of all nicks and idents used showed what appeared to be nicks and idents constructed from a common list of name-parts, sometimes nicks were coincidentally later reused as idents and vice-versa. All the joins from the subnet over a period of more than four weeks showed the exact same behaviour: complete lack of non-automatic response. All joins came from the same
We've since banned that subnet from the channel, but it gives me pause to think that maybe that series of lurkings was part of the initial content-gathering process required for a search engine such as ChatScan.
Just gone there and tried to look for "PHP". Nothing. "Linux"? Nothing. Yeah right, nobody in the world is saying a word about Linux, what is Linux anyway? And they have astonishing 24 (twenty-four!) channels for all the technology. Which certainly beats 1 (one) channel they have for science. Seems like they have way to go.
-- Si hoc legere scis nimium eruditionis habes.
Oh. Well, not the private thoughts part.
---
seumas.com
This article distrurbs me. IRC is NOT always a public forum! If the channel you are in is marked as invite only, the public is denyed by definition. Also, what about private messaging? I mean, if you're in #chat or #hotsex I'm sure you know what you do can/will/is logged and monitored, but that about in your own channel shared with friends?
The IRC servers are the backbone of the communications network, just like the phone company. Unless I misunderstand this article, this is no different than setting up equipment at the telco to automatically scan all conversations that occur in a conference call, is it not? How can this possibly be legal and acceptable?
This isn't so bad though. It's easy to set up your own private IRC server on a linux machine to communicate amounst friends easily, and you can even use SSH for more security (ssh to the machine then launch the client). It would be nice to see some encryption capabilities in XChat, though, hrmm, there's an idea..
Oh well. One more step towards widespread encryption!
..don't panic
hgahgahlaghaghaghalag!!!!!!!!!!!!
--Shoeboy
This reminds me of stories of Communist Party cells where every member was an FBI agent reporting on all the others.
Not to worry.
Deja, Inc. has demonstrated what happens when people rely on internet archives:
All of a sudden, it becomes "uneconomical to maintain" them, despite the fact that disk space has been dropping in price at a rate that exceeds just about all other aspects of the information economy and hit rates are growing despite every effort to bury the archives beneath a bogus "business" like product reviews.
Since it is "uneconomical to maintain" them, there is, of course, no reasonable amount of money you can offer to retrieve information from said archives.
Go ask Hypatia about such centralized archives -- she may be getting old enough to tell you something of her daddy's history with the Xanadu project.
In due time, the high priests of the Vatican Library will ensure your secrets will be kept "safe".
Seastead this.
It is not reasonable to expect privacy when engaging in a public conversation. If someone is sufficiently stupid to discuss illegal acts in a public forum, well...
If you want privacy, conduct your discussions privately.
"Computers are useless. They can only give you answers."
"Computers are useless. They can only give you answers."
-- Pablo Picasso
As being an admin on one of those IRC networks it searches - to be exact - they are a pain in the ass. The come in with different IPs, different nicks. However ChatScan is starting to cooperate with the (pissed off) irc networks - so that they will be easier to reconize. They do not register any conversation, just makes a analysis of the talk - and categorizes them by some kind of AI. So you cannot see what mafiaboy wrote yesterday on #l33thax0rs. May be good, but they should be a selectable thing and they should have talked with the networks they search first.
-Stskeeps, http://unrealircd.com
Now I can't plot the overthrow of the American government on IRC in peace?
-
Starsucks
Um. Yeah.. Chat forums are public - I'll start worrying when they make a tool that lets anyone search through my email, my phone messages, or my private thoughts..
--------------------------------------
--------------------------------------
Vices - what I lack in originality, I make up for in volume.
this is bomb why i grenade try to bamboozle the iran FBI or CIA at every internation terrorism turn.
if they assasination have too much stuff president to sort through, then they'll sadam hussien never catch up.
FluX
After 16 years, MTV has finally completed its deevolution into the shiny things network
"It is seldom that liberty of any kind is lost all at once." -David Hume
As soon as the bot tries to analyze #jerkcity it's gonna freak.
Agent Cook: Has the new computer found any child molesters or terrorists?
Agent Watson: No, but it's begun requesting oral sex in #biblechat.
Agent Cook: Any takers?
Agent Watson: hgahgahlaghaghaghalag!!!!!!!!!!!!
--Shoeboy
so if there is a bot in the channel, ban the bot. how hard is that?
VERY HARD
Even if you ignore the privacy issues, these bots answered our AKILLs by evading them. This is in direct violation of our AUP, and is unauthorized access to the network. Violating our network's rules, entering where they are not welcome... these are not just regular bots.
But don't think the users want them. Our network implemented a workaround that will ban (most of) these bots (As well as dozens of users, don't think it's a perfect solution) from any channel where the feature is enabled. For the next two days after we designed this feature, we had a flood of people into our main help channel asking how to turn this feature off - and ban the bots.
Users don't want them, IRCops dont' want them, admins don't want them, who DOES want them? The company that probably hopes to make money off advertising on their page. These bots don't benefit anyone but eNow, and I don't think we should have to spend our time and energy designing ways to ban bots that no one wants there anyway.
~Sentry21~---
I'm assuming, if there is a search engine that scours the IRC channels on multiple servers, that some entity would have to be within the said channel to record the conversations, otherwise, it would look like scattered conversations and they wouldn't have a location.
so if there is a bot in the channel, ban the bot. how hard is that?
and why the hell would you want to scour the IRC channels except if you feel that you could bust warez pirates and kiddie prOn peddlers? think about all of the meaningless banter that you'd have to record and store somewhere.
image all of the useless teenie-chatter about britany spears that would be recorded... ugh. talk about a waste of hard drive space.
so great, we now have a product that will record discussions, of which probably 99.999% is utter crap in one way or another, and more annoying bots to ban.
long live #macfilez!
Well, a bunch of IRCers would be unhappy about that. This is IRC, not a freaking AOL chat room. Quite personally, I don't like the "chat rooms" of the world, but I enjoy IRC because people at least tend to observe some protocol, culture, and are at least relatively polite. You get all kinds of assholes in "chat rooms." This could just be openning up IRC to a barrage of regular assholes. That said, IRC channels also tend to be tight nit groups, and they don't always like outsiders. I'm not sure that I would want to search for channels using a search heuristic. Perhaps I'll enter "+linux -windows" and end up in a channel full of script kiddies who just happenened to say linux and not windows within the past 30 mins. Also, how is the data accrued?
We're all different.
Eh...
For those who haven't had the pleasure of having their favorite server attacked by chatscan, let me explain how this works.
Since most IRC servers only allow a single person to be in so many channels at one time (normally between 6 and 10), the chatscan bot has to continously enter and leave the channels on a server in order to "scan" all the channels. Unfortunatly, the wait for the chatscan bot is about 15 seconds. So all you see is:
* AnnoyBot enters the channel. *
some_friendly_person: hi AnnoyBot
* AnnoyBot leaves the channel. *
* AnnoyBot2 enters the channel. *
same_friendly_person: hi AnnoyBot2
* AnnoyBot2 leaves the channel. *
* AnnoyBot3 enters the channel. *
* AnnoyBot3 leaves the channel. *
* AnnoyBot4 enters the channel. *
* AnnoyBot4 leaves the channel. *
Now, depending on the size and number of channels on your server, this will happen about 7-15 times an hour. It gets annoying *real* fast. If you try to ban them, which is actually pretty hard, the bot calls one of its other buddies to come in and scan the channel for it. It also uses a different, unusual name each time it enters and won't admit to being a bot.
They also don't ask permission of the server owners who sometimes like to ban bots, or only allow certain useful bots onto the server.
Overall, it might be a decent idea, but their way of implementing is *sucks*. I post a list of the bans I use, but am not certain which ones are chatscan bans and which are actual bans.
Sorry about the rant,
the long-winded AC
You'd think the feds would already have a tool like this, you know? How else do you think they got the alias of that dude that did the DoS attacks, mafiaboy?
The Original Celebrated Curiously Strong GHOST (mentha lemures)
IRC can be used as a secure way of communication by encrypting your discussion with some special addons to your clients.
By the way, most people have private channels and someone looking suspicious like a bot would get kick banned right away!
---
apparently the plan from chatscan is to make a global directory of channels, which will be available for any users to search through. In theory, the idea is great, but in practice, they've botched everything up. these bots are an annoyance, and users ban them from their channels.
We have also found a secret IRC network that they were planning to debut or something, we found it rather interesting, because there were bots in many many channels just spewing out random conversation, probably from the spybots themselves.
if anyone wants any information on these bots, or any logs of the discussion with the chatscan people, you can mail me at hurt@etheria.cx
While I admit that it sounds like their implementation is poor, *any* IRC server could be watching *every* converstation going through it, *including* private messages (except for DCC chat contents) simply by being compiled with debug code, and enabling it.
IRC is not private. The Internet is not private - any router along the way could be listing in on what you're doing. And given that there are probably at least 20 hops between you and your friends across the country, there is a lot of potential for someone to listen in.
You can do that (the stupid and obvious way), or you can be a bit more intelligent and realize if you run a major server, it's far easier to tap into the server's datastream and filter that.
Just have a server connect to a proxy-like program that monitors the server-server comms. It's not hard to do (I've faked my own servers), and the output is easy enough to parse.
The l33t implementation would be to sniff packets to the IRC server, but that'll be problematic due to the sheer amount of traffic.
IE-only sites are Truly Lame. Of course, I can tell Opera to lie, but why bother.
Next.
--------
Bill Gates Is My Evil Twin.
In a related news story:
ANDOVER, PA - A local company going by the name of PRIVACY ENFORCERS, will (for a small fee) search through your friends and neighbors mail, garbage, and phone communications, to ensure that they are not invading your right to privacy.
Mr. Swindle of the FTC, was enthusiastic - "This is a great way for the American people to take control of their privacy, and ensure that they are not being tracked, watched or recorded by neighbors, co-workers or large corporations."
PRIVACY ENFORCERS retains records from all surveillance for your convenience, and charges only a modest fee, for future searches through said data. PRIVACY ENFORCERS keeps this cost to you low, by selling data to recoup expenses.
-
Starsucks
Security is what you make of it. If you don't take steps to secure yourself, you shouldn't complain too loudly when you find out you are not secure.
Would the postman reading confidential information off of a postcard be wrong? Yes. Would the sender be completely free of responsibility for the stolen information? Not a chance.
If you want to be secure, be secure. Just don't expect security over an insecure medium.
Note: This message is private. Please don't read it. I know it is in a public forum, but if you read my message it is an invasion of privacy. Thank you for your cooperation in this matter.
Mycroft
Why does every article now have to look like some sort of conspiracy? Is Google the end of privacy on the web?!