Slashdot Mirror
What Kind Of Logs Should ISPs Keep?
Effugas asks: "An engineer at a rather large ISP recently asked me a rather simple question that I didn't have a particularly good answer for: What logs should they be storing? He wasn't asking about the simple question of whether their own servers should be watched closely--that's obvious. He was asking about his routing infrastructure. I told him they of course musn't record the actual data being routed through their network; however, endpoint to endpoint route logs(since the establishment of those routes is the ISP's raison d'etre) did seem viable. But now, I'm not so sure--if there's one thing we learned from Kenneth Starr's subpeona of Lewinsky's book purchase records, it's that Barnes and Noble stored such records in the first place! But on the flip side, I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either. So I ask, without passing judgement in either direction: What options does a network administrator have for retaining forensic evidence in case of abuse, which ones are ethically justified, and what are the actual router configurations which implement such ethical systems?"
The ISP should feel free to log anything. Anyone who wants their data to be secure will be using https or ssh anyway.
Okay, that might be a bit extreme, but it seems the only workable and enforceable policy. If you choose any other criterion, there will always be some unscrupulous ISPs which ignore it, and it gives people a false sense of security.
-- Ed Avis ed@membled.com
To help keep their logs dry, they should purchase a log rack, or simply arrange the logs on top of a makeshift support system so that the logs do not directly contact the ground.
Moist logs tend to attract bugs and decompose much faster.
Most of logs lose their information for administrator after some one week. In case you spot a hacker attack, that is more than 2 weeks old, you can as well forget to try to track anything in logs. Unless its real amateur, you get no informations there.
Therefore I think that there should be two cathegories of logs. One that is periodically thrown away each 1-2 weeks and one that is kept for longer time.
If programs would be read like poetry, most programmers would be Vogons.
For example:
(are you ready for this?)
Captain's log, Stardate 2433, We found several of our subscribers logging into a known rebellious website called Slashdot.com. They were summarily executed. (Etc. Etc.)
Cool, eh? YOU BETCHA!!
Logging in this country has gone way too far and is an abuse that cannot be permitted to progress any further. Not only does abuse of this cost many of us what they view as their birthright, but it also scares the hell out of those who haven't lost anything due to it yet. Sure, there are definitely some political and corporate interests who benefit by letting this sort of thing run rampant, but can we really afford it? And who's this world for, anyway -- the corporations or the people?
:)
And when this does spiral out of control, efforts to redress the wrongs that have been committed, no matter how good-intentioned or extensive, will never fully wipe out the harm that has been caused within the lifetimes of those who have really been hurt most. Once you go too far, you can never truly come back.
So I would definitely urge keeping logging to an absolute minimum if you can't eliminate it entirely. If you can't really appreciate the wisdom of not logging, I strongly urge you to take a hike.
And then, after you come back from your tromp through tree-lined trails, to reconsider.
It's really not a question of what logs to keep, but how long to keep them. You should keep logs of requests, attacks, e-mails, routing information, anything that you might actually need, but only keep them for the appropriate period of time. You really don't want to have to dedicate a tape changer to this anyway, do you?
E-mail, routing information, and the like, should have a relatively short lifespan, if a person is being harassed, they should report it quickly. You should allow them a week or 2 for turnaround in such cases, and burn the necessary information to a CD or other storage media for any followup needed, when there is a report. You shouldn't, however, keep a long papertrail on your users, this only invades their privacy. If there is a legitimate need for such logs, it will arise relatively quickly.
Attack logs should be kept longer. All attack logs should be analyzed and damage should be evaluated. Appropriate individuals should be informed of the attack based on what has been compromised. Even these, however, should be trashed after a period of time. Do you really care about an unsuccessful attack 2 years ago? Probably not, you might, however, care about someone who root-kitted your server a year ago, since they probably still have the passwords of at least a few of your users.
Eh...
You're joking, right?
I never knew that! What the hell is the US legal system doing prying into that kind of thing? One of these days it'll be illegal to buy books without some kind of ID.
Personally, I would encrypt them all using public-private key crypto. The "public" key is what is used to feed the data into syslog, and the private key can be used to decrypt it if you need it. If your systems are physically or otherwise compromised, the attacker still cannot derive the private key as long as you maintain due diligence in maintaining the security of the logging host(s). This means you can log everything to your hearts content and not worry about privacy concerns, as much. Just make sure to put the standard disclaimers in your AUP.
I suspect, however, that wasn't quite the answer you were looking for. Honestly, in order to compromise most people's privacy requires an ungodly large harddrive to store all that information. Simply monitoring a T1 with a packetsniffer doing decent filtering can easily trash a fast 30GB HDD. The security industry is replete with stories of how crackers were caught because their packet sniffers went amok trying to log everything, and crashed the system trying.
I'd recommend logging the source and destination of mail, and when it was retrieved. If you are using RADIUS servers, log the times they signed on and off, and keep the system clock religiously on-time. Have the facilities to monitor each user (ie, be familiar with how to use a packetsniffer, and have a box on standby if you need to use it). A quick cheat would be to configure the RADIUS server to tell $SUSPECT connection to only use $MONITORED_IP and then tell the packetsniffer to dump everything from $MONITORED_IP to disk. It's simple, but it works.
As far as advice on law enforcement.. it depends on your situation. If you have been compromised, it still may do you more harm than good to report it due to the administrative overhead involved in prosecuting them. Generally, however, they are quite helpful on getting you the information you need to prosecute. Don't expect them to get too involved though unless your SMTP logs say that a message was sent from l335h4x0r@yourisp.com to president@whitehouse.gov with a subject line mentioning what he's going to do with a box of cigars and a can of surgical lubricant. In that case, you probably won't have any choice but to cooperate. :)
Hope this helps,
I think that ELM has a much longer burn time, and a stronger, prouder smell, that reminds me of my heritage.
Eh...
I've found myself asking the same question. How much info do you really need to log? Unfortunately, many ISP's never even think about this, until something bad happens. Right now, I make sure that basic routing info getts logged, and anything that is blocked via our access lists.
Cisco ROCKS!!!!
Um, this is my sig.
A logAlog everywhere and the FBI search for Mitnick in there
**Life is too short to be serious**
ISPs should log to a certain point. It shouldnt become the task with the highest priority.
I guess this should be logged:
Dial-in.
Excessive protocol floods.
Connects made to customers on ports that are used for trojan programs like netbus ect.
-- What we do in life, echos in eternity
In my opinion, the only valid use for keeping logs is to help troubleshoot any problems that may occur in the network. The details recorded and length of time the logs are kept should be consistent with this purpose.
Those who argue that ISPs should not keep any logs are not being realistic.
Without logs, the ISP can only shrug its shoulders when a customer calls about email being dropped. With logs, the ISP has a chance to narrow the problem and fix it.
I feel like picking a fight with everyone who thinks they are right. - Rainmakers
Assume they log everything, for purposes of guaranteeing your own privacy.
Assume they log nothing, for purposes of maintaining your own documentation.
Because the fact is, they probably don't log what you need them to log, and log all sorts of crap you wouldn't want them to.
What they should log, IMHO, is everything they can, but only keep it for a couple of weeks.
Having made use of everything from error logs to snooped IRC traffic to bust intruders on my systems, I recognize both the value of such logs, and the potential for abuse.
--
Since most attacks are perpetrated by people who I can only refer to as true amatures, I said that attack logs should hang around for a while given that the security was successfully breached, whereas an unsuccessful attack, or some moron who is trying to "bitchslap" your webserver (can you believe that there are people who try that on boxen that obviously won't respond to such an attack), should be tossed after a few weeks, given that no repeat attempt has been made. If there has been a repeat attempt, older logs may be of use.
Eh...
"I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either."
Nonetheless, it IS right. The ISP is providing Internet service (duh, that's what ISP means). Period, end of story. If they want to keep (or get back) "common carrier" status, they CANNOT log packet contents.
In my view it should go like this:
Harrasee: Hey, Mr ISP--your user BlahBlah keeps sending me threatening emails, please kick him off
Mr ISP: I have no way of checking the contents of incoming or outgoing emails so I can't verify what you say is true. Furthermore, even if I could, I am not a law enforcement agency and can't take action against this person.
--
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Perhaps in clarification, i should state that i'm speaking more in terms of hoping for ISPS to be more like FreeNet, etc. Than an industry, as far as what is kept on file. Granted, with Centralized servers and the laws being passed, one should lean more towards logs of intrusions to the system, ftp, perhaps telnet (i'm refering to connections directly to the server, not beyond that, etc.
and in reference to my prior post, i forgot one thing, "First Post", ha.
In a much simpler time, I had an experience with an ISP that logged every ounce of traffic that made its way through their Annex portserver. They had enormous filesystems devoted entirely to violating their customers privacy. "Just in case"
I strongly disagree with this form of Orwellian observation, while at the same time, understand the need as an administrator to keep a certain number of logs to keep certain the system is running smoothly and that your users aren't taking down space shuttles, etc...
While more sophisticated users are aware that every keystroke can be logged, and have various facilities at thier disposal to conduct point to point encryption, the bulk of the people are unaware of this blatant violation of their most fundamental right to privacy, or more importantly, how to avoid invasions on said privacy.
In summary, I say that any logs in excess of what are necessary to continue the smooth operation of the server (which obviously vary from place to place) are entirely too many.
gitm
- The pen is mightier than the sword, the court is mightier than the pen, and the sword is mightier than the court.
What it really comes down to, IMHO, is that information itself is rarely bad. Having information is neither good nor bad in itself.
Consider a widespread DDOS attack--in this case tracking down the origin is difficult enough, and having profuse logs would be a real plus not just for the ISP, but for the net at large.
On the other hand, logging routing traffic which shows that users X,Y, and Z downloaded metallica songs which they did not own, thereby making it possible to prosecute and put them in jail for a long time would come under the heading of a Very Bad Thing.
Notice that in each of these cases, having the data in itself is not bad--it depends entirely on what is done with it. The real question which should determine what logs should be kept is, how likely is it that this information will be abused?
disclaimer: I don't think that people will really go to jail for downloading metallica MP3's-that was just an example to illustrate a point-that if the existence of logs in a given situation, in this case a police state situation, were this likely to be abused, it would be a consciencious netizens duty to come up with a convincing reason why logging was impossible. Something about the data bandwidth of (n-1)^10000 exceeding possible logging potential of network based systems under primary load conditions. Impossible to argue with that, now, isn't it?
Be ot or bot ne ot, taht is the nestquoi.
IMHO, an ISP should feel free to keep logs of all traffic that crosses their network. The use of the logs afterwards is really what matters as well as how long the logs are kept. In general, I'd feel comfortable saying that a year worth of logs is a good time frame. However, if that's too short you might want to keep the logs for as long as you can based on available storage. As for what to do with the logs after you have collected them, that's pretty easy. Nothing. Unless of course they are needed by 'the authorities' for an investigation. In that case a method of reporting is needed that will only report on the desired target so that you don't end up looking at EVERYONE with suspicion.
In view of the competing interests and liabilities of the ISP, it is probably pragmatically necessary for the ISP to maintain as comprehensive a set of logs as possible.
Whatever policy is adopted, a breach of ethics would not arise from the maintenance of logs, but rather from the failure to inform customers that such logs are being maintained. By informing the customers, each customer is on notice to take steps to assure the security of any information sent in the clear or over the wire.
I work at a large internet portal in sweden, and, amoung other things, we have a free ISP and mail service. A few days ago a letter was put on my desk from the police asking for a mail that was sent from our service to be traced.
/nutt
The process of digging through hundreds of megs of logs to find out the proper sender IP from a webmail interface is quite possibly less entertaining then counting drops of water on your farhead in a chinese POW camp. And thats WITH the leather whip. I though i was going to go into epileptic shock. I ended up having to pass the job on to a coworker. yuk.
Just my two kronors....
At least one belgian ISP got his password file very often cracked. So, if you can't track the connexion via the phone company, the information is useless. Even more, accesses are often pirated using tools like Back Orifice and such. So the information of what user connected is useless by itself...
Connexions made should definitively not be logged, for privacy and practical reasons. The people who do craking/pirating visit many web/ftp sites, connect to many machines each time they use internet. Those who only make 2 or 3 connexions are those who log on the net, connect to IRC and check their mail. Without forgetting about those web sites with so many ad-banners/counters/... that to visit one page, about 10 different IP are accessed !
Bad formed packet could be logged in order to spot people trying DoS, spoofing and such. again, how long is the question. If you can't track the real people connecting, it's useless.
Mail server use should also be tracked. but no mail content. (remind me of the FIDOnet time when many unscrupulous Sysops spend their time reading the mail going through their machine)
For the rest, AFAIK, log files can be modified at will. So I can't see how they could be used as legal evidence. IMHO, they could only be used as a tool to spot problems. But nothing more. So I think that all what is not needed for such purpose should not be logged.
... consists of tcplog and our RADIUS log.
Essentially that gives us a list of which IPs are in communication with our own, and a list of who was on what IP at the time the communication occurred. We keep our RADIUS log for 6 months for billing purposes and dispute settlements over billing, and our tcplogs are kept for one month.
Well, really, to save their ass, they should log when data is transferred, but not where or how much, or what it is, for that matter. Sort of like how phone calls are recorded at the phone company, except this won't keep the destination, because that can easily saturate a log. (imagine the logs from multiplayer games where you contact other computers for peer.. or someone after one night of Gnutella.. yeeesh). On the flip side, they could always just keep track of things like user login/logoff, any server accesses they did locally (say, to DNS, mail, news, etc) which would do about the same.
I feel that ISPs should be limited to logging who was using a particular IP at a particular point in time.
I do know it is possible using anonimity services to retrieve just about everything you would want off the net. This however is slow, awkward and inconvenient but the biggest downside to these services is that they are only is use by the technically ept.
I feel that 'ordinary' users should be protected by law so that no more information is gathered from them than is gathered from a hardcore conspiracy theorist that routes all his traffic through unregulated offshore servers.
Just because PGP exists and is freely available (well almost freely) doesn't mean that those who are not savvy enough to use it should be punished and have their communications needlessly intercepted.
Yeah, suuure. Get yourself a copy of "1984", read it, and learn why total surveillance is a bad thing. The very existence of such data is a danger in itself, because it can be used to commit crimes, and you can never be sure who eventually gets hold of it.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
The ISP is responsible to maintain a log simply by the fact that they want to service their customers well (assuming that is the case). There is currently no legislation or laws of any kind (that I am aware of) that FORCE an ISP to keep logs of it's users. With this in mind, the logs needed to maintain one's own network, and making logs of abuse issues, as they arise, is probably the best way to go. Now, if you are a large ISP, you should seriously get your bigwigs to consider this little puppy (pdf file): InverseIP Insight It is not exactly "cheap" from what I hear, but this is the best ISP utility I have ever seen and there are many large ISPs deploying/using it. Not only for technical (testing the network and KNOWING when there is an issue), but for technical *support* as well, as this program tells every last detail down to what initialization string their modem is using. You can know what a customer's issue is and how to fix it before they even call. Perhaps ISPs will start calling their customers before they call the ISP?
Bradford L.
Bradford L.
http://www.modemhelp.net
They should all be archived indefinitely.
I think you will find that most states have document retention laws, which specify for how long you are able to keep certain kinds of documentation. Lawsuits have been lost because companies did not comply with these laws, i.e. kept logs/documents for too long.
You might want to recheck the laws in your state before you start keeping stuff "indefinitely".
Let ISP's log as much as they can. They probably hire a cheap MCSE-dude who will never be able to do anything with 'em anywayz... I mean: information is only information if it's interpreted by someone who knows what he's doing...otherwise it's just data. And the more data around, the bigger the chances on certain data getting lost..or untraceable... It's time to start worrying once ISP's start hiring skillfull personell! Not when they start glogging up their own server-disks with logs.
What options does a network administrator have for retaining forensic evidence in case of abuse
This also ties into the carnivore question about faked emails. I've gotten some harrassing emails and considered forwarding them back to the sys admin of the jerk in question. However, realisticly, I could send anything I wanted with FWD in the title, and without digital signatures, they wouldn't know if I was forwarding a real email or not. But what kind of logs could they keep that they could confirm the authenticity of a message without invading the privacy of the user?
Now, bearing in mind that I don't do this for a living, wouldn't it be possible to set up a logging program that ran a metric on each message that came through, based on date, to and from and message content, that could not be reversed to actually produce that data, but would have an astronomically improbable chance of being reproduced by a fake message?
That way, the logs kept, just looking at them (even by the ISP) would tell them nothing but how many messages had gone to and fro from the whole ISP. But if someone came to them with an "incriminating" or "harrassing" email they could (at their discretion or under warrent) confirm the authenticity of that message actually having been sent by their service. If each ISP used their own metrics and kept them private, it would be very difficult for anyone to fake email evidence. This would be useful for both law enforcement/people being harrassed and the innocent but framed.
So, is this kind of log possible, and would it satisfy privacy advocates, since you couldn't even tell "how aften and when used" for any given user?
-Kahuna Burger
...will work for Chick tracts...
Says log everything and forward it to the police on request (well, over simplification of the truth but thats about what it amounts to).
Alot of ISPs are _threatening_ to pull of the UK because of this.
While most Slashdotters will answer a resounding no to those questions, what happens when child pornography comes into play? Should a police officer, or the FBI even, be able to demand an ISP hand over their logs, and examine them for people who have downloaded child porn? (Not exactly the easiest search, but I suppose doable none the less).
I think that determining who has access to the logs is perhaps even more important than determining what to log in the first place.
./configure
make comment
make post
You issue the license plates for the vehicle
they are using. Then they can go anywhere
on the internet and do what they want. If
someone with a warrant comes and says:
who did you give that plate to, then you
give it to them. And only what the warrant
specifies.
If you are going to keep more information than
that, then you should inform your cusotmers that
you are keeping tabs on them. But I dont think
you will keep your customers for very long
if you do.
Remember that a lawyer can get access to almost
anything for any reason. And it can be a civil,
not criminal matter: Custody, Divorce, Libel.
Do you really want to be the vessel used by
an unscrupulous lawyer doing a character assasination for his client on your customer?
And do you really want to get supoenaed every
day for access to your logs?
The ISP should log nothing, out of their own self-interest. Anything they need to log, for their own purposes, should be destroyed after use.
Although it may be useful to Starr to find Lewinsky's book buying history, it's not good press for Barnes & Noble to have the existence of this log disclosed. Similarly it's never going to be in the ISPs interests to be at risk of having logs subpoenaed. The only legally-secure defence against this is to not have the logs in the first place (and this may require a traceable and provabel process to show that any that did exists have been destroyed).
The new Regulation of Investigatory Powers bill is due to be passed in the UK soon, which means that on request (i.e. a warrant issued by the Secretary of State is produced), an ISP must be able to intercept all traffic that a particular customer sends or receives. If you haven't got such a warrant when you intercept traffic coming from or destined to a UK citizen, then you are in breach of the Interception of Communications Act, and so you shouldn't be doing any logging at all.
To be honest, I don't think the harm is in the logging - it's what is done with the logs. Disclosure to third parties is definitely illegal and unethical, but the use of this sort of data within an organisation can also be dubious. How much would your marketing department like to know about the 'real' (read 'secret') interests of all of your customers?
I say you guys have got it pretty easy in the US, but at least we're now getting clear legislation (even if it is b0rked) saying what we can and can't do over here in the UK. To easily answer this question in the UK though, does require a few hours with a copy of the Data Protection Act, the Interception of Communications Act and the Regulation of Investigatory Powers bill. Even then, you're probably wrong.
As far as we what we do is concerned (as an ISP) - we log enough for billing, and we have some machines running an IDS in promisc. mode to pick up scans, viruses, etc. going across the network. Apart from that, it's all pretty standard syslog-out-of-the-box.
--
With the hash, the data can not be retrieved as such, but it is possible to verify objectionable content as genuine and not forged. This would be in the "kiddie porn/death threat/Metallica song" category.
These logs should be expired in a reasonable period of time. Any sufficiently serious death threat could not fail to be investigated within 30 days. Any behavior which is not repeated within that period of time can be considered at an end. Tough for the slowpoke.
Otherwise, no content logging, and no intrusive logging such as unauthorized snooping on what software is being used and how.
Naturally, my ISP keeps logs for that traffic (Inktomi boasts that its Traffic Server can write many different log formats), in part to deal with abuse.
As you might also expect, the privacy policy does not directly cover these logs. It makes promises about some very specific types of information, but does not make any general statements that obviously pertain to types of information not covered in the enumerated, specific types. Result: I think most lawyers would say my ISP could sell access to DoubleClick, the FBI, or anyone else.
Checking your system
So are you using a proxy, but don't know it? You can check pretty quickly (though I should warn you, while a positive/proxy result is conclusive, a negative/no-proxy result may be a result of the proxy configuration, as the systems can be set up to bypass the proxy for certain sites, or to only use the proxy for certain sites, etc.).
Step 1: what's your address?
Check your current address for whatever network adapter (ethernet card, PPP/dialup device, etc.). In Unix or Linux, something like '/sbin/ifconfig eth0' will do; in Windows 9x, run 'winipcfg'; in Windows NT, 'ipconfig'.
Step 2: what address do web sites see?
Go to a URL that will show you the environment variables passed to CGI scripts, like http://www.cgihost.com/cgi-bin/env.cgi or http://www.ualberta.ca/htbin/dumpenv.pl . Look at REMOTE_ADDR. Reload several times. Does it change? You might see some other proxy-specific variables like HTTP_CLIENT_IP and HTTP_VIA, depending on the proxy server's configuration.
Step 3: interpreting the results
If you ever see a REMOTE_ADDR value in Step 2 that doesn't match the local address from Step 1, yet you don't have a Manual or Automatic proxy configured in your browser, then congratulations, you're behind a transparent proxy, and should assume that all your Web traffic is being logged.
http:// vs https:// For regular HTTP, there's a lot they can conceivably record. The URL. Your cookies. Where you came from. Etc. For https:// it's a bit better. All they can do is record where you connected to, and when. Even this information might be deemed valuable, e.g., someone frequently connecting to many banking sites probably isn't eligible for low income tax credits. https:// is somewhat like encrypting your email: they can't tell what you're doing, but they can tell who you're contacting.
I've complained via email a few times, and received a couple polite emails from the technical staff. But nothing has changed in the official policy, so my ISP is still free to share my complete Web usage history with whomever they wish. Highest bidder? Most pushy government agency? I can't say.
-Peter
I don't think there is any purpose of having logs lying around for more than 2-4 weeks. You should log everything, do not look at the logs except if you suspect or know there is something wrong.
I don't know for ISPs since I'm administering the Unix domain of a hosting company. Since we are usually the victim of an attack I know that I'd report to an ISP whitin a couple of days of the attack. If I find that the security of a server/subnet have been compromised for a longer while than 3 days, we usually resoft everything in that area and restore the latest safe backup of the content, the system and binary that are executed with this content are usually replaced with a later version. I seldom find any use in reporting such an incident to an ISP based on the logs that are kept, because they can be pretty different from what happened (as in tangeled with).
I know of a safer method for logging even tho I don't use it. Using a serial port for logging for instance, not beeing able to mess up the log from the server that actually generates the log is pretty secure.
For internet access: All that needs to be logged is who had what IP at what time. So you can track the odd email threat back to it's source, etc.
For web servers: The standard log, with referers, is a nice thing.
--Mike--
Maybe some of you have not worked at an ISP, but ISP's keeping logs is very important, if only to combat SPAM and other forms of abuse.
These logs should include:
* Radius logs - username, port, and time, (Caller ID or npanxx info if you can get it), and IP assignment.
* SMTP logs - SMTP ID. Actual copies of emails would require too much space than available to any ISP.
* NNTP logs - again ID information only (NNTP post ID, date, time, etc).
* Accounting logs as relevant to specific devices - for instance, shell and web servers which allow for telnet/ssh access, ftp servers, etc. This is not spying, this is good system administration.
* DNS - knowing about those lame delegations is a big help. Especially when your customers routinely register domain names with your name servers as authoritative but fail to alert you!
* Most important, accounting logs for root level commands as executed by the system's administrators. This can be a sore spot with some admins, but logging into a machine as root or su'ing immediately to root after login does not present accurate data as to what the admins are doing on a box. Using sudo or one of the other packages and maintaining an adherence policy to its' use should be expected. (Yes, yes there are ways around it..).
Most of these things are standard practices for any of you who have worked for an ISP. I could care less what people were doing online unless they were violating our TOS/AUP and generated complaints. At that point, we needed to know who was doing what in order to fufill our contractual obligations to all of our customers.
All kids love Log!
What rolls down stairs,
Alone or in pairs,
Rolls over your neighbor's dog?
What's great for a snack,
And fits on your back,
It's Log...Log...Log!!
It's Lo-og, Lo-og,
It's big, it's heavy, It's wood!
It's Lo-og, Lo-og,
It's better than bad, It's good!!!
Everyone wants a Log!
You're gonna love it, Log!
Come on and get your Log!
(Everyone needs a... Come on and get your... You're gonna love it, Log!)
Log, from Blammo!
--
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidemessage(this);
* Q
P.S. If you don't get this note, let me know and I'll write you another.
The argument that nobody should mind unless they are committing a crime always pops up when some invasion of privacy is being advocated. What constitutes a crime may well look very black and white when you mention child pornography but what happens when some future or in some case current laws start to gnaw away at basic freedoms.
Regards
To put it simply, it's not what information that is stored by the ISP that should be in doubt; it is what is done with that information. Log whatever is needed to protect people from abuse, offenders, etc, and ONLY use that information to those ends. Lay out strict guidelines on when this data can and cannot be accessed ahead of time and then stick to those rules. If you have the integrity to worry about an issue such as this, then you should have the integrity to keep yourself in check.
Does my ISP have any responsibility to log anything whatsoever? Not unless they explained they had were going to in the term of service agreement, or publically announced they would elsewhere.
However I would expect to minimally see logs of all failed logins to my account. And Ideally every action on every port should be logged and archived. However this is not an ideal world and I'll settle with whatever information directly pertains to my account and data.
I think it's hilarious
--- RFC 1149 Compliant.
Having been in the ISP trade for a number of years until I finally escaped awhile back; the best advice that I have for ISP's is to keep it around as long as the ISP thinks it's necessary. Let me add a note to that statement, when you are determining "necessary" be prepared to have a government agency outside your door someday and take weeks to months sifting through the data. We didn't keep any network level information (we could care less about routing), but we did record all dial-in access authentication for 6 months.
We recorded: login/logoff times, ipaddresses, bytes in/out, and on digital lines the CID
We didn't watch what you did on our network, but we did record the initial logins for general accountability of our customers, tracking of which pop sites need modem/bandwidth upgrades (lots of other tools used here, but the more data the easier to get financing for upgrades). Our necessary was 6 months, for others it may be less, (we were keeping somewhere around a 1/4 to 1/2 million dialin records a day).
We had a very formalized schedule and I suggest any ISP to get some form of it onto paper. We talked with many different government agencies and the resounding word was that the ISP business is unregulated, they don't care how long the records are kept around; but if they were to contact us and someone tells them 2 weeks, another tells them 6 months, and another tells them they never get deleted, that they are going to show up on your doorstep to do an inperson hello. As long as everybody gives them the same answer then they are fine with you giving them the logs, if they smell a hint of something fishy (even if it's only a simple communication error) they are coming in with a warrant to seize your equipment, and will go through all your equipment at their leisure where you may or may not allow your business to run.
Spelling & grammar checker off because I don't care
The purposes of a security audit trail are two: 1. to allow you to reconstruct what happened when your system was attacked or failed in some way, and 2. to serve as legal evidence, when you do go after someone.
The detail in the log and the degree of protection that you give it should reflect your goals, but remember that some malicious intruders trash the system they've hacked if they suspect someone is onto them. That would suggest some degree of care is warranted. If the audit trail is to be used as legal evidence, you'll need to demonstrate that it hasn't been modified after the event. If it is only to help you analyze what happened after the fact, you should at least protect it from malicious access, since the hacker would love to know what you know.
What do you log? As has been said, packet sniffing content would take ungodly amounts of storage, and if you're an ISP, you really shouldn't be doing it. It's Just Wrong (tm). Once again, it depends how tyrannical you want to be, but I think that just monitoring what IP's are hitting your boxes when is sufficient for most security concerns. At the most I'd say take note of traffic patterns, just incase a customer's box has been broken into and is doing things it didn't normally do.
Should logs be permanent? We all should be able to come up with one real simple example of a corporation that was burned by e-mail leaking out that honestly shouldn't have. Corporations are now beginning to take a policy of purging e-mail stores often, so it doesn't come back to bite them in the ass. Is this ethical? Probably not. Which is why you have every right to be dumping your logs too. If corporation XYZ comes to you looking to see if the maintainer of corporationxyzsucks.com is one of your customers... sorry, you dumped the log. Don't get me wrong here, I'm not saying that ISP's shouldn't help big evil corporations if someone from them DoS'd them. I'm just saying that ISP's have a right to 'lose' information just like corporations do. Things are much less of a hassle that way.
Legal issues. If I were a customer of an ISP that suddenly decided to start logging everything, they damn well better tell me that their terms of service are changing. Anonymity is something I value, and is a key factor in my ISP choice. What with all the DoubleClick-ish privacy things going on right now, I would not get yourself into that mess. Let your customers know exactly what you're logging, they have every right to know.
Perhaps this is all remarkably obvious, and the opinions have been karma whored up by now, but I just thought I'd offer my two cents.
Many people seem to complain about being harassed online, whether it be through email, chat programs, or what have you. The thing is, though, that no matter where you go, online or real-world, you run the risk of running into someone who does not agree with you. A prime example, school. I know when I went to high school, many people were abused, harassed, or even assulted because of different beliefs. And most of the time, they would not even be disciplined by a faculty member. I guess this is the way of life. If you want to avoid being beat up in school, either don't go, or hire someone to protect you the whole time. If you don't want to be harassed online, don't use ICQ, mIRC, read you email, or browse the internet. Unfortunatly, in any activity that involves many people in one place, someone is bound to disagree with someone. It should not be the ISP's responsibility to log everything, that should be the software developer's choice. For example, mIRC is able to log a session to a text file, and can be printed out later if necessary. One thing that should be tracked are the origin and destination of packets. This way, someone can be identified if they like to deliver DOS attacks. But again, as long as you are online, I feel that you must accept that there are people out there who don't like you, and like in real life, may do something to make you a little mad.
--- At my sig, unleash hell.
It's a public network. Use encryption. ISP's should log as much as they want.
Of course, what they do with this information is the important part.
ISPs should and probably do log ALL data coming through their routers. After all, that is very valuable data to some people.
And don't kid yourself that many ISPs are not. And unless you are administering the ISP yourself, don't kid yourself that YOU are not having all your network traffic recorded.
It is like Microsoft or Real sending pings of your net traffic back to home base across the net. There is little motivation for an ISP to abstain from such activity. It is very tough to get caught. And some people will pay for your data, especially if you preprocess it properly.
If I were running an ISP, whatever server logs I did decide to keep, I wouldn't keep them long; I'd be too concerned about potential abuse by overzealous law enforcement or litigants to want to retain them. If you consult a tort lawyer today, you'll be told to get rid of your company's old email fairly rapidly so that it can't be used against you in court. I think that this would be a smart strategy for server logs as well.
Drop forged packets (Essentially anything not inside your address range originating from within your address range) and log those too.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
For the wood-burning emergency generator. Duh!
No. Nice try, though. Its the emergency backup heavy duty LART stick.
s/ISP/ASP/g
Asps are adders. Adders need logs to multiply.
-- What you do today will cost you a day of your life.
Logging content not only risks the common carrier
status, but is a 'Bad Thing'. OTOH, logging
connections makes sense, violates nobody's privacy
and is still very useful. Joe logs in at time A,
receives email with header B, etc..
Similar to what phone companies do, they have
records of how long a call lasted, from whom,
to whom, and no record of the call itself (I hope).
ICQ#2584116
-- d'arcy poirot
pointed out (so I'm stealing their post to a degree) that there is a difference between security and privacy.
Should you use encryption, to keep your data secure? *YES* absolutely.
Should your ISP be forced to keep your surfing habits private? *ABSOLUTELY*
Should they be allowed to log as much data as they want for their own analysis later? *ABSOLUTELY*. Why? Because they *can*. It's *THEIR* network. If we say 'they can't' they can just put it in the contract; you want to use @home, you agree that we may log as much information about packet flow as we want. Period.
First, I do not currently work at an ISP, but I have done. I also have administered arrangements for remote access at educational estabishments, thereby effectively being an ISP for the students and staff. This was a VERY thorny question for us in all those cases. We recorded who connected when, with what IP, and who accessed the services we provided, again recording the source IP. Those logs were kept for a few months. Logs of suspected probes were kept for a few weeks, overt attacks for longer. That was it. With this info we were able to pin down the account associated with any abuse reports and spot a few compromised user accounts (usually because somebody used the same password for everything and it got cracked somewhere else) by seeing the same user pop up twice from different locations at the same time.
The logs we kept on OURSELVES though were much more thorough. Anything one of our machines did was watched somewhere and whilst most of those logs were short-term and verbose enough to require scripted assistance to scan in any meaningful manner we made damn sure that we looked into everything that poked up above the background noise level there.
Privacy was important too - in all cases it was clearly understood that discussing logged info with anyone outside the admin team apart from the customer who owned a suspect account was cause for getting fired immediately. To even discuss it with the customer required written authorisation. If anyone else wanted the info it had to go through the head of the admin team. Marketing folks, the billing dept, top level management (by their own request) or support staff did not have access to that raw data and it would only be turned over to anyone outside the company with a court order.
Other guys at the company sometimes accused us (the admin team) of being anal about it and I guess we were, but the complaints sure dried up when the policy saved us from getting our ass sued.
# human firmware exploit
# Word will insert into your optic buffer
# without bounds checking
I had a
Loggin connections is fine, but logging content is invasion of privacy isn't it? Once an ISP starts logging content that is passed through their system, it will be considered invasion of a person's personal privacy. As a citizen of a country, you have rights, and you are guaranteed these rights. Although these logs may not be used until an attack occurs, they are still invading a person's rights online. How would you like it if an ISP keeps track of everything you do. For example, Who you email and what you wrote and who you icq and your message. Even if they wanted to do this and were able to by law, wouldn't they waste a lot of disk space on their system? If you have thousands of packets going through your system, wouldn't logging all the information in each one fill up your hd?
I work for an ISP. I'm not a Network or System Admin., But, I work closley with them. Seems to me that the logs we keep have saved our ass on several occasions. From threatening email sent by one of our Members children from a Hotmail Account(Which we used the logs and headers to determine the IP Address and therefore username) to Attempted Hacks. One doesnt need to log everything just the "Protect Your ASS" bare-bones logs.
"I think you know what I'm talkin' about, Mr. President; We're gonna kill us a mummy!" - Bruce Campbell as Elvis Presley
I don't believe they should keep any by default, other than those pertaining directly too their localized network.
I believe that the option to have whatever type of logging is possible should lie with a given client, and that all clients should be told expressly what logs are being kept by their ISP.
One of an ISP's main responsibilites should be the privacy of their clients.
An ISP is a road that leads out too the world. Some people choose to live in gated communites where a physical log of everyone who passes through the enterence gates is kept and suspicious looking fellows are detained by a security guard. Other people buy a plot of land right next to the highway, and don't care who goes by.
-T
Old truckers never die, they just get a new peterbilt
It is my opinion, from a privacy standpoint, that an ISP should log traffic on it's systems in one of two cases. The first of which is to comply with Federal, regional, or local law enforcement. Now this is an unfortunate one as we have all witnessed abuses by the police in recent years, especially regarding the largely un-legislated domain of the Internet. The other case is if the ISP has an real proof that thier internal user agreement is being broken, a/o policy is being disregarded by a user. Now this is only a precaution to avoid the first circumstance. Other than this no full traffic logs should be written let alone kept. Especially when regarding the rash of recent ISP break ins over the past 6 months or so it seems that more than just a few sysops would have access to thse files. Well that's just my 2cents -=AlterEgo=- its the nature of of my circuitry
I noticed that there was a comment about " router configs for such logging " . Anyway , I do router and switch configs on a daily basis. Logging would put a huge strain on a router and slow down the response of routing request. I feel that logging would be better done on a transparent proxy or a firewall. You can scale the hardware to suit this need ( more processors and such ). Routers usually have somewhat week cpu's ( and don't really need anymore as long as you don't put logging and such on it ). Just my 2 cents worth.
Logs of a DDOS attack can just be tossed if you're not seeking litigation though, since they are relatively useless.
Eh...
Lumber has a million uses.
I have to respond to this for one reason - namely the line "I've certainly had friends be harassed and threatened online, and turning a blind eye to everything but attacks directly against the network doesn't seem right either."
I don't think a lot of people really think about this stuff when they do it. Yes we all want to be safe. We want our friends to be safe. Sometimes we even want those we despise to be safe as well.
But where do we draw the line? This type of thinking is as dangerous as blanket "log everything no matter what!" As the story suggests, Barnes and Noble learned a very important lesson when they kept track of everything.
Remember, the moment you give up just one of your rights to privacy is the moment you have given them all up. Also remember that "protecting the little children", as the religious right likes to say all the time, does not mean that MY rights as an adult should be erroded because of whatever draconian law they want passed.
If it's your policy not to keep logs of any sort, then they can't be subpoena'd in court.
Just be sure to wear the gold uniform when you beam down -- you know what happens when you wear the red one.
I think a reasonable log would be one similar to what the telephone companies keep. A log of who connects to the ISP from which dialup line, calling from whatever phone line (from CallerID), at which time and is given this IP number. This will give them adequate information that they can use to get a warrant (in the States, anyway) for a more intrusive search on THAT PARTICULAR USER.
Scanning the data in packets for anything that is just passing through the ISP's network is not such a good idea. I don't know who works there. I do not know if they are maliciouos or not. I do not even know if they are competent or not.
I'll handle my own security, thank you.
People seem to be willing to give up their rights for security. The problem is - the security is just an illusion. Law enforcement, by design, is supposed to react when a crime is committed, not to prevent crimes from happening. More and more, people are giving up their rights to have a proactive law enforcement body. By giving up your rights, you often create a more pressing problem than the one you are trying to solve by giving up you rights.
I think it was Ben Franklin ( or Thomas Jefferson ) who said "Those who give up their liberties for security, are deserving of neither". Don't give up your rights just to feel safe. It does not work. You end up trading fear of criminals for the fear of the government. I'd rather deal with a script kiddies than fight with the NSA or AOL.
Remember, You are unique...just like everyone else.
Here waer this tracking device, and you are not allowed to leave your house with out asking us first, you shouldnt mind unless your going out to commit a crime. Is such ignorance and stupidity that rampet in our world? Can I move to a more civil and intelectual one? Maybe we should just install cameras in everyones house, they wont mind unless they are commiting a crime. Not all things are black and white. If they were all crimes would be punished by death and you would be screwed. 'Sir, you were speeding back there, Im going to have to ask you to step out of the car so we can execute you.' Yea, your about as retarded as a lobotomised eggplant.
IceBerg
A friend of mine recently lost a database on his site by someone 'social-engineering' a username and password to the web interface from one of his staff. The site logged the ip that the person doing the deletions came from, but it was a proxy's IP. The ISP in question didn't store their cache logs for more than a day or two, and so could not tell him the account he was attacked from. They had backups of the data and were back up in minutes, and the pleb who leaked the password was beaten soundly, but it would have been nice to know who-dunnit :-)
/* Wayne Pascoe
Simply monitoring a T1 with a packetsniffer doing decent filtering can easily trash a fast 30GB HDD.
A T1 is 1.5 megabits/sec. To fill up 30 gigabytes recording _all_ data sent across the T1 (no filtering) would take about 44 hours. If a cracker leaves a sniffer unattended for that long, I have little sympathy for them.
Overflowing a user account I can believe, but I would be amazed if drive overflow was a significant problem for the vast majority of packet-sniffing crackers. Heck, cut out HTTP and take only the first few packets of an FTP or POP session's data and you've reduced your data load by a factor of 100 or more, while keeping the information you're interested in (passwords).
In summary, I don't think that drive space is a problem for a half-way competent sniffer.
- Detection of and response to a security breach in progress requires special attention to legal, regulatory, policy, and ethical matters so that the needs of security administrators and the forensics requirements of law enforcement are balanced with the privacy rights and expectations of users. These matters will be addressed with the Secure Packet Vault, a tool for rapid response to an intrusion incident or for continuous oversight of a subnet. CITI will also investigate the uses of cryptography to address policy-imposed data handling requirements. Vault Architecture The packet vault hardware is composed of two 133 MHz PCI-bus Pentium machines interconnected via a private 100 Mbps Ethernet. One machine (the "listener") is also connected to the network under test, and is used to capture and encrypt the data, which are then sent over the private Ethernet. The listener stores no packet data on magnetic disk. The other machine (the "writer") receives the encrypted captured data and stores them to magnetic disk for subsequent writing to CD-ROM. The two magnetic disks on the writer are attached to a dedicated SCSI bus; a second SCSI bus is dedicated to the CD-ROM recorder (CD-R).
a longer paper is hereUNIX-derived operating systems were chosen for both platforms because of our familiarity with UNIX and the flexibility it provides. OpenBSD 2.0 was chosen for the listener because of its kernel BPF support; Linux 2.0.0 was chosen for the writer because of the early availability of drivers for the CD-R.
All data are encrypted to allow selective release of conversations, where a conversation is defined as all communications between a pair of IP addresses. Packet IP addresses are obscured by substitution, and packet data are encrypted under a symmetric key unique to each conversation. Material needed to reconstruct all conversations is remembered and encrypted under the public key of a trusted third party.
Amateur.
<clickety click>cd /users ; rm -rf *a*<click>
There. Now there's plenty of room. Efficient yet subtle. Generates more confusion than blame.
The only logs that should be kept are ones that are abolutly nessacary to the running of the ISP. Let's face it, unless you meet someone from the net, there is nothing somebody can do online to actually hurt you. Written text is just words, but it so nobodies business where anyone goes on the net, what they do or anything else. There are many ways to catch criminals, and having a proactive big brother is not the best way. One other thing, the tracting of users, only relates to the machine that is online, there is no way of knowing who is actually using the computer. Anyone could be, on someone elses acount. As far as children being harm, the answer there is have stiffer punishments once the crime is committed. Otherwise the continuing trend of giving up our rights to feel safer and not punishing criminals will just keep spirialing out of control.
Yes you must respect privacy, but you shoudl also state clearly and in laymans terms what your privacy policy is, and stick with it even when times are tough.
What you should log depends on what your needs are and also what services you provide. Remeber though that you may be held responsibe for someone abusing your network, so it may be wise to keep track of who is on it and from where.
send flames > /dev/null
Only 'flamers' flame!
What, he didn't see that section in the Echelon Users Manual that the FBI/CIA/NSA sent him when he started his ISP?
Good ol' USA!
I think keeping point to point logs is a good idea, if for no other reason than your own protection. The real question is who should have access to those logs. The answer is no one, unless there is a problem. If it is a cracker problem, then the only person who should have access to those logs is you. If it is a legal problem, then the logs should only be released by court order. Don't volunteer to third parties that the info exists, and let your users know what your policy is. I also agree that the logs should be encrypted. If a user has a complaint or problem, you could offer to monitor data to and from that particular user as a service, but should require written permission from user.
I can tell only about the Finnish law. Currently logging is always prohibited, except for some specific reasons, such as:
- billing, for commercial services (rather obvious).
- resolving cases of abuse, in which case the information may be communicated with other involved organizations.
- "development of the services" (I don't really understand what they mean by this).
- for marketing purposes, if explicitly approved by the client! (This definitely includes using cookies for collecting marketing data, if not approved.)
Interpreting all this is not very clear to me. It seems that logging itself is ok, because you can always expect some case of abuse. But you are not allowed to use the logs for anything except for these purposes.The ethical questions are of course about personal privacy. It's not just the "Evil Big Brother Government", but it might also be...your big brother (the biological one)...snooping at your voyages in the internet. Or your ex-lover, or your wannabe-lover, or your worst enemy...
It also reads: "Who has received or otherwise acquired information about a confidential telecommunication, which was not intended for him, many not unlawfully communicate or use for his advantage the information, or knowledge about the existence of the information."
Start to gnaw away at our freedoms? The law is already doing that. The DeCSS lawsuits is the first thing that comes to mind. Maybe if they wanted to catch real criminals they should repeal the bogus laws that make crimes out of activities that are not unethical. Then maybe we'd be more trusting and allowing of some monitoring. To catch real crooks. With strong legal safeguards. But while they are actively oppressing us? Hell no!
Just because it CAN be done, doesn't mean it should!
> The argument that nobody should mind unless
... before being siezed as child porn in one US state and records of who rented it subpoenad for grand jury indictment proceedings) and the fiasco with the removal the works of David Hamilton, Jock Sturges and Sally Mann (regarded as some of the greatest contemporary photographers by their peers) from the shelves of Barnes and Noble bookstores come to mind as examples of what can happen when one person's interpretation of an artistic work differs from another's.
...
> they are committing a crime always pops up when
> some invasion of privacy is being advocated.
Indeed. This claim is one of the fundamental principles of all totalitarian regimes.
> What constitutes a crime may well look very
> black and white when you mention child
> pornography
Does it, really? The case of The Tin Drum (an excellent movie that won numerous awards
There are even those who consider Michelangelo's sculpture of David to be pornographic, even though a replica of it stands before our very own Library of Congress. There are those who are absolutely shocked and disgusted by some of the great masterpieces of painting displayed in galleries like Sancouci and the Louvre.
Furthermore, there's the issue of the 'net being a worldwide medium. While it is considered perfectly acceptable for women to sun themselves, and for children to go naked, at the public beaches of the French Riviera, they would get arrested for public indecency in California.
So, in an email exchange between someone living in the US, and their relatives living in Europe, which "community standard" gets applied? What should happen when one of the European family members posts snapshots at PhotoPoint of the outing to the beach that include naked todlers playing in the surf?
It's not as black and white as it may seem at first glance.
> what happens when some future or in some case
> current laws start to gnaw away at basic
> freedoms.
It's happening every day. Just look at the DeCSS case.
And then there's the FBI's Carnivore system, which, by it's very nature, violates the fourth ammendment rights of ISP users by reading every email passing through a network that it's connected to, looking for "something illegal".
The FBI defends this Mongolian Hordes approach, invading the privacy of every single subscriber to an ISP in order to find the *one* who *may* be discussing illegal activity in an email, by stating that there is no explicit law to prevent it.
George Orwell, move over. Big Brother's big brother is here
Today, if you did that, you'd be overwhelmed. But it's useful to have the capability and to log such stuff during a peak period now and then, just to get an indication of what junk is out there. Most denial-of-service attacks will show up in such logs, of course.
well, i don't know about ISP's, but i have come under fire for this issue before. i ran the network at a medium size company for several years. i was one of those administrators that logged enouph information to run a few volume reports from and when the HD was full on the logging machine, i deleted all of them. then out of the blue we get a sexual harassment suit against us from a fellow employee. at this point i was forced to sit at a deposition and explain the reasons i didn't keep an archive of our logs. it was a 5 hour experiance that can only be described as less plesant than a tax audit. because we didn't have the logs, we lost the case, and i got fired. that entire experiance has re-written my position on logging and privacy in the corporate world. now my position is very easy, log EVERYTHING and keep it FOREVER. i can say that as long as i am responsible for information systems for any company, i will have a complete history of what has transpired on my network. now i have everything logging to a file server, at the end of the month, i run my reports and burn all the files to CD that then get put in a save deposit box in a bank down the street. (yes, that does include email messages and attachements that go through our email servers) i copy everything. i WILL NOT be put in the situation that i was in before. i also convinced the upper management to ammend the employeement contract to state specifically in plain language that we do this. we have gotten heavy resistance to this, but our offical response is "if you don't like it, there is more than one technology company in this city." now thats corporate america, but i don't see why ISP's should be any different. the way i see it, the only difference is the sheer volume of data to be archived. take from it what you paid for it.
*** I suffer from a colorful array of psychological problems
<>
That's a different issue. Lewinsky's purchase history was based on *financial* records. Financial records MUST MUST MUST be kept to help eliminate errors, and correct them when they do arise. Not to mention for tax purposes...
-JF
MrJoy.com -- Because coding is FUN!
the problem with this is an ISP is not a priest or doctor. it is a business. a business with the intention of making money and covering its ass. it should log and keep the information that it needs to do so.
*** I suffer from a colorful array of psychological problems
now, i realize that my situation was different because it was a corporation and not an ISP, but in our case, i know for a fact that the logs would have proved us inocent. a very big part of the reason the other party filed the lawsuit was because they knew we didn't have those logs to prove our case.
in my view, the people that are afraid of the law and afraid of the litigation are the ones that need to be watched because they are the ones with something to hide.
*** I suffer from a colorful array of psychological problems
How else are they going to have a firewall?
Geek-grrl in training
To truly understand recursion, you must first truly understand recursion.
Somebody put the lid back on please. I don't think we can survive such an overload of BS.
-- I have marked myself unwilling to moderate-- I don't have other accounts to artificially inflate the karma of
Mod this up! He brings up some VERY valid points that most idiots^H^H^H^H^H^Hpeople don't even think about when they advocate laws that erode our rights.
First posting isn't trolling. It's...first posting.
I used to work for an ISP as a network admin and security "expert," so I've got some experience in this area.
When I was hired on I implemented a policy of making sure all the server clocks were correct (via NTP) and synchronized to each other. That way our logs would correspond, and we could check all machines if something went flaky on one. Sometimes cracking happens on only one machine; sometimes it happens on multiple machines almost simultaneously. The first step towards having credible logs is to make sure the date stamp is correct on all of them.
Another policy I would implement, if I had the support of management (and adequate disk space) would be to save everything except for the data that actually travels over the pipe. I would save a record of all outgoing data, so we would know what went out from our site to the outside world. I'd also probably try and log most or all the traffic on the internal network (our IP block). Obviously an ISP would run into privacy entanglements if everything--incoming and outgoing--was logged (not to mention doing so would require enormous amounts of disk space), and might even be operating contrary to its stated policy.
It would be possible (and would require far less space) to just log packet headers, not the actual content of each packet. Knowing what the packet types are is often a useful diagnostic tool, both for knowing what the heck your network is doing and for (somewhat) knowing what people are doing on your network. Of course, this is better done at the router, probably.
We considered syslog, even with everything turned on, to be inadequate for logging purposes. Of course, any sysadmin worth his salt uses tripwire, ISS, COPS, etc, and keeps extensive records associated with these tools.
We installed ZoneAlarm on all of our Windows machines and encouraged our customers to do so as well--this helped give us a heads-up on several potential hackers. We configured ZoneAlarm to keep extensive records as well.
My personal policy is to keep it, no matter how small or insignificant. Don't trash anything. You should see my pine mailbox directory--it's huge. But of course I can't apply this principle to everyone, as I have no desire to be another fascist sysadmin. But I can imagine there are those who would keep everything.
If you do keep everything, or nearly everything, for crying out loud keep it safe. Have a separate machine that performs as a loghost and make it secure. More secure than any other machine on your network. In fact, make it impossible to get in unless you're on console. Make backups religiously and store them off-site. It may save you or one of your customers in case of disaster or overzealous law enforcement.
Many ISP's are not.
As the System and Network admin for a regional ISP, I can tell you that it's simply not practical to log all traffic. The amount of storage required for such an undertaking is OUTRAGEOUS.
It's further hampered by the fact that 99% of users get dynamic IPs, so there's no easy way to correlate the data with individual users.
Security requires that you have to assume that your traffic is *ALWAYS* being recorded. Paranoia is good, but it's not realistic. Just be sure you present it as such, instead of fact.
--
My comments and opinions completely reflect those of anyone and anything I am remotely associated with.
I agree with all of the above. We recently were slapped with a subpeona in a child pornography case. We are a small ISP in a small town, if we had nothing to give the police, it would have been very bad for business in this small town. The only time ANY info about any customer is given out is by subpoena, and it's happened twice since I started at this ISP. Some may cry that it's a violation of their freedom, but people pay to use a service that WE provide, on our equipment, and ISP's need to be able to protect themselves against things you might do 'in their name'. Our owner takes it quite personally when a user does illegal things on our service toting our e-mail address, etc. Granted, the logs we keep do not delve too much into someone's actual activity. Unless a user has static IP, it's nearly impossible to even see what webpages are being looked at in the logs on the cache machine. Sure, if I went to see what dynamic IP was assigned to a person at a specific time, then went to the cache logs to see what matches of the IP within the specific time frame there were, I could probably see, but it's too much trouble to even attempt without a court order involved. Radius logfiles in particular are very useful in terms of technical support. We can see why people get disconnected, etc, and we do have caller ID on all of our modems, and have caught people who were using other peoples' accounts, etc. with it. When someone calls and says that they can't get online, and we see that they are already logged in, but the phone number matches little Timmy's best friend's house, and that's where little Timmy is right now, parents get a tad upset ;-) The same people that would complain that these things are invasions will most likely be *helped* by these records at some point in the future.
Why did we not here of this sooner? When was this put into place? How long did it take to make this public?
Why ask these questions? The anser is this...
With todays technology, a little time to do something we consider wrong may already serve its purpose by the time the "media" breaks the news. These things will be done, serve there purpose, and then it won't make a difference in some cases.
The question is, how do we prevent such thing from happenning?
Another good read is "Discipline and Punish" by Michel Foucault, in which among other things he discusses the autoculpability effect of the panopticon... in other words, if everyone thinks they are being monitored, they will freak out and admit guilt even when innocent.
As a security admin for an international firm, we're currently engaged in enhancing logging on our Unix servers. As a first stage, we're ensuring that all system messages with a facility level of AUTH are getting locally logged and also sent out to a dedicated logging server. To address legal issues, all system's MOTD's explicitly state "You will be monitored, you have been warned." Future stages will include migrating to one of the enhanced system logging daemons that thankfully use TCP and encryption for sending logs to remote systems. Anyone else rolled out a system logging architecture at an enterprise level? How'd you go about it? -harikiri
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
When (not if) you or a user of yours is a target of an investigation or lawsuit, you will have to turn over ALL 'relevant' logs. And 'relevant' can take a rather broad definition.
As a sysadmin, I keep logs of any event that affects the security of my system, along with enough detail to know who was logged into what machine or using which dynamic IP address at any moment in time.
If somebody uses your system to launch an attack, you want enough information to place blame- no more, no less.
I do not deploy Linux. Ever.
The big limit on back-tracing truly abusive internet users is DHCP. A site knows the IP of someone conencting to it, but most ISPS don't assign fixed IPs-- instead using DHCP to assign them dynamicly on connect. Without a log at the ISp of what IP address was assigned when it becoems extremely difficult to find genuine culprits who need to be dealt with.
(Yes I know a serious cracker wil lstage his attackthrough multiple machines, but the problem children generally don't have that skill set.)
If net-abuse can be reported to an ISP and the ISP know who the user was, the ISP can deal with it. Without that ability, one obnoxious kiddie can get the entire ISp blocked from sites and news groups.
> Okay, you menctin the FBI's Carnivore system.
... which leads me to ask "who's watching them while they're watching us?"), and b) because it's not in their best interest as Big Brother. After all, we can't defend ourselves against a threat that we don't know exists.
... though not in a direct sense.
... and that means on a personal level. The Constitution was written by people who believed in personal liberty, and personal responsibility.
... which means we need to get a *lot* more people off their asses and into the voting booths in November.
> This topic hit the "media" and now everyone is
> quoting it as "violating fourth ammendment
> rights" blah blah
>
> Why did we not here of this sooner? When was
> this put into place? How long did it take to
> make this public?
This type of behaviour by government agencies is a relic of the Cold War. They use the premise of "National Security" (though these days the politically correct form of the term is "Anti-Terrorism") to omit or delay informing the public of their oppressive and blatantly unconstitutional activites.
The fact is that they don't tell us about these things a) because they don't HAVE to (no one is forcing the FBI to tell us how it's watching us
I see Carnivore as a threat not only to the fourth ammendment, but to the second as well
The whole purpose of the second ammendment is to make sure that the government cannot disarm the people. The basic premise behind Jefferson's writing of this ammendment was that the government's military apparatus can not take over the country by force as long as the citizenry can shoot back.
The goverment might not be able to become totalitarian by military means in the US, but there are other, subtler means by which the same effect can be achieved. Carnivore is one such way, though now that it's been exposed, the people can defend themselves against it.
The fact that it took so long to bring it's existence to the public's knowledge is more proof that our government is both dishonest and corrupt.
It is a symptom of a government that has become used to not being accoutable to the people for it's actions. Our government has forgotten that *they* are supposed to be working for *us*, and not the other way around.
As Plato said: "Whereas the truth is that the state in which the rulers are most reluctant to govern is always the best and most quietly governed, and the State in which they are most eager, the worst."
> The question is, how do we prevent such thing
> from happenning?
A good start would be for people to do their own thinking, and to take responsibility for their own actions.
If people refuse to exercise some self-control, of course the government will take over that control.
I know it's cliche, but "The price of freedom is eternal vigilance"
If we want to be free of the kind of oppression that systems like Carnivore make possible, we have to watch for them diligently, and take the government to task when we find them.
We also need to elect government officials who will be honestly accountable to the people, and will hold every government agency accountable
We also need to let people know that there are choices other than the RepubliCrat nominees.
When less than a third of eligible voters show up for a Presidential election (less than 27% of those eligible voted in the last one!), the United States ends up with exactly the government we deserve.