Your problem is you (the major ISP you worked for) were not pro-active about preventing spam. Obviously you intended to do nothing but clean up after the mess was made, and so you did.
Dynamically addressed dialup accounts (which is what most spammers have used and still use) should not have direct port 25 access. Those accounts are limited accounts, anyway, and should send all mail only through the designated mail servers run by that ISP.
After having 2 cases of "spam and run" done on dialup accounts at the ISP I used to work for (and continue to consult for) I put in place a block on port 25 outgoing for all dynamic dialups. That was over 2 years ago and there hasn't been an origination spam incident since then (relaying was also blocked separately). I personally use another major ISP for nationwide roaming dialup purposes, and they do this (I tested it). UUnet should start doing this. Although it wouldn't stop SyberSchool, it would stop most of the spamming which comes from their dialups.
If you run your own mail server or an ISP, just block most of the spam via RBL/DUL/RSS from MAPS. If you don't run a mail server, choose an ISP that does these things for you on theirs. That will block most of your bandwidth waste.
If a major ISP wants to allow a spammer to operate, then the way they should do it is to require it be done from a dedicated circuit, and to prohibit relaying through any mail server not listed in the DNS for the destination address. Such an operation is very easy to block on the receiving end. In addition to RBL/DUL/RSS, I also have my own DNS zone to block my own set of IP addresses.
My point of view on this is that if someone actually wants to be a part of this and get spam, they should be allowed to do so. Likewise, someone who does not should not have to. I'm not opposed to an ISP that wants to allow spammers to send bulk email in a legitimate (e.g. identifiable, easy to block) way. Anything less is, IMHO, fraud (and if the ISP knows it's going on, is a party to the fraud, also IMHO).
What I want to know is if Nace and SyberSchool are sending their email direct (doing normal DNS MX lookups and sending to the designated host) or if they are doing relaying through innocent third party mail servers. If it is the latter case, then I think they should be cut off. If the former, then I have no problem with it because I can block them myself very easily (your ISP can, too, if they want to).
We do make a pure IP available. The cost is increased because our cost to offer it is greater. Those costs include making sure we identify who the subscriber is to be sure we are not signing up someone whom we have previously canceled due to abuse. There is also a real cost involved in static IP, which everyone who does want to run an SMTP server usually wants. This is why it is bundled together (almost everyone who is in one group is in the other). Most of the subscribers who do run their own SMTP servers are businesses, but there are some individuals. And most of them have migrated to DSL as well.
I entirely agree with you on the principles. But from the perspective of a business offering where we have to balance between controlling costs (including the impacts that reduce the quality of our service) and offering what the vast majority of customers want, this is what we end up with. Pure IP is not what most of our customers care for.
Basically, both you and I will have to face the fact that what we want as a pure End-to-End IP is a small subset of the telecommunications market demand. Fortunately, the cost of offering it is only marginally higher, so it is practical to offer it as an alternative.
My understanding of transparent redirected proxy was that it worked with the IP address you used. If it uses the separate Host: header to do an IP address lookup, it's a broken proxy server design. Maybe the redirection is broken for not supplying the IP address. I haven't looked into what they actually did design, but I know it could easily be designed to work right even in your situation. My point is the blame goes to the design they are using, not the fact that they are using transparent proxy at all. Now I need to go check Squid and see if it's designed right... *sigh*
In any event, I would sure like "IP purists" (which I count myself as one) to help me out and find solutions to real business problems like these which preserve pure IP. Please tackle the SPAM issue and HTTP bandwidth redundancy issues as the top priority.
I fully understand that, and might well be hunting around like you (if it weren't for the fact that I get the service for free being a consultant there).
There are tradeoffs. If port 25 is left open, and people can sign up for services and send SPAM, then the quality of our service goes down because we have to spend staff time dealing with the aftermath, and our equipment has to deal with more DoS attacks. Before blocking port 25 we were seeing about 1 spammer a month signing up. After blocking port 25, we couldn't tell if there were any trying to or not (didn't log port 25 attempts).
There were alternatives. For example we could have done a background investigation on people who sign up for service. But that would not be cost effective given the time involved (normally accounts go up within 15 minutes) and the fact that there really isn't any information about who spammers really are. We chose to block port 25 because it was the least costly choice, and would have the least impact on the vast majority of our customers.
As for letting people know, it was our policy to answer truthfully if anyone ever asked; no one ever did. The anti-relaying on our servers has caused a few problems for "road warrior" customers, but blocking port 25 on our dynamic dialup never has raised a tech support issue.
As for the web cache, I do think it would be appropriate to notify customers, and I will bring that up. As for opting out of caching, I think that's going to end up having to be an extra cost item, since doing so means greater use of bandwidth. A third of the cost of business is buying bandwidth.
Incidentally, I plan to set up a 2nd optional cache server (not transparent, must configure as a proxy) which also blocks most banner ads. It will pull from the main cache (less the banner ads). Customers will have the option to use it if they choose to. Is that fair? I'm sure it's not fair to the advertisers :-)
We curently offer what I would call a limited access service. All dialup access is restricted on port 25. Outgoing connections to port 25 from dialups are limited to reaching our servers only. This only applies to basic dialup access. We also have a premium dialup service which includes static IP and is not blocked at port 25. The purpose of all this is to pro-actively prevent SPAM from originating from our network, especially considering that we don't really know who these people are who sign up for basic accounts. We've had people call and sign up and then send SPAM the very next day. In the 2 years that we've been blocking port 25, we've not been the origination point for SPAM (relaying has been blocked way longer).
Now, is this something people would consider to be NOT pure end-to-end IP? Considering that the email content is end-to-end, can this be considered to be a valid exception?
BTW, we also block ports 137-139 and 31337 at the border.
We are also considering HTTP caching on port 80 since our inbound traffic is well more than double our outbound, and HTTP inbound can account for virtually all of it. I just don't know how much of it is redundant that a cache would service until I put a cache server in place and try it out and see. But a cache server would definitely cost us way less than the increase we would need to pay over the next year just to add more bandwidth.
Actually, the cost is going down down down. The day of portable 10 digit phone numbers will be here quite soon. The major limiting factor is that not areas can immediately participate. Number portability already exists between local phone companies in most metropolitan areas. The technology has been built; it just needs to be scaled up.
What good is it to have different distributions...
on
An RPM Port Of APT
·
· Score: 2
What good is it to have different distributions if the distributions are all alike? Of course there's one of them I'll pick for my own use. And the reason will be because it doesn't do it the way the other packages do. What I'm trying to say here is please don't try to force all the packages to be like. I don't want a Linux melting pot. I like diversity.
Re:The problem is in the dependency database
on
An RPM Port Of APT
·
· Score: 2
I'm not sure where my first reply here went to. It didn't show up. If it does later, sorry.
My interest is not in the space vs. speed tradeoff. I'm not particularly trying to save database space. Instead, I'm trying to save hassle and time administering the system. Packaging systems promised this, but eventually do not deliver. Your suggestion is also an increase in my time. I would much prefer to see a system audit tool that looks at what is actually installed. You could correct a database with such a tool.
If a package installer checked what was actually installed, instead of what a database claims is installed, then I would be more interested in it.
Re:The problem is in the dependency database
on
An RPM Port Of APT
·
· Score: 2
Based on the book I bought, RPM spec files seem to be overly complex. But then, all I really want to do is say what the package name and version is, and that's it.
The problem is in the dependency database
on
An RPM Port Of APT
·
· Score: 5
What we need is a packaging system that can correctly detect whether or not dependent packages are installed without having to have a database. Inevitably, the database will get out of sync the moment you have to compile something from source because no.deb or.rpm file is available right then, or because you have a local patch to fix a bug you need which isn't important enough for enough other people for the author(s) to fix right now (or maybe is to complicated for them to figure out how to roll it back in without breaking things for other people that you don't happen to need to worry about). Once the database is out of sync, then new problems come up, and those are easily fixed by forcing an install or installing from source, and then it just gets worse.
Without a database, it would mean the installer would have to have a way to detect whether the dependent thing is installed or not, and in the correct version. I won't say that would be easy, but it is what would be needed. Until then, based on my past experiences with Redhat's RPM, I won't at all be interested in a fancy packaging system.
Perhaps RPM is great when you make certain everything you upgrade is done with RPM. The sad fact is that an RPM file usually lags behind by hours or even days, making it necessary to compile source and let the Makefile blinding do the install, throwing the whole RPM database out of whack. And if you have to patch source, as I occaisionally do, then you're screwed.
I used Redhat for 3 versions (5.1, 5.2, and 6.0) and it sucked and didn't show any signs of getting better. Half my problems were with the RPM database being screwed up (saying I didn't have dependencies I really did have, and so on). The other half were with the screwy Redhat rc files, but that's another thread. I tried Debian but never got to see if APT would do things right because the installer in Debian was screwed up (they have announced fixing it, but it's too late for me now). I'm back to Slackware and trying OpenBSD. I'll probably have time to give Debian another look in 2001 sometime.
Back when GIF first came out, no PC had more than 256 colors. So naturally, GIF files were constructed with exactly one image block within the file, even though the GIF87A standard allows for multiple image blocks. This led to a common misunderstanding among the vast majority that GIF itself was limited to 256 colors. By reading the standard you can see that no such limit exists at all because there is no limit on the number of image blocks per file.
To this day Photoshop is incapable of exporting GIFs which use more than one image block to increase the number of colors. To me that says their developers don't read the standards they write code for, but instead just base it on their pre-conceived old notions of the way it works. Ironically, Photoshop will import a true-color GIF. That seems rather inconsistent to me.
Now many of us may not want to even use GIF at all, as PNG is clearly the superior format. At this point, it may not be worth it for Adobe to add on to their GIF capabilities. But it does tell me that they don't have a technical understanding of GIF.
No one has yet shown me why we even need a central "government" to control domains. The domain naming system is nothing more than a commonly implemented, highly distributed, and rather arcane, search engine. And it's not even a very slick search engine.
If we are going to form a representative body to manage it for us, then we have to decide who the "us/we" part is. Are "we" the ones who register names or are "we" the ones who are going to be looking up names. I think it should be the latter, if anything. We are, of course, the ones who decide what goes into our own DNS data files, or DNS lookup list. We decide how we shall see the world.
As I have mentioned before, it is possible for the whole domain naming system to be run with every server having its own root zone. Will that result in confusion? Probably, but mostly only for corporate suits who were (and probably still are) all confused by all this internet stuff, anyway.
I'd rather be below the anarchy line than choosing to leave in place oppression by the wealthy when there is the chance to escape it. That's not to imply that all who are wealthy will oppress. Some do and some don't. But the important thing is, now the tools to oppress are becoming something the masses can choose. While I would not encourage doing things like this, I think it's mere existance helps to reduce the impact of it when done by big corporations and such. Now the courts must acknowledge that the ability to doctor evidence really does exist, instead of dismissing such accusations summarily as they have for decades, just because there is no way to prove such doctoring (because it was good enough to escape detection). Now the fact will be too apparent, and have to be accepted.
Why should we leave the capability to doctor evidence in the hands of highly paid lawyers and big corporations? Now the little guy can doctor his own evidence. I call that a level playing field.
How the fuck do you know that shitface domain won't fucking get rejected later, such as when it goes through to the motherfucking central registry? (language included for the unoffended)
Where can I get... from the French government (not from RIPE or ARIN)... a list of the IP addresses to block... e.g. the ones the French government, or better yet, that judge, thinks are the ones to be blocked.
How would they be able to not allow it? All you have to do is set up your own DNS server (if you don't already have one), change the data starting at the "dot zone", and make it use whatever data you want. You can make.com point to Internic or change it to point somewhere else. You will have that level of control with every top level domain when you run your own "dot zone". And it's your server.
A couple years ago I made this tool to help build a "dot zone". I could bring it back and keep it updated, or someone else can just take the code and do it for themselves (free source code). All that's needed is the willingness to just do it.
You can also do this with DNS servers (for instance an ISP can do it for their customers, and even have mutiple choices of DNS servers set up differently) and choose what info you get per top level domain. A couple years ago I set up a tool to help those making their own "dot zone" to fill it in with their own choices. You can see that tool here.
Slashdot Mirror
Your problem is you (the major ISP you worked for) were not pro-active about preventing spam. Obviously you intended to do nothing but clean up after the mess was made, and so you did.
Dynamically addressed dialup accounts (which is what most spammers have used and still use) should not have direct port 25 access. Those accounts are limited accounts, anyway, and should send all mail only through the designated mail servers run by that ISP.
After having 2 cases of "spam and run" done on dialup accounts at the ISP I used to work for (and continue to consult for) I put in place a block on port 25 outgoing for all dynamic dialups. That was over 2 years ago and there hasn't been an origination spam incident since then (relaying was also blocked separately). I personally use another major ISP for nationwide roaming dialup purposes, and they do this (I tested it). UUnet should start doing this. Although it wouldn't stop SyberSchool, it would stop most of the spamming which comes from their dialups.
If you run your own mail server or an ISP, just block most of the spam via RBL/DUL/RSS from MAPS. If you don't run a mail server, choose an ISP that does these things for you on theirs. That will block most of your bandwidth waste.
Goodbye Anonymous Coward. You posted too much anyway.
If the IP addresses are static (and usually they are on T3 and T1) then smack yourself for not blocking them on the first incident.
If a major ISP wants to allow a spammer to operate, then the way they should do it is to require it be done from a dedicated circuit, and to prohibit relaying through any mail server not listed in the DNS for the destination address. Such an operation is very easy to block on the receiving end. In addition to RBL/DUL/RSS, I also have my own DNS zone to block my own set of IP addresses.
My point of view on this is that if someone actually wants to be a part of this and get spam, they should be allowed to do so. Likewise, someone who does not should not have to. I'm not opposed to an ISP that wants to allow spammers to send bulk email in a legitimate (e.g. identifiable, easy to block) way. Anything less is, IMHO, fraud (and if the ISP knows it's going on, is a party to the fraud, also IMHO).
What I want to know is if Nace and SyberSchool are sending their email direct (doing normal DNS MX lookups and sending to the designated host) or if they are doing relaying through innocent third party mail servers. If it is the latter case, then I think they should be cut off. If the former, then I have no problem with it because I can block them myself very easily (your ISP can, too, if they want to).
We do make a pure IP available. The cost is increased because our cost to offer it is greater. Those costs include making sure we identify who the subscriber is to be sure we are not signing up someone whom we have previously canceled due to abuse. There is also a real cost involved in static IP, which everyone who does want to run an SMTP server usually wants. This is why it is bundled together (almost everyone who is in one group is in the other). Most of the subscribers who do run their own SMTP servers are businesses, but there are some individuals. And most of them have migrated to DSL as well.
I entirely agree with you on the principles. But from the perspective of a business offering where we have to balance between controlling costs (including the impacts that reduce the quality of our service) and offering what the vast majority of customers want, this is what we end up with. Pure IP is not what most of our customers care for.
Basically, both you and I will have to face the fact that what we want as a pure End-to-End IP is a small subset of the telecommunications market demand. Fortunately, the cost of offering it is only marginally higher, so it is practical to offer it as an alternative.
My understanding of transparent redirected proxy was that it worked with the IP address you used. If it uses the separate Host: header to do an IP address lookup, it's a broken proxy server design. Maybe the redirection is broken for not supplying the IP address. I haven't looked into what they actually did design, but I know it could easily be designed to work right even in your situation. My point is the blame goes to the design they are using, not the fact that they are using transparent proxy at all. Now I need to go check Squid and see if it's designed right ... *sigh*
In any event, I would sure like "IP purists" (which I count myself as one) to help me out and find solutions to real business problems like these which preserve pure IP. Please tackle the SPAM issue and HTTP bandwidth redundancy issues as the top priority.
I fully understand that, and might well be hunting around like you (if it weren't for the fact that I get the service for free being a consultant there).
There are tradeoffs. If port 25 is left open, and people can sign up for services and send SPAM, then the quality of our service goes down because we have to spend staff time dealing with the aftermath, and our equipment has to deal with more DoS attacks. Before blocking port 25 we were seeing about 1 spammer a month signing up. After blocking port 25, we couldn't tell if there were any trying to or not (didn't log port 25 attempts).
There were alternatives. For example we could have done a background investigation on people who sign up for service. But that would not be cost effective given the time involved (normally accounts go up within 15 minutes) and the fact that there really isn't any information about who spammers really are. We chose to block port 25 because it was the least costly choice, and would have the least impact on the vast majority of our customers.
As for letting people know, it was our policy to answer truthfully if anyone ever asked; no one ever did. The anti-relaying on our servers has caused a few problems for "road warrior" customers, but blocking port 25 on our dynamic dialup never has raised a tech support issue.
As for the web cache, I do think it would be appropriate to notify customers, and I will bring that up. As for opting out of caching, I think that's going to end up having to be an extra cost item, since doing so means greater use of bandwidth. A third of the cost of business is buying bandwidth.
Incidentally, I plan to set up a 2nd optional cache server (not transparent, must configure as a proxy) which also blocks most banner ads. It will pull from the main cache (less the banner ads). Customers will have the option to use it if they choose to. Is that fair? I'm sure it's not fair to the advertisers :-)
We curently offer what I would call a limited access service. All dialup access is restricted on port 25. Outgoing connections to port 25 from dialups are limited to reaching our servers only. This only applies to basic dialup access. We also have a premium dialup service which includes static IP and is not blocked at port 25. The purpose of all this is to pro-actively prevent SPAM from originating from our network, especially considering that we don't really know who these people are who sign up for basic accounts. We've had people call and sign up and then send SPAM the very next day. In the 2 years that we've been blocking port 25, we've not been the origination point for SPAM (relaying has been blocked way longer).
Now, is this something people would consider to be NOT pure end-to-end IP? Considering that the email content is end-to-end, can this be considered to be a valid exception?
BTW, we also block ports 137-139 and 31337 at the border.
We are also considering HTTP caching on port 80 since our inbound traffic is well more than double our outbound, and HTTP inbound can account for virtually all of it. I just don't know how much of it is redundant that a cache would service until I put a cache server in place and try it out and see. But a cache server would definitely cost us way less than the increase we would need to pay over the next year just to add more bandwidth.
Actually, the cost is going down down down. The day of portable 10 digit phone numbers will be here quite soon. The major limiting factor is that not areas can immediately participate. Number portability already exists between local phone companies in most metropolitan areas. The technology has been built; it just needs to be scaled up.
What good is it to have different distributions if the distributions are all alike? Of course there's one of them I'll pick for my own use. And the reason will be because it doesn't do it the way the other packages do. What I'm trying to say here is please don't try to force all the packages to be like. I don't want a Linux melting pot. I like diversity.
I'm not sure where my first reply here went to. It didn't show up. If it does later, sorry.
My interest is not in the space vs. speed tradeoff. I'm not particularly trying to save database space. Instead, I'm trying to save hassle and time administering the system. Packaging systems promised this, but eventually do not deliver. Your suggestion is also an increase in my time. I would much prefer to see a system audit tool that looks at what is actually installed. You could correct a database with such a tool.
If a package installer checked what was actually installed, instead of what a database claims is installed, then I would be more interested in it.
Based on the book I bought, RPM spec files seem to be overly complex. But then, all I really want to do is say what the package name and version is, and that's it.
What we need is a packaging system that can correctly detect whether or not dependent packages are installed without having to have a database. Inevitably, the database will get out of sync the moment you have to compile something from source because no .deb or .rpm file is available right then, or because you have a local patch to fix a bug you need which isn't important enough for enough other people for the author(s) to fix right now (or maybe is to complicated for them to figure out how to roll it back in without breaking things for other people that you don't happen to need to worry about). Once the database is out of sync, then new problems come up, and those are easily fixed by forcing an install or installing from source, and then it just gets worse.
Without a database, it would mean the installer would have to have a way to detect whether the dependent thing is installed or not, and in the correct version. I won't say that would be easy, but it is what would be needed. Until then, based on my past experiences with Redhat's RPM, I won't at all be interested in a fancy packaging system.
Perhaps RPM is great when you make certain everything you upgrade is done with RPM. The sad fact is that an RPM file usually lags behind by hours or even days, making it necessary to compile source and let the Makefile blinding do the install, throwing the whole RPM database out of whack. And if you have to patch source, as I occaisionally do, then you're screwed.
I used Redhat for 3 versions (5.1, 5.2, and 6.0) and it sucked and didn't show any signs of getting better. Half my problems were with the RPM database being screwed up (saying I didn't have dependencies I really did have, and so on). The other half were with the screwy Redhat rc files, but that's another thread. I tried Debian but never got to see if APT would do things right because the installer in Debian was screwed up (they have announced fixing it, but it's too late for me now). I'm back to Slackware and trying OpenBSD. I'll probably have time to give Debian another look in 2001 sometime.
Back when GIF first came out, no PC had more than 256 colors. So naturally, GIF files were constructed with exactly one image block within the file, even though the GIF87A standard allows for multiple image blocks. This led to a common misunderstanding among the vast majority that GIF itself was limited to 256 colors. By reading the standard you can see that no such limit exists at all because there is no limit on the number of image blocks per file.
To this day Photoshop is incapable of exporting GIFs which use more than one image block to increase the number of colors. To me that says their developers don't read the standards they write code for, but instead just base it on their pre-conceived old notions of the way it works. Ironically, Photoshop will import a true-color GIF. That seems rather inconsistent to me.
Now many of us may not want to even use GIF at all, as PNG is clearly the superior format. At this point, it may not be worth it for Adobe to add on to their GIF capabilities. But it does tell me that they don't have a technical understanding of GIF.
No one has yet shown me why we even need a central "government" to control domains. The domain naming system is nothing more than a commonly implemented, highly distributed, and rather arcane, search engine. And it's not even a very slick search engine.
If we are going to form a representative body to manage it for us, then we have to decide who the "us/we" part is. Are "we" the ones who register names or are "we" the ones who are going to be looking up names. I think it should be the latter, if anything. We are, of course, the ones who decide what goes into our own DNS data files, or DNS lookup list. We decide how we shall see the world.
As I have mentioned before, it is possible for the whole domain naming system to be run with every server having its own root zone. Will that result in confusion? Probably, but mostly only for corporate suits who were (and probably still are) all confused by all this internet stuff, anyway.
I'd rather be below the anarchy line than choosing to leave in place oppression by the wealthy when there is the chance to escape it. That's not to imply that all who are wealthy will oppress. Some do and some don't. But the important thing is, now the tools to oppress are becoming something the masses can choose. While I would not encourage doing things like this, I think it's mere existance helps to reduce the impact of it when done by big corporations and such. Now the courts must acknowledge that the ability to doctor evidence really does exist, instead of dismissing such accusations summarily as they have for decades, just because there is no way to prove such doctoring (because it was good enough to escape detection). Now the fact will be too apparent, and have to be accepted.
Why should we leave the capability to doctor evidence in the hands of highly paid lawyers and big corporations? Now the little guy can doctor his own evidence. I call that a level playing field.
How the fuck do you know that shitface domain won't fucking get rejected later, such as when it goes through to the motherfucking central registry? (language included for the unoffended)
Where can I get ... from the French government (not from RIPE or ARIN) ... a list of the IP addresses to block ... e.g. the ones the French government, or better yet, that judge, thinks are the ones to be blocked.
That's why we need to start doing some suppressing of European governments that are into censorship. France 1st, UK 2nd, then Germany, and on and on.
Or [whatever].health.int.
How would they be able to not allow it? All you have to do is set up your own DNS server (if you don't already have one), change the data starting at the "dot zone", and make it use whatever data you want. You can make .com point to Internic or change it to point somewhere else. You will have that level of control with every top level domain when you run your own "dot zone". And it's your server.
A couple years ago I made this tool to help build a "dot zone". I could bring it back and keep it updated, or someone else can just take the code and do it for themselves (free source code). All that's needed is the willingness to just do it.
You can also do this with DNS servers (for instance an ISP can do it for their customers, and even have mutiple choices of DNS servers set up differently) and choose what info you get per top level domain. A couple years ago I set up a tool to help those making their own "dot zone" to fill it in with their own choices. You can see that tool here.