Slashdot Mirror
Contacting Network Admins Of Large Internet Companies?
lisa asks: "I work as a sysadmin for a national DSL ISP. Unfortunately, we've recently found that @Home.com is not allowing connections to port 25 from some of our primary mail servers: this of course means that our customers can't send mail to theirs. I've called and talked to people in their tech support, and only after several calls have we been able to get them to acknowledge there may be a problem. The trouble is, I can't seem to get in contact with any network admins there. Even the tech support person I spoke with expressed less than hopeful sentiments about being able to get this issue escalated. Has anyone had trouble like this with @Home or other simliar Internet companies?"
"What is the best way to get in touch with a Network Admin or someone who actually can do something about a network issue in cases like these? It would be nice to know that just writing root@home.com would get to their systems department, but I was told all of that mail goes through support first."
You might want to look at this. It's a list of NOC contacts for many major providers.
I don't know how up-to-date it is, though.
--
Today, if something goes down, you have no idea whether anybody knows about it or not. None of the backbone NOC's post trouble tickets to the open Internet anymore. Apparently they don't want anybody to know how lousy their service is. The sad thing is that by keeping these secret, they've caused a thousand-fold escalation in the number of phone calls coming in saying "Hey, did you know your route between Dallas and Atlanta is flapping?". Aside from convincing the rest of us that they have something to hide, of course -- but if you're part of an oligarchy that has collectively decided (illegal cartel?) to screw the customer, there isn't much I can do about you deciding to be a deceitful scumbag (well, I could create a new backbone, but that isn't exactly cost-effective).
-E
Send mail here if you want to reach me.
Also, if you want to be taken seriously, don't mention that you use Linux if you can help it. 5 years ago it would've meant "hardcore programmer on the line escalate to admin", it now means "Windows dork trying to survive with Linux, much hand holding is about to occur. Shield busy admins from harm!"
Don't just drop techno babble. If the technicians don't understand what you said, they'll assume (for their own safety) that you don't know anything. They will not escalate you.
Flat out asking to speak to an admin will probably just make the technician feel insulted and less inclined to help you.
For best results, if possible, work with the technician, try their suggestions (and tell them that they all failed), make him take out a trouble ticket so the whole spiel is recorded and doesn't have to be repeated. In most cases they'll escalate it when all of their suggestions fail.
These are just my observations from the inside. *shrug*
No it doesn't take away the right to send email. It only NOT OFFERS the right to make SMTP connections. There's a difference. By blocking it, they force dialup/DSL/cable users to use the ISP SMTP server as first hop, where they can enforce (not all do, but at least they can) their no-spam policy.
now we need to go OSS in diesel cars
If Pacific Bell allowed the customers to connect port 25 directly, then it would create a massive headache, and high costs, for them to deal with the spam (and it would happen for certain, and probably has happened a lot in the past to get them to do this).
When you sign up for service, you are told what SMTP server to use for outgoing mail. Use it. Or find whatever other way works for you. But they are not offering SMTP connection services to you. The solutions are easy, so deal with it.
now we need to go OSS in diesel cars
Have you made certain that your network never has sent out spam (and I mean EVERY machine on your network) and they your domain is not in one of the domain based anti-spam zones? You say you are running Exchange. Since Exchange has installed with relaying on by default (at least when I last checked it about 6 months ago) you may have been a spam conduit in the past (if not still one now). Test every mail server by getting on the machine and running telnet to mail-abuse.org (standard telnet port 23) and having it check to make sure you are not an open relay.
now we need to go OSS in diesel cars
reply... "Midnight will be when NTP says it is."
now we need to go OSS in diesel cars
I have a Pacific Bell DSL line, running my own mail server with my own domain name (actually a subdomain of stanford.edu). The problem isn't on Pac Bell's side; my parents use Earthlink, and my email to them was bouncing. Some investigation showed that they had configured their mail servers to reject any mail traffic from Pac Bell IP addresses other than the Pac Bell mail servers. This was an explicit decision on their part, again with the motivation of "reducing spam."
Fortunately, I was able to relay my SMTP traffic through Stanford's mail server (since I'm using a valid *.stanford.edu address) for each set of mail destinations that does this access control.
I think it's pretty stupid to assume that a DSL line is going to be using the ISP's email services as well--- especially since Earthlink has no problem _delivering_ mail to that account.
OSPF is explained in RFC1131, later replaced by OSPF V2 in RFC1247.
Explaining the internals of OSPF is beyond the scope of this forum.
You can, however, RTFM RFC1131 and RFC1247. These are in Postscript.
BGP is described in RFC1771
Now, listen up, go get a life!
At the time I was working for a web site, basicly, and the problem we were having is that @home customers in san francisco couldn't get to the site. After talking to a few of these customers, I had a couple do a traceroute to our server, and somewhere in the middle of @home's network a split horizon (i think that's what they are called) happened. It was where the packet just kept getting bounced between 2 of their routers back and forth until one of them finally dropped it. This only happened to traffic destined for our little network. I called @Home and was escalated to the top tech, who finally believed me. Then I was called back by a sysadmin there who required a lot of convincing. So he finally acknowledged the problem and said that they would get to it. Before I left that company I don't think it had been fixed, but it might have by now. It actually seems like it was a problem with their RIP or IGRP config, so maybe when a router was rebooted it would fix it's tables. Who knows. But the short of it is that I got ahold of a sysadmin and nothing was done. So good luck when you get that far. The journey may still not be over.
-Nicodemus
Okay, first off, you're doing it wrong.
You need to call their NOC, *NOT* tech support. Get their NOC number, which is according to my records, 650-556-5599. If that's not the NOC, you can get the NOC number from them.
Once you get to the NOC, make them create a trouble ticket, and get ready to use your "I'm NOT HAPPY WITH YOU" techniques. The ONLY way anything will be done about it is if you ride them. Hard. They probably have the TT from Tech Support, so have that number ready, and give it to them. Start riding them hard. Demand supervisors, etcetera. Remember, the NOC is going to be setup with a front line defense (NOC techs), second line defense (NOC NetEng, NOC Unix Admin, etc), third line defense (NetEng, Unix Admins), and finally supervisors. That's NOT how it's managed, but how it's going to progress. Escalate often. Just keep calling them.
That's the only way I've ever gotten anything done with Crack-Home or any other moronic overly large ISP. If they're big enough to have a NOC, then rest assured you'll only get things done if it gets to the NOC. The NOC will likely scream at Tech Support if they get TT's from them (I know we did when I worked in one) and generally have a fit, and ignore the ticket as much as possible. NOC and Tech Support typically do not get along.
Hope this helps, and good luck.
your company here.
your company here.
shelby != ford
Currently working for a major provider (not AOL, I wouldn't sell out) in tech support, I have to say I get a lot of callers who say they've got an MCSE and a CS degree and they've been in the field for 20 years calling me that make most AOL users look like Linus Torvalds.
Best way to handle tech support is to tell them what you think the problem is, let them run through thier checklist so they can properly document the trouble ticket as required, and if you're cool and cooperative about it and try and be on the same level as the support geek, they'll escalate it for you.
To be honest, I love it when someone outside the Winmac realm calls, because they're almost always the easiest calls for me, ie they're the first to notice a widespread network outage and are very cooperative in giving me details I need to document while I run my tests and document that, it gets escalated to NOC and the problem's fixed by the end of the day, or they just had to reinstall thier OS and lost thier TCP/IP settings and all I have to do is read them off the user ticket...
One problem that slows down reporting of network issues a lot are technically illiterate people who get mad because they can't check thier email right now and won't cooperate with us making sure they're set up right or variations on that theme.
--
Help us build a better map!
...because it's a lot easier that trying to get anything useful out of @home. I've been an @home user for three years now and I can honestly say that if it weren't for the fact that I can't get high speed internet access from a competitor, I would be switching.
I think part of your problem might be that the tech support staff are also kept in the dark. They can't help you if they aren't informed themselves. Maybe there's an @home techie out there who can answer this?
I do tech support for another major ISP in this area and I am proud to say that our users don't suffer from this same problem.
I feel your pain. I too have had huge problems with ISPs either not believing me or not listening to reason. I spent 7 hours (at least) on the phone with Earthlink (7 hours is much less than it takes to get another DSL provider) trying to fix a problem with their PPP servers. I was doing protocol analysis so I was *certain* what the problem was. The bad thing is that it was *very* technically complicated and not on one of their check sheets for their techs.
:)
The point is that there is nothing we can do about this. I am sure there are a lot of *really* smart people here. The problem is that tech support people have to deal with a lot of Microsoft Morons so they just assume we are in the same category.
What is really needed is a way for a geek to say "I have mad Kung Fu and have a Black Belt in Network Engineering" and they would say... "oh... excuse me... I will connect you to our third tier tech support right now". Of course that is not realistic. But what they could do is keep track of people with mad Kung Fu so they can go right through the line.
If an ISP would do this it would SERIOUSLY increase their business. All the geeks would subscribe to their services because they don't want to deal with other ISPs. It would also increase their reliability because they would have *really* smart people fixing their network problems for free! Open Source ISP!
Somehow that is logical so I assume it will never happen. God forbid any Western country undertand Zen philosophy!
I work as a sysadmin in the NOC of a large very well known regional ISP. The only people who have the number to our NOC are the people above us and the managers of the front line support. Customers and the general public are not allowed to call us and I think it's a good thing. I saw a reply above that linked to a list of NOC phone numbers and checked it to make sure that my company wasn't on the list. Face it, if you were bombarded 24/7 by geeks who felt that their problem deserved your full undevided attention you wouldn't get much real work done (face it, there are people who feel that if their insignificant problem doesn't get fixed this second the world will collapse and anarchy will reign).
For the problem that was listed at the top of this thread, I would suggest contacting the abuse@ email address for the domain in question. I do know that our company's email admins do get those messages as well as a few higher-ups. Other than that, route your mail thru your isp's SMTP server and save all of us headaches.
How do they manage to keep track of the IPs used by major ISP's mail servers? Lucky guesses? Laborous investigation?
Step 1. Reverse DNS lookup on domain name.
Step 2. Check domain name entry for IP address
for MX value
Step 3. Compare IP address and MX address. If they are not equal, bounce mail.
We're all geeks over here>>> The Linux Pimp
--It's Pimptastic!--
There was a lot of hassle involved. Netcom, then owned by Mindspring, clearly had a massive wall between their system administrators and their support people, with no direct way of contacting the sysadmins. I'd email the support address, and get an email back from someone clearly too clueless to know what an SMTP server or MX address is, insisting the problem must be my end or with my ISP. In the end I basically had to persist, phoning their 1-800 number in the end, getting names of support staff involved, and following up every inch.
I found it tough. The more you point at RFCs and stuff, the more you look like, well, the sort of people you get on TV claiming you don't need a drivers licence because the states aren't constitutionally allowed to forbid you from using the roads or that banks are allowed to create money because of some legal loophole. The person you're talking to has no idea what an RFC is, or an MX record, or anything like that. All they can do is accept that you've tried it all different ways and can't send email.
In the end they put a ticket in with their system administrators, who knew exactly what the problem was and fixed it.
From what I can figure out, the problem was because my ISP's IP address block is smack in the middle of BellSouth's (BS providing the connectivity), and Mindspring had configured the Netcom servers only to accept email sent directly from BellSouth's email servers, not from BellSouth customer IP addresses - my bellsouth.net account continues to this day to have the same problems but I'm buggered if I'm going through the hassle again. This is stupid anyway, but of course as the complaints were coming from people who deliver their own email, or from people with ISPs in similar positions, of which there are probably relatively few, few enough for it to look like most email is being delivered perfectly and therefore it "obviously" being a problem on the deliverer's end.
Why they did this is anyone's guess. I think, given the problems I have being let onto any IRC servers these days, that a lot of the hacking being done at the moment is being done from Bellsouth.net addresses, but I haven't read anything anywhere to back that up. Mind you, the problems emailing ix.netcom.com started a year ago, whereas EFNet's clamp down is at most 4-5 months old.
My advice? To be honest, just keep trying, and keep piling on the pressure until they relent. Send email to the support addresses. If you don't get a response, start calling - preferably calling the @Home customer's 1-800 support line. Keep calling, get names of support people, and don't stop until the situation is resolved.
If Mindspring hadn't finally relented and put in a ticket to their system administrators, I'd probably have used Usenet or something similar to start embaressing them, a little log of an nslookup, telnet to an SMTP port, and then this posted on an appropriate newsgroup. But as it was, it got fixed.
--
You are not alone. This is not normal. None of this is normal.
Following Lisa's posting about "@Home blocking PORT 25" I would like to add that AOL does the same thing.
Our traveling reps use AOL to connect to the net when they are away from the company LAN. We were having issues with email constantly getting rejected. A quick telnet diag revealed that AOL has a Proxy on port 25 that grabs users requests.
Numerous calls the AOL Joke Support didn't get us anywhere. Just explaining the issue is grueling because the Script Reading techs just spout off canned answers like "We only support AOL mail and not Outlook, etc". I tell them "If AOL is suppose to be a true ISP they shouldn't hinder people trying to use Port 25." When you try to ask for a Senior Tech they keep on with their canned drivel.
We've had to setup another mailserver at the company that listens on Port 2525 in order for our users to send mail. AOL won't acknowledge the issue at all.
Every other port works fine, just Port 25 they capture for some reason.