Slashdot Mirror

There already is a Web page called "Internet Health Report" at http://internetpulse.keynote.c.... It has been reporting the status of the U.S. backbone providers since possibly 1993 (23+ years). At least, that was when the domain keynote.com was first registered.

It reports latency in msec, percent availability, and percent of packets lost. The page is copyrighted. The terms of service indicate there might be a trademark on the name "Internet Health Report".

Sort of, but look at the website of any startup. They all try to list their large corporate customers. Being able to put Campbell's on your website is surely worth something.

They used to call the early Internet, the . The theory being, you didn't have to worry about implementation details. You just had the X.25 network socket connected to the world somewhere in the data center, and your network router somewhere else. In between was one brightly colored cable connecting the two. That's all you needed to know. At least in theory.

That guy will probably have X.25 cloud in his keyword list.

As the Federal CIO sang the praises of Amazon.com-backed Animoto's use of the Amazon Cloud, the Chairman of the Recovery Board decided giving Amazon the contract to host Recovery.gov was the right thing to do, and called on the public to 'imagine if other, much larger federal agencies were to follow our lead.'

Credit for deciding to tap Amazon was given to government contractor Smartronix, who reportedly used AWS in the development and testing of recovery.gov, but did not go live with it in the initial roll-out.

The government planned to find another home for the more than $1 million in computer hardware and software that were previously purchased to host the (apparently) relatively low-traffic Recovery.gov site, but were no longer needed after hosting was switched to Amazon.

Heh, I was wondering why scoreboard showed they were having issues:
http://scoreboard.keynote.com/scoreboard/Main.aspx

*sigh*

So it wasn't just an outage.

you moron, go to the site and see for yourself.
http://scoreboard.keynote.com/scoreboard/Main.aspx ?Period=RH24

right now cogent is having a problem.
http://scoreboard.keynote.com/scoreboard/Main.aspx ?Period=RH1

you moron, go to the site and see for yourself.
http://scoreboard.keynote.com/scoreboard/Main.aspx ?Period=RH24

right now cogent is having a problem.
http://scoreboard.keynote.com/scoreboard/Main.aspx ?Period=RH1

happened in Detroit in the last 24 hours. Apparently all ingoing/outgoing traffic to other Tier One ISPs had problems in that city. Also, Philadelphia had really slow traffic within Level3 (and slower to all the others), and had major problems connecting to Verio. San Diego also had some problems, especially within the Level3 network. St. Louis was the only area without major problems...

For a breakdown, check out this view of the data.

Chicken little just wrote another news article I think.

http://scoreboard.keynote.com/scoreboard/Main.aspx ?xAxis=Destination&yAxis=Origin&zAxis=Metric&nAxis =Period

There is no outage anywhere.

Try the report for the last 24 hours, not the big yellow blocks.

http://scoreboard.keynote.com/scoreboard/Main.aspx ?Period=RH24

Internetwork outtage?

In other news, Internet Health Report looks ok to me at the moment & I didn't see "intermittent" anywhere in the article.

Did somebody say the sky was falling? o_O

The Internet Health Report cited in TFA shows all green now. It looks like whatever problem they had is solved.

See http://scoreboard.keynote.com/scoreboard/Main.aspx ?xAxis=Destination&yAxis=Origin&zAxis=Metric&nAxis =Period/

Take a look at the scoreboard now. The mentioned problems are gone and Level 3 is no longer in the red.

The internet traffic reports indicate Level3 is having severe routing problems. As at 0300 10/21 PDT.

From whirlpool.com.au
On October 21st we began receiving numerous calls from multiple location of routing issues. Upon further investigation we discovered that Level3 appears to be a common factor. We have discovered that Level3 is having major network issues nationwide. We are in the process of shutting down our Level3 peering at all our locations. We have opened ticket 184739 to track this issue. Currently Level3 does not have an ETR.

Ah, keynote also has some nice data. Apparently Level3 & Verio have both been having severe problems in the last few hours, and have a high latency connecting to most of the other Level-2 & Level-3 providers:

http://scoreboard.keynote.com/scoreboard/Main.aspx ?Login=Y&Username=public&Password=public

this will help you confirm your information
just adjust the settings to 24 hours and you'll see
http://scoreboard.keynote.com/scoreboard/Main.aspx ?Metric=AvailNtwk

Looks like Level3 and COGENT may be "working" again.......

Keynote has them back online !!!!

http://scoreboard.keynote.com/scoreboard/Main.aspx ?Login=Y&Username=public&Password=public

Peter

Sure, they have the connections, but routing extra traffic through those peering links will probably only cascade this problem. The intermediate providers will see a jump in trafic coming through from L3 and Cogent, and they will have to consider how to recoup the costs that that is imposing on them.

It's a web, and when one strand breaks, it increases the strain on the other strands.

At the fringes there are really two types of internet service offered: upstream and downstream. Most consumers (individuals) need a lot of downstream and very little upstream. They typically are sold assymetric service that is heavily biased in this direction. My cable connection, for example, gives me ~5Mbps down and 768kbps up. On the flip side are the content providers who typically need a lot of upstream bandwidth and less upstream bandwidth. ISPs have found that these customer are willing/able to pay quite a bit more for their internet connections. Therefore, the law of supply and demand has increased the cost of connections with higher upstream capacity.

Several levels up the ISP heirarchy, however, there are mostly only symmetric lines (T3, OCx, ...) providing equal upstream and downstream bandwith. In order to maximize the use of this bandwidth, many providers try to balance the number of content providers with content consumers in order to use the upstream and downstream capacity equally. In theory, this usage should be well balanced by the time it reaches the Teir 1 providers.

The problem we are having right now is caused by Cogent not subscribing to that business model. They have found that the cost to support content consumers is much higher than the cost to support providers. (If for no other reason than there are far more of them.) So, their business model skews heavily towards the provider customers, reducing their operational costs. This, in turn, means that they are able to offer lower costs to those content providers -- in many cases undercutting the other big service providers such as Level 3

This, of course, makes the other providers unhappy because it cuts into their high-yield business. So, occasionally, one of them demands compensation for "transit" instead of providing free peering. They do this because they feel (rightly IMO) that Cogent is able to make more money on these high paying content providers by using an asset owned by the other service providers -- the online customer/consumer base. Basically, Level 3 is telling Cogent that because Cogent is making money by using that virtual asset owned by Level 3, Cogent owes Level 3 some sort of compensation. It is worth noting that several other Teir 1 providers already take this approach with Cogent and Cogent is forced to pay for "transit" service to those providers' customers.

As long as all the Teir 1 providers cooperate, the system works reasonably well. However, in this case, Cogent is trying to take advantage of that informal cooperation to make some extra money. So, they are being capatalists. In this case, capatalism is at odds with cooperation and the system is not working well.

Many people are calling for government regulation to prevent this sort of situation. I expect this to cause some major problems. The issue could be resolved if all the Teir 1 providers would realize that there is a different market value for ingress and outgress traffic. In a free market, I expect that the ingress traffic (corresponding to upstream traffic of content providers from the lower levels) would have substantially more value than the outgress traffic (downstream traffic to consumers). The outgress traffic might even have negative value (meaning that a service provider would charge to take care of it). In the case that two peers balance their traffic well (the ideal cooperative solution) no money needs to change hands. In the other cases (like this one) the ISP with excess outgress usage should probably be charging the one with excess ingress.

Unfortunately, there is no fluidity to the system between the true market (the upstream and downstream bandwidth consumers) and the core market (the Teir 1 providers). If there were, Level 3 could justify their demand for more money based on the value of the traffic they were accepting from lower down the food chain.

The Internet Health Report shows the interruption, and I noticed there is also the "Pending Assingment" as mentioned in these articles. I'm curious as to who this is.

In case you want to monitor the situation...

http://scoreboard.keynote.com/scoreboard/Main.aspx ?Login=Y&Username=public&Password=public

Internet Health Report

My question, which I am attempting to form into a coherent argument still, is generally this:

Will the new network actually be a distributed network, or will it be a massive, bottlenecked POS like we have now?

I refer, of course, to the 12 major DNS servers which control our access: Internet health report. One of these goes down and those of us still up see a super-slow internet. Two go down and pages fail to load as often as not. I've yet to see three go down completely, but it is bound to happen.

The above are stateside only, to my knowledge. And yes, I know that we *CAN* use raw IP addresses to bypass the DNS, however the world wide web DIES when a few DNS servers go down. I guess that means the above is a complaint about the WWW, not TCP/IP in general.

Bah. Now I realize that I don't really know enough about the guts of the current TCP/IP stack to be able to defend any arguement I might make. I suppose I need to pick up that tome I have lying around here somewhere....

Cheers,

First, although 300 locations with a few users each may sound like a high-volume site, it is not. I don't want to burst any bubbles, but it simple is not high-traffic in today's world. I work with large e-tailing sites that get 200,000 unique visitors per hour.

The first step is to determine the type of load you will receive. Is it call-center type traffic, where they will have dedicated staff accessing the application, or will it be more like Internet traffic that comes in waves when it feels like it? If your application fits the call-center model, then you need to know the maximum number of operator-types that will be online at any given time. If it is more like an Internet site, such as Slashdot, then you need to either project the number of sessions per hour, also called the arrival rate, or examine the web logs to find out.

Concurrent users and arrival rates are not the same--one is the output of the other. In arrival rate mode, the number of concurrent users vary depending on the number of visitors arriving that minute, and the speed of the site. If the site slows down, which is will at a higher rate of visitors, then the sessions will take longer. If the sessions take longer, then visitors continue to come to the site and the number of concurrent users rise. Internet visitors do not know how many users are on the site and certainly won't obey any threshold that you determine.

The second step is to test over the Internet, and from as may remote locations as possible. You said that there were to be 300 remote offices. Are these all in the US, or are any of them International? Testing on a local LAN does not tell you much of anything, because there is no latency and everything runs at the speed of your switch. Very few people have 100 megabit connections to the Internet, so it is not realistic to test that way. Real users have a mix of line speeds, and come from a variety of locations. It is best to test from 5 or more geographically disperse locations, using a distribution of the line speeds that your end users will be using. If each of these 300 site has a T1, and each site has an average of 3 users, then each user should run at 512Kbps, not 1.54Mbps.

Lastly, perform realistic transactions on the site, don't just simply hit the home page. Real users on the site will probably start at the home page and traverse the site, doing various things. You should have an idea of what these actions will be, or you can examine the web logs to determine the top 10 paths through the site. Then write scripts for each path and run them proportionately. You also need to build in think or dwell times into each page. Real users don't go from page to page as fast as possible! They take time to fill out forms. A good load test takes into account how familiar a person is with the site and what the person's patience with the site will be. A person using an SSL connection purchasing something has more patience that someone browsing a catalog. By the same token, an operator-type person does not have any choice about whether they can use the site or not, however their productivity will be directly proportional to the speed of the site.

There are very few open source or free tools that do these things for you. Your options are to 1) wing it as best you can using the SWAG method you described, or 2) seek help. There are various Do-It-Yourself outsourced solutions, such as Test Perspective or some other total outsourced solution. The DIY method will probably get you the best value, but you are subject to your own work, and don't have anyone to blame if things go wrong.

Aren't you just lying?

The truth:

Candidate: George W. Bush
Internet Address: www.georgewbush.com
Web-Site Software: Microsoft-IIS/4.0
Internet Access or Hosting: Exodus Communications in Austin, Texas

Candidate: Al Gore
Internet Address: www.algore2000.com
Web-Site Software: Apache/1.3.9 on Linux
Internet Access or Hosting: Exodus Communications in Sterling, Virginia

More thematically consistent 2000 candidate website stats:

Candidate: Gary Bauer
Internet Address: www.bauer2k.com
Web-Site Software: Microsoft-IIS/4.0
Internet Access or Hosting: Atlantech in Silver Spring, Maryland

Candidate: Bill Bradley
Internet Address: www.billbradley.com
Web-Site Software: Apache/1.3.9 on Sun Solaris
Internet Access or Hosting: Shore.Net in Lynn, Massachusetts

Candidate: John McCain
Internet Address: www.mccain2000.com
Web-Site Software: Netscape-Enterprise/4.0 on Sun Solaris
Internet Access or Hosting: US West in Minneapolis, Minnesota

Candidate: Alan Keyes
Internet Address: www.keyes2000.org
Web-Site Software: Apache/1.3.9 on Sun Solaris
Internet Access or Hosting: SimpleNet in San Diego, California

Candidate: Steve Forbes
Internet Address: www.forbes2000.com
Web-Site Software: Netscape-Enterprise/3.5.1 on Sun Solaris
Internet Access or Hosting: USinternetworking in Annapolis, Maryland

So you're lying to the OSS website to make Gore look bad, and Bush look good. You probably even believe your own lies. What a perfect Bush partisan.

Here's an interesting(?) review of the sites of the Presidential candidates' websites.

Here's another review and commentary about the websites, including a count of the number of errors in the HTML.

Netcraft says that Bush actually was running Apache for a while before the election, but switched to IIS by October (at the latest) and has been stuck there ever since.

That's called the Internet. Why mockup/simulate what already exists.

Now if the concept was slashdot-a-country-per-day that'd be cool. Monitor the effects of page availability of other sites in the country that were not specifically slashdoted, etc. Similar to what Keynote does.

Could we bump a country off the internet just for giggles? Hmmm.

I used to work for Keynote Systems and the hot potatoe routing was of major concern to customers. Many times you see inbound and outbound routes are significantly different due to the "least cost" associated with dollars in a peering relationship. You can see basic peering times from one of their public service sites, Internet Health Report. You can also do a quick one-off measurement from their demo site My Keynote. Check out the diagnostics tab and run some traceroutes from different backbones and you will immediatly see how congestion at major peering points can effect performance.

I used to work for Keynote Systems and the hot potatoe routing was of major concern to customers. Many times you see inbound and outbound routes are significantly different due to the "least cost" associated with dollars in a peering relationship. You can see basic peering times from one of their public service sites, Internet Health Report. You can also do a quick one-off measurement from their demo site My Keynote. Check out the diagnostics tab and run some traceroutes from different backbones and you will immediatly see how congestion at major peering points can effect performance.

Keynote have oublished a reporton the performance of major web sites on September 11th, 2001.

Of course, there's a lot of dark fibre around, so the capacity is there if it's really needed. Once the current recession is over, we can expect to go back to the days of massive overprovision and redundancy as content and bandwidth providers seek to build in capacity to handle peaks. What will really help is multicasting for video streams, and well-designed caches at ISPs.

Arg..I screwed up the link.

LoadPro can be found at http://www.keynote.com/solutions/html/keyreadiness _works.html

DFossmeister

Keynote offers a couple of solutions that will push your site to any limit you want. Their Test Perspective load test tool is a self-service online tool that generates the load for you, and their KeyReadiness Load Testing service is a completely outsourced option.

Their options are services only, so the tools are proprietary. The KeyReadiness tool is really cool though because it runs on a large Linux farm.

Disclaimer: I am affliated with Keynote, but that doesn't mean that the service doesn't kick ass...

Donald E. Foss

So I guess there are even non-political, ethical justifications for DoS attacks.

Moreso, isn't DoS precisely what companies like Mercury Interactive and Keynote do when they try to slam your webserver so you know whether you need to buy more server processing power, etc.?

::Colz Grigor
--

Keynote Systems has a service which should catch something like this. Doesn't look too pricey either...

Keynote Systems has a service which should catch something like this. Doesn't look too pricey either...

I just have a quick question as to the actual project. In what way does this differ from Keynote's service. I am a contract engineer and my company works very close to Keynote and we get full coverage throughout the globe of the companies websites as well as our competitors. Seems like the same thing, and Keynote is quite affordable for it's services. Is this a case of reinventing the wheel or something new?

• Later this week, Nokia will be introducing a new mobile phone that not only has the now-standard built-in Tetris game and GPS receiver, but also wireless voice-over-IP support and a built-in webcam for videoconferencing! (A friend of mine works on this project, which is how I know about it.)
• Also this week (I know, funny coincidence, but everyone's trying to get their new products out in time for the Xmas shopping season), Oster will be introducing new microwave ovens, toasters, and blenders with built-in Ethernet interfaces and firmware support for the new Linux Open-Source Kitchen Automation project (I forget the URL, but it's hosted at SourceForge). These devices will allow you to program and monitor your kitchen devices over your home LAN. They also have built-in webcams so you can watch your kitchen in action, from a much more intimate viewpoint than has ever before been possible (in the case of the blender, the cam actually views up from beneath the whirling blades).
• And speaking of intimate viewpoints, Symbian will shortly be introducing a new vibrator with a webcam built into the tip. Configure it with an URL to upload images to, and watch it go in and out, in and out...
• Lastly, Web monitoring company Keynote will be entering the consumer hardware business with a new PCI add-in board for better DIY monitoring of your own website. Just plug it in and the onboard webcam shows you the packets going in and out through your network interface.