Web Traffic Snarls Sites on Black Friday

eweekhickins writes "A surge of e-commerce traffic on Thanksgiving night and all day Friday apparently caught several retail giants by surprise, with Lowe's, Macys and Victoria's Secret especially hard hit. In fact, almost a third of leading retailers suffered significant slowdowns on Black Friday, according to statistics released this weekend by Keynote Competitive Research, a firm that tracks Web site performance."

Comments

Why...

[kungfujesus]

Why don't these huge stores buy servers that can take the strain? sure, they may be ridiculously overpowered for most of the year, but being able to function on black friday is extremely important for their bottom line.

Re:Why...

[BosstonesOwn]

It's retail they don't spend before it happens , they try and save a buck when they can.

My experience at a retail business was exactly that. We had to have weeks of slow networks and servers in order to get the ok to get vendors in to bid on selling us gear. It was a huge joke.

They should host with companies like akimai who can provide bandwidth on demand.

Re:Why...

[mcrbids]

Seriously, predicting traffic is pretty much a black art. Even if you build out for what you thought would be enough, you still could get caught flatfooted.

Hear Hear! This man speaks wisdom!

A year ago, I purchased a number of 1u 4-way servers in anticipation of rising demand. Based on rough guess of processing speed and current workload, I made an estimate of how long these servers would handle the load.

Now, a year has gone by, and the load has only risen slightly, despite a dramatic increase in traffic! Bandwidth has risen sharply, yet the server load still floats at around 3-5% all day long, while based on my past estimates, would should be routinely hitting 25% and spiking to 200% from time to time.

It's rare that it ever hits 20%. But disk usage is out through the roof - now at about 3x initial guess. Our customers are USING THE CRAP out of our services, but apparently refinements in the software over the past year (caching, etc) have all but completely negated any performance hit from the increased load.

Re:Why...

[Anonymous Coward]

Posting anonymously since I'm too close to this...

I've worked with all of the companies mentioned at one time or another. Some do better planning than others. Quite simply, Black Friday represents a worst-case-scenario for an ecommerce site. Either you build out enough extra capacity to handle it (and we're not talking a couple servers here - we'd be talking more like hundreds of servers, not to mention massive database backends) and pay for it (both hardware, management, bandwidth, storage, etc), or you don't build out any extra and get slammed. Tens of millions of dollars of equipment and management, all for one day. Or what most companies do - you build out enough to handle the brunt of it, make as much profit as you can, and some peaks you just don't handle because it's not worth the massive investment to handle 100% of the traffic. It's a cost-benefit analysis, plain and simple. I can't comment in too much detail, but some of the companies listed did exactly this, and some... well, let's just say they didn't invest nearly enough. That's their choice.

(Please recall that a OnDemand type of initiatives don't handle this - the idea behind them is there is extra capacity that you "switch on" on a moment's notice to handle a spike. The problem is that there's no capacity when everyone is hit at the same time. OnDemand is great if your peak is at a time when someone else's isn't - they get extra capacity at that time (for instance, flowers and greeting card companies on holidays, retailers on Black Friday). Here, everyone needs capacity at the exact same time. It's simply brutal.)

It will be an interesting week as we get more data on Black Friday and everyone filters back in from Thanksgiving (mind you, many of us were working all this weekend, and some serious overtime monitoring and improving the situation however we could).

Re:Why...

[SeaFox]

Why don't these huge stores buy servers that can take the strain?

I used to work for the Dept of Education (by way of a subcontractor) and the site I supported (the one your submit you government college financial aid form through) had the same problem. Several times a year it would slow to a crawl or not really be usable at all. The days this happened were the days that states required forms be filed by to be eligible. Despite the fact there are 365 days in a year, it seemed there were only a dozen or so days that states would choose as their deadline days, so we'd always have at least a few states all due on the same day.

Anyway, the question you just asked used to come up a lot. "Why don't they just buy more servers/capacity?" The answer is because the difference in traffic is so huge between these days and "normal" days it would simply cost too much to maintain that sort of capacity. And (in our case at least) the security required for the app and data keeps a temporary bandwidth solution from being viable solution.

Note: in the case of the FAFSA, it was ultimately the users themselves who were at fault for the slowdown, it not like an 18-hour shopping sale where you have to be there in a small time window, the form was available year round, it just happens that most people wait till the [i]last possible day[/i] to do it, then complain about the traffic jam (I'm looking at you, California!).

blame

[shawn443]

Since the article makes no mention. I will not blame perl, apache, or linux. I will blame .net, IIS, and of course PHB's.

The term "Black Friday"

[popo]

Is it just me, or is the term "Black Friday" being used much more this year than in previous years? Maybe I'm the only clueless one, but I was seeing it so much I Wiki'd it for a little explanation: the root of the term (and if this is well known to all, my apologies... I'm slow that way) is that the balance sheets of retailers are typically "in the black" by the Friday following Thanksgiving.

I can't help thinking it sounds more like a stock market crash than a "good thing".

Re:The term "Black Friday"

[forkazoo]

Is it just me, or is the term "Black Friday" being used much more this year than in previous years? Maybe I'm the only clueless one, but I was seeing it so much I Wiki'd it for a little explanation: the root of the term (and if this is well known to all, my apologies... I'm slow that way) is that the balance sheets of retailers are typically "in the black" by the Friday following Thanksgiving.

Apparently, Black Friday is extremely well known, even internationally. I passingly know a fellow who grew up in Germany and moved to the US this year, and was very excited about his first chance to see Black Friday shopping in person after having heard so much about it. Seeing Americans in a consumerism frenzy must be a bit like watching sharks in a feeding frenzy, I guess.

Re:The term "Black Friday"

[UserChrisCanter4]

Please read the wiki again; the root of the term is the high stress caused to transportation workers. The first citation anyone can find refers to the traffic snarls and the associated headaches for traffic police, cab/bus drivers, etc.

As the wiki points out (and common sense will tell you), bleeding money for 11 months of the year and hoping to recoup it in the last one is one of the most asinine business plans since the "???->profit" joke. Similarly, the wiki points out that quarterly SEC filings from any decent retailer will show you that they do make a profit in the other quarters, as well.

Unless you're a Christmas decoration specialty retailer or something similar, waiting until the fourth Friday in November to turn a profit would be a recipe for failure.

You're not the only one

[Moraelin]

If it makes you feel any better, you're not the only one. Any other application of "black" to a day seems to have meanings a lot more... well, dark.

E.g., "Black Tuesday" is when the Great Depression hit.

Heck, even "Black Friday", other than that particular meaning, was applied to massacres, riots, major financial scandals, you get the idea.

So I can't help wonder what kind of idiot chose "Black Friday" to mean "we're selling lots of stuff". I mean, gee, it must be such a dark and depressing thing.

More importantly, it's the kind of language that obscures instead of informing. For someone who doesn't know that particular pun already, it evokes the exact opposite image. I'll confess that I too, when reading that summary, was left thinking, basically, that it was some great catastrophe that befell them.

On second thought, though, heh, it sounds like what marketers and management tend to do to sound smart... when they aren't. Now I'm not saying that all of them are clueless, far from it. Just that you can often tell the ones who _are_, by the inclination to speak gobbledygook and think that having a buzzword for everything makes them so great.

Why not offer a simpler version

[cerberusss]

When the peak hits, why don't e-commerce sites switch to a simpler interface? The gazillion queries that these sites do for one page can be completely switched off. For instance, I'd rather be able to put a book in my shopping cart WITHOUT stuff like:
- "people who bought this article, also bought"
- Full text search
- Customer reviews
- Editorial review
- Offers "Buy together with hacksaw, 15% off"

And the gazillion datamining queries done by the website.