NYT Paywall Cost $40 Million: How?

An anonymous reader submits this musing from Philip Greenspun's blog: "Aside from wondering who will pay more than the cost of a Wall Street Journal subscription in order to subscribe to the New York Times, my biggest question right now is how the NY Times spent a reported $40-50 million writing the code (Bloomberg; other sources are consistent). Google was financed with $25 million. The New York Times already had a credit card processing system for selling home delivery. It already had a database management system for keeping track of Web site registrants. What did they spend the $40-50 million on?" Maybe the folks behind CityTime were free on weekends.

Comments

Large organization doing something simple

[Anrego]

I can actually see how this happens. Large organizations spending millions and taking years to do something a small team could whip up (and probably do a better job of) in a few months.

Different team sizes are required for different tasks. Some companies get this and put small teams together and have flexible processes that can scale to project size. Other companies can only do things one way, and that’s where you end up with insanity such as this.

You end up with layers and layers of process controlling huge unwieldy teams. You spend months just drafting the process by which you’ll operate under, and then it needs to be reviewed and this is before development even begins! You end up with 5 layers of management, each providing no real value to anything... but adding lots of time and cost.

You’ll need to gather metrics of course, so you need to figure out what metrics you need, and how you will analyse them, and how they will feed back into the dev process. And of course you’ll need someone to actually facilitate all this with some kind of metric crunching tool (which has to be bought and admined as well).

Re:Large organization doing something simple

[headhot]

I'm a consultant in telecom. I see this every day. I'm convinced that any project, no matter how big can be done by 6 people.

Re:Large organization doing something simple

[fruey]

Two well identified principles at work here (and the bigger an organisation, the more likely they are to happen, especially without strong leadership)

1. Parkinson's law : basically, work spreads out to fill the time that was earmarked to complete a project
2. Brooks' law : Adding people to a project increases lateness, because the number of communication channels to manage increases as a square of the number of people on a project

Only very sound management and trusting delegation - along with having a reasonably competent project team in the first place - can make things happen quickly.

Re:Large organization doing something simple

[rwven]

I'm a consultant in telecom. I see this every day. I'm convinced that any project, no matter how big can be done by 6 people.

QFT. In my experience it seems like, for the most part, a small, highly skilled, highly focused team can accomplish at least as much (or in some cases far more) than any large team of developers/architects. Decisions are easier, faster, and cheaper to make when you have a group of people with industry experience and know-how. The amount of code needing to be laid down for most web projects really isn't THAT large....especially when it comes to a project like this NYT example.

I also think (and this probably goes without saying around here) that a top-heavy management structure is an instant doubling (or probably worse) of time and budget for any project. I was on a development team once that had twice as many people-management and project-management positions as there were developers, and it was an absolute nightmare. Developers ended up sitting around twiddling their thumbs waiting for someone to actually make a decision and call down the order to act.

Re:Large organization doing something simple

[PsyciatricHelp]

I actually use to work for the Times as a tech and all I can say is, man you nailed it.

Re:Large organization doing something simple

[dlingman]

I'm a consultant in telecom. I see this every day. I'm convinced that any project, no matter how big can be done by 6 people.

I guess you don't have any kids then. (What am I saying. This is slashdot.)

Re:Large organization doing something simple

[bjourne]

Hah requirement analysis! Aka doing a lot of mind numbing paper work that if you are lucky the engineers are able to ignore and still produce a quality product. Or, if you're not so lucky, have to follow it because clueless managers don't know how software design works resulting in releasing something which is a veritable disaster. People are notoriously bad at reasoning about abstract things, especially if they don't have the technical know-how on how those things really work, which is why "requirement analysis" is a waste of time.

Re:Large organization doing something simple

[jimbrooking]

Agreed. Long before most of you were born, i.e., in the 1960s-70s, it was a given that if IBM told you (a customer) that they were putting another 100 people to work on a piece of late software, they were really telling you the project had been killed. And of course there's the classic Mythical Man Month (http://en.wikipedia.org/wiki/The_Mythical_Man-Month) which, of course, no one reads any more.

Re:Large organization doing something simple

[wisdom_brewing]

Don't be so insulting to trainees!

Re:Large organization doing something simple

[dkleinsc]

I should point out that the NYT Paywall is apparently a much more complex beast than a simple "pay up to see the articles". What they're trying to do is allow search engines, Twitter, and other social media to drive traffic to them, but at the same time not allow people to regularly read their content for free.

My guess as to the approximate cost breakdown:
- consulting fees to convince the top brass to go along with this plan even though the last attempt failed miserably: $20 million
- project management and business analysis: $10 million
- profits for the purchase manager's brother-in-law's IT contracting firm: $9 million
- 8 developers and 2 testers to do the actual work: $1 million
- Watching online readership plummet again: Priceless.

Re:Large organization doing something simple

[SharpFang]

Prerequisite: the team must be equipped with powertools to cut through the red tape.

Say, open line to the CEO who just says "yes" to anything they say and authority to fire whoever stops them from performing their task.

Depending on company structure, 10-60% of the time of any "revolutionary" change is spent actually developing the change, the remainder is asking, waiting, begging, urging, pressing, explaining, escalating and generally overcoming people who while aware of the necessity of the change and futility of their resistance, will resist the change as much as they can (or see it as the opportunity to exercise their decision-making power, which is totally unneeded and unwelcome there but by no means anyone could ever notice that.)

Re:Large organization doing something simple

[sarhjinian]

I think you've pretty much explained the last ten years at Nokia.

Re:Large organization doing something simple

[Anonymous Coward]

The depressing thing about the "big budget" development efforts is that the bigger the budget, the bigger the likelihood that it will fail.

Some of the most successful software I've ever seen was fairly ugly stuff that was knocked out in short order by less than 5 people. Years later, as it finally begins to seriously creak at the seams, the corporation puts together a massive team to bring it up to date. They spend massive amounts of time and money on consultants, fad "silver bullet" development tools and fad "silver bullet" project management techniques. 18 months into the 2-year schedule, they realize they've spent a year and a half generation UML Actor diagrams, panic, and immediately set everyone to work coding, without any coherent blueprint at all. The whole thing falls apart, lots of people get laid off, and maybe 2-3 people go in and hack the original warty system to keep it going. With luck, one can repeat this cycle several times.

Re:Large organization doing something simple

[Anonymous Coward]

I agree. We're a 2 person company that's bid on projects against larger competitors. We've had customers call us in after going months or over a year with the larger companies and not having any deliverables.

We've turned around the project in a fraction of the time it took a larger team to even start. It's absolutely insane.

The common reason that we're not taken seriously is that we don't have a big enough team to deliver on time, and if these larger companies can't stay on schedule then we have no hope in hell.

The reason why we can turn things around in time is that we're committed, we know what we're doing, we work together well and we don't milk the customer.

We split the project and run with our tasks, we don't wait for an architect or management to sign off on anything.

I've worked for a large company and hated it. The politics were the biggest problem. Then management had to be educated in what had to be done. Write-ups by the architects had to be explained and clarified back to management because they knew squat. They were the ones that had to communicate to the customer so they had to know what they were talking about. We've suggested that some of the programmers could go and pretend to be the front guys just so the management training sessions would be reduced. That didn't go over well as the managers felt that they were being pinched out of their jobs.

What's worse is that the programmers that headed the projects were related to upper management. They had no clue but were given huge responsibilities, and all of the perks. One of the kids refused to do the job unless he got a Porsche as a starting bonus. Next day there it was in the parking lot. That's when I decided to leave...

Now, the two of us work from home. We have flexible hours, spend time with our kids and get the work done on time or ahead of schedule. We have control over the flow of information needed from the customers and 3rd parties and don't rely on others to forget to contact someone. It may sound inefficient but we're spending less time waiting for others and more time doing the work. We get together when we need to. Most of the time it's online collaboration or telephone.

Re:Large organization doing something simple

[ShadoHawk]

Yeah. And only one is needed after the project is started... ;)

Re:Large organization doing something simple

[betterunixthanunix]

And of course there's the classic Mythical Man Month (http://en.wikipedia.org/wiki/The_Mythical_Man-Month) which, of course, no one reads any more.

Funny, it was required reading in my software engineering course. Then again, perhaps it is not the developers who need to be reading it; more likely, it is management.

Re:Large organization doing something simple

[Just Some Guy]

What they're trying to do is allow search engines, Twitter, and other social media to drive traffic to them, but at the same time not allow people to regularly read their content for free.

Here you go:

if request['referer'] in (twitter, facebook) or 'Googlebot' in request['agent']: allow_access()

Since I only spent 3 weeks writing that, I'll just charge $25 million for it. Such a deal!

Re:Large organization doing something simple

[steelfood]

What they lack is vision.

You can pull off a big project with a large group of people. Countless people in the past have shown us you can do it, Steve Jobs being the most visible currently. But you need to have vision. You need to have somebody at the top going, "This is what it needs to look like in the end. This is the part you need to work on. Now go do it."

Most management meetings are more about answering the question that a single, lone visionary would've answered in two minutes, than about actually getting there. Rule by committee isn't only inefficient, it's the perfect way to get nothing done (which is why there are heads of state even in democracies). Management meetings are a form of rule by committee. Is it no wonder then that everything crawls?

Re:Large organization doing something simple

[TheRaven64]

The common reason that we're not taken seriously is that we don't have a big enough team to deliver on time, and if these larger companies can't stay on schedule then we have no hope in hell.

Next time you have a project, you can hire me as a consultant. You won't have to manage me, and I won't do any work, and will only require a token salary, but I'll increase the size of your team by 50% and therefore make you seem 50% more credible!

Its easy

[MichaelSmith]

A lot of it will have gone into executive information components of the system. Ways of showing the guys in charge exactly how much money they are making from the paywall this minute. Then you have the configuration interfaces and the teams to design datasets to control how the paywall works. Then you have the engineering which actually implements the paywall. They probably wrote a proxy from scratch to do that. Then they put it through validation. This created 10000 bug reports. Thats a lot of bugs so they outsourced the bug fixing to four companies in India who approached the solutions in 223 different ways. Then the resulting code changes were merged back into the mainline with bugs closed. Nobody wanted to do the tests again which was probably a good idea for the sanity of the people involved. Then they went live.

Well, thats my guess, anyway.

Re:Its easy

[gl4ss]

you actually forgot one bit.

hiring 'shadow developers' (also known as retards) and inflating the team size from the 10 needed to the unnecessary 100+ with testers. the times still receives just the output from 5 good guys but gets a bill for 100+. and every new person introduced to the team is more profit, as the there's more hours and persons to bill for(so they hire an unnecessary new guy/gal to inflate the team, the new person costs say 5000$ a month, the client is billed for 6000$, the company ruining executives keeping the difference . and universities have been just inventing new professions so it's pretty easy to inflate the team with sheep experts, marketing advisors, accessibility destroyers etc. and every single one of them will try to come up with output that would change the end product somehow, to justify their job).

CAL's

[Anonymous Coward]

Microsoft client access licences for all subscrbers?
Paywall is implemented in sharepoint

What did they spend the $40-50 million on?

[Anne_Nonymous]

Consulting.

What else offers so little for so much?

Re:What did they spend the $40-50 million on?

[SharpFang]

The sad part is that there *are* good consultants out there. I'm one of them. I'm extremely skilled, knowledgeable, and I bring a lot to the table.

...but that's what they all say.

A simpler way.

[khasim]

Follow the money.

Someone is getting paid. Find out who and what that person's connection to the person signing off on that expense is.

Re:A simpler way.

[paiute]

If this was government work you'd know without a shadow of a doubt that they're laundering political kickbacks through IT projects

You mean they're overbilling and using the overage to fund black ops projects like unmanned shuttles.

"You don't actually think they spend $20,000.00 on a hammer, $30,000.00 on a toilet seat do you?"

Re:A simpler way.

[a2wflc]

I'm sure there is plenty of slush and black ops, but it's also the military getting around political interference.
In the early 90's it was something like this:
The military says "we like the old helicopters and don't want the new ones".
Congress says "we won't fund spare parts for the old ones so you'll need the new ones by the time they're ready."
Supplier says "pay us $20k for a hammer and we'll throw in $15k of spare helicopter parts."

Re:A simpler way.

[Reeses]

You mean they're overbilling and using the overage to fund black ops projects like unmanned shuttles.

"You don't actually think they spend $20,000.00 on a hammer, $30,000.00 on a toilet seat do you?"

Having worked for the government, yes, I do. If it's a one off project. Or only a handful.

There's no "open market" for the government to allow to absorb research and development costs or to achieve economies of scale.

If you need a special inertialess (non-rebounding, not physics-violating) sparkless hammer that can be used in an explosive gas-filled environment, and you need, say, 3. The government will commission and buy 10. The initial development and testing costs all get rolled into the cost of those 10 hammers, no matter how many man hours and resources it takes.

And, having watched the USMC test a piece of equipment in a "shaker box" designed to imitate driving a HumVee over rough terrain, hearing the salesperson from the company say, "Oh shit, there's no way this is going to....." and watching the piece of equipment explode 6 seconds into the test, there's a lot of engineering involved in meeting weird military requirements.

Is there money hidden and wasted in there? Yes, probably.

Does that hammer really cost $20,000 to develop? Yes, probably.

Re:A simpler way.

[_Sprocket_]

Follow the money.

Someone is getting paid. Find out who and what that person's connection to the person signing off on that expense is.

Sure. But don't expect to always find some nefarious link. The world of bureaucracy is bigger than just corruption. Although bureaucracies do make a breeding ground for corruption and they become more unwieldy the more the system is adapted to eliminate corruption.

I've had a lot of direct experience dealing with government bureaucracy at various levels and organizations (and some experience with corporate bureaucracy). I've seen relatively simple tasks turned in to months-long projects and couple months worth of thorough effort turned in to a multi-year outsourced contract. This isn't because someone was getting a pay-off. This is because The System, of which every good Bureaucrat serves and follows, demands levels of effort far beyond anything anyone not serving The System would think sane. So while the tasks themselves can be simple, performing the tasks within the bureaucracy requires many more additional steps that require many more man-hours to accomplish.

I should stress that corruption can still rear its ugly head. I haven't viewed it very often myself. But I've dealt with rules that have come in to place to close a loophole exposed by someone who had figured out how to game the system and got caught doing so. There will be people that are gaming the system according to these new rules and the process will repeat itself (as well as the occasional case of someone who thinks they won't get caught doing something others got caught doing).

Explanation.

[Seumas]

They hired the same guys the government uses to create $10mil Drupal websites.

Corporation

[moberry]

$45,000 for the implementation, and $39,955,000 in management bonuses.

repurposed joke from recently read comment:

[circletimessquare]

http://politics.slashdot.org/comments.pl?sid=2067216&cid=35703166

An investment firm is hiring mathematicians. After the first round of interviews, three hopeful recent graduates - a pure mathematician, an applied mathematician, and a graduate in mathematical finance - are asked what starting salary they are expecting. The pure mathematician: "Would $30,000 be too much?" The applied mathematician: "I think $60,000 would be OK." The math finance person: "What about $300,000?" The personnel officer is flabberghasted: "Do you know that we have a graduate in pure mathematics who is willing to do the same work for a tenth of what you are demanding!?" "Well, I thought of $135,000 for me, $135,000 for you - and $30,000 for the pure mathematician who will do the work."

thank you, SilverHatHacker (1381259) for the joke

Re:repurposed joke from recently read comment:

[Psicopatico]

That wasn't a joke

Stupid comparison

[Anonymous Coward]

Apples: Creating a search engine from scratch. The main hook is that it is simpler than existing products. User workflow involves typing in a term and clicking a link, and users are interested in it because it's different than competing products. No money (remember, this is before Google Ads) is changing hands.

Oranges: Changing the user experience for a major existing site. Users are already familiar with the existing site and already inclined to react negatively because you're now charging for what was free. Money is changing hands, so a complete system for handling disputes and showing purchase history is required. The whole system has to hook into existing customer service systems. Customer service systems behind the scenes have to be extended. Support personnel have to be trained. Legal considerations for multiple states or possibly nations may be involved. Management needs reporting features.

Re:Why? Rupert likes to keep money moving around

[charlievarrick]

Great theory but Newscorp does not own the New York Times, it owns the New York Post.

Re:Why? Rupert likes to keep money moving around

[wombatmobile]

Rupert Murdoch doesn't own the New York Times. He owns the New York Post.

Good question

[QID]

I work for a large, multinational corporation, full of all sorts of layers of management and unpleasantness, and the current rather sizable program I'm working on--months of development and engineering work, lots of hardware, custom-built stuff ordered from all over the place--is still well under $40 million. If this is one of the supposedly greatest newspapers in the world and they manage to spend that much money on so little, no wonder print is fucked. They've done it to themselves.

For comparison's sake

[osgeek]

The Obama administration's web sites for promoting transparency in government were around $34 Million just to keep them running.

1 million for the code

[DrXym]

39 million for the oracle license.

The Yorkshire Ranter thinks it's reasonable

[Eunuchswear]

The complexity of the rules makes it sound like a telco billing system more than anything else - all about rating and charging lots and lots of events in close to real-time based on a hugely complicated rate-card. You'd be amazed how many software companies are sustained by this issue. It's expensive.

http://yorkshire-ranter.blogspot.com/2011/04/scaling-and-scoping-nyt-paywall.html

Dude, shut up!

[jayhawk88]

Look, we told them we had to program an autonomous artificial intelligence agent to proactively scan cyberspace for hackers looking to bypass the firewall using port cross-scripting. They bought it, don't screw this up for us.

It's easy

[HangingChad]

I watched the Navy burn $27 million on a glorified CRM that used Siebel and never got any working components. While that clusterfuck was going on a small team of four people built a prototype type system that was eventually rolled out to production because it was the only one that worked.

The person responsible for the $27 million dollar disaster got promoted and took over management of the working system, which they promptly turned over to EDS to manage.

When it comes to software development, spending more doesn't necessarily get you more.

Uhh...

[Panaflex]

Well it's quite possible to "book" the value of a project much higher than you may actually outlay at any present time. Remember, these costs get deducted, deprecated and ultimately reduce tax burdens.

Or maybe it was just hookers and blow...

Comment removed

[account_deleted]

Comment removed based on user account deletion

Works the same way in the private sector

[plopez]

I've posted this already but go ahead and review it. Corruption happens in the private sector too.

It's sort of like:
  Contact a golf buddy who you went to school with and were in the same frat with. Skew bids and and keep out other contractors by disqualifying them with statements like "they don't have the depth", "they don't have the scale we need", "I heard they dropped the ball once or twice", "we should look at company X because they also but our services and/or products", etc.

Both execs or PMs on both sides trumpet the accomplishment and bail out with in a year. They agree to give each other very good references.
The company hired doesn't have a clue and also farms out the work to a low bid offshore contractor. On the buyer side, the new exec or PM is left with vague and poorly written requirements and thrown into a new environment working with an inexperienced PM on the vendor side.
Often neither new PM has a background in software.

So hilarity ensues, while the original culprits abscond.

Or variations on that theme.

The corruption is there in the form of soft soaping and self-dealing. Looking after your own self-interest before the interest of the company, which is unethical when you are a manager and charged with looking out for the company.

Does this make sense?

Re:Works the same way in the private sector

[jbengt]

Oh absolutely, I wasn't arguing otherwise, it's just that the private sector is a bit more discreet about it, they try to cover their tracks. In government, a fuckton of money just disappears and everyone whistles and looks away.

In my experience, it's the other way around. Private companies are usually quite open about giving contracts to their preferred contractors; they don't have any obligation to be fair, and long-term relationships tend to breed a desire to do the job well (as long as enough competition remains to keep their prices honest). Government entities produce mounds of red tape in order to "prove" that they are not giving favors to or getting favors from a contractor; their projects are often awarded to a barely competent contractor who submitted a too-low bid, leading to requests for change orders, cost overruns, and corner-cutting.

Re:No surprise

[peragrin]

I only agree about Bush Sr. He was a good guy.

Reagan did kill the soviets through his spending but the stock market crash of 1987 was the result of that spending.

If you look at what happened form 2001-2008 you will realize one fact, the only thing keeping the country going was new home construction and loans, and once that stopped it all stopped.

We never really recovered with a diversified economy from the dot bomb, but it was hidden behind huge home sales and tax breaks.

My personal income tax has yet to recover from that mess. Every year I have to pay more than the previous year though my salary only rarely goes up.

$20k is cheap for a hammer

[name_already_taken]

Supplier says "pay us $20k for a hammer and we'll throw in $15k of spare helicopter parts."

That's not how it works at all.

You want some helicopter parts thrown in? The military guys know to not even try that. That's not what happens at all. Everyone is covering their asses. If some military helicopter part fails, you can bet that the procurement chain will be examined. There has to be a paper trail for everything.

Supplier says "pay us $20k for your crazy over-spec'ed* hammer, and we'll jump through your stupid purchasing hoops, go through all kinds of extra work certifying things that have nothing to do with the performance of the item, fill out reams of unnecessary paperwork, send an employee to a special course so they can learn how to enter invoices in your arcane billing system (btw, commercial invoice ~1 page, government invoice ~30 pages), wait thirty days for our first billing to be rejected because of some minor issue (100% chance first billing is rejected, btw), submit corrected billing, wait 30 days for someone to tell us that the contract was shifted to another department and so it has to be resubmitted again (they knew 2 days after we billed them, but they're not required to respond until 30 days, so they don't), wait another 30 days to find that the billing was accepted, then wait 60 days for the payment to show up".

Many companies turn government business away, because the documentation requirements are onerous, the payment terms are ridiculous, and the project may be cancelled halfway through anyway.

*The requirements on military items would make your head spin. Making a tiny design change to a part to make it easier and cheaper to manufacture can trigger everything from having three government people sign off on the revised drawing all the way to having to run a live fire test at some proving ground where they strap your whatever it is to a tank and drive it around, attach it to whatever gun it's supposed to work with and fire 1000 rounds, or shoot at it, depending on what it is. All for a really minor change that was never going to affect how it worked in the first place.

You may think a $20k hammer is ridiculous, and it is, but once you see the paper and testing trail, it starts to look reasonable, assuming it's not an off-the-shelf-item (very few are). Now, if they're buying more than 10 hammers, that price had better come down, but for a one-off, $20k is a bargain. Heck, it'd probably win an award for cost savings.

Re:Microsoft and Apple

[ColdWetDog]

What did you do this morning? Spill some Reality Distortion Shield in your coffee? Apple (which may or may not use 'small teams of excellent designers and programmers) pushes out stuff that is buggy as hell. Anybody with half a brain and a reasonable grip on reality doesn't buy any Apple product until version 2 or, better yet 3. What Apple does it that it concentrates it's folks on just a few items compared with the thousands of SKUs that Microsoft has to deal with. They don't have to cover the breadth of support issues that Microsoft does. Deprecating technologies really does work for them. They do seem to be better at limiting feature creep when compared to MS and clearly they have a smaller administrative chain to deal with.

But they're perfectly capable of coming out with impressively buggy products. I should know, I own a bunch of them.

No worse than others but not markedly better.

It's their plan to pay Zero taxes

[opusbuddy]

Only not as transparent as GE.

Actually had a twerp from NYT Tech support tell me this morning that they had a new iPad app. True, 2.0.4 (the only version on the appstore) was new on April 1, 2010.

From Wikipedia, the free encyclopedia

According to the Collins English Dictionary 10th Edition fraud can be defined as: "deceit, trickery, sharp practice, or breach of confidence, perpetrated for profit or to gain some unfair or dishonest advantage".[1] In the broadest sense, a fraud is an intentional deception made for personal gain or to damage another individual; the related adjective is fraudulent. The specific legal definition varies by legal jurisdiction. Fraud is a crime, and also a civil law violation. Defrauding people or entities of money or valuables is a common purpose of fraud, but there have also been fraudulent "discoveries", e.g. in science, to gain prestige rather than immediate monetary gain.