Specs of Salons Subscription System
legLess writes "Scott Rosenburg, Salon's VP of operations, wrote an interesting article for Web Techniques about Salon's subscription system. It goes into a fair amount of technical detail (JavaBeans and JSP on Linux for login and authentication; Perl, HTML::Mason and MySQL (CD: and oracle) for content). He also talks about their subscription numbers, what drove them to do it, and their plans for the future (technical and operational). A little fluffy, but still a good read."
Since I am playing with Tomcat and Cocoon myself right now, I was interested to read about how Salon is also gettin into XML publishing techniques.
-Mark
I read this article when the mag arrived, and was mostly fascinated by their decision to build it all in-house. Two or three years ago, they most likely would have thrown a ton of money at another company to develop the whole system. They would have gotten something that was not exactly what they wanted, and no doubt three months late (at least). Companies finally seem to be realizing that they need to make better use of the staff they have, and that even adding a few programmers or other employees can be cheaper than hiring outside developers. That's what I'm seeing at my company now, anyway, and others I know.
In terms of their technology, I think managing two page management technologies (JSP and Perl/Mason) would get a little tired, and is likely unnecessary. While JSP might not be fast enough to handle the Mason-generated pages, you can certainly use Perl to transact credit cards if you want.
From previous postings on this site, it seems that Slashdot will be going to subscription route as well. I think its a good idea. The quality of posts will probably improve (the best posters appear to be the /. addicts who would likely subscribe), and there would be capital in place to provide extensive services on top of what is already here.
Actually that's just because it is badly implemented by Slashdot. All they need is to put a second DB server in replication and, even when the master server is down, it can still be used as a read only backup (which is fine for serving content).
As I recall, when Salon first implemented their premium content, they didn't mark what content was premium, but instead when you viewed the story it would give you the first couple of paragraphs then say that you had to subscribe to read the rest. This is a big mistake, as it really makes non-subscribers resent the wasted click, and I'm glad they finally wised up and starting marking everything clearly as premium.
/. that point to noteworthy stories on the web, Salon is one of the only true purveyors of Internet journalism that provides stories worth pointing to. They have top-notch folks reporting and provides commentary on the whole spectrum of news.
I had no problem subscribing to Salon, though, because these guys are the real deal. While I love sites like
my biggest point of annoyance with the site was when they took their news coverage out of the free portion of the site. they chose a moment when everyone around the world needed good, accurate reporting of the developing events after 9/11, and exploited that moment to expand their subscriber base. that should have been the moment when they kept everything open and freely available, perhaps soliciting for donations to keep the quality of coverage up. instead, they chose the greedy road and shut a lot of people off from a good information source.
"Why is all this crap here?" -- 4-year-old Brandon
There were only two before this article, however. Maybe not a trend, but still prevents one...
No wonder they have 87 million in debt. Why the fuck do you need oracle to serve content? MySQL works fine for slashdot,
MySQL is a piece of shit. (no nested selects? WTF?) And incase you hadn't noticed, slashdot fails all the time. The only reason they stick with it is because the source was originally hard-coded with MySQL API calls. Andover even pored money into the company behind MySQL to beef it up when it was flush with IPO money. Nowadays they're using proprietary extensions (witch cost $$, btw).
MySQL was not designed for a lot of the things its being used for.
You're comment is a lot like saying: "Who the fuck needs a mainframe? my DOS LAN works fine!" or something.
autopr0n is like, down and stuff.
Personally, I would pay only for Slashdot Unpremium (not Jon Katz, no Michael/Timothy, YRO, in other words, just the basic stuff that slashdot was founded on).
If you go to your user prefrences (if you have an account) you can filter out certan authors, (katz, michael, etc). Not that I don't think katz should be thrown off the site or anything.
autopr0n is like, down and stuff.
i.e. they use Java/JSP for login/subscription control). They also use Oracle for "more complex database management needs". Oh, also, they use R:Base (!!! - hey that was old when I got my first job in the late 80's).
Sounds like the typical real-world reality of a production environment:^) I like Grant's spin on using Java/JSP:
"Grant chose to introduce some diversity into Salon's infrastructure"
This month, Salon launched a monthly subscription program for 6$.
Reading the article, I have to give credit to the way Salon deal with their readers.
1. Even the free site is not overwhelmed by ads like those flash based ones that run around the page on wired, or those poping pages on yahoo.
2. The price for the site is really low, compared to the price you would pay for a daily newspaper. They understood that internet users CAN pay for content but at a reasonable price.
3. They give premium content, not only ads-free stories.
Thumbs up, Salon.
Men are born ignorant, not stupid; they are made stupid by education. Bertrand Russel
If I remember correctly, Linux wasn't designed for many things it's now being used for, either...
After reading this article, I begin to see why it will be doubly tough for Slashdot to make the jump to subscription-based revenue.
The biggest problem I see with Slashdot is that Slashdot doesn't have a Cringely or a Coursey or a Dvorak. Sure, Slashot has Jon Katz, but I just don't find his articles as readable as a Cringley column or a Dvorak rant.
I see the real difference between Slashdot and Salon on a couple of other fronts as well. Besides not having several columns by intersting authors, most of Slashdot's content is made interesting by the readers, not by the story submitters. Personally, I am horrified by both the obvious lack of attention given to grammar, as well as the oft-biased one-liners added by the story submitters. Finally, although it has gotten better in recent times, Slashdot seems to crash a lot... even more than an overloaded MySQL database would suggest.
For Slashdot to take a viable community and turn it profitable, the story editors do a lot more than Salon did. The fact remains that Salon's content is mostly unique, whereas Slashdot's content (in terms of story submissions) is mostly regurgitated. Salon's readers will pay because it's hard to find Salon-like articles anywhere else. On the other hand, I can honestly say that if ZDNet had a moderation system, I'd only rarely visit Slashdot. ZDNet's columnists keep me entertained, and their news is grammatically correct and up-to-date because they pay people to go out and write it.
It all boils down to whether Slashdot can successfully differentiate itself from the hundreds of other "Cool Linux Stories" sites. In the end, what keeps Slashdot's readers coming back is the discussion and the attached moderation system. What remains to be seen is whether or not people will pay for that.
Even the MPS/Oracle part is driven by perl. MySQL still holds the content and the users.
I do like the bit near the end about cost and time-to-market, tho. ;)
I'm so tired of the whole "The free web is over" rants. Why do so many people think the commercial part of the web is the only part of the web? People have created content for free for a long, long, long time. Whether it be music, art, comics, poetry, literature, editorials, films, TV shows (yay for public access cable!), books, whatever... Many people are driven by ideas or creativity they just want others to pay attention to, without getting paid for it.
So the "Soon you'll have to pay for all your web content" chant really means "Soon you'll have to pay for all the web content dished up by commercial organizations." Good. I don't need it anyway.
The Internet is cool because all the free content that was out there to begin with can now be put online by anyone and viewed by anyone. So your underground newspaper, or garage band, or your off-the-wall comic strip has the potential to be viewed by a lot more people than just those in your town or school.
I don't care if I have to pay for content owned by AOL Time Warner or whoever. There's plenty of people out there who want me to look at their stuff for free. (gee, kinda like how the Internet used to be)
Macintosh humor! MacComedy.com
"All they need is to put a second DB server in replication and" hehe. You forgot that MySQL doesn't support replication which re-inforces the earlier comment on why MySQL sucks.
However this was the case close to 9 or 10 months ago when I checked. I haven't checked back since. I am under the impression that the replication features have not been implemented in stable form yet. Also slashdot runs the stagble release of debian which is very old. If replication is supported as of today it will be awhile before its running on slashdot.
http://saveie6.com/
Slashdot seems to crash a lot ?? What OS / Browser are you using? I have been "member" of /. for years ID 7110 and I can't remember having a slashdot Crash. (Currently using Opera 6.0 on SuSE 7.3)
Help fight continental drift.
Salon should look into selling/licensing their code to other content providers. Bandwidth is not getting any cheaper, and a cash strapped web company would probably love a chance to buy Salon's code as opposed to spending the money to do this themselves. I know that it is not Salon's MO, but they need money, and so does just about everybody else on the web.
Now I believe they have merged with the Well, but unfortunately it doesn't appear that they have a community message board worth paying for. They would really benefit from using slashcode or another engine. What they have no is pitiful.
http://saveie6.com/
While I agree that there is something odd about users paying for a site where the users generate the content (via posts), its the network of users that one is paying for access to, not a particular user or set of posts.
From the article:
:)
In the end, of course, our subscription plan has worked because a small but significant portion of our users feel that Salon is worth supporting with their cash.
Alas, this happens when you get older: you start paying for support, news and, alas, women.
A little fluffy, but still a good read.
Why do slashdot editors feel that they have to throw in their 2 cents. I'm glad you posted the article, thanks for the summary, but I don't want to hear your lame ass opinion. Especially when it is something as non-specific as a little fluffy. It's like you are afraid that all the slashdotters will be talking amongst themselves and saying "what's up with that fucking fluffy article that was posted", so you are attempting to protect yourself against that eventuality by saying - hey this is fluffy.
Just post the fucking thing.
Thank you.
LA Times, Chicago Tribune, Newsday, Baltimore Sun, Orlando Sentinel... all owned by a firm not on your list.
Oh wait, they are (relatively) conservative papers, so you can't possibly trust their reporting.
IfI want a leftist viewpoint, I read Salon. When I want to see what the extreme right wing is saying, I read WorldNetDaily. Slashdot tends to lean about as far to the left as Salon, but doesn't have as much porn, which I suspect is the real reason many people subscribe.
No, honestly, I just read it for the articles!
I do not deploy Linux. Ever.
Look Here The server has been up for 80 days and 20 hours. Not too shabby.
Help fight continental drift.
While I am a proponent of building it yourself, alot of times naive people look at something like an ecommerce billing system (for example) and decide it should be easy because they are inexperienced. This is the same mentality people use when they decide to re-write big sections of code to make it cleaner! Hello! All those "unclean" parts are bugfixes that someone had to figure out and apply.
On the one hand, the VP of engineering (and apparently the only programmer at the place) had done something like this before. On the other hand, considering the problems they still need to deal with (handling foreign zip codes for AVN) kind of implies they have a long way to go... Obviously hiring someone else to do it does guarantee it will work well but a decent company will have solved all the problems you don't even know your going to have if you DIY.
I am not a number! I am a man! And don't you
Easy ... so when the go down the crapper in 2 months, your only out $12.
I am not a number! I am a man! And don't you
I used to use Encyclopedia Britannica once in a while. But now, I can't use it at all because it's a pay site. I can't justify paying a subscription when I might use it once or twice a month.
I currently pay to subscribe to one on-line content provider : Cooks Illustrated. Its worth it to me. Salon might be worth it to me, too, but I don't buy it. As more and more sites go pay-for-play, I'm not going to be subscribing to dozens of sites. 1) I am only a casual reader, and 2) even if I thought it was worth it, I'm not going to pay hundreds of dollars a month to keep up with all the sites I visit.
And here's where it starts to break down: the (commercial) web can end up just like print media. Sure, any print publication could be subscribed to by everyone, but everyone is not going to subscribe to everything, or even one thing. So you have your subscribers, and you don't reach anyone else.
I know that Salon Premium pretty much drove me away from Salon. I accept that in the future, I will be much more limited web site availability. I'm willing to pay that price because all that casual content isn't worth the thousands of dollars it would cost to maintain subscriptions.
Micropayments, anyone?
Robotiq.com is heavily tested on animals
You are right, come to think of it, that happens fairly frequently. I go thru a NAT server then to Squid for my house network. I always thought it was a problem in my end. Thanks for info.
Help fight continental drift.
oops I was wrong. Recent release info is here. Transaction support is both supported by Mysql and PostgreSQL.
http://saveie6.com/
here. Interview with COO Patrick Hurley from August this year. The interview is very "content provider" oriented and is a good read.
-Kraft
Live and let live
The article doesn't address whether all of the premium content becomes free once archived. I certainly hope it does. I think it's important for Salon's influential content to be widely available for reference use (linking) in other on-line content.
It would be a shame if other authors on the Web could not link to an important Salon article. Linking is practically the definition of influence on the Web (ask Google). Influence makes Salon more attractive to subscribers, and so on.
Geez, I *bought* the (Compact, but perfectly readable with the magnifier) OED for US$300...
"Biped! Good cranial development. Evidently considerable human ancestry."
Does this mean I can get a free subscription if I just edit the cookie file of a friend and grab the cookie?
For you information, one of the benefits of "Premium" is that you can download the daily issue as a pdf. Good for reading on the commute, or on the commode, perhaps?
One possible future model is actually a return to an old one: a model similar to that of CompuServe or America Online before the internet explosion--a package of access and (often exclusive) content.
- /. had more original news pieces
- Jon Katz wrote something worth reading
- Paying members could turn off all ThinkGeek ads (et. all)
- The moderators were fair [troll]
- Paying users could turn OFF the Cowboyneil option in polls
Sapere Aude - Homer
Ever heard of pay-per-view?
Visit sunny Knowumsayin.com, home of the pork shirt.
That more expensive newspaper, you can give it to a friend when you are done reading it. Not sure what Salon would do if you tried the same with them...
The one thing that disturbs me about Salon's model is that it is absolutely not available to those who can't afford it, i.e. poor folks. Something like, say, Time you can go to your local public library and read.
Maybe libraries are subscribing to Salon. I dunno. If not, it seems like their solution might be exacerbating the so-called technological divide.
Note: I say all this as a subscriber.
Visit sunny Knowumsayin.com, home of the pork shirt.
In my experience (as a longtime magazine production person), magazines with some sort of bias are more effective than those that have none. In fact, I can't think of a magazine without some bias or other...
Point being, just because you don't agree with their POV doesn't mean they won't find readers.
Visit sunny Knowumsayin.com, home of the pork shirt.
I introduced static mode I think close to a year ago. Its used when the DB is down or a site update is occuring (like we put in bug features and such). We do the bug/feature additions probably twice a week now, and they normally happen around 1AM Eastern time as of late.
.shtml, everything else is disabled.
:)
During static mode you can only reach pages that are
Its certainly better then the old way which was to just to dish out zero content
You can't grep a dead tree.
Both of you. Please.
What might work is, say, a service that lets you buy topics, like "politics", "literature", or "entertainment industry news" for a flat fee, but covers a large number of publications. Like AdultCheck and PornoPass, but for people who read. The "adult verification systems" are commercially successful, unlike micropayment systems. As usual, the porno industry figures it out first.
In the last few weeks, the site has been overwhelmed by ads.
I seriously dobut that many people would go for a /. subscription. It just isn't worth it, I mean, all the content is user generated. Salon, obviously, is a real magazine with paid reporters and all that and has a valid excuse to charge money for what they're doing. /. just links to other stories and lets people post about it.
Maybe if the subscriptions were like $5/$10 a year or something.
autopr0n is like, down and stuff.
May be a little unrelated but can't stop myself from offering my own comments on the cost associated with subscription
Slate.com which competes for the same market as Salon ran a subscription experiment couple of years back. They ended up signing about 26000 subscribers. They were charging only $19.95/year, which is a pretty low price point considering that it was costing several times that much, even on the very best campaign, to acquire a subscriber. The cost of acquisition was averaging between $50 and $100, so obviously Slate was losing money on every subscriber we signed up.
Eventually, they decided to go free again.
Slate has only 40 employees while Salon has double the number - I therefore except their costs to much more.
I think even with the $72/year that Salon is charging
they are losing money. I would be suprised if they switch back to being a free ad-based site.
Actually I'm running two DB servers with replication and all I can say is, it just seems to work (the slave seems to indeed reflect perfectly the updates received by the master). It was set it in less than 30 min :) . If your web scripts can do "if (master is down) then connect_to_slave and do_read_only" then you are set...
I don't think I'd subscribe to any site. Some studies(sorry, no link) have shown that people will do without or stop visiting a site rather than pay anything. /., and I even click on them because I'm curious, but I get cranky when a news site spawns two windows that are tricky to close(Using IE).
You know, I don't own a credit card. I'm not going to mail a money order(no checking account either) which costs 5$ CDN across the border. I honestly think I can do without. Sure I hate advertising, and I see it is getting worse. I could care less about one little ad on
It was particularly interesting to me since we at Goofball.com went through the exact same process a while back ourselves. We initially were a free site and at our peek were doing close to a million pageviews a day - not including the Apache requests for video downloads (that amount to like 500 GB per month in bandwidth). We were actually leasing 7 Linux boxes (3 running HTML::Mason dynamic content servers, 3 plain Apache image servers, and a MySQL DB server) and paying through the nose for them due to the bandwidth. However, back in the good old days of the CPM advertising model, we were making plenty to afford the costs.
We were of course f----d when the "new economy" came to bear and we suddenly had no income for close to a year. Good thing we saved all that money we made on ad banners. We were eventaully forced to either close it all down or move to a payment model. We polled our viewers and decided to try the payment model based on their feedback.
The site is now 80% pay / 20% free. We're not just charging for access to specfic areas of content, but for the actual utility provided for by the site's various personalization services. We also decided that micropayment were not feasible. Can you imagine the headache of tracking pennies for pageviews and the associated overhead of dealing with people's questions/complaints about charges to their credit cards? A yearly fee of $19.95 (or a nickel a day) was the way we went and you know what - it actually saved the site.
Granted, traffic is at 10% of what it was, but that allowed us to drop off a few machines from the server farm and thus reduce costs further - keeping us slightly in the black each month.
HTML::Mason is the key to the site's success for sure. We can provide dynamic content on the fly pulled from the database, but a key element of the delivery machanism is Mason's built in caching. Only the first page request for a given URL (that uses the DB) actually requests the data from the database - subsequent requests are pulled from a GDBM replica of the DB's content that was cached by the first request. Mason also provides us with the ability to maintain a persistent DB connection during the life of the Apache daemon. Additionally, the same Apache/mod_perl/Mason binary also listens for requests on port 80 and 443 for SSL requests. All credit card transactions are handled through a Mason enabled API gateway. All of our back-end tools are HTML::Mason as well.
I didn't really get the part about "needing Java/JSP" in the Salon.com story. It sounded to me more like the author was not really in touch with the particulars of the technology at hand and was just repating what reasons he may have been given by the development team (who may have been looking to learn something new for the sake of it). I just came from a job where a decision was made to "go with a Java solution" simply because of the name of the programming language more than any other factor. I have nothing againsts Java believe me, but I'm so tired of buzzwords being used to influence decisions that are actually in dire need of pure business and software logic instead.
I'd encourage everyone - if they haven't already - to have a look at HTML::Mason. And also, for a good diversion, take a look at Goofball.com.
The main problem with the web subscription model at this point is that one must pay separately for one's internet access. Imagine having to pay fifty dollars per month for the possibility of watching cable or satellite television. To actually view any content, one would then be required to subscribe to each channel individually, at additional cost. Not many takers, eh?
Not entirely true. To view cable or satellite you'd have to buy a house, pay for electricity, and buy a tv. There are lots of other things your paid monthly internet access fee gives you besides just what you can subscribe to.
With that said, if those subscription services were to offer you a free dialup number to connect to and download their material without the need for an isp....
One possible future model is actually a return to an old one: a model similar to that of CompuServe or America Online before the internet explosion--a package of access and (often exclusive) content.
That would be a great idea. The only thing you'd have to worry about is, will they survive if they put that much $ into the infrastructure of other services they're not expert in. Look at Aol these days, they force ads down *paid* member services, AND their content is 100 miles wide but an inch deep. All they have are things you can already find on the internet: chat rooms, newsgroups, message boards, IM, stocks, basic news...
eTrade SUCKS
what about the other kind of economic discrimination: advertising. Advertisors influence the content of a site. Also, if you're trying to peddle a product or spice up you're corporate image -- you don't exactly want your ad to appear next to an in-depth four part study on homelessness. Finally, ads are geared to a target market which is usually affluent, so there is economic discrimination built in, too. I think advertising is the number one cause of all of the superficial, feel good, smiling prattle in the big media. Subscription is a pretty good way to go in terms of avoiding bias and paying the bills.
When in doubt, have a man come through a door with a gun in his hand.
I agree, $72 per year is a bit much.
That's why Salon only costs $30 per year.
If you want only one month at a time, it's $6/mo.
HBO seems to be doing fairly well, these days, and has been since the early days.
Note: "Early days" in this context is defined as the time when "basic cable" meant paying a sum of money every month to a company which would deliver to your home via coaxial cable a handful of local television stations which could easily be recieved -for free- with an antenna, with the possibility of paying an additional monthly sum for HBO.
Kid-proof tablet..
As someone else pointed out, paying $72/year for any online amature news site is completely outrageous.
I can get magazine subscriptions (PCMag, et al) for like $40/year... I can get a HUGE newspaper at my door *every day* for $60/year, and I can *guarantee* you that it has a lot more coverate and stories and information than any online news site around will have, and it will be a lot better written too. Plus I'm not tied to my computer to read it, which as of right now is a nice thing. Sure, the wireless internet is great, but during a 45min commute to work in a carpool it's great to just flip open a newspaper.
I'd pay for salon if it was $30/year, but I think that's the magic number for me. Any more than that and there is a lot better places to go to get the information that I want.
If God gave us curiosity
use something called "adult check" that works like that. I guess it's successful. I haven't tried it, thank you.