Amazon Cloud Adds Hosted MySQL

1sockchuck writes "Amazon Web Services has added a relational database service to host MySQL databases in the cloud, and is also dropping prices on its Amazon EC2 compute service by as much as 15 percent. Amazon says the new service lets users focus on development rather than maintenance, but it will probably be bad news for startups offering database services built atop Amazon's cloud. Cloud Avenue warns that Amazon RDS should serve as 'a warning bell for the companies that build their entire business on Amazon ecosystem. ... They are just one announcement away from complete destruction.' Data Center Knowledge has a roundup of analysis and commentary on Amazon RDS and its impact on the cloud ecosystem."

Comments

Showing their cards at last

[Mateo_LeFou]

Turns out "the cloud" is just another name for "datacenter". Who knew?

Re:Showing their cards at last

[slim]

I'm getting increasingly fed up of every cloud story getting piles of comments deriding cloud as "just" something else.

- "The Cloud is just another name for datacenter"
- "The Cloud is just another name for distributed computing"
- "The Cloud is just another name for thin-client computing"
- etc.

In this particular case, yes, the backend of the Amazon cloud is a bunch of datacentres.
And you could build a virtual datacentre in the Amazon cloud.

But that doesn't mean that every datacentre is a cloud, because a cloud has properties that most datacentres do not.

Re:Showing their cards at last

[Anonymous Coward]

The Cloud is just a buzz word. It makes non-techies feel clued in without having to understand the differences among a handful of technologies and how they work together.

Re:Showing their cards at last

[base3]

And he forgot "The cloud is just another name for timesharing." The 1960s called; it wants it glass house computing model back.

Re:Showing their cards at last

[slim]

The Cloud is just a buzz word. It makes non-techies feel clued in without having to understand the differences among a handful of technologies and how they work together.

At first I thought you were contradicting me. But you're not, necessarily.

"Cake is just a buzz word. It makes non-bakers feel clued in without having to understand the differences among a handful of ingredients and how they work together."

Combine eggs, flour, baking powder, sugar, flavourings, in just the right recipe, you get a cake.
Combine datacenter technogolies, virtualisation, parallelisation, timesharing, web based management, in just the right recipe, and you get a cloud.

This doesn't mean that "cake" or "cloud" aren't useful shorthands.

Re:Showing their cards at last

[fucket]

So you're saying... the cake is a lie?

Re:Showing their cards at last

[trevorrowe]

But that doesn't mean that every datacentre is a cloud, because a cloud has properties that most datacentres do not.

Maybe not, but that is besides the point. The "cloud" is just a fancy way of saying "other peoples servers" (OPS).

Re:Showing their cards at last

[slim]

It might be beside your point, but it's my point precisely.

To paraphrase myself, replacing "datacentre" with your phrase "other people's servers":

"Other peoples servers" is not always a cloud, because a cloud has properties that "other people's servers" don't always have.

My web site runs on my ISP's server: OPS. But my ISP's hosting is not in a cloud.

Saying "Cloud is just a fancy way of saying OPS" is along the lines of saying "Oak is just a fancy way of saying tree".

Re:Showing their cards at last

[trevorrowe]

Saying "Cloud is just a fancy way of saying OPS" is along the lines of saying "Oak is just a fancy way of saying tree".

Yup, you are 100% correct, "other people's servers" is a nebulous, non-accurate, gaseous term that doesn't quite hit the nail on the head (just like "clould computing"). On the other hand, it doesn't make me feel like a pointy-haired boss when I use it in conversation with others.

Who wants to play buzz-word bingo?

Re:Showing their cards at last

[dissy]

There is a pretty big difference however.

In your companies case, you are not paying for what you use, you are paying for everything, in a lot of cases up front instead of over time.

You need to buy servers, their resources (memory, storage, backup media, interconnect system, etc), as well as the datacenter itself (power, cooling, arrangement, management, staff maintenance, bandwidth, etc), and then once everything is up and running functionally, you still have paid for all of those things like servers storage bandwidth and power, no matter if you are using 100% or 1% of your system.

In amazons case, you don't. You pay for what you use, as you use it, no more no less.
You don't need to pay upfront costs for servers, the infrastructure to support them, and the people to run them. You DO pay for those things, but only a very tiny percentage of, which happens to be the percentage of their resources or skills you use.

At least for the moment, it is much cheaper to buy these resources from amazon, than to pay to build up a datacenter to start with 1 or 2 machines, but be able to scale up to millions. That would have such a huge up front cost that it is not even an option for most small businesses.

There will always be situations where the obvious answer is doing it yourself. This will never change.
That does not exclude the fact there are other situations where using cloud time sharing is the better answer.

Re:Showing their cards at last

[MaerD]

No, he's saying the cloud is a a pie.

Re:Showing their cards at last

[TooMuchToDo]

What you don't mention is that you pay a premium for using only what you need instead of building out your own infrastructure. In some cases, the premium is upwards of 100% (have had to run the numbers for several clients, for some it works out well, for some it's grossly more expensive).

Re:Showing their cards at last

[mwvdlee]

a cloud has properties that most datacentres do not.

Like what?

Re:Showing their cards at last

[slim]

What you don't mention is that you pay a premium for using only what you need instead of building out your own infrastructure. In some cases, the premium is upwards of 100% (have had to run the numbers for several clients, for some it works out well, for some it's grossly more expensive).

I think that's fair. It would be pretty amazing if it worked out cheaper in all cases, and everyone should run the numbers and evaluate the benefits before going into it.

I know Smugmug.com believe they're $500K by using S3 instead of their own storage servers. http://blogs.smugmug.com/don/2006/11/10/amazon-s3-show-me-the-money/

But, there are plenty of scenarios where due to predictable loads (or simply low loads), or simple requirements Amazon's pricing model is a bad fit.

I would be most tempted by Amazon's model if I was starting up a service, was hoping for huge sudden growth, but didn't have the confidence to invest upfront in my own hardware for that capacity.

Re:Showing their cards at last

[TooMuchToDo]

I would be most tempted by Amazon's model if I was starting up a service, was hoping for huge sudden growth, but didn't have the confidence to invest upfront in my own hardware for that capacity.

This is the best example in which you'd want to use Amazon's cloud computing services.

Re:Showing their cards at last

[slim]

a cloud has properties that most datacentres do not.

Like what?

Like, a single node failing is routine, routed around in software, and not considered a problem. Yes, in a traditional datacentre you'd have dual or treble redundant servers, but if one goes down in the middle of the night it's a crisis and an operator's pager goes off. Not in a cloud.

Like: Bringing up a new VM, or hundreds of new VMs, is something you can do on a whim. Yes, newer VM-oriented datacentres have the technology to do this, but because of the way they're managed and financed, usually you have to go through a time consuming approval/requisitioning process to even add one VM.

Like: Dynamic scaling and location. For example, with S3, if your store is getting a lot of hits, you'll benefit from Akamai-like caching wherever in the world Amazon has a presence.

And more. Anything you can imagine that comes from using a small fraction of a really huge pool of computing resources, spread across the planet.

Re:Showing their cards at last

[petermgreen]

At least for the moment, it is much cheaper to buy these resources from amazon, than to pay to build up a datacenter to start with 1 or 2 machines, but be able to scale up to millions. That would have such a huge up front cost that it is not even an option for most small businesses.
The compromise solution is to co-locate your hardware. Afaict provided your needs are gradually and predictably increasing this is cheaper than buying your CPU time on demand from a cloud provider and there is a relatively easy upgrade path from servers in a shared rack to racks to cages and so on.

Personally I think things like S3 are mostly useful as a contingency plan for adding a lot of capacity in a hurry while you work out

Re:Showing their cards at last

[petermgreen]

It's not very clear from their post but they seem to be comparing the purchase cost of their own hard drives and the enclosures to go with them (plus the cost of hosting them for a year but that is small compared to what they are spending on the drives) to the rental cost of S3 storage.

While I can see that makes some sense for a company in a fast growth phase when they stop growing those S3 bills are going to keep on coming while the hard drives and thier enclosures will keep on going for years. Also while he called "xserve raid enclosures" dirt cheap the ones I see listed on apples site are anything but cheap.

Re:Showing their cards at last

[slim]

While I can see that makes some sense for a company in a fast growth phase when they stop growing those S3 bills are going to keep on coming while the hard drives and thier enclosures will keep on going for years.

In the specific case of Smugmug (an photo sharing site), their storage needs will never stop growing (as long as users don't abandon them). When their user base stops growing, they'll still be uploading images every day, and those images don't get deleted. I daresay as time goes by, the average size of each file will increase (higher res cameras; more consumer bandwidth; video).

But you're right, many applications will hit a ceiling on their storage needs. Their owners, as Smugmug did, should do the sums and decide what's best value for them. Amazon et al should be manipulating their price structure to compete with owning your own disks.

Re:Showing their cards at last

[slim]

I should add... one would expect Amazon to reduce their per-byte prices year on year, in line with falling costs.

Big question: can the industry adopt standards to make migration between services easy, such that customers aren't locked in to one provider, and competing providers can push each others' prices down?

Re:Showing their cards at last

[mr.dreadful]

pie in the sky?

Re:Showing their cards at last

[cmdotter]

Being up in the clouds is a wonderful happy place that I go to whenever I'm feeling down. Shame on you for speaking in a less than blissfully unaware state.

Re:Showing their cards at last

[turbidostato]

"Personally I think things like S3 are mostly useful as a contingency plan for adding a lot of capacity in a hurry while you work out"

Yes. Or a case for a startup bussiness you still don't know how well it will make out: reducing you up front costs reduces your risks.

Re:Showing their cards at last

[TooMuchToDo]

SmugMug took into account that if they shelled out for the disks themselves, they were at a disadvantage from a tax perspective, as you have to depreciate the disks over a period of time. With Amazon, they can write off the S3 usage as an operating expense immediately and let Amazon deal with the tax issue of the capital expenditure.

A Little Disappointed

[Comatose51]

I was a little disappointed that MySQL was the only choice offered. I was hoping for Postgres to be offered along side. It's strange to me that most ISPs/hosting companies still don't offer Postgres. MySQL is prevalent but its future is a bit shaky at the moment. Postgres is open source and offers some great features.

Re:A Little Disappointed

[skgrey]

For a first step, MySQL was the obvious choice and it shows a move in the right direction.

I do enjoy how everyone is trying to beat down Cloud Computing. It's basically a new technology, and just like every other new technology there are going to be bugs and issues that affect SLA right away. If you are putting all your eggs in the Cloud basket, it's the same as using that brand new bleeding-edge Cisco product or virtualization platform. You have to expect some pain until they hone the "technology".

Sure there's no good overall definition, and it's become kind of a joke in certain circles, but there are some solid ideas behind clouds. Sometimes I think it's because engineers are more worried about their jobs five years from now, because if clouds do catch on their jobs will be in jeopardy.

Re:A Little Disappointed

[Fotograf]

well i am not IT pro or something, but what exactly is "new" on this cloud? I mean i have been clouding my websites, databases and file storage for years, they are called ISP. My provides, virtual or physical PC, SMB host, or full windows host, managed. Storage as much as one can pay for business and home users...?

Re:A Little Disappointed

[maxume]

In the context you speak of, the new thing is the billing.

(The ability to use automated systems to quickly add and remove virtual machines is also an advancement from traditional virtual hosting)

Re:A Little Disappointed

[dingen]

well i am not IT pro or something, but what exactly is "new" on this cloud?

The fact that you pay only for what you actually use and the services scales automatically to fit your needs.

Re:A Little Disappointed

[slim]

If you're buying a physical server from your ISP, that's not a cloud. If you're buying a virtual server, is it hosted dynamically across hundreds or thousands of physical machines? If not, that's not a cloud.

Now, this probably doesn't matter to you. What you actually care about is price, performance, capacity, availability, resilience, flexibility etc.

Many believe that running a cloud is the easiest, cheapest way to sell fast, reliable hosting services, which can be commissioned and decommissioned in a very flexible manner. You can buy a VM from Amazon in seconds, and have it running instantly. You can close it down and stop paying just as fast.

One open question is, should the marketing use the buzzword? You don't actually care that it's in a cloud. You just care about its cost and features. But then again, being told it's in a cloud gives you clues about its features.

Re:A Little Disappointed

[Fotograf]

question is, does it support basic needs (like protected encrypted storage, web and SFTP up and downloads), DVD send-ins uploads, VPN...? from their website it looks rather simple and need another service provider to build a solution over it.

Re:A Little Disappointed

[Rogerborg]

And that's different from Software As A Service how?

Re:A Little Disappointed

[slim]

Well, I was talking about generalities - what "cloud computing" means.

In the case of Amazon EC2, you get a virtual Linux server, on which you can run whatever services you like - VPN, SSH, WWW, whatever. It's simple, in so far as it gives you pretty much total flexibility.

I'm not sure about sending them DVD uploads. If they don't, a third party with lots of bandwidth could offer that service.

Re:A Little Disappointed

[Rogerborg]

Pretend that I don't work in marketing, and thus don't enjoy the frisson of hearing new terms for old rope. If one provider offers me "cloud computing" and the other offers "software as a service", what does that tell me about the likely functional differences in their offerings?

Re:A Little Disappointed

[dkf]

And that's different from Software As A Service how?

It isn't different. SaaS is one of the ways of delivering things in a cloudy way (particularly when selling direct to end users) but there's also I(nfrastructure)aaS (where Amazon's strength has been for a good while now) and P(latform)aaS, which is where a good number of companies are getting excited (new ways to lock customers in, I suppose...)

If you find it surprising that businesses and media are getting excited over a rebrand of what was there before, you've not been watching this industry for nearly long enough. It happens again and again, over and over. The trick is to spot what will be the next hot BS buzzword ahead of time (and to remember that sometimes the underlying technology can actually be worthwhile).

Re:A Little Disappointed

[slim]

what does that tell me about the likely functional differences in their offerings?

Probably very little. It will give you clues about what price/performance/reliability to expect.

If you were buying a business-critical web hosting service, you'd likely be asking the company questions about their setup. Do you have RAID? What's your disaster recovery procedure? How do you achieve high-availability on the database server? How long can you run on UPS? What's your SLA?

"Cloud" is a shorthand answer to many of these questions. The fine details are there for the reading, too.

Re:A Little Disappointed

[tcopeland]

> It's strange to me that most ISPs/hosting
> companies still don't offer Postgres.

Heroku offers Rails application hosting on PostgreSQL only. 38K apps and growing... their setup is very slick.

Then again, I'm a big fan of Rails on PostgreSQL.

Re:A Little Disappointed

[dingen]

I always though "software as a service" as a design principle for software development and "cloud computing" as a form of hosting.

So you could, for example, create software as a service by using cloud computing. You could also choose another form of hosting for your SaaS. Or provide something else using cloud computing.

Re:A Little Disappointed

[Rogerborg]

You're among friends here: it's OK to say "I have no idea".

Re:A Little Disappointed

[slim]

I'll try again then.

If one provider offers me "cloud computing" and the other offers "software as a service", what does that tell me about the likely functional differences in their offerings?

Firstly, the provider that offers you "software as a service" has not told you that his SaaS offering isn't cloud hosted. That's an implementation detail.

The provider that offers you "cloud computing" has given you a bit of extra information, about the infrastructure they plan to use to offer you your service.

Your end users needn't know or care whether they're interacting with a physical server, a virtual server in a cloud, or a man in a box.

But being told it's a cloud implementation gives you clues that it'll be cheap, that it'll be fault-tolerant, that (assuming you program for it), adding or removing capacity will be instantly available.

Re:A Little Disappointed

[dingen]

Flickr, Facebook, Wikipedia, Google, Nokia and YouTube seem to disagree.

Re:A Little Disappointed

[dingen]

My question for you is, why don't you just give up and use what the entire world is using

Mr Ballmer? Is that you?

Re:A Little Disappointed

[raylu]

Your question actually doesn't make much sense.

What software are they offering as a service?

Re:A Little Disappointed

[snaz555]

With cloud computing you lease virtual servers to host YOUR software. It doesn't have to be web services, you could run DNS or host NFS servers, or your own custom Unix daemons (or WIndows Services) to do whatever you want. If you're a developer you can use it to host svn, trac, and build server. Unlike a colo it's a virtual machine instance. In a cloud you install all your stuff on an instance, then take a snapshot. You can then automatically spin up additional instances in response to load, according to your metrics of load. So if you build the right tools you could have a build server that scales dynamically - more checkins could spin up more instances to run more builds in parallel. When you lease storage you can have it shared across your instances.

Clearly, with EC2, one of the most installed software is mysql. You run it in one or more instances and put the db on S3. With Amazon offering mysql as SaaS you no longer have to deal with provisioning and sizing - it adapts to your usage. I think it sounds like a brilliant time saver and my company will take a close look at it for sure. (We run a lot of backend infrastructure on EC3.)

With SaaS you lease services not servers. Like collaboration tools, web hosting, etc. You don't know what the provider runs the software on - and don't care. You pay for the use of their software, not your own. I get the impression Amazon didn't want to get involved with SaaS for various reasons previously. (There are others who sell SaaS that's hosted on EC2, perhaps Amazon didn't want to compete with their own customers.)

Re:A Little Disappointed

[TheRaven64]

Remind me, which of those companies is hosting anything on MySQL that is actually important (i.e. people would care if it's lost)?

Re:A Little Disappointed

[dingen]

It doesn't really matter what they use it for, now does it? The fact that they choose to use MySQL at all shows they put an amount of faith into it. You don't store data in a database because you want to lose it, right?

But, to answer your question, a quick Google learns Facebook, YouTube and Wikipedia store all of their important data in MySQL databases. I know Google doesn't use MySQL for searches, but they do store other stuff. I'm not sure what Nokia does, but they do seem to like MySQL a lot.

Re:A Little Disappointed

[TheRaven64]

People usually use MySQL because it is fast for simple queries. Wikipedia uses it because they can run a lot of queries against it, and if it loses the occasional set of edits no one is really going to care. Same with YouTube - who cares if a few comments get lost? I tried to read the link about Google, but it was a page of sentence fragments with no overall coherency, so I've no idea what it was trying to say, but last time I visited Google they had posters up everywhere encouraging their employees to use BigTable for all of their storage needs.

Re:A Little Disappointed

[Blakey Rat]

I'm sure this has been in the planning long before the Oracle business was announced. Amazon can't predict the future any better than anyone else.

Re:A Little Disappointed

[jcnnghm]

Very true. Hourly billing and the ability to quickly provision systems is what makes these services. For our newest application, we only purchased enough equipment to handle the application base load. Our application then monitors the acceleration of system response times, load, and requests to automatically provision cloud servers. Essentially, we'll transfer messaging servers to the cloud, then internally re-provision to handle the new application loads, depending on what the actual load looks like. When the load falls, we'll transition back.

The benefit of cloud computing is that for a few dollars a month, we can provision a few extra servers for the relatively few hours of peak load. This allows us to reduce our upfront cash outlay, while also allowing us to maximize our server usage.

Re:A Little Disappointed

[dingen]

I find it very hard to believe the folks running Wikipedia, YouTube or Facebook wouldn't care if some of their content would disappear every once in a while, just for the sake of a little speed.

Either there is another great benefit about using MySQL, or it doesn't really lose data all that often, if ever. Either way, it's clearly not as worthless as the GP suggests.

Re:A Little Disappointed

[slim]

Remind me, which of those companies is hosting anything on MySQL that is actually important (i.e. people would care if it's lost)?

Let's take "important" as meaning "of monetary value", just to simplify the question. Facebook's data is valuable for two reasons:

1. The users treasure it. Their goodwill is vital, if Facebook is to keep them coming back to be served adverts
2. It's a treasure trove of minable information about demographics and connections between people's consumer preferences. Facebook makes money by selling that info.

Re:A Little Disappointed

[Anonymous Coward]

Amazon provides "Hardware as a Service". Much more useful (to me) than SaaS.

That seems to me like less vendor lock-in. You can move your software and data to your own servers and datacenter if you are dissatisfied. You just have to pay for the hardware you need at peak load rather than paying for what amounts to average load.

Re:A Little Disappointed

[plague3106]

Sorry, how did "big company uses X" lead to the conclusion "X is reliable?" It could simply mean they've taken steps to account for the fact it loses data, but that doesn't make it reliable.

Re:A Little Disappointed

[ShinyBrowncoat]

Although usage varies, most people (vendors and analysts at least) consider "Software as a Service" to mean you are renting an application (Google docs, Concur expense app, salesforce CRM app), while "Cloud computing" typically means you are renting infrastructure (Amazon EC2/S3) and/or a platform for developing custom apps (Google app engine, Force.com). Sometimes "cloud computing" is used generically to refer to all three: infrastructure, platform, and/or applications running outside your datacenter, managed by a 3rd party vendor.

Re:A Little Disappointed

[Xtifr]

If one provider offers me "cloud computing" and the other offers "software as a service", what does that tell me

Not bloody much. That's like saying (yay, car analogy ahead!) one provider offers "rental car service" and another offers "transportation", what does that tell you? One term is more vague and broad than the other, and they're definitely not synonyms, but the both offerings could be described as "transportation".

Not that I don't think "cloud computing" is an overused, overhyped, ambiguous and frequently misleading term itself. I do. But it's still only a subset of "software as a service".

Re:A Little Disappointed

[fimbulvetr]

They don't support DVDs, but you can do hard drives (USB2/eSATA).

http://aws.amazon.com/importexport/

Re:A Little Disappointed

[dkf]

I get the impression Amazon didn't want to get involved with SaaS for various reasons previously. (There are others who sell SaaS that's hosted on EC2, perhaps Amazon didn't want to compete with their own customers.)

SaaS is much more boutique than the mass market that Amazon aims at. There's lots of money to be made in the area, but it's about a high value-add through deep expertise in the hosted applications.

The people who do need to watch out are the people selling services based on fairly commoditized applications (e.g., MySQL). Those have the potential market scale to make it as not just an end application, but as a platform that other SaaS providers can build on top. That's the point (Platform as a Service) when it makes sense for the Big Boys (Amazon, Google, Yahoo, IBM, etc.) to turn up and commoditize things, which is pretty much the kiss of death for the small scale high-profit-margin way of working.

The fix for the really small guys? Get closer to your customers, Make your living out of taking the mass offerings and tailoring them to particular clients' exact needs. That's an area where there will continue to be plenty of work, as long as you're not dogmatic about it.

A real worry though is that there seems to be little room for medium sized providers. In particular, there's quite possibly not enough of a space to support the growth of a company in this area from a small player to a very big one. Instead, there's this forced vast gulf between the giants and the minnows. I suspect that there will need to be more regulation round this stuff to prevent such an oligarchy ("gigantarchy"?) from getting overpowerful, which doesn't exactly fill me with glee. But it can't be good for the market overall for there to be hardly any organization of middling size involved.

Re:A Little Disappointed

[MarkWatson]

I wouldn't be surprised if PostgreSQL support will be forthcoming.

BTW, I spent 2 hours experimenting with RDS - I'll probably start using it when it is out of beta.

Re:A Little Disappointed

[Kadin2048]

All of them, insofar as they're all relatively successful sites (well, Nokia isn't a website per se, but the others are) and they would be out of business if they started losing users' data. People would care quite a bit if it was lost.

If Flickr dumped a large amount of user data, that would be the end of the line for them -- people would move to another service, unless they could somehow reassure users that it was a complete one-time fluke. But the damage to their reputation would be tremendous. Same with Facebook, although in some ways FB can jerk its users around more, because there are fewer serious alternatives in most people's minds. Wikipedia is all data; that's all it is, a bunch of bits somewhere. Maybe if it disappeared people would rebuild it, but it would take years.

Your question seems to be assuming that if a company isn't doing financial OLTP or something similar, that the data isn't "important." This is stupid, and a quick look across other industries will show you that there is lots of money tied up in industries like entertainment (which Facebook and Flickr basically are), and the consequences of data loss in those industries, while it may not outright kill anyone, may still be dire.

Re:A Little Disappointed

[turbidostato]

"I do enjoy how everyone is trying to beat down Cloud Computing. It's basically a new technology"

They are beating down because, repeat after me, it-is-not-a-new-technology; it is a buzzword.

"Sure there's no good overall definition"

As it's usually the case for buzzwords: they are not technical jargon but marketroid speech for PHB characters, no wonder it's fuzzy.

Of course faster provision, better templating and higher manageability is good news for everybody but it is just an evolutionary trend lasting years now, its last step being bringing virtualization on the x86 platform and it certainly is no news for anyone in the field but just "more and better" of the same.

Do you want cloud?
  * Virtualization: checked
  * System templating: checked
  * Remote monitoring: checked
  * Centralized administration: checked
  * Fast user-managed provision: checked
  * Fast deployment and redeployment: checked
What am I talking about? Amazon? Not: just my three racks server room and that has been the case for years, each year faster, more flexible and capable. Maybe it's because I'm the unknowingly inventor of the cloud... except this has been "best practices" for any sysadmin worth his salt for years.

Re:A Little Disappointed

[turbidostato]

"If one provider offers me "cloud computing" and the other offers "software as a service", what does that tell me about the likely functional differences in their offerings?"

The one providing "cloud computing" is offering you the barebones for your services; the one ofering SaaS is offering you the service.

So generally speaking, the one offering you SaaS will be the client of the one offering him Cloud.

Re:A Little Disappointed

[turbidostato]

"how did "big company uses X" lead to the conclusion "X is reliable?" It could simply mean they've taken steps to account for the fact it loses data, but that doesn't make it reliable."

They are companies that already showed the monetary ability and the acumen not to fail in obvious ways. Since "taking steps" always costs money, such a big company using "X" instead of "Y" indeed leads to the conclusion that "X" is a better value for its bussiness case than "Y".

Re:A Little Disappointed

[skgrey]

Yes, but you don't have the flexibility and the elasticity of resources, billing for your exact usage, and the ability to rapid provision. Though you do have a cost for technicians to maintain and know every aspect of your data center, which is something that's built into the cloud model. Clouds are great for companies that don't want to hire and pay for that expertise, or the totally redundant internal infrastructure. You said it yourself; your network. I'm a systems engineer too and I have "my network" as well, but I can still see where there is some definitive value in a cloud, even more than an ISP or SaaS offering (although SaaS and cloud can go hand in hand). Trust me, I'm worth my salt, and I know I and every other engineer need to understand this technology's good points and bad points. There's definitely something there with the offerings, the economies of scale, and the service levels. See my point?

Re:A Little Disappointed

[jadavis]

Just a tip, about 10 years ago when MySQL started, PostGRES had been abandoned for several years with no maintenance.

That is utterly false. Mailing lists, commits, steady releases, and community activity all around the world over the last 10 years prove otherwise.

I've seen a suspiciously similar post in response to another article, so you obviously intend to misinform (I don't know or care why).

Re:A Little Disappointed

[turbidostato]

"Yes, but you don't have the flexibility and the elasticity of resources"

Does it Amazon? Don't think so (I don't mean Amazon's users but Amazon itself).

"billing for your exact usage"

That's true: in our little company we never found useful billing for that kind of services to our internal users though we could with not too much effort.

"and the ability to rapid provision"

Did you read what I wrote? Three or four years ago "rapid" meant "as soon as your new iron comes and then less than a day more". About two years ago it meant "less than a day". About a year ago it meant "about two hours". Now it's a matter of autoprovision and it takes the time for the template to copy, and surely less than it would take to get it running on Amazon. As I already said, nothing new but a continous improvement of our services.

"Though you do have a cost for technicians to maintain and know every aspect of your data center, which is something that's built into the cloud model."

Do you think Amazon doesn't pay its technicians?

"Clouds are great for companies that don't want to hire and pay for that expertise"

Not. *Outsourced* cloud services are great for companies that don't want to hire and pay for that expertise.

Re-read your post. The only real novelty is about billing: if you outsource this, billing gets more complex though with finer granularity. That I concede, but even if that meant a total revolution in the way people do bussiness it certainly doesn't makes for "a new technology" as stated in the grandparent post. And then, wrapping a new billing ability in fuzzy PHB-oriented terms in order to better market it looks like the very definition of "buzzword" just as I said.

Re:A Little Disappointed

[TheRaven64]

No, my question is assuming that these sites will not suffer financial loss over data loss. How many people abandoned GMail over any of the instances where they lost emails stored on their system? How many people abandoned Wikipedia because they suffered a power failure and half of the MySQL instances didn't recover cleanly, so they lost a few days of edits (and, astonishingly, then commented that they were impressed with MySQL because half of them didn't suffer data loss. Apparently ACID doesn't mean anything to them). How many users of these sites know or care that the sites periodically lose data? All of the sites listed have had data losses reported on Slashdot, but they all seem to be doing well. They are not, you'll note, using MySQL to store things like their customer invoice data, or anything that would cost them money if they lost it.

Re:A Little Disappointed

[plague3106]

No, it doesn't. They could be taking more of a hit to maintain their position, because MySql is supposed to be fast, and that's more important to them than reliable.

Warning Bell

[slim]

I guess the warning bell is, if your business model is to host something simple and obvious on EC2, then resell it, you can expect direct competition - in this case from Amazon themselves.

To be sustainable, you need to add something difficult, or non-obvious, or that fills a niche, or stands out in some other way.

Cloud Avenue could still do OK, if they can make their offering better than Amazon's, by whatever means - a nicer UI, better management tools, better customer support, etc.

Re:Warning Bell

[Attila Dimedici]

I guess the warning bell is, if your business model is to host something simple and obvious on EC2, then resell it, you can expect direct competition - in this case from Amazon themselves.

To be sustainable, you need to add something difficult, or non-obvious, or that fills a niche, or stands out in some other way.

Cloud Avenue could still do OK, if they can make their offering better than Amazon's, by whatever means - a nicer UI, better management tools, better customer support, etc.

If you base your business model on using the services of a bigger company to offer services to your customers, it is just a matter of time until that bigger company decides that they would rather get the money you are making than the money you are paying them. The only exception to that is if the service you are providing is a lot of work on a day to day basis (as opposed to being very difficult to develop, but then it basically runs itself), and is only of interest to a small niche market.

Re:Warning Bell

[MyDixieWrecked]

It really depends what kind of service(s) you're launching on the cloud. If you're building generic infrastructure to cover some area of the market that AWS doesn't cover well or at all, then you may be in for a rude awakening in the future. This doesn't mean that such a service should not be built, it's just that one should realize what kind of risks are involved when developing something like that.

There are plenty of services that build on top of AWS that will probably be safe from competition well into the future. Those include services that are very specific such as Heroku's Rails app hosting, which will actually benefit from additions such as this MySQL instance type and the price cuts of EC2.

Also, when building apps that essentially turn you into a reseller of AWS services, although there may come a time when amazon starts competing directly with you, you've got your app built. If you built it properly, it should not be difficult to re-wire your backend to utilize some other service or build your own cloud infrastructure. If you're big enough and have the necessary capital, it may actually be a cost savings to do such a thing.

Re:Warning Bell

[slim]

If you base your business model on using the services of a bigger company to offer services to your customers, it is just a matter of time until that bigger company decides that they would rather get the money you are making than the money you are paying them.

There's quite a lot of precedent for smaller companies reselling services from larger companies.

IBM used to offer EDI interchange services. A lot of the sales were through industry specific resellers. So company X knows about, say, the insurance industry, and sells EDI services to insurers. Company X has its own helpdesk, and only refers the harder questions to IBM. IBM is very happy with this arrangement. The subs roll in month after month. IBM doesn't need to train anyone in the foibles of the insurance industry. Company Y, meanwhile, is reselling the same service to the automotive industry.

Then there's the thousands of gambling websites that are merely thin rebrandings of the same few underlying sites, which get referral fees. The larger company that writes the software and runs the service is effectively outsourcing consumer marketing, while also protecting themselves from risk.

Re:Warning Bell

[CastrTroy]

Same goes for all those people who develop plugins for VS.Net. Eventually all those plugins, if remotely popular, will become core features of VS.Net, putting you completely out of business. I still don't understand why MS hasn't built a PDF converter into MS Word. If they did, they would probable wipe out half of acrobats user base.

Re:Warning Bell

[Full Metal Jackass]

So, would Cloud Avenue's business model not be threatened if they hosted the databases on their own physical servers?

Re:Warning Bell

[samkass]

I still don't understand why MS hasn't built a PDF converter into MS Word. If they did, they would probable wipe out half of acrobats user base.

I don't understand why they don't have print-to-PDF as a core part of the Windows OS like MacOS. It's awfully nice to be able to print anything and everything from any app directly out to a PDF as a basic feature of the OS.

Re:Warning Bell

[slim]

Excellent point. And by hosting on EC2, at least they haven't spent a fortune on hardware, should Amazon drive them out of the market.

Re:Warning Bell

[TheRaven64]

Because that would encourage people to share PDFs, rather than proprietary format documents, and would make it much easier for people using other software stacks to interoperate with those people. Print to XPS will probably be a standard Windows feature soon enough though...

Re:Warning Bell

[CastrTroy]

Exactly, and in windows Vista, you can print to Microsoft XPS Document Writer, which is basically MS's version of PDF.

Re:Warning Bell

[Kadin2048]

The only explanation I can come up with (now that PDF is an open format) is that Microsoft's hatred of open formats / love of their own proprietary formats has overcome their desire to crush a competitor.

They're willing, in other words, to let Adobe maintain their little Acrobat fiefdom, because in doing so they keep PDF as a sort of second-citizen format that's obnoxious and annoying for most users to view and use. They know how bad the Acrobat toolset is -- huge, bloated, painfully slow to do anything -- and as long as things stay that way, they're happy. It's just one more way to encourage users to send around DOC/DOCX files instead of PDFs.

If the dominant PDF reader was Foxit's (small, lightweight, fast) instead of Adobe's, they might just bite the bullet, admit that PDF is here to stay, and build their own tools into Windows (or more likely, into IE). But they don't seem to have come to that point yet. They still seem to think that they can displace PDF, with a combination of proprietary Office formats and the new XPS crap.

Goodie

[Anonymous Coward]

Do they offer any good databases?

Quadruple Extra Large

[vagabond_gr]

With the two new types, their instance list looks like the McDonalds menu.

I'd like a Quadruple Extra Large with cheese please.

Re:Quadruple Extra Large

[friedo]

Don't forget the Diet Coke.

Re:Quadruple Extra Large

[kchrist]

What do they call that in France?

Re:Quadruple Extra Large

[rgo]

It's called Relationnelle with Cheese.

Not competitive enough

[Richard_at_work]

While not directly comparable, the Azure platform being launched next month by Microsoft includes two relational database options:

1. Small database (1GB)- $9.99/month
2. Large database (10GB) - $99.99/month

Each SQL Azure database is triple redundant automatically, and you do not pay for storage or load balancing. The Amazon model has you paying for the instance ($81 per 31 days for the small instance) plus storage charges and other costs.

Not too impressed at the moment.

Re:Not competitive enough

[raylu]

What?

http://www.microsoft.com/windowsazure/pricing/

# Web Edition: Up to 1 GB relational database = $9.99 / month
# Business Edition: Up to 10 GB relational database = $99.99 / month
# Bandwidth = $0.10 in / $0.15 out / GB

Web Edition Relational Database includes:

* Up to 1 GB of T-SQL based relational database
* Self-managed DB, auto high availability
* Best suited for Web application, Departmental custom apps.

Business Edition DB includes:

* Up to 10 GB of T-SQL based relational database
* Self-managed DB, auto high availability
* Additional features in the future like auto-partition, CLR, fanouts etc.
* Best suited for ISVs packaged LOB apps, Department custom apps

http://aws.amazon.com/rds/

# Small DB Instance: 1.7 GB memory, 1 ECU (1 virtual core with 1 ECU), 64-bit platform.
# Large DB Instance: 7.5 GB memory, 4 ECUs (2 virtual cores with 2 ECUs each), 64-bit platform
# Extra Large DB Instance: 15 GB of memory, 8 ECUs (4 virtual cores with 2 ECUs each), 64-bit platform
# Double Extra Large DB Instance: 34 GB of memory, 13 ECUs (4 virtual cores with 3,25 ECUs each), 64-bit platform
# Quadruple Extra Large DB Instance: 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform

(Price per hour)
Small DB Instance $0.11
Large DB Instance $0.44
Extra Large DB Instance $0.88
Double Extra Large DB Instance $1.55
Quadruple Extra Large DB Instance $3.10

Provisioned Database Storage

For each DB Instance class, Amazon RDS provides you the ability to select from 5 GB to 1 TB of associated storage capacity for your primary data set.

* $0.10 per GB-month of provisioned storage
* $0.10 per 1 million I/O requests

Data Transfer In

* All Data Transfer $0.10 per GB

Data Transfer Out

* First 10 TB per Month $0.17 per GB
* Next 40 TB per Month $0.13 per GB
* Next 100TB per Month $0.11 per GB
* Over 150 TB per Month $0.10 per GB

Data transferred between two Amazon Web Services within the same region (e.g. between Amazon RDS US and Amazon EC2 US) is free of charge.

The minimum on Amazon is 5GB, so let's compare 10GB. For Amazon at 1 month, you're paying $0.10 * 10 = $1 for storage and your $81.84 is about right. Note that this $82.84 is not comparable to the "Web Edition" offering from Microsoft, as that's for 1GB of storage. The "Small DB Instance" offering from Amazon is for an instance, not for storage, which you pay for completely separately.

So this $82.84 figure is really only comparable to Microsoft's "Business Edition" offering at $99.99, both before bandwidth costs. Bandwidth costs apply to Azure too under a different pricing model. The data in cost is exactly the same and the data out cost is $0.02/GB more expensive for Amazon for the first 10 TB and cheaper after that. You do have to pay Amazon an additional $0.10 per 1 million I/O requests, though.

On the other hand, Amazon allows you to buy way more than 10GB of storage, different instances, and

Re:Not competitive enough

[Richard_at_work]

You entirely miss the point that the Azure offering is fully redundant and highly available (each database has two redundant hot copies, and you don't care about managing that or handling the load balancing), while the Amazon offering is .... not, so you triple the costs for the same redundancy.

Re:Not competitive enough

[PCM2]

You sound sarcastic about Microsoft's T-SQL claims. Maybe you're unaware that a lot of people really do use Microsoft SQL Server as a mission-critical database, and that it's really fairly competitive in that market. Most of the complaints I've heard about it have to do with pricing, and less often, being forced to run it on Windows.

Cost

[tibman]

The smallest instance is 11 cents an hour or ~$80 a month. That just seems like a lot to me, atleast for a personal DB. That $80 only gets you a virtual box with "1.7 GB memory, 1 ECU (1 virtual core with 1 ECU), 64-bit platform." with a max of 1 TB storage (also an additional cost). It just doesn't seem worth it, tbh.

I guess if a company is counting hardware costs, payroll, electricity, and stuff like that.. $80 might be a good deal. But i think most people would rather have a normal server hosted for $10-20 a month.

Re:Cost

[dingen]

The smallest instance is 11 cents an hour or ~$80 a month.

That's assuming the database is used every hour of every day. For a website that is only accessed occassionally, you pay a lot less of course.

Re:Cost

[slim]

If it's a personal DB, you'd probably not want to run it 24/7. Remember Amazon VMs are trivial to bring up and down, and you only pay while they're up.

If you're thinking about the backend for a personal Wordpress page (etc) this probably isn't the right platform.

Re:Cost

[JPDeckers]

No, you don't.

You pay on run-time, not CPU time consumed, so pay 0.11*24*31 for a month, regardless of usage.

Unless ofcourse you have a script that fires up an instance on the moment your website is accessed, and shuts it down afterward, but that might be sub-optimal in responsetime :)

Re:Cost

[Richard_at_work]

Nope, thats the instance cost - the instance is up all the time. The storage and transfer costs depend on traffic.

Re:Cost

[Savior_on_a_Stick]

It all depends on what sort of site you're talking about.

I have several apps at work that are only used maybe 3 or 4 days a month.

They could be cloud based and called from an internal portal as needed.

That would make it 0.11*8*6 for me.

$5.28 per month.

To get an annoying app up out of my way.

Gotta think about that

Re:Cost

[dingen]

Well... that's just stupid. What's the benefit of running your database in a cloud if you don't pay for just how much you use it? Isn't that the whole point to begin with?

Re:Cost

[Richard_at_work]

No, the point of running it in the cloud is to remove the question of the infrastructure. Amazons offering isn't really 'cloud' imho, its more hosted infrastructure but you still care about the infrastructure. Note how you still need to build in redundancy and availability yourself with the Amazon offering, which means multiple instances and other systems to handle the load balancing/clustering/whatever - a true cloud option will 'just be' fully redundant and highly available.

Re:Cost

[slim]

All databases are for web sites now?

Re:Cost

[QuestionsNotAnswers]

a script that fires up an instance on the moment your website is accessed, and shuts it down afterward

Except: "Each partial instance-hour consumed will be billed as a full hour." - from http://aws.amazon.com/ec2/#pricing

Optimization

[dingen]

I wonder if programming for cloud services will bring back the need for code that is optimized for speed (or using as little resources as possible), since you pay for the actual usage of these resources.

Re:Optimization

[slim]

I wonder if programming for cloud services will bring back the need for code that is optimized for speed (or using as little resources as possible), since you pay for the actual usage of these resources.

In server land, if you've got lots of clients, you were always paying for those resources. You max out a server, you have to buy another, or code something more efficient.

Usually, the cost of more computer resource is vastly lower than the cost of a programmer doing optimisation. Jeff Atwood has written frequently on the subject.

Re:Optimization

[dingen]

Is that still true for cloud computing? Because you don't just "get a new server" when your code is a bit bloated. Instead, you pay too much every single day your service is online. This could really add up over time.

Re:Optimization

[slim]

For non-cloud computing, you pay too much every single day, until you reach optimum usage level. Then you exceed the optimum usage level, and have to buy another server, and pay too much again. So it's a series of server-sized steps, approximating a curve.

If you were paying by the timeslice, the cloud equivalent would show a smooth curve, matching the growth in usage.

OTOH with EC2 you pay by the hour of uptime, rather than by processor usage, so CPU usage isn't of the essence for many applications.

You might well optimise to minimise the stuff you really are paying for. Web designers already try to minimise download bandwidth. You might also strive to compress data before storing it -- but as always, it's a tradeoff between how much it saves, and how much it costs to do.

Re:Optimization

[dingen]

OTOH with EC2 you pay by the hour of uptime, rather than by processor usage, so CPU usage isn't of the essence for many applications.

Ah, right. It seems I understood incorrectly how Amazon's service works.

Re:Optimization

[ELitwin]

Since when did code optimized for speed go out of fashion?

Re:Optimization

[slim]

Since when did code optimized for speed go out of fashion?

Since 1974, when Donald Knuth claimed "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.103.6084&rep=rep1&type=pdf

Of course there's a balance to be made - don't write stuff that's blatantly inefficient. But neither should you spend a week optimising a routine, when a $1000 worth of hardware removes the problem.

Re:Optimization

[thinkloop]

Usually, the cost of more computer resource is vastly lower than the cost of a programmer doing optimisation. Jeff Atwood has written frequently on the subject.

That's not necessarily true in the cloud. Consider a site that processes 100 requests per second, and on every request the site needs the same 100 row recordset. If you had a traditional, fixed-cost, non-cloud environment, and the site was performing nicely, it wouldn't matter whether that recordset was being pulled from the db on every request or from cache or wherever.

In a cloud environment like Amazon's, however, you are charged for all transfer and requests in and out of the db. So even if your site is fast, there may be a very compelling monetary argument to optimize that process by having it come from ram rather than hit the database.

At 100 requests/second and if the data were only updated monthly, you could be paying many orders of magnitude more money by not spending that tiny bit of developer time.

--

The cloud is nothing more than a datacenter, only as much as twitter is nothing more than updating your finger file

Re:I did some maths

[vagabond_gr]

If you run your site on a single server then it's much smaller than slashdot, no matter how many cores or ram you have. Also, it means that your site is down much more often than it should. If you want a serious infrastructure with redundancy, EC2 is a quite cheap solution, with many advantages in terms of maintenance and scaling.

Re:I did some maths

[dingen]

Are you taking into account what the 8core xeon server with 16 GB RAM costed you and what it will cost in the future to replace it?

Re:I did some maths

[slim]

This surprises me. Even if you go for "Extra Large" at 80c/hour that would only account for $584/month. And it's much cheaper if you go for a reserved instance.

So you must be using $1000/month worth of bandwidth and storage: wow.

If you've done your sums right, though, I'd take it as a sign that you've got a fairly unique set of requirements, that are a bad fit for the Amazon billing model.

What happens when your Xeam server with 16GB of RAM develops a hardware fault, incidentally?

Re:I did some maths

[ionix5891]

Are you taking into account what the 8core xeon server with 16 GB RAM costed you and what it will cost in the future to replace it?

its rented, no upfront costs, no setup costs

and it runs an nginx web server and php-fpm instances running the site as well as the DB

cpu: E5410 8x @ 2.33GHz
ram: 16GB DDR2-667 ECC Registered
disk: 32GB SSD (OCZ-VERTEX)
bandwidth: 10TB/month

google analytics visitors/month: 7,026,784
google analytics pageviews/month: 27,389,317

mysql queries/month: 2,149,784,446

alexa ~1000 (/. is ~1100)

I'm not sure what's so compelling

[HangingChad]

It rids the customers of any need for time consuming database administration tasks.

I'm sorry but administering a db just isn't that difficult or time-consuming. It takes a certain level of technical knowledge to write good SQL. If you can do that, usually you have enough skill to handle the little bit of maintenance MySQL requires. This isn't like running an Exchange or SQL Server with a ton of overhead, licensing fees, and required add-ons. You can scale MySQL for the cost of hardware. I'm not seeing a compelling reason to let Amazon run my databases.

And then there's no question of who owns the data, who has access to it, and what happens to your data if you can't pay the hosting bill? If your application or web site is so wildly successful that you have to manage failover and load balancing, then you can afford to hire people to solve those happy problems.

Re:I'm not sure what's so compelling

[Blakey Rat]

You could just not use it, you realize. Or is an Amazon rep in your office with a gun to your head?

I'm sorry but administering a db just isn't that difficult or time-consuming. It takes a certain level of technical knowledge to write good SQL. If you can do that, usually you have enough skill to handle the little bit of maintenance MySQL requires.

Then...

This isn't like running an Exchange or SQL Server with a ton of overhead, licensing fees, and required add-ons.

WTF, man? Administering MS SQL Server is tons easier than MySQL. You've got this exactly backwards.

Re:I'm not sure what's so compelling

[slim]

I'm sorry but administering a db just isn't that difficult or time-consuming.

It's quite a lot more difficult and time-consuming than not administering a DB.

• Not monitoring security patch announcements
• Not feeding and watering a backup system
• Not needing to procure storage hardware in advance of requirements

... and so on.

Re:I did some maths

[ionix5891]

see my reply before yours:

http://aws.amazon.com/rds/#pricing

Extra Large DB Instance 15 GB 8 ECUs $0.88 USD / hour

thats $642.4 / month

2,149,784,446 queries/ month @ $0.10 per 1 million I/O requests

thats another $214.9

data transfer is about 5-10mbit month between database and php/server @ First 10 TB per Month $0.17 per GB

thats another $800 or so

this amazon thing is an absolute ripoff

and so far i have had no downtime this year, compare that to the much publicized amazon downtimes ;)

Calling all attorneys

[cellurl]

There needs to be a 5 year "promise" of service by Amazon. I for one wouldn't move my junk to them without a 5 year "community pledge" or something. When a big company starts providing everything, it scares me. The only way I would sleep at night using them is if/when my servers were burning up.

wikispeedia

Re:I did some maths

[ionix5891]

actually i had 0 downtime this year

i dont see how amazon can justify and order of magnitude difference in costs

What is new about "cloud computing"

[bradley13]

IT develops in a spiral, with old ideas being re-introduced in new and better ways every few years. Sure, remote hosting has existed a long time, and virtualization was invented more than 40 years ago.

So what is new about cloud computing? The idea that a virtualized guest can run on any server, anywhere in the cloud. If you boot up an EC2 instance, you neither know nor care what the underlying hardware is, or whether it is in California or Timbuktu. In fact, one day your instance may be in one data center, and another day somewhere else entirely. With live migration, it is even possible for an instance to move from one host to another while running.

This degree of dynamic resource allocation is entirely new. It is made possible by (a) some pretty snazzy virtualization technology (Xen & co), plus (b) the hardware support (virtualization extensions) built into Intel and AMD processors since 2006.

Re:I did some maths

[Anonymous Coward]

data transfer is about 5-10mbit month between database and php/server @ First 10 TB per Month $0.17 per GB

It's $0.00 (per GB) if it's within the same availability zone, and $0.01 (per GB) between zones. If you're using AWS for the database, you should probably be using it for the php/server too, and you can control the zone your instances are launched in, so you can get the $0.00.

Not sure if the rest is accurate, but I (hopefully) just cut your bill in half.

Why the confusion?

[Maudib]

I am not sure why people are so confused about what cloud computing means in this context. It is pretty straightforward-

(1) Yes, the underlying technology is "just" a data-center that you could provision through standard channels.
(2) Yes, it is "just" a normal MySQL server that you could manage and scale through normal means.

Now take those above functions, and put them behind an API that we can call into from our software. Could you manage the same things directly? Of course! However there are use cases where being able to control these functions through is very desirable.

Now take a bunch of other infrastructure resources and put control of them all behind APIs too. One ends up with a very different thing then traditional hosting. You can't provision 100x servers/databases/hadoop nodes for a single hour or night at a traditional host based on some event your software manages, and then pay less then $100. Sure the underlying tools are the same, and there are many traditional use cases where AWS is actually more expensive. However there are an equal number of situations where the reverse is also true.

As for who owns the data, thats just FUD resulting from an unfortunate overlap in terms with things like Facebook. The AWS TOS and contract is quite clear on who owns the data. Just like any other data center, if you don't secure/encrypt your stuff it is possible for the host to look into it, but this is no more likely in AWS then at Rack Space or Data Pipe.

Re:Why the confusion?

[teknopurge]

You can't provision 100x servers/databases/hadoop nodes for a single hour or night at a traditional host based on some event your software manages, and then pay less then $100.

You can't do this with the cloud either. Show me what a 100 CPU instance costs for an hour, each with 2GB of RAM and a 5 GB disk.(bare bones) Come back when I can get a single instance with 1024 CPUs and 50 TB of addressable RAM. I want my one instance to scale automatically - because that's what the cloud promises: automatic scalability.

Re:Why the confusion?

[Maudib]

Sure you can. I have applications online that scale up and down automatically multiple times a day. Some scaling is scheduled, some is based on load.

Though your terminology is a little wrong. Its not a single "instance", it is N instances. Scaling up and down is based on adding or removing an instance. It can be done automatically right now using things like Scalr, Cloud watch or right scale. Or it can be implemented from scratch pretty easily too.

100x small ec2 instances will cost you a total of $10/hour. Each has 1.7 GB of memory, 1x core and 160 gb of storage.
100x large ec2 instances will cost you a total of $40/hour. Each has 7.5 GB of memory, 2x Cores and 750gb of storage.

Re:Why the confusion?

[slim]

I think the GP wants to be able to write an application targeted at a single system, and have it magically scale by adding resource.

Sorry, nobody's achieved that. You have to program with clusters in mind. As long as you do that, however, EC2 lets you add and remove machines from the cluster on demand, via API calls.

Google AppEngine loses you some flexibility, but gives you access to an API that takes full advantage of their massive distributed resources.

Re:I did some maths

[ionix5891]

theres 745 Queries per second avg (see http://pastebin.com/m7f6415a1)

there are many background tasks running all the time, crunching alot of data, beside serving pages

most of the data fits nicely into the RAM (database is 12GB, most of that is log data) and the SSD drive helps alot, average loads are 2.5 @ 50% idle

also the application/site is highly tweaked and designed from the ground up

backups are done to another server on local network and to backup server on another continent which is configured to take over if theres hardware failure

OPS

[mcgrew]

"The cloud" is simply Other People's Servers. Not too buzzwordy. Of course, buzzwords' only real use to to make people think you understand things you don't, so since people are starting to understand what "the cloud" is, you can use the new acronym "OPS". If they ask what "OPS" is you can tell them, but they won't ask because they'll be afraid you'll think they're stupid.

Re:I did some maths

[ionix5891]

forgot to mention, the site makes heavy use of javascript/ajax so the pageview figures are deceptive, considering there are alot of asynch requests

It's a screaming deal

[Wee]

I guess if a company is counting hardware costs, payroll, electricity, and stuff like that.. $80 might be a good deal. But i think most people would rather have a normal server hosted for $10-20 a month.

"Might be a good deal"? Are you kidding? It's a raging deal! You get patching, sysadmin, hosting, etc for that $80. You likely even get more in terms of resources than you would on your "normal" $20/month hosted server (which is probably going to be some pokey virtualized instance on a grossly overloaded server some place).

You also get backups and redundancy for that eighty bucks. The PSU blows in that hosted server and you're looking at downtime. You lose a disk and then you're looking at paying one of your employees to re-install everything, reload the DB, test it, etc.

You can do a hell of a lot with what they're giving you. I wouldn't use it for a personal web site or anything, but for a small business who needs a basic DB-backed web site/service, it's quite a deal (especially if they are short on internal IT resources). Given MySQL's popularity in its nice, I'd say the DB choice was appropriate as well.

-B

Re:It's a screaming deal

[mr.dreadful]

I wouldn't use it for a personal web site or anything, but for a small business who needs a basic DB-backed web site/service, it's quite a deal (especially if they are short on internal IT resources).

You wouldn't host your personal stuff on it, but you'd put business stuff there?

Re:It's a screaming deal

[kchrist]

Correct. For personal, non-business use, cheap shared hosting is usually sufficient. But the $10-20/month that costs you will not get you anywhere near the speed, reliability, and everything else you get with a hosted system like this, making it usually not a good choice for business (and I'm talking about a database that powers a web business, not a blog or something for an off-line business).

Re:It's a screaming deal

[Wee]

What kchrist said. It's way overkill for hosting a personal web page or whatever. There are much cheaper options which don't really need the reliability and extensibility and so forth.

-B

Can you guarantee PCI compliance in writing yet...

[ducomputergeek]

I wonder how many sites are accepting and or storing credit card data on the Amazon cloud without knowing they're breaking the terms of their merchant account contracts.

Until Amazon, or any other "cloud" provider can guarantee PCI-Compliance, we can't even consider them. Our current data center guarantees Level-I compliance and we have it in writing.

Re:Can you guarantee PCI compliance in writing yet

[slim]

Until Amazon, or any other "cloud" provider can guarantee PCI-Compliance, we can't even consider them. Our current data center guarantees Level-I compliance and we have it in writing.

It's a valid observation.

If I was to launch a retail web site -- say, hypothetically, ThinkGeek hadn't been invented yet, and I got there first today -- then I'd expect (once I got a Slashvertisment out there) huge numbers of moochers looking at the T-shirt designs but not buying, along with a much smaller number of buyers.

So I would consider hosting images and perhaps the catalogue site on EC2/S3/RDS or some other cloud service - where I can dynamically scale to a slashdotting - and pass buyers to a secure checkout service hosted somewhere else.

Re:I did some maths

[guruevi]

Not necessarily. Amazon's offering as far as the 'cloud' goes is that you pay for a virtual server at a specific data center. Although they do have a level of hardware redundancy at one location, for full redundancy you would have to pay Amazon for 2 instances and tell them to host it at both data centers. That means it's double as expensive. But that would be the same with his setup. He would have to pay for another dedicated server at another hosting company to get redundancy.

But his point stands, if you have legitimate use for a full-blown server it would be much cheaper to host it or buy it from a random hosting company. Amazon only comes in where usage of a single server is on average 1% or less. But if all you care about is serving a webpage, there are shared hosting (even redundant) solutions that are much cheaper than Amazon. I would only recommend Amazon for custom applications where web sessions don't work but that you would want a shared server for and that is something hardly any regular hosting company accepts.

One example where Amazon makes sense would be data crunching - a customer enters some information on a website and behind the curtains, that data has to be crunched. It can't be crunched in real time for whatever reason (it is crunched by a cron job or so) and it would be too expensive to get your own virtual or dedicated server.

Re:I did some maths

[PhillC]

Someone mentioned reserved instances earlier.

If you get the High-CPU Extra Large, and pay for a 12 month reserved instances ($1820), then $0.24/hour ($173), your monthly cost is down to £325. Reserve it for 3 years ($2800) and $0.24/hour and you're down to $251 per month.

It sounds like your website needs to be up 24/7, so when considering EC2 a reserved instance might be a better way to go. If you didn't need your instance 24/7, then just pay for it by the hour.

The reserved instance does take away some of the advantage of scalability, but you have the same issue with your ISP hosted dedicated box anyway.

Re:I did some maths

[Logic and Reason]

Ignore this new Amazon RDS thing; it's not worth the premium if you can set up MySQL yourself. An Extra-Large EC2 instance is only $584/month ($497/month with the just-announced price drop).

Regarding your queries/month calculation, each MySQL query does not necessarily result in disk I/O, especially if your entire database fits in RAM. I would be surprised if you had to spend more than $50/month on this.

I was going to suggest hosting both your web server and MySQL server on EC2, but the bandwidth costs would kill you: $1,700/month for the 10TB you mentioned in another post (assuming it's all outbound). Ouch.

Re:So jaded at people being jaded

[slim]

[applause]

Can I quote you in full, when the next cloud story hits?

clouds

[smoker2]

They are just one announcement away from complete destruction.

That statement sums up the whole "cloud" debate for me.

yes I know it was referring to the start-ups offering services on top of the amazon services. But my point stands.

latency?

[Colin Smith]

15 milliseconds to my customers?

Re:latency?

[slim]

15 milliseconds to my customers?

Perhaps more words on either side would cause your question/observation/whatever to make sense?

Re:latency?

[Colin Smith]

"can they deliver me "

HTH.

 

Re:latency?

[slim]

"Can they deliver me 15ms to my customers?"

Why do I have to use more words asking you to make sense, than you use to hint at some meaning?

You could start by telling us what application you have in mind. Obviously not HTTP. Nobody needs a web page to load in 15ms.

I recommend you don't play Tux Racer over X on an Amazon EC2 instance. That's not what it's designed for.

Re:latency?

[Colin Smith]

Why do I have to use more words asking you to make sense, than you use to hint at some meaning?

Pass.

Obviously not HTTP. Nobody needs a web page to load in 15ms.

Obviously you need to take a fresh look at what the web has become. HTTP is no longer (and hasn't been for years) a method for delivering web pages. HTTP is now a method of delivering remote procedure calls and responses between browser side javascript applications and application servers. It's obvious that the performance of such applications is dependent on the latency between the client and server.

Can Amazon EC2 deliver me 15ms network latency between my application and my customers?

 

Re:latency?

[slim]

Can Amazon EC2 deliver me 15ms network latency between my application and my customers?

I guess it depends where your customers are in relation to the Amazon farm you use.

It would take you less than half an hour and a dollar to find out: provision an EC2 instance, ping it from a typical customer site, then bring it down.

Do follow up with your findings!

I still think it's an odd requirement. I use a lot of Ajax heavy sites, they feel perfectly responsive and I don't have anywhere near a 15ms response time to any of them.

In the realm of applications that really do need low latency, I suspect it would be a mistake to run a game server from an EC2 instance because there are specialist hosting services for that kind of thing. But I'd be interested to hear how it went, if anyone's tried it.

Re:Postgres scales worse

[JSG]

Care to post some details? "Apparently " doesn't even count as personal anecdote and don't forget that anecdote is not a synonym for data.

continuation

[petermgreen]

what you are going to do long term.

I tried it first thing this morning

[MarkWatson]

Copying from my blog (http://markwatson.com/blog/2009/10/i-just-tried-amzons-new-relational.html):

Amazon just released a beta of their Relational Database Service (RDS). You pay by the EC2 instance hour, about the same cost as a plain EC2, but about $0.01/hour more for a small instance, plus some storage costs, and bandwidth costs if you access the database outside of an Amazon availability zone.

RDS MyQL compatible (version 5.1) and is automatically monitored, restarted, and backed up.

Currently, there is no master slave replication, but this is being worked on (RDS beta just started today).

Here are my notes on my first use of RDS:

        * Install the RDS command line tools
        * rds-create-db-instance --db-instance-identifier marktesting123 --allocated-storage 5 --db-instance-class db.m1.small --engine MySQL5.1 --master-username marktesting123 --master-user-password markpasstesting123
        * Wait a few minutes and see if the RDS instance is ready: rds-describe-db-instances
        * Open up ports for external access, if required (note, here I am opening up for world wide access just for this test): rds-authorize-db-security-group-ingress default --cidr-ip 0.0.0.0/0
        * Use a mysql client to connect: mysql -h marktesting123.cyvbi77nio5f.us-east-1.rds.amazonaws.com -u marktesting123 -p
        * create database recipes;
        * in another bash shell: cat recipes.sql | mysql -h marktesting123.cyvbi77nio5f.us-east-1.rds.amazonaws.com recipes -u marktesting123 -p
        * In the mysql client: use the remote RDS hosted database and be happy :-)
        * delete RDS instance (to stop paying for it): rds-delete-db-instance marktestng123 --skip-final-snapshot

Any mysql client libraries should work fine.

Cloud Computing vs. Software-as-a-Service

[DragonWriter]

Pretend that I don't work in marketing, and thus don't enjoy the frisson of hearing new terms for old rope. If one provider offers me "cloud computing" and the other offers "software as a service", what does that tell me about the likely functional differences in their offerings?

SaaS is a model of software delivery and licensing (which predates cloud computing, though since cloud computing has been available, it is mostly deployed using cloud technology) in which software is remotely hosted and access is provided over the web, usually for a fee (which may be by number of users, or by some other measure of usage.) The essential feature of SaaS is that the vendor provides a particular suite of packaged software, and the client pays for access.

Cloud computing is a model of server provisioning in which logical/virtual servers are dynamically provisioned under software control in a manner which abstracts the underlying physical hardware. Except for the software which actually manages server instances, and operating system software for the instances, there may be little or no actual software provided by the vendor -- e.g., the more basic EC2 instances -- with the application software provided by the client. The essential feature of cloud computing is dynamic server provisioning, not who provides the application software.

Cloud computing is frequently used by SaaS vendors, sometimes on their own clouds, sometimes on someone else's cloud (e.g., a third-party SaaS vendor may use Amazon EC2 or Google AppEngine to host their applications, which is why some have dubbed EC2, AppEngine, and similar cloud offerings "Platform-as-a-Service" offerings.)

But cloud computing can also be used within a company's own datacenters for its own internal applications -- again, the defining characteristic is the dynamic provisioning that abstracts physical servers -- and Software-as-a-service can be done without using cloud technology as all. Cloud computing has approximately the same relationship to Software-as-a-service that freeways have to automobiles; they're related and often used together, but very diffferent things.

Re:I did some maths

[ionix5891]

yes really

use with mysqlfs?

[gumbi west]

So could you use this with mysqlfs to do backups?

Re:use with mysqlfs?

[slim]

I guess you *could*. But it's likely not the best/cheapest backup option using Amazon Web Services.

Alternatives:
  - rsync to an EC2 instance - you can even then take snapshots to S3.
  - use one of the many backup utilities that store on S3

Re:I did some maths

[Kadin2048]

Clearly, you've gone to a lot of effort to make sure your application fits in the hardware footprint you have for it. While I applaud that on technical grounds, I question it from a business perspective; particularly in the early stages of a startup, it might be a much better use of resources to work on features or other things that will appeal to customers, rather than on extensive tweaks or ground-up custom coding. That's the sort of thing you can do later, when you have money coming in.

The advantage of EC2 and other services like it is that they let you get a product out in front of customers, hopefully generating revenue, very quickly and with little upfront cost (either in hardware or in development time/effort), which you can scale out linearly as demand requires. Once you have something out there and are generating revenue, there are often better hosting options and lots of ways to improve performance.

There's a lot of risk in producing a highly tweaked, built-from-scratch application before you have users and cash coming in. If the project or company fails, that's a lot of time wasted. Cloud services allow what are sometimes little more than "working proof-of-concept" apps to go live, and then be improved iteratively if they actually make money (or attract more VC cash, as the case sometimes seems to be...).

I'm sure there are a lot of people who aren't fans of that development or business model, but it's pretty close to dominant in the startup world and that's where I see a lot of cloud services being used right now.

Re:Postgres scales worse

[jadavis]

Unfortunately Postgres still scales out very poorly compared to MySQL

...

Sloany cluster degrades very quickly

Where did you get your information?