Amazon Introduces Bidding For EC2 Compute Time

ryanvm alerts us to Amazon's beta announcement this morning for what it is calling Spot Instances, which represent a name-your-own-price way of using the elastic compute service. Here is Amazon's documentation on the feature. "For customers with flexibility in when their applications can run, Spot Instances can significantly lower their Amazon EC2 costs. Additionally, Spot Instances can provide access to large amounts of additional capacity for applications with urgent needs." Customers can use the EC2 API to see recent spot prices.

Comments

Permutation city

[Brian Gordon]

Reminds me of:

and civilisation has accumulated vast amounts of ubiquitous computing power and memory which is distributed internationally and is traded in a public market called the QIPS Exchange (QIPS from MIPS, where the Q is Quadrillions)

Great book. http://en.wikipedia.org/wiki/Permutation_city.

Re:Permutation city

[afidel]

In short scale Quadrillion is only peta, so we are very much there today with the largest computers being able to do quadrillions of floating point operations per second let alone integer. Not sure if worldwide processing power is yet to the yotaflops scale though I suspect we probably aren't far off.

nifty

[Anonymous Coward]

if only i needed more cpu power for something..

Indictment of cloud computing?

[teknopurge]

I wonder if amazon overestimated the uptake of EC2 and needs to have a fire-sale to get _some_ income on the investment. They need to have capacity to service new orders but by default this means that they have un-used resources that are costing them more then the unit bring in.

Re:Indictment of cloud computing?

[afidel]

Considering that the 3 year cost of a Double Extra Large Instance 34.2 GB of memory, 13 EC2 Compute Units (4 virtual cores with 3.25 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform is $36k and that the 3 year TCO for such a machine in my environment today would be about $8k I don't think they are losing money on EC2 unless their utilization is *really* low, but this does allow them to maximize utilization (and profit) as it allows near perfect price discrimination. It also allows them to scale *their* resources for things like cyber monday by bumping these low priority jobs off the cluster and using it to run their own dynamic site.

Re:Indictment of cloud computing?

[fuzzyfuzzyfungus]

I suspect that we'd have to watch the prices pretty closely to find out.

Variable pricing(down to the point where electricity costs make it cheaper to just turn them off and suck up the losses, of course) is completely logical for dealing with any unused capacity, and it can't have cost Amazon all that much to hack this pricing scheme into their existing system.

If there is usually some capacity available, with a small discount in exchange for having no availability guarantee(or even implication), with occasional chunks of cheap time, then it is probably just an expected side effect of the fact that "cloud" demand(like a fair bit of classic datacenter demand) is somewhat bursty. If there is constantly capacity available at 20 cents on the dollar, well, then Amazon would seem to have a bit of a capacity problem...

Re:Indictment of cloud computing?

[enoz]

In order for EC2 capacity to be highly available (I haven't yet heard of people waiting in a queue for hours for an instance), it seems obvious that Amazon must have a large amount of computing power in standby.

This process of auctioning off the extra processing power based on fluctuating capacity seems like a win-win situation for Amazon AND users. Users who want increased processing, but are not time-bound, can get "off-peak" rates. Meanwhile Amazon can make money off the "idle" processors which are still available to be reserved as an EC2 instance.

Re:Indictment of cloud computing?

[NeilO]

No, I don't think so. More likely they already own outrageous capacity to handle the December retail crunch but don't use anything close to that capacity in any other month of the year. So I doubt there's any additional investment in capacity for EC2. EC2 utilizes what Amazon already owns.

If you read Werner's blog entry on this new feature you'll see they reserve the right to interrupt a Spot Instance and essentially restart it later on. You need to make sure whatever you're doing with that instance you can checkpoint and resume. I think that means Amazon is not trying to "fire sale" underutilized resource. More like they're filling in the cracks between larger "full price" instances in order to maximize utilization.

Re:Indictment of cloud computing?

[afidel]

OK, they fire me, who's going to setup, configure, and maintain the cloud VM's? Oh, that's right they still need someone to do that.... There is no magic pixie dust despite what IBM and PHB's would like to think, companies need digital janitor's to feed and care for the complex machines that run the company and that's exactly what I do. If you are smaller than Amazon or Google and you have someone who spends most of their time worried about the physical infrastructure then they are doing something terribly wrong.

Re:Instance creation?

[emcron]

It's not a matter of getting custom quotes -- they just want to know what you're intended use is going to be. The price tiers (I believe) remain the same. It's a basic check to try to limit malicious users who would spawn thousands of instances for spam or other nefarious purposes.

Re:Instance creation?

[Anonymous Coward]

Initially when you get your EC2 account, it is limited to 20 concurrent VMs. You then have to request to have this limit increased, but the same pricing applies (at least up to 200 concurrent VMs)

Having accounts limited to 20 VMs means that a single user can't sign up and start to flood the system. Also they make no guarantees that a request to start up a VM will succeed unless you have a reserved instance, which has a yearly cost attached.

What this spot pricing does is provide a financial incentive for customers to move load to off peak times rather than adding to the load during peak demand. This in turn should flatten out the load on EC2 (depending on how much load can and will be moved to the off peak times).

In the end this is simple supply and demand economics, and a lot of people in academia have been carrying on about this and a "cloud marketplaces" for quite some time...

Re:It's too proprietary

[Alpha830RulZ]

but their ability to exert lock-in in the medium term is pretty weak

Which is related to their strength, for me. I use them precisely because of the short term availability. We ran a test job just today on their service to test the scalability of a routine. Bought 20 2 core machines for 4 hours for I think $64 or so. I'd have to have at least 4 big multi-core servers lying around idle to have that available for a test bed, best case about $12,000 for the bunch (I've priced them in the last month, this is the best choice I found. That's 4 separate servers in a single 2U chassis. I have several boxes from these guys, they are very solid, are 2/3 the price of Dell, and don't need a blade cabinet.)

I wouldn't use EC2 for long term use, but for short term, or speculative use, it's a pretty good deal. I've found them to be pretty stable and easy to use.

Re:Brilliant market for surplus

[Chuck Chunder]

Since an EC2 server not running an EC2 instance is literally burning cash for Amazon

I know the USD has been slipping but surely grid electricity is still a cheaper source of energy then using your own greenback powered generator?

good. Now can I buy mobile bandwidth that way?

[MancunianMaskMan]

seems it would be be a brilliant way to marked mobile data bandwith as well: when the cell is not too busy and neither it the upstream link, they can sell it off cheap and people can facebook all they like (while i read /.), and when the traffic heats up i still have my work email going but the web2.0rhea flow is temporarily halted.

ELASTIC Compute Cloud

[Slashdot Parent]

3 year TCO for such a machine in my environment today would be about $8k

Ahh, but what would the 1 hour TCO for that machine? And how quickly could you get me 100 of them? And what if after a week, I don't need them anymore. Do I have to continue paying for them?

What if I need 1000? But only for a few days? Can you even fit 1000 in your data center?

EC2 is for elastic computing needs. The price will never compare favorably to static computing on a 3-year basis.

By the way, your 8k is very low compared to what you'd have to pay to get the same featureset of Amazon Web Services. What happens if your data center catches fire? How quickly could you get that machine up and running in a new data center, and at what cost? How quickly could you upgrade the storage? Backup online to fault-tolerant storage? Clone that machine?

What if I want to to load testing of my application? Can you get me a full copy of my production environment and let me quit paying for it once my load testing is done? How much would that cost?

What about staging my application before production deployment? Do I have to pay a full year for a server I plan on using for like 100 hours of that year, tops?

Bottom line: There are a many use cases that call for elastic resources. Comparing EC2 with an ordinary server makes no sense, because they are different tools for different jobs.