Amazon EC2 Now More Ready for Application Hosting

For months now, I've been geeked about Amazon's EC2 as a web hosting service. But until today, in my opinion, it wasn't ready for prime time. Now it is, for two reasons. One, you can get static IPs, so if an outward-facing VM goes down you can quickly start another one and point your site's traffic to it without waiting for DNS propagation. And two, you can now separate your VMs into "physically distinct, independent infrastructure" zones, so you can plan to keep your site up if a tornado takes out one NOC. If I were developing a new website I'd host it there; buying or leasing real hardware for a startup seems silly. If you have questions, or especially if you know something about other companies' virtual hosting options, post comments -- let's compare notes.

Comments

IPv6

[rubeng]

Nice, don't suppose there's any chance of IPv6 support - give each instance, running or not, a unique address.

Re:IPv6

[tolan-b]

I suspect only tunneled over IPv4.

What I'm personally waiting for from EC2 is European datacentres, as I have an application that's latency sensitive. :(

Re:IPv6

[nacturation]

What I'm personally waiting for from EC2 is European datacentres, as I have an application that's latency sensitive. :( You can use Amazon's S3 Europe for serving static files from their European datacentre.

Re:IPv6

[tolan-b]

Yeah I need low latency to the server running the app. Hopefully the fact that they've opened a Euro datacentre for S3 is an indication they might do the same for EC2 though.

Re:IPv6

[Karrots]

Each instance is provided a IPv4 address. Its just that they are randomly assigned when the instance is started. This allows you to receive a static IP you can assign to one of your instances.

It's too proprietary

[Animats]

If you're using Amazon for hosting, you can't switch hosting services; their system is too nonstandard. Do you want to be in a position where they can raise prices or cut off your air supply?

Re:It's too proprietary

[nacturation]

If you're using Amazon for hosting, you can't switch hosting services; their system is too nonstandard. Do you want to be in a position where they can raise prices or cut off your air supply? EC2 allows you to setup your own servers on their infrastructure. Ultimately, this is as standard as getting a virtual or dedicated server at any one of thousands of other hosting providers. Switching is as easy as replicating the environment you've created for yourself (which is likely a standard LAMP stack anyways) and then doing a DNS change.

Re:It's too proprietary

[Anonymous Coward]

Forgive me for being uninformed if that is the case, but aren't these simple virtual machines running linux ? It should be no harder to move to a different hosting place than it is to move any hosted linux. I have done "brute force" migrations by literally copying every file from a hosted system (RedHat) to an in-house server, and jumping through hoops with editing fstab and grub.conf and etc until it booted and ran, and then fixing driver issues. I have also done the same from a hosted system into a VMWare image. Of course you could do it "right", if a deadline wasn't breathing down your neck and your business had properly documented the initial setup, then you would take the opportunity to put all the latest versions on the new place, and migrate a module at a time doing what code and config updates were needed.

But seriously, what is the proprietary lock-in in a Linux virtual machine ?

Re:It's too proprietary

[LukeCrawford]

it's pretty much a standard i386/PAE Xen image... I've not tried, but if you take a image of your filesystem, you should be able to move it to another Xen hosting provider that supports i386/PAE. Of course, most competitors don't have Amazons wiz-bang provisioning technology. Uh, not to whore out my own links, but I run a small Xen hosting provider (btw, ec2 kicks my ass when it comes to price per megabyte of ram) - and I (and I assume many of my competitors) provide a read-only rescue image where you can mount your partitions without booting them up, meaning that if you have a dump or tar of your old filesystem, you could move your image fairly easily.

I do think that another host using an automated provisioning system that is compatable with EC2 would be a good thing- If I wasn't absolutely swamped by my dayjob, I'd try to implement such a thing.

Re:It's too proprietary

[dogas]

Your comment makes it apparent that you really don't understand how hosting a website works.

My company uses EC2 (plus a few other amazon services, which I find to be spectacular) for hosting our application. If we wanted to move to another server or company or datacenter, it's just a matter of setting up the new server and repointing the DNS. Also what is nonstandard about their servers? You basically set them up however you want. You want to run linux? cool. FreeBSD? awesome. Basically you can run any *NIX clone you please. Fortunately lots of people provide excellent templates, so rolling your own is not really necessary.

Re:It's too proprietary

[Anonymous Coward]

Yes and no. Since it's not persistent, you have to set up some kind of backup/replication from day one -- S3 being the common choice here. My startup uses EC2+S3, and just getting a Linux image serving a webpage is completely standard (yay), but setting up all the replication and monitoring and whatnot that a real server actually needs is kind of a pain. You end up with a lot of EC2/S3-specific fun, at least on the administration side.

As just one example, we don't do full backups, but rather have our images install everything on startup (which is rare, and fast anyway), and simply hot-sync our RDBMS while it's running. Simple and fast! Then we wanted to install a blog, so we installed Wordpress. Later we discovered that Wordpress uses our database for posts (so it replicated them just fine), but *also* writes to the filesystem for file uploads. So we needed to install the "Wordpress S3" extension, so file uploads got backed-up to S3 too.

Were we stupid to assume that web apps that use an RDBMS use only the RDBMS? Or to use Wordpress? Or to hotsync only and not do full backups of our image? Maybe all of the above. But it seems about equal to me to say "EC2 is standard Xen" as "Perl is cross-platform": it's certainly possible to use it that way. You can also tie yourself down to a platform really easily, if you don't make portability a priority. We don't think EC2/S3 is going away soon, so we took the easy way.

True, we could just move our image somewhere else. It would probably run, mostly. Things like "back up EC2 to S3" we'd need to spend time changing, because I don't think anybody else in the world uses the same interfaces there. That, I assume, is what the GP was speaking of.

Re:It's too proprietary

[TheRaven64]

Uh, not to whore out my own links, but I run a small Xen hosting provider Can you drop me a mail if you start offering FreeBSD domUs? I have a dedicated server running OpenBSD at the moment, but I'd be interested in a backup host.

Re:It's too proprietary

[mr_da3m0n]

>You basically set them up however you want. You want to run linux? cool. FreeBSD? awesome. Basically you can run any *NIX clone you please.

No you don't. You have to run Linux. And they pick the kernel. It runs on Xen after all.

Also, why does everyone seems to ignore the fact that the virtual machines are automatically wiped/reset to base image state whenever they terminate?

While inconvenient, their API is simply fantastic. My EC2 machines boot, add-remove certain components, and then deploy data from S3 on boot.

You just have to build your thing with one thing in mind: EC2 Virtual machines are one shot, disposable machines.

Re:It's too proprietary

[Tephyrnex]

If you use EC2 as your server, aren't you are pretty much stuck using S3 as your data store, unless you want all of your data requests going out over the interweb and back to populate pages? This means you end up coding to S3 instead of a more generic database structure. That sounds fairly proprietary to me.

Also, although the pay as you go pricing structure is good for a business, it's a little pricey for someone to just try out or use for a hobby site. I can't afford $75 / month to set up a server to play with in my spare time over a couple of months...that buys almost a full year with a shared hosting plan.

I have a question:

[Megaweapon]

Is this a Slashvertisement?

Re:I have a question:

[jamie]

Just in case you were serious... :)

Slashdot, and the company that runs it Sourceforge Inc., aren't using Amazon Web Services for anything that I know of. Slashdot runs on real hardware, not VMs, and we're not planning on changing that anytime soon. I don't know anyone using AWS, which is part of why I'm looking for Slashdot reader feedback. My experience with it is limited to starting up some instances and playing around with installing Apache to see how it all works, and I did that on my own nickel. I chatted with someone at Amazon about AWS last year, but I didn't sign an NDA so I learned about today's news through their public mailing list.

Re:I have a question:

[Anonymous Coward]

No, it's MoreReady(patentpending)!

Re:I have a question:

[Anonymous Coward]

That is avoiding the question. The question is if Slashdot is getting some sort of kickback or favor for running this article. Nobody asked what hardware Slashdot runs on. In fact, I suspect that most things that get advertised are not actually used by the people running the advertisement.

Re:I have a question:

[Metaphorically]

Doug Kaye from IT Conversations has been doing some pretty heavy stuff on EC2. He did a podcast with an Amazon guy on Technometria where they got in to a lot of detail have a listen.

Re:I have a question:

[dyefade]

Even if it is an advert (and I suspect it isn't, though I have no proof), it's still an interesting discussion. I've been looking at the AWS line-up, and it'll be interesting to see what this thread throws up in terms of for and against.

Re:I have a question:

[jamie]

That is avoiding the question. The question is if Slashdot is getting some sort of kickback or favor for running this article. The answer is: of course not. We never do that.

Re:I have a question:

[KillerBob]

*grins* Slashvertisements don't necessarily have to be paid to /.... Just a submission coming from somebody who works for the company, if it's interesting enough to get posted, could be called a "Slashvertisement". The Slashdot Effect is well known, especially in IT circles.

Re:I have a question:

[Anonymous Coward]

Good joke! Sure not on company accounts, but the odd money wire some of the slashdot staff gets doesnt count ;)

Re:I have a question:

[sentientbrendan]

>don't know anyone using AWS, which is part of why I'm looking for Slashdot reader feedback.

I smugmug www.smugmug.com uses AWS, or at least S3. This is a pretty big for pay photo sharing site.

Re:I have a question:

[adpowers]

FWIW, Amazon Web Services did host the Seattle Slashdot anniversary party. I'm not suggesting there was any impropriety, however.

Bandwidth Limits/Costs?

[Doc Ruby]

How much bandwidth transfer a month can I get there, and how much does it cost? What's the max sustained bandwidth that I can get from one of their servers?

And if I'm competing with Amazon by running a popular streaming radio station (even paying the required royalties, but of course not to Amazon), will they start shutting me down?

Re:Bandwidth Limits/Costs?

[brunascle]

pricing and bandwidth is oulined here about halfway down the page. and a nifty pricing calculator here.

looks pretty reasonable to me, but i dont really have anything to compare it to. no minimum fee. it's completely based on bandwidth, resources, and usage.

Re:Bandwidth Limits/Costs?

[Doc Ruby]

The calc shows that data transfer costs $0.18:GB out ($0.10:GB in), with no maximum (or minimum) charge. It doesn't show max bandwidth, but I'd expect Amazon to have some fat connectivity, though I'd want a CIR (Committed Information Rate, or guaranteed minimum rate) for any real pro application.

But I can get data transfer (in or out) for $0.05:GB up to 2TB:mo, with root access on an actual dedicated server, not VPS ($0.03:GB for VPS). At a datacenter I've used for a couple years, with good support, >99.99% uptime, never any bandwidth capping, on fast machines with good amounts of storage. With years of experience. GB after the cap are expensive, like $1:GB, but I use geographic diversity already, so I split my loads across multiple servers, pooling their caps across the balanced loads. Transfers within the hosting WAN are free.

So while Amazon looks interesting, I think I'll keep my existing hosting company, which is anywhere from 2-6x as cheap as Amazon's new, relatively untested service, with potential competition from Amazon's own services.

Re:Bandwidth Limits/Costs?

[KevinKnSC]

Um, you forgot to tell us who your existing hosting company is.

Re:Bandwidth Limits/Costs?

[gfilion]

It doesn't show max bandwidth, but I'd expect Amazon to have some fat connectivity, though I'd want a CIR (Committed Information Rate, or guaranteed minimum rate) for any real pro application.

They don't have a CIR, but I remember reading in the docs that they have 250 Mbps per virtual machine.

So while Amazon looks interesting, I think I'll keep my existing hosting company, which is anywhere from 2-6x as cheap as Amazon's new, relatively untested service, with potential competition from Amazon's own services.

It's true that Amazon is more expensive than a dedicated server, but the idea is that it's elastic: you could run 3 servers during peak time (let's says 4 hours per day) then scale down to one the rest of the day. This is cheaper than 3 dedicated servers.

Re:Bandwidth Limits/Costs?

[Doc Ruby]

Well, 250Mbps is (250 * 30.48 * 3600 / 8) 82.296TB:mo max. That would cost $12,439.28 per month at Amazon, and (at $0.05:GB->2TB/$1:GB@2TB+) $79,896 at my "cheaper" host, but only because the punitive per-GB cost of exceeding the 2TB cap at my host. Since I load balance already, the 82.296TB would be split across 41 or 42 $100:mo servers, which would cost $4200 at my host.

Spending that $4200 at Amazon gets 23.3TB, which at my host would cost $1200. That $1200 at Amazon would get 6.7TB, which would cost $400 at my host. $400 buys 2.2TB from Amazon, which is just a little more than what I get for $100 now.

So the 3-4x cost of Amazon buys me instant scaling to the (more expensive) bandwidth tiers, without the lead time to buy and start up a new server. Conversely, my bandwidth has to drop to 1/3-4 at Amazon before I'm spending less there than at my cheaper but oversupplied host.

So if I know far enough in advance to deploy new servers that I'm going to grow more than 3-4x (or drop extra ones if I shrink that much), then my current host is cheaper. But spikes are a different story, because I do have to buy a whole month at least (and there's startup fees, but that's probably matched by all the other fees other than bandwidth at Amazon, which are included in my current host costs cited here). If my bandwidth spike doesn't consume at least 25-33% of the cheaper bandwidth, I'll have bought too much extra bandwidth, and could have saved at Amazon.

It's good to have the options. And maybe some of the customers at my current host will be tempted over to Amazon, because that choice is better for them. Which could spur my host to offer similar spike pricing models, at the current lower price rates. So, GO AMAZON!

Re:Bandwidth Limits/Costs?

[vidarh]

At my last company we were looking at EC2 as a "backup" solution to handle spikes - for that it may be cost effective. But looking at our bandwidth graphs, and the cost differential, spikes of the kind of magnitude where it'd make a difference were incredibly rare. We had maybe one event over 2+ years where it'd made a difference. If you prepare your system for virtualization anyway, you could handle that by bringing up just extra capacity on EC2 and using your cheaper host for normal day to day use.

In fact, if you prepare for that, the cost differential is likely to grow, as if you have EC2 as a fallback you can afford to get closer to full load and bandwidth usage before you add more physical boxes...

EC2 is just too expensive to be worth it for normal use...

Re:Bandwidth Limits/Costs?

[a9db0]

I know you've been asked before, but I'll try again:

Who is your "cheaper" host?

I'm not asking to be a pain, I'm just interested in alternatives.

Re:Bandwidth Limits/Costs?

[Anonymous Coward]

Why would they? You could compete with them just the same using a different hosting provider.

Re:Bandwidth Limits/Costs?

[Doc Ruby]

Because they can compete with me that way. If it drives me away, well, they got my money for a while, and maybe dealt me some blows in shutting me down and making me move around instead of competing.

This is always the general problem with the network vendor also competing in the other layers in which the network customers are competing.

Re:Bandwidth Limits/Costs?

[Doc Ruby]

That is an excellent compromise.

Is there a way to start a minimal account at EC2 which just idles away doing nothing, and a virtualization account somewhere else (like at my cheap, but flat rate per server, provider), and then very quickly clone my running virtual session over the Internet to the EC2, where it quickly starts running to handle the spikes (then shuts down)? With that setup, I could use EC2 as purely generic spike capacity, and just need as much advance warning that a spike is going to max out my cheap provider as far ahead as the time it takes to send the virtualized session over to EC2 and start it up. Now that is some 21st Century load balancing, and really an example of using virtualization where nothing else will really do. "Just in time" virtual cloning for spike handling.

Re:Bandwidth Limits/Costs?

[Doc Ruby]

I avoided mentioning it because it's a competitive advantage I'd prefer not to share with all of Slashdot (and my competitors, who read Slashdot, especially stories like this one).

You can feel free to believe that I'm making it up. I probably would, if I were in your shoes. I'm not making it up, but I'm not going to give it away here. Sorry.

NOC

[joshcornejo]

I think you are confused ... all the NOCs of Amazon could go down and your servers (which are in a Data Centre) will continue to operate. http://en.wikipedia.org/wiki/Network_operations_center

Re:NOC

[jamie]

Ah, I was using the term to mean data center. Didn't realize they were sometimes physically separated. My misunderstanding.

I might move my website over

[Anonymous Coward]

If the prices are good I might go with them. I don't know if you guys know but I invented the roller blade. Someone stole the idea from me and get a patent on it before I could. I would have been rich beyond my wildest dreams. My new website will be for helping people get their ideas patented. If anyone has any information on who stole my roller blade concept, please let me know. Thanks and god bless.

Re:I might move my website over

[Anonymous Coward]

I heard it was Monsieur Petitbled. He must have stolen it around 1819 when he patented the inline skate.
You should have a serious talk with him.

Some more about EC2

[jamie]

So here's a little about what EC2 actually is, for those of you who don't know. You don't have to reply here, start your own comments ;)

The Elastic Compute Cloud was originally designed as a way to host applications that needed lots of CPUs, and the option to expand by adding more CPUs. It's a hosting service that lets you start up virtual machines to run any software you want: they have a wide variety of pre-packaged open-source operating systems you can pick to start up your VMs with.

Starting up a VM takes just a minute or two, and it's point-and-click thanks to the Firefox extension. Each VM comes in one of three sizes: small (webhead), large (database), and extra large (bigass database). They cost respectively $72, $288, and $576 a month (billed by the hour), plus bandwidth ($0.18/GB out, somewhat cheaper for data going in and there's a price break at 10 TB).

One of the concerns everyone raises with hosting on virtual machines is that if a VM instance goes down, you lose everything on it. It comes with hard drive storage (160 GB on the small size), but if something goes wrong, that data's gone.

I think the rejoinder here is that, on real hardware, if something goes wrong, your data's gone. You never set up an enterprise-level website on the assumption that any particular hardware has to survive. Single points of failure are always a mistake, and backups are always a necessity. When any machine explodes - real or virtual - the question is how fast your system recovers to "working well enough" (seconds, hopefully) and then how long it takes you to get it "back to normal" behind the scenes (hours, hopefully). Those answers shouldn't depend on whether there's a physical drive to yank out of a dead physical machine that may or may not retain valid data.

Which brings up what I think is one of the selling points of EC2: free fast bandwidth to S3, Amazon's near-infinite-size, redundantly-replicated data storage platform. That's a nice backup option to have available. That's part of why, if I were starting a new web service, I wouldn't host it on real hardware. I'd like not having to worry about backups, tapes, offsite copies... bleah, let someone else worry about it.

Slashdot hasn't run many stories on EC2 (none that I know of) because until now it's been a niche service. Without a way to guarantee that you can have a static IP, there had been a single point of failure: if your outward-facing VMs all went down, your only recourse was to start up more VMs on new, dynamically-assigned IPs, point your DNS to them, and wait hours for your users' DNS caches to expire. That meant that while it may have been a good service for sites that needed to do massive private computation, it was an unacceptable hosting service.

Now with static IPs, you basically set up your service to have several VMs which provide the outward-facing service (maybe running a webserver, or a reverse proxy for your internal webservers), and you point your public, static IPs at those. If one or more of them goes down, you start up new copies of those VMs and repoint the IPs to them. No DNS changes required.

I know there are other companies offering web hosting through virtual servers. Please share information about them, the more we all know the better.

Re:Some more about EC2

[Anonymous Coward]

There are other options for web hosting through virtual servers. I use Mosso ( http://www.mosso.com/ ) for several of my small sites and have always had 1st rate support from them. The sites I host there are just personal sites but have had zero hosting problems of any kind in the year plus I have had the account. Disclaimer: I have my Mosso account as Rackspace employee company perk so not sure of pricing. However with that being said if for some reason I were not employed at Rackspace any longer I would keep my Mosso account.

Re:Some more about EC2

[Darkforge]

Slashdot hasn't run many stories on EC2 (none that I know of)

There have been a few.

Amazon EC2 Open To All
Amazon and Hardware As a Service
Amazon Betas 'Elastic' Grid Computing Service

Re:Some more about EC2

[jamie]

Adding to Related Links, thanks. Should have done this myself (doh).

Re:Some more about EC2

[sugarmotor]

>> One of the concerns everyone raises with hosting on virtual machines is that if a VM instance goes down,....

I don't quite understand this one. I've heared it before and was puzzled. Do these VM's "go down" more frequently than regular hardware would?

Or is it just the dynamic IP that makes it more problematic?

Thanks --

Stephan

Re:Some more about EC2

[42forty-two42]

It's not that they go down particularly often, but if there is a hardware failure, amazon generally will not take heroic efforts to recover your data - your instance is terminated, hope you had a backup. Sometimes they'll try to reboot it temporarily, but they won't put the drives in another machine or anything like that. So make sure your app can recover from a complete and permanent server failure, with total data loss on that server. Also, previously, there was the IP problem, and also you couldn't ensure your instances were hosted at different locations. With these new changes, you could have a backup instance sitting at an independent datacenter, and move your IP over in a matter of minutes (possibly less; it seems to take longer when you're actively watching how long it takes to move, based on my testing)

Re:Some more about EC2

[Anonymous Coward]

near-infinite

This makes no sense. The definition of infinity makes nearness to it impossible. If you mean "very large" say so; if you mean there are limits that are in practice unreachable then say something like "practically infinite" or "effectively infinite". "Near-infinite" is meaningless.

Re:Some more about EC2

[Anonymous Coward]

You forgot to mention that the system is based on the Xen virtual machine software, the commercial sponsor of which was recently snapped up by Citrix.

AG.

Re:Some more about EC2

[swb]

One of the concerns everyone raises with hosting on virtual machines is that if a VM instance goes down, you lose everything on it.

That has to be an Amazon limitation.

With, say, VMWare ESX, its almost a functional requirement that your data is stored on hardware other than the host CPU (SAN, usually). I also like critical data stored on a raw SAN LUN (as opposed to a virtual disk on a VMFS) since even if the entire VM execution infrastructure tanks, those same SAN LUNs can be accessed by physical machines with SAN access.

VMDKs have some advantages for smaller datasets and obviously for OS boot volumes, but for most large datasets it can get awkward to manage them and I don't think it does much to enhance performance to add the abstraction layer vs. a raw LUN.

But either way, you never have to lose your data just because a VM crashes even if the host crashes.

Re:Some more about EC2

[Stu Charlton]

FWIW, Elastra (who I just started working for) does clustered MySQL and PostgreSQL via EC2, at pennies on the dollar (basically an uplift over Amazon's hourly EC2 costs). It automatically uses S3 to ensure persistence, along with a messaging backbone to detect down instances. We also built our own static dynamic DNS mapper to deal with the static IP issue ;-) With the new zone feature, geographic failover is next....

The main goal is to enable flexibility in designing and configuring an infrastructure stack, by building-in a lot of the "rules" and dependency management "gotchas", and also providing markup languages to describe how new pieces of infrastructure fit in the cloud. The point is to make the "cloud" something a lot more automated than a traditional blade array or even virtualization, which is necessary for a public cloud like EC2 but even would be nice for a private cloud like your own Xen or VWware cluster...

Showing My Age

[Rary]

Okay, I really feel old now. Do "The Kids These Days" really say things like "I've been geeked"? And if so, why? Where the fsck did such a stupid saying come from?

Oh, and get off my lawn.

Re:Showing My Age

[maxume]

Huhhuh, you said fsck.

Re:Showing My Age

[jamie]

If you think I'm one of the "kids these days," you really are old ;)

check out Mosso

[tnhtnh]

I use Mosso - they are inexpensive and are hosted and owned by Rackspace. Therefore the service is fantastic!

Re:check out Mosso

[ibookdb]

Interesting..

Re:check out Mosso

[lb746]

What's also interesting about Mosso is their billing method being based on requests and not on the type of requests/media or demands your applying to the servers. What would cost you $1.89 a month on S3, costs you $100 on Mosso. You can easily max out your monthly request amount on Mosso with 3 small websites, so make sure you look into this factor carefully before considering their services.

A Few Basic Questions

[smackenzie]

The more I learn about Amazon's AWS offerings... the more confused I get. I've read a TON of material, reviewed the APIs, looked at sites built on this platform and have read many blog entries. I feel like I "know" a lot, but understand very little. Someone help?

1. What is a perfect "typical" application for AWS? (And don't answer, "one that needs to scale...". I'm looking for a realworld example.)

2. Anyone here on Slashdot using these services? Nervous about single point of failure? (And I don't mean just technical, but also financial, legal, security, business continuity, etc.)

3. EC2 / S3: is there any value in using just one? I've noticed there are additional services now, too

4. In the days of SOx / PCI / CISP compliance, is it even possible to set up a financial app on AWS?
5. Also, finally, maybe a question to Amazon... why? Someone did the financials recently and it was a fascinating study. The short of it is that at max capacity, the net income from all of AWS for Amazon is so tiny, you have to wonder why they even bothered... [need citation]

A classic case of wanting to like the technology, but not really sure how to use it. Thanks.

Re:A Few Basic Questions

[maxume]

I'm pretty sure that the Amazon store represents a good real world application. Or really, they need similar technology to run the store, so they just have it lying around.

I like the theory that they are mostly running it on their "Christmas capacity" as far as explaining why they are doing it.

Re:A Few Basic Questions

[PsychoKiller]

1) Don't limit your ideas about using EC2 to hosting. You can run whatever you want on their instances. Think about a company that does some kind of data acquisition/processing. You could set up a system for them that does a run in 1 hour (since that's the minimum billing slice) instead of their current process that takes a month on a single workstation (or even cluster of workstations in their office). The results get stored on S3 where they download them over an encrypted connection.

2) Yes, very nervous. Especially with the privacy laws in the States. I'm Canadian, and I would be talking to lawyers about data storage issues before having sending customers' data down South.

3) EC2 is useless without S3, since your images are stored on S3. S3 is useful without EC2, as you can use it for static storage and BitTorrent hosting.

4) See my response to #2.

5) I don't work for Amazon. :P

Re:A Few Basic Questions

[Skidge]

I like the theory that they are mostly running it on their "Christmas capacity" as far as explaining why they are doing it.

If that's the case, what happens when Christmas rolls around?

Re:A Few Basic Questions

[Anonymous Coward]

2. Anyone here on Slashdot using these services?

Yep. I'm using S3. I use a custom app to sync up most of my data to S3 nightly (custom only because at the time I started using it there wasn't anything out there configurable enough for my needs, there might be now). It's actually fairly cool, and has saved me from one machine crash so far. Not only that, but since S3 gives me infinite bandwidth, I can quickly pull down documents in a secure fashion when I'm away from home using a little GUI app I wrote.

Nervous about single point of failure?

Not really nervous, but annoyed. My previous backup strategy involved prayer, so this is an improvement, even if it disappears one day, but still the API isn't perfect. It's gone down from time to time, and there's always the always lovely "We've encountered an internal error" error. The later has reduced in frequency, but still is annoying.

maybe a question to Amazon... why?

As I understand it, this is largely a subset of some of the technologies they use internally, so it's a way to try and monetize their internal infrastructure. It doesn't have to bring in a profit today, but one day it could be quite profitable.

Re:A Few Basic Questions

[imroy]

As far as using S3 on its own - it would make a good store for static content. You have a site, say www.example.com, but have a separate host for static files e.g static.example.com. This has long been common practice - having a simple light-weight web server for static files (style sheets, icons and other images, etc). For S3 you setup a CNAME record in your DNS that points to s3.amazonaws.com and create a 'bucket' in S3 with the hostname (static.example.com). Bingo, cheap and scalable off-site storage for your website. I'm not sure if S3 would be best for small files though. It would probably be better suited to bigger media files like podcasts/vodcasts, FLV/MP4 videos for playing in flash, etc.

Re:A Few Basic Questions

[maxume]

I don't know.

Superficially, it seems like they should be able to set things up so that ec2 increases their hardware utilization without really requiring a whole lot of extra hardware beyond what they are using at peaks.

It could just be that the extra hardware is nearly free compared to what they have to spend maintaining what they need for the store.

Re:A Few Basic Questions

[Anonymous Coward]

I use S3 by itself for off-site on-line backup.

Only pennies per month.
Automated access via s3cmd; http://s3tools.logix.cz/s3cmd
or http://s3sync.net/wiki

Re:A Few Basic Questions

[sofla]

I use S3 to mirror some of the larger files from my website. Costs me like $.07 a month. Your S3 files are accessible via a public facing URL, so any content you have that is client-side would only need S3. Also, any files you post are automatically accessible using BitTorrent as well. Last, it might interest you to know, that Linden Labs used S3 to rollout one of their more recent updates to the Second Life client.

I've been eyeing EC2 for awhile now, for this reason: I get more control of the configuration than with a typical VPS. For me, the appeal here is that I prefer Gentoo Linux to other distros, and it isn't easy to find ISP's that support Gentoo. Also, there is a nice comfort factor in knowing that if the hardware on which the instance is running fails, then it should be a "simple" matter of bringing up another instance to bring the server live again (provided the S3 storage that hosts the VM image didn't die as well...).

As for single point of failure... I don't see Amazon as any less reliable than other ISP's. On the contrary, I think that as a company Z is much more likely to be around than some fly-by-night hosting company. As for why they do it, my understanding is that they're renting out the extra capacity of their own server farm. So its very close to free money for them. Sure they could pull the plug on that, but so could any other ISP.

Re:A Few Basic Questions

[Anonymous Coward]

Here's a simple example that may make more sense.

Say you're an indie 3D animated movie creator. You've been doing your modeling, render some scenes, done some low res proofs, some nice single frames.

But now you want to render the entire movie, in HD. But you're not PIXAR.

So you set up a VM to be a rendering slave, log in to amazon, "hi, I'd like 1000 machines please". Load everything up, render your movie, and you're done.

Amazon is charging by CPU (and bandwidth). The rendering time for a movie is fixed, it's going to take X CPU Hours to render the movie, let's say 24000 hours. (Why yes, I am pulling these numbers out of thin air.) So, you can either spend 3 years rendering that on a single machine, or shop it out to Amazon and have it rendered in a day.

That's a great example of the kinds of tasks the Amazon cloud can do readily. Now you can gene sequence at home without building out the 2 car garage in to a monster data center.

It's also great if you, say, want to demo a web site for a week, but don't have a spare server you can use.

Stuff like that.

While the meme of the article it something permanently hosted on Amazon, it's real niche is spontaneous, short term, computing needs. You can get a VM at Amazon faster (perhaps cheaper) than your internal IT department.

Re:A Few Basic Questions

[Jack9]

Why?

I remember all initial articles/hype quoting Amazon reps as saying it was a method of monetizing devalued/obsolete hardware rather than writing it off and disposing (all) of it.

Re:A Few Basic Questions

[scribblej]

I can answer the PCI question. VISA and the approved auditors do not currently allow virtual machines, even if you are running them yourself on your own hardware. And to have them run on someone else's hardware.... forgetaboutit.

That may be changing soon though, with the popularity of VMs, they will have to do /something/. I hope.

Re:A Few Basic Questions

[nahdude812]

I have a hard time placing this in the enterprise for large business, but I can see how it would be valuable for medium and small business. S3 would let them serve large media files to lots of customers and pay per usage instead of buying a fat pipe, while also having a substantial amount of burst capacity should they have an event where lots of people seek out their goods.

From a personal level, I'm using it for two purposes. For one, I use Jungle Disk with S3, and have some specific utilities and files which I want synchronized access to from multiple locations. I update a file at a client site, and it's updated at home too. All this comes with the convenience and speed of editing as if it was a local file.

The second purpose is also through Jungle Disk. I'm a photography enthusiast, and I have a couple of dozen of gig worth of photos that I would be crushed to lose since there's nothing in the world which could replace them. Jungle disk has a backup feature, and I use it to back up my photos (and some other data) to Amazon's services. I'm incredibly nervous about the loss of this information. I keep it on my desktop, and rsync it nightly to my server in the closet. I also rsync it nightly to a NAS RAID 5 storage device, which itself backs up to an external USB hard drive. So I have 4 copies of this data in my house at all times. But none of that would protect it from a fire or sufficient power surge, and that's why I use S3 as my last line of defense. For a couple of bucks a month, I get another level of peace of mind.

Re:A Few Basic Questions

[nahdude812]

Many S3 tools, like Jungle Disk, support encryption before you upload the files. So what you're storing on Amazon's servers they couldn't give to the government even if they wanted to.

Re:A Few Basic Questions

[kbob88]

Re #5 (why?)

My guess is that they set up all the infrastructure to run their own systems. Then someone realized, hey, we could market this! So most of the fixed cost is already covered by their own internal needs.

Re:A Few Basic Questions

[Thing 1]

Many S3 tools, like Jungle Disk, support encryption before you upload the files. So what you're storing on Amazon's servers they couldn't give to the government even if they wanted to.

In general, it's far better to do no wrong, rather than dream up ways to avoid getting caught.

In this specific case, your VMs "automatically" read the data back in when you start them up, don't they?

So, the warrant includes the VMs. You're not safe using EC2 for criminal activity...

Re:A Few Basic Questions

[Anonymous Coward]

1. For EC2, we currently use it for full web application development. Basically giving developers and QA a full stack of "hardware" that pretty much matches production. We also use it to grow our production server farm on a semi-on-demand basis, so when we know ahead of time that there is going to be a spike in traffic, we have the system resources available.

It's not really good for clustering where really low latency network connections are required or really IO intensive operations due to the nature of VMs. But for basic web serving kind of stuff, works pretty alright.

2. EC2 instances go down or otherwise become unavailable on a fairly regular basis. Sometimes you have warning, like when the physical host that the VM is on starts to "degrade." Other times, like last Thursday, a router went belly-up a random set of our instances were unavailable to the Internet, but were still up. If you need serious up-time, spend the money on building out your own infrastructure.

3. At a previous company we used just S3 for disk storage. It's not as fast as a CDN like Akamai, but it was cheaper and faster than what we could provide for the terrabytes of data that we were serving. I'm not so certain that using EC2 w/o S3 would be really feasible, because you want to have backups of your instances in S3 for quick recovery at a minimum.

4. Anything is possible. It's like anything regarding security and compliance; there is a balancing act. When you consider that plenty of companies have violated SOX and none have them have been fined or prosecuted, you kind of have to wonder whether you should even care. Last place I was at that was audited for SOX compliance (and were not anywhere close) were told, "We'll be back to redo the audit next year. Fix it." And nothing else.

5. The AWS services maybe "cheap" for us users, it has way better margins than the traditional retail space that Amazon.com runs in. Considering how much hardware and network resources they have just for amazon.com, it becomes a question of how to better utilize the extra hardware you have around when the website isn't pushing it. Adding some more headroom in order to support other users of their infrastructure becomes trivial. Initially, AWS users are just using extra capacity that Amazon already had. Now it's making money enough to pay for its own growth.

Re:A Few Basic Questions

[daboochmeister]

I use S3 for personal backup. Simply get JetS3t, and setup a cron job using their synchronize.sh script. Easy peasy.

I backup our growing collection (~7GB at this point) of photos/movies/music/personal docs nightly, costs about $1.20/month.

For now, I'm just backing up everything, brute force, once a week, and doing synchronize nightly (gets the additions/deletions). When I get around to it, I'll actually setup an rsync to a separate LVM volume, and backup just the diffs, or something fancier.

Re:A Few Basic Questions

[nahdude812]

I'm not familiar with what your options are with EC2, but with S3, my concern isn't, "How do I get away with illegal activity while getting someone else to house my data for me," so much as, "How do I protect my constitutional right to protection from unreasonable search and seizure." In the US of late, simply having a constitutionally guaranteed right doesn't really guarantee anything.

People have a right to privacy, and corporations have a right to protect trade secrets. Or maybe you're so naieve as to believe that government agents are incorruptible. The thing is we can't even tell if they are already corrupted and selling the information of other people, because the domestic spying program they've established, in addition to being illegal and unconstitutional, lacks any form of oversight or checks and balances. If they were selling your identity, if they were selling your medical records, if they were selling your company's trade secrets, there is no one who 1) would even know it, or 2) would be able to stop them even if they did.

The data you store on Amazon's web services could be given to the government via several channels already. They have that unfiltered backbone monitoring station, courtesy of AT&T, and maybe some other backbone providers. They also have National Security Letters which they could use to demand this information from Amazon itself, and Amazon wouldn't even be able to admit they'd received a letter at all, much less who it targeted.

Even independent of corruption (which I assume is a given since all humans are corruptible at the right price), there's always the chance that your personally identifiable information, trade secrets, or other private information ends up on a stolen laptop (which has happened numerous times in the last few years).

Even if it were not in direct contradiction with one of this country's founding principles, even if it were not horribly illegal and immoral, there's still plenty of reason to want to protect your own privacy which does not involve wanting to engage in illegal activity.

Re:A Few Basic Questions

[strudslev]

2: We're using EC2 on Momondo.com. Currently we have about 30 hardware hosted machines as our backbone and when we need to to scale our robot servers we need to we just add x extra servers to handle the load. It works great for our purpose.

Re:A Few Basic Questions

[dave420]

Here's what I'm using it for:

I'm currently developing a web-based specialised video sharing site, and I'm going to use EC2 to convert uploaded videos to .flv (and possibly h264, spec depending). That way, the main web server can create new instances should the upload queue get too long, and close them down when the queue is empty. The user uploads their videos directly to the S3 service, and get added to the queue. Each EC2, when booted, asks the web server for a video to process, and all the instances continue munching on the queue until it's empty, at which point all the instances close down. That way we don't pay for EC2 unless we actually need it.

I didn't get how to use these services before, but now I've seen the light. It's not necessarily the technologies themselves, but how Amazon lets you interact with them. The only problem with any of this is that the S3 service doesn't have a way to move files between buckets, rename them, or edit metadata, without having to upload the object again.

Re:A Few Basic Questions

[try_anything]

If that's the case, what happens when Christmas rolls around?

Easy -- everybody is so busy buying presents on Amazon that nobody has time to visit any other sites until after Christmas.

Re:A Few Basic Questions

[try_anything]

In general, it's far better to do no wrong, rather than dream up ways to avoid getting caught. You're not safe using EC2 for criminal activity...

You slept through high school civics class, didn't you? That's not the point. When someone in the government sees you as an enemy, whether it's because

• They sincerely think you've done something bad;
• You're a business competitor of their friend, patron, or ally;
• You support the wrong party in a significant and meaningful way;
• You support an organization or proposal that threatens their bureaucratic fiefdom;
• Or, they simply think you're hindering them from doing their very important job;

... they will treat you like a criminal who happens not to have been convicted of a crime yet. Most people are incapable of seeing you as an enemy without developing a feeling of moral disgust toward you. That's why we're so paranoid about the rights of criminals in the United States. You don't see nearly as many people bitching about the rights of criminals who have been convicted. (A few people get excited about it, but not as many.) Criminals are fined, confined against their will, forced to wear uniforms, given very limited privacy, forced to report to parole officers, etc. We're fine with that. What we worry about is the rights of criminals before they get convicted, because we know (and you should, too) that every citizen is a criminal in the eyes of somebody in the government.

Personally, I'm probably only a criminal in the eyes of the lady at the DMV who had to admit (after wasting five minutes telling me I was wrong) that a mistake HAD been made and needed to be corrected. She was involved in neither the mistake nor the correction; nevertheless, she was seething with anger because I had persisted in disagreeing with her. If she had the power, she would have suspended my license on the spot. In some countries that would have been possible. Even in the Western world, in the old days, you could be arbitrarily stripped of your rights by an accusation that you were a criminal. It seemed like a common-sense thing to do, but at some point we realized it worked out very poorly in practice, so we used the concept of rights, buttressed by judicial oversight, to prevent abuses.

Now, in the new old days, there's a suggestion that you can be arbitrarily stripped of your rights by an accusation that you are a terrorist. It's an awfully handy tool for an official to have if, say, they wanted to do a favor for one of your business competitors. They could open a terrorism investigation, take a quick look-see through your Amazon EC2 servers, tell your competitor what they saw, and close the investigation with the finding that you weren't a terrorist after all. (See? The system works.) What kind of judicial oversight applies to this? Didn't the Bush administration say that they had the constitutional authority to ignore any laws passed by Congress, and any decisions handed down by the judiciary, that might hinder their ability to fight terrorism? You can't discount the possibility they are living up to their words and entrusting unreasonable powers to government agents, who are just human beings and not beyond the reach of blackmail, bribery, partisanship, and the allure of powerful friends.

In other words, now is a bad time to trust the government to respect your rights. (As if there were ever a good time.) If you harbor any hope that your business or other endeavor will succeed enough to gain the attention of powerful people, you should think like a criminal. Without transparency and meaningful judicial oversight, we're all criminals. And don't we all harbor the dream that someday we'll make a big enough splash that we get some important shoes wet?

Re:A Few Basic Questions

[Thing 1]

I completely agree with you about protecting your rights through encryption. I was just saying that I saw a hole in the argument.

If they were selling your identity, if they were selling your medical records, if they were selling your company's trade secrets, there is no one who 1) would even know it, or 2) would be able to stop them even if they did.

There is someone: the employee in the know who later leaves the corporation that purchased the illegally-gained information.

He then has some serious blackmail capability against his former employer. And, he could also go for a whistle-blower reward.

You are absolutely correct that "everyone is corruptible," so it's great to know that there's "someone" on the other end of the deal you mentioned. It may take a while, but eventually that (or those) individual(s) will get greedy for more than they were most likely handsomely paid, and will try to coerce funds with their information.

I actually really like that, the idea that the other side is corruptible as well. We could even start a campaign to help corrupt them. Well, actually I guess that's what wikileaks is.

Re:A Few Basic Questions

[Thing 1]

... they will treat you like a criminal who happens not to have been convicted of a crime yet.

There's a quote from Ayn Rand's "Atlas Shrugged" that fits here perfectly:

"Did you really think that we want those laws to be observed?" said Dr. Ferris. "We *want* them broken. You'd better get it straight That it's not a bunch of boy scouts you're up against- then you'll know that this is not the age for beautiful gestures. We're after power and we mean it. You fellows were pikers, but we know the real trick, and you'd better get wise to it. There's no way to rule innocent men. The only power any government has is the power to crack down on criminals. Well, when there aren't enough criminals, one makes them. One declares so many things to be a crime that it becomes impossible for men to live without breaking laws. Who wants a nation of law-abiding citizens? What's there in that for anyone? But just pass the kind of laws that can neither be observed nor enforced nor objectively interpreted - and you create a nation of law-breakers - and then you cash in on guilt. Now that's the system, Mr. Rearden, that's the game, and once you understand it, you'll be much easier to deal with."
-- Ayn Rand, _Atlas Shrugged , Ch. III, "White Blackmail"

She understood what the government was beginning to do, back in the 1940s and 50s. Which makes me wonder, have there been any governments that didn't encroach? There was a reason Jefferson wrote "The tree of liberty must be refreshed from time to time with the blood of patriots & tyrants" -- he was not promoting war, he was understanding human nature, and humans run the governments.

Now, in the new old days, there's a suggestion that you can be arbitrarily stripped of your rights by an accusation that you are a terrorist.

I think it's always been that way deliberately. The witches, the Jews (hi Godwin!), the Red Scare, and now the terrorists. There's always a "thought crime" that governments choose to enforce, because it's the one that requires the least amount of evidence to prosecute. "If she floats she's a witch and we kill her; if she sinks, well it's too bad that she drowned but at least her soul's in Heaven." Scary, but keep the population religious and they'll believe anything.

Re:A Few Basic Questions

[nahdude812]

I hear you on that. Further, it's happened a few times with the current administration already. People who make it public knowledge that there's a locked room in an AT&T backbone data center which a spliced fiber runs into, giving the government access to all data moving across that line, whether or not they have a right to access it, and without any sort of check or balance in place to make sure that they only access what they should.

But it should be noted that in light of the PATRIOT act, even revealing that a request was made is a crime of the magnitude which gets you disappeared. Whistleblower protection only applies if you are given due process, and even US citizens are no longer eligible for due process if a suit at the FBI/CIA/NSA uses the words, "national security," or, "terrorism."

No

[I Like Pudding]

Amazon just has a very interesting service architecture. This is why you keep seeing articles all over the place about it.

Re:No

[ShieldW0lf]

Doesn't look very interesting to me. Nothing novel, just Amazon "Now with hosting"

Why would anyone choose to leave their data on someone elses server when the price of owning your own hardware has become so low?

Pretty boring Me Too kind of stuff.

Re:No

[tolan-b]

Because having your own hardware you can't scale up to 50 server instances for half an hour and then scale back down to 1 when traffic decreases, just as one example.

Re:No

[Anonymous Coward]

Why would anyone choose to leave their data on someone elses server when the price of owning your own hardware has become so low?

The hardware is the least of my company's worries. The fact that ISPs like comcast and at&t are actively threatening "content providers" (that is: everyone with a server) over "using their bandwidth" (that the ISPs are being paid for by their customers, in addition to our ISP paying them (directly or indirectly) for transit, and for which we pay our ISP) make it more and more difficult to justify hosting any sort of serious application in a closet hanging off of a leased T1 line. Hosting with Amazon (or any other large consolidated data center, virtualized or not) at least brings collective bargaining power to the table when these large ISPs finally decide to break out the crowbars and say "ok, give us a million bucks or nobody will ever see your site again".

Re:No

[jamie]

For me, it'd be more about hassle than price. If I'm developing a new service, it starts with just one server and I don't want the hassle of figuring out where the best host is. I want the flexibility to cancel the whole thing with no contract (billed by the hour) and just walk away if it turns out not to be a good idea. I also want the flexibility to scale quickly from 1 machine to 10 and 100 without having to worry about picking out the hardware, billing, power, cooling, network architecture, backup, fixing dead machines, and of course whether the host has room for me to scale.

When the VCs want to know what issues are involved with my service scaling to 100x its current size, they would much rather hear that the single hardware issue is "dollars," rather than that whole long list of unknowns. Dollars are easy.

And from what I can tell, EC2/S3 would scale from one server up to Slashdot size and beyond without much problem. Probably not to Wikipedia size, but I wouldn't be surprised if it could get close. And as someone else noted, they don't have data centers anywhere but the East Coast... but I wouldn't be surprised if they're working on that too (I don't have any inside info, I didn't sign an NDA).

Re:No

[Metaphorically]

Amazon's angle is that it scales up and down with the application demands. That's the 'elastic' part. It's more cost-effective than owning your own hardware for many applications. Suppose you had a web application that did some image processing. If you only get a handful of visitors then any decent hosting will do the job. When you get a traffic spike then your app can bring down the server or at least get your site cut off due to CPU quotas. Otoh if you design it to take advantage of EC2 then you can scale up as you get more visitors very quickly (dunno if it's seconds, minutes or hours) and only pay for the extra computing power while you're using it.

Re:No

[teh kurisu]

Funny you should say that. Just today my boss was telling us at work how he'd bought a Dell PC for about £170, including VAT and shipping. Pretty damn cheap. The problem is that there are other costs involved if you want to turn that PC into a web-facing development platform, which is what I'm currently using EC2 for - mostly man hours, but also stuff like electricity, backup solutions, a reliable internet connection, and physical space. To a company with five employees, these costs aren't insignificant.

We use virtual servers hosted by Pipex Webfusion at the moment, but their current Linux offering is a version of Fedora Core that's years out of date and quite inadequate. We've also had one major outage with them, where our server was inaccessible for a number of days. Last week I finally snapped, and dived head-first into EC2; fortunately EC2 is one of my boss's 'new toys', so he didn't seem to mind that I'd added several days onto the length of the project.

The advantages to us are that the servers are there when we need them, and costing us absolutely nothing when we don't. They're cheap, and much, much more flexible than our Webfusion service.

Re:No

[CastrTroy]

What's the downside of that though? Say your site gets slashdotted, and now, instead of having your site taken down, you get a huge bill, for all the resources you used. What provisions are in place to ensure you don't get a giant bill at the end of the month that you can't afford to pay, because someone linked to you on slashdot?

Re:No

[Wesley Felter]

What provisions are in place to ensure you don't get a giant bill at the end of the month that you can't afford to pay, because someone linked to you on slashdot? If you want more servers on EC2, you (or some code written by you) has to tell Amazon to start them. So you could choose not to "go elastic" if you can't afford it.

Re:No

[vidarh]

Yes you can. Nothing stops you using EC2 for "overflow". EC2 for instances you use most of the time isn't cost effective compared to a number of other hosting providers, which is no surprise since you pay for Amazon to keep a huge amount of spare capacity to handle surges.

Re:No

[TooMuchToDo]

The advantages to us are that the servers are there when we need them, and costing us absolutely nothing when we don't. They're cheap, and much, much more flexible than our Webfusion service.

That's exactly the reason Amazon started offering S3/EC2/etc. It's cheap for them to build the infrastructure up since they have to use it anyway for peak shopping periods (Christmas). But the other times of the year when they don't need the capacity, they can make a buck renting it out to others. You shouldn't use it for every situation, but their product/offering definitely has it's place.

Re:No

[iowannaski]

Oh. Well I guess that is interesting then, isn't it?

Re:No

[tolan-b]

So to translate your answer, "no you're right, you can't do that without EC2 or a similar service."

Glad we agree.

Slicehost.com

[casings]

Cheap, affordable, reliable VPS solutions: www.slicehost.com

I have been with them for a few months, and their interface's ease of use, and the level of support they provide are just what I was looking for.

Re:Slicehost.com

[barryp]

Agreed. I've got 256MB slice running Apache, Django, PostgreSQL, Exim quite nicely. It's a Xen setup, my particular machine is an Ubuntu Gutsy server, was awfully easy and quick to setup. Lots of clear tutorials for getting your VM tuned up for firewalling and so on - I found those especially handy since I'm more of a FreeBSD guy and wasn't up on some of the Linux-isms.

Re:Slicehost.com

[stevey]

Although recently a Debian Developer was critical of slicehost, and seemingly in a valid way.

Personally I host a reasonably high-traffic antispam service and I think Amazon's offering looks good, but as mentioned a little pricy.

I love the idea of adding extra nodes on-demand, but I think I'm not yet at the level where it would be a worthwhile use of my time or budget.

Re:Slicehost.com

[Skylinux]

Same here, I have been using Slicehost for three months now and would never go back.
I setup a small slice running CentOS 5 server hosting all my domains for $20 a month. I have tested the site availability with hostracker set to 1 minute intervals for 5 domains and result has been 100% uptime.

Slicehost.com rocks!

Re:Slicehost.com

[speculatrix]

in the UK, bytemark.co.uk get very good ratings for service. I am not a bytemark staff member, but I am a customer!

Re:Slicehost.com

[TheRaven64]

Since this seems to be thread for whoring our hosting providers, I'd like to recommend mine. I get a dedicated server for about the price of a VPS, and I have a human in my IM roster that I can bitch at if anything goes wrong. I've been with them now since a few months after they were on /. and have been a very happy customer. They had a few reliability issues early on, but nothing recent. The hard drive on my machine died just under a year in, and Apple refused to honour the warranty, so the co-lo company picked up the tab for the replacement.

Oh, and OpenBSD on a Mac Mini is really nice.

Re:Slicehost.com

[smallfries]

As people are recommending their hosting providers this seems like a good place to ask for a suggestion. I've wanted to move my filesystem off of the machine at my place, and onto an online provider. Unlike most people looking for shell account I'm not so interested in bandwidth costs, I figure I'll only need 200GB/mon. But for storage I want about 2TB, and this seems to be the killer. Each hosting company that I've checked has built their pricing plans around the assumption that people are hosting web-apps - lots of bandwidth, not very much disk.

What's the most cost effective way to do this? S3 looks ideal, but it is rather pricey. Dedicated hosting seems a waste as all I want is access to the storage and it's going to cost about £100/month to get 2TB of storage.

Has anyone got any ideas / experience?

Re:Slicehost.com

[TheRaven64]

My advice would be for a dedicated server, and talk to the hosting company. Mine will get external disks for you if you want them, so you could chain a load of FireWire disks to give you 2TB, but you're probably better off finding someone that focusses on 1U rack spaces and getting a server with a few big disks in it. Generally, if you talk to someone at the company they can provide you with something that fits your needs. If they don't have a human you can talk to, then run away and don't look back - you're much better off somewhere (anywhere) else.

Re:Slicehost.com

[Anonymous Coward]

Not trying to be a shill or anything but slicehost rocks. Their differentiator is that you (seem) to get exactly what you pay for all the time, and can burst higher. A lot of Xen hosting shops oversubscribe their infrastructure so that anywhere from 25% to 75% of your domU RAM is swapped by the host, which sucks hard if other people on your host start getting busy. At least most places admit that they swap some RAM on their domU's if you read the fine print.

The only downside is that they don't support custom kernels. I can appreciate the support and security issues, but it would be great if they partitioned off a chunk of their service or a small datacenter, and have that be a free-for-all with the acknowledged risks of running a domU on that infrastructure.

No persistent storage; not great value

[saterdaies]

There's still one glaring problem. There is no persistent storage (other than shuttling data to S3). That means that if your website is database-backed, you need to figure out what to do should your instance crash. Hourly backups? Mounting S3 as a slow FUSE filesystem that you can put your database on? It's all ugly.

And it's still not a great value. It seems cheap. $72/mo for a 1.7GB RAM server. Well, look at Slicehost and you can get a 2GB RAM Xen instance (same virtualization software as EC2) for $140 WITH persistent storage and 800GB of bandwidth. That doesn't sound like a great deal UNTIL you calculate what EC2 bandwidth costs. 800GB would cost you $144 at $0.18 per GB bringing the total cost to $216 ($76 more than Slicehost). That 18 cents doesn't sound like much, but it adds up. The same situation happens with Joyent. For $250 you get a 2GB RAM server from them (running under Solaris' Zones) with 10TB of bandwidth. That would cost you $1,872 with EC2. Even if you assume that you'll only use 10% of what Joyent is giving you, EC2 still comes in at a cost of $252 - and without persistent storage!

EC2 really got the ball rolling, but it just isn't such a leader. Other operations have critical features (persistent storage) that EC2 is lacking along with pricing that just isn't more expensive. I want to like EC2, but their competitors are simply better.

Re:No persistent storage; not great value

[jamie]

You get database backup by replicating to another VM, presumably one in a different "zone" for physical separation. Then that backup VM every n hours stops its replication, dumps to S3, and starts replication back up (exactly like a physical machine would stop, dump to tape or to a remote disk, and restart).

Database high-availability is similar. In the extreme case, you replicate your live master to the master database in another zone that entirely duplicates your live zone's setup (same number of webheads, same databases in same replication configuration, etc)... then if the live zone falls into the ocean you point your IPs to the webheads in the HA zone and resume activity within seconds, having lost only a fraction of a second of data stream.

Having dealt with Slashdot's webheads and databases losing disk, and in some cases having to be entirely replaced, I don't see how persistent storage is a big selling point. I mean it's nice I guess, but not something that I'd sacrifice any functionality for. Applications have to be designed to run on unreliable hardware.

Re:No persistent storage; not great value

[mveloso]

It's $72/month if you're at 100% cpu all momnth. It's $.10 per cpu hour, which is retardedly cheap, because you only pay for what you use.

I haven't used it because of the lack of a static IP. Now, it's a viable solution for the real world.

Re:No persistent storage; not great value

[saterdaies]

Billing is based on instance-hours not cpu-hours. So, for every hour or partial hour your instance is running, you get charged. It doesn't matter if you're a 1% cpu usage or 100% cpu usage during that time: http://www.amazon.com/ec2

Re:No persistent storage; not great value

[dogas]

It seems like you answered your own question about persistent storage. S3 is persistent storage.

If you are running a database backed application on EC2 without a master/slave setup, and your master goes down, to me, that seems like a failure to plan for the worst on your end. It's really not an argument that even though you DO have persistent storage, your data is safe on that server. Your data is never safe. Hence, a backup/replication plan is ALWAYS needed. Services like EC2 force you to think about those plans and address them so that they are not a problem.

As for the slicehost option, we looked at them. We found that Amazon seemed to provide a much more robust service. Plus, Amazon has a LOT more resources to keep their service robust, as opposed to a startup with one datacenter.

Re:No persistent storage; not great value

[jpnewservers.com]

Newservers.com is a competitor that provides a simpler solution with better value. We offer real instead of virtual servers with regular harddrives, hourly billing, redundant hardware load balancing and a simple webapp for adding, removing, and monitoring servers.

Re:No persistent storage; not great value

[mini me]

Mounting S3 as a slow FUSE filesystem that you can put your database on?

They have SimpleDB, although it's still in limited beta by the looks of it. At present you can store your data anywhere. If you're using some kind of SQL server they don't scale all that well vertically anyway, so there's not much point in using them on a EC2 type setup, even if storage was persistent.

Re:No persistent storage; not great value

[dubl-u]

Well, look at Slicehost and you can get [...] WITH persistent storage

The Amazon machines offer storage that persists for the life of the virtual instance. That's until you kill the instance or until the hardware fails. (It does persist through reboots and OS crashes.) And unless Slicehost is running some crazy magic beyond the RAID-10 setup they mention, a hardware failure could still wipe out your data, and will certainly cause downtime during which you will have an opportunity to wonder when and whether your data is coming back.

If you run some stats, it could well be that the Slicehost "persistent" storage does indeed persist longer. That'd be my guess. But it's possible that the Amazon "non-persistent" storage is actually more stable. That depends on the quality of hardware and maintenance at both companies, factors that you cannot know. Meaning that if reliability is really important to you, you must plan for either kind of storage eventually failing. And if reliability isn't that important to you, then you're planning to depend on your backups anyhow, in which case Amazon doesn't seem so bad either.

I think the main difference between Amazon and the more typical providers regarding persistence is that Amazon's experience has taught them to assume that everything fails, and so you should engineer for that. EC2 and S3 are built around that, and are very frank about what they provide. It seemed weird at first, but now I like it better.

Re:No persistent storage; not great value

[mveloso]

Whoa, that's a change. Before they used straight CPU time. I guess they weren't making enough at that price point.

Re:No persistent storage; not great value

[Wesley Felter]

S3 is persistent storage. With weak (i.e. useless) semantics that are totally different from conventional storage.

Your data is never safe. Hence, a backup/replication plan is ALWAYS needed. Sure, but if your plan involves a SAN, EC2 cannot support it. There are so many best practices that EC2 does not support; effectively you have to design your app for EC2.

Re:No persistent storage; not great value

[Creepy Crawler]

In all honesty, accounting for S/N ratio, how much is a slashdot post worth?

Thats what would matter for the failover time of lost data. But really, I'd be interested in how much a post is worth (it is content, albeit small).

Re:No persistent storage; not great value

[dogas]

I think if your setup requires a SAN, you're too big and enterprisy for EC2.

S3 has been working well for us. While the semantics are different than typical storage, I would argue that they are far from useless. Since files on S3 can be made publicly accessible via a web address, we use S3 to host our assets for our website (css, javascript, images), as well as db backups and other backups.

We have not had to design our app for EC2. We do make use of S3 for storing user data, so we have S3 libraries in our app, but I fail to see where the "designing our app for EC2" comes into play.

Re:No persistent storage; not great value

[Wesley Felter]

No, the EC2 billing has not changed. It was always by instance-hour.

Re:No persistent storage; not great value

[colin7151]

I would tend to agree about the value Issue. Please tell if I am misunderstanding there pricing, but from what I see for $.80 / hr you get a box with 8 "EC2 Compute Units". They specify that a Compute Unit " provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor." This seems insane, for this XL instance I would be paying $576 / month for. I can lease a box with 2x the CPU power (8, 2.83 cores) for $274 / month. Now, peak capacity ... you don't have all the instances running at once I get, but Powering up a new master "on demand" is not a simple task with a decently sized dataset. So, I guess for EC2 to be valuable 2 things need to be true for you. You must have a spike that is high enough from your norm to justify those prices, and you have to have an app that is easy to "shard" into small ephemeral pieces. Do not underestimate how hard the ladder can be, doing this with a DB that has a large data set is not trivial. In addition, there tend to be a lot of things you can be doing during periods where you have the capacity, IE: you know basic sales stats, but what about all the complicated relationship that are too expensive to calculate live ? Are they running at night or whenever you idle time is ?

Re:No persistent storage; not great value

[Anonymous Coward]

EC2 with S3, hadoop and hypertable would solve a lot of issues regarding the database backup problem. Keep an eye on these emerging projects. The problem when you database grows in size is always how to backup.

There are LVM snapshots solutions, slave backup solutions, but I definitely think that some project like hypertable (http://www.linuxworld.com/news/2008/020608-hypertable.html) and thrudb (http://www.igvita.com/2007/12/28/thrudb-faster-and-cheaper-than-simpledb/) are the future of data storage when we thing about database.

No virtual ip or failover

[dominic.laporte]

My major concern (last time i checked) was fail over & virtual ips. I think they fixed this with the new elastic ip. I will have to check again.

However, another issue i had was to send traffic between 2 EC2 nodes. They don't mention (maybe i missed) nor guaranty the bandwidth between the nodes in the same availability zone. This is crucial if you are trying to run a very fast performance tests between the 2 nodes and you need minimum delays. I am not sure if the bandwidth between the EC2 nodes is caped or no as well.

IPV6? How about an SLA!

[Jeremiah Cornelius]

"More ready" is wonderfully relative.

"Less unready" is just as accurate, and perhaps more precise.

Without an SLA, EC2 or SimpleDB, or "Head in The Cloud" is an experimental platform.

Anyone Use 3Tera's AppLogic?

[Trail_of_Dead]

We looked at the EC2 solution when we started developing our hosted offering and didn't care for the new IP address when, and if, something went down. We went with a hosting company called LayeredTech. They offer public and private VPS and VPDC solutions. The really cool thing that has impressed me is they run 3Tera's AppLogic platform. It lets you visually (through a web ui) create "applications" based on "appliances". There is a standard portfolio of prebuilt applications (SugarCRM, etc.) and templates for LAMP, etc. So, we build our application by taking a firewall appliance, a CentOS appliance, a gateway, a MySql appliance, glue them together, customize them, and then create our own template. You can specify down to the appliance level, the amount of cpu, memory, disk, and bandwidth each are assigned which let's you scale up your capacity simply by tweaking values through the UI. We can now deploy our Rails/Java hosted offering for new customers in about 20 minutes on our grid. AppLogic has automatic failover so that if anything goes wrong, it reploys your application to a new node in your grid and restarts it. It's not as cheap as EC2, but much more powerful. It's definitely worth a look.

Re:Anyone Use 3Tera's AppLogic?

[Wesley Felter]

No, no one has used AppLogic because the minimum price just to try it out is hundreds of dollars per month. EC2 is somewhat flawed but they are getting a lot of business because it is so cheap to try.

Re:Anyone Use 3Tera's AppLogic?

[Trail_of_Dead]

When you use LayeredTech as your hosting provider, it's included with no separate license price. LT's prices are very reasonable as well.

Expensive; Especially to Debug

[Anonymous Coward]

The small model is $0.10 per instance-hour. 24*30 = 720 * $0.10 = $72.00/month ... for a computer you don't own.
Considering today's hardware, that's a very expensive "lease".

Moreover, from TFA:

Paying for What You Use
You will be charged at the end of each month for your EC2 resources actually consumed.

As an example, assume you launch 100 instances of the Small type costing $0.10 per hour at some point in time. The instances will begin booting immediately, but they won't necessarily all start at the same moment. Each instance will store its actual launch time. Thereafter, each instance will charge for its hours (at $.10/hour) of execution at the beginning of each hour relative to the time it launched. Each instance will run until one of the following occurs: you terminate the instance with the TerminateInstances API call (or an equivalent tool), the instance shuts itself down (e.g. Unix "shutdown" command), or the host terminates due to software or hardware failure. Partial instance hours consumed are billed as full hours.

So, if your instance starts up and then crashes immediately due to software failure? That's $0.10 you just spent.

Better make sure your instance is running production quality code, otherwise crashes just after the hour mark can get pretty expensive pretty quick.

Summary: Amazon's hosting is neat, but expensive. There are cheaper alternatives.

Re:Expensive; Especially to Debug

[jamie]

"Software failure" in that case refers to a failure of Amazon's Xen software that runs your virtual machine.

Amazon doesn't know or care whether your software is "production quality code" or not. You pay $0.10/hr whether your code is debugged or not :)

Our experience using EC2 + Rightscale

[dogas]

My company uses EC2 + S3 + SQS + Rightscale (http://rightscale.com) to manage our infrastructure.

First off, Amazon has an excellent product. It is essentially Hardware As A Service, and the tools they provide abstract it as such.

The most common argument against using EC2 for hosting is that if your server goes down, you will lose any data created since the last time you saved a snapshot. While this is true, it forces you to bring a backup + recovery plan to the front of the table. Provided you have a backup + recovery plan in place, you no longer have to worry about fixing a server ever again. If something goes wrong with one of our application servers, I would simply fire up a new instance, link it in with DNS, and terminate the old server. With rightscale, this is all pushbutton.

Consider that scenario with running your own colo server. You could potentially spend hours diagnosing + fixing an issue with a server before you could bring it back up. Ok fine, the way to mitigate that is to have a hot backup running. But now we're talking about a ton of cash to support 2 servers on a month-to-month basis. We have found that amazon's costs to run EC2 instances are very competitive for the specs.

Note: I'm not a shill for either rightscale or amazon, I just find that these 2 companies are the forefront of where hosting is going, and their products are awesome. It's all about virtualization!

Re:IPV6? How about an SLA!

[Phurge]

is that true about no SLA?

That would make me pretty nervous committing my company's business to Amazon without an SLA....

linode

[realmaestro]

I use linode for my VPS hosting. Though I only use it for blog/family photos, I love how easy it was to set everything up. Linode only lets you run linux though, but offers a choice of distribution. I use their CentOS 5 build. It comes with persistent storage, and a nice little web console to manage. It's great to play around with, as their cheapest option is $20 a month (I bought when it was at a special and only $10 a month for the lowest option).

Highly recommend it, the guy who runs it is very responsive in the forums, though I have never had a problem (don't think my site gets much traffic though ;-)).

Only thing that would potentially prevent me from using VPS for a business though is company-sensitive data. Do you guys who have used VPS for company servers do anything to protect your data? That's the one advantage of having your own hardware that I don't know if any VPS really matches. Who knows, maybe having your own box in someone else's rack gives you the same issue anyways. I think I'm probably just too paranoid :-).

Being Geeked

[TheNinjaroach]

"Being geeked" is a term my friends use to describe someone that is very high on drugs.

Kids these days..

Re:IPV6? How about an SLA!

[Jeremiah Cornelius]

Hmmmnn. I been looking. The DB is definitely no SLA. Looks like EC2 and Storage got one, six months ago.

FreeBSD under Xen

[LukeCrawford]

FreeBSD support for xen hasn't been merdged into mainline just yet (acutally, last time I looked there was a patchset for FreeBSD 7 that was broken by xen 3.0.3 that had been idle for a while.. looking again, Kip Macy looks to have updated it to work with FreeBSD-Current: http://wiki.freebsd.org/FreeBSD/Xen, so maybe I'll look into it again when I get a chance.)

NetBSD/Xen is quite stable on i386/non-PAE and netbsd-current has i386-PAE and x86-64 support for xen... If you like OpenBSD, NetBSD might be a better choice than Free (OpenBSD is very close to NetBSD) Any xen provider that can handle i386-non-PAE should be able to give you good NetBSD images. (I won't have a non-PAE box available for 3 weeks or so)

Within the context of ec2, last time I looked ec2 was i386-PAE, so you should be able to run netbsd-current (or even freebsd-current according to the above link) on it.

E-mail me if you want to continue this discussion within the context of my hosting company.

Cloud Services Comparison

[diegoc]

Some days ago I posted an article on my blog in which I try to compare different cloud services and also give my 2 cents opinion about the technology itself (disclaimer: I directly tested only two services, EC2 and GoGrid.)

Beyond the comparison, in my post I say that I was wrong trying to use a utility computing platform as EC2 like a web hosting platform; also, there other very interesting uses of the technology behind the clouds (e.g. creating disposable environments for application testing.)

An interesting idea, but...

[davmoo]

This is a service I find interesting and appealing in many ways, and I intend to investigate it further after reading this thread. But upon using Amazon's handy calculator, my costs for comparable services would be roughly *6 times* what I'm currently paying for leasing two physical machines and the bandwidth to go with them. For quick projects to test out something, this would be a good service. But for a day in/day out stack, I don't think this is it, at least not for me.

Re:IPV6? How about an SLA!

[vidarh]

Most SLA's are worthless. Unless there are SIGNIFICANT financial penalties for downtime they make little difference.

Re:IPV6? How about an SLA!

[Jeremiah Cornelius]

They help you sue - and to reach out-of-court refunds.

EC2 is great but check out Joyent

[JeffHunt]

We've been using Joyent and are very happy. I've used EC2 for a few things and I think Joyent is more economical for many applications.

Re:IPV6? How about an SLA!

[aminorex]

That depends a lot on the scale of your operation and the scale of your hosting service. The value of an SLA is that you can sue to recover damages in case of non-compliance. But it may not be possible to recover real damages in court: Your provider may not have pockets that deep, you may not have pockets as deep as your lawyers' thirst for money, and the law may not allow for full recovery in your circumstance.

EC2 is up and stays up. Reliabilty counts for a lot more than legal recourse, in my book. SLAs don't create reliability, they *help* (hopefully) to create legal recourse, which is a very poor substitute.

I have a few questions too.

[arete]

I understand that EC2 is *nix only, with nonpersistent filesystems, and that S3 is an apparently very reliable remote filesystem that you can get to really fast from EC2 for free.

I understand the huge value of this for transient (1 month) intensive very bursty workloads. Which, mostly, seems to be what's it's targeted at.

But for actual normal servers I don't quite see it... I mean one option is that it's cheap. Which it might or might not be, depending on who you compare it to. Maybe it's the most reliable option out there at some price point, but the static IPs (for instance) are pretty young to consider this true, and it's not necessarily cheaper than the discounter's dedicataed servers. If we just assert for the discussion that it's not cheaper per power, then the question is, is it advantageous in other ways?

It seems like you can take your same instance and reboot it bigger, which is nice, but not _really_ that different than most hosting, which will move your drives to newer machine. And it seems like coordinating this would still be a fair headache, and you don't really want to ever have 0 instances during the transition, so you're talking about some overlap.

So it seems like you'd only be really interested in this if you were always going to have your main instance up, and then you were sometimes going to have none but sometimes going to have many other instances up. Past a certain scale it might be worth your time to have more instances 9-5 and less at night, or something (depending on your users) But the setup seems like a lot of issues (not to make it just run, but to really take advantage of it.)

I'm also curious whether it supports automatic instance restarting... e.g. if a zone goes down, can you tell it you definitely want it to put your instance up again in a new zone? (Understanding that your instance has to be smart enough to boot into a useable state unmanaged... but it's a lot better than you having to monitor it.)

Re:I have a few questions too.

[Wesley Felter]

So it seems like you'd only be really interested in this if you were always going to have your main instance up, and then you were sometimes going to have none but sometimes going to have many other instances up. Past a certain scale it might be worth your time to have more instances 9-5 and less at night, or something (depending on your users) Yeah, that is pretty much the use case for hosting on EC2, but it's a small fraction of the market. It seems like EC2 could be much more general purpose, but at this point it isn't.

I'm also curious whether it supports automatic instance restarting... e.g. if a zone goes down, can you tell it you definitely want it to put your instance up again in a new zone? Nope, so you have to have at least two instances running at any one time so they can keep watch over each other.

Re:I have a few questions too.

[Eivind Eklund]

But for actual normal servers I don't quite see it... I mean one option is that it's cheap. Which it might or might not be, depending on who you compare it to. Maybe it's the most reliable option out there at some price point, but the static IPs (for instance) are pretty young to consider this true, and it's not necessarily cheaper than the discounter's dedicataed servers. If we just assert for the discussion that it's not cheaper per power, then the question is, is it advantageous in other ways?

What I've found intriguing about this has always been for startups with an unknown growth curve: You can start with fairly little money, and if things take off, you're able to throw computing power at it VERY quickly. You pay a trifle more for the actual computing power, but it is available effectively immediately. This means you don't have to keep spare power around yourself for the case where you "hit the jackpot".

Eivind.

do they not use pygrub

[LukeCrawford]

or some other method to let you pick the kernel? (even then, you are going to have something of a hard time with FreeBSD, it's xen support is rather, uh, alpha)

Can't really trust anything hosted by a US Corp

[Anonymous Coward]

While this technology seems great and I'd love to play with it, my company is highly distrustful of America and we basically can't use anything which is under the legal hammer of the US Govt. We've just heard too much about secret data mirroring, warrantless searches, endless "monitoring" of everything - it's a nasty state of affairs.

And it gets worse the larger the US Corporation. I don't seriously think that the smaller hosting providers have any kind of NSA supervision, we're using a US Colo facility right now. But it gets easier and easier to believe, as the company size goes up and transparency goes down, that there's data monitoring at some or all of the "big" facilities.

Is this groundless, irrational, kneejerk paranoia? Maybe. But there's a lot of circumstantial evidence. It's all to do with the CIA and NSA - they're masters, absolute masters, of the long-term strategy game. Through their investment arms they choose likely-looking candidates and "bless" them - with their money, and immense influence in diverse circles, they choose, fund and henceforth influence the winners. It's a very clever game, I have to hand it to them.

How to detect such compromises in a company? It's almost impossible but at the holistic level there does seem to be a certain "smell" around the "blessed" players. Google's technology stood by itself, but Facebook? Facebook's rise has been too many ducks in a row for me, and the TLAs couldn't ask for a better information-gathering service. And now they're adding IM - how convenient. I'm sure the privacy of arabs who are members of groups critical of America writing each other messages in farsi will be utterly inviolate!

What has any of this to do with EC2? Well, nothing, except that it has the same kind of smell to me. Large, opaque corporation with obvious dependency on Govt. benevolence? Check. Opportunity to create a single mega-source of intelligence? Check. Good fundamental technology? Check. Slightly suspicious "blessed" feeling around media coverage all choosing an "official" solution to the on-demand problem? Double plus check.

This may all be paranoia, I certainly can't prove anything, and I doubt I'll be able to any time soon. But historically there is ample precedent elsewhere. Many, many intelligence agencies have played the "kingmaker" role everywhere in the world - pick a winner, make sure he's the winner, and make sure he's yours. Historically it's been many different types of company - I am thinking especially of media but it's everywhere. And it's not even solely about surveillance or control - it can just be to strengthen the country's economy by hidden "we're all on the same team" support - look no further than Japan Inc. in the 80s or the Chaebols in Korea today for the textbook example of how that's done. And do I even need to mention NSAKEY?

Am I crazy? You might think so, but I'm not alone. I might not be able to actually disclose who wrote that writing on the wall, why, when or how, but it's pretty obviously there by now.

Especially in the Information Technology realm, it's now USA Inc., and IMO anyone who values their data and/or privacy should at least consider this before entrusting it to an American corporation.

OK, replies denouncing me as a nutcase to start in 3, 2, 1 .. ; )

Does anybody use this as failover?

[Fjan11]

Does anybody have experience with using EC2 as failover? Can it be fully automated?

I operate a regular database backed web site, and have spare servers sitting around in case something goed awry. It would be great if I could avoid that redundancy and set things up so that EC2 instances get fired up if my heartbeat server detects the site is down, pipes the database over (or the latest backup if that's unavailable), and then redirects the load balancer to the EC2 instances. I'd like to do all of this without human intervention (since I hate getting up at 3 AM and I make mistakes a lot)

If this can be done reliably it would be an awesome service, but I have no clue how the business model for Amazon would work, since I would only be really using it for a few hours a year

Re:Does anybody use this as failover?

[dave420]

You can easily start up and terminate instances using the utilities they provide, or the API. Your heartbeat server would easily be able to (re)start instances. Just create and configure your "blank" EC2 and use the utilities provided to upload the image of that EC2 to S3, then use the code it returns when creating instances, and that image will be loaded automatically. EC2+S3 is very powerful, in my opinion. But then that's me. I'm using it to process videos in a queue. I can start and stop instances depending on how many items are in the queue (or even have one instance per item in the queue - it's the same price), and only pay for the time the instances are actually being used. Empty queue = no instances = free.

Anyone using this with Java/Tomcat

[scarolan]

I have a web application that runs on a fairly standard Tomcat/Linux/Oracle stack. Is there anyone else out there using EC2 for something similar? I would be very interested to hear about your experiences with AWS. Contact me at scarolan[at]gmail[dot]com if you have any info.

New Youtube Screencast

[tvoutour]

Hey I spent the last few days learning a bit about EC2. I am definatly still a newbee but if any one is interested I recorded a screen cast of my session creating an instance, lunching a webserver and connecting to the Fedora instance with SSH. You can see it here: http://www.youtube.com/watch?v=bBajLxeKqoY