EC2 Vs. App Engine Vs. GoGrid Vs. AppNexus

snydeq writes "InfoWorld's Peter Wayner delves into the ill-defined realm of 'cloud computing,' providing a deeper look at four shared services: Amazon EC2, Google App Engine, GoGrid, and AppNexus. Offering wildly divergent amounts of hand-holding at various layers in the stack, the services simplify your workload but force you into a set, 'ball-and-chain-computing' routine that you may not prefer. Sure, the services allow you to pull CPU cycles from thin air whenever you need to, but they can't solve the deepest problems that make it hard for applications to scale gracefully, Wayner writes. He describes these 'clouds' as an evolving experiment, rife with potential but 'far from clear winners over traditional shared Web hosting.' The sobering look at the trend includes a QuickTime tour of each service — EC2, App Engine, GoGrid, AppNexus (those links all .MOV)."

Comments

The definition of cloud computing is still vague

[strider2k]

Even after reading the wikipedia article on Cloud computing, I still can't give a good definition of it. I know the general concept but if a non-tech person asked me to describe it, I'll give a blank stare.

I prefer Google for Cloud Computing...

[PC and Sony Fanboy]

I'd choose Google App Engine. Since no one really knows what cloud computing is, and no one knows what google does, I think they make a good fit.

oh wait. I do know what google does - It makes the internet better... and it prints money (I guess...)

Re:I prefer Google for Cloud Computing...

[albee01]

Of the four, Google seems to be the most limiting, at least on the surface. If I understand correctly, Google's offering requires the app to be written in Python and it denies some Python functionality such as file writes.

Amazon's offering, sitting at the other end of the spectrum, allows you to run a full instance of Linux complete with root level access.

The other two are not as confining as Google but more restrictive than Amazon.

On a side note, spam raining from the cloud has become a problem for at least Amazon. Some blacklists are blocking IP addresses owned by Amazon's EC2. If you want to run a mail server in the cloud, it just might rain on your parade.

Re:I prefer Google for Cloud Computing...

[jlar]

"I've been looking into this a bit, and the amazon option seemed the best."

I also looked a bit into Google and Amazons offerings for at Python project. Google was definitely the cheapest and if I could squeeze my project into the limitations they have established I would have chosen it. Unfortunately it is not possible to install C and Fortran extensions to Python (due to security reasons, you can install pure Python modules). This was a showstopper for me.

The critique of not providing access to a local file system is in my opinion misguided. One of the main strongpoints of the Google service is the Google File System (http://labs.google.com/papers/gfs.html). If you do not want to use it, you are probably not building a scalable and distributed web application anyway.

I found the Amazon offering too expensive for my project at this point. I ended up using a minor hosting provider specialized in my type of Python applications (webfaction.com) which does a great job but miss the distributed aspects of cloud computing (not that I need that feature at this point).

Re:The definition of cloud computing is still vagu

[Anonymous Coward]

you mean the definition of cloud computing is still cloudy?

Wow

[BasharTeg]

Finally, a burst of common sense on the latest hype. Hosted servers have offered many of the benefits you get out of "cloud" computing for years, without locking you into a particular vendor or platform. With virtualization, you should be able to build your own images and farm them out to hosting companies, using your technology and platform of choice. Clustered ESX and SANs already give us the resource scalability we need for most systems, partitioning finishes the job. You can just pay a hosted server company to host your vmware image on their ESX cluster and scale up your storage as needed on their SAN. The key is that YOU build a scalable design.

I highly doubt a majority of businesses are going to lock themselves into one hosting provider's specific development platform just to take advantage of hosted servers that push themselves into the next layer.

Re:Wow

[snuf23]

I highly doubt a majority of businesses are going to lock themselves into one hosting provider's specific development platform just to take advantage of hosted servers that push themselves into the next layer."

This depends on the service. Amazon's ec2 is basically just Xen virtual servers provisioned on the fly. s3 is a little weird but there are plenty of tools available to use it in whatever your application is running on. Code changes to support it are not all that difficult.

Re:Wow

[BasharTeg]

SQL's scalability is next to nil for any real web application these days if all you are going to do is partitions. If you shard you get significantly more mileage but you still don't come anywhere near the reaches of Amazon and Googles simple database solutions. They didn't put all the time and effort into making those solutions because SQL scales well, they put the time and effort into it because SQL (and relational databases in general) do not scale at all past a certain threshold. Relational, partitioned SQL is for small to medium sized companies. If you're one of the big boys good luck keeping SQL up to speed with any type of real usage/growth.

I'll let Oracle and their customers running Oracle RAC that the "big boys" can't run relational SQL databases and that it is only for small to medium sized companies.

The "big boys" have been solving this problem for a lot longer than Amazon and Google have, so to appeal to their authority and the "time and effort" they put into making this product as proof that SQL doesn't scale is ridiculous. On the low end, there's sharding, and on the high end, there's scalable clustered SQL systems like Oracle RAC and IBM DB2 ICE. Making broad statements about some overall lack of scalability of SQL speaking from your MySQL and/or Postgres experience makes you look a little underinformed, when there are enterprise class solutions to SQL server scalability problems from major vendors, on top of the roll your own solutions you can do by doing partitioning / sharding.

On top of that, consider that database servers are optimizing for multi-core systems with things like parallel index scans, breaking up single indexes and joins into sub-processes, dispatched to different cores. This kind of scale up only serves to complement the scale out provided by sharding.

People who say that SQL is dead are just bored. Get over it.

PS. Go check out the TPC benchmarks for the biggest and baddest SQL servers you can buy to see how far people are scaling SQL up, and then explain why a few shards of those "big boy" SQL servers can't handle the load of any "real web application". Or go read the MySpace case study.

Re:Wow

[BasharTeg]

You're right, I did have those confused.

So it sounds like you have hosted virtual servers and some hosted SAN storage. That's cool, and it's a smart way to do business.

It's only when people call it "cloud" and act like it's something crazy and new beyond a combination of virtualization and SAN storage, managed by someone else just doesn't seem like it's worth all this hype.

I have a Vmware ESX cluster. I have an EMC SAN. They're supported through contracts with my reseller. When there are problems, the high availability features of my ESX cluster and my EMC SAN protect me. When I need to add storage, like the 15TB of storage I just added, I add storage. When I need to add another ESX node to expand my CPU or memory availablility, I add another ESX node. Each time I do, my HA gets a little better too. I use pre-built virtual machines ("appliances") for certain things either from vendors or that I build myself, just like the images Amazon offers.

If you can find someone that hosts this for you, that's great. It would probably save a lot of headaches that people with real equipment on site have to deal with. There's probably quite a bit of cost savings associated. My only point is, this isn't a massive shift of computing as we know it, it's just a turn-key solution made out of existing parts. Nothing wrong with that under you hit the vendor lock-in part.

When you start replacing SQL with Amazon's SimpleDB or Google's database, just so you can have a hosted virtualization/SAN solution, you're probably locking yourself in too much. I think your use of Amazon's service makes the most sense, because you're taking advantage of the hosting without the lock-in.

OK, but let's look at the big picture

[dedazo]

Comparisons are OK, but let's look at reliability. EC2 is not the same as S3, but the recent fiasco with S3 and SQS should give people pause before considering using any other Amazon cloud services. Two of my clients were hit with this over the weekend.

I don't know what kinds of volumes (traffic and hosting) Google AE is handling at this point, but at this point I think I would trust Google more than Amazon. One of the issues with the S3 downtime for many people was the fact that Amazon itself (and all its properties) continued to run perfectly while all the sites that hosted images and other content with them failed. Does Google use its own infrastructure to host AE? I don't know, but if they do I'd trust them a hell of a lot more than AWS.

At this point I'm thinking I'm not going to recommend AWS anymore.

EC2 is pretty sweet

[donnyspi]

Just get the EC2 and S3 plugins for Firefox and it's really easy to fire up instances and manage them. Sure, there's a learning curve, but once you really get it, it's awesome.

Missing the point?

[DragonWriter]

Sure, the services allow you to pull CPU cycles from thin air whenever you need to, but they can't solve the deepest problems that make it hard for applications to scale gracefully, Wayner writes.

AFAICT, they aren't intended to. The deepest problems are software problems for which there is no general solution, only problem-specific solutions for each particular task; what they are intended to deal with is the hardware problem that having a scalable software solution is of limited value if you have a fixed pool of hardware and have to go through disruptive upgrades when you expand that pool of hardware (and deal with the associated capital costs.)

Cloud computing services are, largely, tools to help dynamically "right-size" hardware, changing it from a capital investment that requires predicting the future well to plan right to an operating costs that can be quickly adjusted based on changing needs. Complaining that they don't solve the fundamental problems of software scalability seems to be missing the point.

Re:Missing the point?

[Teilo]

Yes, this whole article, I think, misses the point. The cloud, by its very nature, forces you to develop a solution that is intrinsically scalable. It doesn't develop it for you. Who the heck imagines that it is the responsibility of a hosting provider to make your platform scalable? Give me a break!

EC2 is not your typical co-hosting service, nor should you ever treat it like one. To properly implement a platform upon it, you, the programmer / admin need to implement machine images which have the ability to come up, plug in, and start handling requests - all without further intervention. It's a lot of work to get to that point.

No cloud service should be judged by how much engineering this requires, because it's not their responsibility. You can run crap on the cloud, and that's not the cloud's fault. But - you do the hard work first, and the cloud opens up a whole new world. Imagine - the ability to automatically scale your capacity up or down, based upon load. Pay only for what you use, and not a penny more - something that traditional colocation/hosting cannot possibly give you.

I don't know about you, but if I were running a .com start-up, with the potential for huge growth, I would really the thought that I am ready for almost any amount of traffic, no matter what.

Amazon EC2 wins

[orionr]

I run a small startup in the Boston area and have been using Amazon EC2 (plus S3, SQS, and the rest of the AWS family) for the last year. It's worked for us like a champ. A little downtime in the beginning plus some S3 outages, but with the right backup, failover, and restore procedures in place we've gotten reasonable uptime.

The big requirements for us were the following:

1. Ability to move our website (and code base) elsewhere if needed. Could be in-house, to another cloud provider, etc.
2. Minimize up-front cost and allow for massive scaling if needed
3. Cost competitive servers/computing over time
4. Cost competitive storage/disk over time

App Engine fails the first criteria, since (at least currently) you can't build a BigTable application on anything but Google App Engine. "Cloud computing" in general beat out traditional hosting on the second, third, and fourth points. I hadn't checked out GoGrid or AppNexus at the time, but other competitors (Sun, etc.) couldn't match Amazon's price-performance specs.

So, with all of those requirements, Amazon EC2 won out and I'm a happy customer.

Amazon does not automatically scale

[slashkitty]

I'm not familiar with all of them, but with amazon's service, it doesn't "spin up more servers to handle demand" by any stretch of the imagination (unlike what the name infers). You'd have to build an application that does this. Sure, it makes ordering and setting up new servers easy, but it still has to be done by your program. With google's system, there is no need to even worry about scaling up, because it just looks like one system. Unfortunately, google's system is way to limited for anything but customized, simple db apps. I can't wait for it to expand it's feature set.

Moving to ec2

[snuf23]

The cost analysis was really what did it versus our managed hosting plan (1/10th the cost per month). Auto scaling and healing of the application cluster was also a benefit. To scale with a traditional host meant getting locked into a contract for the added server(s).

One thing about ec2 is that it forces you to use best practices for disaster recovery. Instances don't commonly just "disappear" but you need to plan for it. Well tuned ec2 images can have your site up and restored from backup automatically within minutes.

ec2 / s3 is far from perfect and certainly won't meet everyone's needs. The downtime s3 has seen (like last weekend) would be devastating to some businesses. Of course even with a traditional host you may have downtime due to truck crash or other random act.

Re:The definition of cloud computing is still vagu

[hedronist]

So you're saying you fail Rowell's Extension to Einstein's Test of Comprehension? Sad, sad day.

Einstein: You do not really understand something unless you can explain it to your grandmother.
Rowell's Extension: You totally don't understand something if you can't even explain it to a bunch of geeks. Also, they'll probably laugh at you.

Sound familiar?

[psmears]

From the article:

...any Web site filled with an endless stream of mostly forgettable comments trolling for reactions from the rival fans

I can't think of any site to fit that description...

Re:The definition of cloud computing is still vagu

[chabotc]

I think the best way i've heard it explained is:

"When details of implementation are sufficiently hidden away that you no longer have to think about them, people often draw a 'cloud' around it, just like you do with the internet where (most of us) don't have to worry about all the wires and the protocols but it's just there, and it just works.."

Cloud computing is trying to draw the same cloud around.. computing (resources), you don't have to worry about connectivity, electricity, how to make db's and file systems scale across systems.. it's an abstract cloud that's just there without having to worry about it.

Re:The definition of cloud computing is still vagu

[Bazzargh]

'the cloud' is old networking/telephony terminology. Describing interconnection of two sites, you'd diagram the systems at either end, and their local links, but once the links enter the network you don't know or care how the routing happens (generally). This part of the network was 'the cloud' (and was diagrammed as a cloud).

By inference, cloud computing would be where you know the computation is happening somewhere on the network, but you neither know or care exactly where.

See this thread back in 1995 -
http://groups.google.co.uk/group/bit.listserv.techwr-l/browse_thread/thread/d6384bd640275c43/14da0963ed1c294a?hl=en%0Eda0963ed1c294a

Or the first diagram in RFC 1587 (1994):
http://rfc.dotsrc.org/rfc/rfc1587.html

I joined a telecoms company the year before that and the term was in use there, can't vouch for earlier.

Re:GoGrid Beta

[HighTechDad]

Don't worry, I'm pretty thick-skinned. You have some valid questions that others probably have so I'm glad to help in what little way I can.

Currently you can't make your own images, but that is a feature request that is a common one. We do have other Linux distros, specifically, CentOS. We are working on Ubuntu among others as well.

Obviously, we are slightly smaller in terms of infrastructure compared to Amazon. However, we do have a 20,000 sq ft data center and understand hosting. Other products of our have multiple POPs around the world so things similar to "availability zones" is an obvious extension. Ok, I won't talk about the SLA with the exception of we do pro-actively issue credits. Did Amazon do that with S3 when it went down earlier?

You are right about competition. It's good for Cloud Computing in general and helps to drive standards and maintain good pricepoints. We encourage it. Hopefully we can move a bit more rapidly than the likes of Amazon or Google, but we don't have the vast resources as they do. We are 100+ and growing company (hiring as well!). I love your ideas.

In terms of failover for EC2, we whole heartedly agree in this stragegy. It makes sense. Currently, our persistent storage is tied to machine images but stay tuned on that as well.

Don't forget that $100 trial promo code I'm offering. Just enter "GGMS" in the signup page. Test it out and let me know what you think.