The Amazon Technology Platform

Don420 writes "Jim Gray has an interview with Amazon CTO Werner Vogels for ACM Queue. It is filled with a lot of details about the Amazon architecture that we have not seen before: 'If you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.' But also quite a strong statements about developing software at Amazon: 'Developers of our services can use any tools they see fit to build their services. [...] Whatever tools are necessary, we provide them, and then get the hell out of the way of the developers so that they can do their jobs. [...] Developers are like artists; they produce their best work if they have the freedom to do so, but they need good tools.'"

Comments

100 Services ?

[jonv]

'If you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.'

and this a good thing ?

Re:100 Services ?

[simonjp]

It didnt say that it collected 100 pieces of personal data etc from your computer, more likely its all internal requests :) I mean it usually provides suggestions on your past orders, what pages you've recently viewed in the system (which isnt exactly personal data being collection IMO as its all "internal" to Amazon.

But the question is does this make for a better shopping experience?

When I've used Amazon it's always been helpful.

However, Amazon UK seem to be a little less reliable in shipping recently...

Re:100 Services ?

[a16]

I think that was the OP's point: one stinking page sucks down a shit load of badwidth for information that's not really needed or it's not need every time. It's kind of sickening when I bring up a custom page and it's hitting servers and using bandwith for information that's not updated that frequently.

It is worth noting that in TFA he mentions how services are cached, therefore presumably (for Amazon) the service delivering the data for "today's popular items" or similar to the main page wouldn't need to transfer it's full result, this would be cached where possible.

Good Tools?

[Anonymous Coward]

My father always said it was a poor craftsman who uses his tools as an excuse.

Re:My question is

[Anonymous Coward]

I worked at Amazon. It was a nightmare. And I can say this: absolutely none of the above. Amazon tools are largely undocumented, not requiring any kind of standardization on platform/libraries/processes, are totally heterogeneous and are often scary bits of random code floating out in the ether.

This attitude sounds like it's great, but something that this attitude absolutely encourages is non-standardization and a huge lack of planning for the future. They encourage you to go 'just do it' - and then 10 different people have done the same thing because they had no idea that someone else did it first. Or more likely, they looked at what someone else did, saw the horrible documentation and support and decided it was easier to do their own thing.

Re:My question is

[Anonymous Coward]

I currently work at Amazon, and typically most projects aren't done using obscure technologies, they're usually pretty standard technologies. However, there are a *lot* of standard technologies, and every project really does use a different one. So if you're on one team and you're using Java/Spring/Hibernate, you may move to another team and be using Perl, or be using Java/Struts/SomethingElse.

Good documentation, however, is something that is definitely *not* required and something that I hope will change in the near future at Amazon. Without that (and dedicated support teams), you end up with a bunch of developers doing tech support and maintenance work and never getting any actual development done.

Re:Thanks

[stlhawkeye]

Or maybe I got you wrong and you meant that developers are like artists: Poor, starving, living for their work and only valued once they are no longer available.

Move away from the coasts. I make $75,000/year working 40 hour weeks. I'm not on-call, have flex hours, get 3 weeks of vacation, and unlimited sick time. Quit working for IT sweat shops. Move somewhere where family time is valued and it's impossible to hire people unless you are willing to give them that flexibility. I've been through four employers in the St. Louis area and been able to land jobs with a deal akin to this one at all four. Developers are "poor"? No. Elementary school teachers are "poor." Starting salary for a developer in a low-watt market is close to $40K without a degree. That's not "poor." That's not starving, and that's not living for their work, unless by "living for their work" you mean that you're expected to show up on time and do your job.

Re:My question is

[Anonymous Coward]

I started at AMZN as a dev - hoped to do development of the kinds I never did. And I'm not saying I didn't. However, in retrospect, what I think fucked me up the most is my inability to estimate the time needed between being on a pager 24 hrs/day, 6-7 days out of every month and the time needed to maintain/fix old legacy (perl-based mostly) code - which left me with little time to do any project work. I suppose I've got no one to blame but myself for not knowing how to predict time required for projects, however, one thing I can say is that the environment at AMZN, especially for devs, is untenable at best (hostile, at worst, I'd say). Now, I'm no longer there - I get to sleep at night and not walk in to work like a zombie, I have the weekends/holidays to myself and my family, and my work is of higher quality, and on time. I have AMZN to thank for teaching me a lesson - 1) know thy limits between pager-duty/dev work/maintenance work, 2) be ultra-pessimistic when giving time estimates, 3) say 'no'/complain a lot/cover your ass.

If you do that well, you will survive at AMZN. To get ahead, you need to be an illusionist.