Slashdot Mirror


Amazon Makes It Almost Impossible To Calculate Their "Virtual CPU" Equivalent (informationweek.com)

dkatana writes: AWS started out defining its virtual CPUs as being composed of EC2 compute units, or ECUs, which it defined as an equivalent to a physical Xeon processor. However, a virtual CPU now looks suspiciously variable... A virtual CPU is whatever Amazon wants to offer in an instance series. The user has no firm measure to go by. From the article: [B]y doing a little math, you could actually compare what you were getting in virtual CPUs in EC2 versus Azure. Also by doing a little math, you knew how to compare one Amazon instance to another based on the ECU count in each virtual CPU. Microsoft didn't look too bad in the comparison. That is one of the casualties of the nomenclature change. I have searched for updated information on how a virtual CPU is measured and found nothing comparable to the definition of the 2012 ECU measure. I have questioned Amazon representatives three times between Oct. 27 and Dec. 21, and don't have much of an answer."

56 of 114 comments (clear)

  1. Totally true by Anonymous Coward · · Score: 1

    In fact Amazon doesn't know themselves... I have asked them to compare to the i7 series and they simply don't have a clue.

    1. Re: Totally true by mSparks43 · · Score: 2

      when I did some testing a while back. $20 a month would get you processing of a crappie mobile you could buy for $50. Amazon has never had good cpu.

      so it's probably more that they don't want to say.

      anyone who's benchmarked them against alternatives has nothing good to say.

      http://openmymind.net/Why-I-Di...

    2. Re: Totally true by lgw · · Score: 2

      That's a good link. My real complaint about AWS is what's mentioned at the end:

      It also feels like a lot of services are stuck at version 1.0, lacking that polish and continual improvement

      This is what annoys me. SQS is a good 1.0 version of message queues, but the features are the just above the minimum you could possibly call a message queue. DynamoDB is a good 1.0 version of a NoSQL DB, just above the minimum you could possibly call a NoSQL DB.

      These services are years old, but look like what most software does at version 1.1 or so: minimal features, no glaring bugs, but nothing great either.

      I can't say anything about EC2 CPU performance, as I've never benchmarked that myself, but aren't they just (mostly older) Xeons? The way the T2 instance is described makes it sound like the cores are oversubscribed, but I haven't heard that about the other instances.

      In any case, unless you're CPU-bound, it doesn't matter. I'm more concerned about price for the memory I need. For compute-intensive jobs, EC2 Spot is cheap if you're really fault tolerant.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    3. Re: Totally true by mSparks43 · · Score: 1

      each insrance is one of who knows how many virtual machines running on old xeons.

    4. Re: Totally true by lgw · · Score: 1

      Right, but they clearly promise you a number of cores. The easy assumption is that the cores aren't oversubscribed, and none of the docs suggest they're oversubscribed, except the docs for T2 (seems like it must be). Can't tell for sure, of course, unless someone can find something definitive from Amazon.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    5. Re: Totally true by mSparks43 · · Score: 1

      what they promise is by "virtual core".

      that has no L1/2/3 cache.
      and will be balanced on the machine based on actual cpu cycles (my "real" 3.5ghz cpus can "turbo mode" to nearly 7ghz on heavy loads - Linux cpu-governor)

      it's an extreme version of the old hdd spec lies.

      http://tiemensfamily.com/TimOn...

      300...

      compared with 10s of thousands for a "real" core.

    6. Re: Totally true by lgw · · Score: 1

      I can't make much sense of your post, and the linked article is a bit confusing as well. But if the claim is that the 2 cores from an m1.large are about what you'd expect from 2009, I can totally believe that - the m1 instance is pretty old. I'd expect them to be sub-2-GHz Xeon cores. Xeon is usually damn slow compared to the consumer cores from the same year, and it's not sensible to compare Xeon performance to consumer performance (or, at least, it's Intel's brain damage, not Amazon's).

      From the docs, m4 is 2.4 GHz Haswell, and m3 is Ivy Bridge - they don't say the frequency so probably pretty low.

      As far as cache, I don't know what hypervisor Amazon uses. I know VMware (which I'm sure they don't use) protects the CPU cache as long as you don't oversubscribe cores. "World switching" a core, as they call it, is the most expensive operation you can do in a hypervisor, so any hypervisor worth a damn will try to avoid that.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    7. Re: Totally true by godefroi · · Score: 1

      I believe they use Xen.

      --
      Karma: Poor (Mostly affected by lame karma-joke sigs)
  2. Re:Quit whining by pijokela · · Score: 2

    That is not really possible. Amazon is getting to a monopoly position if you want to use all the modern Cloud stuff. Sure, you can get boxes from many different providers, but AWS has a ton of other services that you cannot buy from others and even more importantly, all the 3rd party Cloud services are running on AWS so they are faster if you are on AWS too.

    I would not be surprised if in 10 years Amazon would be a verb for running server software like google is now for search.

  3. Mostly irrelevant to most people by AmiMoJo · · Score: 3, Interesting

    Most people don't care about the exact performance, so it's not with spending the money and effort to precisely define or guarantee it.

    Amazon is generic and cheap. Microsoft has really good integration with visual studio and .NET. Those are the factors people choose by.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    1. Re:Mostly irrelevant to most people by Anonymous Coward · · Score: 1

      Considering that most modern CPUs have performance counters and such, there is no excuse not to be able to measure it, particularly when you are charging for it. How big is this house you want to sell me? Well, somewhere between 100 and 300 sq. m., nobody cares, so why are you whining about it, am I right?

    2. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      CPU built-in performance counters are as reliable as the size of the fish catch described by your alcoholic uncle.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    3. Re: Mostly irrelevant to most people by TheReaperD · · Score: 4, Funny

      After rereading my post, I realized I needed to post a clarification for the easily offended: "your alcoholic uncle" is used as a generic example and not referring to the AC's uncle.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    4. Re:Mostly irrelevant to most people by StormReaver · · Score: 1

      This behavior is what I (and others) predicted when this whole "cloud" stupidity began: as providers become more entrenched, the prices and complexity slowly increase. Those who become dependent on this particular brand of idiocy will be taken to the cleaners.

    5. Re: Mostly irrelevant to most people by Dunbal · · Score: 1

      Fuck the easily offended politically correct cunts. They are useless anyway.

      --
      Seven puppies were harmed during the making of this post.
    6. Re: Mostly irrelevant to most people by TheReaperD · · Score: 2

      The uncle clarification was to make sure that you and others knew I wasn't just trying to be a troll. When I get marked as troll, I want to earn it. As far as the counters, it's not that their random, it's that they are optimized to give inflated tallies for benchmarks. So, in some areas, they're going to vastly overstate their performance (and thus, the "fish story" analogy) but, when you actually call on that level of performance, it isn't there. Which areas that do this will vary by manufacturer and model. The companies that try to offer serious and fair benchmarks have had a nightmare time trying to resolve this. They often have to reverse-engineer where each processor is lying, figure out an estimate of percentage it is lying on that particular test and modify the final results by that percentage. Of course, the company that made the chip will say the adjustment is unfair, then they rebuttal, etc. If that alone wasn't bad enough, chips (though this is more common on graphics chips) that are labeled as mid-range are, in fact, higher-end chips that have been stepped down by firmware, that remove the firmware imposed speed limiter for benchmarks only. These are just the examples I know of; I have no doubt there are more.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    7. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      When it comes down to names of CPU calls, I'm in over my head. My former father in law is the engineer, not me. He said the specifics, I just couldn't retain them. But, the short was, when it came to internal performance marks, CPUs lie. Whether that specific function call does (or even can) lie and which CPUs do it, I'm unable to say due to being out of my area of skill. Sorry about the graphics kludge in my statement. I was mixing apples and oranges. I was meaning that both types lie, in their own way, to fool performance benchmarks, not that they had the same type of internal metric. I've followed this nightmare more from the side of the benchmark suites, not from the engineering side.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    8. Re: Mostly irrelevant to most people by TheReaperD · · Score: 2

      Unrelated side comment: Where the hell are all my line breaks going?

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    9. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      OK, you win the internet for me today.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    10. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      First link didn't work but, the second one did. Thank you, that was informative. Sadly, I don't have the background to follow some of it. It's helpful to have this now assuming the manufacturers don't try to game it too much. Of I hadn't already posted, I would mod you up.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    11. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      I found the fix. ...and it sucks. Apparently, Slashdot now carries over your desktop posting preferences to the mobile site. This didn't used to happen. My desktop was set to HTML codes posts. There is no way I'm imputing proper HTML code using a smartphone touchscreen keyboard!

      So, to the fix. You can only do this on the desktop version of the site. There is no mobile accessible setting for this. Login to the desktop site, open any story, then reply to any post. Select options from the post window (not the main Slashdot title bar) then go to the Formatting drop down box in the pop-up window and select "Plain Text" then click the save button at the bottom of the window (you may have to scroll to see it). You will still be able to put in HTML tags but, you don't need to post proper code anymore.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    12. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      Hey, I've been staying analytical and being honest when I'm outside my skill set. The personal attack is uncalled for.

      You don't need to take my word on it for benchmark cheating. A Google search will do. Here's an older one but, it went all the way to a class-action suit that Intel had to settle: Intel lawsuit. The most recent article about CPU cheating was 2009 though GPU cheating is still rampant in the news. On the CPU side they have either cleaned up their act enough to stay out of the news or just haven't been caught. I can't say for sure which.

      Though none of this will be detailed enough for you as your knowledge is more specialized and most news is not going to go to that level. I can't fix that.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    13. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      Today's complexity creates a lot of the trouble in getting reliable performance data. That makes perfect sense to me.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    14. Re: Mostly irrelevant to most people by TheReaperD · · Score: 1

      Sorry. That's the hard part of having a long conversation with an AC.

      --
      "Be particularly skeptical when presented with evidence confirming what you already believe." -
    15. Re: Mostly irrelevant to most people by Cederic · · Score: 1

      I'm not a specialist. I took a career choice to become a generalist, on the grounds it lets me get involved in solving problems with a broader scope and greater scale. I call this fun.

      I have no post-graduate qualifications either. I got on with learning pragmatic useful shit instead.

    16. Re: Mostly irrelevant to most people by BronsCon · · Score: 1

      Of course, your post wasn't at all masturbatory.

      --
      APK quotes people (including myself) without context and should not be trusted. Just thought you should know.
    17. Re: Mostly irrelevant to most people by phorm · · Score: 1

      I try to use the phrase "one's alcoholic uncle" which is less likely to be misconstrued as personal. :-)

  4. Variable Units by TheReaperD · · Score: 1

    I read this a they're worth whatever they say it's worth (at the moment) and you're going to whine and complain about it but, not do anything about it. Until they start loosing a large dollar value of customers over it, they're not going to fix it.

    --
    "Be particularly skeptical when presented with evidence confirming what you already believe." -
  5. Re: Quit whining by t1oracle · · Score: 1

    Google and Microsoft will always have their own clouds. Those are two real competitors.

  6. Re:Quit whining by vikingpower · · Score: 4, Interesting

    Alternative: build your own private cloud out of the smallest servers you can find that still suit your need. I did so: a private cloud on HP Microserver (gen 8). The things consume almost no power when idle. Taken together, they provide quite the computing power ( 64 cores, Xeon E3 ) and quite the storage (32 TB). Cost me around € 450 in electric power per year. Am not dependent on Azure or Amazon. Use no bandwidth when doing cloudy things.

    --
    Religous speak to God. Insane are spoken to by God. When all shut up, one can finally hear Shostakovich in peace
  7. what a surprise by climb_no_fear · · Score: 2

    that the "Cloud" is nebulous....

  8. So just takes the worst measurement you can find? by WoLpH · · Score: 1

    Imho there is only 1 valid measurement for cloud solutions, how much will certain performance cost you. And you can only really find out by testing it for _your_ specific goal. Some apps require more cpu, others more disk, others more ram... there's no single number to indicate your price/performance index.

  9. Re: Quit whining by DThorne · · Score: 3, Insightful

    I'm not sure I'd call it whining - it's a significant investment for many to set up cloud computing the way they need it so I think it's a fair demand that if your gas station insists on measuring it's product in frackles that it give you a fair conversion rate for litres to frackles. My needs would be something predictable like rendering 3d images, so the first thing I'd be doing is measuring render times for the same image there and here. That's easy and relatively cheap to do, and the numbers scale directly, but I can see this being a serious issue for other uses.

  10. If you need well defined performance... by drolli · · Score: 3, Insightful

    then buy dedicated instances.

    I like micro instances/instances which do not occupy full physical processors at Amazon because of availability and price for low-impact/bandwidth applications. For all other use lambda or dedicated instances.

    Virtual CPUs are anyway difficult to asses - to me it may be very relevant to have the 1st level cache of the core which i run on undisturbed by other applications (since changing the cache hits is a big deal for specific numerical problems), and for you 20% more share of the CPU may be important.

    1% of computation time not spend in my task on a physical processor can do as much damage as 50% change in speed.

    A small side remark: the price for the different VCPUs also varies.

    1. Re:If you need well defined performance... by Sarten-X · · Score: 1

      This is exactly the problem, and solution.

      From a computer science standpoint, modern algorithms that rely on single-thread speeds are obsolete and should be avoided for new development. From a sysadmin perspective, CPU speed is the least important metric compared to the rest of the system components. For project management, the system management toolset and feature support should be the prime concern.

      Amazon's never really tried to hide the fact that their hardware is dissimilar behind the scenes. That will impact your computation speed, naturally.

      --
      You do not have a moral or legal right to do absolutely anything you want.
  11. Re: Quit whining by Anonymous Coward · · Score: 4, Interesting

    Microsoft is a joke and since Satya insist on moving everything to India, treating people like shit, and firing everyone after a few years for "IP" reasons, it's a trainwreck of a company that only still survives because of its initial brand-name and market penetration.

  12. Re:How much can you pay? by Anonymous Coward · · Score: 1

    Except there's still a lot of potential market to capture. Microsoft and Google can charge less and get those people before they get locked in to Amazon.

  13. Re:welcome back to the mainframe by Anonymous Coward · · Score: 1

    This is a scam by ISPs. They won't give you a high-bandwidth uploading network connection at your home or office. So you have to pay rent to run some slow computer at some remote facility.

    It's not 1990 anymore... why can't you host a server at your own place.

  14. Re:welcome back to the mainframe by known_coward_69 · · Score: 1, Insightful

    where can't you get high speed connections? NYC and LA are up to 300mbps and higher for time warner. other markets as well.

  15. Re:Quit whining by ShanghaiBill · · Score: 2, Informative

    Alternative: build your own private cloud

    The whole point of doing computing in the cloud is to handle variable demand. I spend 98% of my time writing and debugging code, and only 2% running the final model. But when I run it, I want to do it at scale. On AWS I can rent a dozen K80 GPUs for a few hours a week. There is no way it would make sense to own them, even if I could afford to do that.

  16. AWS by fluffernutter · · Score: 1

    I was considering using AWS... but it seemed to complicated with their calculator to get any kind of estimate of what it would cost me I went with dedicated servers instead.

    --
    Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
  17. Re: Quit whining by NormalVisual · · Score: 2

    And with all the crap they're pulling with the telemetry and whatnot in Windows 10, they seem hell-bent on losing that as well.

    --
    Please stand clear of the doors, por favor mantenganse alejado de las puertas
  18. Re:welcome back to the mainframe by Anonymous Coward · · Score: 1

    where can't you get high speed connections? NYC and LA are up to 300mbps and higher for time warner. other markets as well.

    The first commonly used modems were a *thousand* times faster than that. Connections here in the US are very slow, like the typical 1.5 Mbps T1 or DSL connections in the Seattle area for businesses, but the claim that it takes more than three seconds to transmit a single bit, is a lie.

  19. Microsoft makes it almost impossible to be license by mrmaster · · Score: 2

    Big deal. Try asking azure support about licensing and they will flat out say I don't know. Assuming you bugged them at least a half a dozen times about the same questions beforehand.

  20. Article is wrong on two things by pgn674 · · Score: 1

    > "A virtual CPU is whatever Amazon wants to offer in an instance series." No. The vCPU (Virtual CPU) aspect of an AWS EC2 Instance is the county of virtual cores that are exposed to an OS. In desktop computers, a quad core Intel CPU will appear to have four courses when looked at from inside the OS (my go-to way to count them in Linux is to run top and press 1). A quad core hyperthreaded Intel CPU will appear to have 8 cores. The vCPU metric simply tells you what the OS will show you, and tells you how many processor threads can run concurrently. > "If you deal with server sizing and instance price comparison, then the measure -- previously expressed as an EC2 Compute Unit or ECU -- is kaput." ... "It's the closest thing you'll find to an acknowledgement that ECUs are still in use behind the scenes, but Amazon no longer wishes to define them due to the changing nature of its underlying hardware." Yes, ECU (Elastic Compute Unit) metrics are still used behind the scenes, but Amazon does publish them. Even for new Instances. Check out these URL's: http://a0.awsstatic.com/pricin... http://a0.awsstatic.com/pricin... Of course, this isn't very parsable by human eyes. So someone started an open source project to display this data, and its available at http://www.ec2instances.info/ So yeah, TFA is wrong.

    1. Re:Article is wrong on two things by pgn674 · · Score: 1
      Ugh, sorry about formatting. Here, I think this is right:

      "A virtual CPU is whatever Amazon wants to offer in an instance series."

      No. The vCPU (Virtual CPU) aspect of an AWS EC2 Instance is the county of virtual cores that are exposed to an OS. In desktop computers, a quad core Intel CPU will appear to have four courses when looked at from inside the OS (my go-to way to count them in Linux is to run top and press 1). A quad core hyperthreaded Intel CPU will appear to have 8 cores. The vCPU metric simply tells you what the OS will show you, and tells you how many processor threads can run concurrently.

      "If you deal with server sizing and instance price comparison, then the measure -- previously expressed as an EC2 Compute Unit or ECU -- is kaput." ... "It's the closest thing you'll find to an acknowledgement that ECUs are still in use behind the scenes, but Amazon no longer wishes to define them due to the changing nature of its underlying hardware."

      Yes, ECU (Elastic Compute Unit) metrics are still used behind the scenes, but Amazon does publish them. Even for new Instances. Check out these URL's:
      http://a0.awsstatic.com/pricin...
      http://a0.awsstatic.com/pricin...
      Of course, this isn't very parsable by human eyes. So someone started an open source project to display this data, and its available at http://www.ec2instances.info/

      So yeah, TFA is wrong.

  21. Re: Quit whining by lgw · · Score: 1

    It's odd how much Azure seems to struggle to keep up with AWS. MS has no shortage of cash, and at the very least they could offer "MS SQL instances in the cloud" as cheaply as they needed to go get people locked in, It's not like NoSQL and message queues are rocket science these days, and the AWS versions have fairly minimal feature sets.

    In the old days MS was all about lock-in, and while you never wanted to touch v1.0 of anything from MS, by the time 2.0 came out they were usually ahead of the pack in terms of feature checklist (quality not so much). I guess they've lost that spirit.

    Google spent so long just offering a limited set a quirky services in the cloud. It seems like they're smart enough to stomp the competition: it always baffled my why they didn't. (They certainly know how to operate at scale as good as anyone.)

    --
    Socialism: a lie told by totalitarians and believed by fools.
  22. Re:welcome back to the mainframe by tepples · · Score: 1

    Seattle's zoning law is anti-utility. For acceptable Internet performance, leave Seattle.

  23. Mbps vs. mbps spelling lame by tepples · · Score: 1

    Back when Usenet was popular, members of certain groups in alt. used to call that kind of post a "spelling flame" or later a "spelling lame". If you want to complain about mbps vs. Mbps, please also have something helpful to say about the rest of the post.

  24. Re: Quit whining by Lumpy · · Score: 1

    But they not only measure in frackles but they also break your fuel gauge so you dont know how much they actually put in your car.

    --
    Do not look at laser with remaining good eye.
  25. Re: Quit whining by Cederic · · Score: 1

    (They certainly know how to operate at scale as good as anyone.)

    Operating at scale when you're bespoke from the chip up is easy.

    Well, relatively easy.

    Operating at scale when any twat out there can do whatever the fuck they like to your commodity infrastructure (which must be relatively commodity or you can't sell it) is a very different proposition.

    I'm confident Google could step into that market if they chose, but it's a deviation from their standard operating model.

  26. Re:Quit whining by Cederic · · Score: 1

    We have multiple tier 5 data centres. Amazon/Azure is still useful for dynamic scaling for some of our workloads.

    Some of the more optimised and specialised services can also really exploit cloud pricing. When part of our business can service millions of consumers for a few hundred dollars a year with Amazon we just can't get close to that price point in house.

    But fuck it, The Good, The Bad and The Ugly just started on TV so I'm going to kick back, drink some vodka and enjoy cinematic perfection.

  27. Re: Quit whining by mlts · · Score: 1

    Microsoft survives for one reason: They are so well entrenched in the enterprise. Other than small companies that LDAP can work with, AD will be found as the core authentication and management mechanism of most companies out there.

    Because of this, if MS sees losses on other fronts, they can just ratchet up Windows Server license fees, and still come out ahead, as they have a captive audience.

  28. Re: Quit whining by soulhuntre · · Score: 1

    What? Have you even seen MS's offerings? They can't build DCs fast enough because so many people are signing up. Their stuff is at least in par with Amazon if not better.

    This. All of this. More and more clients who traditionally went AWS (advertising campaign back ends, social media startups and so on) are picking Azure. The AWS tools are just crude in comparison and the Azure offerings are typically more complete, more robust and much better documented.

    3-4 years ago suggest Azure hosting was suicide for a potential contract, these days it is an advantage and makes the guys pushing AWS look like they aren't keeping up. The perception is shifting fast.

    --
    --> Fight tyranny and repression.... read /. at -1!
  29. history repeating again? by Kishin · · Score: 1

    The old VAXen that DEC sold came with a measure called VAX Units of Performance (VUP's). You bought stuff with a certain amount of VUP's rather than raw performance specs. Now Amazon is selling some Amazon-specific measure of performance that benefits them more than us. More signs the cloud is just re-inventing the mainframes and minicomputers of old. Cheaper, faster, more flexible, and so on for sure. Same concepts, though, plus what it seems is the same bullshit. :)

  30. Deliberately obscure? by spud1955 · · Score: 1

    Maybe its all meant to be like mobile phone plans. So complicated that no one can workout which better or cheaper