Slashdot Mirror

← Back to Stories (view on slashdot.org)

ATI Distributing Spurious HL2 Benchmarks

Posted by Zonk on from the game-card-makers-get-involved-in-the-process dept.

BatonRogue writes "Apparently ATI provided a few Half Life 2 benchmarks to the press and some websites are actually using the benchmarks for their Half Life 2 performance reviews. AnandTech and HardOCP seem to be the only reputable sources of Half Life 2 performance data as they both put together their own benchmarks representative of Half Life 2 gameplay. AnandTech apparently went through every Half Life 2 level and put together a list of the 11 most stressful levels and then created 5 demos, while HardOCP put together two long benchmarks for their review. AnandTech and HardOCP's results appear to agree with each other, while the ATI-backed benchmarks show ATI with a huge performance lead in Half Life 2. Apparently (according to the AnandTech article), ATI was allowed to make their demos while at Valve before Half Life 2 was released, while Valve would not let NVIDIA remove any data from their time at Valve until the game was released. Politics at work as usual."

40 of 52 comments (clear)

  1. Not at all surprising... by Paladin128 · · Score: 4, Informative

    Gabe Newell seems to have something personal against NVIDIA -- he went absolutely batshit towards NVIDIA, as if it was a personal assault that the GeForce FX 5900 didn't run HL2 well. And yeah, the GeForce FX architecture totally sucks with the source engine. The X800 Platinum is the best at running it, but not by the margin ATI is claiming.

    Not to say that given the chance, NVIDIA wouldn't post absurdly inflated numbers. I still personally favor NVIDIA, mostly because thier Linux drivers are of such high quality. And although ATI's Win32 drivers have improved greatly over the past 2 years, in my experience, they aren't quite up to the level of NVIDIA's. Maybe another year and they'll get there. My biggest beef is the lack of support for older products -- the new Catalyst drivers are good, but drivers for the original Radeon and All-in-Wonders suck. NVIDIA's detonator drivers support everything they've ever made, other than the craptastic Riva128 ZX. I'm still using my trusty old TNT2 -- plays a mean game of Quake3 under Linux.

    --
    Lex orandi, lex credendi.
    1. Re:Not at all surprising... by justforaday · · Score: 2, Funny

      and were those drivers WHQL certified?

      --
      I'll turn into a supernova and burn up everything. Well I'll turn into a black little hole and you'll turn into string.
    2. Re:Not at all surprising... by Fweeky · · Score: 1

      And don't forget that I can even run official nVidia drivers on my FreeBSD workstation. Plays a mean game of UT2k4 under Linux emulation ;)

    3. Re:Not at all surprising... by cassidyc · · Score: 1

      that's because you're a moron

      helps if you use a NVIDIA card

      CJC

    4. Re:Not at all surprising... by jackbird · · Score: 2, Informative
      I support 3D graphics workstations from time to time (dual monitors and/or multiple video cards; the whole 9 yards)as part of my job, and I suggest 3 avenues of troubleshooting:

      1. Get your PCI interrupts in order. There are only actually 4 PCI interrupts (A,B,C, and D) which are assigned to PCI/AGP slots in hardware by the motherboard (and some of which are assigned to onboard stuff like the IDE/USB controller - chack your motherboard manual). If it can possibly be helped, don't have anything sharing the AGP slot's PCI interrupt (requires moving cards from slot to slot in the case). Stay away especially from a sound card and the AGP on the same PCI interrupt - Sound cards like to open up long bus mastering sessions that the AGP port absolutely hates.

      2. Cooling/Power issues. An overheating card or underpowered PSU (or dust-clogged intake) can cause a lot of flakiness with graphics cards that suck up obscene amounts of power. Make sure not only that your wattage is correct overall, but that the 12V, 5v, and 3.3v rails all are capable of the load being placed on them individually. This information can be dug up on the benchmarking sites. One more thing - I once troubleshot a problem like this that came down to a failing fan inside the PSU - any funny noises in the case?

      3. Try a total driver reinstallation using one of the 3rd-party driver removal utilities like Detonator Destroyer (now deprecated in favor of something else - I haven't needed to do this in a while). Oh, and never, never listen to windows update when it tells you you have an Nvidia driver update - it'll roll back the version to something over a year old and probably hose the driver in the process.

  2. Always question PR benchmarks by obsid1an · · Score: 1
    The ATI PR benchmarks are crap, as should be expected of any benchmark in a PR release. The thing that sucks is ATI is distributing demos made by them for people to benchmark off as if they are good overall representations of gameplay which clearly isn't true. Also the fact that ATI was even able to make these demos ahead of time while others were denied seems wrong.

    On a side note, the NVIDIA SLI PR benchmarks were actually fairly close to those done by Anandtech.

  3. Re:What's the point? by obsid1an · · Score: 1

    I'm not sure you want to compare Halo 2 and HL2. I too was very skeptical of Valve since they have never made their own engine from scratch, and they really only have one solid game they've sold. However, HL2 has sold me. The game really is very good and most likely the best FPS of the year.

  4. so? by Anonymous Coward · · Score: 1, Funny
    AnandTech apparently went through every Half Life 2 level and put together a list of the 11 most stressful levels and then created 5 demos, while HardOCP put together two long benchmarks for their review.
    Anyone know if Anandtech is hiring?
  5. Could we see some reasonable cards reviewed? by Qrlx · · Score: 3, Insightful

    Could we have a review for the common man? The one who can't afford to spend his rent check on a $600 video card?

    1. Re:Could we see some reasonable cards reviewed? by cassidyc · · Score: 1

      They become resonably priced when teh next generatino comes out, or you could always look towards the mid range card.

      Hell I run HL2 on my Geforce 5 (I forget the exact model, but it's not the best in class) and it plays nicely indeed.

      I'm not that concerned that it doesn`t run at 100 frames/sec, or that there is another card that beats mine by a whole 5 frames/sec.

      Which, by the way, is about the margin of difference, in real terms, of the two companies top line cards.

      CJC

    2. Re:Could we see some reasonable cards reviewed? by Lord+Bear · · Score: 2, Interesting

      My 'common man machine' (2.8Ghz P4, 1G RAM, ATI Radeon 9600 pro) gets between 30-70 fps depending on the stituation. It typically sits a 50-60 when not much is going on and drops down to about 30 in a heavy firefight. Its very playable.

      I'm using these settings: 1024x768, all settings at high, 2x AA, 2x Ansio.

      In other words it runs a heck of a lot faster than doom 3. :)

    3. Re:Could we see some reasonable cards reviewed? by Qrlx · · Score: 1

      Thank you. I had heard rumors that a 9600xt was good enough to play HL2, I'm becoming more and more convinced that that's the case.

      Aside from DOOM III, which is a special effects tour de force, sort of a advanced pixel shader proof-of-concept if you will, I can't see any reason to spend more than $150 on a video card right now.

    4. Re:Could we see some reasonable cards reviewed? by snuf23 · · Score: 1

      I've noticed that newer games are starting to increase the usage of DX 9 features but it's mostly in the form of additional eye candy. So no they aren't necessary - but the visual experience in HL 2, Medal of Honor Pacific Assault, Doom 3 etc. is enhanced greatly by being able to play at high res with AA and Aniso on. I find the experience much more absorbing especially combined with surround speakers or a good headphone set.
      It's not necessary - but these new cards bode well for pc gaming in the future. They are that imnpressive.
      The next six months will see high volume shipments of the mid range and lower end cards as well as price drops. And concurrent with that - more games released that can actually push these cards to the limit.

      --
      Sometimes my arms bend back.
    5. Re:Could we see some reasonable cards reviewed? by Qrlx · · Score: 1

      Well I hope you're right. I would love to be able to get more card for my buck. But a decent video card from three years ago (say a Ti4200) is still around the $100 dollar mark. I would have thought that would be like $30 by now.

      The "entry" dx9 cards like the nvidia 5200 and the radeon 9000 have far, far less horsepower than the previous generation dx8 cards like the Ti4200 or the Radeon 8500. It's kind of disappointing, and it feels like we're being taken for a ride.

      Did top of the line video cards used to cost $600 when they came out? That seems like a new phenomenon to me. They've found a way to make a computer cost $5000 again, this wasn't the case a few years ago.

    6. Re:Could we see some reasonable cards reviewed? by PixelSlut · · Score: 1
      In other words it runs a heck of a lot faster than doom 3. :)

      This is partly due to the fact that ATI has historically had crappy OpenGL support in their drivers. They claim to be working on a new OpenGL implementation now, which is good news for everyone I think. I'm using NVIDIA hardware primarily because they support OpenGL well and they support Linux well, but I'd love to see some real competition for them in this area.
  6. The AnandTech review goes down to 9700 Pro by Kelmenson · · Score: 1

    And includes 2 other sub-$200 cards, the 9800 Pro and 6600.

  7. ATi's results are spurious for not sucking? by neura · · Score: 1

    I think what is spurious here are the claims that ATi is releasing false data. The poster specifically states that Anand went through and found the parts of the game with the heaviest load and based their tests on that. If HardOCP's tests give similar results, one could safely assume they did something at least similar.

    I don't see ATi's benchmarks as "spurious" simply because they're different than someone who benchmarked only the worst performing parts of the game they could find. >.>

    Would it not be more accurate to benchmark the game as a whole?

    If anything, I'd consider any results that consist solely of the lowest performing parts of the game to be "spurious".

    donut hat

    1. Re:ATi's results are spurious for not sucking? by Mike+Hawk · · Score: 3, Insightful

      I'd consider any results that consist solely of the lowest performing parts of the game to be "spurious".

      Why is that? They could go into a crouch-high pitch black corridor and record the FPS there, but that wouldn't be interesting. The worst situation that the user can expect on each card IS very interesting to the consumer. Even if the worst spot for each card is different, its still very interesting.

      For example, if one card consistently runs at 30fps at the highest detail setting no matter what, and another runs at 50 most of the time, but drops to below 10 during every bossfight, which is the better buy? I dont know, thats up to the consumer. But this information is very relevent.

      It may not be "scientific", but the consumer isn't buying an abstract number on a chart.

    2. Re:ATi's results are spurious for not sucking? by Paladin128 · · Score: 1

      No -- actually testing under the heaviest load is the best way to measure performance. The card that has a good framerate at the heaviest loads will be able to handle the lighter with ease. Who cares if you can get 500FPS when there's nothing on screen if you can only get 20 under load. I'd frankly rather a consistent 60FPS across the whole game rather than that kind of swinginess. For that reason, it's similar to benchmark based on glxgears -- so geometrically simple, there's nearly nothing for the GPU to do.

      --
      Lex orandi, lex credendi.
    3. Re:ATi's results are spurious for not sucking? by metalhed77 · · Score: 1

      The worst performance is THE performance. Generally in these fast paced games the worst performing parts are the only parts that matter. The last thing I want is to drop down to 10 FPS when the going gets rough.

      --
      Photos.
    4. Re:ATi's results are spurious for not sucking? by obsid1an · · Score: 3, Informative

      You don't see a problem with a company picking and choosing which parts of a game perform better on their card and then releasing benchmarks saying this is how the whole game performs?

    5. Re:ATi's results are spurious for not sucking? by Seahawk · · Score: 1

      I think you are missing his point!

      HardOCP and Anandtechs performance figures is just as bad as ATI's as they took some of the most stressfull levels.

      A proper benchmark should use all the game, or a couple of levels that represent the average of the game.

    6. Re:ATi's results are spurious for not sucking? by obeythefist · · Score: 1

      I don't see a problem with it, but I sure as heck wouldn't buy a video card based on that "information".

      I suppose if you consider the general public, however, it is very misleading for the company to make those specific examples and then attempt to pass them off as an overall performance indicator. Give companies an inch and they'll take a mile of course. When advertising first kicked into swing, and there wasn't anyone to regulate the industry, companies used to tell all sorts of lies. About how tonics cured diseases, increased lifespans, "Reduced tired blood" and all sorts of other baloney.

      Some people might say that anyone who believes those adverts deserves to suffer because they've lost money on an inferior product. But there's a responsibility that we all share to keep business honest. So we shouldn't put up with companies trying to mislead consumers this way. The behaviour of ATI and Valve with regards to HL2 has been deplorable.

      --
      I am government man, come from the government. The government has sent me. -- G.I.R.
  8. Most important by Schemat1c · · Score: 2, Insightful

    Here's my benchmark for HL2:

    It's a lot of fun to play.

    Isn't that the only one that really matters?

    --

    "Nobody knows the age of the human race, but everybody agrees that it is old enough to know better." - Unknown
    1. Re:Most important by Llevar · · Score: 1

      Isn't that the only one that really matters? Not if you can't actually play the game because your video card gives you a steady 10 frames per second.

    2. Re:Most important by parliboy · · Score: 2, Insightful

      If you're benchmarking HL2, that's relevant.

      If you're benchmarking video cards, it doesn't mean a damn thing.

      --
      "You're never ready, just less unprepared."
  9. Still doesn't answer my question by Prien715 · · Score: 2, Informative

    Maybe there exist people who don't buy a graphics card every year. Even the worst card gets ~54 FPS which is definitely playable. Past that, it's really academic or else you're looking to the future.

    I have a Geforce2 GTS w/64 MB of memory and an Athlon XP 2200+ w/512 MB memory. I can play UT2004 fine with 32 players on any given map without frame loss (lowest detail settings, but the framerate's smooth, which is what's really important). Doom3 is a no-show. Would I have to fork over cash for a new GFX card for this game to play reasonably well or not?

    --
    -- Political fascism requires a Fuhrer.
    1. Re:Still doesn't answer my question by Nos. · · Score: 1

      I guess it depends what you consider reasonably well. I can't say for sure, but my setup is an AMD 2600 1G Ram and a GFX5600. Running at 800x600 with basically everything on highest details (no AA). Runs great... couple times I've noticed a bit of lag, but nothing that really interfers with game play. I haven't actually checked FPS yet.
      With what you've got, I am guessing you could run at a reasonable framerate at 640x480 with low details, but its hard to say.

    2. Re:Still doesn't answer my question by obeythefist · · Score: 1

      Every 18 months if you're serious about playing these games, unfortunately. It's not necessarily easy, and cutting edge gaming isn't for everyone. This is what the console market is for. Understandably, you can't have the amazing graphics of Half-Life for the low low price of a console. PC gamers pay for the hardware because they want the performance and the quality. Much like hi-fi enthusiasts spend loads of cash upgrading their home cinema. Got a family, not a lot of money? Those things are more important than gaming or buying new subwoofers for your hi fi, so you adjust your financial properties.

      nVidia and ATI wouldn't be the companies they are unless people *wanted* to spend money on those cards to play games like HL2.

      You could probably play HL2 with very low settings. It wouldn't look very good. Your CPU will also hold you back. That gear is very, very dated though. You need to define for yourself what you consider "reasonable" quality to be. I remember playing Quake with some pretty low graphics settings back in the day. It ran, it wasn't as spectacular as it could have been, but I played it.

      As a side note, the reason DooM3 is a side note and HL2 wouldn't be directly relates to the graphics quality. HL2's graphics aren't particularly revolutionary. Most textures in the game are only 2 passes for the shaders to work on. DooM3 has about 3-5 passes from bumpmapping and so on for the dynamic lighting in the game, which makes truly breathtaking visuals (if a bit dark if you don't have many light sources). But you pay for the amazing capabilities - you need a computer that can handle it. You won't get Quake running on an XT, and you won't get DooM3 running on a 286. Sometimes to move forward, well, we all need to move forward.

      --
      I am government man, come from the government. The government has sent me. -- G.I.R.
  10. The benchmark I want to see by MrHanky · · Score: 1

    I want to see a benchmark of the newest cards (hey, a GeForce 6800 non-GT is quite affordable) on somewhat older CPUs as well. It's great to see that an Radeon X800 or GeForce 6800GT rock at high resolutions with P4 @ 3.2 GHz, but what about those of us stuck with Athlon XP 1600+ and a motherboard that can't take much more. Can we play HL2 with the latest offerings from ATI and nvidia, or should we just stick to our old Radeon 7200s and play Tuxracer instead? A modern GPU should offload the CPU quite a bit, but it's impossible to tell just how much from the benchmarks Anandtech et al do.

    1. Re:The benchmark I want to see by drinkypoo · · Score: 1

      AFAICT the 6800 only comes in GT or Ultra flavors. The cheaper one is the Ultra 128MB, at $305 and up. For $271.39 you could get a Gigabyte GA-K8NS with an Athlon 64 2800 and 256MB of PC3200 DDR. Maybe your money would be better spent on a mb/cpu/memory upgrade.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    2. Re:The benchmark I want to see by MrHanky · · Score: 1

      Yes, that's a possibility. But I think a new motherboard also demands a new PSU (don't the newer Athlons need the same plug that was introduced with the P4?), so in the end the cost will be about the same. And my graphics card still sucks -- it's 1st gen Radeon, halfway between GeForce256 and GF2. It's not going to let me play Doom3 at all, but there is a chance that HL2 will load...

      The problem is, I have no idea which upgrade is best for my immediate gaming needs. However, I do know that none of the options are very future proof -- AGP is on its way out, and so is socket 754. But then again, how many times have you been able to upgrade the CPU significantly without also changing the MB?

    3. Re:The benchmark I want to see by drinkypoo · · Score: 1
      If your power supply provides enough power to run an Athlon 64, which IIRC doesn't really consume any more power than an Athlon XP and might even consume less, then you can get a little adapter widget which turns one of your drive power supply connectors into the ATX 12V connector. One of my motherboards actually came with one just in case my power supply didn't have one (it does.)

      As you say, the coupling of CPU to motherboard is not especially troubling. The only reason to even put it in a socket is being able to replace it if it fails, and to prevent a stockpile of slow motherboards from existing. This was significantly less true before motherboards came down to about a hundred bucks for a quality product. Granted, it's probably time for you to upgrade your video card, but it's clearly also time for you to upgrade your processor... :)

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    4. Re:The benchmark I want to see by snuf23 · · Score: 1

      It really is a disappointment that sites don't tend to do cpu scaling tests. Tom's Hardware will run them once in awhile. I think it's mostly because they are very time consuming to perform.
      If I were you - the upgrade answer would depend on your budget. Create an upgrade plan and move towards it. If you don't plan on going to a PCE motherboard in your upgrade then it could be worthwhile to pickup a nice AGP video card now. You will get a boost and you can take the card with you when you upgrade.

      --
      Sometimes my arms bend back.
  11. No nVidia Conspiracy by delus10n0 · · Score: 1

    NVIDIA did tell us that honestly their limited time at Valve wasn't solely dictated by Valve. Valve extended an invitation to NVIDIA and things just ended up working out so that NVIDIA only had two (albeit long) days with the final version of the game.

    Way to start a spin, Slashdot crew.. Michael Moore would be proud!

    --
    Not All Who Wander Are Lost
  12. Re:What's the point? by Vengeance_au · · Score: 1

    Just a small quibble - Valve didn't build the engine from scratch - note the splash screen at the start talking about the Havok engine. Also referenced here. And considering the high reliance on physics in the game, its not something to overlook. Now, back to watching the clock until home time and getting back in to HL2 ;-).

  13. Re:On the tit by snuf23 · · Score: 2, Informative

    Well... if performance is the question - the latest and greatest cards from Nvidia and ATI are both pretty sweet. Sure Nvidia has a lead in some games and ATI in others (see Doom 3 and Far Cry) but I can't see anyone giving either of the cards a bad review. Mostly all they can do is highlight some of the not so good - i.e. 6800 Ultra is power hungry, ATI drivers are a bit wanky.
    Reviews for anything other than the top of the line are mostly "bang for buck" reviews and in this generation the numbers from ATI and Nvidia are highly competitive.
    As I recall the Nvidia FX series got some well deserved flak but the last set of outright suck video cards has to have been the Cryo series chips - ack.

    --
    Sometimes my arms bend back.
  14. Re:What's the point? by syrion · · Score: 1

    Havok isn't an engine; it's the physics system. Source is the engine, and was, in fact, created from scratch. (Painkiller is another game that uses Havok physics, but not as effectively as Half-Life 2.)

  15. fairness by d_strand · · Score: 1

    ATI's demos may be biased (of course they are), but dont claim HardOCP's are any more dependable. They used an overclocked Nvidia card for example (and Kyle is a well known Nvidia fanboy)

    And Anand looking for the most stressing parts of the game? Well it seems to me the slowest parts might just as well be CPU-limited as graphics limited. And [H] benching an entire level? Come on! You have to choose GPU-stressing parts, if you bench an entire level your average scores will be much more even since the cards are good at different things.

  16. Re:On the tit by Goosey · · Score: 1

    "All these little hardware sites"

    Sorry, I really can't consider HardOCP or Anandtech to be 'little'. If these are little hardware sites, what is your definition of a big one?

    --
    --- "End Of Line" - MCP