Slashdot Mirror

← Back to Stories (view on slashdot.org)

Intel Releases 5,000 Pages of Open-Source Haswell Documentation

Posted by samzenpus on from the pages-for-the-people dept.

An anonymous reader writes "Intel has ended out 2013 by publishing 5,000 pages of new GPU documentation about their latest generation 'Haswell' graphics hardware. The new documentation complements their longstanding open-source Linux graphics driver that has supported Haswell HD / Iris Graphics since last year. The new documentation covers the hardware registers and special information for 3D, video acceleration, performance counters, and GPGPU programming."

19 of 111 comments (clear)

  1. I received them earlier ... by Anonymous Coward · · Score: 4, Funny

    ... thanks Snowden!

  2. is it complete? by waddgodd · · Score: 5, Funny

    Does it include APIs for the NSA backdoors?

    --
    Just because you're paranoid doesn't mean they aren't out to get you
    1. Re:is it complete? by TheGratefulNet · · Score: 4, Funny

      what a relief to find that the arguments (of the api calls) only need rot13 applied twice in succession. its not at all cpu-intensive, so that's a relief.

      alarmingly, the result is returned in plaintext. waiting for nsa api v1.1 for the fix to that one.

      --

      --
      "It is now safe to switch off your computer."
    2. Re:is it complete? by RightSaidFred99 · · Score: 2, Insightful

      God, I know. We get it - the NSA is spying on us. All these "NSA turr hurr!" jokes remind me of the "Scary Movie"/"Epic Movie" type pseudo-spoofs. They seem to think making references to other movies, alone, makes it funny. "Turr hurr, Gandalf is breakdancing turr hurr!". Same level of infantile humor.

  3. Dear Nvidia... by KazW · · Score: 5, Insightful

    Please take notice, this is how to support GPU hardware correctly.

    --
    Geeks don't grock information, they grep it.
    1. Re:Dear Nvidia... by bill_mcgonigle · · Score: 5, Interesting

      What I wonder is what really makes it harder / impossible for Nvidia or whomever to do it but works for Intel? If anything.

      The standard rumor is that they they all violate bogus patents rampantly and only by keeping their code secret (and possibly backdoored) can they stay afloat, in face of the patent trolls.

      A deep cynic might claim that Intel can survive more of these trolls than nVidia could so this could be a competitive move. IIRC Intel and nVidia had a cross-licensing deal that involved Intel staying out of the discrete market - maybe that's due to expire soon.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    2. Re:Dear Nvidia... by jedidiah · · Score: 4, Interesting

      I'm still waiting for Intel drivers that are on par with their Nvidia counterpart.

      Despite all of the noise made about Intel's cooperation, this is the first time we've actually had full disclosure from them. Prior to today what they offered was incomplete. It was all empty promises despite of all of the rhetoric from the political purists about how Intel does things better.

      Someday, this might lead to a proper driver. Although Intel hardware will probably still be just as lame then.

      --
      A Pirate and a Puritan look the same on a balance sheet.
    3. Re:Dear Nvidia... by gigaherz · · Score: 2

      I doubt that. The firmware, maybe, but probably not drivers. Normally the difference between high-priced and low-priced models is that the low-priced models have some internal fuses blown, so that some of the cores are disabled. Sometimes those cores were defective, other times they disable cores just to meet the demand. It could be that they disable the cores with firmware instead of fuses, and somehow the drivers could reenable cores in the latter cases, but my guess is that the people who give the orders simply think of their precious architecture details as information that needs to be kept secret, in case the competition gets too many ideas from those details.

    4. Re:Dear Nvidia... by gnasher719 · · Score: 2

      The standard rumor is that they they all violate bogus patents rampantly and only by keeping their code secret (and possibly backdoored) can they stay afloat, in face of the patent trolls.

      The other reason would be that if NVidia engineers could read the documentation needed to write ATI drivers and vice versa, they would figure out some clever ideas that their competitor had and reproduce them. If you make more and more powerful cards, you always need new clever ideas how you turn more transistors into more speed. You run into bottlenecks that weren't bottlenecks when you had a quarter of the transistors. Since Intel integrated graphics is quite a bit behind in that respect, Intel is probably doing clever things that ATI and NVidia would have been glad to know four years ago.

    5. Re:Dear Nvidia... by Luckyo · · Score: 2

      Market disagrees. Vast majority of PCs sold, both desktop and laptop run intel's intergrated graphics. Most of them are the older GMA chipsets, but quite a few desktops nowadays run on intel's HD GPUs that come with the CPU.

    6. Re:Dear Nvidia... by tibman · · Score: 2

      That's a trap. The mobo comes with integrated intel, yes. But in most cases the end user also has a discrete card. You can guess which one is actually used.

      --
      http://soylentnews.org/~tibman
    7. Re:Dear Nvidia... by tlhIngan · · Score: 3, Insightful

      The other reason would be that if NVidia engineers could read the documentation needed to write ATI drivers and vice versa, they would figure out some clever ideas that their competitor had and reproduce them. If you make more and more powerful cards, you always need new clever ideas how you turn more transistors into more speed. You run into bottlenecks that weren't bottlenecks when you had a quarter of the transistors. Since Intel integrated graphics is quite a bit behind in that respect, Intel is probably doing clever things that ATI and NVidia would have been glad to know four years ago.

      The other thing is, well, the documentation simply doesn't exist. Having worked with quite a few ASIC vendors, inside and out, I can tell you the modern SoC if heavily undocumented. For a good chunk of the blocks, unless they're external IP (from say, Synopsys, Cadence or ARM), there IS no documentation other than a register list.

      Yes, the documentation may just be a register list. Nothing to tell you how you should drive the hardware, if the registers have to be programmed in a certain way, or if there's tricks and other things that are important to know. Or even how some data structures need to be laid out in memory for hardware (you may get a brief layout, but that's it). Any my favorite - how the hardware reacts to an error. Sometimes it just plain locks up requiring a full reset. Other times it sets some strange error bit that must be cleared in order to resume functionality. If you're lucky to know how to clear the error.

      Oh, and how does a software developer find out such information? Easy, they ask the ASIC designer how they envisioned the thing to work. Sometimes they're helpful and tell you exactly what you need to know, but they can be terse and expect you to figure out their thinking. And sometimes what you want to do LOOKS possible from the registers, only to find out that no, you just cannot program the bits that way and no, you shouldn't try.

      That's probably why Intel took so long to get the documentation out - Haswell has been out since what, April? And only now they release the documentation? Well, it's probably because Intel was getting the whole whack together from random designer notes, software design, etc. Then bringing all that together (Intel creates some of the best documentation around) and editing it and all that.

      For the most part, nVidia and AMD probably DON'T have much in the way of documentation on the latest GPUs. Intel does, and only because they spend time and effort doing it - but Intel's also huge and has a lot of cash to have a bunch of engineers sitting around writing documentation exclusively.

      If you think OSS people hate writing documentation - it's no better when it's commercial development - again, the project usually doesn't allow for much documentation to be created. And things like GPUs which change frequently, well, by the time the documents are written, it's obseleted 3 times over.

  4. Code. by ledow · · Score: 3, Insightful

    Documentation is ONE PART. It says what the design was supposed to be like.

    Then you have errata and variations - when some of the hardware doesn't correspond to the documentation and acts differently.

    Then you have examples - where someone shows you how to, e.g. draw a simple triangle using the documented opcodes and all of the boilerplate and set up necessary.

    And then you have actual working code. Where you give away, for example, a complete implementation that conforms to a higher, standardised API and issues instructions to the hardware to perform those actions.

    Out of all of those, documentation is the easiest thing to do. You can just (for example, just flicked through a PDF from that site) say that instruction X transposes a matrix. No idea of performance, whether that's the recommended way, what it contends with, how it works, whether the Intel drivers use that themselves, whether it's a legacy function, whether it has huge constraints on its use.

    Without some code, it's all just fancy tech sheets. Sure, better than nothing, but a long way from actual co-operation. I'm not saying Intel don't co-operate in other areas, but documentation like this? That's the "quick reference" stuff for when your thousands of lines of existing example code don't act like you expect when you tweak them and you look up what that operand is supposed to do and how.

    Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.

    Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.

    This is why I've never been that bothered by documentation releases, or even unmaintained source-drops. Supposedly Broadcom did something similar for the RPi's graphics chips. I think we're still waiting on anything that's not a binary driver there. And we have this sort of stuff for some ancient 3D graphics cards - it's just not as easy as reading it all and then sitting down to write a driver.

    Intel, nVidia, ATI: Give us drivers with code that have no reliance on "black box" information/code, and we'll be happy. Until then, it's just lip-service. And you know that. That's why you don't release this kind of stuff for graphics chips, and nor does anyone else. Because you can drop this in someone's lap and years later STILL end up being pestered to the ends of the earth for an open-source driver (or assistance to help write one) because it doesn't exist.

    Code is a lot more than writing things to perform a protocol described in the documentation. If only it were that easy.

    1. Re:Code. by Anonymous Coward · · Score: 5, Insightful

      You make a good point, however you are incorrect. As an author of a handful of drivers, and contributor to a handful more - we like specs. If you are incapable of taking an RFC or spec and outputting a working driver, then you aren't quite the programmer you think you are. Specs are often all a driver author ever receives, and you should be able to produce working code with nothing else. It's *nice* if the vendor sends best practices or additional notes about deprecation of certain methods...but it's not the norm, and should not be expected. Being a good developer means being able to benchmark methods described in specs and determine what performs best, and when that applies.

    2. Re:Code. by Anonymous Coward · · Score: 5, Informative

      lots of blah blah blaa

      The Haswell GPU driver source code has been in the upstream kernel and userspace parts for maybe a year now.

    3. Re:Code. by pentagramrex · · Score: 2

      Documentation is ONE PART. It says what the design was supposed to be like.

      Then you have errata and variations - when some of the hardware doesn't correspond to the documentation and acts differently.

      Then you have examples - where someone shows you how to, e.g. draw a simple triangle using the documented opcodes and all of the boilerplate and set up necessary.

      And then you have actual working code. Where you give away, for example, a complete implementation that conforms to a higher, standardised API and issues instructions to the hardware to perform those actions.

      Out of all of those, documentation is the easiest thing to do. You can just (for example, just flicked through a PDF from that site) say that instruction X transposes a matrix. No idea of performance, whether that's the recommended way, what it contends with, how it works, whether the Intel drivers use that themselves, whether it's a legacy function, whether it has huge constraints on its use.

      Without some code, it's all just fancy tech sheets. Sure, better than nothing, but a long way from actual co-operation. I'm not saying Intel don't co-operate in other areas, but documentation like this? That's the "quick reference" stuff for when your thousands of lines of existing example code don't act like you expect when you tweak them and you look up what that operand is supposed to do and how.

      Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.

      Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.

      This is why I've never been that bothered by documentation releases, or even unmaintained source-drops. Supposedly Broadcom did something similar for the RPi's graphics chips. I think we're still waiting on anything that's not a binary driver there. And we have this sort of stuff for some ancient 3D graphics cards - it's just not as easy as reading it all and then sitting down to write a driver.

      Intel, nVidia, ATI: Give us drivers with code that have no reliance on "black box" information/code, and we'll be happy. Until then, it's just lip-service. And you know that. That's why you don't release this kind of stuff for graphics chips, and nor does anyone else. Because you can drop this in someone's lap and years later STILL end up being pestered to the ends of the earth for an open-source driver (or assistance to help write one) because it doesn't exist.

      Code is a lot more than writing things to perform a protocol described in the documentation. If only it were that easy.

      It's been a few years since I worked in hardware, but even when you build it for a commercial company; not open source, you don't get the things you (poster) think are important.

      In the past I found AMD far more helpful than Intel, and I was building high end workstations then (Motorola and MIPS based, but lots of AMD chips). MIPS and AMD at least gave you the documentation for free if they thought you were serious: that was LOTS of thick books. Sometimes they helpd you figure out if things didn't work properly.

      If you are in a small company, even when you buy a reference design kit you don't get any help. you have to work it out for yourself, even if there are some errata sheets. No properly working drivers for the reference design. No source.

      These days I'm glad I have a little company in China making our boards, that while being a bit clumsy are very fast to fix things. Things get broken as quickly.

      I'm even more glad that there are less complier bugs than I had to deal with on Microsoft compliers, or GNU on obscure ARM architectures I tore my hair out over.

      Luckily I have a lot of hair.

      Rob.

    4. Re:Code. by Kjella · · Score: 2

      Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.

      Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.

      So in short you're saying the open source community is a bunch of liars and nVidia is right when they say "Well if we gave you X you'd ask for Y and Z until we're really doing all the job anyway, so why give you the little finger when you'll chew off our arm? If you're going to leave it all to us anyway, use the binary." For sure, there are hardware bugs and maybe undocumented ones too because the closed source driver never tripped it but this is like saying you can't program an x86 processor because Intel might decide that 2+2 = 3.99999999999978. You're just looking for an excuse to throw in the towel. For example you could start by writing the code to take Mesa from OpenGL 3.1 to OpenGL 4.4, it's all work and no hardware documentation required. There's tons of work that could be done, and the same is true for drivers as well. The documentation is there, the manpower is not. Stop pretending it's Intel, nVidia and AMD's fault that the open source drivers are lagging far behind the blobs.

      --
      Live today, because you never know what tomorrow brings
    5. Re:Code. by NixieBunny · · Score: 3, Insightful

      Tee hee. I have a fine counterexample.
      About 15 years ago, my company (a producer of VMEbus and CompactPCI boards) designed a video module. We used a Trident mobile graphics chip. Unfortunately, we were attempting to use it with a PowerPC, not an x86 CPU. We had the big user manual for the chip, but when we programmed all the registers according to the published configuration info, it refused to initialize.
      We then were given the BIOS object code from the factory (they wouldn't share the x86 assembly source code). We disassembled the code. It was such a tangled web of spaghetti that we never did figure out how to get the part initialized, and the factory app engineering team was unable to tell us how to do so either.
      We eventually dumped the part and used an Intel part with C source code available. It worked just fine.

      --
      The determined Real Programmer can write Fortran programs in any language.
    6. Re:Code. by skids · · Score: 2

      Have you seen the quality of the code written by the hardware design guys?

      Yep. Atrocious. Topped only by BIOS coders. I have a laptop that you have to limit your BIOS sessions to under 1 minute or it will overheat. Animals.

      --
      Someone had to do it.