Slashdot Mirror
Ask Slashdot: What Is the Most Painless Intro To GPU Programming?
dryriver writes "I am an intermediate-level programmer who works mostly in C# NET. I have a couple of image/video processing algorithms that are highly parallelizable — running them on a GPU instead of a CPU should result in a considerable speedup (anywhere from 10x times to perhaps 30x or 40x times speedup, depending on the quality of the implementation). Now here is my question: What, currently, is the most painless way to start playing with GPU programming? Do I have to learn CUDA/OpenCL — which seems a daunting task to me — or is there a simpler way? Perhaps a Visual Programming Language or 'VPL' that lets you connect boxes/nodes and access the GPU very simply? I should mention that I am on Windows, and that the GPU computing prototypes I want to build should be able to run on Windows. Surely there must a be a 'relatively painless' way out there, with which one can begin to learn how to harness the GPU?"
I don't think he is looking at making a game, I think he is looking for some cheap parallel processing. I have done some cuda, it was a pain to set up a few years back. There probably are better tutorials now.
I get the impression that CUDA/OpenCL is still the best option. This thesis on Obsidian presents, a Haskell set of binding which might be easier and also covers the basics quite well. Haskell lends itself really well because the language inherently is designed for parallelism because of purity and out of order computation. That being said, I think Obsidian is a bit rough around the edges but if you are looking for a real alternative, this is one.
Anyone who tells you differently is selling you something.
Ordinarily I'd agree with you (programmer's time is worth more than anyone else's) but that means stopping now not even bothering with the GPU, since he already has code that works on the CPU. He's done. The project is complete. Next work order.
As soon as we start saying he's not already done, we've violated the principle and should stop trying to use it. His target is clearly end-user-enjoyed performance, and he's willing to put in more programmer time. So it's time to hang up the rapid prototype hat, and seriously get his hands dirty.
What's wrong with a higher level language that interfaces with OpenCL? You're still writing OpenCL, you're just using Python for loading/storing datasets and initialisation. If you're starting out, something like PyOpenCL might be better as it'll allow you to focus on writing stuff in OpenCL.
So there's nothing really easy about GPU programming. You can look at C++ AMP from Microsoft, OpenMP or one of the other abstractions but you really need to understand how these massively parallel machines work. It's possible to write some perfectly valid code in any of these environments which will run SLOWER than on the CPU because you didn't understand fundamentally how GPUs excel at processing.
Udacity currently has a fairly decent intro course on GPU programming at: https://www.udacity.com/course/cs344
It's based around NVIDIA and CUDA but most of the concepts in the course can be applied to OpenCL or another GPU programming API with a little syntax translation. Also you can do everything for the course in your web-browser and you don't need an NVIDIA GPU to finish the course exercises.
I'd suggest running through that and then deciding on what API you want to end up using.
If you are going to program a GPU, and you are looking for performance gains, you MUST understand the hardware. In particular, you must understand the complicated memory architecture, you must understand the mechanisms for moving data from one memory system to another, and you must understand how your application and algorithm can be transformed into that model.
There is no shortcut. There is no magic. There is only hardware.
If you do not believe me, you can hunt up the various Nvidia papers walking you through (in painful detail-- link below) the process of writing a simple matrix transpose operation for CUDA. The difference between a naive and a good implementation, as shown in that paper, is huge.
That said, once you understand the principles, CUDA is relatively easy to learn as an extension of C, and the Nvidia profiler, NVVP, is good at identifying some of the pitfalls for you so that you can fix them.
http://www.cs.colostate.edu/~cs675/MatrixTranspose.pdf
Agreed 100% about CUDA and OpenMP! Already invented a new multi-core string searching algorithm and having a load of fun playing around with my GTX Titan combing CUDA + OpenMP. You can even do printf() from the GPU. :-)
The most _painless_ way to learn CUDA is to install CUDA on a Linux (Ubuntu) box or Windows box.
https://developer.nvidia.com/cuda-downloads
On Linux, at the command line fire up 'nsight' open the CUDA SDK samples and start exploring! And by exploring I mean single-stepping through the code. The NSight IDE is pretty darn good considering it is free.
Another really good doc is the CUDA C Programming Guide.
http://docs.nvidia.com/cuda/cuda-c-programming-guide/
Oh and don't pay attention to the Intel Propaganda - there are numerous inaccuracies:
Debunking the 100X GPU vs CPU Myth: An Evaluation of Throughput Computing on CPU and GPU
http://pcl.intel-research.net/publications/isca319-lee.pdf
I teach this stuff daily, and the huge advance over the past year has been the availability of OpenACC, and now OpenMP 4, compilers that allow you to use directives and offload much of the CUDA pain to the compiler.
There is now a substantial base of successful codes that demonstrate that this really works efficiently (both development time and FLOPS). S3D runs at 15 PFLOPS on Titan using this and may well win the Gordon Bell prize this year. Less than 1% of lines of code modified there. NVIDIA has a whole web site devoted to use cases.
I recommend you spend a day to learn it. There are regular online courses offered, and there is a morning session on it this Monday at XSEDE 13 if you are one of those HPC guys. A decent amount is available online as well.
BTW, with AMD moving to Fusion, the last real supporter of OpenCL is gone. NVIDIA prefers OpenACC or CUDA and Intel prefers OpenMP 4 for MIC/Phi. So everyone officially supports it, but no one really puts any resources into it and you need that with how fast this hardware evolves.
Incorrect. That is certainly a valid approach and the GP should be modded up.
Using textures and shaders you can very easily do massively parallel floating point operations in XNA on the GPU, and it's a language the asker is familiar with.
Think outside the box a little bit.
Or when hardware engineers design chips, do they actually model out the components of every single transistor?
Chip design is absurdly complicated (even on the digital side), and involves several layers of abstraction. In roughly increasing level of detail:
* Spec level: high-level behavioral description of the functionality of a digital system, something like "8-bit 115.2kbps UART" or "2MHz PWM with 0-100% duty cycle in 0.1% increments".
* HDL/RTL level: software-like description of the complete system design. Can range from higher-level (describing behavior) to lower-level (describing specific logic). When people talk about buying, selling, or creating "IP" in the chip design world, they're usually talking about RTL for a single functional unit.
* Gate level: Logic gates and flip-flops and their connections.
* Transistor level: The transistors that make up the gates, and their connections.
* Device level: The behavior of an individual transistor.
* Physical layout: Just what it sounds like; the actual arrangements of metal and silicon.
There are some more in between, but that should give you an idea. HDLs are not necessarily low-level. For large designs (like modern SoCs), it takes some *very* expensive and complex software to go deeper into the list, and the process is not entirely automated. So I wouldn't say hardware design can't be high-level. The difference is that in hardware, you always have to care about the lowest level when you're doing your high-level design, while in software you can take more things for granted. So even though a board-level design might just be a bunch of off-the-shelf chips hooked together, it still takes a lot of work to make sure everything comes out right.
Visit the
The thing that is hard about gpu programming isn't getting code that works, its getting code that is fast. One of the most significant issues is how the data is arranged and accessed on the GPU. A big portion of this is going to be related to how the data is setup/transfered/accessed over PCIe from/to main memory.
Basically, your going to want to access that data in a manner that is fairly low level on the cpu side as well. So, the advantages of phython/etc are nullified when you have some binary blob like format your trying to access as a big pinned block of memory. This is the kind of programming that C/C++ specialize in and are really good at. Hence, notice how openCL and CUDA both are very similar to C.
I'm not saying python isn't going to work, what I am saying is that much like C++ doesn't make a good batch/text manipulation language, python doesn't make a good bit banging language
I have written code for computational biology - CUDA is a lot easier to pick up if you're just converting from C. They have great examples and documentation, great plugins but you're stuck on a single hardware platform. OpenCL on the other hand is a lot less 'nice' to begin with (pouring over 250 page PDFs with minimal explanation) but allows you to leverage both CPU and GPU efficiently and a lot less hardware independent although these days it's just nVidia for serious GPU computing and maybe Intel is starting to get into the game (don't know, haven't come across their hardware yet), AMD is a joke, not even all their GPUs (or drivers) have support for GPGPU yet and their drivers just suck.
Custom electronics and digital signage for your business: www.evcircuits.com