Slashdot Mirror
Ask Slashdot: What Is the Most Painless Intro To GPU Programming?
dryriver writes "I am an intermediate-level programmer who works mostly in C# NET. I have a couple of image/video processing algorithms that are highly parallelizable — running them on a GPU instead of a CPU should result in a considerable speedup (anywhere from 10x times to perhaps 30x or 40x times speedup, depending on the quality of the implementation). Now here is my question: What, currently, is the most painless way to start playing with GPU programming? Do I have to learn CUDA/OpenCL — which seems a daunting task to me — or is there a simpler way? Perhaps a Visual Programming Language or 'VPL' that lets you connect boxes/nodes and access the GPU very simply? I should mention that I am on Windows, and that the GPU computing prototypes I want to build should be able to run on Windows. Surely there must a be a 'relatively painless' way out there, with which one can begin to learn how to harness the GPU?"
GPU programming is painful. A painless introduction doesn't capture the flavor of it.
Since the whole point of GPU programming is efficiency, don't even think about VBing it. Or Pythoning it. Or whatever layer of a shiny crap might seem superficially appealing to you.
Learn OpenCL and do the job properly.
When all you have is a hammer, every problem starts to look like a thumb.
Those are game engines. They will do nothing to help him use the GPGPU capabilities of his graphics card.
I don't think he is looking at making a game, I think he is looking for some cheap parallel processing. I have done some cuda, it was a pain to set up a few years back. There probably are better tutorials now.
I get the impression that CUDA/OpenCL is still the best option. This thesis on Obsidian presents, a Haskell set of binding which might be easier and also covers the basics quite well. Haskell lends itself really well because the language inherently is designed for parallelism because of purity and out of order computation. That being said, I think Obsidian is a bit rough around the edges but if you are looking for a real alternative, this is one.
Anyone who tells you differently is selling you something.
Check out the Udacity class on parallel programming. It's mostly CUDA (I believe it's taught by NVIDIA engineers): https://www.udacity.com/course/cs344
CUDA is generally easier to program than OpenCL. Of course, CUDA only runs on NVIDIA GPUs though.
OpenACC is what you're looking for. It uses a directive based programming model similar to OpenMP, so you write ordinary looking code, then annotate it in ways that tell the compiler how to transform it into GPU code.
You won't get as good performance as well written CUDA or OpenCL code, but it's much easier to learn. And once you get comfortable with it, you may find it easier to make the step from there into lower level programming.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
VB.NET background. Wanted to get into GPGPU to accelerate some of my more complicated math calculations. Tried CLOO (open source .net GPU wrappers) and couldn't get it to work, tried AMD's OPENCL dev gui, couldn't get it to work. Eventually found the answer in python. GPGPU in pyopencl is well-documented thanks to the bitcoiners, and from .net you can either run the python in a shell, or write a little python kernel to listen for, and process commands. Only catch is the opencl abilities are limited, and you have to start dabbling in c++ to get it to do any real work (and even then it's a dumbed-down c++ and many existing extensions don't install or work quite right). All in all I found the entire thing very rewarding though. :) Best of luck.
Coursera has some courses on GPU programming, like this one, and what's nice about them pretty slow, and I'm assuming that they explain things well. Other online courses probably offer the same, and I think the video lectures would be helpful in understanding the concepts.
I have left slashdot and am now on Soylent News. FUCK YOU DICE.
So there's nothing really easy about GPU programming. You can look at C++ AMP from Microsoft, OpenMP or one of the other abstractions but you really need to understand how these massively parallel machines work. It's possible to write some perfectly valid code in any of these environments which will run SLOWER than on the CPU because you didn't understand fundamentally how GPUs excel at processing.
Udacity currently has a fairly decent intro course on GPU programming at: https://www.udacity.com/course/cs344
It's based around NVIDIA and CUDA but most of the concepts in the course can be applied to OpenCL or another GPU programming API with a little syntax translation. Also you can do everything for the course in your web-browser and you don't need an NVIDIA GPU to finish the course exercises.
I'd suggest running through that and then deciding on what API you want to end up using.
Never under any circumstances use cuda. We don't need anymore proprietary garbage floating around. Use opencl only.
If you are going to program a GPU, and you are looking for performance gains, you MUST understand the hardware. In particular, you must understand the complicated memory architecture, you must understand the mechanisms for moving data from one memory system to another, and you must understand how your application and algorithm can be transformed into that model.
There is no shortcut. There is no magic. There is only hardware.
If you do not believe me, you can hunt up the various Nvidia papers walking you through (in painful detail-- link below) the process of writing a simple matrix transpose operation for CUDA. The difference between a naive and a good implementation, as shown in that paper, is huge.
That said, once you understand the principles, CUDA is relatively easy to learn as an extension of C, and the Nvidia profiler, NVVP, is good at identifying some of the pitfalls for you so that you can fix them.
http://www.cs.colostate.edu/~cs675/MatrixTranspose.pdf
Agreed 100% about CUDA and OpenMP! Already invented a new multi-core string searching algorithm and having a load of fun playing around with my GTX Titan combing CUDA + OpenMP. You can even do printf() from the GPU. :-)
The most _painless_ way to learn CUDA is to install CUDA on a Linux (Ubuntu) box or Windows box.
https://developer.nvidia.com/cuda-downloads
On Linux, at the command line fire up 'nsight' open the CUDA SDK samples and start exploring! And by exploring I mean single-stepping through the code. The NSight IDE is pretty darn good considering it is free.
Another really good doc is the CUDA C Programming Guide.
http://docs.nvidia.com/cuda/cuda-c-programming-guide/
Oh and don't pay attention to the Intel Propaganda - there are numerous inaccuracies:
Debunking the 100X GPU vs CPU Myth: An Evaluation of Throughput Computing on CPU and GPU
http://pcl.intel-research.net/publications/isca319-lee.pdf
I teach this stuff daily, and the huge advance over the past year has been the availability of OpenACC, and now OpenMP 4, compilers that allow you to use directives and offload much of the CUDA pain to the compiler.
There is now a substantial base of successful codes that demonstrate that this really works efficiently (both development time and FLOPS). S3D runs at 15 PFLOPS on Titan using this and may well win the Gordon Bell prize this year. Less than 1% of lines of code modified there. NVIDIA has a whole web site devoted to use cases.
I recommend you spend a day to learn it. There are regular online courses offered, and there is a morning session on it this Monday at XSEDE 13 if you are one of those HPC guys. A decent amount is available online as well.
BTW, with AMD moving to Fusion, the last real supporter of OpenCL is gone. NVIDIA prefers OpenACC or CUDA and Intel prefers OpenMP 4 for MIC/Phi. So everyone officially supports it, but no one really puts any resources into it and you need that with how fast this hardware evolves.
You would probably see a multi-fold increase in performance by simply converting your project from C# to C++.
Incorrect. That is certainly a valid approach and the GP should be modded up.
Using textures and shaders you can very easily do massively parallel floating point operations in XNA on the GPU, and it's a language the asker is familiar with.
Think outside the box a little bit.
We have come a long way since the days of assembly and assembly in another name Fortran. But the overheads of the higher level languages have been masked a lot by the ever increasing speed and memory availability. Whole generations of programmers have come up, higher level languages with IDE and CASE tools from day one they fundamentally don't understand how the code actually works. They are continually stumped by the fact the code does what they tell it to do, not what they meant it to do.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
True, and this is even more true on GPUs than CPUs. They do a lot less to shield you from the low level details of how your code gets executed, so those details end up having a bigger impact on your performance. And to make it worse, those details change with every new hardware generation!
But for a new user just getting into GPU programming, it's easier to learn those things in the context of a simple programming model like OpenACC than a complicated one like CUDA or OpenCL. That just forces them to deal with even more complexity and hardware details right from the very start. OpenACC can produce good results if used well. And once you've learned to do that, you're in a better position to tackle the harder technologies.
"I'm too busy to research this and form an educated opinion, but I do have time to tell everyone my uninformed opinion."
How many boxes do you want to go through before you get to the solution? Sure, he could write it as a shader, but that hardly requires pulling in something like Unity or XNA to build the project.
Writing GPU programs is hard. Not only do you have to learn a new sets of APIs, you also have to understand the underlying architecture to extract decent performance. It requires a different approach to problem solving that requires months if not years to develop.
Fortunately you don't need to read the entire cuda programming guide to program on the GPU. There are several excellent libraries out there which hide the complexities of the GPU architecture. Since you are doing image processing, I would recommend Arrayfire (http://www.accelereyes.com/products/arrayfire). It is a free library which provides several image processing functions which have been optimized for the GPU. You should also look into Thrust and NPP(included with the CUDA toolkit), although these libraries are more verbose and require greater understanding of the C++ and GPU to program.
OpenACC may be higher-level(easier to use), but it still generates CUDA/OpenCL code. Your wording sounded like "OpenCL support is gone." I want to correct you on that. OpenCL is the future and wraps CUDA also. If you code for CUDA, you can only target CUDA hardware. If you code for OpenCL, you can target not only AMD, but also CUDA hardware. That was the point of the OpenCL spec in the first place. OpenCL can also transparently take advantage of the local CPU cores. OPENCL has one drawback. OpenCL does not support all types. It is highly constrained to certain kinds of types relevant to graphics/3D. There have been some kludge patches to make CUDA/OpenCL work with string types (i.e. parallel grep with CUDA), but these aren't well suited because the hardware was not intended for that and it requires a lot of moving of memory from the main motherboard memory to the graphics card memory which wastes a lot of time. String parallelizing is better done with mechanisms like OpenMP. OpenMP can support any kind of types and crunch with them and OpenMP is designed to co-exist with MPI(RPC-like many computer parallelism).
Start learning OpenCL, OpenMP, MPI, GNU & boost library parallelism. To make it easier try running golang opencl examples: ./all.bash /home/youruser/goopencl/src/github.com/tones111/go-opencl/cl/demo/rotate/
apt-get install mercurial meld
hg clone -u release https://code.google.com/p/go
cd go
cd src
#Put this into your ~/.bashrc:
export GOROOT=/home/youruser/yourgo
export PATH=$PATH:$GOROOT/bin
mkdir -p ~/goopencl
cd ~/goopencl
mkdir -p ~/goopencl/pkg
mkdir -p ~/goopencl/src
export GOPATH=/home/youruser/goopencl
go get github.com/tones111/go-opencl/cl
go get github.com/tones111/raw
cd
go run rotate.go -i="i.png" -o="o.png" -a=15