Slashdot Mirror
Metaprogramming GPUs with Sh
Martin Ecker writes "With the advent of powerful, programmable GPUs in consumer graphics hardware, an increasing number of shading languages to program these GPUs has become available. One quite interesting language that - in many ways - has a very different approach than other mainstream shading languages (such as Cg or the OpenGL Shading Language) is Sh. The recently released book "Metaprogramming GPUs with Sh" by Michael McCool and Stefanus Du Toit, both major contributors to the Sh project, explains the basics of the Sh high-level shading language and the corresponding API and also goes into some of the details of the implementation. The book is intended for an audience that is already familiar with traditional shader development for programmable GPUs. Also, a firm background in 3D graphics programming and C++ is a must for the interested reader." Read on for the rest.
Metaprogramming GPUs with Sh
author
Michael McCool, Stefanus Du Toit
pages
308
publisher
A K Peters
rating
7/10
reviewer
Martin Ecker
ISBN
0321197895
summary
A book that describes an interesting shading language and accompanying API to program GPUs.
Before discussing the book in more detail, I will try to give a basic overview of Sh, since most readers will not be familiar with it. For a more in-depth look at Sh, I recommend taking a look at a recently posted Gamasutra article by Michael McCool (http://www.gamasutra.com/features/20040716/mccool _01.shtml), the paper on Sh from the authors presented at the recently held SIGGRAPH 2004 conference (http://www.cgl.uwaterloo.ca/Projects/rendering/Pa pers/#algebra), and of course the Sh homepage at http://www.libsh.org.
Sh started out as a research project at the University of Waterloo (http://www.cgl.uwaterloo.ca), and it is both a shading language and a runtime API to use the Sh shaders. As a shading language Sh is embedded into C++ as a domain-specific language, which is made possible by using C++ operator overloading and by defining special tuple and matrix types that are used extensively in shader code. So instead of defining its own language that requires a full compiler, like other shading languages do, Sh uses regular C++ syntax to describe shader code, which is then dynamically (at runtime) compiled to a specific backend, such as a GPU or possibly even the CPU. In addition to compiling to a specific GPU or CPU target, Sh can also be used in a special stream mode where a shader is applied to a stream of input tuples. This is very useful for general purpose GPU programming where the GPU is basically used as an additional processor to the host CPU (see http://www.gpgpu.org for more information on the subject). Finally, Sh code can also be executed in an immediate mode where every Sh statement is directly executed on the host CPU (without being compiled into a shader program), which makes it very easy to debug shaders with any host debugger running on the CPU.
Due to the way Sh is embedded into C++, the full range of abstraction mechanisms offered by C++ can be used to structure and modularize shader code. Abstract base classes, regular functions, templates, and any other features offered by C++ can be used to develop shaders. This is an interesting consequence of the metaprogramming approach of Sh that also allows the use of software engineering principles in shader development, such as object orientation, that other shading languages currently cannot offer.
This kind of metaprogramming in C++ is used by an increasing number of libraries. For example, the Spirit parser framework (see http://spirit.sourceforge.net) uses a similar approach to describe and generate parsers directly in C++ instead of using traditional external tools, such as yacc or bison.
One of the most fascinating features of the Sh toolkit is the possibility to combine and connect shader programs to form new shader programs, which allows one to easily build complex shaders out of simple shader fragments. In a more general sense, Sh provides what can be called a shader algebra (see also the aforementioned SIGGRAPH 2004 paper), where shader programs are the objects on which special operators to combine and connect them are defined. An interesting application of this shader algebra is to specifically bind certain varying shader inputs to uniform variables and the other way around (this is what functional programming languages usually call currying). Also combining a matrix palette skinning shader with any light model shader (or any shaders that perform specific tasks, for that matter) is easily possible.
After this short introduction to the Sh toolkit, we shall now take a closer look at the book "Metaprogramming GPUs with Sh".
The book is split into three parts, an introduction, a reference, and an engineering overview.
The introduction consists of the first five chapters and discusses the basics of the Sh shading language and the API. In particular, the tuple and matrix types and the operators defined on them are presented. The way shader programs are defined and how parameters and attributes are handled is discussed, followed by the way textures are represented. Finally, the stream and channel concept used to feed data into shader programs is discussed. These introductory chapters contain a number of examples that demonstrate the presented concepts. Chapter three contains a quite interesting sample shader that uses constructive solid geometry techniques and metaprogramming in Sh to render text. While not the most useful use case, the shader shows some interesting capabilities of Sh, in particular the shader algebra operators. Chapter four on textures has some more nice sample shaders for doing shiny bump mapping, rendering wood and marble, and using Worley noise.
The second part of the book is a reference on Sh. Unlike references in many other computer books, this is not just a technical listing of the available features of Sh but is written in regular prose (with the occasional reference-like table here and there). The six chapters of the reference section describe how to setup and use the Sh library, and then discuss the available types, operators, and standard library functions more thoroughly than in the introduction. Additionally, the available backends are mentioned in the last chapter of this part of the book. A draft of the reference manual can also be found online at http://www.libsh.org/ref/online.
The final part of the book deals with engineering aspects of Sh. These final five chapters of the book discuss the details of the current implementation. The intermediate representation for shaders that is used by Sh is presented as well as how streams and textures are managed and stored internally. The interface between the Sh frontend and the various specific backends is discussed, as well as the current state of the optimizer including some further improvements that are planned in the future.
The images in the book are all in black and white except for 14 color plates in the middle of the book. The color plates and other images usually show teapots or animals, so they aren't all that exciting, but do demonstrate what the sample shaders presented in the book look like.
The book does not come with a CD-ROM, but with such a young library that is still under heavy development, putting a snapshot of the library's source code base on a CD-ROM would be a waste of resources. Sh itself as well as all sample shaders presented in the book can be downloaded from the Sh homepage at http://www.libsh.org. This website also has additional documentation, including some papers and the API reference documentation generated with Doxygen from the sources. Sh is distributed under a very liberal open source license (based on the zlib/libpng license) that also allows commercial use.
For the reader with enough expertise in 3D and shader programming, this book provides a concise and well-written introduction to Sh. The book will definitely contribute to enlarging the currently relative small user base of Sh and hopefully help the library grow and get more refined in the near future. Everyone familiar with "regular" high-level shading languages, such as Cg or the OpenGL Shading Language, should take a look at this book to see a new and interesting way of programming GPUs that the aforementioned languages do not offer.
About the review author:
The author has been involved in real-time graphics programming for more than 9 years and works as a games developer for arcade games. In his rare spare time he works on a graphics-related open source project called XEngine http://xengine.sourceforge.net.
Before discussing the book in more detail, I will try to give a basic overview of Sh, since most readers will not be familiar with it. For a more in-depth look at Sh, I recommend taking a look at a recently posted Gamasutra article by Michael McCool (http://www.gamasutra.com/features/20040716/mccool _01.shtml), the paper on Sh from the authors presented at the recently held SIGGRAPH 2004 conference (http://www.cgl.uwaterloo.ca/Projects/rendering/Pa pers/#algebra), and of course the Sh homepage at http://www.libsh.org.
Sh started out as a research project at the University of Waterloo (http://www.cgl.uwaterloo.ca), and it is both a shading language and a runtime API to use the Sh shaders. As a shading language Sh is embedded into C++ as a domain-specific language, which is made possible by using C++ operator overloading and by defining special tuple and matrix types that are used extensively in shader code. So instead of defining its own language that requires a full compiler, like other shading languages do, Sh uses regular C++ syntax to describe shader code, which is then dynamically (at runtime) compiled to a specific backend, such as a GPU or possibly even the CPU. In addition to compiling to a specific GPU or CPU target, Sh can also be used in a special stream mode where a shader is applied to a stream of input tuples. This is very useful for general purpose GPU programming where the GPU is basically used as an additional processor to the host CPU (see http://www.gpgpu.org for more information on the subject). Finally, Sh code can also be executed in an immediate mode where every Sh statement is directly executed on the host CPU (without being compiled into a shader program), which makes it very easy to debug shaders with any host debugger running on the CPU.
Due to the way Sh is embedded into C++, the full range of abstraction mechanisms offered by C++ can be used to structure and modularize shader code. Abstract base classes, regular functions, templates, and any other features offered by C++ can be used to develop shaders. This is an interesting consequence of the metaprogramming approach of Sh that also allows the use of software engineering principles in shader development, such as object orientation, that other shading languages currently cannot offer.
This kind of metaprogramming in C++ is used by an increasing number of libraries. For example, the Spirit parser framework (see http://spirit.sourceforge.net) uses a similar approach to describe and generate parsers directly in C++ instead of using traditional external tools, such as yacc or bison.
One of the most fascinating features of the Sh toolkit is the possibility to combine and connect shader programs to form new shader programs, which allows one to easily build complex shaders out of simple shader fragments. In a more general sense, Sh provides what can be called a shader algebra (see also the aforementioned SIGGRAPH 2004 paper), where shader programs are the objects on which special operators to combine and connect them are defined. An interesting application of this shader algebra is to specifically bind certain varying shader inputs to uniform variables and the other way around (this is what functional programming languages usually call currying). Also combining a matrix palette skinning shader with any light model shader (or any shaders that perform specific tasks, for that matter) is easily possible.
After this short introduction to the Sh toolkit, we shall now take a closer look at the book "Metaprogramming GPUs with Sh".
The book is split into three parts, an introduction, a reference, and an engineering overview.
The introduction consists of the first five chapters and discusses the basics of the Sh shading language and the API. In particular, the tuple and matrix types and the operators defined on them are presented. The way shader programs are defined and how parameters and attributes are handled is discussed, followed by the way textures are represented. Finally, the stream and channel concept used to feed data into shader programs is discussed. These introductory chapters contain a number of examples that demonstrate the presented concepts. Chapter three contains a quite interesting sample shader that uses constructive solid geometry techniques and metaprogramming in Sh to render text. While not the most useful use case, the shader shows some interesting capabilities of Sh, in particular the shader algebra operators. Chapter four on textures has some more nice sample shaders for doing shiny bump mapping, rendering wood and marble, and using Worley noise.
The second part of the book is a reference on Sh. Unlike references in many other computer books, this is not just a technical listing of the available features of Sh but is written in regular prose (with the occasional reference-like table here and there). The six chapters of the reference section describe how to setup and use the Sh library, and then discuss the available types, operators, and standard library functions more thoroughly than in the introduction. Additionally, the available backends are mentioned in the last chapter of this part of the book. A draft of the reference manual can also be found online at http://www.libsh.org/ref/online.
The final part of the book deals with engineering aspects of Sh. These final five chapters of the book discuss the details of the current implementation. The intermediate representation for shaders that is used by Sh is presented as well as how streams and textures are managed and stored internally. The interface between the Sh frontend and the various specific backends is discussed, as well as the current state of the optimizer including some further improvements that are planned in the future.
The images in the book are all in black and white except for 14 color plates in the middle of the book. The color plates and other images usually show teapots or animals, so they aren't all that exciting, but do demonstrate what the sample shaders presented in the book look like.
The book does not come with a CD-ROM, but with such a young library that is still under heavy development, putting a snapshot of the library's source code base on a CD-ROM would be a waste of resources. Sh itself as well as all sample shaders presented in the book can be downloaded from the Sh homepage at http://www.libsh.org. This website also has additional documentation, including some papers and the API reference documentation generated with Doxygen from the sources. Sh is distributed under a very liberal open source license (based on the zlib/libpng license) that also allows commercial use.
For the reader with enough expertise in 3D and shader programming, this book provides a concise and well-written introduction to Sh. The book will definitely contribute to enlarging the currently relative small user base of Sh and hopefully help the library grow and get more refined in the near future. Everyone familiar with "regular" high-level shading languages, such as Cg or the OpenGL Shading Language, should take a look at this book to see a new and interesting way of programming GPUs that the aforementioned languages do not offer.
About the review author:
The author has been involved in real-time graphics programming for more than 9 years and works as a games developer for arcade games. In his rare spare time he works on a graphics-related open source project called XEngine http://xengine.sourceforge.net.
You can purchase Metaprogramming GPUs with Sh from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Is this the longest headline blurb ever?
Is the book review template broken or something? This is the second review in as many days that's been misclassified.
"They redundantly repeated themselves over and over again incessantly without end ad infinitum" -- ibid.
it really has to be, I thought I had hit the wrong link when I got to the page! someone wake up Tacoman and see whatup on the whois!
CB
free ipod and free gmail!
Dude,
(1) Don't link article
(2) Set blurb = article
(3) ???
(4) Profit!*
*by saving on bandwidth and the cost of replacing a toasted server