And if you include lighting into the equation, with N = L lights + S geometric surfaces, he's advocating for O(c * 2^backtraces * N) over O([c1 * S] + [c2 * L]) that you would get with ray-casting + deferred shading&lighting, where it could be proven that c2 c1 c, seeing as how you also have to cast multiple rays per recursive back-trace to get a decent approximation of lighting in your scene, where you only need to perform a single sample per pixel with what I'm advocating for.
Yeah, not to mention that with full ray-tracing, as you add more lights to a scene, it increases the overall complexity per pixel per ray bounce linearly as well. That's why doing deferred shading is so nice because it unbuckles the lighting from the rasterization or ray-casting/tracing step and lets you scale up the number of lightings independently with a fixed amount of overhead and linear cost per light for the entire scene, instead of per pixel or per ray or per polygon.
Going with pure ray-tracing doesn't let you take advantage of that.
You still run into the same problems regardless of whether you're using a vector processor with no branch-prediction and no cache, to if you're using a bunch of in-order cores with cache coherency, or full-blown out-of-order cores with cache coherency. You end up pulling in a combinatorial explosion (ie. exponential number) of cache lines or memory accesses per recursive ray tied to the complexity of your scene.
People like to talk about how you can just throw more cores and distributed computing architectures to increase the performance of ray-tracing graphics. Yes, it works, but only up to a point. You can cut the number of days or weeks or months off of fully rendering a CG movie by building out a distributed render farm, but it's not going to hold for when you get single shared-memory machines with enough cores and you expect it to be able to do it all in real time.
It'll still be a much better use of those hardware resources if you go with algorithms which are much more orthogonal in nature when it comes to memory access patterns. Ray-casting + deferred shading should scale better than fully recursive ray-tracing, even on many-core shared-memory CPU architectures.
I admit, I made my post before watching the video. Now that I've watched it, I think my position still holds. He talked a lot about using ray-tracing in content preproduction and preprocessing tools during development, and he talked about building ray-tracing engines in the past to experiment with the performance characteristics. This is the same research he did where he came to the conclusion that ray-casting will be better than ray-tracing. Where his position has changed is in that he used to think it would only be feasible using voxel as your input data set, but he's starting to come around on ray-casting against analytical surfaces... polygonal meshes, curved surfaces, etc. in real time.
I can't tell if you're trolling or you just wrote up a reply without fully reading my post. I never said rasterization is better. I alluded to the fact that ray-casting will win out over rasterization, and in the very near future. What I said was the ray-casting will win out over ray-tracing. The algorithmic complexity of ray-casting + deferred-shading is better than recursive ray-tracing.
I also forgot to mention that conventional rasterization and ray-casting is also just as parallel in nature as ray-tracing. In fact, more so because it has much better memory access patterns as I mentioned in a previous post. Memory-access is the biggest limiting factor in building scalable, parallel systems. If you don't have good memory-access patterns, you might as well being doing sequential work, because it's getting serialized by the hardware memory controller anyway.
And this situation isn't improving, it's actually getting worse on each subsequent hardware generation... memory access is increasingly becoming a bigger and bigger bottleneck. I suggest you educate yourself on how memory works on modern cache-coherent hardware.
The video you posted is not real-time frame rates, it's interactive frame rates. It takes a few seconds to fully recompute the scene once you stop moving the camera. And note how there's only a single car model. imagine scaling up the amount of geometry to a full world. With ray-tracing, as you scale up the complexity of the geometry, you end up scaling up the required computational complexity as well due to radiosity computations. Full real-time ray-tracing on huge worlds in real-time is a pipe-dream. What you will be able to do with ray-casting or rasterization with deferred shading composition to simulate things like reflections or radiosity will always be more than what you can do with ray-tracing, and so games developers will always choose the former.
Yes, you do lose out on reflectivity, so you have to handle reflections through other means. If you use deferred shading, you can handle it in a separate reflective lighting pass in combination with something like cube-mapping. Even though you do a second pass to handle it, it'll still end up being faster than had you tried to implement fully recursive ray-tracing.
Ray-casting would still have the benefit in providing better scalability than polygonal rasterization as there is a fixed cost per pixel, whereas with polygon rasterization, as you scale up the complexity of the geometry, you increase the computational cost. And that's why Carmack thinks it will eventually win out against rasterization: ray-casting becomes a better use of the available computational resources you have at your disposal once you get past a certain level of detail in your scenes.
The Euclideon infinite detail technology was exposed in a recent interview to be efficient sparse voxel octree ray-casting with contours (instead of axis-aligned cubes), which was previously investigated by nVidia researchers and others.
It's still not quite competitive enough with conventional rasterization techniques if you also try to do decent lighting with it along with all of the other stuff that goes into developing a game. Maybe in couple of more GPU generations. And in the Euclideon demos, you can see they're only getting 15-25 frames per second, and yet they're using a simplistic lighting model with a single global light.
Also, you can merge polygon rasterization and voxel ray-casting under the same rendering architecture if you employ deferred shading and composition. That way you can have your static models as a voxel-octree that you ray-cast into your geometry-buffer, rasterize your dynamic polygonal models in a second pass into the same geometry-buffer, and then perform your lighting passes with the geometry-buffer as input, and then your final scene composition and post-processing passes. In fact, I bet you that in 5-6 years time, this will become a standard practice among PC game developers. Unfortunately, I think the upcoming next console generation, with hardware from Sony and MS due out next year, will miss out on being able to handle this.
It should be noted that John Carmack believes that Ray Casting, not Ray Tracing, will win out in the long term.
Unfortunately, many people outside of the graphics field confuse the two. Ray Casting is a subset of Ray Tracing in which only a single sample is taken per pixel, or in other words, in which a single ray is cast per pixel into the scene and a single intersection is taken with the geometry data set (or in the case of a translucent surface, the ray my be propagated in the same direction a finite number of times until an opaque surface is found). No recursive bouncing of rays is done. Lighting is handled through another means, such as with traditional forward shading against a dataset of light sources, or using a separate deferred shading pass to un-hinge the combinatorial explosion of overhead caused by scaling up the number of lights in a scene.
John Carmack has been quoted on saying that full-blown ray-tracing just isn't feasible for real-time graphics due to the poor memory access patterns involved, as casting multiple rays per pixel, with multiple recursive steps ends up touching a lot of memory in your geometry data set, which just thrashes the cache on CPU and modern/future GPU hardware alike.
When people talk about real-time ray-tracing, they almost always invariably are referring to real-time ray-casting.
Position/location isn't stored at all, it's inferred from memory location and reconstructed incrementally as you traverse the data structure, much like how an array index isn't stored for each element in the array. And if you haven't noticed, everything is static and fixed. Even high-end multi-GPU systems can only handle N-body simulations with tens of thousands of particles at interactive rates, which is essentially what you're calling for... and yet with Euclideon's demo, they purportedly have tens of billions of voxels in a given scene.
You do not need a GPU with lots of RAM, although the more the merrier of course.
For example, Id Software's upcoming RAGE has texture data sets that are many GBs in size in one big contiguous world without different areas. How do you think they manage to load it all in and display everything on your typical 512MB to 1GB GPU, or even on an iPhone with 256MB of user mode memory? They dynamically stream in the content from disk based upon your position within the world and use LOD techniques.
Many current console games also do dynamic LOD streaming of texture data, polygonal meshes, animations, etc.
In the Gigavoxel and Efficient Sparse Voxel Octree papers, they discuss streaming to handle data sets larger than what is possible given GPU RAM. In fact, voxel octrees a good fit for streaming and LOD.
I guarantee you Euclideon is streaming voxel data from disk as needed and caching it on the GPU dynamically. You can do the streaming asynchronously in different threads running on different cores using asio/epoll/kqueues, so it scales nicely on multi-core systems.
Also, they did not say that their renderer is running on the CPU. They said their renderer is running "in software". Guess what GPGPU renderers are? They're software renderers.
No it's not. He mentions "point cloud" in his video, but he also used the term "atom" to describe a voxel or point, so his terminology is all off kilter to begin with. The qsplat sample images in your link look nothing like the sparse voxel octree or gigavoxel demos--the qsplatting has a lot of roughness along splat subdivision boundaries. The Euclideon demo yields results that are more in line with what you would expect from a voxel ray-casting renderer.
Furthermore, with qsplatting, you still end up with polygonal data that you need to texture, meaning you need to generate texture maps and texture coordinates for your modesl at the same time. With voxel raycasting, you do not need to bother generating texture maps, the voxel octree encodes material/color information intrinsically (you can achieve 1-bit per voxel effective compression with nVidia's efficient sparse voxel octrees implementation with diffuse and specular material colors), you just render the diffuse color into the frame buffer for each ray-voxel intersection, or if you're doing deferred shading, you render per-pixel material values into your g-buffer before you do your lighting pass.
And what do you mean you can't anti-alias point clouds? Of course you could anti-alias them. You can do anti-aliasing as a post-processing pass on any image data. Take a look at FXAA for example.
From a technical standpoint, I think it's obvious he's doing some sort of sparse voxel octree, and not qsplatting.
You still run into the same problem as you increase the resolution of your voxel octrees, thus increasing the depth of your octrees or spatial lookup structures, which requires that you need to recurse an more levels as you perform a spatial query, although it scales logarithmically instead of linearly. Still no reason to call it "unlimited."
Correct, but they actually use ray-casting, not ray-tracing. Ray-casting only involves a single ray collision test and sample per pixel, and then you need to use an alternative means to compute lighting, such as a deferred shading and lighting compositor. Full ray-tracing doesn't scale well for real-time graphics in shared memory systems due to memory access patterns involved. Intel has some demos with some simple car models doing full recursive ray-tracing, but it only runs at a few FPS even on 64 cores.
The technology itself isn't bullshit, but what is bullshit is that Euclideon is taking credit for other people's work.
They say they've invented the methods and algorithms behind it all, well that's just pure fantasy. Here's what Euclideon is basing there technology off of:
Furthermore, it's not "unlimited detail", you will always have a limited amount of RAM and disk space, therefore you have limits to your detail, even with procedural content generation. But your effective compression rate is 1-bit per voxel, so 20 billion voxels = 2.5GB, so you need to utilize dynamic content streaming from disk to the GPU.
Euclideon is just spinning up the marketing bullshit and trying to make a profit off of it all. They don't even have good lighting, they're just doing forward shading for each voxel ray-cast intersection using diffuse lighting with a single global point light source. And they haven't demonstrated robust animation yet.
Guess what, it is possible to animate voxel octrees, but Euclideon never came up with the method either. Some researcher in Germany came up with a working solution for his bachelor's thesis: http://www.youtube.com/watch?v=Tl6PE_n6zTk
Euclideon is unjustly taking credit for other people's hard work. They say they've invented the methods and algorithms behind it all, well that's just pure fantasy. Here's what Euclideon is basing there technology off of:
Furthermore, it's not "unlimited detail", you will always have a limited amount of RAM and disk space, therefore you have limits to your detail, even with procedural content generation.
Stop virally spreading this video, you're only helping Euclideon profit from more page views and attention.
I've recently started using Tundra, a very light-weight and scalable (CPU core wise) build system, with very fast and reliable dependency analysis written in C, using pthreads/winthreads and taking advantage of SIMD instruction sets. It supports multiple build configurations and variants side-by-side, multiple disjoint build targets/deliverables, and has very DRY LUA-based DSL for project configuration. It doesn't hide CFLAG details, it gets out of the way and lets you precisely control what you're passing to the compiler for each tool chain you want to support. It's released under the GPLv3, was written by a DICE employee and is being used to build Battlefield 3.
I was able to get rid of ~25 CMakeLists.txt that made up thousands of lines of text and replaced it with a single ~350 line tundra build file, in building a 200,000+ LoC multi-platform project and combined with a very stripped down autoconf script for environment detection and config.h generation, was able to support all of my requirements. I had to hack and hack to get CMake to try to support what I needed.
Because of GCC having adopted GPLv3 with GCC 4.3 and later. Apple doesn't want to use it. Clang/LLVM trunk is actually starting to get pretty decent at x86/x86-64, but ARM is still pretty crappy.
Think is, Clang has come a very long way in just a couple of years, if they keep up the same pace of development, it will most likely surpass GCC either next year or the year after.
Slashdot Mirror
And if you include lighting into the equation, with N = L lights + S geometric surfaces, he's advocating for O(c * 2^backtraces * N) over O([c1 * S] + [c2 * L]) that you would get with ray-casting + deferred shading&lighting, where it could be proven that c2 c1 c, seeing as how you also have to cast multiple rays per recursive back-trace to get a decent approximation of lighting in your scene, where you only need to perform a single sample per pixel with what I'm advocating for.
Yeah, not to mention that with full ray-tracing, as you add more lights to a scene, it increases the overall complexity per pixel per ray bounce linearly as well. That's why doing deferred shading is so nice because it unbuckles the lighting from the rasterization or ray-casting/tracing step and lets you scale up the number of lightings independently with a fixed amount of overhead and linear cost per light for the entire scene, instead of per pixel or per ray or per polygon.
Going with pure ray-tracing doesn't let you take advantage of that.
You still run into the same problems regardless of whether you're using a vector processor with no branch-prediction and no cache, to if you're using a bunch of in-order cores with cache coherency, or full-blown out-of-order cores with cache coherency. You end up pulling in a combinatorial explosion (ie. exponential number) of cache lines or memory accesses per recursive ray tied to the complexity of your scene.
People like to talk about how you can just throw more cores and distributed computing architectures to increase the performance of ray-tracing graphics. Yes, it works, but only up to a point. You can cut the number of days or weeks or months off of fully rendering a CG movie by building out a distributed render farm, but it's not going to hold for when you get single shared-memory machines with enough cores and you expect it to be able to do it all in real time.
It'll still be a much better use of those hardware resources if you go with algorithms which are much more orthogonal in nature when it comes to memory access patterns. Ray-casting + deferred shading should scale better than fully recursive ray-tracing, even on many-core shared-memory CPU architectures.
I admit, I made my post before watching the video. Now that I've watched it, I think my position still holds. He talked a lot about using ray-tracing in content preproduction and preprocessing tools during development, and he talked about building ray-tracing engines in the past to experiment with the performance characteristics. This is the same research he did where he came to the conclusion that ray-casting will be better than ray-tracing. Where his position has changed is in that he used to think it would only be feasible using voxel as your input data set, but he's starting to come around on ray-casting against analytical surfaces... polygonal meshes, curved surfaces, etc. in real time.
I can't tell if you're trolling or you just wrote up a reply without fully reading my post. I never said rasterization is better. I alluded to the fact that ray-casting will win out over rasterization, and in the very near future. What I said was the ray-casting will win out over ray-tracing. The algorithmic complexity of ray-casting + deferred-shading is better than recursive ray-tracing.
I also forgot to mention that conventional rasterization and ray-casting is also just as parallel in nature as ray-tracing. In fact, more so because it has much better memory access patterns as I mentioned in a previous post. Memory-access is the biggest limiting factor in building scalable, parallel systems. If you don't have good memory-access patterns, you might as well being doing sequential work, because it's getting serialized by the hardware memory controller anyway.
And this situation isn't improving, it's actually getting worse on each subsequent hardware generation... memory access is increasingly becoming a bigger and bigger bottleneck. I suggest you educate yourself on how memory works on modern cache-coherent hardware.
https://lwn.net/Articles/250967/
The video you posted is not real-time frame rates, it's interactive frame rates. It takes a few seconds to fully recompute the scene once you stop moving the camera. And note how there's only a single car model. imagine scaling up the amount of geometry to a full world. With ray-tracing, as you scale up the complexity of the geometry, you end up scaling up the required computational complexity as well due to radiosity computations. Full real-time ray-tracing on huge worlds in real-time is a pipe-dream. What you will be able to do with ray-casting or rasterization with deferred shading composition to simulate things like reflections or radiosity will always be more than what you can do with ray-tracing, and so games developers will always choose the former.
Yes, you do lose out on reflectivity, so you have to handle reflections through other means. If you use deferred shading, you can handle it in a separate reflective lighting pass in combination with something like cube-mapping. Even though you do a second pass to handle it, it'll still end up being faster than had you tried to implement fully recursive ray-tracing.
Ray-casting would still have the benefit in providing better scalability than polygonal rasterization as there is a fixed cost per pixel, whereas with polygon rasterization, as you scale up the complexity of the geometry, you increase the computational cost. And that's why Carmack thinks it will eventually win out against rasterization: ray-casting becomes a better use of the available computational resources you have at your disposal once you get past a certain level of detail in your scenes.
The Euclideon infinite detail technology was exposed in a recent interview to be efficient sparse voxel octree ray-casting with contours (instead of axis-aligned cubes), which was previously investigated by nVidia researchers and others.
http://www.tml.tkk.fi/~samuli/publications/laine2010tr1_paper.pdf
It's still not quite competitive enough with conventional rasterization techniques if you also try to do decent lighting with it along with all of the other stuff that goes into developing a game. Maybe in couple of more GPU generations. And in the Euclideon demos, you can see they're only getting 15-25 frames per second, and yet they're using a simplistic lighting model with a single global light.
Also, you can merge polygon rasterization and voxel ray-casting under the same rendering architecture if you employ deferred shading and composition. That way you can have your static models as a voxel-octree that you ray-cast into your geometry-buffer, rasterize your dynamic polygonal models in a second pass into the same geometry-buffer, and then perform your lighting passes with the geometry-buffer as input, and then your final scene composition and post-processing passes. In fact, I bet you that in 5-6 years time, this will become a standard practice among PC game developers. Unfortunately, I think the upcoming next console generation, with hardware from Sony and MS due out next year, will miss out on being able to handle this.
It should be noted that John Carmack believes that Ray Casting, not Ray Tracing, will win out in the long term.
Unfortunately, many people outside of the graphics field confuse the two. Ray Casting is a subset of Ray Tracing in which only a single sample is taken per pixel, or in other words, in which a single ray is cast per pixel into the scene and a single intersection is taken with the geometry data set (or in the case of a translucent surface, the ray my be propagated in the same direction a finite number of times until an opaque surface is found). No recursive bouncing of rays is done. Lighting is handled through another means, such as with traditional forward shading against a dataset of light sources, or using a separate deferred shading pass to un-hinge the combinatorial explosion of overhead caused by scaling up the number of lights in a scene.
John Carmack has been quoted on saying that full-blown ray-tracing just isn't feasible for real-time graphics due to the poor memory access patterns involved, as casting multiple rays per pixel, with multiple recursive steps ends up touching a lot of memory in your geometry data set, which just thrashes the cache on CPU and modern/future GPU hardware alike.
When people talk about real-time ray-tracing, they almost always invariably are referring to real-time ray-casting.
Position/location isn't stored at all, it's inferred from memory location and reconstructed incrementally as you traverse the data structure, much like how an array index isn't stored for each element in the array. And if you haven't noticed, everything is static and fixed. Even high-end multi-GPU systems can only handle N-body simulations with tens of thousands of particles at interactive rates, which is essentially what you're calling for... and yet with Euclideon's demo, they purportedly have tens of billions of voxels in a given scene.
You do not need a GPU with lots of RAM, although the more the merrier of course.
For example, Id Software's upcoming RAGE has texture data sets that are many GBs in size in one big contiguous world without different areas. How do you think they manage to load it all in and display everything on your typical 512MB to 1GB GPU, or even on an iPhone with 256MB of user mode memory? They dynamically stream in the content from disk based upon your position within the world and use LOD techniques.
Many current console games also do dynamic LOD streaming of texture data, polygonal meshes, animations, etc.
In the Gigavoxel and Efficient Sparse Voxel Octree papers, they discuss streaming to handle data sets larger than what is possible given GPU RAM. In fact, voxel octrees a good fit for streaming and LOD.
I guarantee you Euclideon is streaming voxel data from disk as needed and caching it on the GPU dynamically. You can do the streaming asynchronously in different threads running on different cores using asio/epoll/kqueues, so it scales nicely on multi-core systems.
Also, they did not say that their renderer is running on the CPU. They said their renderer is running "in software". Guess what GPGPU renderers are? They're software renderers.
You obviously didn't watch the video all the way through. They have scenes with tens of billions of voxels.
No it's not. He mentions "point cloud" in his video, but he also used the term "atom" to describe a voxel or point, so his terminology is all off kilter to begin with. The qsplat sample images in your link look nothing like the sparse voxel octree or gigavoxel demos--the qsplatting has a lot of roughness along splat subdivision boundaries. The Euclideon demo yields results that are more in line with what you would expect from a voxel ray-casting renderer.
Furthermore, with qsplatting, you still end up with polygonal data that you need to texture, meaning you need to generate texture maps and texture coordinates for your modesl at the same time. With voxel raycasting, you do not need to bother generating texture maps, the voxel octree encodes material/color information intrinsically (you can achieve 1-bit per voxel effective compression with nVidia's efficient sparse voxel octrees implementation with diffuse and specular material colors), you just render the diffuse color into the frame buffer for each ray-voxel intersection, or if you're doing deferred shading, you render per-pixel material values into your g-buffer before you do your lighting pass.
And what do you mean you can't anti-alias point clouds? Of course you could anti-alias them. You can do anti-aliasing as a post-processing pass on any image data. Take a look at FXAA for example.
From a technical standpoint, I think it's obvious he's doing some sort of sparse voxel octree, and not qsplatting.
You still run into the same problem as you increase the resolution of your voxel octrees, thus increasing the depth of your octrees or spatial lookup structures, which requires that you need to recurse an more levels as you perform a spatial query, although it scales logarithmically instead of linearly. Still no reason to call it "unlimited."
Correct, but they actually use ray-casting, not ray-tracing. Ray-casting only involves a single ray collision test and sample per pixel, and then you need to use an alternative means to compute lighting, such as a deferred shading and lighting compositor. Full ray-tracing doesn't scale well for real-time graphics in shared memory systems due to memory access patterns involved. Intel has some demos with some simple car models doing full recursive ray-tracing, but it only runs at a few FPS even on 64 cores.
Sorry, that first URL is missing a character at the end, it should be: http://research.nvidia.com/publication/efficient-sparse-voxel-octrees-analysis-extensions-and-implementation
The technology itself isn't bullshit, but what is bullshit is that Euclideon is taking credit for other people's work.
They say they've invented the methods and algorithms behind it all, well that's just pure fantasy. Here's what Euclideon is basing there technology off of:
http://research.nvidia.com/publication/efficient-sparse-voxel-octrees-analysis-extensions-and-implementatio [nvidia.com]
http://artis.imag.fr/Publications/2009/CNLE09/ [artis.imag.fr]
Here's video from 2009 which looks better than Euclideon's videos: http://www.youtube.com/watch?v=HScYuRhgEJw [youtube.com]
They might also be using other people's source code (legally though, it's released under the Apache license): http://code.google.com/p/efficient-sparse-voxel-octrees/ [google.com]
Furthermore, it's not "unlimited detail", you will always have a limited amount of RAM and disk space, therefore you have limits to your detail, even with procedural content generation. But your effective compression rate is 1-bit per voxel, so 20 billion voxels = 2.5GB, so you need to utilize dynamic content streaming from disk to the GPU.
Don't bother, they're taking credit for other people's work. You want to know how their technology works? Here's a couple of research papers:
http://research.nvidia.com/publication/efficient-sparse-voxel-octrees-analysis-extensions-and-implementatio
http://artis.imag.fr/Publications/2009/CNLE09/
Want some source code? http://code.google.com/p/efficient-sparse-voxel-octrees/
Want a video? http://www.youtube.com/watch?v=HScYuRhgEJw
Euclideon is just spinning up the marketing bullshit and trying to make a profit off of it all. They don't even have good lighting, they're just doing forward shading for each voxel ray-cast intersection using diffuse lighting with a single global point light source. And they haven't demonstrated robust animation yet.
Guess what, it is possible to animate voxel octrees, but Euclideon never came up with the method either. Some researcher in Germany came up with a working solution for his bachelor's thesis: http://www.youtube.com/watch?v=Tl6PE_n6zTk
Euclideon is unjustly taking credit for other people's hard work. They say they've invented the methods and algorithms behind it all, well that's just pure fantasy. Here's what Euclideon is basing there technology off of:
http://research.nvidia.com/publication/efficient-sparse-voxel-octrees-analysis-extensions-and-implementatio
http://artis.imag.fr/Publications/2009/CNLE09/
Here's video from 2009 which looks better than Euclideon's videos: http://www.youtube.com/watch?v=HScYuRhgEJw
They might also be using other people's source code (legally though, it's released under the Apache license): http://code.google.com/p/efficient-sparse-voxel-octrees/
Furthermore, it's not "unlimited detail", you will always have a limited amount of RAM and disk space, therefore you have limits to your detail, even with procedural content generation.
Stop virally spreading this video, you're only helping Euclideon profit from more page views and attention.
I want to see them suffer while the rest of us dine on Replicator synthesized gourmet cooking.
Sorry, a stray comma got into that URL. It should be: https://github.com/deplinenoise/tundra
I've recently started using Tundra, a very light-weight and scalable (CPU core wise) build system, with very fast and reliable dependency analysis written in C, using pthreads/winthreads and taking advantage of SIMD instruction sets. It supports multiple build configurations and variants side-by-side, multiple disjoint build targets/deliverables, and has very DRY LUA-based DSL for project configuration. It doesn't hide CFLAG details, it gets out of the way and lets you precisely control what you're passing to the compiler for each tool chain you want to support. It's released under the GPLv3, was written by a DICE employee and is being used to build Battlefield 3.
I was able to get rid of ~25 CMakeLists.txt that made up thousands of lines of text and replaced it with a single ~350 line tundra build file, in building a 200,000+ LoC multi-platform project and combined with a very stripped down autoconf script for environment detection and config.h generation, was able to support all of my requirements. I had to hack and hack to get CMake to try to support what I needed.
https://github.com/deplinenoise/tundr,a
http://j.mp/emwZ9Z for the PDF presentation slides.
Disclaimer: I have no relations with the author of Tundra or EA/DICE.
Yes. They must be compile time and they must not induce side-effects.
Because of GCC having adopted GPLv3 with GCC 4.3 and later. Apple doesn't want to use it. Clang/LLVM trunk is actually starting to get pretty decent at x86/x86-64, but ARM is still pretty crappy.
Think is, Clang has come a very long way in just a couple of years, if they keep up the same pace of development, it will most likely surpass GCC either next year or the year after.