Slashdot Mirror
Examining the User-Reported Issues With Upgrading From GCC 4.7 To 4.8
Nerval's Lobster writes "Developer and editor Jeff Cogswell writes: 'When I set out to review how different compilers generate possibly different assembly code (specifically for vectorized and multicore code), I noticed a possible anomaly when comparing two recent versions of the g++ compiler, 4.7 and 4.8. When I mentioned my concerns, at least one user commented that he also had a codebase that ran fine after compiling with 4.6 and 4.7, but not with 4.8.' So he decided to explore the difference and see if there was a problem between 4.7 and 4.8.1, and found a number of issues, most related to optimization. Does this mean 4.8 is flawed, or that you shouldn't use it? 'Not at all,' he concluded. 'You can certainly use 4.8,' provided you keep in mind the occasional bug in the system."
If it ain't broke, don't fix it. No need to upgrade.
Thanks for another worthless uninformative article.
Holy fuck, I sure won't be using this for anything mission-critical.
Though the code behaves differently with, and without optimisation, and does not work on the new compiler whereas it did on the old,
this does not mean it is a bug in the compiler.
GCC, Clang, acc, armcc, icc, msvc, open64, pathcc, suncc, ti, windriver, xlc all do varying optimisations that vary across version, and
that rely on exact compliance with the C standard. If your code is violating this standard, it risks breaking on upgrade.
http://developers.slashdot.org/story/13/10/29/2150211/how-your-compiler-can-compromise-application-security
http://pdos.csail.mit.edu/~xi/papers/stack-sosp13.pdf
Click on the PDF, and scroll to page 4 for a nice table of optimisations vs compiler and optimisation level.
_All_ modern compilers do this as part of optimisation.
GCC 4.2.1 for example, with -o0 (least optimisation) will eliminate if(p+100p)
This doesn't on first glance seem insane code to check if a buffer will overflow if you put some data into it. However the C standard says that an overflowed
pointer is undefined, and this means the compiler is free to assume that it never occurs, and it can safely omit the result of the test.
I've only run into a few compiler bugs (like the one in this article, most always due to the optimizers), and it was always so incredibly aggravating, because it's easy to believe that compilers are always perfect. Granted, they might not produce the most efficient code, but bugs? No way! Of course I know better now, and most of the bugs I came across were back in the Pocket PC days when we had to maintain 3 builds (SH3, MIPS and ARM) for the various platforms (and of course the bugs were specific to an individual platform's compiler, which made it a little easier actually to spot a compiler bug, when a simple piece of code worked on 2 of 3 architectures).
Better known as 318230.
Some people see "bugs," others see "features."
I've seen solution features designed around security holes before, and when we finally patched the breach, we received emails demanding that the decision be reversed and how dare we break customer solutions by surreptitiously patching things!
Sometimes you never can win.
-- "Simplicity is prerequisite for reliability." --Dijkstra
The article basically says:
"GCC 4.8 includes new optimizations! Because of this, the generated assembly code is different! This might be BAD."
Like, duh? Do you expect optimizations to somehow produce the same assembly as before, except magically faster?
The linked "bug" is here: http://stackoverflow.com/questions/19350097/pre-calculating-in-gcc-4-8-c11 - which says, "Hey, this certain optimization isn't on by default anymore?" And to which the answer is, "Yeah, due to changes in C++11, you're supposed to explicitly flag that you want that optimization in your code."
So, yeah. Total non-story.
And that's true: for microcontrollers. Knowing the assembly language is useful because you can actually know what is going to happen next in terms of instructions fetched, decoded and executed. It's a whole different ballgame with multiple cores, dynamic scheduling, out-of-order execution, several layers of on-die cache and pre-executed branches. While the compiler (in this case) may not always get it 100% right (yet), at least it's going to do things *consistently*.
One of the projects I work on will compile and run perfectly with GCC 4.6 and any recent version of Clang. However, compiling under GCC 4.7 or 4.8 causes the program to crash, seemingly at random. We have received several bug reports about this and, until we can track down all the possible causes, we have to tell people to use older versions of GCC (or Clang). Individual users are typically fine with this, but Linux distributions standardize on one version of GCC (usually the latest one) and they can't/won't change, meaning they're shipping binaries of our project known to be bad.
So, as has always been the case: use optimizers with caution, and verify the results. This is standard software development procedure. Some aspects of optimization are deterministic and straightforward, and are therefore pretty low risk; others optimizations can have unpredictable results that can break code.
He actually observed that different assembler code was generated - well how do you think can you generate _faster_ assembler code without generating _different_ assembler code?
The article does _not_ make any claim that any code would be working incorrectly, or give different results. The article _doesn't_ examine any user-reported issues. So on two accounts, the article summary is totally wrong.
I _cannot wait_ to see how much hilarity ensues in the Gentoo world, where it's real common for random clowns with no debugging (or bug reporting) ability to have -Oeverything set.
Well you are correct, but other languages just throw an exception or error when you do something that ought to be undefined behavior. Many C / C++ undefined behaviors allow your program to keep running when you do something stupid.
I'm not saying the problem is the language, if it did the same things the other languages do to prevent that kind of stuff the performance hit would be noticeable.
I forgot to mention that some compilers leave to the operating system to crash your application on some undefined behaviors. Which brings its own cross-platform problems.
Having been somewhat involved in the migration of a lot of C++ code from older versions of gcc to gcc 4.8.1, I can tell you that 4.8.1 definitely has bugs, in particular with -ftree-slp-vectorize. This doesn't appear to be a huge problem in that almost all the (correct) C++ code we threw at the compiler produced good compiler output, meaning that the quality of the compiler is very good overall. If you do find a bug, and you have some code that reproduces the problem, file a bug report, and the gcc devs will fix the problem. At any rate, gcc 4.8.2 has been out for a number of months now, so if you're still on 4.8.1, you may want to upgrade.
Please correct me if I got my facts wrong.
Protip: Don't confuse compiler devs with people who specify programming languages. It makes you look stupid.
A successful API design takes a mixture of software design and pedagogy.
I haven't tried this with the latest version by even a version 4.x GCC cannot generate inline code with the 8 bytes version of cmpxchg with 32bit code. Doing this in a function is OK.
I think the problem is that this instruction almost takes up all of the registers and GCC cannot cope with this if you want to do it inline.
cmpxchg8b is useful for lock-free code.
Government cannot make man richer, but it can make him poorer. - Ludwig von Mises
Just because invoking an undefined case can essentially cause any effect (even launching nuclear missiles) doesn't mean doing that is a goal.
Here's some others: Don't confuse undefined behavior with useless behavior. Don't confuse undefined behavior with a free ticket to generate whatever crap code you'd want.
Ah, you must have gotten the condensed version, I think the complete quote is supposed to be something along the lines of:
Trust the compiler, the compiler is your friend. But it's one of those asshole friends that will draw on your face if it catches you sleeping, and will sometimes make stupid mistakes with the best of intentions. So you probably want a good understanding of at least the basics of what it's doing, and the most common ways it gets confused.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
>GCC 4.2.1 for example, with -o0 (least optimisation) will eliminate if(p+100p)
Seriously? Wait, no, I thing Slashdot just ate your <, and that should be if(p+100 < p)
edit: Wait, Slashdot silently swallows malformed "HTML tags", but doesn't parse < properly? How the $#@! are you supposed to include a less-than sign?
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Well you are correct, but other languages just throw an exception or error when you do something that ought to be undefined behavior. Many C / C++ undefined behaviors allow your program to keep running when you do something stupid.
Well, in some cases. Problems like a [i] = i++; go away in Java because it defines what should happen: The expression is evaluated strictly left to right, so the old value of i is stored into a [i], then i is increased by 1. Bad array indexes an nil pointers throw exceptions (which themselves will cause trouble if this was unexpected). But there are things like "restrict" in C which causes undefined behaviour if used wrong, but that is explicitly intended because it allows serious compiler optimisations.
(there is no such thing as C/C++)
This is Slashdot, not HRhangout. We know they're two different languages. We also know they're closely related and share a lot of similarities, since C++ is, after all, a descendant of C (with a lot of big changes, obviously). So "C/C++" is a convenient way to save some keystrokes.
We can't help it if a bunch of HR morons think they're the same language.
<< LIke This >>
Hint: the trailing ';' is not optional.
Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
On a related note, does anyone have any suggestion on how to track down such bugs? Are there for example code-analysis tools that will highlight code with undefined behavior likely to give different results when optimized, or valid code that may trigger known compiler bugs? It seems like such a thing would be immensely valuable - if I have a compiler-related mystery bug *somewhere* in my codebase, being able to narrow that down to even the 0.1% of lines containing "suspicious" code could make the difference between it being impossible to solve and merely difficult.
In fact I'm rather surprised that "this code may cause undefined behavior" isn't a standard compiler warning. I mean C and C++ are performance-oriented languages that practically invite developers to come up with "clever" solutions, a warning that they have exceeded the sometimes non-obvious limits of defined behavior would probably save more debugging hours than any other warning on the planet.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
C does what you tell it to.
If you tell it to do something stupid, it will still try to do it.
If it's stupid, then the compiler should have issued an error.
It's up to YOU to not tell it to do stupid things.
Which is silly, because the reason computers exist in the first place is to help us slow, error-prone humans by doing logical computations for us.
Maybe you need a static code checker?
Yes, but the static code checker should have been built into the compiler from day one.
Agreed. Iit seems like "This code may cause undefined behavior" would be one of the single most time- and aggravation-saving warnings a compiler could possibly emit. Granted 100% detection might be challenging, but even detecting the 80% of most common and non-obvious culprits could save millions of man-hours hunting down obscure bugs.
Heck, it would seem to me that any time n optimizer is able to completely eliminate *anything* from my code (outside of instances of inlined or templated functions) then there's probably a problem - either I've written pointless code, or I've invoked undefined behavior. Even if the tie to source lines is tenuous at that point a "Optimization has completely eliminated some code in function Foo" warning would be valuable draw attention to potential problem spots, though I imagine false-positives might be a major issue for a naive warning implementation.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Humans occasionally make mistakes, invariably. Regardless of talent, skill, intelligence, or experience.
There's no need to change the language, a simple "this code may invoke undefined behavior" compiler warning would eliminate countless hours of debugging.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
> counting backwards is faster because comparing to the static value 0 is faster
Quite so. However if you're counting there's a pretty good chance that you're traversing an array, in which case caching optimizations that presume a forward traversal will tend to completely overwhelm any potential gains from comparing to a constant instead of a variable.
While we're ranting, why is _++ even a distinct operator from ++_? Are there really that many situations where i++ can streamline the code significantly? We're only human, it's physically impossible for us to retain and apply every subtly of an 800+ page language specification indefinitely, even if it were completely unambiguous to begin with.
As for your example, actually there's a very good reason for it to be undefined behavior: The fewer restrictions placed on the order of evaluations of generally non-interacting elements, the more opportunities are exposed for optimization, especially once you bring parallel and vectorized CPUs into the equation. And imposing deterministic evaluation order only when side effects may be present then introduces numerous special cases which require expanding the standard and drastically complicating compiler implementation, with a corresponding increase in bugs and reduction in optimizations due to excessive caution. Warnings would be nice though. Even if the fact that Funky( Foo(), Bar() ) causes undefined behavior because of side effects buried 63 function-calls down would likely still slip by, at least the most obvious problems would be caught and perhaps serve as a reminder that caution must be exercised.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Is gcc 4.8 the one where the compiler source was completely converted to C++?
/me ducks.
Stick Men
The start of the summary was just so bizarre to me. Of course different versions generate different code, that's what happens when you change how code is optimized. Why would someone set out to investigate this, except as a question about how it improves the code.
Now if there's a bug that's a different issue, and all compilers are going to have some sort of bugs somewhere as these are complex pieces of code. But a change in the output should never be treated as evidence of a bug.
Well, it kindof is, but the spec doesn't try to call out every stupid thing you need to warn users about. Compiling with warnings, extra warnings, and pedantic warnings enabled will find a lot of stuff. But those are warnings, not errors, for a reason: sometimes you really do know better.
Socialism: a lie told by totalitarians and believed by fools.
Optimizers "completely eliminate" code all the time, at least for good code. You'd drown in such warnings in the last C codebase I worked on. E.g., our standard called for all variables to be initialized at declaration, in some trivial way that can't itself have errors. So you get a lot of: ... // do something with foo
int foo = 0;
foo = NonTrivialFunction(bar);
We expect the optimizer to remove the "= 0" wherever it's redundant, so there's no performance cost to this helpful technique.
Or again ...
MyClass *pFoo = NULL;
pFoo = FindIt("bar");
if (pFoo == NULL)
{
return ERROR;
}
Perhaps the compiler can prove that FindIt will always return a non-null value for the constant input, and can optimize away the null check. That doesn't in any way make the null check bad code (nor the initial NULL assignment, which likely got optimized away as well).
Socialism: a lie told by totalitarians and believed by fools.
Wow, slashcode randomly removes newlines in code blocks. WTF? Well, there's always tt.
Socialism: a lie told by totalitarians and believed by fools.
While we're ranting, why is _++ even a distinct operator from ++_?
Some platforms has a preincrement and postdecrement opcodes, but not the converse. Some vice versa. Once upon a time, a programmer would care about such things, and C would never have gained the acceptance it did without the ability to use what looked fast on your platform. Heck, there are still coders who use the "register" keyword, as if that still did anything.
Socialism: a lie told by totalitarians and believed by fools.
The GP has a valid point. Most people complaining of these optimizer "bugs" likely have undefined behavior. In C & C++, the compiler/optimizer/linker is given full freedom on what to do. Often, the compiler will just eliminate the code. It could, in theory, format your hard drive. Yes, compiler bugs do happen, but they tend to be rare and infrequent. Last GCC bug I saw was on a minor revision of 4.1.2 that caused an ICE (internal compiler error) when you had an anonymous namespace at the global namespace level.
Um, the multi-threaded memory model for C++ was just recently defined in C++11. Prior to that standard, there wasn't even an acknowledgement of multithreading/concurrency.
Ugh, okay. I might still argue that the compiler is in fact detecting a problem, but there's no need to make the programmers drown in warnings they're not allowed to fix.
But... point made. There would need to be a sanity line in there somewhere - like say if he compiler uses non-trivial contextual information to judge that things could safely be eliminated. Or if the elimination is only made possible by the choices made in interpreting undefined behavior. But the optimizer may well have already lost the contextual information necessary to make those calls.
Yeah, that newline thing is annoying. IIRC it mostly applies to "..." and possibly some other punctuation. And might not be restricted to code, let's see: ...
line 1
line 3
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Ah, that would explain it.
And I imagine there's some compilers for embedded systems that still take "register" seriously.
--- Most topics have many sides worth arguing, allow me to take one opposite you.
Just because *you* write undefined behavior, doesn't make the code generated by the compiler any less or more valid. The compiler/optimizer are trying their best to generate the fasted code possible from the information given (by you). If you give it invalid or incomplete information, don't be surprised that the behavior isn't what you expected.
Your compiler will then also be able to program your application, mission accomplished I guess.
Wait, what? How do you figure? All I'm saying is that the optimizer, which is already exploiting all manner of non-obvious opportunities for code elimination, could include a heuristic to warn about optimizations which may cause unintended behavior.
And apparently the STACK project sort of does this already, though it isn't actually compiler per se. If I understand correctly it detects "dangerous" code by:
1) Performing all normal, boring dead-code elimination
2) Performing optimizing code elimination on (1), without the benefit of exploiting undefined behavior
3) Performing optimizing code elimination on (1), while allowing maximum exploitation of undefined behavior.
4) Reporting any code eliminated by (3) but not (2) as a likely problem spot
--- Most topics have many sides worth arguing, allow me to take one opposite you.
In the bigger picture it's the right thing to do (it's mostly about pointers). You init the pointer to NULL in the init block. If you accidentally use it before initialization, you get a well-understood error condition, and you follow a well defined pattern in your exit block of freeing-if-not-NULL. It rarely comes up for ints, really.
Socialism: a lie told by totalitarians and believed by fools.