Slashdot Mirror
Code Quality In Open and Closed Source Kernels
Diomidis Spinellis writes "Earlier today I presented at the 30th International Conference on Software Engineering a research paper comparing the code quality of Linux, Windows (its research kernel distribution), OpenSolaris, and FreeBSD. For the comparison I parsed multiple configurations of these systems (more than ten million lines) and stored the results in four databases, where I could run SQL queries on them. This amounted to 8GB of data, 160 million records. (I've made the databases and the SQL queries available online.) The areas I examined were file organization, code structure, code style, preprocessing, and data organization. To my surprise there was no clear winner or loser, but there were interesting differences in specific areas. As the summary concludes: '..the structure and internal quality attributes of a working, non-trivial software artifact will represent first and foremost the engineering requirements of its construction, with the influence of process being marginal, if any.'"
Or the summary is completely incomprehensible?
Of course, I could try to RTFA, but hey, this is Slashdot, after all...
No sig for the moment.
I can't wait to see what kind of ignorant anti-Windows screeds the 13-year-olds who post on Slashdot during recess will come up with in response to this article! I'm betting five minutes, tops, until someone posts a comment that spells "Microsoft" with a "$". Don't let me down, Slashbots!
That you have neither capitalized on your shared synergies, nor have you recovered your cherished paradigms.
Oh. Wait. This is about propeller-head stuff rather than management stuff. Lemme get my "Handbook of postmodern buzz words"...
"If god did not exist, it would be necessary to invent him" --Voltaire
Results inconclusive. How does he define what is trivial and what is not. Also this is all based on x86... he should compare the code bases of stuff that runs on Alpha, and PowerPC as well.
Which of the samples were closed source? And........ how did he get a hold of it/them? Open source software is still open even if it came from microsoft.
Final line in the paper: "Therefore, the most we can read from the overall balance of marks is that open source development approaches do not produce software of markedly higher quality than proprietary software development."
Interesting, but not shocking for those who have worked with disciplined commercial teams. I wonder what the results would be in less critical areas than the kernel, say certain types of applications.
..that Open Source code is of quality, but at least the point of things like the GPL is that you have the power to change that, and improve that code..
"The OpenSolaris kernel was a welcomed surprise: it was the only body of source code that did not require any extensions to CScout in order to compile."
Given that the Solaris kernel has been compiled by two very different compilers (Sun Studio, of course, and gcc), it isn't that surprising. Because of the compiler issues, it is likely the most ANSI compliant of the bunch.
If I am understanding correctly, you were looking for 'winners' and 'losers' (weasel words in and of themselves, but anyway...) in terms of 'quality' (another semi-subjective term that could make someone go crazy and drive motorcycles across the country for the rest of their lives).
You found that '..the structure and internal quality attributes of a working, non-trivial software artifact will represent first and foremost the engineering requirements of its construction, with the influence of process being marginal, if any.' -- or in plain English: "the app specs had a much bigger influence when compared to internal efficiencies".
I would wonder if you're just seeing a statistical wash-out. Are you dealing with data sets (tens of millions of lines and thousands of functions) that are so large, that patterns simply get washed out in the analysis?
Oh dear, my post is no more clear than the summary...
davejenkins.com |
Well, you lose your bet, been over five minutes and no anti Microsoft screeds let alone spelling it with a $.
Just so everyone understands, the tactic used here is known as "Poisoning the well." The idea is the discredit an argument's source before the argument is presented. Here, our AC friend is trying to ward off criticism of Microsoft by insinuating that anyone who does so is a 13 year old "Slashbot."
The fallacy is in the fact that even someone who is 13 and often goes along with the Slashdot zeitgeist may still have legitimate criticisms of Microsoft, such as the fact that Microsoft sucks giant hairy donkey balls.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
I'm sorry, but if this is what passes for serious academic computer-science work, close the schools. This all appears to boil down to: quality code (definition left to the reader) is produced by good programmers (can't define, but I know one when I see his/her code) who are given the time to produce quality code. Rushed projects by teams of average-to-crappy programmers results in low-quality code. All the tools and management theories in the world have little impact on this basic fact of life. My PhD, please?
So while looking at the data collected, I had to wonder if some of the conclusions reached were not something of a matter of weighting - I saw some things pretty troubling about the WRK. Among the top of my list was a 99.8% global function count!!!
This would explain some things like lower LOC count - after all, if you just have a bunch of global functions there's no need for a lot of API wrapping, you just call away.
I do hate to lean on LOC as any kind of metric but - even besides that, the far lower count of Windows made me wonder how much there, is there. Is the Windows kernel so much tighter or is it just doing less? That one metric would seem to make further conclusions hard to reach since it's such a different style.
Also, on a side note I would say another conclusion you could reach is that open source would tend to be more readable, with the WRK having a 33.30% adherence to code style and the others being 77-83%. That meshes with my experience working on corporate code, where over time coding styles change on more of a whim whereas in an open source project, it's more important to keep a common look to the code for maintainability. (That's important for corporate code too - it's just that there's usually no-one assigned to care about that).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It is easy to explain why the results are not conclusive, those metrics do not measure the actually quality of code.
the worst for what applications?
If good code and bad code were a simple automated analysis away, don't you think everyone would be doing it? What methodolgy could possibly give a quantitative weighting for "quality"?
"To my surprise there was no clear winner or loser..." Not really a surprise at all, actually.
Then I realized it is rather pointless. That did not prevent me from presenting it anyway, after all most of the academia works this way. When I get out of the university, I will read and understand the code, maybe maintain it for a year, before judging its quality.
"Therefore, the most we can read from the overall balance of marks is that open source development approaches do not produce software of markedly higher quality than proprietary software development."
That phrase should be reversed.
Have you ever seen the code that was leaked on Windows 2000? It was absolutely exactly like the same quality of code in Microsoft's SDKs (Software Development Kits) for Visual C++ and some of their source code examples for MSDN subscriptions are pretty much a mirror image of the quality. Absolutely fucking horrific. No wonder they take forever (if they can even bother to try) to fix bugs and security holes. It's just a god damn nightmare from hell.
People make claims about the need for closed source all the time, usually revolving around the need to a predictable level of quality, or some other factor. The fact is, this results proves that its a wash whether you choose open or closed--so why not choose open?
There's a deep significance here I'm failing to capture completely. Someone else word it better if they can. But there didn't need to be some blow-out victory of open source over closed source for this to be a victory. All open source needed to do was compare--which it did, clearly--with closed source, in terms of value, to secure its worth.
The way you choose to license your software doesn't coralate with software quality... Seems logical to me. As how you license your software has very little to do about the code inside the OS.
Closed Source Developer: I will try to do my best job as I possibly can so I can keep my job and make money because that is what I value.
Open Source Developer: I will try to do my best job as I possibly can so I can help the comunity and feel better about myself/get myself noticed in the comunity/Something cool to put on my resume... because that is what I value.
People who choose to license their software OpenSource vs. Closed Source says nothing about their programming ability. There are a bunch of really crappy GNU projects out there as well as a bunch of crappy closed source projects... Yea there is the argument of millions of eyes fixing problems but really when you get millions of people looking at the same thing you will get good and bad ideas so the more good ideas you get the more bad ideas you get and the more people involved the harder it gets to weed out good ones and bad ones. Closed source is effected often by a narrow level of control where bad ideas can be mandated.... All in all everything really ballances out and the effects of the license are negledgeable.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
I haven't seen anybody else comment on the fact that the statement that the quality of the code had more to do with the engineering than the process through which the code was developed is quite interesting.
;)
From my personal experiences, it typically seems code is written to solve a specific need. Said another way, in the pursuit of solving a given problem, whatever engineering is required to solve the problem must be accomplished - if existing solutions to problems can be recognized, they can be used (for example, Gang of Four/GOF patterns), otherwise, the problem must have a new solution engineered.
Seeing as how there are teams successfully developing projects (with both good, and bad code quality) using traditional OO/UML modeling, the software development life-cycle, capability maturity model, scrum, agile, XP/pair programming, and a myriad of other methods, it would seem to be that what the author is saying is, it didn't necessarily matter which method was used, it was how the solution was actually built (the.. robustness of the engineering) that mattered.
Further clarification on the difference between engineering and "process" would strengthen this paper.
I went to a Microsoft user group event some time ago - and the presenter described what they believed the process of development of code quality looked like. They suggested the progression of code quality was something like:
crap -> slightly less crappy -> decent quality -> elegant code.
Sometimes, your first solution at a given problem is elegant.. sometimes, it's just crap.
Anyways, just my two cents. Maybe two cents too many..
SixD
Don't feed the troll, compadre!
No sig for the moment.
It would be nicer to say it's a Piece of WoRK.
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
"Linux excels in various code structure metrics, but lags in code style. This could be attributed to the work of brilliant motivated programmers who aren't however efficiently managed to pay attention to the details of style. In contrast, the high marks of WRK in code style and low marks in code structure could be attributed to the opposite effect: programmers who are efficiently micro-managed to care about the details of style, but are not given sufficient creative freedom to structure their code in an appropriate manner. "
How ever I was left wondering how it was possible to compare fairly? He already stated:
"Excluded from the kernel code are the device drivers, and the plug-and-play, power management, and virtual DOS subsystems. The missing parts explain the large size difference between the WRK and the other three kernels."
and reading I see even more of the drivers aren't there:
"The NT Hardware Abstraction Layer, file systems, network stacks, and device drivers are implemented separately from NTOS and loaded into kernel mode as dynamic libraries. Sources for these dynamic components are not included in the WRK. "
http://www.microsoft.com/resources/sharedsource/licensing/researchkernel.mspx
So it's not like for like. Maybe you would draw different conclusions if it was, maybe the Linux style issue is because of all the drivers the WRK lacks. So even though I think his conclusion sounds probable, I don't feel I can state it as so with any confidence.
such as:
define how_good
return crash count
end
now that would give you a winner,
Interesting, but not shocking
Considering that few open source developers are strictly open source, that's hardly a surprise. I'd be willing to bet many open source developers are also part of disciplined commercial teams.
The flip side of that coin is just as intriguing. Open source development models don't produce software of notably inferior quality either. That should send a shivey through Castle Redmondore.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
The metrics used in this paper are lame. They're things like "number of #define statements outside header files" and such.
Modern code quality evaluation involves running code through something like Purify, which actually has some understanding of C and its bugs. There are many such tools. This paper is way behind current analysis technology.
Sorry, I've been in the business for over 25 years and had to hear one pin head after another spout about code quality or productivity. Its all subjective at best.
The worst looking piece of spaghetti code could have fewer bugs, be more efficient, and be easier to maintain than the most modular object oriented code.
What is the "real" measure of quality or productivity? Is it LOC? No. Is it overall structure? no. Is it the number of "globals?" maybe not.
The only real measure of code is the pure and simple darwinian test of survival. If it lasts and works, its good code. If it is constantly being rewritten or is tossed, it is bad code.
I currently HATE (with a passion) the current interpretation of the bridge design pattern so popular these days. Yea, it means well, but it fails in implementation by making implementation harder and increasing the LOC benchmark. The core idea is correct, but it has been taken to absurd levels.
I have code that is over 15 years old, almost untouched, and still being used in programs today. Is it pretty? Not always. Is it "object oriented" conceptually, yes, but not necessarily. Think the "fopen,"fread," file operations. Conceptually, the FILE pointer is an object, but it is a pure C convention.
In summation:
Code that works -- good.
Code that does not -- bad.
A couple of issues with your metrics:
... ...
... ... ...
... ... ...
...
...
1. "Figure 6: Common coupling at file and global scope."
Shouldn't the "coupling" be weighted by the actual "scope" size? For example:
- Having a small file with lots of file global symbols shouldn't increase complexity much.
- Having 1 global symbol in all files is far better than 10 global symbols in 1/10 of the files.
2. "Strictly structured functions are those following the rules of structured programming: a single point of exit and no goto statements."
Having lots of gotos/labels doesn't increase the code complexity per se. Having nested gotos, mix of gotos jumping forward and backwards, intermixed gotos and labels does. For example, this is simple:
goto a;
goto b;
goto c;
a:
b:
c:
while this is unreadable (but uses the same number of labels and goto statements):
b:
goto c;
a:
goto b;
c:
goto a;
b:
Rui
It's a well known fact that code will always resemble the institution that produced it, to some extent. To describe the Microsoft code as "poorly structured" is likely a bit out of touch.
The absolutely best kernel code is generally extremely beautiful and descriptive when dealing with the system's abstracts (with nice, long descriptive names for each function) and then unbelievably hellish and ugly in the sections that deal with hardware. Kernels represent an intersection between the idealistic system code and the hideously complex and inhuman machine interaction code. For this reason, we gauge the value of the systems based on how cleanly they compile into assembly, their performance, and ideally how well they do what they were written to do.
Kernel code fills such a complex role in the computer science paradigm that it is likely impossible to gauge the value or quality of any of them through any sort of automated means. What we have here is a mess of a research paper that comes to no obvious conclusions because they didn't really discover anything. If it were of any value, its final summary and conclusions wouldn't be so obfuscated. The researcher may or may not have mastered the art of understanding the zeitgeist of kernels but he certainly hasn't mastered the research paper.
hmm ....i see
By the time you get a car/plane that's accepted by the "market", the insides are going to be of a certain quality, no matter what the process used.
:)
;).
Did I get that right?
So now the question is how long did it take to get that level of quality. And maybe there is a difference in quality but the measurement used is not sensitive enough, or not as appropriate or his conclusion isn't quite correct - he measured a difference, just didn't show in his conclusion
are as far as modules are concerned then BEST way to handle partial setup in the module loading when the module doesn't register fully and must unload itself.
That's going to up the GOTO count in Linux, especially since drivers were left out of Windows.
Statements like this: "Indeed the longest header file (WRK's winerror.h) at 27,000 lines lumps together error messages from 30 different areas; most of which are not related to the Windows kernel." Allow me to feel smug in my anti-Windows bias
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
This leaves just one closed-source kernel and two open source kernels. If you were to perform a comparitive study between drugs on the basis of three subjects, you'd be laughed out of the research field so fast you'd hit escape velocity and be the first human on Mars.
Now, what OS' could potentially be studied? This depends on who can be talked into supplying software source and whether the researchers are willing to sign NDAs. I'd argue that you want at least three OS' in the closed and open categories, and two in the hybrid - one in each direction.
The closed-source OS' you'd want to examine - if you could get the code - would be Windows (as is already the case), LynxOS-178 (a moderately battle-hardened RTOS) and Trusted IRIX (massively hardened secure OS). In principle, the three design methods and requirements are different to such an extreme that this will give you the best possible picture of the range of code development practices. The idea would be to not test code that would have had similar constraints, because that only tells you about one aspect of software design. If, on the other hand, you pick three points that are more-or-less on different extreme points on the boundary of closed source design, you can get a picture of what that boundary might look like. Substituting any RTOS and Trusted OS for LynxOS-178 and Trusted IRIX respectively would work just as well, the idea is to get the extremes, not get specific implementations of those extremes.
The open-source OS' would include Linux and one of the BSDs - but it has to be a BSD that follows a markedly different ideology, so OpenBSD might be a good choice - but would probably also include one of the BeOS clones and/or a research OS like ExoPC. Again, the idea is to mark out the boundary as carefully as possible. The more data points you have, from points as far apart as possible, the better the understanding of that boundary.
What you will end up with is not a handful of values that have no meaningful basis for comparison, but two overlapping shapes. It'll look something like a Venn diagram. Hybrid methodologies will not merely appear somewhere between the two furthest extremes between the two shapes, but must exist only within the intersection between the two groups, and only there. No other midpoints would be valid.
To test this, we'd use a selection of hybrid designs. Plan9 started closed but has been open the longest, so I'd expect that to be in the intersecion but closer to the midpoint of the open source group than the midpoint of the closd source group. OpenSolaris has been open a relatively short time, so I'd expect the converse to be true for that.
This would still not be a great study, but it would increase the number of data points and provide some measure of control over the experimet. Since you cannot sensibly have a control group in this, the only controls you can assert are those of ensuring the samples are representative of the diversity in techniques, ensuring that the software is correctly classified, and ensuring that the results are sanity-checked.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Comparing Linux to the Windows Kernel and concluding there's no difference between them ignores a decade of operational experience. A "study" that concludes otherwise is wrong. Most people working on Linux would consider it flamebait.
Go ahead and mod yourself up.
I am a name troll of Westlake. Visit my homepage to learn why.
I haven't read the entire paper; I was mainly interested in the metrics you chose. I have to say that I'm not very impressed.
One of the things you've proven is that closed source code is more likely to have consistent conventions for typedef identifiers and aggregate tags, and to spell check their comments. This would be expected, as it's all about house style, conventions, and of course a homogeneous development community will have more consistent application of conventions than a heterogeneous one (as found in an open source project). That doesn't have very much to do with actually code quality, however.
One thing I found very telling is that in the Windows Research Kernel, the percentage of identifiers with wrongly global scope is twice as high as that in Solaris, and twenty times as high as that in Linux! That really is a metric of code quality.
One I was looking for, accepting the limitations of your data mining approach, was some metric with regard to garbage collection - that should be something that your approach would give you, and you are dealing mostly with C code, where garbage collection definitely matters.
Other real metrics for code quality, unfortunately, are qualitative, and so don't lend themselves to a data mining approach: how explicit are the comments? How well do they match the lines they are commenting? Are the comments high in signal-to-noise, or is a lot of unnecessary junk thrown in there, too? How well do the names of the variables and functions match their uses (i.e., is the code self-commenting to any degree)? Are there a lot of unnecessary structures in the code, representing developmental cul-de-sacs, unused code left behind when a new block replaced an obsolete one, etc.
I also think it's a methodologically disingenuous for you to use the Microsoft Research Kernel. You say that the size difference isn't significant because you've found that the quality of modules created by the same community tends to be consistent. What you haven't explained, though, is how research code that was prepared for publication can in any way be said to be representative of the quality of closed production code produced by the same community. Also, by using the kernel only to represent Windows, you're excluding the primary source of heterogeneity in an operating system: the device drivers. Since most Windows device drivers are also closed source, including them would lend statistical truth to your findings.
Obviously, though, convincing both Microsoft and a significant number of driver vendors to allow you to analyze their closed source code for quality would not have been a very easy thing to manage. Unfortunately, I would argue that this skews your results to near uselessness. Over all, what you've proven that a chunk of the Windows kernel cleaned up for publication is orthographically cleaner than the entirety of Linux. That's not a particularly insightful result, I'm afraid.
Eh. I stopped reading at To my surprise there was no clear winner or loser.
I mean, talk about shilling for Microsoft!!
Isn't NetBSD the system filled with academics who insist upon clean, manageable, and portable code above all other standards? Too bad the NetBSD kernel didn't get judged here, I suspect it would have taken the cake.
I still recall this exhaustive report comparing several kernels' performance back in 2003 in which NetBSD pretty much beat the pants off of everybody else (note the two updates with separate graphs). The initial poor performance was due to an old revision, and upon seeing that there were some places in which the newer revision wasn't so hot, the developers fixed them and in only two weeks, NetBSD beat out FreeBSD on every scalability test. Their pragmatism and insistence on quality code finally paid off.
Ever since seeing those charts, I've been waiting for Debian/NetBSD to come out...
Use my userscript to add story images to Slashdot. There's no going back.
Oops, it needs a trailing slash: http://bulk.fefe.de/scalability/
Use my userscript to add story images to Slashdot. There's no going back.
wtf.n0x.org
Am I the only one who noticed that he's comparing releases of various kernels to pre-releases of others? For instance, despite the fact that he's using a release version of WRK, he's using a checkout of HEAD from FreeBSD, which is just the latest code to be checked into CVS. FreeBSD goes through many months of vetting before release, and that bears heavily on release quality. HEAD isn't even always guaranteed to compile.
I'm not familiar with how OpenSolaris versions work, but since there is a date listed, this seems to be a pre-release of some sort as well.
The Linux version perplexes me further. Isn't the -0.5 part of the version a distro-specific patch-level? And if so, which distro? I couldn't find that mentioned anywhere. And why evaluate Linux based upon the work of a distribution at all?
You: "There's obviously a problem with a study that takes 8GB of data and concludes that there's no difference in quality between kernels with legendary uptimes and those that can't manage memory well enough to stay up more than a few weeks." From the summary: "The areas I examined were file organization, code structure, code style, preprocessing, and data organization." These have no direct correlation to uptime. Yes, indirect, perhaps, as in "a better-organized kernel is easier to understand, debug, and reason about", but not direct as in "implementing the scheduler in 3 files instead of one guarantees stability." That said, what defines this as an interesting but impractical study? Doesn't it say something that there's something more fundamental than just high-level software engineering principles at work in the relative qualities of kernels?
you have data and mathemtics to back that OP like the OP did
Most of the criterions used in this paper are syntactic and/or not clearly related with the actual quality of the code.
E.g., measuring the average length of the files or the average number of files in a directory reflects an underlying assumption: software quality is better when granularity is higher. But it is not true.
The reality is that you need an *appropriate* level of granularity in your software, and this also depends on your overall design. It's a little bit nonsense to try measuring such properties of the code as if they were somehow absolute and context-free !
Such criterions and the related underlying assumptions seem very debatable: thus its no wonder that no significant difference can be extracted out of these measurements.
You get what you measure.
If you measure wrong, you get wrong results.
If you measure X, don't reject things with non-X in your random samples.
That subjective conclusion is precise effect reading too much into the metrics.
Sun or Microsoft programmers need to support 2 and 2 platforms respectively. (Sun: SPARC and AMD64; M$: IA32 and AMD64). All portability are of boolean complexity.
But FreeBSD and Linux run on dozen of platforms. I do not know how it is in BSD land, but in Linux first and foremost requirement for platform support, is that it has no negative side-effects on other platforms. Consequently, for example, under Linux most (all? - all!!) locking is still implemented as macros: on uni-processor system with preemptive kernel feature disabled all in-kernel synchronization would miraculously (thanks to preprocessor) disappear from the whole code base. To make sure that on such platform, kernel would run as efficiently as possible - without any locking overhead, because all the locking is not needed anymore.
And that's single example. There are many macros for special CPU features: depending on platform it would be nop or asm statement or function call. No way around using macros.
I think one of the points the author needed to factor in, is portability of OS. Without that, most metrics are skewed too much.
P.S. Actually, Linux affinity to macros is often (at least from words of kernel developers) stems from poor optimization of inlined functions in GCC. Many macros can be converted to functions - but that would damage overall level of performance. In many places significantly.
All hope abandon ye who enter here.
Don't even try. That's twitter you're arguing with, and all he cares about is bashing Microsoft, not about the engineering principles applied within the respective kernels.
For a site about things like basic rights, Slashdot users sure do like to censor "dissent".
Ahh... yeah, I did look at his profile and notice pretty much all of his replies were scored -1.
what was the most foul comment you encountered :D ? and where did it reside
No honestly twitter, using your own sockpuppet account to complain about sockpuppets isn't going to fool anyone.
http://www.gnu.org/philosophy/open-source-misses-the-point.html
ctrl+f for "better".
In his not-so-humble opinion, Open Source is about better software, Free Software is about ethics.
A Microsoft OS is shitty by design. It represents "first and foremost the engineering requirements of its construction" ...
Therefore, it is designed to be the way it is.
Those statistics are useless; nobody has any idea how these measures translate into correctness, robustness, or performance.
Twitter - YOU said this? You really should be ashamed of yourselves.
Faster! Faster! Faster would be better!
You could have horribly written code that is reliable and bulletproof. OpenSSH comes to mind (unmodded by Debian).
Or you could have beautifully written code that is useless and full of holes. Handed-in college CS homework assignments come to mind.
Forget what you *think* you're measuring (code quality). Instead, consider whether you're measuring anything at all. That is, is there any information in the data you've measured?
Now given your sample space, it is highly unlikely that you'd measure something (anything!) and the measurement comes out more or less the same for every sample. The more likely conclusion, is that you haven't measured *anything*.
The summary means that getting your requirements right is the most important part of the project's life. How you implement those requirements is less important.
Of course, this is just my interpretation; and the meaning of "requirements" is quite fuzzy. For example, if my specs go beyond the scope of a "usual" SRS, and include programming style guidelines, there will also be an impact on how code is written.
--
SRS = Software Requirements Specifications
The saddest poem
Open Source Software promotes Competition.
Closed Source Software promotes Collusion.
If you've got so much data and still can't find differences, it's because your measurements are not informative enough. The only thing the author can conclude is that his way of measuring code quality cannot pick up differences in the part he has labeled "process".
Well, Visual Studio is nice minimalistic development environment. Using it doesn't make my cry out loud in frustration, unlike what happens with most of its commercial competitors. Microsoft can be proud of it.
But of course, for doing real work there is still no alternative to Emacs.
...where he got the Windows source code from?
... I still recall this exhaustive report comparing several kernels' performance back in 2003 in which NetBSD pretty much beat the pants off of everybody else (note the two updates with separate graphs)...
The link appears dead. Can you please point to an active page with the same material you were trying to show?Recommendation to guy who wasted his time on a study of quality, without having any idea what quality actually is:
Read "Zen and the Art of Motorcycle Maintenance
Further advice:
When your analysis of the geometry of the planet concludes that the Earth is conical, despite obvious prima facie evidence to the contrary, don't announce your discovery that the Earth isn't spherical after all to the world; take another look at your method of analysis.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
the question is not a troll, though I'm a Linux server and desktop advocate there are applications for which it's the very worst choice. Like serious CADD / CAM, on Linux or BSD? forget about it.....
But go one step further: a commercial software company using OSS can reduce the development costs by using FLOSS coders; therefore produce the SAME / BETTER quality code for LESS money.
Methinks this lesson is too often ignored....
Ultimately, AT&T plagiarized huge chunks of the BSD code into SYSV (which is why they sheepishly settled their case with the Berkeley Regents), and then what was left of SYSV and BSD were merged into Solaris.
Yes, it's now had 15-20 years of closed source development, but Solaris was built on top of a strong body of Open Source code.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
winerror.h is also shipped with the WDK. It contains all the error codes for the Win32 environment subsystem, which isn't implemented by the kernel. Win32 exists mostly in user mode: although a sizable chunk of the Win32 server process was moved from winsrv.dll into win32k.sys in NT4, which does run in kernel mode, Win32 really isn't part of the kernel. They did that to cut down on IPC overhead, not to integrate it into lower layers. Win32 services can only be used by user mode processes.
ntstatus.h is the header that contains the kernel's error codes. That file is 16272 lines in the WDK 6000 (with 8 comment lines to every code), and contains few codes not relevant to the kernel.
It's not surprising that winerror.h contains messages not related to the kernel because it's not what I'd consider part of the kernel. AFAICT, this file is included only to support win32k.sys, and possibly display drivers. I'd be interested to know how many of the kernel source files actually include winerror.h.
the layout of the bottom half of Eclipse's 'About' dialog is slightly better than the one in VS. apart from that it's a steaming pile of crap. i curse every time i have to use it, which is far too often these days.