Slashdot Mirror
Linux Is the Largest Software Development Project On the Planet: Greg K-H (cio.com)
sfcrazy writes: Greg Kroah-Hartmant, the Linux superstar, delivered a keynote at CoreOS Fest where he gave some impressive details on how massive is the Linux project. Kroah-Hartman said the latest release (4.5) made two months ago contains over 21 million lines of code. More impressive than the amount of code, and what truly makes Linux the world's largest software project is the fact that last year around 4,000 developers and at least 440 different companies that contributed to the kernel. Kroah-Hartman said, "It's the largest software development project ever, in the history of computing -- by the number of people using it, developing it, and now using it, and the number of companies involved. It's a huge number of people."
"Greg Kroah-Hartmant, the Linux superstar,"
Uh, what? There are only two superstars in Linux: Linus and the guy who came up with systemd.
I don't think it has a larger number of software engineers of lines of code than Google's core codebase (hundreds of millions of lines of code, tens of thousands of software engineers).
That is because unit tests are useless and so is most QA/QC. Let me guess, you think Agile development improves quality too. By the time you have defined all your "unit tests" we have already built the entire system and have it running in the real world.
My impression is Linux isn't inherently bigger and more complex than things like Windows and AIX; it has many fewer users; and yet it draws on a much greater pool of developer time. It's inefficient, costly, and wasteful. The project just isn't well-run; it's mostly arguments, bits of code in and out, and no planning or goals.
Support my political activism on Patreon.
"by the number of people using it, developing it, and now using it" ...I was using it before BUT now, after developing it, I'm *really using it now... :)
At first, I think this must make it more robust to long term changes. It raises a follow up question, though. How do those developers/companies group up by contribution? I'm sure most are working on server/enterprise applications, but any changes there might be equally interesting.
For comparison, I found articles citing 1000-2000 developers for Windows 7. I had no luck finding estimates for windows server.
PS: I don't reply to ACs.
The article never says it is the greatest or best. Just the largest.
Modern app appers app apps on Appdows 10, not LUDDITE software on LUDDITE Linux!
Apps!
Talking about BS, your source is a article about Linux as a Desktop computer... Linux core business is servers, not desktop. When you look at servers distro (like RHEL), many of the issues they are referring are irrelevant.
In the case of Linux, you don't have to argue whether unit tests or QA/QC are useless. What matters is simple: what developers want to spend their time contributing. They have historically not done unit tests and the QA/QC is probably as varied as the developer. Linux is a community effort by a self selected community. It's not a corporate driven, profit-seeking entity with a singular management structure.
PS: I don't reply to ACs.
The VMS operating system was estimated to contain over 25 million lines of code, and that was measured over 10 years ago - I'm sure it's quite a bit more by now.
There is no God, and Dirac is his prophet.
That is because unit tests are useless and so is most QA/QC.
In your first sentence you demonstrate your total and complete ignorance of the subject matter at hand.
Let me guess, you think Agile development improves quality too.
No, it improves the productivity of shitty prima donna developers who think they can spend 8 years writing 4 lines of code to get it just right. It allows middle managers to reel in asshole developers who think their shit doesn't stink while actually producing nothing of use to the business.
Its a really shitty development pattern for a good developer, but 99.99999% of developers are shitty and Agile is far better than free roam for them. Your average run of the mill developer in most software shops is actually less useful and efficient than most janitors.
For every Linus there are 7 billion people who aren't.
By the time you have defined all your "unit tests" we have already built the entire system and have it running in the real world.
Of course mine will work and do what its supposed to do, yours will be buggy, not meet the design requirements, not function as expected in random ways.
You are the definition of ignorant on this matter. You really should just keep your opinion to yourself because the only people who are going to agree with you are going to be the same people in the unemployment line with you.
Software development is an engineering practice when done right, which you clearly have no fucking clue how real engineers work. Fortunately people like you will never be allowed to do anything that actually matters like build bridges or buildings, and your ignorance and arrogance will prevent you from even understanding why you'll never be able to do it.
So congratulations there snarky ... you've shown us you're nothing more than a completely ignorant, smug asshole who has no fucking clue when to shut his stupid mouth :)
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
My impression is
WRONG
it has many fewer users
a billion android devices will disagree with you
what's the alternative? prayer and incense?
That is because unit tests are useless and so is most QA/QC. Let me guess, you think Agile development improves quality too. By the time you have defined all your "unit tests" we have already built the entire system and have it running in the real world.
Only if you think nine 5's is acceptable reliability.
Some of us have higher standards than that.
Because when you build systems of systems, it doesn't take many constituent systems built with the attitude that "unit tests are useless and so is most QA/QC" to turn the entire assemblage into a steaming pile of unreliable shit.
But hey, if you like producing steaming piles of unreliable shit, go right ahead.
Just get of your damn high horse - because it's stinking up everything with horse shit.
which has very minimal, I can even say non-existing, QA/QC and no unit tests at all.
So it's your claim that this doesn't exist?
http://autotest.github.io/
And this also doesn't exist?
http://linux-test-project.github.io/
That bit about shitty programmers...that's gold. But only a corporation would have to resort to such methods to rein in developers. In Linux land, if you fu is weak then your code goes nowhere (hopefully, at least).
PS: I don't reply to ACs.
I don't see anyone making that assumption. Can you quote the specific claim that you're reacting to?
Full system tests are good. Incense helps.
Oh look: another butthurt QA person who is jealous because he doesn't make as much money as a prima donna developer! I've been writing software for decades that you are currently using every day. Guess how many units tests Linus has written? Zero.
Comment removed based on user account deletion
The VMS operating system was estimated to contain over 25 million lines of code, and that was measured over 10 years ago - I'm sure it's quite a bit more by now.
This is just the kernel. But most of it is arguably "not" kernel code... it's drivers. This is directly addressed in TFA:
All versions of VMS and OpenVMS together come nowhere near to running on as many different hardware platforms as Linux, so it would be shocking if Linux's drivers weren't massive in comparison.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Is Linux a steaming pile of unreliable shit? No it isn't. Is Windows? Which one do you think has a large QA/QC department assigned to it?
Which it also isn't. Windows NT (and successors) is bigger by almost every measure, including man-hours, scope and user base.
And then I'm not even talking about the worst flaw in the story yet: the implicit assumption that it's good, or even meaningful, to be the largest project. To me, it smacks too much of IBM's kloc worship in the 80s.
Shitty programmers are shitty programmers. No amount of unit tests and Agile development is going to fix that.
It's not a corporate driven, profit-seeking entity with a singular management structure
Not 100% right now, but Red Hat is working on it.
Linux is more widely used than any NT or NT variant. Give me a break.
I'd say having a university programming project become one of the biggest operating systems in history, and with the vast number of contributors and collaborators, all for a project that you can freely download, yeah, that's a pretty impressive badge of honor.
If Linus died tomorrow, he'd be viewed as one of the pantheon of great computer innovators, not so much because he produced anything in and of itself unique, but rather because he transformed the *nix ecosystem, and lead to greater penetration than I suspect Unix's original creators could ever have imagined.
The world's burning. Moped Jesus spotted on I50. Details at 11.
Oh look: another butthurt QA person who is jealous because he doesn't make as much money as a prima donna developer! I've been writing software for decades that you are currently using every day. Guess how many units tests Linus has written? Zero.
Way to cogently rebut GP's argument!
Are you going to play on the monkey bars or swings during recess?
Android anyone? Not to mention all the embedded variants.
My thinking is that Linux has at least as many users as Windows, if not more.
The world's burning. Moped Jesus spotted on I50. Details at 11.
Monkey bars! Those are for REAL men!
It's The Matrix - the fake reality we all live in. I heard The Architect made a fortune in stock options.
Is Linux a steaming pile of unreliable shit? No it isn't. Is Windows? Which one do you think has a large QA/QC department assigned to it?
Why, yes it is!
The core parts of Windows don't behave like that. You can drive Solaris into hard swapping that lasts for hours and it will come back just fine. But Linux? Write to a memmap()'d file under even moderate memory pressure and the core OS shits the bed performance-wise.
I need to reboot my Linux-based Android phone more often than I had to reboot old Win95 systems.
It's that exact kind of unreliability in corner conditions that make large-scale Linux systems less reliable then Solaris or even Windows systems - as long as you can get past the crappy third-party drives and fight through all the GUI-based cruft you'll find in Windows in order to just use the core of the OS kernel.
You need higher standards.
This is not entirely true when the software in question is being directly converted into hardware. This happens for VHDL, Verilog, and SystemVerilog. People call these hardware design languages, but the reality is that they are pure software. Other tools (Synopsis Design Compiler for example) turn this high-level software into low level netlists which in turn are used to produce silicon. A mistake in the high level software can cost millions of dollars to fix and spin a new piece of silicon.
The definition of 'unit', however, is not necessarily always agreed upon. What is the 'smallest possible' unit that can be tested? I would argue that it is the smallest possible unit that makes sense to test. For example, a PCI-Express root-port has at least one very well defined interface (the actual PCIe bus), but often also has very well defined internal-interfaces for how the root-port 'unit' plugs into the rest of the chip. Since all interfaces are very well defined, I can write unit level tests to fully exercise and prove that the root-port works correctly, and there is very little need to write full-chip tests which would end up taking days to run vs the hour or two it takes to run against just the unit.
The full-system tests are much harder to develop as they require writing assembler to get the CPUs to send traffic down to PCIe. But instead I can directly send write/read commands which is much simpler test case to develop. IT also gives me finer grain control of my test and the timings, as I don't have nonsense like cache-hits, misses, etc getting in the way and perturbing my stimulus timings.
The logic portion of developing hardware is almost entirely a software problem these days, with the exception that this software has to meet some real world constraints of physical setup and hold timings. But software for real-time embedded systems have similar timing requirements that must be met, so it's not a completely foreign concept to software development.
Unit testing is critical to certain sectors of software development, and large corporations spend hundreds of millions of dollars a year doing unit testing because in the long run it saves a ton of money and time.
You are right! Some esoteric JVM problem is total proof that Linux sucks. Thanks for the tip!
Unit testing of software is a great way to prove that a given software routine performs the wrong function with perfection.
Exactly.
I nearly fell out of my chair when I read your LKML post claiming that the 2.2.x/2.4.x kernels were more stable than they are now. I'm guessing you either weren't involved with Linux back then or have a short memory.
Here is a short history lesson:
The development ended up dragging out to the point where the base kernel features needed for new devices not existing.. so you could use the stable kernel which didn't support much, or you could try the development kernel which tended to be wildly unstable or you could use the disto kernels (RedHat etc) which attempted to patch drivers from the unstable branch into the stable branch resulting in something almost as unstable as the development branch.
I still have nightmares of the day someone presented me with a brand new IBM server in which the Old kernel couldn't see the disks, the dev kernel crashed on boot, and the distro kernel would crash several minutes after starting. (I ended up going with a hand built custom kernel + patches).
It doesn't matter how many times you whine on slashdot or troll the kernel list. No one who remembers the old stable/unstable branches wants to go back to that nightmare.
Also FYI: There are several projects that QA the Linux kernel testing branches and report the results back to the kernel devs.
You have a way over optimistic view of Agile as generally implemented.
I contribute to a project that has CI going to run unit tests on every commit. Those unit tests take hours to complete. I have not seen those unit tests catch a single bug (other than style formatting issues, which are non-functional). The human testers have. The problem being that people equate hours of test as the sole metric of quality. The quantity of test does not by itself assure quality of test. The other facet of this is the phrase '100% coverage', which I've seen *many* project managers jump up and down all excited. I have seen QA teams *fired* at that milestone, because it's a misleading phrase. When management sees '100% automated test coverage' they think 'Human QA is no longer needed'. They don't understand those unit tests are just as likely to have problems, and almost certainly will not be all-encompassing (in fact unit tests tend to hit the corner cases that the developer *recognized* would be tricky, and had those edge cases in mind as they wrote the code anyway)..
My problem is less that unit tests are a waste of time or that they can't be good, it's that too many advocates of the process oversell the benefit and lead to poor decisions with respect to quality based on overconfidence in the process. The conundrum is that in practice, either you oversell it and cause these bad consequences, or you try to describe it accurately and project management will express frustration on 'wasted effort' in writing unit tests since they can't really replace human testing.
Incidentally, the sentiment that the project burdened by unit tests will be beaten to market is a likely consequence. It's not necessarily a *good* thing, but if 'good enough' hits the market before 'truly good', most will ignore 'truly good'.
XML is like violence. If it doesn't solve the problem, use more.
Nice troll reply. Unwilling to except that you might be mistaken in your assumptions, and when presented with a rational argument for unit testing you choose to instead ignore the information.
Would you please be so generous as to point out what part of Linux you help code so I can be sure to avoid that code in the future?
A slightly higher percentage of Americans have desktop and laptop computers than have smart phones; and half of Americans have iOS. Across the world, almost 90% of desktops and laptops are Windows, and more people own Windows PCs than Linux-Android tablets and phones; while 58% of devices are Android. In terms of sheer software installation volume, there are many more PCs than people, thanks to office PCs and one-PC-per-family-member.
Java has the claim of running on almost every god damn thing and having more users than anything ever made besides tits. Even with Android, the Windows installation and user bases are pretty far in the lead.
In terms of new devices and software installations per year, it's definitely Android; those are short-lived devices, though, with many people replacing their smart phones after two years (two weeks if you're an iOS user). PCs did that in the late 90s.
There's a disturbing amount of non-smart-phone embedded Windows; it's eclipsed by every other OS, both in-house and QNX. Linux has its embedded home in small-form-factor PCs, such as home routers and set-top boxes; most of everything that's not a PC-by-another-name runs a customized OS, either modified QNX or some proprietary ARM microkernel written specifically for the device. Too bad they're not running Linux on medical devices and ATMs; they're all Windows.
Support my political activism on Patreon.
Without comment on the "merits" of all these arguments, I can make the interesting observation that Linux haters seem just as rabid as Microsoft haters.
All of which, ultimately, is unnecessary. If you hate Linux and love Microsoft, run Windows. If you love Linux and hate Microsoft, run Linux. There's no need to "prove" that one or the other is "better." Run whatever works for you. Heck, run Apple stuff if you want.
The point is: what you do as an individual doesn't affect me much, if at all. And vice-versa.
Now, if nearly everyone suddenly abandoned Linux, there would be a problem for those who remained. But that's not going to happen any more than Windows people starting to adopt Linux in droves.
But I can run Linux without feeling the need to destroy Microsoft. In fact, "hating" is a waste of time and energy better put into doing some productive work.
Taking the numbers at face value you get the following stats:
- with 4000 developers
- 2.7 lines of code added per day per developer
- 1.3 lines of code removed per day per developer
- 0.47 lines of code changed per day per developer
The problem is that a lot of efforts bearing the banner of 'QA/QC' are not really that good, and they embolden bad decisions. At the core of this is a commonly held belief that 100% coverage with unit tests == quality job done. If you just have unit tests, the system and user experience will inherently be taken care of. This phenomenon has led a lot of justified folks to get pissed at the concept of unit tests altogether, even though the practice isn't really bad in and of itself. Some people find it a waste of time, but I think the majority of the vitriol comes from bad quality outcomes from this phenomenon.
XML is like violence. If it doesn't solve the problem, use more.
You are right. I think we can all agree that we all hate OSX.
Core is mobile and embedded systems....
Android, *krhaem*
In electrical engineering circles we call these two aspects of testing 'verification' and 'validation'. Verification is verifying that the code matches the specification. Validation is making sure that the specification is really what we want in the first place.
Fortunately, us real engineers demand complete specifications, and when we find mistakes in those specifications we make sure they get fixed. Doing things in your haphazard land of writing code by the seat of your pants must be stressful at times.
It must be nice working for the government.
...because they still don't support Linux with their official logins, tax, banking etc. services.
In fact - BankID which is the most used login method just came out and said they'll drop Linux (which they did) support as the userbase was too small.
I also use Telia ID card login - with a card reader, even this is "hackishly" supported (officially unsupported, but I've gotten it to sorta kinda work), so people with Linux can't officially even register at the unemployment offices unless they have windows or a smartphone.
But hey, hopefully we're heading in the right direction - I've been a registered Linux user since 1998, and now exclusively use Linux for my Desktop. Every time I go to work and use Windows - I'm constantly reminded of an inferior system with endless updates, endless disk-trashing and endless limitation as a user.
What this world is coming to - is for you and me to decide.
I love it!
I have never, and will never, work for the government. Real engineers use specifications and test against them.
The core parts of Windows don't behave like that
As someone who uses a Windows desktop a lot, Windows is no saint when it comes to performance going to shit. My Linux system has not gone off to unbearable performance garbage under my particular workload.
Different platforms are currently weak to different things. For you, Linux had a key weakness for your workload. For me, Windows has some weaknesses. The difference being my ability to actually characterize what the OS is doing under Windows is much reduced.
XML is like violence. If it doesn't solve the problem, use more.
I'm just joking. I'm sure you are a better engineer than I am.
The problem being is that many managers *think* that some magical process can allow shitty programmers to do great work, and then lay off the great programmers to get cheap-o developers because Agile means quality, no matter how bad your workforce is.
Of course this isn't how most of the well-liked software is done, but it's done *all the damn time* in enterprise software land. Leading to a paradox of free and cheap 'consumer' software generally being a *ton* better than equivalent enterprise software packages which generally cost a whole lot more (when equivalents exist).
XML is like violence. If it doesn't solve the problem, use more.
I thought the Linux wins were: serving the web,
Yep
cloud services,
Yep
android devices,
Yep
chromeOS [...],
Yep
medical devices,
What?
aerospace,
What?
routers,
What?
cars,
What?
ATMs,
WHAT?
Good idea.
Your comment has too few characters per line (currently 5.7). Your comment has too few characters per line (currently 7.1). Your comment has too few characters per line (currently 8.7).
CLI paste? paste.pr0.tips!
> Not so greatest then, considering that people get tired of adjusting their code to new APIs which inevitably
> leads to even more regressions.
Developer tiredness isn't a metric.
> You know, Greg, you don't sound convincing. You sound like a marketer of some dietary supplement.
I think he's too busy developing to worry about slashdot whiners.
I'm hardly a linux fanboy, but I use it, and, like other genuine fanboys and other developers I couldn't care less when enough other computer users use linux such that a pointless post can appear on a pointless blog declaring that `finally it's the year of linux on the desktop`.
...Said the troll who has six or seven different versions of MSVC++ Runtime installed alongside one another. And apparently twelve different versions of Not-OpenGL over the last 20 years is perfectly okay.
Editor, A1-AAA AmeriCaptions
All that emotion and fallacy. I doubt your engineering skills on that basis alone.
And yet it's eaten everyone's lunch with it's informal but successful QA. Just because it doesn't have a manager reading up on the latest fad and implementing it everywhere, badly, doesn't mean it isn't getting the job done.
Software development is an engineering practice when done right, which you clearly have no fucking clue how real engineers work. Fortunately people like you will never be allowed to do anything that actually matters like build bridges or buildings
I just had to laugh at this one. You praise Agile but it is everything but the way you'd build a bridge or building. You absolutely don't create a building one room at a time, you have architects and construction engineers and blueprints all ready to go before you start implementing. If you did it the Agile way you'd rewire the house ten times over before you're done. And you don't in the middle of construction find out you want to add another floor and an upstairs bathroom, not unless you want a huge replanning. Agile is more like Star Trek, full power to shields/weapons/engines/life support and things magically rewire themselves to serve the most pressing business need.
To be honest, I feel like waterfall overplan and overthinks things, Agile underplans and underthinks things. I like waterfall projects that act "agilish", small initial scope and iterative releases, clear deliverables, priorities and rapid prototyping. Or Agile projects that act "waterfallish", someone with a clear long term vision of what the system eventually will look like and how the components we build will scale and fit the big picture. Bad waterfall is just mental masturbation over plans that'll never work in reality. Bad Agile is just timeboxed cowboy coding, making up shit as you go along. The problem is some project managers think you can make uncertainty go away by sitting around a table and discussing it further. Some things you just won't know until you've tried and seen what kind of progress you actually make.
Live today, because you never know what tomorrow brings
How can you run a project without knowing the bug burn rate and having daily standup meetings???? The horror! Linux will never take off!
Bigger projects means more competing requests. Which means more sacrifices to meet all the requests.
Small apps that do what you want is far better than a big one.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
So, your advice is to put the best guys in straitjackets so you can keep the idiots you hired from poking themselves in the eye with a fork when they go to lunch?
Linus chose to let the best guys do their best work and if anyopne comes back from lunch with a fork in their eye, send them to the doctor and invite them not to return.
I notice you didn't include things like servers, routers, automobiles, industrial machinery, televisions (esp. the "smart" TVs), all the IoT shit (e.g. thermostats), avionics systems, and etc.
The embedded stuff outnumbers smartphones and 'PCs' by at least a factor or two... and nearly 50-60% of those run some embedded variant of Linux... embedded Windows ekes out maybe a sliver of that market.
Quo usque tandem abutere, Nimbus, patientia nostra?
I read your link. The part I find surprising is that the author finds it surprising that modifying mmap-ed memory might involve file I/O. Why wouldn't it? So the answer is to NOT safeguard from data loss by letting it hang around uncommitted forever? I can make it WAY faster, I'll just make all calls that write to a file be a NOP, that'll be really fast as long as you don't care about your data.
The solution is to use the APIs correctly. Either put the file in tmpfs or use SYSV shared memory.
TL;DR: When you french fry when you should pizza, you're gonna have a bad time.
I can write unit level tests to fully exercise and prove that the root-port works correctly
Anybody who took you seriously after that absolutely should not be allowed near a compiler.
You're making one big mistake, and that's equating software development with construction. It's not. Software has achieved automation of construction of final artifacts based upon detailed specs: it's called compilation. What programmers do is much closer to creating specs than to construction according to specs. People writing recipes are not cooking.
By a lot.
I think you confuse "the art of programming" with "the craft of programming." If it were the former, you'd be allowed to put the noses on sideways, like Picasso, and call it "art". But as a craft, you need to produce the best work possible, even though it certainly contains an element of art.
I work on safety-critical software. We are required to do rigorous unit testing on every line of code by The Authorities. And I regularly find bugs in my code in the process. In fact, since I will be developing unit tests anyway, I (try to) make it a practice to do develop a module and its test simultaneously, which is more efficient than doing the unit test at the end, as is the common instinct by those who regard unit tests as "useless."
I also think you confuse "useless" with "efficient." Depending on the use of the software, unit tests may be an inefficient use of development time - as the Linux folks evidently believe. So, I don't do unit tests for every category of software. For example, there's no need to do that for a simple utility script. But I wouldn't want to fly in an airplane whose autopilot code was never unit-tested, and whose developers instead simply assumed that all the code they write is bug-free.
Systems don't always exercise every code path of the underlying units. Many times issues in the units are obscured by the complexity of the system tests. Isolating a unit and having tests can catch a lot of bugs early, and the earlier you catch bugs, the easier it is to fix them.
Of course mine will work and do what its supposed to do, yours will be buggy, not meet the design requirements, not function as expected in random ways.
Exactly true. I've found over many years of engineering that there is a vast difference between something that nominally works and something that works perfectly under all conditions. Once you've achieved the former, you've done perhaps 1/3 or less of the work required to get to the latter.
I need to reboot my Linux-based Android phone more often than I had to reboot old Win95 systems.
Counterpoint: my first Windows-based programs won't even start. Therefore, Windows is worse.
Zippy is the largest pinhead on the planet. Over 18 billion polka dots.
And apparently twelve different versions of Not-OpenGL over the last 20 years is perfectly okay.
It's either eleven versions if you count major releases, or 16 if you also count point releases...
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Like many other things in life, unit test also follow the 80/20 rule. You get 80% of your benefit with only 20% of the required time investment. You don't need anywhere near full coverage to have useful unit testing.
Unit test are only useless if you make useless unit tests. Behold the power of cargo cult programmers. You need to test your project somehow, how else do you know it's working at all? It compiles? The difference is how you test it, not if you test it. You probably test it by compiling an exe with a bunch of parameters or variables. Instead of that, just create a unit test and pass in the state, it's not much different.
Your code probably doesn't play well with units tests because instead of passing in state, you just have global state.
In my experience, developers - even prima donna ones - make far less than pretty much everyone else in the industry short of level 1 helpdesk.
Millions of lines of code does not necessarily make a good software project. Which is better...a project that does X in 2 million lines of code...or a different project that also does X but only needs 500K lines of code to do it? In most cases the smaller code base is better; but you can sometimes make a function do something in just a few lines, but it takes you all day to figure out what those few lines mean. A bigger function with more lines might be much better because it is easier to follow, has more error checking, and is easier to maintain and update. I have also seen functions that were 2000+ lines long with GOTOs all over the place. It was a mess.
A joke and a rant are different things.
You're making one big mistake, and that's equating software development with construction. It's not. Software has achieved automation of construction of final artifacts based upon detailed specs: it's called compilation. What programmers do is much closer to creating specs than to construction according to specs. People writing recipes are not cooking.
No. In the waterfall model, first you write the spec then you implement the spec. The developer breaks down the spec to code, the same way the compiler breaks down code to machine instructions. Implementing the spec is not supposed to be a creative process, in fact figuring it out everything you're going to do first so you know how to best implement it and don't go down dead ends and waste time rewriting, refactoring or switching tools/libraries/architecture is pretty much the cornerstone of waterfall methodology. Then you write it once and test that everything was done according to the spec, at least that's the theory.
In fact cooking is a pretty good analogy, say the customer wants to have a pizza. The requirements are that it be tasty, crunchy and spicy. So you make a recipe for a pizza. Then you create a big and expensive industrial robot to make a pizza exactly like that. And you test it extensively, but not in terms of the requirement but that you added exactly the right ingredients, in the right shape and amount placed in perfect order and arrangement, that it's baked for exactly the right duration at the right temperature. And after ensuring everything is exactly to spec, you hand it to the customer and he goes "Yuck! That's not what I wanted"
Live today, because you never know what tomorrow brings
I thought Wikipedia and its sister Wikis, which together make up the Wikimedia project, are the largest software project on the planet, based on any number of metrics from number of contributors, code base (yes, Virginia, HTML, wiki markup and natural languages such as English, German, Russian, etc. are and should be considered as code), cultural impact, and geographical dispersal. For better or for worse, politicians and plagiarists alike use Wikipedia.
Well they should. How can you have a known correct working system if you haven't tested every possible system path?
Um, right. So the average QA is paid higher than the average developers? I didn't know that. Thanks for the tip!
Wow. It took you many years to figure that out? I'm pretty sure everyone in the industry knows that the last 5% of the project takes 90% of the time.
The security of Android has nothing to do with the security of Linux. Android is built on Linux. It is those addon pieces and the Android security model that is the issue. A lot of the insecurity of Android is by design. Google doesn't care about your security.
Just because you don't unit-test doesn't mean you assume the code is bug-free. There are other ways to test that aren't based on unit-tests, as you non-Linux folks evidently believe.
That pretty much sums up modern corporate software development: keep the idiots you hired from blowing up the place.
You said "Linux" but what you're actually talking about is "software environments that sit on top of Linux." Ubuntu, KDE, glibc, and Gimp aren't "Linux." It seems in your response comment further down that you selectively focus what you were talking about to exclude whatever categories of things you were blatantly wrong about. How does "Android devices are 'short-lived' devices" change the fact that your statement as presented is provably incorrect based upon the massive amount of Android devices in use? It's okay to be wrong. Man the fuck up.
So is IBM, HP, Dell, ...
Still won't be corporate driven. And definitely not by a "singular management structure".
Actually, yes I have had to reboot Solaris when it goes into hard swapping that lasts for hours...
People wanted to get work done and the job causing the swapping couldn't be aborted due to the thrashing...
And bad performance happens on both Solaris and Linux when applications go berserk doing stupid things. With Windows, it is hard to keep it from thrashing and and crashing.
In the safety-critical world I work in, we have to do both unit tests and what we call "system tests," aka black-box tests.
As an analogy, you could test test that a mechanical clock "works" because it tells time accurately, but if you really need to be certain that it will be reliable under all conditions, you're also obligated to individually test all of its gears and other various parts.
I'm glad the people who write code for things like airplanes and medical devices do that, even if the Linux folks don't. That my be why Linux can only be used in airplanes in non-safety-critical applications like the wi-fi router that lets you watch movies as you fly the friendly skies. If it were to be used for something at the highest level of safety-criticality like an autopilot, The Authorities, worldwide would require you to produce evidence that you had requirements-based module tests that covered every bit of the object code that the compiler generates.
Maybe I'm not as smart as you if it took me longer years to figure it out. Even so, based on our figures, it sounds like it takes you about three times as long to get something working well after the point that it nominally works. That might explained why you noticed this effect almost immediately. ;-)
... but it's not a project.
Who's the sponsor? Who's the project manager? What are the goals and when are they scheduled to be completed? When is it scheduled to end?
If you can't answer those questions clearly and unambiguously, then whatever you're looking at, it's not a "project".
Unfortunately that's because people who think like 110010001000 are busy building the medical devices. Devices that will need patching in the middle of your future surgery. Devices that will be causing the system monitoring your heart rate and blood pressure to suddenly quit.
It's the 21st century. The question is not should something have software in it. The question is when? And how cheaply can I get it on the Internet of (Insecure) Things?
You know its just about the basic things. For example I've heard that there are driver regressions with features from certain devices stopping to work. How hard is it to wire up some computer to a test rig that pulls each day from mainline, builds it, and then tests the resulting kernel on various devices, testing that all features of the devices still work. Then it would make it easy to find the bug as you only have to sift through a day's commits.
No, unit testing is no silver bullet, but a type system isn't either, or the hope that "nothing will go wrong".
In fact such a system wouldn't be new, the coreboot project already has it as well.
One of the funny moments in the SCO vs IBM (over linux), where Darl McBride deliberately drove a company into a wall and gave most of the lucrative legal work to his brother, was a bit about a guy getting off a light plane in Germany with the entire printout of linux source code in his briefcase. They just had no clue at all about the size of the project and made such a silly gaffe, which when it comes down to it, is some very pointless petty perjury.
and nearly 50-60% of those run some embedded variant of Linux
Have some numbers to back up that?
Talking out of my ass, I'd bet that more than 90% of embedded devices produced since '00 have no OS at all, and if you count only the segment of non phone stuff with an OS, more than 50% of that wouldn't run linux but some variant of RTOS.
I've been writing software for decades
I'm the new RPGLE programmer in an AS400 shop and I'm surrounded by developers like you. Long story short, the 20+ year old code base is crap, none of the neck breads know what the hell is going on in the system and then blame it on bad data. Everything in the system from a user's perspective is confusing. And after a few years of using the system these folks wrote, our clients scream for SAP or RedPrairie. And these folks keep telling themselves, "They don't know what they're missing! We can roll out changes in days not months! They'll be sorry!" All the while I sit there and see our company bleeding money out because people like you think you can do no wrong and are artist and the code you write is somehow magical.
I'm not saying unit test and agile development fix bad programmers, but what I am saying is the free roam model of programming leads to a lot of crap and loss of money. And the self righteous nature means they'll never understand why our clients go elsewhere and never return. It just is so frustrating to work with people like you because you all believe that you can do no wrong. People want software that works. Sometimes that means unit testing, sometimes it doesn't, but if just go around yelling, code is art and the users exist only to ruin it! You're just blowing your company's money out the window. Just bloody hell, and you wonder why they keep outsourcing? Well just look in the mirror one day, there you go. Please never work within a 500 mile radius of Atlanta.
I'll take my troll mod now. For this person it was worth it. And trust me bud, I know that everything I've said has fallen on deaf ears.
Many computer users run a modified version of the GNU system every day, without realizing it. Through a peculiar turn of events, the version of GNU which is widely used today is often called “Linux”, and many of its users are not aware that it is basically the GNU system, developed by the GNU Project.
There really is a Linux, and these people are using it, but it is just a part of the system they use. Linux is the kernel: the program in the system that allocates the machine's resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the GNU operating system: the whole system is basically GNU with Linux added, or GNU/Linux. All the so-called “Linux” distributions are really distributions of GNU/Linux.
Many users do not understand the difference between the kernel, which is Linux, and the whole system, which they also call “Linux”. The ambiguous use of the name doesn't help people understand. These users often think that Linus Torvalds developed the whole operating system in 1991, with a bit of help.
Programmers generally know that Linux is a kernel. But since they have generally heard the whole system called “Linux” as well, they often envisage a history that would justify naming the whole system after the kernel. For example, many believe that once Linus Torvalds finished writing Linux, the kernel, its users looked around for other free software to go with it, and found that (for no particular reason) most everything necessary to make a Unix-like system was already available.
What they found was no accident—it was the not-quite-complete GNU system. The available free software added up to a complete system because the GNU Project had been working since 1984 to make one. In the The GNU Manifesto we set forth the goal of developing a free Unix-like system, called GNU. The Initial Announcement of the GNU Project also outlines some of the original plans for the GNU system. By the time Linux was started, GNU was almost finished.
Most free software projects have the goal of developing a particular program for a particular job. For example, Linus Torvalds set out to write a Unix-like kernel (Linux); Donald Knuth set out to write a text formatter (TeX); Bob Scheifler set out to develop a window system (the X Window System). It's natural to measure the contribution of this kind of project by specific programs that came from the project.
If we tried to measure the GNU Project's contribution in this way, what would we conclude? One CD-ROM vendor found that in their “Linux distribution”, GNU software was the largest single contingent, around 28% of the total source code, and this included some of the essential major components without which there could be no system. Linux itself was about 3%. (The proportions in 2008 are similar: in the “main” repository of gNewSense, Linux is 1.5% and GNU packages are 15%.) So if you were going to pick a name for the system based on who wrote the programs in the system, the most appropriate single choice would be “GNU”.
But that is not the deepest way to consider the question. The GNU Project was not, is not, a project to develop specific software packages. It was not a project to develop a C compiler, although we did that. It was not a project to develop a text editor, although we developed one. The GNU Project set out to develop a complete free Unix-like system: GNU.
Many people have made major contributions to the free software in the system, and they all deserve credit for their software. But the reason it is an integrated system—and not just a collection of useful programs—is because the GNU Project set out to make it one. We made a list of the programs needed to make a complete free system, and we systematically found, wrote, or found people to write everything on the
So what?
I am a shitting developer, you insensitive clod.
This being true and working for myself, the only way I get anything out the door is to 'go agile' and I still don't like it. As 'silicon valley' put it "this just became a job."
It should be unlikely that the unit tests run during a CI build pick up anything.
The unit tests run on a CI system are normally checks you haven't broken anything. They are also a framework to show that the module you have designed meets the specifications/ user stories which were used to determine its functionality.
Any programmer worth his salt has tested their module against the unit tests before adding it to the repository.
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
The bug burn rate is known. Grep the commit logs for bugfixes & timestamps... No standup meetings though, no formal reports at all, no planning in advance who works on what. The only administrative effort is in rejecting/accepting patches. Proves that all the planning/admin/management hoops & methodologies that sw developers are forced to endure - are unnecessary for writing software. It may or may not be necessary for running a business, but certainly not for making sw.
Because almost every driver is part of "linux-kernel", it is big and bloated - I bet there is still ancient 3com ISA-card code by this nasa guy, Becker..
Windows at least has DDK and working driver model.
This sort of testing is done, but in a very limited fashion. You can test on "various devices" but certainly nowhere near all. Linux runs on over 15 different processor architectures. Within each family, quite a few differences among those processors. Then, most processors come with a selection of boards on which they fit, with varying amounts of memory and i/o capabilities. Then, some of these platforms (such as the ibm-compatible PC) has hundreds of thousands of optional but supported peripherals. And then, some platforms are incredibly expensive - one-off hardware or massive machines where some bugs only show if you install a thousand processors.
And not all bugs shows themselves immediately. Some appear only after hours or days of running. Some needs special circumstances, such as a bad reaction to a malformed network packet or someone yanking a usb device at the wrong moment. And some bugs needs to be seen by a human, such as a video acceleration feature that unfortunately swaps the red & blue channels.
As someone who don't use windows at all, I have this observation:
Windows users, even enthusiast power user, don't know what goes on in their machine. They 'reinstall' to get rid of 'cruft' and make the machine 'faster'. I never understood that mindset. I have replaced computers that was used till the hardware wore out, running a single install all the way. And then copied that linux installation over to the new machine - so I kept the setup and did not need to (re)install. Not even when replacing the hw. If I brought a machine to its knees, I bough more RAM or uninstalled the problem software.
Do they still 'reboot on a schedule (daily or weekly)' in order to keep windows 'snappy'? Never did that either. I reboot to start an upgraded kernel every other month or so. home machines also get turned off to save power occationally.
It is fun seeing students reinstall linux 'to get rid of some problem'. They are used to make things work that way, but then they hit exactly the same problem in the new install. Reinstalling is not a solution - rather, it is a problem if you actually need to do it. When there is some problem, just fix the problem. Much faster than a reinstall, and unlike a reinstall, it helps.
that just makes me wonder, whats gonna happen if linus dies :x
That would be "unwilling to ACCEPT".
With your level of attention to detail, I bet you are a really good coder.
Did he also check how many people work on M$ windblows?
I notice you didn't include things like servers, routers
You mean "Linux has its embedded home in small-form-factor PCs, such as home routers and set-top boxes" doesn't include routers?
If we want to bring servers into the mix, can we talk about how there are fewer servers than *people* involved with any given business, and how we're reducing the number of actual server devices by doing combined services in "The Cloud"? I.e. you need 1/10 a server, I need 1/10 a server, they all need 1/10 a server, let's all rent 1 server from some other guy. Meanwhile, at the companies renting the server, they *already* had more people--and one or more work PCs per person, almost always running Windows (and typically running MacOSX otherwise)--than servers, thus for every 1 Linux they had many Windows. That's not even counting the businesses running non-Linux servers, most of whom are running a mostly-Windows server base if not a zero-Linux server base (colleges and anyone in medical do this a lot).
Linux is big, but it's not bigger than everything else.
Support my political activism on Patreon.
The CI jobs on this thing take hours. I'm not going to run the checks on my system before letting the system take a crack at them. It's not like my push is even a commit to master, it's staged in a code review for humans to have a chance to look it over while the CI tests are running. Why in the world would I delay getting human feedback in code review to make sure I'd pass unit tests first, when the only potential downside would be it catching a bug in public?
XML is like violence. If it doesn't solve the problem, use more.
Linux is great and Linus deserves a ton of praise. But there's nothing good about being biggest. I suspect SystemD has played a big role in the Linux project becoming so big. If the kernel without the bootloader, et al is 20+ million LOC - YIKES!!
There are some very convenient things for desktop users now.... That's an important part of growing the user base....
All this post does for me is stress me out.
Every rule has more than one consequence.
Then those are low-signal tests and you should stop running them...
I'm not disputing that there is something wrong with that scenario. I'm just frustrated that the project *thinks* those tests alleviate need for human testing. This is not an anomaly, in most places I see unit tests prominently on display I see the pattern of making too much out of the unit tests.
XML is like violence. If it doesn't solve the problem, use more.
With that JVM issue, it would be solved, REALLY EASILY by having /tmp being mounted as tmpfs instead of a real filesystem.
This is why you need a real sysadmin....
Some systems by definition do not exercise all paths the first time 'round. I have many of libraries that have many features that aren't all in use "yet". Heck, most libraries are not part of a system. If by definition your code is not part of a system, how do you do a system test? And if you're not the one writing the system tests, how do you guarantee your own code paths are being exercised properly?
I am responsible for my own code, not some other person who so happens to be using it.
LOL. Android has WAY more users than Windows.
Unit tests are an 80/20 rule. They can find 80% of your bugs and very quickly. Unit tests allow you to get near realtime feed back if your change will break something. Integration tests can take a very long time and means you are less likely to use the tests until after you've made a bunch of changes. Quick feedback is extremely useful.