Linus Drops BitKeeper

ravenII wrote in to mention a story running on CNet, which discusses Linus Torvald's decision to no longer use BitKeeper. From the article: "Linus Torvalds is looking for a new SCM for his project's source code after a conflict involving the current management system, BitKeeper. 'I've decided to not use BK (BitKeeper) mainly because I need to figure out the alternatives,' Torvalds said in a posting. 'Rather than continuing things as normal, I decided to bite the bullet and just see what life without BK looks like.' Coverage on the BitKeeper announcement from earlier this week is also available. Update: 04/10 16:36 GMT by Z : Updated to reflect the story's origin.

Comments

*NOTE TO MODERATORS*

[Anonymous Coward]

People who observe that stories concerning bitkeeper and linux have existed before should be moderated "redundant" so that the rest of us don't have to look at them. This is a new story on a new development, and it is difficult to intelligently discuss it in an article crapflooded by people complaining that this is a dupe when it is not.

Q & A SCM?

[Anonymous Coward]

Lets start with an easy question.

What qualities does a replacement have to have, and what are the present alternatives missing?

Re:Q & A SCM?

[arose]

Linus has to like it.

Re:Q & A SCM?

[tomhudson]

Grandparent post:

Lets start with an easy question.

What qualities does a replacement have to have, and what are the present alternatives missing?

Parent post:

As a non-programmer, but a Linux user, and generally intrested in 'the way things are done', I wish the above post had a higher rating, and a few more answers.

Me: ROFLMAO - "Let's start with an easy question ..."

As a programmer, I'm going to let you in on the BIG secret: getting the initial requirements right is NOT an easy question. It's the difference between success and failure, between on time and late, between on the money and over-budget.

Ask any coder how much time they spent convincing people that what they thought they required was

1. stupid
2. a waste of resources
3. going to cost too much
4. better put off until a later iteration.
5. not practical
6. at cross-purposes to the projects' goal
7. all the above

There's a reason we have terms like "feature creep" and "kill your first baby" (to kill your first baby means "don't get too attached to the code you wrote, because you're probably going to have to throw it all out at some point". Peopel can actually get attached to a chunk of code they wrote, that seemed elegant at the time, but just doesn't cut it, and they won't throw it out because of the psychological investment they have in it)....

Being able to accurately spec out any non-trivial project is not easy.

Re:Q & A SCM?

[cakoose]

Actually you do (if you're in the US)

The USA has a lot of people laid off due to outsourcing and the dot-com bust.

These people literally having nothing better (or at all) to do.

person_who_loses_job_after_bubble_burst != donald_knuth

Re:Other reasons...

[Senjutsu]

Bitmover dropping it's free version of Bitkeeper isn't a good enough reason?

dupe, but anyway.....

[diegocgteleline.es]

..I hate how everybody has critized Linus (specially the RMS/GNU front), when he never forced people to use BK.

I mean, when Linus started to use BK he promised things would be at least as good as they were before - and it's true (they've been better in fact), people still gets -rc's in GNU diff format at kernel.org. The official way of distributing patches has always been "clean", I don't know why people whined so much about BK, it's OK for me if Linus wants to use a propietary tool himself, as long as I'm not forced to use it. I've certainly not used or needed to use it for years, and I'm one of those people who tries -mm and -rcs all the time...

(and those who claim that people should behave differently and "give example" just because they're "leaders" can go to hell)

Re:dupe, but anyway.....

[mrchaotica]

Then RMS should apply that to himself because he needed to use non-free software when he started to develop GNU. And until linux was there, I guess he couldn't use a free kernel either.

If RMS had refused to use non-free software, he would have to program the entire system by literally twiddling in bits one by one. In contrast, if Linus had refused to use BitKeeper he'd be inconvenienced, at the worst. Linus used BitKeeper because it was the best tool for the job, but RMS used non-Free software because it was the only tool for the job. I believe that if RMS could have bootstrapped GNU any other way, he would have.

Re:BitKeeper Website

[diegocgteleline.es]

Never?

The quote continues being true. IMO the reason why Linus is dropping BK is the license and flamewars - BK is great, but maybe the free alternatives have got better in those 3 years and now they're "good enought" to use with the kernel. It'd be certainly better if bitmover would make BK free, but that's not going to happen.

Mindshare and image bloodbath for BitKeeper

[bADlOGIN]

Until Linus started using it for Linux, I'll bet most developers had never heard of BitKeeper. CVS, Perforce, VSS, and ClearCase for the past 4 years mostly seems to be what people would be using. Now all anybody knows is that these idiots dropped Linux support and burned a a great source of publicity (e.g. "our product is so good, one of the biggest OpenSource poster-child projects uses it despite it being closed and commercial!").
Now they'll be known for screwing over Linus Torvalds. I wonder how well they'll fare in future technology evaluations? I can hear the discussions now: "Gee boss, BitKeeper is nice and all but if they screwed over the guy who writes Linux , how do you think they'll treat us after they have our money?"

Re:Mindshare and image bloodbath for BitKeeper

[frank_adrian314159]

BK is tired of people trying to reverse engineer their software...

Uh, last I heard, that's neither illegal, immoral, or unexpected when you release a product to the world. I'm sorry they're unhappy about it, but it seems a bit naive to think that it won't happen and a bit stupid that they'd PO one of their most visible references over it.

Re:Mindshare and image bloodbath for BitKeeper

[tftp]

s/tired/delusionally paranoid/

Re:Mindshare and image bloodbath for BitKeeper

[civilizedINTENSITY]

tired of people trying to reverse engineer their software

This is exactly the core issue, and exactly why BitKeeper isn't worth considering.

Re:Mindshare and image bloodbath for BitKeeper

[Baki]

And you know that Linus reverse engineered UNIX? He recreated a system with the same API. What is wrong with that? There would be no progress at all if we weren't "stealing" each others ideas all the time. Neither in science, nor in art, nor in (software) engineering.

I find the current brainwashing efforts of "intellectual property" proponents to make us believe there is anything wrong with reverse engineering highly immoral, contradictory with human civilization and ignoring its history.

Re:BitKeeper Website

[tftp]

There is no need to remove the quote, it is correct. The BK software works fine. The problem is that Linus is not happy about BK licensing any more.

Re:open source

[Jeremi]

If they had used an opensource SCM from the start, they would not have to worry about changing things around now.

If a comparable open-source SCM had existed at the time, I'm sure they would have considered it. (and no, CVS didn't count as comparable :^))

Re:Actually...

[eraserewind]

So why doesn't our Open Source hero just fork the codebase, oh wait...

The question no one has asked

[Anonymous Coward]

How much has Linus use of Bitkeeper hampered the development of a free software alternative? I mean, if Linus had used a free SCM since the beginning, chances are that much more effort would have invested in that very same system to make it better. Unfortunately all the past publicity has helped only Bitkeeper instead of the free software movement.

Thank you

Re:The question no one has asked

[SA Stevens]

One of the prominent facts of Open Source programming history, going waaaay back, is that something that works good and commercial has to exist first to copy and replace. This can be said about large parts of the GNU toolchain. It's part of the 'culture' for the name of the GNU replacement to be a pun or joke on the name of the original tool.

Big conglomerate groups of random people on the Internet aren't that good at developing something totally original. It's much more common to let some commercial entity 'develop the product spec' first.

Re:Actually...

[killjoe]

If this whole absurd scenario is not an object lesson on why not to choose proprietary software nothing is.

Imagine that. A company pulling their license and then refusing to even sell licenses to you or your employees.

Re: Actually...

[po8]

Oooh---there's an attractive offer. Quit your job, but receive a free BitKeeper license. I wonder if Larry would make me the same generous offer?

Re:not surprising though

[HalfFlat]

So which do you take issue with?

1. Writing non-free software is antisocial

or

2. If a job is based upon antisocial behaviour, one should get some other job

Personally I can't see how you could argue against 2, unless you are in fact antisocial and just don't care. RMS has been making a case for point 1 for years now, and the recent news with regards to BK is more supporting evidence for his argument.

Reading your comment to the interview, I can't understand your vehemence. It's irrational. If you have a choice of doing something bad or doing something good, why insist that it's okay to do the something bad? I don't want to put words into RMS' mouth, but I get the impression that when he states something like: "It is better not to program at all than to program non-free software", it's not a statement about the utility of free versus non-free software -- as you noted in your comment, a lot of non-free software is very useful -- but instead it is a statement concerning how programming as a practice in today's society should be carried out. Programming non-free software is not bad because it makes useful software -- that would be silly. It's bad because it supports a system in which non-free software proliferates and causes huge amounts of waste. It is essentially the same point as the one you quote.

The horrible irony of it all is that most non-free software is non-free by default, not because the owners of that software are profiting significantly by having it be non-free. Most software is written for internal use for internal projects. Embedded software is usually specialised to its hardware and of limited use to competing manufacturers. In as much as it is not limited, a culture of free software allows the programmers to write original software, not re-write what others have done hundreds of times before.

If hypothetically free software were to become the only legal form of software as of tomorrow, the vast majority of programmers would still be in a job. And those jobs would be better.

Perhaps the SCM Solution is not the problem

[Anonymous Coward]

In all these discussions the question seems to be:
What tool can we find to fit Linus' methodology.

The question that nobody ever asks is:
Do Linus' methodologies actually benefit the project?

I'm going to say something that will make people angry, but I don't mean it as flame bait:
The Linux kernel is in absolute disarray, has been since 2.2.x, and is only getting worse as the project grows.

The kernel development process is a highly distributed one, with individuals all over the world implementing their own corner of the universe and sending it in diff form to the Great Linux Mutex in the Sky: Linus.

All these people, all over the world, doing things their own way. Limited cohesion, inability to pick a good design and stick with it for more than 5 minutes. Here are two examples from the stable kernels ...

Firewalls:
2.0.x: ipfwadm
2.2.x: ipchains
2.4.x: iptables

Event Handling:
2.0.x: None
2.2.x: None
2.4.x: dnotify
2.6.x: inotify

The VM system has been swapped out - several times! - in "stable" kernel branches. Nowdays Slashdotters are saying that it is the distribution's responsibility to maintain a stable kernel tree.

So, here's my point. Maybe the BSDs really have something going for them by maintaining a centralized development model where major changes are planned, documented, and subjected to stringent review across the board before being included in the kernel.

Here are the above examples for FreeBSD:
Firewalls:
2.x: ipfw & ipf
3.x: ipfw & ipf
4.x: ipfw & ipf
5.x: ipfw2 & pf

Event Handling:
2.x: none
3.x: none
4.x: kqueue
5.x: kqueue

FreeBSD consistently implements well thought out systems and incrementally improves them over the years. Time and time again FreeBSD issues solid stable releases. Even the releases that are publicly declared not ready for production are more stable and have fewer dark hairy corners than Linux.

Maybe it's time for Linus to re-evaluate his model and bring some true software __engineering__ to the Linux development process. Instead of being the Great Mutex in the Sky, he can be the great organizer and enabler of an organization of people working together to implement well thought out, well designed systems.

Just a thought from an anonymous coward.

Re:Perhaps the SCM Solution is not the problem

[DarkMan]

I'm not convinced by your arguments, although I do note that most of the sub-points are valid. Specifically, I reject your conclusion.

If Linus did Linux the *BSD way, a few things would have to change.

Firstly, and foremostly, he would have to develop, and guide, the userland as well. That is part of the reason for the apparent stability of BSD interfaces - for some kernel->userland interfaces the implemetnation is hidden (for example, things that are used only by libc). Whether this is a good or bad thing on the partof the BSD's is irrelevent, it is a fundementaly different approach.

Currently, Linux provides a set of kernel level interfaces, and the GNU libc utiliese some of these to provide the standard C library. Some of the kernel interfaces are standard, (POSIX file stuff, sockets etc), some are not (Firewall, some threading interfaces etc).

Most fundementally, the reason I reject your conclusion can be best expressed as a question: "If the linux process is so poor, why is it more popular than *BSD?"

One can discuss that question ad nauseum, but I suspect that part of the reason is that it is _not_ tightly controlled. If you have a method for making some part better, the Linux apprach says: show me the code, convince us it's better. The *BSD approach says: Show me the code, convince us it's better, get somone on the core team to sponser it, shedule it for the next (or prehaps one after next) release, and wait.

Thus, Linux has a lot more mobility, and can utilise new approaches sooner than *BSD - for good or ill [0].

My personal thoughts is: Linux is not broken. The development model works as well now as it used to, thus there is no need to change. Let the two approaches co-exists, let people use whichever they feel like. If that means that Linux becomes a testbed for ideas, and *BSD cherry pick, then so be it, that's fine. If that means that Linux evolves and *BSD stagnates, so be it, that's fine. If that means that Linux is unstable and *BSD is stable, so be it, that's fine. In practice, the last two will be indistinguishable for a long time after one of them occurs.

Let them be different - choose your favourite model, and be happy with it. But if the other one results in something you prefer, consider that both are trade-offs, and neither is perfect.

[0] This can bring problems. The various VM saga's are prehaps a canonnical example here.

Re:not surprising though

[HalfFlat]

A lot of software wouldn't get developed if someone weren't paid to do it - after all, most people, given a choice, would rather "scratch an atch" - do what they enjoy working on - instead of working on stuff they find boring. So the boring stuff ends up having to be either paid "work for hire", or "contracted out", or "proprietary".

You are casually conflating "work for hire" and "non-free"!

Almost all the programming work I do is for a private company. It is work for hire in that sense. Some of it incorporates GPL software and thus is by necessity GPL licensed itself. The other software could just as well have been licensed as free software as well. It doesn't matter. Being free or non-free is orthogonal to being payed or not for programming.

It is only an issue when it comes to selling software itself, unenhanced by other software, as a product. Because it is what makes the Microsofts of the world, and because it is very visible, it feels like this is what programming is all about. But it really carries a disproportionate impact.

On your other point: we all have a choice, it is true. But you could say it is a choice to be antisocial :) Just like other behaviour which supports institutions which are bad in the long-term, such as wasteful use of energy, water, etc., there is some degree of culpability. As to how antisocial it is to support non-free software, or for that matter, to drive SUVs in an urban environment, is a separate but important question. Personally, I am no environmental saint, but I am aware that I am choosing not to be one. This is a form of antisocial behaviour, just a very pallid and common one.

Begin with a solid theoretical foundation..

[Anonymous Coward]

I think we can learn from the fiasco of relational database products on this one (i.e., the standard, SQL, is not relational at all, and the true power of relational databases has still not been implemented in an actual product).

FIRST come up with a rational, rigorous theory of version control. What is a "change"? What does it mean to apply changes and their inverses? What is a conflict? Etc, etc. Come up with theorems and rules based on that foundation.

THEN implement it all in a product. Or better yet, many products, but base them all on the same theory. I.e., let them compete on ease of use, security, speed, etc., rather than just who has the best ad-hoc collection of features.

The author of Darcs does have a "theory of patches" but it's not rigorous (by his own admission). However it's a start.

I think having a solid theory, divorced from any particular implementation, would be a great idea, would yield a consistent set of terminology, and would allow those tough corner cases to be solved with more understanding.

Does anyone else agree?

PS: McVoy himself has talked about the mix of theoretical and practical that BitMover employed to create BitKeeper. It's not just an academic argument. You'll get a better product if you THINK then ACT rather than leaving out the first step. And yes, besides the fact that it's not open source, BitKeeper *is* the best SCM system out there.

Linus didn't scale.

[tres3]

For those of you that don't remember the lost patches and snails pace of Linux development from 2000-2002 I invite you to browse through the LKML history. We had a real problem with the fact that Linus was unable to keep up with the vast number of patches that were coming in through email and when he went on vacation there was no reason for anyone else to even try to submit patches. We all said that Linus doesn't scale! There is no way that 2.6.x would be anywhere near what it is today without some form of Source Control Management (SCM). Developers had been trying to get Linus to use some form of SCM for years but none would do what Linus wanted them to (most were not distributed enough). Larry offered Linus, and other developers, Bitkeeper for free but insisted that they not use it for non Open Source projects and that developers of competing SCM not use it at all (without paying for it). It was nice of him to give us that chance but everyone knew that the strings would, sooner or later, tie us in knots. Bitkeeper served its purpose well (relieving the Linus load) for years; all developers had to do was to send an email to Linus saying "pull the patch from my tree". This solution was a vast improvement on the previous way of doing things and the pace of kernel development increased significantly. It also enabled Linus to hand off large chunks of the kernel to other maintainers so he could focus on the parts he is good at.

The most important thing that Bitkeeper did was to show the developers what was possible with the right tools. Anyone that reads the LKML knew that this day would come sooner or later and many of the SCM developers have used the time to improve their tools. What we really need to do is thank Larry for the use of his program (it was a great help) and move on. I don't think that Linus and the other kernel developers will ever go back to the days before fine grained changelogs, distributed source trees, and the ease with which patches from any one tree can be applied to any other tree. I think, if anything, that the biggest thing that dropping Bitkeeper will do at this point is to accelerate the development of better (and more distributed) SCM's.

Thanks Larry! And more importantly: Thanks Linus, Alan, Andrew, Marcello, Rik, et al. Your work and dedication is much appreciated! ;-)

P.S. Kernel.org has a new SCM written by Linus (in his directory) that is available for your perusal.

Sarcastic post...

[rice_burners_suck]

You know, I wasn't going to say this, but I can't help it...

As we all know, Linus has more resources at his disposal than Bill Gates. Why can't he just have some of his minions design, from scratch, a distributed source configuration management package that can do everything he needs, and have it ready within six months? Then, he and his crew could suffer along with Subversion and the distribution problems it will pose for them, for six months, before Linux can be hosted on Linus' own DSCM software.

It shouldn't be quite that hard to do, with all the resources he has... When Theo had a problem with (I believe it was) the license for the SSH program included prior to OpenBSD 2.6, he thought about it for a while and then busted out his own implementation. If he could do that, then Linus with all his resources can bust out a DSCM.

Yes, this post is totally sarcastic. But seriously, who said you can't take Subversion, rip out its guts, and make it distribution-aware.

Re:Better headline might be:

[Cramer]

More accurately: BitMover loses all of it's advertising.

Seriously, have you ever head of BitKeeper outside of Linux discussions? Have you ever seen an ad for BitKeeper anywhere? Even if the quoted figures are right ($500,000/year), that's pretty cheap advertising for the amount of coverage.

(I've known about BK/BitMover for a long time simply because I know of Larry McVoy. But aside from that association, I've never heard of either.)

C'mon, You Knew This Was Going To Happen

[cmholm]

Yes, Linus had a perfectly good reason for selecting BitKeeper. And, one could reasonably anticipate that Bitmover would do what it did. Hell, if you read what Larry McVoy had to say back when Linus jumped onboard back in '02, he made it quite clear that he was going to get pissed if someone tried to reverse engineer the code. Richard Stallman doesn't have to say "I told you so", 'cause he already did, three years ago.

So, in conclusion, big frickin' deal. BK got a few years of valuable, free beta testing, Linus got some work done, and the Open source folks got a reminder as to why the Free source folks got religon.

Re:RIchard Stallman Knew This Would Happen

[Some Bitch]

Maybe he is not a mad man, but actually a very wise man.

He's both, he's an insane genius. In 20 years time we'll all be saying "We should have listened to RMS". At times he can appear petulant but if I'd been spent a couple of decades warning people about non-free software and been derided while been proved right time and again I'd probably not be a happy bunny either.

One day we'll listen to him BEFORE things start to go all runny but that will probably cause the universe to end.

Re:Actually...

[Anonymous Coward]

> according to the original story, someone who was doing paid work for OSDL (Linus' employer) was reverse engineering the bitkeeper client in his spare time. For bitkeeper, this meant that OSDL was indirectly funding the creation of a free bitkeeper competitor, which goes explicitely against their license terms.

Yes. And I heard someone studying at the university of Berkeley also tried to reverse engineer bk. Larry must remove all the free license to student, as it helps university and they train people that want to reverse engineer bk.

And, I think someone had a grant from darpa, and, in its free time, tought about reverse engineering some part of bk. Clearly, larry must stop selling bk to any government entity.

And, beleive me or not, but some not-that-brillant computer user used microsoft windows to surf to slashdot, and thought about reverse engineering bk. Larry must prevent microsoft to get an bk license, as they are indirectly helping to create a competitor.

Bk may be a fine product. Larry used the kernel for marketing advantage. But, by acting like he did, instead of "preventing" competition, he actually send *hundreds* of the most talented developers in the world in SCM systems.

Hacking is about "scratching an itch". Thanks to his paranoia, Larry made himself a big itch to scratch.

Bitmover is fucked. He could have played nice and let people reverse eng bitkeeper. Linus would have still used the "real" one, it would have evolved into a standard, and bitmover would have made a truckload of money, possibly *owning* the scm space.