Slashdot Mirror
Linus Defends Proprietary File Formats [Updated]
Simon (S2) writes "Torvalds launched a blast against OpenOffice.org, and defended Microsoft's right to keep its binary Office formats proprietary. 'I'm happy with somebody writing a free replacement for Microsoft Office. But I'm not fine with them writing a free replacement just by reverse engineering the proprietary formats,' said the Linux founder. 'Microsoft has its own reasons for keeping them proprietary, and I can't argue with that.'
At the heart of Torvalds' decision to refrain from using Bitmover's BitKeeper source code management tool last week, a day after BitKeeper decided to drop its limited functionality free client, is a dispute between BitKeeper developer Larry McVoy and Samba developer Andrew 'Tridge' Tridgell. It has subsequently emerged that Tridgell was working on a clean room reverse engineered implementation of McVoy's proprietary software, and Torvalds has come down on the side of his friend McVoy." Update: 04/13 17:24 GMT by T : As reader Daniel Callahan points out, this is a goof. "The Register article made up the Torvalds quote. The article offers the quote
and then continues: 'Actually he didn't - we just made that quote up. But what Torvalds really
did say this weekend is only slightly less bizarre.'"
MS shouldn't be forced to open any application source code, but _should_ be forced to have open file formats. They can 'innovate' all they want, but their customers shouldn't be locked into their software. IMO, of course.
Anyone who RTFA will find that the quote in question is false. Its spelled out in the page that it is a false quote.
Indeed, editors need to keep tabs, but asshat submitters need to shape up as well.
Slashdot: News for Trolls, Stuff that's Bullshit.
What's wrong with reverse engineering? In the past it's been considered legal if it is done in a 'cleanroom' type environment, meaning that none of the participants had or have any connection with the company that originated the format (in this case Microsoft). Of course laws like the DMCA cast some legal doubt on some reverse engineering... But ethically it seems just fine.
the false headline will hit google news and spread further, whereas the correction in the comments will go unnoticed.
This story should be yanked now.
Congratulations, submitter! It's not every day you can successfully troll on the front page. Ten points to Slytherin.
This is really unforgivable: to quote the 'Linus quote' from the Register verbatim, and then to not quote the bit immediately after:
Actually he didn't - we just made that quote up.
It doesn't matter how well the quote summarizes Linus' position. The Register makes it very clear that the quote is not really Linus' by denying it right afterward. Slashdot should too.
This is worst kind of out-of-context quoting I've seen in here quite a while, in a story at least. Both the submitter and CowboyNeal should apologise.
(even after I sent an email to the "on duty editor" after seeing this in the "mysterious future")
yeah, I did the same thing. Appearently they just don't care.
I'm a virgo and on Slashdot. Coincidence? Yes.
People will argue about whether the quotation is accurate, but there's no doubt that Linus right now has more conservative views on intellectual property and the development of ideas than many in the software community, even proprietary software developers. You might call this hypocritical, considering how early releases of Linux were so closely modelled along the lines of Minix, including components like the cloning of the Minix filesystem with absolutely no modification or improvement on its design.
I don't really care. He's a kernel engineer and as long as his kernel continues to kick ass, I'll use his software. In the same way, I don't use GNU's silly excuse for a kernel, but think a lot of their politics is insightful and their userspace software unrivalled.
I mean, shit, I'm very tempted to stop reading this site.
Slashdot, the Weekly World News of tech journalism.
The Doormat
If you're not outraged, then you're not paying attention.
The slashdot summary is definitely incomplete, and represents a falsehood by omission. On the naming of this article as a dup, I think this is a worthwhile followup because TFA effectively reframes the issue, and clarifies the clean-room aspects of Tridge's implementation.
:)
In this reframing: Linus has clearly come down against reverse-engineering. TFA is further correct in pointing out that this is inconsistent with what Linux, OpenOffice, gcc, and a bunch of other open source projects are all about.
So, Linus is inconsistent and chose to side with his friend over his principles in this case. I can understand that even if I don't agree with it. Even Linus is entitled to make mistakes now and then
Regards,
Ross
Indeed, editors need to keep tabs, but asshat submitters need to shape up as well.
Then submit unasshatted stuff yourself. You have the opportunity to fix something that annoys you, so do so.
Reverse engineering of file formats and protocols is a right, and it's an important one to ensure a competitive and free market. The real question is whether we shouldn't just force formats to be open. Legislatively, that's a dead end, but big (eg government) can just make open formats a requirement.
Don't be an idiot. Relying on the masses to send you stories is one thing, but relying on them to do all the editorial legwork (fact checking for instance) is just naive. Even if every submitter is making a genuinely good effort to provide nothing but good stories (and believe me, that's not that case) you're still going to get a lot of crap. The fault here lies with the editor.
Sigs are awesome huh?
I wish the article had spent a bit more time justifying the analogy, as you have done. The way it's written, it seems to conflate three things:
Perhaps people with more experience in writing software can correct me, but it seems like these are three distinct, inequivolent things. From what I know Linux is an example of #1; Samba, Gaim, and Open Office are examples of #1 and #2. I guess what McVoy is claiming is that Tridge is doing #3, while Bruce seems to be claiming it's actually #2. It would seem Linus can only consistently object to #3. Can one draw a clear, unambiguous division?
"You call it a new way of thinking; I call it regression to ignorance!" -- Operation Ivy
Because Linus didn't snoop out undocumented proprietary formats in order to figure out how to make Linux compatible with Unix operating systems. He simply built a free operating system that conformed to the published and open Posix standards. If he had examined Solaris binaries to figure out how to make Linux a binary compatible Solaris clone, that would have been reverse engineering. Implementing a published standard is not reverse engineering.
Don't get me wrong, I disagree with Linus' opinion here. I don't think there's anything wrong with reverse engineering, as long as you don't steal trade secrets or perform some other such corporate espionage tactic to facilitate the reverse engineering. That doesn't make Linus a hypocrite, though. It just means that he's not in line with the predominate opinion in the open source world (which anyone who follows Linus' opinions already knew anyway).
"If English was good enough for Jesus, it's good enough for everyone else."
Except that very few of those projects are entirely based on reverse engineering. Linux was a reimplementation, Linus didnt reverse engineer anything, the vast majority of the specs were in the open for him to adhere to. OpenOffice is an attempt to provide Linux and other OSes a quality office package, that otherwise does not exist, the inclusion of .doc capabilities isnt the main reason for it and dont compete with MS Word on Linux. GCC reimplements the C and C++ specs, no reverse engineering there. Tridgell reverse engineered something that already had a capable and popular client on Linux, the other projects didnt have a comparable alternative, and as someone said in the last story, Tridgells main reason was to circumvent the license for Bitkeeper.
Im hardly shocked that Linus came out with a stance that pretty much noone expected him to take, and I have great respect for him doing it. He doesnt really care much for the FOSS philosophy, and that is entirely his right to do so, although I am shocked by the number of people who expected Linus to have a similiar outlook as RMS or Alan Cox.
No they're not. They're attempting to figure out how the binary behaves under all applicaple conditions, and then produce their own code that mimics that behavior. What you're describing is decompiling.
Again - no it's not. Copyright has nothing to do with actual functionality. You're confusing copyright with patents.
If you have a problem with the morality of this process, you may want to take a hard look at the IT industry. Reverse engineering has played a key role in the advancement of technology. Numerous times.
Hypotheically speaking, as we know you don't host kid porn.
/. devolving into the National Enquirer for the tech-set?
Is
How the Register gets away with what they did is amazing. They make up an entirely fake quote, attribute it to Linus and then say, almost parenthetically, "we just made that up, he didn't really say it".
Think about it. How would like it if somebody did to you what they just did to Linus?
Shame on you Register!
"I'd rather be a lightning rod than a seismometer." -Ken Kesey
Sad that in this case it comes from an actual quote from The Register, a reputable news source. They made it easy to take the quote out of context, and that's bad writing. I'd expect to see this from J. Random Blogger and repeated on Slashdot, and I'm disappointed to see it in The Register.
Whatever. They did it for effect, it's a question of style.
I don't think it's fair to require that writers do all kinds of things to avoid their writing being "easy to take out of context". Good writing usually isn't easy to take out of context, sure, but I think journalists ought to be allowed to pull the sort of things that the Register pulled here.
At some point you have to just force someone to accept responsibility for what's being resyndicated and RTFA in its entirety.
I haven't yet RTFA, but since the issue of file formats is near and dear to my heart (and what I do professionally), I figured I should say something.
I'm working on a Digital Archiving project for a government agency. And what we have determined thusfar is that proprietary file formats are -very bad- for long term preservation.
Now, you may ask, who cares about long term preservation? To which I would respond, clearly you are not a fan of history- or at least, good history. Innocuous documents end up being primary sources! People find new uses for and interest in old documents!
Still you seeem doubtful. Fine. But, should Microsoft disappear (unlikely as it may seem) or otherwise leave us with a bunch of proprietarily-formatted files that we cant read save through- shudder- emulation of something like Windows XP, a lot of people will be unhappy. And a lot of data may not be fully recoverable.
You may say that if such things really bother people, then they should only purchase software using open standards. I sort of agree. But we are dealing with a field in which -certain- companies are convicted monopolists, so....
Proprietary formats are still the bane of my existence.
It's not what you know, or even who you know- It's how many people recognize your damn
I was product manager for 2½ years at a software company, whose product was partly open-source based (it was our own OS webserver). I was in the business when Eric Raymond tried to convince s/w companies to "go open source." "It's much better, bugs get fixed, security holes shut much faster." And so on. But the truth is that open source is about free (gratis) software, and software companies are about selling software. There are one or two exceptions, those who can sell support and so on, but the whole _concept_ of having a software company is to charge people for the software you develop. This doesn't mean that I'm against open source, possibly I'm more against software companies.
The bottom line is that open source may one day cover all possible software need for every person, but it will come out of academia, non-profit organizations, and hobbyists. Software companies will not be the primary drive behind open source. I think Stallman has known this for a long time. And if you _do_ have, or plan to start, a software company, there is nothing wrong with keeping some parts of your code proprietary. Alternatively, just don't start a software company.
Journalists have a rule that anything between quotation marks has to be an exact quote. You're not even allowed to correct the grammar or make irrelevant changes to help it fit into your sentence better. That is, if your subject says, "I like tapioca", you're not allowed to write, "Bob says that he 'likes tapioca.'" There are a few things you can do, like using editor's brackets and asking them to verify a rephrased quote, but in general if it appears in quotes (and not in brackets) it had better be exactly what they said.
So I slap the wrist of The Register for screwing up, and further for putting the retraction AFTER the advertisement (though I don't know if that was deliberate or not.) Both Slashdot and the poster also screwed up, but The Reg is the one who really blew it, IMHO.
Would I love it if Slashdot took responsibility? Sure. But I'm not going to expect it, so I live with it. I haven't got any "force" to apply except voting with my feet, and I like Slashdot too much (warts and all) to do that.
Is it just me or has there been way too much 'factually incorrect' information in front-page Slashdot articles lately? A very simple peer-review system for facts in Slashdot articles before they go on the main page would do wonders. Additional "+5 Informative" comments could potentially be appended to the article, such as the parent, and more factual and well-balanced news for the general reader would appear on the main page without the need to read all the "+5 Insightful" opinions and "+5 Funny" jokes to just get the facts. It's a humble opinion. What do you guys think?
(This was a response to another terrible article, but reusing it saves time and energy. Dupes are a way of life on Slashdot.)
This place is going south, fast. I have paid my last subscription since I can no linger justify even 5 USD on this site. This just confirms my theories that the editors won't even bother to read to paragraph 3 in an article. And won't even do it when people spell it out for them via e-mail. Slashdot has become a shadow of its former self.
This has nothing to do with what Open Source is all about.
Correct, but there are many open source projects that rely on reverse-engineering to duplicate the features of another system, which is why I said, "...a bunch of other open source projects..." and didn't claim the value for open source as a whole.
GCC wasn't created by examining the bytecode output of an Intel compiler.
True, but several of the optimizations that used to be found only in commercial compilers were figured out through a reverse-engineering process.
[Linus] is not in favor of reverse-engineering someone elses implementation against their wishes.
1) When would anyone ever be in favor of someone else reverse-engineering their work?
2) Linus is inconsistent with his principles.
3) Linus is inconsistent with current law and the current ethics surrounding reverse engineering.
4) Linus is going after the wrong guy. He should be acknowledging that his decision to go with BitKeeper was always at odds with much of the Linux development community and was bound to eventually blow up in his face. Which it has.
As it turns out, all of these things are okay. Linus seems to have some very good skills that, along with the work of other kernel developers, benefit millions of people every day. This doesn't mean that he should be infallible or that anyone should take his advice when he speaks outside his area of expertise. As for the ill-fated decision to go with BitKeeper, there was value, but there is now cost.
Regards,
Ross
The proprietary file format in question is that of BitKeeper; Tridge reverse-engineered it so that people can have access to their own data when BitMover pulls the plug on the free-as-in-beer BitKeeper (which hadn't happened yet at the time he did it, but which was inevitable as Larry kept changing the license and threatening people with losing their rights to use the software). Linus sided with Larry, despite the fact that Linux, GNU, Samba, and everything else we run has had to rely on reverse engineering of proprietary formats, devices, and protocols since forever just to function.
Reverse engineering has played a key role in the advancement of technology.
Indeed, the x86 clones that are the most popular deployment platform for linux wouldn't exist at all if Compaq hadn't reverse-engineered the IBM PC BIOS.
BitKeeper
Samba
That was great!
Now, who has devoted more time, energy and resources to community development of software?
BitMovers
The Samba Team
You know, I think you really have this thing down by now. Last one:
Who would you rather be stuck in an elevator with?
Larry McVoy
Andy Tridgell
Wow! 100%
I'm sure glad that Andy did raise his hand in class and ask to go to the potty in Professor Bill Gates' class. And I have to wonder how many Samba installations are cooking on the machines of BitKeeper employees.
"Flyin' in just a sweet place,
Never been known to fail..."
Except that very few of those projects are entirely based on reverse engineering.
Do they have to be based entirely on reverse-engineering to qualify as being reverse-engineered? How about if they qualify as benefitting from reverse engineering? I don't differentiate between those two levels of reverse engineering. I also think that reverse engineering is good for competition and the markets in which you and I make economic decisions.
Tridgell reverse engineered something that already had a capable and popular client on Linux,
You and Linus appear to have a problem with that, but for the life of me, I can't see what it might be. Reverse engineering a duplicate of a working existing product is legal, ethical, and highly beneficial to free markets (whether open or closed source).
as someone said in the last story, Tridgells main reason was to circumvent the license for Bitkeeper.
I don't mean to sound condescending, but why else would he put the time and effort into such a project? He wanted an open-source alternative to a closed-source tool that he didn't want to have to use. So he reverse-engineered an implementation of the client to achieve that goal.
Seems pretty straightforward to me. Also seems pretty ethical and completely legal.
Regards,
Ross
This is perhaps the first time I've strongly disagreed with Linus, but I think he's completely wrong here. How do you think we got Samba? All of Samba was reverse engineered, and Linux has gained a huge amount of functionality from that.
There's nothing dishonest about looking at how someone else did something and using their ideas. If Larry Mcvoy has a problem with that, he can take the low road and apply for software patents.
AccountKiller
I don't get how some of these comments are being tagged as "Insightful" or "Informative" if they're just the 5487235th time someone pointed out that CowboyNeal was misleading in his post. A big part of the reason I come to /. is to read intelligent and sometimes funny additions to the articles that are posted, not to read 50 flames that somehow scored high. Let's talk about the morals of open source development and Linus, not CowboyNeal's mistake.
I personnaly don't have much respect for the reverse engineer
If you do not respect reverse engineers, then you do not respect the people who bring you the documentation necessary to add support for new hardware in operating systems published by entities other than Microsoft Corporation and Apple Computer Inc. Why do you want GNU/Linux and *BSD to have poor hardware support?
And Tridgell didn't reverse engineer something that already had a capable Free Software client. You know, the PC actually has a "capable and popular" operating system. So, presumably, it's absolutely wrong for the Linux developers to continue to use reverse engineering to develop a Free Software alternative. Right?
Wrong. This is about freedom. The Bitkeeper people had no business discouraging those who want to use free tools yet who want to interoperate with those who lack the same standards from writing free tools to spec. It was bad enough that the protocols were undocumented and proprietary to begin with. It's worse that this kind of vengeful stance was taken against third parties for daring to have an association with someone trying to create those free tools.
I'm absolutely amazed. He may not be the loudest proponent of FOSS in the world, but he's at least made himself look like such a proponent, and he has relied upon the very people he attacks doing exactly what he's attacking them for doing to make his kernel usable and what it is today.Your respect for him may have risen, mine has dropped. I was prepared to handle the fact he adopted BK in the first place because, well, people do often see themselves as pragmatic when making decisions that essentially defy good practices. He should have learnt something from this lesson, but essentially it looks like he's merely digging himself into a deeper hole while yelling "You all suck!" at those trying to get him out.
You are not alone. This is not normal. None of this is normal.
I haven't heard Tridge's exact words on the subject, but what little I have heard suggests that he didn't particularly care about Bitkeeper's great features and he wasn't trying to write a replacement for the entire Bitkeeper system. He was just writing a replacement for the non-OSS Bitkeeper client - and purely so he and/or others could interact with other developers on Linux kernel development without being several technological steps behind.
The fact that the non-OSS Bitkeeper client had a morally repugnant license was probably a major reason that Tridge thought this a worthwhile project.
Everything I've heard so far indicates that Tridge was doing nothing more than the OpenOffice.org and /or Abiword developers do in reverse-engineering a closed file format, thus enabling people to access their own goddamn data if they cannot (for whatever reason) use the "usual" software. For example, if you're running Linux on PPC hardware, you can't run Microsoft Office and so you can't access information locked up in Microsoft Word documents from that platform - or at least you can't without the open-source applications that have (mostly) reverse-engineered that file format.
And if you're a Linux kernel hacker and you want to develop on an equal footing with other kernel developers, but you happen to occasionally hack on source-control software too - well, you're not allowed to use the free Bitkeeper client. A roughly equivalent opensource client is your only option.
Well, aside from convincing Linus to stop using Bitkeeper :). And as that's now happened (or is in the process of happening), I think you'll find that Tridge has absolutely no interest in continuing development of his OSS bitkeeper-ish client.
I think you don't really understand why people do reverse engineering, nor what a slow and painful process it can be. People don't reverse-engineer stuff to get a cheap thrill. Writing software using your own file formats and/or your own networking protocols and/or accessing your own bits of hardware is much more fun and interesting (and productive) than reverse-engineering someone else's.
But often it can be of enormous benefit to the wider community to be able to open and modify that Microsoft Word document, or use that interesting piece of hardware, or view those Sorenson Quicktime videos, with free and open-source software. And the reverse engineers are the wonderful people who make it possible for us to do this.
He also has a proven track record of sound common sense.
This does _not_ however imbue him with infalibility.
We have two issues, and a side point, here:
(1) is reverse engineering wrong, HELL NO, it is the basis of most human scientific progress, in fact, you do the research, publish the paper and wait for collaborators to reverse engineer aka confirm your results.
(2) are Corporations unconditionally entitled to develop, or incompatibly extend, data formats or protocols and then claim them as patents, trade-secrets, or Intellectual Property, or semble to claim Copyright protection for them HELL NO.
The side issue is, was Andrew Trigel morally entitled to take the view he did.
So, if you try to extend an existing format or protocol, if you document it it is a _derived_work_ and your publication is infringing, unless it is fair use, so the M$ Kerberos extension fails.
To have a trade secret you must keep the secret.
Reverse Engineering is legal almost everywhere.
To protect against Reverse Engineering you need a patent.
If you are a monopoly, so M$ is, and Bitmover is not, different rules apply. Sherman & Mann, acts; see existing settlement(s) and the compliance process in the US and EU.
So, if the EU requires M$ to disclose its Office Formats, for example, then that will mean that they are in the public domain and can be used anywhere, whether Linus likes it or not.
All the above, simply restate the law.
Now, as a matter of opinion, I believe Andrew was: (a) fully within his rights, and (b) the resulting furore was a consequence of Linus lack of legal and commercial accumen in accepting Larry's licence with its in-built poison pill
He should have demanded that the 'free-licence' was irrevokable and that the BK source was in escrow before confering the benefits on Bitmover.
If you work in a large company, and made that sort of mistake, you would be be big trouble.
Honestly, this is why The Register is a dangerous source for news. NO half-decent news source would ever -- and I mean *ever* -- make up a quote from someone and then go on to say they made it up...
-----
"Cogito Eggo Sum: I think, therefore, waffle."
Why are Fox "beating the pants off" the other networks? I've got my own hypothesis: because the average-to-dumb outnumber the smart by a wide margin. This is no secret: look at IQ statistics for the general population. Got a lot of money and Want a successful TV network? Write dumb, loud, shiny content which appeals to the cross-section of the population covering "average" down to "foolish mush brain", and you are guaranteed the widest audience. And you get to perpetuate that audience by filling their heads with your mush! Bonus!
It's the same reason that PBS and NPR historically must struggle to survive, while the Dr. Phils of the world turn into megamillionaires. There just aren't enough smart people to go around, and the dumb ones multiply faster than the smart ones to boot.
So congratulations, stupids, you're extinguishing your best hope for long-term survival. Who wants their kids to get physics degrees, when everybody else's kids are becoming latte-slurping pinheads
with MBAs?
At least you can take solace in the probability that when the end comes, you'll probably all be - statistically speaking - too stupid to realize it.
"Oh brave new world, that has such people in it!"