Why Open Source Doesn't Interoperate

bergie writes "There is an interesting article on Advogato on why it is so difficult for Open Source projects to interoperate or support common standards. Often cultural differences between projects, egoes, and many other issues stand in the way. The article outlines some practical ways for improving the situation, based on experiences from OSCOM efforts to get support WebDAV, SlideML and other standards into Open Source CMSs. Examples of successful interop projects include freedesktop.org, the cooperative effort between GNOME and KDE."

Comments

Because free software is not planned

[Anonymous Coward]

To make software interoperate, developers need to create interfaces before writing their software. Often no planning is done and developers start writing their code without a clear vision of what they want to write.

There is a certain overhead in creating interfaces. They can take time to develop and they aren't any good, you'll be stuck with them for years. Even Linus is against the creation of standard interfaces internal to the Linux kernel. That decision inhibits the creation of a truly modular system.

Re:Because free software is not planned

[akadruid]

Unfortunatly a lot of code writing is done with inadequate planning, but this is an inherant problem. Without a crystal ball you cannot predict every twist and turn you will face.
This is not just a problem with free, Free or open source software, but also with planned, structured development of commerical model software such as Windows.
After all if we could predict the future why would ever need new versions? :)
However I don't think that a neatly integrated environment is impossible, just difficult.
Besides, getting around integration issues is part of the fun!

Re:Because free software is not planned

[You're All Wrong]

Your subject line says it all. Well, nearly...

I'd say the two biggest sins of the open-sourcers are
a) over-generalisation (it'll be able to do everything)
and
b) over-specialisation (it does one task, but can't do similar ones)

I'm finding it hard to think of examples, but I guess GNU grep's an OK example of something that's just about right.
Expanded to do enough things like context greps (e.g. give me 4 lines before the line containing "Name:" and 1 line after), and a few other features (e.g. '-c' so that you don't need to '|wc-l') that add to its functionality, so it isn't over-specialised. Likewise, it's not sed, awk or perl, they realised that just keeping it simple and lightweight was the way to maximise its usefulness.

YAW.

Re:Because free software is not planned

[akadruid]

So you're telling me that all those managers have actually been useful over the years?

some sort of functional and or design specification
That just about covers some of the best stuff I've been given to work with :)

Re:Because free software is not planned

[truthsearch]

From an interview with Rob Short, the vice-president of Windows Core Technology, regarding Microsoft® Windows(TM) Server 2003:

Why is there no command line only version?

We're looking longer term to see what can be done, looking at the layers and what's available at each layer and how do we make it much closer to the thing the Linux guys have -- having only the pieces you want running. That's something Linux has that's ahead of us, but we're looking at it. We will have a command line-only version, but whether it'll have all the features in is another matter... It's a very tangled subsystem.

<plug style="shameless">
As I also explain at MS Versus, Bill Gates has testified in federal court that Microsoft® can't modularize their operating system or document all of its APIs because it's written by groups of developers haphazardly binding software together without any clear overall design.
</plug>

Re:Because free software is not planned

[jc42]

Well, I dunno about that. I've worked on any number of "planned" projects, and in every one, the same thing has happened: The managment comes along and changes the requirements. So you have to kludge the plan to handle the new requirements. Then they do it again, and again ...

I don't think I've ever seen a planned, commercial project that interoperated with anything else. Usually you even have to write new editors because all the file formats are proprietary. Just looking at the data in a file often requires a new program, because it's a new, proprietary format, and not even your vendor's software can read it.

OTOH, I have some "C/unix" programs that I wrote 20 years ago. I've reused them on dozens of different kinds of computers, and usually about the only problems are that the #include lines need to be rewritten. Sometimes I have to change a few variables to a different size int. But there is rarely any problem with my programs interoperating with other programs. This is shown by the fact that I always have a lot of scripts that combine my programs with other programs.

The project I'm working on now consists essentially of "raiding" the data at a big corporation, and extracting as much info as I can from their old, poorly-documented files that are in formats that can't be read on any new computer. My code converts them to plain text, mostly flat files, a few HTML and XML files. They can be read on any computer that we copy them to. They can be dropped into a web directory and read from any browser on any computer.

I'm doing all my work on linux boxes. I keep pointing out to them that my converted files no only work ("interoperate") with most any linux software, but will work on nearly any computer out there, and will continue to work for years. They believe me, because when they copy the files to their other random computers, their programmers don't seem to have problems with the file formats.

Of course, one of the things they really want is to take the data and stuff it into a commercial database. The result will be data that can only be used by a timy amount of software from that database supplier. Well, that and the perl DB package (which we will note is Open Source). But note that they are doing what corporate management always wants to do: Put the data into a format that can't be easly read by other programs.

In summary, I'd say that the Open Source environment, and more generally the non-proprietary parts of commercial unix systems, are by a large margin the best place to find interoperability. Closed, commercial systems can generally be summarized as "Your data won't work anywhere else, and won't even work on the same vendor's systems 5 years from now."

At least that's the history that I've seen so far.

One factor is obviously...

[Anonymous Coward]

...that open source authors prefer solutions they like over "standard" solutions.

Industry standards, particularly those created by committees, are often abominations that people only use if they have to. In my experience, the extent to which people like things like CORBA and XML often seems to be inversely proportional to their level of technical sophistication.

RFCs have much more respect in open source circles than committee-created standards.

Blindered developers

[wowbagger]

One of the biggest problems I see with Free Software development is the problem of the blindered developer.

This is the guy who doesn't bother to raise his head from the computer to look at how his project works in any environment other than *his* system. You know, the guy who requires you to have libfoo.so.5.1.2.pl6-thursday-0741am-fred-mutant1 installed just to compile his code, and by $deity no other version of the library will work.

A concrete example: The developer of the GATOS project (a driver for the TV tuner/video capture (but not video out) functions of ATI All-in-Wonder cards) requires you to use HIS kernel module and HIS radeon driver. As a result, you may EITHER use his code XOR the DRI accelerated 3d code, but not both.

True, he does (to an extent) track the DRI development, but rather than working with DRI and XFree and coming up with a way his drivers can play nice with the standard builds (e.g. having hooks in the standard driver and having the standard driver load his modules if present) he is off on his own little branch.

He also uses libraries and packages that are not part of the standard installs of common distros - as a result just getting his code working is a real slog. So many people don't do it, and his project does not get as much support as it might.

Now, I am not picking on him - developing stuff like that is hard, since it is very poorly documented. And with DRI making changes, XFree making changes, and him making changes, you WILL have times when things don't play well together. But rather than that being a transient state of affairs it is the normal state the GATOS project w.r.t. DRI.

Unfortunately, it take time and work to stop, get a fresh install of RedHat/SUSE/Gentoo/... and see what it takes to get your code to build and install. It takes work to make sure that you really NEED the latest version of libfoo, rather than just any version. Especially when your code interoperates tightly with other people's projects it is difficult to plan interfaces that won't change frequently. If you can accept help from others this isn't so bad, but many project "leaders" have the attitude of "HOW DARE YOU IMPUNE MY PROJECT! IT IS PERFECT UNTO ITSELF! I CANNOT HELP IT IF YOU ARE NOT 31337 ENOUGH TO HAVE THE LATEST STUFF! L@M3Rz! IT IS UNDER DEVELOPMENT!"

But that is the difference between a hack and a software engineer - just "getting something to work" and "getting something to work well, under as many circumstances as possible, as smoothly as possible."

Advogato's number

[infinite9]

No, it's because there are 6.02 x 10^23 developers working on the system at any one time.