Slashdot Mirror
Moving from Binary Drivers to Open Source?
GerryGilmore asks: "We are a division, specializing in telecommunications equipment, of a very large hardware manufacturer. Our equipment, DSP and PSTN boards, has been supported under Linux through a set of binary driver modules and binary libraries implementing our API set. We are in the process of migrating to a completely open source-based software infrastructure to be more in sync with the rest of the Linux industry. However, not having any real experience with moving from proprietary to an open source model, we wanted to see if the Slashdot crowd has any similar experiences to share: The Good; The Bad; The Ugly; and how best to avoid the most common pitfalls."
need... link... to.. *twitch*.. click.
HELP!
I have never made a migration on a scale like this, so I have no horror stories.. but I would like to commend you and all peoples making the move to open source where possible.
** "It's not my job to stand between the people talking to me, and the ones listening to me." -- Pego the Jerk
So the really important thing is to really look through your code first - you need to:
1) Check it is all really your code - you didn't buy any in 5 years ago? If you have a source code control system then actually being able to trace your code is great.
2) Read the comments - ok, so lots of closed source contains rather dodgy comments that you might not want to be public.
3) Check that releasing it wouldn't be revealing any information you got under NDA from any of your suppliers/partners.
Read the codingstyle document and look at what others are putting into the kernel!
The biggest mistake is some idiot using unusual function names, spreading his driver over atleast ten files and using 2 character indents or no indentation at all.
Especially if your source is ported from windows (or the programmer has only windows experience) make sure you do this right.
Jeroen
Secure messaging: http://quickmsg.vreeken.net/
This may be really useful for sales, but it may also lead to a serious amount of bug finding!
Are you really sure you want your device drivers debugged?
Sent from my ASR33 using ASCII
You could find more information at LKML. The archives are here, http://marc.theaimsgroup.com/?l=linux-kernel.
Many vendors have been moving from proprietary to open source. You can join LKML at http://www.kernel.org/.
The worse thing i've seen in a (Windows) driver.
// stage 1 // stuff here // stage 2 // stuff here // stage 3 // lameness filter mary had a little lab, blah, bla, 1 3 5 7 11 13 17 19 23 29 31
for (i=0;i10;i++) {
switch(i) {
case 1:
case 2:
case 3:
}
}
how long until
Make sure that you release all documentation and tools (preferably with source) for the hardware and the drivers. The last thing a "free" developer wants to do is re-invent all of the wheels that your company created.
Provide Good Documentation
If you provide well organized and complete documentation to a quality product, developers will most likely flock to it.
Support the Developers
You will want to have staff on hand to answer questions about the technical details of the product. Create a forum that is monitored by the engineers who designed and create code for the product. Make sure that questions are answered thoroughly and quickly.
be sure you dont have any SCO code in there ;)
Sort through the code. Take out everything you do not want shown.. comments and the like.
Sweep through the code to make it/make sure it is readable. This will attract more developers to your project.
Open a project. Still release the binary drivers, just let people get into the code and start making the fixes for the bugs they find. Once it's sufficiently linux-ized by members of the linux community, switch them to your main drivers instead of your binary drivers.
Your binary drivers work right now (I assume), so leave them as your defaults, until the open source community can go in, change, break, fix and test out your open source drivers with you.
And thanks for your future contributions to the comunity. Please post a follow up when they do go open source. This will generate more interest in your products, and I, and many other admins who are part the decision making process of picking hardware for their companies, will definately give your products another look if they have open source, stable drivers (Like 3Ware....they rock. Because of thier long time commitment to linux, they are the only hardware raid cards I buy for my linux servers).
Although I understand that NDAs can be involved, it often amazes me that hardware manufacturers keep to closed driver implementations.
While it's true that windows and linux are the biggest games in town, offering potential customers who run other OSes a way to use your hardware seems like a no-brainer: larger potential customer base -> more customers -> more profits.
It often seems like pulling teeth; take a look at the recent (and ongoing) attempt OpenBSD is making to get more documentation and relaxed licenses for hardware. Being able to point to $1 million of hardware already running under an OS and getting little or no response from a vendor for better support -> larger customer base -> greater profits? WTF is wrong with the PHBs?
And now, back to the topic:
Document, document, document. Although I don't have any directly relevant experience, I've occasionally taken over 5000+ lines of code with abysmal documentation; on one occasion, it became so painful I rewrote major portions because it ended up taking less time than having to figure out what was going on.
Include reasonable amount of documentation, like a README and an INSTALL file. Keep both short.
Try to use autoconf scripts, since that may help in the long run when people tries to build it on all kinds of strange platforms. Be clear of which platforms that are supported, and which are not. Be also clear of platforms known not to work.
Set up a bug report tool. Bugzilla is a well-known tool. Bugs will be reported, and you may also get fix feedbacks that way.
A clear versioning strategy is also necessary. Avoid a multitude of branches if possible. The preferred way is to have a public read-only CVS archive. (you can use cvsup to create a mirror of the real archive in case you have a security breach on the public server.)
Have a reasonable licensing for your software, it will pay off in the end. You may want to take a look at MySQL. Try to be flexible and not too complicated.
This seems to be what I could come up with on a short notice.
And GOOD LUCK!
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
OpenBSD is striving to open up all drivers, and refuses to allow binary modules. Currently, they are targeting Adaptec, trying to get the management interface opened up. We don't want them to write code for us, or to support it, we just want docs, so *we* can write the code, and support it.
1 8231311&mode=expanded
http://undeadly.org/cgi?action=article&sid=200503
It's great you're doing this. Make sure that people (i.e. potential customers) hear about it. Given a choice between two comparable products, if I know one of them supports the free software community, I'll choose them. I know I'm not alone. You're not only going to benefit (eventually, don't expect instant gratification) from code feedback, your sales will tick up, assuming you market yourselves well. Try to measure it, and then show the evidence to your company's other divisions.
Because hardware has a bad tendancy to hang around long after the vendors lose interest in it?
By making the drivers open source & letting the OS supplier recompile them for new releases they lessen the future load on their support desk from people complaining that their Linux 2.8 binary drivers won't work in Linux 4.2
As an observor on the sidelines, here's a few points that sometimes cause issues.
- Expect a rough transition.
Releasing your app to the community is like hiring a bunch of new developers but not giving them any management. If they like what you have, they will work with it; if they don't, they might re-implement things or openly disagree with your existing design. Get as much relevant information online as possible so others can make informed design decisions.
- Provide direction, but be flexible.
One benefit of OSS is that others can suggest fixes that may directly contradict your current view of the problem. By carefully accepting some of these changes, your software will become better.
- Don't expect the OSS community to do all the work.
Several major bloopers have come from companies saying, "fine; we're open-sourcing it; let them do the work". This is the road to stagnation. The community will support things that are useful to them; don't assume that your alpha-OSS release will generate immediate support. A small OSS community is excellent for porting existing software to new systems; they are generally slow for actual development work.
- Keep providing support.
During the initial transition, you will probably have more work than normal as people flock to your project asking questions. Then only those who like what they see will stay. At a minimum, your company should host an email list and an anonymous CVS or Subversion server.
- Advertise the transition to your users.
Make sure your customers understand that they can now customize things in-house. Make OSS a "value-added" feature. Encourage them to return their improvements back to the community.
- Make a good testing framework available.
Most of your end-users will only have access to the hardware they actually use. Your current Q/A process probably tests against a range of hardware. As such, you own a range of test machines. Network these to a test framework that can validate community changes as they are submitted. Maintain a "stable", in-house tested branch and an "unstable", bleeding-edge branch.
http://www.intel.com/software/products/opensource/
http://www.intel.com/cd/ids/developer/asmo-na/eng/ 52779.htm
I am the maintainer of a driver in mainline Linux. An competing driver is offered by the actual hardware vendor (also Open Source). While working with their engineers has been quite pleasant, we have never been able to agree to work on the same driver.
So the people who know the hardware best are paid to work on a driver that few people use. Meanwhile, the driver in mainline keeps up with the frequent changes of in-kernel APIs but lacks the resources to make use of all the features the hardware offers.
A few companies (e.g. Intel with their eepro) seem to get it right: Have someone work with the community to write and maintain a driver in mainline. You are still largely in control as long as you are competent, and you are pushing the code people actually use.
1) As others noted, do a full code audit and make sure there's no proprietary code in there at all. When in doubt, take it out. You don't want a lawsuit on your hands. Make sure you have the rights to distribute all of your source.
2) Clean up your code. If the comments are incomplete, complete them, if there's something that's obfuscated for no good reason, unobfuscate it, etc. Remember that for it to be useful someone who's never seen it, and doesn't know how your stuff works. While doing that clean up any bad language in the comments and code.
3) Make sure your code builds completely to a final useful state on standard compilers (GCC on Linux, VisualStudio on Windows). If there's any special options that need to be set, document them. Don't release something that won't compile without tweaking, it should be ready to go.
4) Don't neglect binary versions. Keep them at least as current as the source versions, if not more so. Many (most?) people don't like fucking around with compiling their own stuff. It takes time, and the compiler is scary to non-programmers. Have an easy to install binary version as you did before. Goes double for Windows.
5) Do it for the right reasons, that being to get feedback from the world at large and to help out. Don't do it expecting the OSS community to pick up your slack and develop your drivers for you. You might get lucky and find that some extremely talented individuals do just that, but more than likely if you open them up and ignore them, they'll become crap.
6) If you take community contributed drivers that you have nothing to do with (like ports to an unsupported OS), make sure you make it clear on your site that they are different. Have a clear demarcation between drivers you created and supported (with or without community help) and drivers someone else did, but you didn't make and can't support.
In general I think it can work to your advantage, but only if you treat the OSS community as an additonal asset, not as your core development. Maintain the same team you have now, same standards for testing and quality (I'm assuming they are good here) and so on. Take any useful contributions the OSS community provides, but don't rely on them to start doing your job for you.
Make sure you tag each release with a tag appropriate to its stability (prealpha, alpha, beta, stable etc.) Also you might want to do what redhat did with fedora and have a section if the team working on the drivers in an open source manner. naturally keep the binaries available for download too. And make sure it compiles on GCC and mingw. Good Luck!
Open-Source > *
Interesting, a study we did showed that in terms of productivity and readability 2 space tab indents was optimimum. "Why?" I hear you ask - any developer that's worked on a project of any size above "tiny" will know that large indents don't aid readibility, unless your code is very 'squished'. Which brings us onto one of the most important aspects of any project - white space.
Let's look at the following chunks of code;
Many would write a simple for-loop like this (using standard 8-space unexpanded tabs);
for(int l=0;l10;++l){
printf("%d\n",l);
}
versus the more experienced coder (using 2 space expanded tabs);
for ( int l = 0; l 10; ++l )
{
printf ( "%d\n", l );
}
Maybe it's just my opinion (and many others on projects I've worked on), but the second is far more readable and when a new coder comes along to maintain the code she/he would have a much easier job going through the code to figure out how it works. You may not believe me, but take any large chunk of code you may have and take the time to space it out and re-indent it - I guarantee that any decent coder will see my argument is correct.
"Some vendors have closed driver support for Asterisk, eg. Intel/Dialogic which means their drivers can only be sold through a non-GPL Asterisk License. This however means that they rely on sales through Digium, who hold rights in Asterisk. The irony is that Digium are also a telephony interface card vendor and thus a competitor."
d ev/200 4-July/005203.html
:)
:)
I wouldn't be surprised if this is one of Gerry's motivations for switching to open-source. Do a bit of Google searching, examples are:
http://lists.digium.com/pipermail/asterisk-
Gerry works for Intel/Dialogic.
As another poster who figured out who you work for said, you might want to get in touch with people in your parent company familiar with open source, such as the eepro maintainer. They'll probably give you better answers than Slashdot. Although they won't give you as much free publicity as Slashdot.
retrorocket.o not found, launch anyway?
Get into the official kernel. If a driver isn't in Linus's tree, it doesn't matter, so you will be on an endless treadmill of API breakage. Once your driver is in the official kernel tree, the kernel hackers will take responsibility for most of the API refactoring.
Know the politics. Most Linux kernel developers aren't accountable to anyone and don't negotiate. You will have to put up with whatever requirements they give you if you want your code to be part of the kernel.
Know the effort. You will probably be asked to rewrite your drivers, possibly more than once. This will take months. If you don't do it, then open-sourcing the drivers was mostly wasted effort.
As others mentioned, coding style is important. Also, wrappers are not allowed in the kernel, so call kernel APIs directly instead of wrapping them. The result is a totally Linux-specific driver, but the rules are the rules (see above).
As a developer of binary drivers you are used to having complete control over your code, and "pushing" all updates down-stream. But once your code goes in the kernel (which I highly recommend - don't let it languish outside Linus' tree), you'll have to consider how to deal with code changes coming TO you FROM the kernel developers.
This is both a good thing and a bad thing. It's good because you'll get fixes from people testing your code on all sorts of weird platforms you've never heard of. It's bad because you can wake up one morning and discover the kernel API for your type of driver has changed overnight, and your code won't even compile until you re-write half of it (there go your plans for the weekend). A certain amount of lag is acceptable, and you can restrict support to the stable kernel series only if you want, but expect to hear a lot of whining from users who will demand that you keep up with the cutting-edge development code.
I can think of two things to look at with respect to comments.:
First of all, you would like your comments to be meaningful, understandable and accurate. (I'm sure I could find you some juicy counter-examples in my own code).
The second thing (which, I think the grandfather post referred to) is: You might want to edit out comments like
- "I was going to shoot George Bush but this seems even more insane",
- "This is the worst IP violation I've ever committed", or
- or "This code stinks worse than our hardware."
You know -- stuff that just might embarrass your PR group if it got published on slashdot. There probably isn't a whole lot of stuff like that, but you should hire a couple of young code monkeys to go a quick read thru your code, and flag anything even vaguely questionable for your more senior programmers to vette.Murphy's law says that you won't necessarily catch everything that might be embarrassing, but if only one or two nasty examples make it past the review, you can always blame it on too much coffee. If there's lots of stuff that you find on a quick audit, then you might want to delay the public release for a couple more months while you go over the code with a fine toothed comb.
If you can find some code monkeys with OpenBSD style auditing experience, then you could possibly add in cleaning up the actual code to the benefits of such an audit. This code is going to represent your company (unless you release it anonymously), so it'd be good to release the best code your resources allow you to generate.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
For whatever my word is worth, we're doing this for several reasons: 1) Sell more stuff. I am a greedy, capitalist pig and I want to sell more stuff. I know from having participated in the Linux industry since installation was on a pile of floppies that open source drivers allow hardware companies to sell more stuff. 2) Help our software be better. As mentioned above, I know that open source leads to better software, even if you do start with a train wreck. :-)
3) Help our customers. It's only somewhat altruistic in that I know that open source will help our customers in making their products better. By doing this, it helps me and my company.
We are going to continue to own and maintain the code and, quite frankly, we are not anticipating much initial assistance. It's a long-term effort to work better with our customers so that they won't have similar horrible experiences as you, sadly, had.
Right. And autoconf has nice structures to test for things like kernel versions, the presence of any specific libraries, GCC versions which can affect code compilation, the presence of x86 or ppc or x86_64 or ia64 hardware, etc., etc., etc.
.deb installation packages. But hey, if you want to write magical mystery Makefiles and #ifdef statements to deduce what the person doing the compilation actually wanted to do, go for it. Have fun re-inventing the wheel for every single program you write.
It even has nice structures for building one version of a kernel module that's not the one you're running at the moment, or building RPM or
Just as IBM is being asked to pull out 30 years worth of source code for SCO, your company might be asked to do the same by some company abusing the legal system. If your whole source evolution may be viewable, that suggests that only a ground-up rewrite can hide some of your nastier ethical and social improprieties. The best option in this situation is making it open-source from the start (even basing on open source code) and black box testing against your old code. Making it open will do a lot to keep it clean and test it hard.
True, your company may have signed all sorts of legal entanglements with your old source, but you'll probably have to rebuild it all from scratch anyway in eighteen months or whenever Longhorn is finally unleashed. Starting over is inevitable; get a jump on things and start that re-construction effort from scratch now with an open source driver. Whatever changes you have to make for Longhorn, you'll be starting from a trunk of source that's stable in execution and legality.
That's just fine. But if they want control over everything, they need to be responsive by getting things done and have useful information on the school website.
As a blatant example, it's a disgrace to the school to have sports and music schedules that are 3 years old attached to "upcoming events" on the main page. Equally lame was that PR would not allow departments to post lists of classes being offered.
But I think the most absurd was that PR would not even allow the School of Business, which has professors being entrusted with teaching things like marketing and branding, to have input about what was on the website for the School of Business.
It's really sad because while the website is pretty, it's really devoid of useful information. If I were a prospective student I would quickly dropped the school from consideration just based on how lame the website was. In my case, I worked at the school (systems administrator) and got a nice discount.
It was frustrating telling professors and department heads, "Sure, you COULD have your course schedules put online, as well as a staff directory. In a technical sense, it's quite simple to do, really. But you'll have to get it past PR first."
Going back to responsiveness, I was assisting a professor with creating documents for a series of seminars he was contracted by the state to do through the university. We created brochures for the seminars, giving descriptions, agenda, etc. Because it was done through the U, PR demanded final approval on the things. I would agree with that in general. But, even with persistent hounding, they did not bother to take a look at them until after the seminars had already passed. How were we supposed to get the word out, and people registered (and paying) for the seminars if we can't send brochures?
So, maybe I'm tainted by my bad experience with PR where I worked for several years. But my general feeling is that PR folks are arrogant asses who don't give a shit about what other people are trying to accomplish and who feel that as long as it looks pretty, there's no need to have current or useful information available for people.
That's not good branding. It's just being an ass.