OpenCM Alpha6 Released

Jonathan S. Shapiro writes: "The new OpenCM Configuration Management system has just released it's first external alpha (0.1.0alpha6) over at opencm.org -- brought to you by the same folks who are working on the EROS operating system. Being a 0.1 alpha release, it shouldn't be used for prime time development yet, but it's definitely ready for a first look. We've been using it internally for OpenCM development for a while, and we have officially moved EROS development over. It looks like most of what needs to be done toward a first release is cosmetic at this point. Alpha6 has been tested on RedHat 7.3. Porting and cleanup are underway for solaris and (soon) *BSD. People seem to be excited about this -- especially the OpenBSD folks. While you are over at the web site, have a look at the two papers on OpenCM from this year's USENIX and USENIX Security conference."

Comments

I like Slime

[Anonymous Coward]

Although the newer systems are pretty nice too...

Details, details...

[hackwrench]

It should have been mentioned in the article that OpenCM is a replacement for CVS. Also OpenCM (Configuration Management) is a bad name for this sort of project Configuration Management sounds like the functional equivalent of Control Panel not Code Versioning Management.

Re:Details, details...

[Twylite]

"Configuration management" is an industry term for precisely what this product does, which is product revision control, and not source file revision control. You will notice that other tools like AEgis and many commercial tools also refer to themselves as "configuration management" rather than "source control".

Congratulations

[mlinksva]

When the poster announced this project a little over two years ago I asked why in the world spend time developing another configuration management system with many extant, more on the horizon, and so much work left on EROS. I certainly thought OpenCM (then DCMS) would never see the light of day. I was wrong, and glad about it. I'm really looking forward to disconnected operations.

Finally,

[Anonymous Coward]

A source control system that doesn't suck. And I don't count bitKeeper - I've used it, it does suck, no matter what marketspeak McVoy spews on LKML.

But why the **** is it called a "Configuration" Management system? That's just confusing...

Re:Finally,

[MadAhab]

I'm hoping it becomes the Mutt of cms/version control. You know: "All configuration management systems suck. This one just sucks less."

I'm still trying figure out how this one will suck a little bit, but less than other systems.

But why the **** is it

[oliverthered]

Because that's what it is,
It manages the configuration of the code base etc...

Manual:

[Hellkitten]

I just love this little pearl from the manual on the site:

This tutorial assumes that your name is "Jack."1

And then as a footnote at the bottom of the page

1. If your name isn't Jack, and you don't know your name, contact your local administrator for support.

The boldness is mine, the italics is theirs

Re:Manual:

[josepha48]

guess that means you don't know Jack.. (lol)

On another note EROS spelled backward is SORE. Do you really want to use a sore OS?

I thought linuxconf was supposed to be the start of a 'generic' configuration management system. Supposedly it would work on other UNIXes as well.

Oh well guess just one more. I actually prefer using the very basic tools, like useradd/adduser and eiting the config files sometimes.

Update: OpenCM port submitted

[Jonathan S. Shapiro]

Todd Fries just submitted the patches getting OpenCM to build under OpenBSD. They'll be included in the next alpha.

Re:Update: OpenCM port submitted

[Anonymous Coward]

*chuckle*

Oh, I guess this news will matter to 3, maybe 4 people.

What a loser.

excuse me...

[Anonymous Coward]

..this isn't a troll. What exactly is this and what is it for? When I read "browzilla released" I got a clue, this thing, no clue. thanks in advance.

What configuration?

[The Pim]

"Configuration management" is an industry term for precisely what this product does, which is product revision control, and not source file revision control.

It's still a stupid term. Tell me, where is the configuration that is being managed? Industries have lots of meaningless jargon--doesn't mean us sane folks have to parrot it. "Revision control" works just fine and makes sense.

Re:What configuration?

[Twylite]

Confusing, maybe, but certainly not stupid. OpenCM, like many other tools, is mistaken as a revision control tool, but isn't.

Revision control systems like RC and CVS are concerned with tracking revisions of files. The have some limited capability to track revisions of entire systems. But the lack the capability to properly track and manage the way a system is put together from its components.

Maybe the easiest way is to think of a system as one of those cool (well, they were then) transformer toys. In its airplain configuration is goes really fast, but change it to its robot configuration and it has laser weapons.

In a software system, configuration management is concerned with which revisions of which files make up a particular "distribution" (for lack of a clearer word). It can also track this information across what would normally be separate projects in revision control systems. In its simplest for, CM can be a text file which says "This is label 1.2.14 of the main system, and is built against label 2.7 of libraryA, which in turn is build against label 0.96 of libraryB". A proper CM system gives you far more flexibility (I am not familiar with OpenCM's capabilities).

What software geeks usually refer to as "configuration" should more often than not be called "preferences". A Linux distribution is a particular configuration of Linux and associated tools, but your installation of that distribution has been specialised with a number of your preferences.

Re:What configuration?

[The Pim]

OpenCM, like many other tools, is mistaken as a revision control tool

It's more natural for me simly to think of RCS and CVS as semantically limited revision control tools, and ClearCase, subversion, and BitKeeper as more complete and powerful revision control tools. Ie, I acknowledge the fundamental differences between more primitive and more advanced systems, but I reject the notion that they deserve different names.

Maybe the easiest way is to think of a system as one of those cool (well, they were then) transformer toys. In its airplain configuration is goes really fast, but change it to its robot configuration and it has laser weapons.

Yeah, I've thought of that argument, but it's a big stretch. "SCM" systems don't let you use a single codebase as either an airplane or a weapon, at least not the way they are normally used. And even if I let you get away with calling the specification of library versions as "configuration", that is hardly a core feature of these systems.

Face it: the primary purpose of any of these systems is to manage the various versions in some body of content. So I reassert that "version control" or "revision control" or "change control" is the best name.

And by the way, the fact that the name is confusing does matter, especially for a free software project. If you're trying to appeal to developers, its best to pick a name that won't confuse all the students who haven't been corrputed by industry jargon. :-)

Re:What configuration?

[dutky]

on July 6, 2002, Pim wrote:

Face it: the primary purpose of any of these systems is to manage the various versions in some body of content. So I reassert that "version control" or "revision control" or "change control" is the best name.

No, as the previous poster correctly stated, that is the job of a version control system like RCS, SCCS or CVS. A CM tool keeps track of a bunch of information that is not part of the source code in any way. CM allows you to capture how to install a program (or system of programs): what files need to go where, what directories need to be created, what environment variables (or symbols, logical names, etc.) need to be defined and what their values should be, and a whole host of other things that are not directly captured in the program's source.

You can use a version control tool to keep track of this information, but it is a lot of work and is pretty hard to keep straight (keeping the install scripts in CVS is often confused for a CM solution, but it doesn't help when your old CM guy leaves and you need to bring the new CM guy up to speed on what goes where and why). The real purpose of CM is not simply to make it easy to install and configure the system, but to make it easy for humans to understand what that process entails and detect when something has been changed or overlooked.

Some folks might be forgiven for thinking that CM and version control are the same thing: if you have never worked on a real production system before (one with many programs and a complex set of directories and configuration files, as opposed to some little toy program for a class project) you have probably never encountered the need for CM. Unfortunately, there is almost no way to demonstrate the need for CM in the small: unless the size of the system is larger than what one person can comfortably think about at one time, the need for CM just doesn't arise.

Re:What configuration?

[The Pim]

Ok, if configuration management is this big thing that is fundamentally more than version control, answer these challenges:

1. Why does the OpenCM home page call itself a "replacement for CVS", which you say is a version control tool, not a configuration management system. Why does the site (at least the overview and features sections) list only version control features? Where are the CM features in OpenCM?
2. If CM is needed for programming "in the large", how have most large free software projects gotten away without it? Or, if they are using CM in some form, where is it?
3. I've used ClearCase for several medium-sized (5-10 full-time programmers) projects. As far as I can tell, all of our use of ClearCase falls under what I would consider version control. What parts of ClearCase are configuration management? (Of course, you can skip this if you haven't used ClearCase.)

I know about programming "in the large" (or at least medium-sized). I've used tools more powerful than CVS. These tools are powerful because they do version control better than CVS, not because they offer something fundamentally different.

And your argument that CM is about what files go where seems like nonsense to me. First, even if that is important, it hardly requires a sophisticated tool. You might as well use (a subset of) RPM for that. Or a 100 line Perl script (that reads the file locations, etc, from a config file). Second, code should generally be buildable and runnable right in the source control system, for quick testing. So if you need some fancy layer just to get the code into a runnable state, you're probably doing something wrong.

Re:What configuration?

[shoreman]

1. Because CVS is a very popular system being used currently for an "80 percent solution" for true configuration management. The user interface to OpenCM could have been designed to any paradigm, but why not use the most widely used (open source) system since a LOT of developers are already familiar with that? BTW, OpenCM only manages configurations, or sets of files. You can't (yet) grab only a single version of a single file unless it's the only member of a configuration. That's the whole point of CM, as previous posts have exhaustively pointed out.

2. Most large free software projects have "gotten away without it" because "it" hasn't been available! Again, one of the best (open source) solutions to true CM out there currently is CVS, and it's only an 80 percent CM solution! Ask _several_ CVS users what they don't like about CVS and you'll have a list of requirements for a true CM system!

3. ClearCase is a marvelous tool. (I'm not an expert at ClearCase, but I've used it.) Unfortunately, it costs more money than I have! However, don't you have to have network access to your ClearCase server (repository) in order to work with it? Wouldn't disconnected development be a wonderful addition? (It doesn't look like OpenCM actually supports that yet, but it sounds like it's in the works.)

Re:What configuration?

[The Pim]

1. I agree that having deltas that span multiple files is a good thing. But this is just an improvement on the version control theme: managing changes in groups of files, not individual files. If this is "the whole point of CM", then CM is clearly just version control.
2. I know (everyone knows) that CVS has many shortcomings. But the shortcomings (at least the ones I've heard about, eg lack of atomic changesets, poor branching/merging, centralized) are still about version control.
3. ClearCase has some disconnected functionality (snapshot views), though it has many flaws (like, due to the fascist license regime, you can't run any clearcase commands while disconnected!). But what is the point of your question? ClearCase is commonly termed a CM tool, but as a user, all I see is version control. What part of ClearCase is CM, as opposed to version control?

Again, my challenge is to evince some important function of "configuration management" tools that is fundamentally different from version control.

Re:What configuration?

[dutky]

again, Pim wrote:

Ok, if configuration management is this big thing that is fundamentally more than version control, answer these challenges:

1. Why does the OpenCM home page call itself a "replacement for CVS", which you say is a version control tool, not a configuration management system. Why does the site (at least the overview and features sections) list only version control features? Where are the CM features in OpenCM?

Well, I wasn't addressing the question of whether or not OpenCM was a valid CM tool (which it may or may not be, I have no direct experience with it) but whether or not there existed a discipline of CM separate from that of simple source code version control. Similarly, your assertion was not that OpenCM wasn't really a CM tool, but that the term CM itself had no use or meaning separate from the concept of version control.

2. If CM is needed for programming "in the large", how have most large free software projects gotten away without it? Or, if they are using CM in some form, where is it?

Probably the same way that many large commercial projects have gotten away without any real version control: badly. Free software projects are actually at something of an advantage in this respect. There are fewer time and budget constraints on free software than on commercial projects, so free software can afford more wasted effort making up for the lack of CM. As well, free software often has a freeer hand in designing the system so that CM might be less of an issue. Finally, most free software projects are simply not large enough to need CM: those that are large enough, probably have some form of documentation that covers CM issues outside of the version control arena (FAQs, HOWTOs, ReadMes, etc.)

3. I've used ClearCase for several medium-sized (5-10 full-time programmers) projects. As far as I can tell, all of our use of ClearCase falls under what I would consider version control. What parts of ClearCase are configuration management? (Of course, you can skip this if you haven't used ClearCase.)

5-10 programmers is a pretty small project compared to the kinds of things I've encountered that required active CM: think of something like 20-30 programmers and 3-5 subsystem architects (remember, BTW, complexity increases exponentially to team size, not linearly) As far as ClearCase is concerned, I've never used it so I wouldn't know.

I know about programming "in the large" (or at least medium-sized). I've used tools more powerful than CVS. These tools are powerful because they do version control better than CVS, not because they offer something fundamentally different.

See above. 'nuff said.

And your argument that CM is about what files go where seems like nonsense to me. First, even if that is important, it hardly requires a sophisticated tool. You might as well use (a subset of) RPM for that. Or a 100 line Perl script (that reads the file locations, etc, from a config file). Second, code should generally be buildable and runnable right in the source control system, for quick testing. So if you need some fancy layer just to get the code into a runnable state, you're probably doing something wrong.

The only reason that you think it is nonesense is that you've never worked on a large production system. CM isn't about a technical fix for a technical problem, it is about managing the complexity of large systems, their installation and configuration, and allowing humans to keep track of all the fiddling details. A tool can help with some of this, but the real heart of the matter is a managerial issue.

As for doing something wrong if it takes so much effort to get the code into a runnable state: you are probably correct. The sad fact is, however, that most large software projects are doing something wrong in exactly this way (many somethings, in fact), but there is little hope of being able to turn the tide.

Really large software projects are too big for even a large team of people to tackle every aspect of the system. Therefore, large projects must be built on top of smaller pieces, often pieces that have been built by other people. Each of these component pieces has its own fiobles and requirements, all of which combine to present the system architect with one giant mess of fiobles and requirements. The job of CM is to manage this mess of foibles and requirements in such a way that the system can be installed and operated without bankrupting the company in the process.

Re:What configuration?

[Twylite]

We use a commercial configuration management tool which has no revision control capability.

Pre-release activities include labeling the source code in the various parts of the separate revision control tool and archiving it (.zip or .tar.gz), then putting the code archives, design documentation, hardware diagrams, Bill of Materials, installation programs, user manuals, and an image of the final (distributable) CD into the CM tool.

Every unit of data which is stored in this manner is assigned a unique identifier, which is propegated back to the source of the data (VSS trees are labeled with the source code UID, source code is updated to contain the UID so that it appears in DLL versioninfo, documents have their UIDs added into them, the CD has the UID in a file in root and printed on the label, etc).

Finally, the CM also tracks the development of systems as a whole (hardware, software, documentation in their entirity), as well as delivery to customers. Patches are also entered into the CM separately, and their delivery tracked.

In short, given any program or part thereof, or piece of hardware, documentation, CD, whatever, we can identify all the source code (main application, tools, firmware), design documents, hardware processes, material providers, previously discovered bugs (linked to a separate system), patches available and applied, and so on.

For a commercial environment which values engineering processes, the benefit of this sort of knowledge should be clear. It is also something which can't be done with most revision control tools (notably, it would be possible to do it with VSS because of the availability of linking and cross-project labeling, although still not an ideal solution).

Re:What configuration?

[Jonathan S. Shapiro]

Actually, software CM is in some sense the dual of system configuration management. SCM is about getting consistent configurations in to a repository. System CM is about blowing those configurations out to a bunch of machines. The first step, of course, is to know what constitutes a consistent set of configurations.

System CM is a lot more than preferences. Think about what happens when you get dynamic library versions wrong.

OpenCM actually could be used for system CM when the bundle model goes in, but it's different enough from the way existing system CM is done that I don't expect it will catch on for that purpose.

Re:What configuration?

[Jonathan S. Shapiro]

To answer your questions in turn:

1. CVS uses various tricks to straddle the boundary between revision control and configuration management. Features like "tags" are an approximation to configurations. There are things we need that CVS doesn't (and can't) do, like rename, but CVS is presently the best available tool (or it was two weeks ago :-)

   We describe OpenCM as a replacement to CVS because the command interfaces are intentionally similar and because most of the target users of OpenCM are currently using CVS.

2. CM isn't needed for programming in the large. It's only easiest to see there. CM starts to be needed the minute your program is spread across more than one file and you might need to recover older versions. The missing piece in RCS is that it does not know how versions relate between one file and the next.

3. If you are managing sets of versions of files, instead of versions of a single file, then you are doing configuration management. I understand why you don't want to give a separate name to configuration management, but the naming debate ended in the SCM industry almost 15 years ago. Arguing about it now is pointless.

SCM isn't about what files go where. That's system CM. SCM is about which versions of which files go together to build a particular application version.

Re:What configuration?

[Jonathan S. Shapiro]

Pim:

You are arguing about lexicon. As I said earlier, the software industry made it's selection here 15 years ago, and it isn't going to change. I understand why you think a new term isn't called for. In fact, I probably agree. That said, arguing about it is kind of like trying to reopen because you don't think the armistice lines were drawn in the right places.

Re:What configuration?

[dutky]

holding blindly to his course, Pim wrote:

1. I agree that having deltas that span multiple files is a good thing. But this is just an improvement on the version control theme: managing changes in groups of files, not individual files. If this is "the whole point of CM", then CM is clearly just version control.
2. I know (everyone knows) that CVS has many shortcomings. But the shortcomings (at least the ones I've heard about, eg lack of atomic changesets, poor branching/merging, centralized) are still about version control.
3. ClearCase has some disconnected functionality (snapshot views), though it has many flaws (like, due to the fascist license regime, you can't run any clearcase commands while disconnected!). But what is the point of your question? ClearCase is commonly termed a CM tool, but as a user, all I see is version control. What part of ClearCase is CM, as opposed to version control?

Again, my challenge is to evince some important function of "configuration management" tools that is fundamentally different from version control.

First, CM has nothing to do with file deltas. You may assert it as many times as you like, but still, CM is not about version control.

Second, the shortcomings of CVS are not the issue. CVS is a version control tool, not a CM tool.

Third, I've never used ClearCase, so I don't know what features it has that may (or may not) be relevant to CM. However, unless you are the CM guy, it is likely that you never use any of the CM features of ClearCase. As a programmer, you don't really need to do anything other than version control, but that doesn't mean that version control is the only task that is important or that the tool supports.

One final time: CM is all about keeping track of exactly what versions of what files went into a specific build of a system (you may be shipping different builds of the system to different clients, for example), where those files are stored in a specific build, what was done to install the build on the client hardware and what environment variables/registry entries/symbols/logical names/etc. were set to what values for any specific installation. Much of this information can be (and often is) stored in the form of installation and configuration scripts that are kept under revision control. You may also want to associate user documentation, discrepency reports, enhancement requrests, work orders, and even client specifications or contracts with a specific build for a specific client. On top of all this you would like to have the information in a form that can be readily digested by humans (assorted install/configure scripts do not meet this requirement) so that new recruits and non-coders can understand what goes into the system and its installation.

To say that just becuase you have the installtion and configuration scripts in revision control, you are effectively managing the system configuration is like saying that just because you have been given the compiled executables, you have all the documentation you need to maintain and enhance the software! CM is not about the source code, but about the meta-data of the built and installed system. It tells you not only what you can tweak in the system, but also how that tweaking was done for a specific installation.

In summary, CM records the following facets of the system:

1. what versions of the OS, compilers, development tools, OS utilities, libraries, etc. were used to build the system.
2. where are the compiled executables and configuration files stored during installation.
3. what environment variables, registry entries, logical names, symbols, etc. need to be defined for the system to operate, and what does each value do.
4. anything else that needs to be done to build and install the system, including: file system structures (do we need to create specific directories with specific permissions?) network connections and hosts (do we need to have specific hosts that the system will try to talk to?), peripheral devices (do we need a printer/tape library/spot-welding robot attached to the machine?), etc.
5. what 'bill of materials' was included in a specific release to the customer (or should have been), including: database structures, static database entries, data files, executables, documentation, etc.
6. what client requirements were met by a specific build, including: contracts, specifications, enhancement requests, bug fixes, etc.

Re:What configuration?

[The Pim]

3. If you are managing sets of versions of files, instead of versions of a single file, then you are doing configuration management. I understand why you don't want to give a separate name to configuration management, but the naming debate ended in the SCM industry almost 15 years ago. Arguing about it now is pointless.

Ok, I'll relent and accept that as the defining characteristic of configuration management. At least it is a clear criterion that is stronger than "version control", and has some amount of sense to it. As opposed to some of the divers definitions offered on this thread.

Even though to me (as I said before), it is just "better version control".

Re:What configuration?

[The Pim]

I certainly agree that the functionality you described is useful. But most of the systems I've heard described as "configuration management" don't do that, at least not as their primary function. They primarily do version control. For example, OpenCM calls itself configuration management, but describes itself as a "replacement for CVS". So either someone (you or the OpenCM people) is confused about what CM is, or CM is just an amorphous buzzword.

That said, I do now have a better appreciation for your view of configuration management. Let me try to fit my experience into that framework. ClearCase (which I do know fairly thoroughly) has some features in this area. For example, it can hook into your build process and remember which source-controlled files contributed to built files. It can give you a "view" onto arbitrary versions of different modules. But bundling up a release, producing a manifest, tracking installations, etc, are generally outside its scope (and are left for tools such as ReleasePro). So by your definition, ClearCase is not a configuration management system.

linuxconf

[Jonathan S. Shapiro]

Linuxconf is for configuring a system. OpenCM is for managing software source code configuration. Different problems, different tools.

Re:linuxconf

[josepha48]

So then that would make it more like MS SourceSafe or PVCS, both of which handle source code control and configuration. I could not see what features it has as the connection was refused. Does it handle binary files????

Oh and don't take my comment about eros to heart, where I live we have a club called eros.. and it is for just that getting 'sore'....

Re:linuxconf

[Jonathan S. Shapiro]

Yes. Like SourceSafe or PVCS, only better. :-)

We have been having some crash problems with the OpenCM server, so if the connection is refused it's always worth trying back later.