How to Fix the Unix Configuration Nightmare

jacoplane writes: "There's an interesting article on freshmeat talking how sorting out some kind of standard for configuration could really help Unix systems could be more user friendly. The article points out that since Apple has managed to build a quite usable system on top of NetBSD, it should be doable to do the same for open-source interfaces."

Comments

Re:If I remember correctly

[hype7]

but like the article said, Apple managed it.

Says something about everyone's favourite fruit company when they can manage what has previously been impossible :)

-- james

There are valid reason

[evilviper]

There are situations were I've wondered why they went off and created their own configuration file format just for that one application, but that's not too common. The problem with a universal format is that each application need to do different things. Typing ALL apps to one format would in many cases make configuration changes more difficult.

i.e. instead of changing one thing, you have to make the change for every interface/terminal/etc it is running on. Just an example mind you.

Re:There are valid reason

[walt-sjc]

Which is why you use XML. Human readable, parsable via common code library, extensible (that's the X part...)

The cool part is that one tool would be able to configure new things that didn't exist when the configuration tool was designed as long as the tool was well thought out and feature complete. Since the current crop of configuration tasks is quite diverse already, I see no reason why this wouldn't be doable.

With this system, you would also never have to write your own GUI config tool for each app ever again. KDE, Gnome, Samba, Apache, Mozilla, Galeon, etc. could all dump their custom X / web-based tools if they desired (you could even have a web-based version of the tool.) Hell, you could even use this to configure the kernel.

For migration, separate utilities could import / export from the new XML based files while all the software that uses config files get's updated.

Some software such as Apache is already close as it is already somewhat xml like.

The one thing I ask that of ANY config tool is that it NOT be like the Windows Registry, Don't put everything in one big-ass binary database. That's the most hostile, fragile pile of crap on the planet.

Re:There are valid reason

[4of12]

Which is why you use XML.

Yes!

The Software Carpentry project, though loaded with good intention, in my view made a mistake in rejecting a really promising looking project (TAN, I think it was called) that proposed to store configuration information in XML files.

Based on all the experience with config.cache on UNIX and the problems with Windows Registry, there is a real opportunity for designing an improved system, one that is open, hierarchal and extensible.

Perhaps there just needs to be a little shove over the bridge, in some sense. Something like a tool for converting

ac_config_has_glibc=yes
ac_config_glibc_version=2.1

syntax into appropriate XML.

There's a real trick, though, getting an easy-to-use XML hierarchal database for configuration to work well when local customization happens, when user/developers don't want to inherit the system wide default cached libraries and include files, for example. It can be done, but it needs to be done well.

Re:There are valid reason

[evilviper]

There's no way an appication can be forwards compatible in such a scheme. Configuration for GNOME and KDE would be nothing like configuration for Afterstep & IceWM. How would it pull the word 'Wharf' out of the air? And all the options that go along with it?

Obviously, every single section and applicable option would have to be put in the configuration file. That would seem fair enough, if a little bit large, except when changing one option changes all the applicable options to dozens of other options.

At which point, the conf file would be so damn big and complicated, you wouldn't dare edit it by hand, and it would be just as easy to make it a self-configuring executable script.

Re:There are valid reason

[evilviper]

Well first of all I've had over a dozen of registry failures, and I haven't even used Windows very much. It got better after 98 because of the backups, but sometimes that didn't work... I'd say the only reason you don't have registry problems is because some other failing component renders the system unusable before the registry has a chance to fail.

At least in Unix, once you find the config file you know that all the configuration is done there. In the registry there may be 2000 entries for one application, under different key names, and worst of all you can't always even guess what each key does (MWORD3=0x0004332h). If it wasn't for the cryptic nature of the registry, shareware would fail as you'd only need remove the reg key, then the app starts from 0.

With Unix, the location of the config files is in the man pages, they are much easier to read and understand than the registry, and you can actually fix things if anything ever goes wrong... Instead of just puking, apps will say there is a problem on line#16 of the config file and sometimes tell you how to fix it.

Finally, with Unix you CAN easilly modify the configuration with simple text processing commands. I've made a shell script that works around a bug, and sticks my current dynamic IP address into my Napshare client. It may not sound like a lot but the IP address it fetches is from my router, not local.

Isn't Linux getting there ?

[keshto]

These days, one can usually get away with the "./configure; make; make install; " routine and not need to worry so much. In any case, some of the charm of Linux does lie in your being able to tweak things. And Apple had the luxury of making one decision about software questions and implement them across all hardware.

Maybe the author should try webmin

[evil_roy]

Have a look. webmin

Don't confuse Syntax and Standards

[osolemirnix]

I like the idea, such a tool would certainly be nice. Unfortunately the article only focuses on the Syntax part, the file format and access. I would think that is rather easy, use some kind of XML and you're already halfway there.

The part that makes such a system really useful however, is a standard agreement of which information is stored and what it means. This is where the Windows Registry falls down. And Unix is even worse, because all it has is some common soft-of-agreed-upon shell variables, like $EDITOR etc.
Apple is able to do this better because they set the standards for the OS (even more than MS). The can have one central "registry" for something like default associations of MIME-types with particular applications and define an API so every application can use it and a user doesn't have to change his settings in his browser AND his mail client AND his ftp client, etc.

Given the diversity of the unix crowd, the latter seems difficult to me. Maybe they can include it as part of LSB for a start?

Re:Don't confuse Syntax and Standards

[j7953]

I would think that is rather easy, use some kind of XML and you're already halfway there.

No no no. XML is not the magic solution to all problems in communication, data storage and usability. Using XML does have many advantages for some applications, like communicating over the internet, but I don't see how

<DocumentRoot> /some/path </DocumentRoot>

is so much more usable than

DocumentRoot /some/path

The only thing XML will do if widely used for configuration files is make maintaining them harder for those who'd like to do that by using shell scripts or simple text editors. As a very basic example, using XML makes configuration files harder to grep. The above example might also be written like this:

<DocumentRoot>
/some/path
</DocumentRoot>

This is completely legal XML, but if you grep DocumentRoot this file, you'll get the following result:

<DocumentRoot>
</DocumentRoot>

which is probably not what you wanted.

Re:Don't confuse Syntax and Standards

[Elbereth]

There's nothing stopping you from setting up most configuration files in that same un-grep-able way.

For example:

ACL James \
Bob \
Mary \
Lauren

Most UNIX programs will even ignore whitespace in the config file. I would say that your problem with XML is not related at all to XML, but that you have a problem with a specific style.

IMHO, a standardized way of writing config files is definitely needed. XML would fit the bill, especially since it's so popular. Writing a new configuration language might be fun (and even a seemingly better solution), but every single person running Linux would have to learn that configuration language. Not everyone needs to learn XML.

Re:Don't confuse Syntax and Standards

[Chainsaw]

I don't see how

<DocumentRoot> /some/path</DocumentRoot>

is so much more usable than

DocumentRoot /some/path

Try to nest several DocumentRoots which have different properties. If we ignore those possibilities: XML is a standardized format. There are several usable parsers for almost every platform out there. Having an Apache-style file follows no specific standards. Parsing will also be a bit more difficult.

The grepping problem that you mentioned is really a non-problem: use xmlgrep, xmltree or similar utilities to find the data note that you are looking for.

Re:Don't confuse Syntax and Standards

[rbeattie]

I agree - though I think that XML is a GREAT idea for *nix to use, the format is just too flexible to just say "use XML" and leave it at that. There needs to be some sort of standard way of using XML as a registry so it's not a complete nightmare.

Other variations:

<DocumentRoot>

<setting name="someSetting">SettingValue</setting>

<someSetting>SettingValue</someSetting&gt ;

<someSection value="SettingName">settingValue</someSettin g>

<someSection>
<settings>
<settingGroupName>Test</settingGroupName& gt;
<setting name="SettingName" value="settingValue" />
</settings>
</someSection>

</DocumentRoot>

Etc. The variations on hierarchies and values are endless and is my main beef with XML. It sometimes can be too flexible and people who are using XML (especially newbies) make it overly complex or not suffieciently complex enough for expansion. In other words, this needs to be thought out.

But again, XML is a good idea. I hope it gets adopted as a general mechanism in *nix, but not in some random way...

-Russ

Re:Don't confuse Syntax and Standards

[walt-sjc]

How about:

Notice the "/>" syntax defining the the item as an empty element.

Really, you don't need to make things more complicated than they need be. This uninformed defeatist attitude will solve nothing.

Re:Don't confuse Syntax and Standards

[walt-sjc]

Ahh crap. stupid slash deleted my example, but the point remains (as others have shown) that there are alternate syntaxes that work and are simple.

For the other XML naysayers, styleguides solve most issues, and since the files would normally be generated by code, it's a non-issue.

Re:Don't confuse Syntax and Standards

[j7953]

And if you man grep you'll see the -C switch, which you can then do:

grep -C2 DocumentRoot MyFile.xml

<DocumentRoot>
/some/path
</DocumentRoot>

Yes, but how does that help me if I want to process the data filtered by grep in a script which expects a single line as input? Anyway, several other posters have pointed out the existence of grep-like tools for XML, so I think I should take a look at those before I continue to claim that using XML config files makes scripted maintanance harder.

Re:Don't confuse Syntax and Standards

[rbeattie]

I totally disagree with this. We need a generic XML settings document that is both flexible and standard. Having 30 different DTDs is just making maintenence and support a nightmare.

If all the tools used the same settings file format (expressed in a DTD) then GUI tools, remote administration and general sanity would all benefit. I've got better things to do to have to learn a new file type every time I want to tweak a new tool.

We're talking about pretty basic stuff here. Setting names in a hierarchy with values. Using separate files and XML makes it both standard and much more flexible and recoverable which is one of the main complaints about the Registry. It's just too big and cumbersome.

-Russ

XML as a starting point perhaps?

[andykuan]

In the writer's outline section he has a few bullet-points that scream "XML!" (I'm paraphrasing here):

1. A core system to handle parsing, verification, etc. -- If app configs were based around XML, you could use any of a dozen XML parsers out there.

2. A configuration format description file. -- Hmmm, sounds like a DTD...

3. OS-neutral. -- XML was meant to be portable from the get-go.

The other items lined up pretty well with what XML is all about as well. Anyway, that said, I'm not so sure this'll ever happen since, from a developer's point of view, it's a lot simpler to slurp in a bunch of lines from a text file with fgets and chop it up into words than to have to somehow link in a third-party XML parser.

Re:XML as a starting point perhaps?

[roundand]

Other points in the article that scream XML to me:

* A key element would be the configuration format description file. This would list the configuration options for a given piece of software, giving for each one the name, type (boolean, list, string, etc.), options, category (for subsections within the config), and help text (short and long).

This looks very much like XML Schema to me - it can specify all these data types, including enumerations, and a schema-aware XML editor (eg XML Spy in the Wondows world - anyone know the best Linux option?) will prompt you with help-lists of valid elements and attributes, or list of enumerated values. Doesn't do help text by default, but has an expansion mechanism (xsd:appinfo) for adding application-specific to be added to a schema.

* It all needs to be language, distribution, and operating system neutral, so as to avoid turning off any potential software developers who might find it useful.

I don't know of any programming language or O/S that doesn't have an XML Parser (many don't yet have XML Schema support, but if you stay off specifying default values you don't need that at run time). And XML is good for natural languages - UTF 8 or UTF 16 from start.

There aren't many formats that are equally machine and human readable, even fewer that allow document struture and data typing, and still fewer that have open or free implementations on practically all platforms.

Re:XML as a starting point perhaps?

[Twylite]

First, XML is not a silver bullet. The thing this article screamed was not "XML" but "API". Define the application programming interface for a library, and stop thinking that each application must directly access the native data.

Second, XML Schema is a definition for an XML language, not a document described in a particular XML language. XML is a meta-language, you have to MAKE a dialect using a schema or DTD first. XML Schema could not, for example, include primitive types for IP addresses (very common in config files), or validation constraints for integer values.

Third, XML parsers are no-where near as ubiquotous as you would like to believe. To date there are no (zero, zip, nada, nothing) fully compliant XML parsers. Read the XML spec. carefully and understand just how many IFs and BUTs there are to creating a parser. Xerces and MS's parsers are the closest, but still have a number of problems. Expat-derived parsers happily handle a large subset of XML, but not everything. A vast number of parsers out there ignore the DTD, which means they cannot handle entities (which is a big issue).

Finally, there is nothing intrinsically human-readable about XML. This is common bullshit sprouted by those who haven't actually considered the requirements for human understanding. A simple .ini-style line delimited key=value is the most readable and understandable (from a psychological viewpoint) configuration format.

Re:XML as a starting point perhaps?

[cobar]

What really should be done is to throw a wrapper around the XML parser so that you can do something simple like value = getpref(name) without specifying anything more than the config file location and file format (although probably better handled using a header at the top of the file.

XML is great as a data interchange format, but it's a pain for humans. Compare /etc/passwd in it's current form to an xml representation. Which is easier to understand at a glance? Simply throw a comment at the top of the file explaining what each field does and the non-xml version works extremely well. Writing scripts to work on flat files is much easier than xml, as is searching.

Really, there's no reason why the config library can't handle multiple file schemas according to developer or user preference.

Re:XML as a starting point perhaps?

[khuber]

In the writer's outline section he has a few bullet-points that scream "XML!"

I'd agree that XML is a good basis, but "XML" really doesn't provide much by itself. It's just a file format that is human readable. If you just use XML with a bunch of proprietary tags, your own XML language so to speak, you really don't gain much over the existing different syntax config files.

An automated tool has no clue what your ipaddress (or whatever) tag means at all. You need to provide additional context for tools to understand the semantics of the configuration data. To make configuration files understandable in a more intelligent sense, you need to either restrict the tags you use to your own configuration language, or you need to provide metadata of some sort.

Why is this intelligence necessary? Well there are all sorts of dependencies and relationships in configuration files. You might want a GUI to let you know if you change something that may break another setting, and so on. Plus ideally you would only allow legal values to be set. Data typing could be done with W3C XML Schema Definition Language, or RELAX NG schemas.

Which brings me to RDF, which I think would be better suited to this task than XML alone. If you use RDF (see http://www.w3.org/RDF/ ) you make it much easier to have a self-describing format that tools can do more intelligent things with than raw XML. While I don't think RDF, DAML+OIL, et al is enough to create a Semantic Web as Tim Berners-Lee is hoping, it _is_ a step in a higher-level direction that will support more intelligence in processing data.

Mozilla already uses RDF for various configuration files and I'm sure there are other applications that do too. Mozilla has a whole bunch of stuff about their RDF here.

XML is just a tree of "stuff" in human-readable format. RDF lets you set up properties and relationships in the data in a standardized way. I don't have a brilliant example to prove this to skeptics, but really it is a better way to represent a lot of types of data you want to be able to query. There are many knowledge bases, expert systems and other query engines already out there using RDF and even higher-level languages like DAML+OIL.

-Kevin

Re:XML as a starting point perhaps?

[Florian Weimer]

If you want to do basic syntax checking at the XML level, you end up with configuration files which are no longer human readable. Imagine mapping all those sendmail rulesets to XML! There's no better way to reduce readability, and you cannot use all the existing HOWTOs and books.

Re:XML as a starting point perhaps?

[costas]

I have played around with this exact idea in the past. I got a semi-working prototype working, but then gave up as the huge amount of work needed became apparent. Here are the highlights:

• Create a parallel configuration system depository somewhere outside of /etc, like /conf. That depository should only contain system and meta-system level stuff.
• Keep application-specific configurations in the application directory.
• Get rid of the ages old /usr, /bin, etc, etc. Blatantly copy OS X. OS X has solved most of the metadata management issues already.
• Maintain a separate, read-only hierarchy where programs can still get their config files (/etc), libraries (/usr/lib, etc), etc.
• Have a system deamon that reads the new metadata repositories and creates the legacy ones for the legacy parts of the sytem.

XML is not that bad if you are not creating it or reading by hand (but the cool part is that you can fix it by hand as a last resort). And it's scriptable as hell. Check out the Gnosis XML utilities in Python (disclaimer: I did contribute to this project). XML_Objectify for example allows you to build a full object tree of the XML repository, complete with attached methods (on a per *tag* basis) and everything. Creating a /etc file from an objectified XML one is a cinch that way.

Re:XML as a starting point perhaps?

[foobar104]

In the writer's outline section he has a few bullet-points that scream "XML!"

I spent most of last week screaming "XML!" myself.

In a lot of cases, XML can be convenient for either humans or for parsers, but not both. I'm working on a web front-end to a database-like program that outputs XML, and you would not believe the headaches that come from trying to parse poorly designed XML. And I turned myself positively crosseyed trying to read the XML output as I debugged my programs, but at least in my case once I get my parser written I won't have to deal with the XML stuff myself anymore. If we're talking about system configuration files, these are things that humans have to deal with every day. Speaking personally, that would drive me apeshit.

Basically I've formed this opinion: in situations where data will be parsed by a general purpose parser that has no built-in knowledge of that data format, XML can be a good thing if used properly.

But that's not what we're talking about here. Configuration files aren't free-form self-describing data. They're very specific, and the programs that read them will have intimate knowledge of their formats.

To see my point of view, just compare setting up an Apache instance to setting up a Jabber instance. I have the Apache httpd.conf syntax pretty well memorized-- keyword whitespace value-- so I can set up Apache very easily. But for pretty much the same job, Jabber uses a needlessly complex XML config file. Every time I want to make a change, I have to pull out the API reference!

That's just wrong.

Re:XML as a starting point perhaps?

[Graff]

I'm not saying that XML is perfect, but I do know that Apple's MacOS X uses XML files a lot to store configurations. The file type is a .plist (property list). It works out very well for MacOS X because there are specific graphical editors which hide the implementation from the user. The format is also very human-readable, so you can easily go in by hand and edit the plists if need be.

Here is an example plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList .dtd">
<plist version="0.9">
<dict>
<key>Search Node PlugIn Version</key>
<string>Search Node PlugIn Version 0.5</string>
<key>Search Policy</key>
<integer>1</integer>
</dict>
</plist>

Here is the DTD file which shows how the plists are formatted:

<!ENTITY % plistObject "(array | data | date | dict | real | integer | string | true | false )" >
<!ELEMENT plist %plistObject;>
<!ATTLIST plist version CDATA "0.9" >

<!-- Collections -->
<!ELEMENT array (%plistObject;)*>
<!ELEMENT dict (key, %plistObject;)*>
<!ELEMENT key (#PCDATA)>

<!--- Primitive types -->
<!ELEMENT string (#PCDATA)>
<!ELEMENT data (#PCDATA)> <!-- Contents interpreted as Base-64 encoded -->
<!ELEMENT date (#PCDATA)> <!-- Contents should conform to a subset of ISO 8601 (in particular, YYYY '-' MM '-' DD 'T' HH ':' MM ':' SS 'Z'. Smaller units may be omitted with a loss of precision) -->

<!-- Numerical primitives -->
<!ELEMENT true EMPTY> <!-- Boolean constant TRUE -->
<!ELEMENT false EMPTY> <!-- Boolean constant FALSE -->
<!ELEMENT real (#PCDATA)> <!-- Contents should represent a floating point number matching ("+" | "-")? d+ ("."d*)? ("E" ("+" | "-") d+)? where d is a digit 0-9. -->
<!ELEMENT integer (#PCDATA)> <!-- Contents should represent a (possibly signed) integer number in base 10 -->

Re:XML as a starting point perhaps?

[Steveftoth]

Writing scripts to do simple things with flat files is much easier, but doing complex things is very hard.

With XML you can add new fields to a /etc/passwd file without breaking the existing structure of the file. (assuming the dtd is written correctly) And the old scripts that you wrote will still work. XSLT can transform XML if you want to reformat the data easily.

Basically, XML is a pain to use if your only tool is a text editor. If you use xml based tools then it's not that bad, and much more powerful.

Re:XML as a starting point perhaps?

[iabervon]

XML is almost certainly the right solution for structured data, but is a major pain for unstructured data. If you don't have at least two levels of sections which actually need to be sections (i.e., they get repeated or items in them have different meanings than identically named items in other sections), then XML is needlessly verbose, and therefore unfriendly for humans to deal with.

I think the right choice of standards would be an outer layer of RFC822 headers (which everyone has a parser for already, and which is trivial to implement anyway), with values which may be XML documents with predefined schemas, if there are structured configuration items.

RFC822 headers have the advantage of being as compact as possible, easier to parse (although there are libraries anyway), easier to use after parsing (it's a map of keys to values, nothing complicated like a tree; compare SAX to "get(key)"), and just as standard.

It'd probably be worth putting a header on top which let you choose the character set, and it would be good to include something to say which program (or library) the file is supposed to configure, and where the help/format file is.

If keys were required to not include '=' (treated the same as ':'), and to start with a letter or number (or the line would be ignored as a comment), this would take care of many existing config files.

Re:XML as a starting point perhaps?

[ttfkam]

And don't forget feature creep. If you were to add a parameter to your config file, will it break your scripts? Very likely if they have delimited values such as found in /etc/passwd. And I challenge anyone who has worked in Linux for more than two years to tell me with a straight face that config file structures never change. Or, even worse, you don't add what you need to the config file because it will break all of the scripts and utilities out there.

Parsing XML means that you can write out all parameters verbatim that you don't care about, but add new items easily as my parent post mentioned.

Re:XML as a starting point perhaps?

[ttfkam]

Sendmail rulesets are your example of something that is already human-readable?

Bad example dude.

Re:XML as a starting point perhaps?

[Florian Weimer]

They are, but only in isolation. Getting the big picture is indeed a difficult task.

Re:XML as a starting point perhaps?

[extrasolar]

In the writer's ouline section he has a few bullet-points that scream "Lisp!" (I'm paraphrasing here):

1. A core system to handle parsing, verification, etc. -- If app configs were based around Lisp and symbolic expressions, you could you use any of the dozens of parsers/interpreters out there.
2. A configuration format description file. Of course, lisp functions that are not defined give errors.
3. OS-neutral. Lisp source code is just a text file and requires only an interpreter or sexp parser.

The nice thing about it is that you don't necessarily need a full lisp language. Even if that is desirable, the lisp derivative Scheme is quite minimal for the task.

It seems to me that you want a scalable format. By this I mean that complexity should only increase with need. Lets say the developer only needs a bunch of key/value pairs:

(define docroot "/usr/share/doc/")
(define fg-color black)
(define bg-color white)

A little macro magic could turn that into any syntax you prefer.

If the needs of the software go up, lisp can be readily used as an extension language (see emacs, gimp, xchat, etc.)

The advantage over xml is that you don't get that horrible syntax (or we all don't have to get special XML editors that are useless for anything else), you don't have to have the pains of XML compliance (unicode, doctype declarations or schemas, and well formness), and I think this is far more straight-forward than XML parsing (with all of XML's little quirks such a them elements with a backslashes in them like <blah/> and attributes and character entities (&amp; &lt; etc)).

Re:Apple + NetBSD?

[Debian Cabbit]

Yes, it's based off of FreeBSD. FreeBSD 3.2-RELEASE IIRC.

Re:Apple + NetBSD?

[nycdewd]

yeah, OS X is based on FreeBSD... and yes the person was talking about OS X in the blurb when he mentioned Apple.

and yes OS X is freakin' nice. leave it to Apple to do what no one else has/can.

flames? who cares? look at my email addy.

Ganymede?

[Futurepower(tm)]

Is Ganymede a starting point for a complete centralized configuration tool?

I'm not an OS X user but...

[Tony.Tang]

I recall some discussion earlier on /. that talked about how OS X-on-NetBSD wasn't as clean as many thought. In fact, it turned out that the /etc was pretty much bare except for some file structure that was like a Windows registry (except for it was for Mac). Can someone verify this?

Re:I'm not an OS X user but...

[Spencerian]

What you appear to be describing is NetInfo, a Windows-registry-like component that more or less functions mostly in place of certain typical UNIX config files. It's a NEXTStep legacy, but, unlike the Registry, it somewhat sidesteps an otherwise nightmarish way to handle OS configuration. And, it doesn't scare the bejeezus out of you when you look at it. The Netinfo Manager application will be used commonly by most experienced UNIX users to reactivate the root process, which Apple switches off in OS X for security/safety reasons.

OS X is based on FreeBSD, albeit an older version. It's pretty damned nice for an older kernel.

Re:I'm not an OS X user but...

[Spencerian]

One other thing--OS X has its own file directory structure--User for usr, for instance, and a few other changes. Functionally, this is FreeBSD, but Apple's directory parallels allow mere mortals to grasp its naming. Doesn't make for smooth going for experienced UNIX heads, but a trip to the Terminal shows that what they need is indeed there. There are just symlinks to the common directories, I believe.

OS X is also one of the very, very few *nix systems that is CASE-INSENSITIVE.

Re:I'm not an OS X user but...

[T-Punkt]

> i can verify that there is no connection between OS X and NetBSD

Oh, yeah. From the Darwin FAQ (OS X's core)
http://www.opensource.apple.com/projects/darwin/ fa q.html

"Q. How does Apple intend to work with other BSD groups?
A. [...] We already synchronize our code periodically with NetBSD for most of our user commands ..."

No connection, sure...

Re:I'm not an OS X user but...

[psxndc]

This is a slightly incorrect statement. Case sensitivity is dependent on the filesystem format. To keep compatability with OS 9 (for users that upgrade), OS X can be installed on a HFS+ partition. To maintain that compatability though as a consequence, it is case insensitive. OS X on a clean install presents you with an option to install as a HFS+ install or a UFS (Unix File System) install. The UFS is a case sensitive file system. I think the default install on a machine you get from Apple right now is an HFS+ version. However, if you are doing a complete reinstall or new install and want "classic" mode, I'd recommend setting up two partitions, one UFS for OS X and one HFS+ for your older apps.

psxndc

Re:I'm not an OS X user but...

[bnenning]

I'd recommend setting up two partitions, one UFS for OS X and one HFS+ for your older apps.

I wouldn't do that, unless you really need case-sensitivity for some reason. HFS+ is the "preferred" format in OS X; it has much better performance, and deals with forked files better.

Re:I'm not an OS X user but...

[bnenning]

Can you point me to a link that discusses the pros and cons?

Apple's tech note #25316 has a brief discussion. They make you sign up for an account before you can access it, but here's the relevant part:

When you install Mac OS X you can choose to optionally erase and format a hard disk or volume using the Mac OS Extended (HFS Plus) or UFS (UNIX File System) format. (In some cases you may be required to erase the selected volume.) Unless you have a specific reason to use UFS, you should use the Mac OS Extended format since it provides a more familiar experience to Macintosh users.

While I don't have a source, I recall reading a comparision that found that HFS+ was much faster than UFS. Also I've been at a Mac OS X presentation where Apple engineers stated that because HFS+ was the preferred filesystem, a lot more work goes into it than UFS.

JavaBeans solves all these problems

[p-n-wise]

JavaBeans does everything that this article wants now. It suports introspection, special field editors... Then you can build general purpose editors for any type of user. Couple this with javaspaces, for storage. JavaBeans is a shipping technology, but JavaSpaces is still a bit dodgy last time I looked (6 months ago.) Sun have spent alot of time getting this right, there is no good reason te reinvent it.

GConf

[Metiu]

As it has been already said in a comment on [fm], the answer is GConf:
it has the ability of configuring apps and has the option of using a DB or XML as a backend.
Start porting!

Other side to this

[larien]

This is something I think would help linux become more widely accepted, but there's more to it.

I was thinking this morning how I hadn't udpated my Apache server in a while, and wondering whether I should apt-get the latest version (Apache is kind of important for security, as it's the only open port on my system from the internet). However, I've done various tweaks to config files which would get overwritten if I accepted the Debian standard file, but I don't want to miss out on any new settings that could be important. I know I can do a diff, but that's effort I'd rather not have to go through.

This kind of situation is where a registry-like interface is useful; the install program just has to do a quick 'if-not-exist then add' to any new settings and leave the rest alone (or ask if you want an overwrite of all settings, with appropriate disclaimers).

This kind of thing is difficult to do in a flat file split into different sections (if there isn't a concept of sections, you can just tag the setting on), but trivial in a registry structure, especially when the tool (dpkg-upgrade/apt/rpm) has to handle all the different file formats. However, linux/Unix users would rebel en masse if the registry got inflicted on them (with damn good reason! I like being able to fix problems from single user mode using vi!), but some form of layer between the text file and settings may provide the best of both worlds (programmatic ease and editability). apt/dpkg/rpm could use the interface to add/modify settings without splatting your custom tweaks while still adding the new required settings.

Unfortunately, we're starting from a difficult point, with thousands of applications with many different requirements for their settings. Hopefully we can get some covergence over time.

Re:Other side to this

[Psiren]

If you are worried about security, why would you let *any* automated install upgrade your configuration file. Thats what the diff ability is there for, so you can see what's changed and make the necessary modifications yourself. If you haven't read up on the new changes, why they're there and what else they affect, then just putting them in is a Really Dumb Idea (TM).

Re:Other side to this

[spitzak]

The "registry-like" solution would be *many* *tiny* files. This would use the file system as the registry and you could still fix things with vi.

The main reason this is not done (by MicroSoft either) is the awful performance of the typical file system that assummes files are several K in size on average.

There are some new file systems that address this. Also a "special" file system like /proc could solve the problem.

Apple fixed it...

[Adnans]

Because there is only one captain on the ship, Apple. Good luck fixing it in the Linux world. The only way it might have a chance of working IMHO is if such a proposal gets included the Linux Standard Base. Here's a bold idea, why not copy the way Apple does it??? No need to reinvent the wheel...

-adnans

Re:Apple fixed it...

[praedor]

We're talking about the way config files are organized, designed, placed. There is NOTHING about the way Apple does it with OS X that cannot be copied in *nix. The config file system can be done in ANY arbitrary way, with config files scattered around EVERY directory on the system so long as the app that uses it knows where it is. There is no reason at all that copying the nice way OS X does it would break or be incompatible with anything on any BSD or *nix system.

Re:Apple fixed it...

[iabervon]

Because the way Apple does it is totally unlike what any existing applications use and what any present users know how to use. OS X is already mystifying anyone accustomed to UNIX and/or MacOS by being significantly different from either.

"Hey, if you learn this totally new and unmotivated system, it will be user-friendly" is not a solution unless you're MicroSoft or Apple and have users who are used to having the system changed on them regularly (but, admittedly, each time to something consistent).

Re:Apple fixed it...

[alexjp]

Speaking as a longtime UNIX (BSDi and Linux) user who's been running a production Mac OS X server for about six months, I can say that OS X is a breath of fresh air, simply for the Server Admin tool.

It's actually faster and easier to use the GUI to add, configure, and start a virtual web server than it was the old way (edit httpd.conf, apachectl restart). Some settings cannot yet be controlled using the GUI, but for those, I can easily edit the same old config file, the same way I've been maintaing Apache for years.

To be fair, Apple hasn't implemented everything in a way that's immediately intuitive (i.e. system startup script locations). And the documentation of some things (like system startup scripts) is nonexistent. But things improve with every release, and already I prefer maintaining OS X server to a linux box. (This was not the case until last fall's 10.1 release, though.) -Alex

Re:Apple fixed it...

[namespan]

It's actually faster and easier to use the GUI to add, configure, and start a virtual web server than it was the old way (edit httpd.conf, apachectl restart)

Can you elaborate? I'm still doing this the old way on OS X....

1) I add host names to netinfo
2) I add the virtual host info to httpd.conf
3) I apachectl restart

WebMin...not the Right Thing but damn good

[Spoing]

I've mentioned this before, so I'll just quote myself;

1. Get Webmin. Setup Webmin. Use Webmin. Show Webmin to all other administrators. Teach them Webmin. Eventually, when the time is right, show Webmin to management. Drive home the idea that Webmin is a one-stop-shopping, simplified, and robust server management web app that anyone can grasp (if not master). Point out that Webmin supports Unix (including OSX) but not Windows servers (except for Windows with Cygwin). Over time, point out that on the server side, the few remaining Windows servers take much longer to manage and are less reliable then the multitude managed under Webmin.
2. www.webmin.com/webmin

The complaints about Webmin is that it isn't perfect. It doesn't solve the desire for a universal config mechanism or encourage the editing of files directly. OK. It doesn't. Yet, it exists now and is a lifesaver when using multiple UNIXs. Anyone who wants to know what any Webmin module is doing probably also knows how to use find -cmin, ls -lart, or read the module itself since they tend to be in Perl.

When a universal config comes along, I'll use it and guess what -- it'll work with WebMin.

Webmin might not be the right thing, but it is good enough for a vast number of uses.

Re:WebMin...not the Right Thing but damn good

[SealBeater]

I wasn't going to get into this, but webmin is and always has been a security nightmare. Go search bugtraq for webmin bugs. Everything from insecure files in /tmp to session hijacking. Not to mention that you often find yourself in a simular trap found in package management, once you use it, you better keep on using it as webmin mangles any pre-existing conf files.

Also, ease of use often equals lack of knowledge. I am sure somebody is gonna mod me down for being a troll or flamebait, whatever. I learn the apache conf syntax, I only have to learn it once. That knowledge is portable across any systems I would be installing apache on. I would bet someone who can do it by hand can do a better job then someone who can only do it if they have webmin installed.

I also hope no one is implying that a generic gui based configurator can handle anything but the most basic and common configurations. If I (using the apache example) install and implement a little known module, mod_witch or mod_log_multicast, are you saying that webmin can handle that? Well, go in and configure it by hand you say? Well, I can't because webmin has mangled the config and if it hasn't, it still is not going to be aware of the changes.

Rant on: what is wrong with vi? Seriously? Or pico or whatever editor you want to use? What is wrong with taking 5 minutes out of your day and actually reading the sample.conf that comes with almost all software? Not only can you then go in and actually have the power to do what you want, with confidence and not having to rely on another piece of software or hope that a module exists for you to do what you want, you actually will have increased the amount of knowledge that you have and make yourself that much more valuable.

I would love to meet an administrator who could configure everything and get it working...oh sorry, cavent, said admin can only do it with webmin or something simular. Right, I can see that being a showstopper in a job interview.

SealBeater

Re:WebMin...not the Right Thing but damn good

[Spoing]

Like rants? So do I.

True: things like package management can be a problem -- any automated tool can be. It's not the same as a bare-metal hands-on I-wrote-the-code installation. Never will be.

Yet, there are people who can't or won't learn every nuance of every system file. There are people who know and like one OS or distribution or server application and hate everything else. Those people are called your co-workers. I know, I have the same co-workers.

There are other groups, call them managers, who want to know that when you get hit by a bus they are capable of finding someone who can figure out what the hell you were doing just before you got your Greyhound tatoo.

If Webmin causes you security problems, be that person who can check for those problems before deployment. Most modules are in Perl, and the permissions can be checked system-wide on any static files easily enough. Hell, create a crontab to check if you don't trust others, or even write a Webmin module to do a basic check when you're not around!

[Full-Rant Mode]

1. ...If we want a chance of getting Unix to the unwashed masses -- to enlighten them a bit, maybe to get them to do the right thing.

   ...If we want to break out of the present canabilism between different Unix systems.

   ...IF IF IF...we _NEED_ something that works like a camel's nose.

   We _need_ to get in more tents before we get any figs. Right now, most non-Unix folks don't give a fig about Unix. :)

Re:WebMin...not the Right Thing but damn good

[HalfFlat]

...If we want a chance of getting Unix to the unwashed masses -- to enlighten them a bit, maybe to get them to do the right thing.

Much better in my view is to wash them.

While there is no excuse of course for needlessly hard or complex configuration files and such, when simplification comprimises usability, security and capability it has gone too far. Rather than dumb-down the software, let's encourage people to actually learn about the tool they're using. Perhaps once people understand what security is and why it's important, they might finally start adopting security-aware proceedures.

To summarise: smarter people, not dumber tools.

Re:WebMin...not the Right Thing but damn good

[SealBeater]

...If we want a chance of getting Unix to the unwashed masses -- to enlighten them a bit, maybe to get them to do the right thing.


You think you can bring enlightenment by encouraging ignorance? By not encouraging learning and the proper methodalogy to not have to rely on yet another tool to get the same job done? That you can get anyone to do the "right" thing by encouraging them to not read and learn, just to find the "easy" way out?

Re:WebMin...not the Right Thing but damn good

[helixblue]

I'll begin to use webmin when it supports revision control hooks. Things like this you actually need when you've got multiple admins.

I hacked together revision control hooks once upon a time for some webmin features, but it was frustrating because there was no common saveConfigFile() function or anything.. each module did it individually in their own way, which meant I had to modify each module to execute cvs after it's save functions. I didn't get as far as having webmin prompt for a change description, however.

So, as webmin started progressing through versions, while we began rolling out our cvs-based revision control wrappers for /etc & such.. we abandoned webmin.

Yes, I've got the source code to change it.. do I plan on it? Not especially. Not worth my time right now.

c'est la vie.

Re:WebMin...not the Right Thing but damn good

[SealBeater]

I am going to address some other points to this.

Yet, there are people who can't or won't learn every nuance of every system file.

Then they should not be in charge of administering a box. "Can't" implies an inherent lack of ability. Good enough reason not to be an admin. "Won't" implies an inherent lack of motivation. Good enough reason not to be an admin. Both are reasonable assurances that a machine placed under their control will suffer.


There are other groups, call them managers, who want to know that when you get hit by a bus they are capable of finding someone who can figure out what the hell you were doing just before you got your Greyhound tatoo.

They can, it's called hiring someone who knows what they are doing, as opposed to hiring someone who knows how to install webmin. By your logic, it sounds like you are advocating that they should be able to hire someone who doesn't know what they are doing. Doesn't work in the real world and places that do that inevidably fail.


If Webmin causes you security problems, be that person who can check for those problems before deployment.

Right, fix the known problems, ie, make more work for yourself cleaning up after a known insecure app that is supposed to reduce your workload and make your job easier. Without even touching the problems of new hereinto, unknown security issues that you have introduced in the name of ease of use, you just added an additional layer of complexity in an effort to make things less complex. See the flaw in logic?

SealBeater

Re:WebMin...not the Right Thing but damn good

[Spoing]

You think you can bring enlightenment by encouraging ignorance?

Absolutely not. Right now -- by encouraging only one way of managing Unix systems -- we're not enligtening anyone except ourselves.

Simplify them, and then we can coax the Unix ignorant into learning a wee-bit more...a universe more then what they have now.

What you suggest is preaching to the chior. I'd like to see a larger chior, even if they can't cary a note. Later on, we can teach them how to sing. Right now, they aren't.

Re:WebMin...not the Right Thing but damn good

[Spoing]

All good points -- and I agree with them if not with the same emphasis.

Where we differ is that I desperately want more Unix-friendly work environments, and Webmin is one way to increase Unix adoptation instead of giving in to the flood of Windows systems. If you have this goal also, then tell me how you would address it. So far, I've heard lots of complaints about Webmin but no solutions beyond work harder + be smarter. OK, done. :)

There are problems with anything, Webmin included. With about 200 plugins, I'd be stunned if there weren't any security issues. A security audit, though, occurs per-app on installation and on a regular basis network wide. Those can't be avoided regaurdless of what you do. But, under most Unix systems, you have a chance to fix security issues.

Since it will not be possible to fix problems with Windows or any closed source environment/server, why not fix the problems where we can?

If it takes 2, 3, 5 years to get a unified configuration system why not start with what we can use and fix right now?

Re:WebMin...not the Right Thing but damn good

[SealBeater]

Where we differ is that I desperately want more Unix-friendly work environments

I am actually not sure where I stand on that. I would like a more unix friendly work environment in which I can either use *nix myself or have a *nix infrastucture in place. I have been lucky in that that has not been too uncommon, but from a user standpoint, I don't know. I came from an enviromnent were if someone asked if they should use *nix, the answer was no. I thought at the time that that was elitest and repungnant, but after having tried twice to teach people, I can understand. If you're interested, you'll get it, if not, you won't.

My main objection to making things easy is that often, the student, for lack of a better word, doesn't ever feel the need to progress. It works well enough so why should I learn more, is the previlant attitude. Argh, what's the use?

My main thing is, a unified configuration system is not a *nix thing, for lack of a better word. Every app out there would have to adopt it, it would be an additional overhead, and what would happen in terms of security holes, updates, etc?

If I updated the xml lib, would that render my config unusable? World-readable? What about some hard-core customization? A one-off, if you will. What about a brand new module or app? Will I have to wait for the "configurator" app to catch up? What if I update an app and not the config, and the app parses configs differently? I just think it would create more problems then it solves.

SealBeater

Re:WebMin...not the Right Thing but damn good

[Spoing]

Hmmm. I thought I posted a response earlier. My mistake.

Much better in my view is to wash them. ... To summarise: smarter people, not dumber tools.

The right thing is the ideal way to go. Unfortunately, most people aren't interested. Even those who know better, and I include in this two people I know -- a C/C++ guru (even wrote compilers and decompilers), and a theoretical chemist with a strong background in information system design and theory at the PHD-level. Both know Unix. Both like Unix. Both can use different OS environments and feel at home.

Neither likes diddling with server or operating system issues. At that level, they are unabashed users. Now, they do know the right thing to do. They even do the right thing when they manage thier own systems -- but still as little as possible. Both use GUI tools under Unix if they don't know exactly what to change or don't want to know what makes one Unix different from the next. Webmin fixes that problem.

Yet, my two friends who are capable don't care one whit and aren't taking the MCSE jobs -- they have other interests.

The majority of MCSEs (not all!) who have the "It's a job" attitude. Some care, but are just clicks and don't actually know how these systems work. Should these techs and admins know better? Sure. It is thier job, so they should care and they should learn the details but...they mostly do not. [insert your favorite reasons for this here]

What we need to do is change behavior, not attitudes. There are plenty of attitudes in both the Unix and Windows worlds. Changing the behaviors in the Windows world will not be possible if they use Windows exclusively.

So, stuck with the actual people in those positions, what are we going to do? Leave them as MSCEs and perpetuate Windows-only networks? Leave them as MCSEs who wipe out Unix as soon as they have an option to? Or, educate them and show them there is a better way? Webmin might seem like it's a step back, but it is a step forward. The behavior of the MCSEs changes, they learn Unix isn't alien or ancient. At that point, you can introduce them to VI, scripting, and other tools -- things that are available on both Unix and Windows systems.

It would be less painful to have them go cold turkey. Yet, few of us have that authority. Instead, having them slowly learn on the job is much more effective. Who knows, maybe they will get enthusiastic and start switching over some Windows servers?

Then, and only then, are you going to get smarter people.

Re:WebMin...not the Right Thing but damn good

[NutscrapeSucks]

I came from an enviromnent were if someone asked if they should use *nix, the answer was no.

I've seen that a bit too, and the main reason is usually not Unix itself, but the sort of people that you need hire to maintain Unix systems. They're generally expensive elitiest hard-to-replace assholes, and that fact has cause a certain repugnance against the entire platform.

Everyone's dancing around the core issue here, which is that a 99 conf file formats, symlinks forests, and jerryrigged scripts all basically add up to a job protection contract for the Unix Administrator, and that increases the overall cost of the system by quite a bit, and ultimately has crippled Unix's adoption in many markets.

The fact is that there's a large number of normal LAN admin tasks that don't require detailed understanding of the system. From a managerial standpoint, hiring cheap, dull people that aren't that interested in "progressing" to do that sort of work is just fine.

The Unix Guru needs to understand that rather than boobytrapping his box like the Last Mohican, that if Unix becomes signicantly cheaper to own and maintain, the installed base will explode and so will the market for his Guru services. Then instead of adding users and maintaining the print queue, he can focus on the large scale planning and architecture.

(You'll note that my rant hasn't mentioned the magic "MCSE" word yet - but look at the organization of large NT networks. One or two "gurus" that understand ActiveDirectory and other archtiectual stuff, and lots of low level guys performing the reboot duties.)

Re:WebMin...not the Right Thing but damn good

[geekoid]

if you got a tool that makes your lofe easier, don't tell managment! there going to cut staff, and give you more work.

gconf, LDAP anyone?

[DocSnyder]

"gconf" does exactly what the Freshmeat article describes: a unique way of storing configuration. Its frontend is abstracted from the backend which uses a hierarchy of small plain XML files per default but could also use a SQL database or a LDAP directory.

LDAP is nice on its own, too, with configuration being stored platform- and host-independently, as well as global, group-specific and user-specific settings. Netscape Roaming works this way.

Re:gconf, LDAP anyone?

[FooBarWidget]

Damn right. That explains why when I click on the Apply button in Galeon's preferences dialog, and I kill the Galeon process and restart it again, the configuration is still there.
As you can see, GConf is totally broken, because it saves the configuration when it's supposed to fuck up your configuration files.

Oh wait, that's only the behavior that the anit-GConf trolls invented.

Duh

[autopr0n]

Who hasn't though this? Unix/Linux config is a mess. Diffrent files in diffrent folders with diffrent formats. Sure it's all plain text, but who has time to learn diffretn config formats/languages for each program they want to use?

As for hard to do? I don't see why that would be the case. Just a little bit of XML and viola.

There are a lot of things in the Unix world that are hard or time consuming to do just because they can be. sendmail.cf. hello!?

I hope this goes somewhere

[Twylite]

A brilliant article, and a subject which I have tried before (with little success) to broach with various OpenSource developers. Unfortunately the comments demonstrate the lack of understanding that is all too common in the free software world: an inability to distinguish between apis/formats and software implementations; and an unwillingness to move to a more user friendly system.

GConf and Webmin, for example, are not solutions that address the problem posed by the article. Other comments completely lose the plot: this is about a common configuration format which can use metadata to provide for configuring just about any application, it is not about creating an all-encompassing configuration that will give settings to all applications everywhere for all time.

Gconf is half of the solution at best. It provides a library (common abstraction) for accessing the data. But an application still has to have explicit knowledge of the data in order to act on it. A configuration framework describes this knowledge to allow any compliant application to act on it.

Sometimes an example is worth 1000 pictures (and hence about 1000000 words), so consider a simple packet filtering application with an ACCEPT list and a DENY list, which can listen on several interfaces.

Such an application would have a meta-data configuration file (say /etc/conf/meta/filter.cfg) which specifies: section name=interface count=multiple type=ipv4 ; section name=accept count=multiple type=cidr ; section name=dent count=multiple type=cidr. Now any application can access the packet filter's meta-data file, and the real configuration file (say /etc/conf/filter.cfg), and understand the meaning, to a degree, of the configuration. The types "ipv4" and "cidr" will have to be primitives for the configuration framework.

Thus a general configuration application could present a tree which has three branches (interface, accept, deny), each of which displays a screen with a list. Appropriate entries can then be added, modified or removed, reconfiguring the packet filter without the configuration application having any prior knowledge of the filter's configuration or requirements.

With adequate forethough (for the categorisation and types required) such a system can be extremely powerful. A single configuration library can be used by the application, as well as GUI and CLI configuration clients. You could easily have a command in a script: config pktfltr add -section interfaces -value 1.2.3.4 .

A common library like this can also provide powerful features that a lot of applications lack, like local (user) settings which override global defaults. The library can maintain global, group and user configurations for each application, and getting/setting preferences will take the overlays into account.

Implementing such a system would also provide an ideal opportunity to introduce configuration versioning. CVS (for example, or arch, which includes directory versioning) could be used to maintain various versions of the configuration files in /etc/conf.

Arguments about diversity are largely moot. The data format is independant of the data in any well-written program. .INI files, Bind-style files, XML files and SAMBA files can all represent the same information, but they look different. Some are arguably more efficient in some situations. Even the feared Windows registry has the same functionality ... it just makes the (fatal) error of storing everything in a mostly inseparable datastore (sortof like an /etc/conf directory in which you can't treat the various files separately).

This is a suggestion which needs to be followed through, and supported by the community.

Re:I hope this goes somewhere

[sillysally]

nicely said.

I would deviate slightly. folks should stop thinking of "the solution" as being "a solution" or "it" or some new unified thing. Instead, think of it as "what incremental things can we do that would make the mess less of a mess". If we then implement a few of those things, the world will be a better place, even if we do no more than that. If we keep getting incrementally better, someday we'll arrive at a unified world.

so, for instance: we need a little API that allows reading and writing of hierarchical data, and an implementation of it that supports one file format. Use it in a project. Implement a second file format and now that project could write its data in either format. Approach other projects that use the same file formats and offer to switch them.

Note that the two different file formats represent polymorphism: but this should not dictate C++ (or whatever). Rather, there should be implementations of this small "project" in every common language, at least perl, C, C++ and "shell" if anybody has the 'nads.

I hope I made my point? each little step of this is one incremental thing and is good on its own.

I've got some other issues, but I don't want a long muddy post. But, f'rinstance, the .* files do not belong in my login directory, or I shouldn't log into that directory, and .* files that I hand edit should not be interspersed with .* files that are filled with computer gen'ed stuff. the ~ (home) directory is a huge stinky mess quite apart from the different file formats. Copying your .dot files to a new login dir is a nightmare. However, notice that with a new API being used to read and write them, the new API could silently move them whenever you're ready.

Re:I hope this goes somewhere

[Zapman]

You said:

Such an application would have a meta-data configuration file (say /etc/conf/meta/filter.cfg) which specifies: section name=interface count=multiple type=ipv4 ; section name=accept count=multiple type=cidr ; section name=dent count=multiple type=cidr . Now any application can access the packet filter's meta-data file, and the real configuration file (say /etc/conf/filter.cfg), and understand the meaning, to a degree, of the configuration. The types "ipv4" and "cidr" will have to be primitives for the configuration framework.

and killed yourself. Making a configuration file system that can:

1) have enough 'primitives' to control network interfaces (cider and ipv4), gui's (resolution, dpi, mouse type, mousewheel), terminal types (delete char, color codes, color depth, escape codes, control flow), kernel config (pc-parallel port, IRDA-system), and ANYTHING ELSE THAT COMES ALONG will have even more complexity than the system already in place. You will need a lot of a priori knowledge to edit the file. You must know all the relevant primitives.

2) How on earth will you lay out the filesystem for this? Your example of /etc/conf/meta/filter.cfg you said would apply to SOME interfaces. So the control must go into that file for WHICH interfaces. This meta directory will end up with thousands upon thousands of files. How will you lay out network interfaces? and terminals? and gui's? (this is far easier to crack than the first though)

3) You also destroy backwards compatability. You could write 1 app per app to 'translate' the configs, but that's a huge chicken and egg problem. You also break the origional app, which has to be re-written to conform to this new, all singing, all dancing beast.

It's nice to proclaim 'unified config file format'. But the reality is that it would make life even harder than it is now. You end up just moving the complexities around.

Re:I hope this goes somewhere

[jasno]

Well put.

I'd like to see something like this, but managed but a 'config demon' which has access controls and allows sharing a centralized config with other hosts over the network (ok, you could do it by mounting the config dir via NFS or Samba...).

The demon would provide the API/abstraction and would store each APP's config file in a human readable format. Then supply X and ncurses front ends so those without X aren't stuck editing some scary XML file.

So besides just talking about it, is anyone interested in doing it? I'd be interested in getting involved in a project. I've been tinkering with LFS so I'd have a blank slate to experiment with this very problem.

Now if we could find a way to fix the init scripts while we're at it...

Re:I hope this goes somewhere

[Kiwi]

You will need a lot of a priori knowledge to edit the file. You must know all the relevant primitives.

No, the configuration program itself does not need to know anything about the data each program uses. Just as web browsers do not understand the significance of data in form fields, our configuration program will not need to know anything about the actual significance of data in the configuration file.

How on earth will you lay out the filesystem for this?

Simple. Have a directory, /etc/conf/meta, and put all configuration data there. In fact, we don't even need a meta configuration file if each file in this directory has a summary field, we just go through /etc/conf/meta by hand and load up all the summary fields.

You also destroy backwards compatability

First of all, this is not true; programs can accept files from both legacy formats and from the new XML format in the transition period, or we can write programs which read in the legacy file using the legacy parser and output the same data in XML format.

That said, expressions like this ("You also destroy backwards compatibility") are expression I have heard over and over again, especially in the world of UNIX users.

This is a sentiment which I have seen time and time again, and a sentiment that makes Microsoft (or whoever else belives in proprietary OSes) thrive in the server space.

I have seen this same sentiment with people who complain that Red Hat is horrible because they use xinetd, which, heaven forbid, uses a different file format than /etc/inetd.conf.

This is the same sentiment that the people who make the DNS standards have, dragging their feet and not accepting any DNS extension which has non-ascii characters, forcing Yahoo! to have the host name "espanol.greetings.yahoo.com" instead of "español.greetings.yahoo.com". (Se puede decir que estas personas necesitan un nuevo ano porque ellas no pueden aceptar que esto es un nuevo año).

This is the same argument used by people who do not like CML2 (ESR's proposed kernel configuration utility) because it uses Python; they want everything to be done with shell scripts and the C programming language, and can not see the limitations that those languages have.

This is the same sentiment that makes it so I have to go a step backwards in terms of technological advancment to email UNIX-using people than mailing my friends, since many UNIX-using people are still using mail systems from the early 90s before HTML mail existed.

It is this fear of change which holds back free software, or, if you prefer, software libre. The computing industry, more than any other industry, is an industry that is undergoing constant change. Anyone who can not see this is going to be left behind.

- Sam

Re:I hope this goes somewhere

[renehollan]

You've obviously put a good deal of thought into this, but there are some "gotchas" that need to be addressed in more detail:

Distinguish between metadata syntax (of which there should be one and only one) and data syntax (of which we have several different types).

The first describes what kind of data is in a particular configuration set, rather like an XML DTD (except the configuration set syntax need not be XML). Note that one can think of the configuration data itself as virtually stored in XML, consistent with some DTD, even if it is not: one can imagine a transformational grammer for converting between XML and "legacy" versions.

A generic configuration editor would require two things: a DTD description of the XML version, and a transformational grammer description of the translation between the XML and legacy versions. The transformational grammer could be one-way or two-way depending on how the configuration data was read, IOW you have an input format and an output format.

The quick and dirty approach then, is to edit an XML version of one's data, and export, via a transformational grammer to legacy format files. Of course, this is frought with the usual problem of being incompatible with manual editing of the legacy format file. Add a reverse transformational grammer and you can now read XML and legacy versions of the configuration file -- so this desirable property can be added incrementally, if necessary.

Sharing configuration data between different applications requires them only to agree on the DTD describing the data, and the transformational grammers used to read the configuration files (or, for legacy apps, the implied DTD and transformational grammers). It is reasonable to assume that existing applications that share configuration data do so by virtue of each application understanding the legacy file format. New applications could be oblivious to this format, if a DTD and transformational grammer were available. So, when the legacy file format is abandoned, new applications do not have to change: just the transformational grammer to use changes.

For simplicity's sake, one should settle on a single configuration file for each configuration data set -- this may be initally driven by the need for legacy application compatibility. One will also have to settle on a single DTD describing this data, or at least compatible DTDs. Otherwise, the problem has just been moved from having different configuration file formats to different metadata descriptions.

Re:I hope this goes somewhere

[Zapman]

First off, notice that I listed the 'destroy backwards compatability' last. It's the least important of my arguments. However, your 'transition period' requires a LOT of work on the maintainer's part.

Second, (and most important) I still claim that you're going to just move the complexity around, and not make anything really 'simpler'. You'll unify all the config files, but make it harder to find where you want to change something. You still have to fingure out what 'form data' (style and keywords) the app is willing and able to take. Which still involves looking up obscure syntax in the man pages. So no win comes of it.

All you've really done is unify what character is a comment.

Re:I hope this goes somewhere

[Twylite]

At the risk of /.'ing my ISP ... take a look at a doc on my home page which goes into a whole lot more depth than my Slashdot post.

I tried to get an effort like this off the ground about 4 years ago, with little success. I'm still interesting in playing with it though.

no, and furthermore...

[nycdewd]

"Ernest Prabhakar of Apple Computer informed attendees at the USENIX BSD Conference earlier this week that BSD (upon which Mac OS X is based) is now "three times more popular" than Linux on desktop computers."

http://www.osopinion.com/perl/story/16382.html

Uh...

[autopr0n]

That link isn't particularly helpfull. I hope you don't expect me the dig around on that site for hours to find the answer.

Perhaps you could give me a more precice link.

The only thing I could find was: But good looks are only the beginning. At the foundation of Mac OS X lies an industrial-strength, UNIX-based core operating system--called Darwin--that delivers unprecedented stability and performance. Darwin provides Mac OS X with powerful, advanced features such as protected memory, preemptive multitasking, advanced memory management, and...

Etc. Didn't say anything about what it was 'based' on.

Re:Uh...

[dark_panda]

From http://www.opensource.apple.com/projects/darwin/:

What is Darwin? Darwin is the core of Mac OS X. The Darwin kernel is based on FreeBSD and Mach 3.0 technologies and provides protected memory and pre-emptive multitasking. Darwin runs on PowerPC-based Macintosh computers and a version is also available for x86-compatible computers.

J

Re:Apple + NetBSD?

[BiggyP]

i too was under the impression that it was based upon FreeBSD and Mach rather than netBSD, quite consideribly different, to start with they even tried to poach Linus and get him writing their kernel, but he refused so they went with the FreeBSD project leader instead, [sarcasm]and haven't they been so helpfull to opensource as a whole with OSX and Darwin.....?[/sarcasm]

I /like/ the Unix Configuration Nightmare

[Combuchan]

Why fix it? It's good right where it is.

I /like/ having daemons and programs having their own configuration file. I like having to be able to manually configure something and fix it myself, without having to fight with some meddling front-end. I like learning how configuration is done with each program. I like applying existing text editing and stream tools to work with configuration. This is the way it should be done, and this is why I like Unix far more than other OS's that aren't predominantly CLI-based.

Don't fix it. It's not broken. Things work great as they are now. It's not difficult if you want to learn how to do it. Your "Unix Configuration Nightmare" is actually an elegant symphony of applications, clients, daemons, sockets, and streams that is UNIX. Take it or leave it.
:)

And you know, when I was your age... wait, you get my point.

Re:I /like/ the Unix Configuration Nightmare

[osolemirnix]

I beg to differ. While I like Unix, I would not call this "great":

ls -a $HOME
.Xdefaults .Xmodmap .Xresources .addressbook .addressbook.lu .bash_history .bashrc .dayplan .dayplan.priv .dvipsrc .emacs .exrc .gimprc .grok .holiday .hotjava .jazz .kde .kermrc .lyxrc .muttrc .nc_keys .pgp .pinerc .profile .seyon .signature .ssh .stonxrc .susephone .tex .uitrc.console .uitrc.vt100 .uitrc.vt102 .uitrc.xterm .urlview .vimrc .xcoralrc .xfm .xim .xinitrc .xserverrc.secure .xsession .xtalkrc
You get the idea...

Sure, wget has a command line option for my http proxy. But why isn't there a system-wide default that it uses?
Wait there is! But is it $HTTP-PROXY? Or $HTTPPROXY? Or $PROXY?
What's my default printers name? Yeah right it's probably $PRINTER, but then again, maybe not.

Sorry man, but I don't call that good. I call it ugly. Using an Apple, now that is pretty, and for a reason. Thinking about merging pretty (Apple) and powerful (Unix) sounds truly appealing to me.
The article was talking about user configuration as much as server configuration. I could live with seperate conf files for servers, but for user conf files it's just plain ugly.

Re:I /like/ the Unix Configuration Nightmare

[Shanep]

that is UNIX. Take it or leave it.

I agree. I'm reading all this wondering why people are complaining so much. How much time do people spend in /etc anyway?

I need to get something set up, I go to /etc and either see a file or directory named something along the lines of what I'm configuring. Get it up and if I don't know it well, either quickly read the comments or the blah.conf man page... configured... working... I'm done in here for a while until I think of some other cool thing to do.

When Win95 went to a single, non-text config file (registry), I instantly thought this was going to be a really bad move. And it proved to be. Registry rot is incredible. Anyone ever notice that if you reinstall Windows about once every year or so, you get a massive boost in performance? I installed 98 the other day (just for D2:LOD, Starcraft and RainbowSix), it boots (PII-300, 512MB) in about 30 seconds and shuts down in 1-2 seconds. Prior to this, 98 was taking an age, *if* it would shut down at all, and I had all sorts of stability problems. BTW, I put the C: on a 700MB partition and used a D: of 2.5GB for the games. I burn the C: to a CDR and when things get bad, just dd it straight back to hda1.

My point is (back to the OSes that are more fun than a Frost Nova up Andariels arse), *I* want to mess with those config files, rather than have some app(s) messing with one really important file.

I like it too.

Re:I /like/ the Unix Configuration Nightmare

[SealBeater]

Sure, wget has a command line option for my http proxy. But why isn't there a system-wide default that it uses?
Wait there is! But is it $HTTP-PROXY? Or $HTTPPROXY? Or $PROXY?


Actually it's $HTTP_PROXY. Its called reading the documentation, which is why its there.

As for things like .bash_history, .bashrc, .ssh, .signature, etc

its called multi-user enviroment. Keeping individual settings, information and the like in $HOME is actually pretty elegant. How do you propose keeping seperate users settings intact? If its one system-wide configuration, obviously everyone is stuck with root's preferences, a bad idea I am sure everyone can agree.

SealBeater

Re:I /like/ the Unix Configuration Nightmare

[osolemirnix]

Read my comment carefully.

This is not about a system wide conf file for all users. It may be about a single conf file for each user (in their respective $HOME). Of course that would make perfect sense.

And it's not about how the environment variable is called either. Of course I can look that up in the program manual. It's about the fact that the variable is not standardized across different programs (which I was trying to illustrate by giving it different names). So one program may call it $HTTP_PROXY, while another may call it something else. And one program may expect it's value to be a URL, while another may want to cover more info, e.g. avoid the proxy for certain domains.

The point is that such config info is not standardized across several different apps, so you may end up changing several different config files for one setting. That's the problem.

Re:I /like/ the Unix Configuration Nightmare

[SealBeater]

The sole purpose of the dot file system seems to be to not confuse the configuration files with the user files. This could simply be achieved by using a ~/config/ directory. It would make things clear for everyone at first sight.

So, what does that achieve? You still have all the files in your home dir, only now you have them in a dir in $HOME, instead of in $HOME. I thought the whole point was to make things simpler. Everyone who uses *nix knows that a dotfile is a user config file. Their hidden so you don't have to see them all the time, if you want to, alias ls -a to ls. My only complaint with dotfiles is that there isn't a central information site that has what all the dotfiles are and what they do. The fact remains however, that your solution does nothing, as far as I can see. All you have done is move normally hidden files into an unhidden dir. Half of the dotfiles you have to actually put there, like .forward and .signature

SealBeater

Re:I /like/ the Unix Configuration Nightmare

[SealBeater]

Sorry I have you marked as a foe, cause of the page widening, so just now saw your post.


This is not about a system wide conf file for all users. It may be about a single conf file for each user (in their respective $HOME). Of course that would make perfect sense.


Ok, suppose I take a vacation. Instead of touching .vacation, I have to open up a config file (by what method is left up to the reader) and uncomment the vacation section. How about .signature? At home, I have a perl script that changes .signature every 5 minutes by recreating a symlink to a dir that contains various sigs. How is that supposed to work with your system?

Same issue, slrn. I have multiple .slrnrcs for diffrent NNTP providers. I guess I'll just have to add them all to the flat file. My .muttrc, nicely customized, is part of this one file, I guess. So, if I install it on another box, instead of just copying over the .muttrc, I have to what? cut and paste the revelant section to the flat file on anther box? And how does the single conf file know what settings, etc to have? Does it get appended to the conf file on installation by root? For every user? Only users with $HOME dirs? Do I need to say why this would be an incredibly bad thing?

And suppose there is a new whiz bang app that comes out. What do you have to upgrade to get the settings into the single config file. Would it mudge the settings? Change the format of the file to take advantage of newer XML or whatever features? Would that break existing apps so you have to upgrade everything? All except that one app that hasn't upgraded, so your configs are broken?

There is such a thing as single point of failure...this is one of them.


Of course I can look that up in the program manual. It's about the fact that the variable is not standardized across different programs (which I was trying to illustrate by giving it different names).

I would be very interested to hear what program that parses $ENV_VAR for proxies (to use your example) doesn't use $HTTP_PROXY. w3m does. mpg123 does. lynx does. wget does. w3mir does. Please find one that uses $PROXY or some subset for me.


The point is that such config info is not standardized across several different apps, so you may end up changing several different config files for one setting. That's the problem.


As far as I know, most programs check for a config in $HOME, then /etc. Please give examples of what settings or apps you are refering to.

And you're right, I didn't fully understand what you meant. Sorry.

SealBeater

Re:I /like/ the Unix Configuration Nightmare

[Cardhore]

Lets say I want to write a program for KDE, to configure my system, so that people who don't want to learn all the different styles of a comment in a configuration file can still configure their computers. Do I want to write a new module for every program that has a configuration file, and then test each module with each configuraiton file and all of its permutations? No. Do I want to release a new version of my tool every time a new software package comes out? No. This is what the current crop of configurators do. They have code for each type of file. That is also why they are broken. What we are wanting is two have a standard configuration format, and a standard DTD for that file, so that we can write a standard configure program that works with past and future programs.

Re:I /like/ the Unix Configuration Nightmare

[bnenning]

*I* want to mess with those config files, rather than have some app(s) messing with one really important file.

You're right about the registry, but a better designed system can give you the best of both worlds. Mac OS X's plist files are human readable and writable, easily accessible via a standard API (1 line of code to get or set any property), and do not involve a single point of failure.

Re:I /like/ the Unix Configuration Nightmare

[Shanep]

I see the merit in leaving traditional Unix the way it is, but I also see the merit in laying down foundations properly from the start, OS X style. It would be nice to see Linux, BSD's etc adhere to some standards for individual config files though, but I wonder if everyone can agree, in a pseudo hierachy where heads are rarely on any chopping blocks and more often than not motivation is driven by individual ego when it comes to policies like this?

No doubt we'll get a bunch of different standards, most will die pretty quickly, leaving 2 to slog it out with different distros choosing opposing standards, just to make it harder for us. ; )

I love Linux, Free and OpenBSD, but I personally can't wait to get my hands on OS X. I think the people at my local Apple shop are getting sick of me hanging around. ; )

Re:I /like/ the Unix Configuration Nightmare

[SealBeater]

No, it's $http_proxy. Yeesh.


Actually, you're wrong. Enviromental Variables are always expressed as all upper-case.

SealBeater

Install/configure

[autopr0n]

Sure, installing software is easy. Unpack and compile. We are talking about configuring software.

Like getting sendmail to rout and forward mail, apache to serve up web pages you want, BIND to bind names to IP address, etc.

These tasks arn't "hard" or anything, but they do require a lot of reading on the part of a Newbie. In the windows world (don't know much about OSX) Most of that stuff can be done via am intuitive GUI.

Flame me if you want, but I'd greatly prefer a system that didn't require me to learn diffrent config file formats for each service I want to have running... or deal with a hodgepoge of 'easy config' program hacks.

A simple, standard configuration system is definetly the way to go.

Re:Install/configure

[lessthan0]

There is no simple, standard way to configure complex things. A nice GUI works fine if all you want to do is something simple, but when things get hard, you have to dig into the guts and really understand what is happening.

Now, using a standard syntax may hold some promise of making configuration a little easier, but the hard things will always be hard. No GUI will ever change that.

Also, text files are easier to manage remotely. Just give be ssh and vi and get out of my way :)

Re:Install/configure

[walt-sjc]

This is a defeatist attitude. Did you even read the article? If MS or Apple can do it, we can do it, and we can do it better.

Now a GUI won't make people understand what they are doing or do the work for them (ever configure MS Exchange? That's a bitch....) but what it DOES do is make the configuration easier to navigate.

By using an XML based file format (as I proposed in a previous comment) you get the best of both worlds. The Apache config file is a close example of what I was thinking (it's not XML, but it is XML "like".)

An application would actually have 2 configuration files: one that actually contains the configuration data (like httpd.conf for apache) and the other to describe the options, help for the options, defaults, etc. to the configurator GUI. Both would be XML, would be extensible as hell, and provide a single tool for configuring any application, current or future.

As I mention in a previous post, a set of importers / exporters would provide a way to migrate until the various applications were updated to use the new format (sendmail may NEVER change...)

So you get what you want. Hell it would probably be better as you would no longer need to memorize all the differet quirks / rules for different applications's configuration language.

Re:Install/configure

[Cardhore]

You are exactly right. Have one or two files for the actual configuration (/etc/app.conf and $home/.app.conf), and one file that describes the configuration: its options, parameters, and help strings (perhaps in /lib/etc?). XML would probably be the buzzword to use here.

Think about it--with this method, you only need to write ONE gui configuration tool, for everything!

Re:Install/configure

[walt-sjc]

Configuring Win2000 as a file / print server that authenticates against a domain controller (or IS a domain controller) is TRIVIAL. Configuring a Linux box to do the same thing is a PAIN. This is the MOST common use for corporate servers.

You don't need regedit to do configure this unless you are REALLY into self-inflicted pain, and decide not to use the other tools. Regedit would be the same as vi in my example. It's a way to manually edit data. I am NOT describing regedit. You are thinking that my tool would be an XML browser that would basically manualy edit the first file I described. The second file is the key. It's the one that gives you all the smarts in the config tool such as whether to do radio buttons, check boxes, tabs, text entry, value validation, online help, etc. This is alot of the stuff that's hard-coded into so many different tools.

It's also the one that would define where in the scope of configuration of everything a particular task belongs. Obviously the schema, taxonomy, etc. need to be thought out, but this is a VERY doable project.

Re:Install/configure

[Max von H.]

Administing servers and systems is hard, no GUI will take that part away.

Exactly! Each time some admin tool gets a GUI, it's a reason for your boss to downgrade you since for him GUI=easy, hence putting your salary upgrade in jeopardy since basically anyone can use a freakin' GUI.

Hards things must look tough, for else they lose all credibility :)

Just my twisted thoughts.

/max

Re:Install/configure

[k8to]

By using an XML based file format (as I proposed in a previous comment) you get the best of both worlds. The Apache config file is a close example of what I was thinking (it's not XML, but it is XML "like".)

Yes, you get the best of both worlds. First, it requires more software machinery to process it, but at least it's from a standard library (reliance on which damages portability across unices). Secondly, it's at least nearly human readable, unlike our current textfiles which are entirely human readable.

So XML provides two benefits: more brittle and dependent and bloated software; and harder to edit, read, and manage files.

hardware != software.

[autopr0n]

Hardware, for the most part, can be automaticaly configured. In fact, I've never once had an issue with hardware configuration in linux (except for tonight, ironicaly. cdrecord can't find the burner I put back in the system, even though it worked before)

The problem is software config once you've already got the machine setup. Programs generaly have diffrent formats for their configuration. even windows 3.1 was better then this with their .INI files. geez.

Could you be more specific?

[wirefarm]

Your post was a bit vague on the name of the actual product you are endorsing...

;^)

(I just counted the word 'Webmin' like 17 times in your post...)

Actually, I couldn't agree more.
A couple of weeks ago, my boss called me 2 hours before I was supposed to be at work to tell me that something needed to be fixed on the server. I literally rolled over, grabbed my laptop, logged in using the ssl option, fixed it and went back to sleep.

Now you might say that all I really needed was SSH and VI, but I hadn't had any coffee yet and Webmin just required a bit of clicking. It rarely screws up and doesn't mess with your config files.

I wrote a bit more on it on my web site a while back - scroll down about half way.

Cheers,
Jim in Tokyo

Re:Could you be more specific?

[Spoing]

(I just counted the word 'Webmin' like 17 times in your post...)

Well, I went to both the Creation Science and L.Ron Hubbard schools of logic -- both known for scientifically proving a point by repeating it as many times as possible no matter how correct. Unfortunately, I feel as if I've shamed them since I did include some information that could be verified. Facts are such pesky things. :)

Needs Meta configuration file and config server

[Anonymous Coward]

I've been thinking about this for a while and definately think that there needs to be various elements to a Unix configuration system:

1) The concept of local and group configuration. There needs to be a Meta config file that specifies what individual users can and can't configure. So for instance, if a company wanted to allow users to configure their word processors however they wanted, exect that they couldn't make any changes to the dropdown menu of corporate document templates, that would be specified in the Meta file.

2) A configuration server. I see no reason why this couldn't work with http to deliver XML config files to individual workstations. The Meta files could reside on the server, the local files on the hard disc. You could imagine in this senario that in order to configure your machine all you would have to do is enter the appropriate URL and press go:

http://intranet.company.com/configs/standard.xml

Then you could have different configs for different types of staff, and supported and unsupported configs (from the point of view of IT support).

You could also perhaps have 'approved' config files for supported hardware.

Having this type of Meta config file also allow IT support to have a clear idea of all the hardware used.

I know that probably lots of people on Slashdot don't like the idea of IT support having control over the config of their machines, but big companies like it and if Linux is going to conquer the desktop then this is the type of tool that will help get it there.

Psychological, not technical holdup !

[ehack]

It's the revenge of the geeks. A mindset, not a technical issue. If you cannot figure out how to configure the OS the geeks wrote, you do not deserve to be using it.

In an interview of an IBM director, I asked why their then current OS had no GUI (when Windows was chipping at the userbase). Reply:

We do not need mice - We are not children !"

Mindsets change - at the speed at which glass flows :)

Some Stupid Questions

[heideggier]

Would it be possible to get rid of dotfiles? While alot of people like them I don't know why you couldn't have a ~/etc in the home dir. I get lot's of weird looks when I say, you keep all the configurations in /etc but user config's in dotfiles. There's must be a good reason for this could someone please explain

On that thread what the hell is /usr/etc used for, or /usr/local/etc? won't it make more sense to move /etc here instead?

Do we really need a central registry type file like the article proposes, surly isn't a front end enough, I can't understand what difference this would make. Having a central file which contain's everything like System.dat seem's to be a major security floor in windows since if you can crack that file you pretty much have root on the machine. Not being able to set different permissions for something like fstab or shadow would be pain. Although to be fair the article proposes it as a backup. If it ain't broke don't fix it.

Apart from these questions I think that this was a pretty good/fair article, although I thought that it was a bit unfair to compare linux to windows/OSX since they are completely different OS and something like creating a central config app might work to make linux less customisable IMHO. Most distro's tend to have app's along these lines anyway, what the hells wrong with DrakConf?.

Re:Some Stupid Questions

[Twylite]

These is no technical requirement for dot files. If the entire community decided to standardise on ~/etc, it would happen.

The general theory of /etc, /usr/etc and /usr/local/etc is quite simple. System configuration goes into all of these directories (no user config). Configuration needed at bootup goes into /etc. Configuration needed for OS-default applications goes in /usr/etc, which is available once /usr (containing the OS-default applications) is mounted. Configuration for application you have installed specific to your system (stuff that should be in /usr/local) goes into /usr/local/etc.

The article is not proposing a central configuration file, but a standard for configuration. That would be like saying all application MUST store their configurations in a specific directory, and they must all use the same file format (say XML). Storing all configuration files in the same directory is not the same as storing all configuration in one file.

The benefit of a common location and format is that every app or admin knows where to look for files, and how to interpret them. The structure of each file is identical, so you don't need to know if you use #, //, ' or $ for comments, and whether each line must start with a TAB or end with a ; . What each file says is still different, its just how it says it. Currently trying to work with config files on Linux is like being in a building and trying to talk to people, but everyone is in a different room and talking a different language. Standardize the language and the location, and you only have to content with understanding what is being said.

Incidently, system.dat is not a security problem. In Unix terms that file is r/w by root only, and access only occurs through a kernel-level API. So "cracking" system.dat is no more or less hard than "cracking" shadow.passwd .

Re:Some Stupid Questions

[hacker]

On that thread what the hell is /usr/etc used for, or /usr/local/etc? won't it make more sense to move /etc here instead?

/usr/local/etc exists because you screwed up when you configured the package, by not properly adding the autoconf --sysconfdir=/etc flag, which is assumed if you also use the --prefix=/usr flag.

Re:Some Stupid Questions

[nehril]

the only way linux would get to such a state would be for a distro vendor to bite the bullet and rewrite ALL the included apps/utilities to use the new format.

that means little stuff and big stuff, like crontab -> xml format, smb.comf -> xml format and .GnuXDesktopWidgetFoo -> xml format.

who will step up to the plate? the design is the easy part.

Re:Some Stupid Questions

[Junta]

Though far from alleviating your troubles with the many many many many applications that like to put .files and .directories right in your home directory, I thought you might find this interesting.

The LOX Desktop (http://rox.sourceforge.net/) is a litttle project to bring an Risc-OS like environment to X desktops. Mostly a file manager (but a bit more), but one huge thing about it is the recommeded AppDir packaging. Basically, every package is almost entirely self-contained, except for preferences. For system-wide prefs, something like '/usr/local/Choices/' is used and for personal, ~/Choices/. That way all config info is in a separate location and it is *very* easy to uninstall packages. They provide ROX-Lib for Python writers to *very* easily use this structure. Although it diverges from the path significantly, and provides a Python-Centric API, it has some really good ideas. I know a lot of little utilities exist including a media player (PythonTheater, http://xtheater.sourceforge.net/) that obey this little standard.

Re:Some Stupid Questions

[greed]

OS X, for instance, gets rid of the dotfile dependency [...] with NetInfo, a database for handling this stuff.

NetInfo sits in places of the same sort of things that NIS, NIS+ and DCE registries can replace: passwd, aliases, exports, fstab, hosts, group, printcap, and stuff like that.

Configuration files are stored in ~/Libraries/Preferences. Apple recommends the use of XML, but there are many preferences files that are in more traditional MacOS resource format. And then there's Mozilla....

So, yes, technically your preferences don't go in dotfiles. And it is nice that they're all in one directory which has an obvious purpose. But each application can still choose its own file format.

Re:Some Stupid Questions

[praedor]

Well, to /etc, /usr/etc, and /usr/local/etc, we can also add such as /usr/share/config, /usr/local/share/config. There is all sorts of config files there as well. I would think it nice if it were narrowed down a bit: core system/os/daemon configs independent of wm or environment go into /etc (or better, rename it to /config so it is self-explanatory) and all other configs go into /usr/share/config or /usr/config. For personal settings, you have
~/config.

In the end, I don't care HOW it is organized but that it be standardized and streamlined the same way on ALL distros. I don't like having to do "find" or "locate", I want to know where I will find something upon install, be it Suse, Mandrake, Redhat, Debian, etc, etc.

HAH!!

[autopr0n]

You expect the open source community to use Java!? It's plain C all the way! (just look at Genome... hell, there are some guys (dotGNU IIRC) who want to implement .net in plain C!)

Ok, so that's not entirely true. Some OSS people like java (Freenet, Apache Jakarta). But I seriously doubt that anything Java based would ever make much inroads in the free Unix world. (Maybe Solaris could make some headway, though :P).

And there's a few good reasons why it shouldn't. Java is a little top-heavy, and it's not platform-neutral, it's its own platform... and connecting it to other technology via the JNI is a pain in the ass. I tried JavaBeans once, on a windows box at work. It was a nightmare. I ended up using TCP/IP for interprocess communications. (then again, TCP/IP might not be a bad choice for a central config server, you could do things like remote config. hrm)

Anyway, I think XML plus some schemas might go a long way to doing what we want, without requiring everyone to link in a JVM.

It seems you're missing the point

[Nailer]

The problem isn't with the lack of a standardized configuration tools, its a lack of standardized configuration file formats. This allows:

• Systems administrators to work with new applications config files without worrying about how exactly a comment should look
  
• Automated parsers that don't have to be re-written for every config file - you'd have a whole buynch more webmin modules if this was the case.
  
• GUI folk to be able to use their graphical interfaces knowing that they won't screw up a config file, because there's a very rigidly defined format for the config file and a standard parser that's very well tested
  
• Better manipulation, imports, exports, etc of information into config files, as a popular format allows for the development of tools to perform these tasks across many different applications
  

Re:It seems you're missing the point

[Spoing]

All that is important. Yet, for now Webmin really does do a good enough job.

Even if these changes were added in right now, it would still take a few years for them to be so common that they could be relied upon.

Re:It seems you're missing the point

[thing12]

Webmin really doesn't do a good enough job. Really -- try using webmin against the latest version of most software and you'll be sorely disappointed. The problem is that the people who release the webmin modules are NOT the people who release the software that webmin is configuring.

What we need is exactly what the author is proposing. A generalized configuration system that works off meta data that the software developers supply.

• All the possible configuration options in logical groupings
• descriptions of each option
• default values
• validations on input values
• being able to label options as 'experimental', etc...
• and of course option dependencies - that is if this option is turned on, then enable these 5 other options.

To do it right you'd ultimately you'd probably need to have a very light scripting language with flow control and variables.

But the important thing is to get a workable standard in place, one that the majority of developers can rally around and will be happy to develop configuration scripts for their applications. Most developers wouldn't even have to change the way their application reads the config files - documenting everything in a form that this 'configuration manager' can use would be enough. It's already been too long - we shouldn't wait any longer to get this process moving forward.

Re:It seems you're missing the point

[ttfkam]

First you have to choose the "standard." Right now everyone is bickering about their pet implemention (myself included). Unfortuantely, different programs have different needs. And rather than simply accept the obvious choice of the format that handles all configuration types (flat, hierarchical, etc. -- smells like XML to me, can you tell I am biased?) people still cling to their "my program only needs three config parameters so I'm going with my own custom, unique, weird-delimiter format" ideals.

Re:It seems you're missing the point

[Nailer]

Why not use the standard tools we're already using today... autoconf, m4, automake, libtool.. and have them build a "standard" config file for us at configure/build time?

Probably because a lot of the configuration files which are editied by m4 aren't considered human readable.

Some similar thoughts

[Eloquence]

I had some similar thoughts when I posted this, which describes a system that would be capable of reading and writing various configuration files and at the same time try to establish a common configuration file standard. The idea is not new, a lot of similar attempts have already been made. I think the only way this can be successful is through massive collaboration in writing filters to support existing config formats. This means that the filters themselves have to be fairly simple (possibly even simpler than something like perl or awk, but everything should be supported), and that the configuration manager, as the Freshmeat author calls it, should offer direct access to the filter library so that you can easily add modules to support new programs.

There's lots of cool stuff you can do with standardized configuration files, and dynamically generated GUIs are only one part of it. But developers are not going to change their file formats unless there's a real push for a specific format. Thus, to establish any format, you first have to develop lots of filters to support legacy software. As the FM author correctly points out, such a system would make software configuration under *x potentially much easier than under Windows.

Finally, to all the hackers who fear that this will take away their file-meddling options: If properly implemented, it won't -- and if it does, it will not get accepted as a standard, exactly because of stubborn people like you :-)

human interface guidelines

[dunkelfalke]

maybe the community should agree about common human interface or at least about a common configuration file interface so a kde/gnome/... app can be written using the data for providing an easy setup tool.

i think it would be a good addition to the linux standard base

What configuration nightmare?

[phaze3000]

As someone who has used Linux for some time, and also had the misfortune of using Windows (and doing those MCSE things for all my sins), I can honestly say that I don't think the configuration is the problem. The varying standards used by programs aren't a particular problem as long as there a decent comments in the file (and there usually) are.

The real issue is finding the configuration file; if you're using a distro you're not used to, then one often has to resort to using find. This is exactly what the Linux Standards Base was designed to avoid, and if all distros were to follow this model then I can't see that there is any real problem.

At the end of the day, we're not talking about things like setting background pictures in the window manager, we're talking about setting up mailservers, webservers and the like. If you're not cluefull enough to edit a text file then you're not cluefull enough to be put in charge of setting one of these servers up either.

Some *correct* information about MacOS X config

[lcracker]

Yes, I know this is a comment on freshmeat (I wrote it).. but it pains me to see so much incorrect/incomplete information here that people just believe for lack of experience with OS X.

---

MacOS X's solution to this problem is a couple levels
deep. There are three (?!) systems for configuration:

(1) /etc (which is a symlink to /private/etc):
It's still there, used for a few things. But not much.
More or less just stuff that runs before everything else,
or stuff that apple doesn't have much control over, like
Apache.

(2) Netinfo database:
basically takes a lot of stuff out of /etc and puts it in a
"registry", this registry is accessible over a network
connection for remote administration and such. You
can also do some funky server/client stuff with this too.
There's a command line interface to it that lets you
read/change 'keys' and also dump out certain keys in
well known /etc style formats

(3) Property Lists (.plist):
This is the way that just about all configurable
information in MacOS X is stored. There are a couple
different kinds of plists that the Core Foundation
libraries understand, but the most prevalent of which is
the XMLified version, the others are either vestigal or
are used primarily for localization of strings. Every
application (well, bundled app) on the system has an
info.plist file that has application settings that the OS
reads from on app startup, but you can also put your
own tags in there. Applications also have the option of
putting their own tags in the Resources subdirectory in
their bundle, which is the preferred way. OS specific
preferences (requires admin privs) go in /System/
Library/Preferences, System-wide app preferences go
in /Library/Preferences, Network-wide app prefs (only
applicable if you have OS X Server on the network) are
in /Network/Library/Preferences, and User app prefs go
in ~/Library/Preferences.

The Core Foundation framework manages more or less
all of the work. They'll find the plist for you, let you
work with the data structures, and serialize/deserialize
it to plist format. It's really quite nice to use. Just
about all of the data structures in Core Foundation are
serializable, even the property list data structures, so
sometimes you'll see property lists with serialized
property lists inside them.

As far as unified configuration interfaces go, there
really isn't a need for any in Mac OS X.

You're on your own with /etc, as nothing there is meant
to be novice-tweaked anyways (In OS X).

There is a GUI NetInfo Manager and command line tools
to configure the stuff in that database (nothing a
novice needs to touch).

Configuring the plists is primarily done inside the
applications that you use (since it's so easy for the
developer to use Core Foundation to read/write
property lists), there are no separate configuration
programs, but there are command line tools for working
with plists. If you're using the developers kit, you get a
little (not very poweful and pretty inefficient) program
to edit plist files graphically, but otherwise you either
do it by hand with a text editor or the command line
tools and risk breaking things or let the apps take care
of it themself.

BTW, all of the Core Foundation stuff is available in two
flavors: C, and Objective C. The C libraries are pretty
straightforward and work in all situations, but if you're
writing an app using the Cocoa framework in Obj C, it's
way easier.. though there are a few little peculiarities
about the Obj C implementations. The bonus is that
you can use both at the same time, and Core
Foundation types / Cocoa instances are interchangable
by some voodoo that Apple did. If the Cocoa
implementation doesn't work right (As with serializing a
plist that has NSNumber Boolean types), you just use
the Core Foundation implementation and a typecast..
voila, one line of code changed and everything is
happy.

Re:Some *correct* information about MacOS X config

[Jordy]

NetInfo exists for the same reason NIS, NYS, YP, etc exist; to centralize system configurations across a network of machines. Things like printer configurations are typically the same in a corporate environment.

NetInfo is actually Open Source. Though I'm not sure how up-to-date it is.

As for /etc... some of those are synchronized from NetInfo, not the other way around. A good number of them are just stub files, but there are a few that do not exist at all in NetInfo at all.

NeXT, FreeBSD, Apple

[ZigMonty]

Darwin (the Unix bit of Mac OS X, the kernel of which is called xnu) is mostly based on whatever NeXT's OS was (NeXTSTEP?). If you want to classify it as it is now, it's a Mach 3 based BSD4.4 OS with a lot of the userland stuff taken from NetBSD and FreeBSD and kernel code from all over the place (Apple mentions FreeBSD). Some man pages still say OpenBSD, so I guess it has some of that in it too (or at least the man pages :-).

People keep trying to make out that Apple ripped the entire OS off one of the open source BSDs. Can't people accept that a lot of it is either NeXT derived or from Apple? A lot of it isn't, of course, but the other BSDs copy each other's code all the time. I'm sure the others will start copying Darwin as soon as Apple lifts its non-BSD license (if ever).

If you want more info check out the Darwin project page and the developer site.

Thoughts from someone who adminsters both

[weave]

Having had to administer both windows and multiple unix server, some thoughts (and since I'll be negative to both platforms, it guarantees zealots from both sides will flame me, bwahahahaha)

• The registry in Windows seems to be a logical choice. There are standard tools to use it, it can be manipulated remotely, and except for those horrible clid crap. It is, however, difficult to understand for a human except for those common areas like HKLM\Software\Microsoft\Windows\Run.
• The Windows registry implementation is horribly flawed. It's too likely to get corrupted. A lot of this is from being part of a roaming profile. Losing your registry is like losing all of your application's and user preferences. It really sucks.
• *NIX is a mess when it comes to location of config files, as stated in the article. Even various Linux distros. We have Redhat boxen doing a lot of work now, having switched from a proprietary UNIX (dg/ux) a while back. Some of my techs think we should switch to Debian. I installed it on my workstation in vmware. It's nice, but it'll just require re-learning where the hell everything is. Maybe no big deal but I've got too much to remember already.
• Windows registry trees are not commented. You need to know how to find various reg hack sites and own a ton of resource kits, just to keep a leg up on the crap. Even then everything is not revealed. "You should configure it through the GUI." Yeah, right, on 2,000 machines?
• UNIX config files generally only have one per app. Configuring an app is simply a matter of loading the config file into an editor, reading the including commentary, and adjusting to taste. The exception here is the redhat /etc/sysconfig tree where everything is basically just loading of env vars for other scripts. Not commented, minimal defaults, if you need to figure out something it's dig through docs or read the rc scripts yourself to figure out what to set in it. Yack...
• Windows configs are often done through a maze of menu entries, dialog boxes, tabs, "advanced" buttons, etc... It always leaves you wondering if you've convered everything...
• UNIX config files are easily replicated to another box for a poor man's backup/failover situation. I had a 2000 server in a SAN go down and while I could easily mount that boxes disks into another 2000 server, moving the printer and file shares over was a problem because that shit is all stored in the registry. Instead of a simply copy command, I'd either need to write some sort of program to extract and merge into the backup's registry or figure out another way to replicate the shares. Keeping config crap out of a common database means the service isn't tied to a box so much. Need to move it to another box? Install, copy config files, change a virtual DNS name to point to new location.
• Windows registry is horribly insecure, not by design, but implementation. Loads of apps insist on writing per-user stuff to HKLM during runtime. I should be able to make HKLM r/o for all users but if I do that, shit breaks horribly. Damn it, HKLM should only be scribbled into by an application during its install process.

Re:Thoughts from someone who adminsters both

[cygnusx]

Windows registry trees are not commented. You need to know how to find various reg hack sites and own a ton of resource kits, just to keep a leg up on the crap. Even then everything is not revealed. "You should configure it through the GUI." Yeah, right, on 2,000 machines?

For newer software, wmic.exe helps. Plus seems like Windows.NET server is moving away from the registry model, starting with IIS6 using a XML based meabase for storing config info.

Re:Thoughts from someone who adminsters both

[Skapare]

For newer software, wmic.exe helps. Plus seems like Windows.NET server is moving away from the registry model, starting with IIS6 using a XML based meabase for storing config info.

So, now we can use NOTEPAD to edit the configuration, if we can just figure out what the tags really mean, and whether what we need to provide is supposed to be an assignment inside the tag, or data enclosed between tags (I've even seen both ways used at the same time). DTD's anyone? Do people even read DTD's?

Re:Thoughts from someone who adminsters both

[Mnemia]

I don't think the idea of databases to hold config files is *necessarily* a bad idea. We all know how the Windows registry sucks, and the hell it can cause, but I think the general principle could be a good thing if we addressed some of the points you mentioned above. For instance, there could be seperate config databases for each user and for local system specific configuration. If enough attention was paid to security and reliability I think the idea could work well.

Re:Thoughts from someone who adminsters both

[weave]

What would be sweet is if each domain had a registry database for all domain machines and users and all settings would be recorded, pulled, from there. It would get rid of the stupid-ass roaming profile model while still allowing settings to roam. It would also allow administrators to update settings site-wide or any subset using simple SQL calls to change, for example, all registry preferences for a certain class. Then your stupid mandatory profiles, policy files, and everything else ugly in that area would be solved too. As it is now, we have to beg users to leave their machines on so we can run jobs to contact their machines and update HKLM shit as needed... It's horrible.

I'm getting a woody just thinking about how cherry a central prefs/settings database would be. But no, whatever Microsoft has up their sleeves wouldn't be so sweet. It'll be some horrible kludge I'm sure.

(For home (non domain) boxes, the issue becomes more complicated. Each box would have to have its own database server on it to handle this kind of thing locally (non-roaming).... Either that, or tie your prefs into your passport account on some central microsoft prefs server (insert evil maniacal world-domination laugh here))

Now if *NIX would permit something like that, a common API for settings which could be implemented locally or centrally without knowledge of each app doing it, that'd be great!

Re:Thoughts from someone who adminsters both

[ZxCv]

IIRC, MS started moving most of the IIS config into the metabase with 5.0. Maybe with 6, they're completing the process. But I certainly remember having to deal with the metabase when I had to use 5.0 at work.

A modest configuration proposal

[cobar]

This would be really nice for allowing configuration utilities to work on programs they've never even heard of. Imagine for a moment, that the developer of an application provides lots of useful information for each configuration option, such as the level of importance of an option, comments and descriptions, web links, the data type of the variables, etc. in a xml template file that comes with the application. The template could then be queried by whatever tool you're using and provided you have enough info linuxconf, web frontends, dialog apps, etc. could all build an interface by simply reading all those options from the template via some kind of library function.

Likewise, changes to the config could be passed through the library functions, get validated for correctness and then written to disk in whatever format you want. The level of functionality would be such that you could generate white space delimited files, xml configs, or whatever you want and because the template is descriptive enough, comments could be inserted/preserved for those editing with vi.

Now that I think about it, this really wouldn't a rewrite of existing applications, they could simply keep their existing formats and build the template file for other config utilities. Then, if at a later date they decide to drop their own config processing, they use the template driven library to do the processing work. That way backward compatibility could be maintained.

This does nothing to reduce the myriad different configs that you deal with when editing by hand, but it would give the authors a chance to move a more common format later on down the road. And I think it is more useful than trying to get everyone to switch to some unified format that appeases new users.

I have half a mind to try a limited version of this out for handling Apache virtual domains. And it's a lot more fun than hard-coding a parser into your frontend.

A few things

[HalfFlat]

The current system for starters isn't too bad - the configuration files for important system tools are typically well documented (in man pages if nowhere else), and aren't too hard to edit in a text editor. Certainly there is a problem with the use of automated GUI tools for system configuration, but I can't really see why GUI tools are so important.

Consistency though would be a nice thing, if only so that as more programs are used and require configuration, we don't all have to learn more and more ad hoc configuration schemes. There is the potential for the current system to get well out of hand. Consistency is nice from an aesthetic point of view - ad hocery is rarely pretty.

Tools such as linuxconf or webmin don't help the problem, they just obscure it. Often badly! (Don't get me started on linuxconf, ugh!)

The earlier suggestions of XML, or a recognised subset of XML, together with XML schema to describe the formats, seem ideal as far as file formats go:

• It's standard and well-documented.
• There are a large number of parsers that work with a useful large subset of XML, for many platforms and languages.
• In a pinch, it's not hard to write a quick and dirty and correct-enough parser from scratch on an ad hoc basis, to parse your own particular XML-based configuration files.
• It's human editable (or should be, if used wisely).
• It is rich enough to encompass all the text-format configuration files currently in common use (under the Linux-based operating systems I know of at least).

The big problem of course is inertia: changing configuration files is a big change! Personally, I think the cost though would be worth it in the long term.

The important thing to me is that there is no API specified as mandatory. While a standard API for accessing and manipulating configuration data would be nice, requiring it to be used will not only make adoption of a standard config format slower and harder, but will also limit software developers to the tools for which the API has an implementation. I'd go for:

1. Use XML + schema as the mandatory config file format in the new scheme.
2. Have a preferred mechanism for signalling to applications that configuration data has changed (eg SIGHUP) so that changes can be made atomically.
3. Provide an API for manipulation of config data, but only as a convenience to developers and automated config tools.

I also agree with the earlier comment promoting the use of ~/etc over dot-files. Makes good sense! Again, it's only inertia that will make such a change difficult. Though if other changes are going through ... in for a penny, in for a pound!

Lastly, whatever scheme is adopted, it seems to me essential that there be some flexibility in where the configuration files are located. Standard places are important, but there must also be some way to say to an application that their config data is in /usr/local/etc, or in /opt/package-name/etc, or wherever. This is necessary for sane package isolation and management, especially if not all software on a system is maintained by someone with superuser priveleges.

Classic Excuse

[ZigMonty]

You're using the classic excuse: "They control the hardware so of course Macs are easier to configure!"

Name a piece of current (not ISA etc), statndard (not some home built/unknown vendor PCI card) PC harware, other than the motherboard or the processor and it will most likely work on Mac OS X. It may fall back on a generic type (missing some features) but it will most likely work. Macs support: PC133 RAM, ATA hard disks, PCI cards, AGP cards, VGA monitors, USB, IEEE 1394 (FireWire), etc. I think that would be more than 50 types of hardware. Don't confuse the fact that the iMacs are closed with other Mac hardware. The G4 Towers won awards for how easy they are to open up and upgrade. There's a large handle on the side of the case that you pull and that side of the computer lowers down with the motherboard attached to it, giving easy access to all components.

Could it be that Macs are easy to configure because Apple engineers are clever?

Note: I'm ignoring the fact that this article is about software not hardware configuration.

Check out some more myth debunking.

It took this long?

[Ageless]

It took MacOS X for people to realize that this was a problem in UNIX? Please.

The reason UNIX and UNIX applications are hard to configure, in most cases, is because Open Source programmers are lazy.

This is obviously a blatent generalization so I will explain.

The old adage is that an Open Source program gets written when a programmer has an "itch" they decide to scratch. The problem is that very few people are itched by configuration. You may write the best web server in the world (Apache!) but by time it comes to writing the configuration manager for it the volunteers start falling away.

It isn't very fun writing a bunch of dialogs, windows, buttons and such to make a nice configuration for a program. It's kinda like documentation (and we all know the state of docs for many UNIX programs).

I see examples of this every day. I have a Mac OS X using friend who sends me the URL of every new program he decides to use. It's incredible how many of them are UNIX ports with a beautiful configuration manager stuck on. Mac programmers hold themselves to a higher level of user experience and UNIX people need to get on the boat.

What's needed isn't a global, all dancing, all singing configuration system. What is needed is responsibility in programming.

P.S. Everyone always whines about the Windows registry because it's binary, you can't edit it blah, blah, blah... But the fact is: It works. The average user never cares to edit it because they config their programs from WITHIN their programs. If something is truly needed, do the Windows registry in text file format. Make it /etc/registry.conf. There is no reason it has to be binary.

Re:It took this long?

[Rentar]

The old adage is that an Open Source program gets written when a programmer has an "itch" they decide to scratch. The problem is that very few people are itched by configuration. You may write the best web server in the world (Apache!) but by time it comes to writing the configuration manager for it the volunteers start falling away.

I just want to add my personal reason for never having written any configuration-frontends. (Which I assume is not only true for me, but many others as well). The way it goes for me is usually the following:

• I find a new cool software.
• I have no idea how I have to configure it correctly
• I search the web for configuration front-ends and find none
• I decide to do it the hard way (as there seems to be no other) and configure it by hand
• While doing so, I find out that it's not that hard and I decide that I'll write a frontend for the configuration, as soon as I really know the configuration format.
• When I finally reach the point where I got enough knowledge to be able to write such a frontend I no longer need it, 'cause a.) I allready configured my system and b.) I'm faster doing it by hand. Ergo: No more itch to scratch.

So this is the sad story of at least 10 configuration frontends that never even began to exist.

Re:It took this long?

[Ageless]

If you read my post over, I am not really advocating the use of a single database. I am saying that if that's what people really want (as some other people have posted) then a text based "registry" would work. I have no problem at all with a million different configuration files all in different formats spewed about the filesystem... as long as the program that owns them manages them for me.

In old versions of Windows (pre 95) most programs used .ini files which worked okay. They were generally in \windows or \windows\system but some programs put them in other places, or even used other config file formats. The difference is for most people it didn't matter where they were because the program that made them had a nice little configuration dialog for managing them. THAT is what I would like to see. If I can do everything related to configuration from the program, I could care less how it stores it.

Re:It took this long?

[evi1b0b]

It isn't very fun writing a bunch of dialogs, windows, buttons and such to make a nice configuration for a program.

It still might not be fun in OSX, but making interfaces is very easy because of the included dev tools and the cocoa application model.

A programmer of marginal skill can literally drag together a standard compliant UI in 5 minutes and have it work.

It's a little bit more than a lazy programmer issue. A rich, standardized library of Cocoa classes and dev tools that almost force oyu to do things "the right way" also contribute to better front ends on osx -- lets not forget the fact that the userbase probaly would not use an app alot if it didnt have a good interface.

Re:It took this long?

[geekoid]

What's needed isn't a global, all dancing, all singing configuration system. What is needed is responsibility in programming.
amen, brother.

Re:It took this long?

[Ageless]

I am mostly talking about the applications available for UNIX and UNIX copies (Linux included). A great, great many of the common applications used in **IX now-a-days are open source apps and fit my description above.

Re:It took this long?

[J. J. Ramsey]

"P.S. Everyone always whines about the Windows registry because it's binary, you can't edit it blah, blah, blah... But the fact is: It works."

Actually, the problem with the Windows registry is that it *doesn't* work. It's a single point of failure that has been all too easy to corrupt.

Re:It took this long?

[Phroggy]

P.S. Everyone always whines about the Windows registry because it's binary, you can't edit it blah, blah, blah... But the fact is: It works.

Yeah, for about six months. For the next six months, it starts falling apart. If you really know what you're doing, or if you're just lucky (Windows is so inconsistent, it's not even consistently bad), you can keep it going longer than that. Often, the system will run fine until you try to make some configuration change, and the whole thing breaks - for example, install a NIC, load the drivers, and now the system hangs on boot.

If something is truly needed, do the Windows registry in text file format. Make it /etc/registry.conf. There is no reason it has to be binary.

I'm a big fan of plain text, although I really like the promise of XML (it's text, sort of, but it can be edited with a GUI front end). However, a registry is NOT the answer - if you think it is, you haven't spent enough time on a Mac.

One of the great things about the Mac OS is that the settings for each software component (applications, and pieces of the OS) are stored in seperate preferences files. If one of them corrupts itself, you can figure out which one, trash it (the app will generate a new one with default settings automatically), and everything is fine.

Sure, the Registry is binary, and that's a bad thing, but that's not the problem - Mac preferences files are (usually) binary too. The problem is that all apps store their settings in the same place, and there's nothing to keep an app from mucking about where it doesn't belong.

Another great thing about the Mac: instead of a database of which drivers and components and things need to be loaded at startup, you just move them into a folder, and everything in that folder gets loaded. This is amazingly simple for users to manage - they can configure what loads at boot time the same way they manage their own documents.

The ability to easily delete corrupted settings and replace them with a default copy, on a per-app basis, is something Microsoft will probably take years to catch up with.

Re:It took this long?

[Ageless]

I am actually a huge fan of application specified prefs done correctly. My favorite implementation (I have not used a Mac much) was in BeOS. In Be, every file had attachable attributes that could be as simple as a boolean value up to multimedia files. The supplied API was simple to use and it worked great. Most applications kept their application specific settings in attributes, so that if you moved the app the prefs went with you and kept their system specific prefs in /boot/home/Settings.

All the attributes in BeOS could be accessed, modified, removed or added from the command line with a few simple UNIXy tools too.

I miss BeOS :)

It's not more HUMAN readable...

[Greyfox]

The nice thing about XML is that parsers and standards exists for the file format itself and for metadata describing the file format. While DTDs are a start, XML schema are even better, telling you (The generic config file program reading the XML file) exactly what values an element is expected to have and how they can be nested.

You can hack out a CSS file that would describe to any XML enabled web browser how to display your file in a much more readable format and XPath lets you slice and dice an XML file however you want to. I've run across an XML enabled grep-like program too, though I've never actually had much need for it.

Mac OS X PLists

[ZigMonty]

You are describing Mac OS X's Property list (Plist) format. All Application information (file types etc) and user preferences are stored in .plist files, which are XML. There is easy API for apps to use. Here's the carbon docs for it. The Cocoa plist overview docs still aren't up yet but I can assure you that it's even easier in Cocoa.

Quick example taken from the above docs:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList .dtd"><dict>
<key>Year Of Birth</key>
<integer>1965</integer>
<key>Pets Names</key>
<array/>
<key>Picture</key>
<data>
PEKBpYGlmYFCPA==
</data>
<key>City of Birth</key>
<string>Springfield</string>
<key>Name</key>
<string>John Doe</string>
<key>Kids Names</key>
<array>
<string>John</string>
<string>Kyra</string>
</array>
</dict>
</plist>

Note: The <array/> tag is specifying an empty array.

Re:It's FreeBSD...

[ZigMonty]

No they didn't. A lot of their code is from FreeBSD, and they're big enough to admit it. They built Mac OS X on NeXT's OS which is a Mach based BSD. They didn't just take FreeBSD and slap a GUI on it. Read their page more carefully.

NO ! From an elitist point of view.....

[CDWert]

Making *NIX systems easy to configure maintain, manage and etc will be the downfall of the IT industry, Green Skinned *NIX admins everywhere will revolt the people tring to bring *NIX to the masees.....

Alright so im joking...big suprise...

I think this would be a good thing all in all, IF they DONT MUCK with my existing configurations. There are many things that can be tweaked far beyond the iditot proof gui configuration tools on other OS, In windows you can resort to registry tweaking for some items, others you just cant do a damm thing about, its those thing Im afraid will be removed from my control.

Dummning down on the surface is fine, as long as they dont do it to the underlying configurations. I still cant figure out how the hell to edit (or find) the samba configuration file on OSX, Now granted its not my machine, and I dont really use BSD, so im not suprised. But does it even exist ??? A I am used to it on Solaris and Linux ?

Re:Apple + NetBSD?

[MaxVlast]

I'm not sure how Aqua is the illegitimate child of GNU. The core of the whole shebang are the OpenStep libraries that Apple got with NeXT. The compiler is gcc, but that hardly implies parentage. In fact, A qua is the least third-party dependent component of the whole OS. And as Hurd uses the Mach in the same fashion as OS X, it hardly means that Mac OS is descended from the GNU effort. They're more akin to cousins.

I'm always impressed at the GNU folk's clamor to make everything appear to exist at the grace of Stallman. What he did is good for the world, but he isn't the second coming that he so often portrayed to be.

Also, all versions of Darwin (incl. Mac OS X) have apt available to them through the fink system.

Re:Apple + NetBSD?

[T-Punkt]

OS X is based on Darwin and Darwin takes bits and pieces from all three *BSD, Mach, GNU (compiler) and a lot of other projects. E.g. most of the userland stuff is taken from NetBSD, you can read it up in the Darwin FAQ. Or look at the Darwin-contributors page which lists them all in alphabetical order.

So saying "It's based on FreeBSD" is as wrong/as correct as saying "It's based on NetBSD".

But since FreeBSD's J. Hubbard now works for Apple and FreeBSD doesn't run on the range of machines Mac OS X supports yet (but NetBSD does) expect OS X's roots to FreeBSD to be promoted more than its roots to NetBSD... (sigh).

NOT XML

[Hard_Code]

No XML is not suited for this. XML is not a panacea. XML is for representing large amounts of structured data whose primary purpose is to be interpreted and transformed, and frequently eventually presented to some user. The semantic, syntactic, and performance overhead of XML is WAY too great.

Instead we just need a simple text-only configuration file format which has support for a hierarchy of values (this gets you pretty much what XML would give you). It would also be nice if the format was position-independent, so it could be streamable. A proposal was posted somewhere, but now I cannot find the link. The format could be as simple as:

toplevelkey=value
[tab]branchname1
[tab]branch-key1=value
[tab]branch-key2=value
[tab][tab]branchname2
[tab][tab]branch-key3=value

you get the idea. Please don't throw XML into the mix, it will not simplify things.

Every journey begins with a first step

[Quixote]

I think most of us agree that the conf system is getting out of hand. With the proliferation of distros, its only going to get worse.
A solution would be to first implement a generic "configuration" library. A small, efficient library with various language bindings. It should provide the coder simple functions to manipulate the conf file, sortof like "getopts".
Once the above library is implemented, encourage people to use it in any new projects that they are starting. At the same time, volunteers (ie you) can work on "porting" the existing packages over to using this new fangled library.
Over time, this approach should work.

Re:No

[MaxVlast]

Do you know what that means? "Based on NeXT" doesn't mean anything. NeXT was a company.

Mac OS X is based on OpenStep 4.2, which, itself, was based on NEXTSTEP 3.3. NEXTSTEP is a BSD operating system running on a modified version of the Mach microkernel. OpenStep is a API specification and a set of libraries that conforms to that API. OpenStep 4.2 (the operating system) is an implementation of those libraries on top of NEXTSTEP.

When Apple bought NeXT, they planned to build on top of OpenStep. They first produced Rhapsody for PPC and Rhapsody for Intel. They were the same OS running on two hardware platforms. On top of Rhapsody, Apple put the Blue Box, which was a Macintosh compatibility environment. At no time was there any need for a "BSD compatibility layer." It was all software running on top of BSD. Apple then killed Rhapsody for Intel (and the Yellow Box, but that's tangential.)

What was left was released as Mac OS Server.

Mac OS X 10.0 and Mac OS Server 10.0 (and further versions) are also BSD operating systems. They have the Cocoa (OpenStep) and Carbon libraries available, and the imaging system is called Aqua (replacement for Display PostScript.) At no point in any of this is there a need for any UNIX compatibility layer, as it is all real UNIX. The only compatibility environment necessary is for Mac OS 9 (Classic.) Only certain older applications (Carbon) can run natively on OS X, so for running non-Carbon apps, Mac OS 9 is run in a compatibility environment (similar, but not the same as VMWare.)

I hope that clarifies things.

Re:Yup

[MaxVlast]

Have you ever seen the PARC interface (from the Alto and the Star)? I have. It is nothing like the current WIMP interface that is so popular. Nothing! No overlapping windows, really strange mouse button concepts, much more. Apple licensed (not stole) certain parts of the interface and expanded greatly the rest. Things like window clipping (allowing overlapping), menus that are sane, the modernization of the scroll bar, and much more.

If you're going to deploy a really cool sound system, doesn't it make sense to buy a proven core and put your own interface on it? SoundJam provided the playing engine, and Apple made it a powerful, friendly program.

I don't see a problem there.

Why not XML? Not tab-count-dependent!

[Nonesuch]

I'm not a huge proponent of XML, but it does make for configuration files that are easily parsed by the application, a human, and a syntax checker.

Yes, XML imposes overhead. But it also brings some real advantages. Look at the Apache configuration file, particularly 2.0.

I really really despise configuration files that are dependent on the number and type of whitespace in a line to determine how the data is parsed, particularly what syslog does.

If cut-n-paste (generally converting tabs to spaces, etc) breaks your configuration format, then your configuration format was broken the day it was designed.

Nightmare?

[pmz]

What nightmare? If your sys admins are worth their salaries, then you shouldn't be losing any sleep.

I think the article is referring to a heterogenous network with lots of history, where mediocre sys admins have come and gone and six different flavors of OS exist. This is not a problem with UNIX, it is a problem with your high staff turnover and patchwork network architecture!

Planned well and managed well, a UNIX network is very nice, and you don't need glitzy GUI tools to get you there.

Certain minimum criteria (From a Unix pro)

[Nonesuch]

It'd be great to have better mechanisms for configurations, particularly some means for handling inheiritance of default values, parsing for correctness, and change management.

There are are certain advantages to the current general way Unix software configuration files are handled. I'd hate to see these advantages lost in the name of making things 'easier' or 'more centralized'. Some of these advantages include:

• Storage of configuration data in flat files, without the need to compile to another format before use.
• Each independent application generally has just one or two configuration files, and settings for just that one application are thus easily migrated to another host/platform/etc.
• Being decentralized, an error on automatic or manual editing of configurations for one application doesn't corrupt the configuration for all.
• Most configuration file formats are both easily human-readable and machine-readable.
• Configuration parsing does not rely on linking against a large, potentially insecure configuration file handler library.

Some of these strengths are also weaknesses- for example, having each programmer write his own configuration parser leads to many disparate security flaws as each author makes their own unique mistakes.

I've rescued a number of systems from otherwise-fatal low-level configuration errors by manuaully correcting the configration, either by mounting the drive on another system, or simply hex-editing the raw partition to replace the first character of the offending line with a '#'. I'd hate to lose this last-ditch rescue option.

Re:no, and furthermore... If it

[alfredo]

If it was cool enough for Apple, then maybe the rest of the world with see just how cool the world of the the Nix's is. UNIX/Linux is more than a Kernel, it is a reflection of all of you who have contributed. That is what makes the Nix's so fucking cool. Apple knew that.

For once Apple found someone cooler than itself and wanted to be part of it.

'Advanced' config on windows

[TeknoHog]

I agree that Unix configuration can be a bit hairy, but one thing I've always liked about it over that of Windows is a certain openness, democracy, or equality.

What's advanced to one is trivial to another.

Many people here complain that unix is elitist, but at least it doesn't make assumptions about your level of expertise. There are many situations where a beginner needs to configure something that happens to be under 'advanced'. It's very discouraging to give the idea that you must be 1337 to use those options. Also it's impossible to split people simply into beginners and experts because an expert in one field is a beginner in another.

I guess it goes naturally in the whole of OSS and unix philosophy. If you need to fix something, there are guides to there even if the path proves difficult. In Windows things are decided for you, and if something is decided 'difficult' the system reminds you that it takes some black magic and you probably shouldn't do it.

So, whatever the silver bullet to config is, let there be an option to hack the files with emacs. Because computing is not black magic which should be hidden behind candy houses.

NOT a file-format, a parsing library

[camusatan]

The proposal isn't actually a bad one - we're talking about just a simple C library which can read, write, and parse configuration files using 'plug-in' stub libraries. That's what the author of the article is proposing. It strikes me as a good idea for several reasons:

• Certain configuration file-formats we're kinda 'stuck' with. For these files, we just have to write the plug-in which can read, write, and parse them. We don't have to re-write the application to comply with The New Way. The app doesn't need to change, or even be aware of the configuration library.
• A configuration utility written to use the library would be able to configure all applications which have the appropriate plug-in installed. So if someone writes a nice pretty GUI X-based app, it can configure everything. If someone writes a curses-based text menu app, it can configure everything. A web-based configuration app could also configure everything. Some config-apps could be oriented for gurus, and some could be designed for newbies.
• Eventually, applications themselves could use the library to read in their own preferences. Bonus - it's a little easier to write applications now.

All that being said, I foresee some 'gotchas' -

• The library system would need to not only know about configuration file formats, but some meta-configuration info such as - application X uses config-file format Y in location Z. Application X can be forced to reread its config by sending it a SIGHUP, it's process name is 'blah'.
• The design of the API for the main library has to be made veeeeeeery carefully. The number of different types of data that you'll have to deal with here are huge - we're not just talking 'integer, string, float' - but some relatively complex data structures like IP Address, or 'Firewall mapping' (two IP addresses, two ports, and a UDP/TCP choice).
• Configuration file formats that are unbelievably miserable and allow for the user to put script bits in them (isn't this why we all hate MS Word?) would be almost un-programmatically-configurable. Sendmail.cf comes to mind. It gets to the point, there, where the only thing that's ever going to understand the config file is the application itself.

This is a "Hard" problem.

[paj1234]

This is a non-trivial problem, and one I have some experience of (failing to deal) with.

My program is called "createusers". I wrote it because I wanted to make the following for 100s of new users in one go: user login, Apache home page, Samba account, PostgreSQL database, and MySQL database. I also did my best to make it extensible.

You can find my program here:
http://freshmeat.net/projects/createusers/
and here:
http://www.lfsp.org/

The fundamental problem I faced when writing createusers is there is no standard programmatically-accessible way of:

1) finding out what can be configured in each application

2) applying and undoing the changes.

Furthermore, I would like to make createusers programmatically-accessible. Unfortunately at present I cannot, because there is no standard way for me to let other programs know what facilities my program has, and no standard way for me to make them easily turn-off-and-on-able by other programs.

I received some add-on contributions to my program from other people. I noticed that their programs said "Have you enabled X in createusers.conf?", "Have you enabled Y in createusers.conf?", etc.

If I had standards-based turn-on-and-off-ability for my program, then my contributors' programs would have been able to do it for themselves. They would be able to easily check what things my program is set up to do; AND they would be equally able to change it.

This is a problem for people Much Smarter Than Me. If it is solved, then I promise to put the necessary bits in to my program so its setup can be easily, securely queried and changed by other programs.

absolutely

[No-op]

You hit it dead on there. while you or I have no problem popping in and futzing with the configs by hand or killing processes, those pesky coworkers tend to be completely ignorant of such things.

nothing sells a free UN*X system to your management like this:

you: "we'd need to stretch our WinNT admins further and have them understand how this app works on this system; additionally we'll need 500 more server licenses for NT."

boss: "but that would cost more!"

you: "well, we do have this test environment set up, but it's running under FreeBSD (insert your OS of choice here)."

boss: "But isn't that hard to understand? what if you get hit by a bus?"

you: "well, before you say that, check out this nifty web interface to controlling the system. very nice and straightforward. very easy to use. the NT admins are ALREADY USING IT for other systems, too."

boss: "wow, that's pretty slick. and this is free?"

you: "yes, it's free, and we are ALREADY USING IT in the environment with no problems whatsoever. it also allows for authorized admins to remotely control the systems for 24/7 coverage! and what's more, FreeBSD (insert your OS here) is free, and we have to spend money only on server hardware."

boss: "well, looks like that's the best solution then. write up a proposal and go for it."

not that it always works like that, but sometimes it does, and that's a beautiful thing. plus, scripting your own needed code and adding it on as a webmin module is pretty easy, and can handle most anything you might need to do by hand, or via a script you're running from an ssh term. why bother? just *click*.

Re:absolutely

[Spoing]

Agreed. I'd add that using Grace Hooper as an inspiratin at times can be quite an asset; It's easier to appologize then to get permission. Facinating person.

Along those lines and other trechery in the name of thwarthing bad management attitudes and behavior, I've used this tactic to CYA with great sucess;

1. Problem: Something has to be done but you know that management won't sign off on it. If you don't get them to agree, you will end up in a heap of trouble even if you warn them months in advance.

2. Solution: Take the thankless job of doing required documentation, or add in a progress/status report.

   On a very regular basis(!), get management to review the documents till they are just not interested and want the summary. If possible, get them to sign off on the document.

   After it looks like nobody is reading what you're writing anymore, drop in a clause or section that adds in a stern warning about what you are concerned over. If you can do this with a minimal amount of changes, that's even better.

   When the shit hits the fan, point to the document and thier signatures -- but do so in a non-confrontational manner; don't say 'I told you so'.

   It is important to not to hide anything, to distribute these documents widely within the company or project areas.

This can also be used to 'authorize' certian activities without asking flat out. Yes, I have been refered to as evil at times.

Config File Compiler?

[mcrbids]

GCC is probably a good example - it compiles to metacode, then to actual machine code.

This "two-step" operation is what we should use - with that ANY config file could be used!

So you start with the original text-based config files, and thru a translation layer is converted into a format that this "Configurationator" can handle. Options are picked, and once done, it's uncompiled and saved in the original source format.

It wouldn't be terribly hard to create a metaformat that would allow for this - it would in essence be a modified regex definition!

-Ben

The BIG problem with Linux/UNIX configuration

[dentar]

The BIG problem with Linux/UNIX config programs is that they aren't stateless. Programs that try to keep their own versions of all the config files (ahem... linuxconf) get all freaked out if you go in and change a file by hand the manly way (with vi, not emacs). Webmin (I think) does not try to do this, so it appears to work better. If someone were to write a NON-STATE-KEEPING tool then it has a much better chance of working nicely.

Re:THAT, SIR....

[Mr. Neutron]

What's a "code fag"?

Re:Just some thoughts

[be-fan]

Some consistency might be nice, but in the end would probably be quite irrelevant. Imagine trying to come up with a unified syntax to incorprate Apache, Netnews, Sendmail, XFree, and Gnome. It would turn out to be a logical OR of all the configurations, and wouldn't save anything.
>>>>>
It really wouldn't be that hard, actually. XML can describe pretty arbitrary structures in its syntax, so a unified config file format should be within reach. Besides, Windows does it with just a key/value registry, surely the OSS community of uber-hackers can one up that?

Read the docs, follow the instructions. Or is our society forgetting how to read?
>>>>
Or, rather, the docs are uniformly garbage? Programmers shouldn't be allowed to write their own documentation without consulation from an English major...

Oh yeah, we like things easy. Or what we *perceive* as easy. I argue that easy is different for everyone. For me, easy is being able to fix something with a few quick keystrokes in vi. I find navigating up and down many layers of menus and submenus quite burdensome and slow. But hey, you go tinker with GUIs and pull-down menus. While you're doing that, I'm finished and getting on with my life.
>>>>>
Except that's really not true. 90% of config tasks take no longer in a GUI than via the CLI. Hell, by the time you've opened up XF86Config, scrolled down to the different places you need to get to to change resolution and color depth, somebody using the GUI could have cliked a couple of buttons and brought back a coffee from the local Starbucks.

That said, I think the beauty of current system is that you have options. You can use Webmin or Linuxconf, and I can use a text editor.
>>>>>>>>
You really don't have options. Because of the differing syntaxes, you have tons of formats to deal with, GUI tools that work sporadically (I've never been able to get Linuxconf to do anything but set the runlevel) and leave the files uneditable by humans. If everything were standardized, people would have the freedom to actually edit files by hand, or use a GUI.

What would be a real pain is having to reinstall the whole OS because a hard drive error during a power outage corrupted a single line of your X config, and without that, you can't get to your vaunted GUI to fix it. It would be a simple fix with a text editor.
>>>>>>>>>>>
And having a standardized format that ALLOWS a GUI prevents you from doing that how?

Look, nobody is trying to change everything into a Windows-like binary registry. People want a standard that ALLOWS for a GUI and ALLOWS for text editing.

Screw XML, use Python code

[Tony Hammitt]

We should use actual Python code as the configuration file format. It's callable directly from C, and by extension, all other languages. It's nice and clean. It supports heirarchial inclusion of other configuration files. It has easily readable comments. In short, it's the perfect configuration file specification language.

Re:Apple + NetBSD?

[Dahan]

Well, FWIW:

[greyfox:~] root# uname -a
Darwin greyfox.azeotrope.org 5.2 Darwin Kernel Version 5.2: Fri Dec 7 21:39:35 PST 2001; root:xnu/xnu-201.14.obj~1/RELEASE_PPC Power Macintosh powerpc
[greyfox:~] root# grep -l FreeBSD /bin/* /sbin/* /usr/bin/* /usr/sbin/* | wc -l
12
[greyfox:~] root# grep -l NetBSD /bin/* /sbin/* /usr/bin/* /usr/sbin/* | wc -l
135
[greyfox:~] root# grep -l OpenBSD /bin/* /sbin/* /usr/bin/* /usr/sbin/* | wc -l
24

Interpret that however you wish :)

Re:XFree 86 Configuration..

[TeknoHog]

Now, I realize that this might be impossible to implement without breaking several other things, but I just wanted to throw the possibility out there.

IMHO one of the great things about text config files in unix, is that they can be edited either directly or with custom tools. The application never gets to know how the config was edited, as long as it works. So in principle it can be done without breaking anything.

The only such tool I've used is Linuxconf, but it's rather limited and doesn't seem to work properly. Now I'm back to editing by hand and with scripts of my own.

One problem behind XFree86 is that there's usually a single main config file owned by root, although I believe it is possible to use personal XF86Configs. (Never tried this - no need on my personal system.)

Nevetheless, I could also argue that configuring X is not something that users should do. The system should be installed properly to fit the hardware in the first place. Those who want to tweak the performance for movie playback or such, probably need to have root access anyway. This might be one reason why your suggestion has not been implemented.

Wrong wrong wrong

[ttfkam]

The thing this article screamed was not "XML" but "API". Define the application programming interface for a library, and stop thinking that each application must directly access the native data

Yes, and the discussion went to the next step and started talking about possible APIs. The parsing of XML (DOM and SAX -- DOM for in-memory data structure and SAX for streaming access) are well understood and, most important, already done with innumerable tools already written.

XML Schema could not, for example, include primitive types for IP addresses (very common in config files), or validation constraints for integer values.

Wrong! Wrong! Wrong! You haven't read the spec and are obviously making assumptions based on what you've heard from others who haven't read the spec either. XML Schema allows not only the specification of integer, string, float, date, etc. values, but integer subsets (greater than and less than) and string patterns (through regular expressions). And RELAX-NG can use XML Schema's datatype support as well.

Do you have something that does validate IP addresses for example or were you just complaining about how one thing doesn't (even though it does) without showing anything that will solve the problem. If you say regular expressions, I must remind you that XML Schema supports regex matching of attributes and element data.

To date there are no (zero, zip, nada, nothing) fully compliant XML parsers.

Could you please enumerate these non-compliance issues? An encoding issue? Bugs in DOM or SAX support? What? I wonder about the vitrol of your argument without any specifics. Especially since I been using both parser that you mention (and a few others) without any noticeable issues associated with the standard. Any problem were usual to do with parser extensions to the standard which are fairly simple to avoid.

A vast number of parsers out there ignore the DTD, which means they cannot handle entities (which is a big issue).

You're right! That is a big issue. Unfortunately for your argument, it is not an issue with the leading parsers that you mentioned yourself: Xerces and MSXML. The default parser for Java (Crimson through JAXP), the default parser for Windows (MSXML), the default parser for Perl (XML::Parser I think...haven't used it yet) to name a few ALL handle entities. This is like saying that there are many web servers out there that don't support CGIs with the obvious intent of dissuading use of web servers.

A simple .ini-style line delimited key=value is the most readable and understandable (from a psychological viewpoint) configuration format.

I agree, but you're missing an important aspect here. INI files are usually one and at most two levels deep. They do not handle hierchical structures well at all! Let's look at a snippet from a default Apache configuration file.

# The following lines prevent .htaccess files from being viewed by
# Web clients. Since .htaccess files often contain authorization
# information, access is disallowed for security reasons. Comment
# these lines out if you want Web visitors to see the contents of
# .htaccess files. If you change the AccessFileName directive above,
# be sure to make the corresponding changes here.
#
# Also, folks tend to use names such as .htpasswd for password
# files, so this will protect those as well.
#
<Files ~ "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</Files>

What do we see? Hmmm... Because the key/value pair format doesn't allow hierarchy, they fell back to something that looks a bit...er...XML-like. And it isn't key=value, it's key[whitespace]value. Yet another issue: one file has the equal sign and others don't. I thought we were aiming for consistency? And what if the parameter is a message and that message has a newline character? A '\' at the end of each line? A '\n' where appropriate? These are problems that XML has solved.

And let's not forget Unicode support. We English speakers may have a hard-on for English, but can't you imagine a case where a program intended for a non-English speaking audience would want -- if not the names of the parameters themselves -- parameter values in alternate character sets than ISO-8859-1 (Latin1)? And before you rebut with UTF-8, do you want to write the UTF-8 translator? What about the other codesets? Are you strictly limiting to UTF-8? This is another area of XML parser that keep it simpler for the programmer. Transcoding is done for you and is most likely better implemented and supported than what you or I would come up with.

And when you have multiple "Files" definitions? Well, multiple blocks right? The "Order," "Deny," and "Satisfy" parameters are associated with the "Files" definition, yes? So why aren't you associating a group of "Files" definitions together? Because that would require further hierarchy? Would you end up with something like this?

<FilesDefinitions>
<Files ~ "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</Files>

...other Files definitions...
<FilesDefinitions>

Set up a relatively complex configuration where you have virtual hosts and those virtual hosts have behavior different from the default. What's that? You include them in tags by host? Tags that look somewhat like XML anyway you say? Imagine that!

And what about the comments? For some formats, the comment is the ';' character. For others it is the '#'. And none of which I am aware differentiate between implementation comments and actual directive comments. For example. the comment above describes the configuration directive so that any configuration editor could conceivably read in that info and display as help to the user (assuming that all comments were kept correctly before the directive and not after or on the same line). But what if you wanted to make comments about your specific installation. You are making comments so that others in your working group can see why you made certain changes to the config file, right? So how are they differentiated from the primary directive comments. This is the type of problem that XML namespaces are intended to solve -- a way of giving demarcation points to distinct pieces of (sometimes) unrelated data or simply the clean combination of multiple schemas so that your schema definition doe not bloat to immeasurable levels and different schemas can easily share with one another.

Something keeping back Linux/BSD/UNIX is the stubborn, 1337 coders who spent all day figuring out a config file that some joker on the net thought would be a keen config format but forgot to comment it. Let's face it, Apache is not the norm. Config parsers are a known problem with known solutions. Writing a config file parser is not the primary focus for most programmers out there. The config file is a necessary evil to them. Let's make it easier and just say, here's your DOM interface. It's well documented, works well for the limited dataset with which you a working (config files are usually less than 50K), will handle any configuration organizational type you want (hierarchical, flat, etc.) and will save you the time of writing a parser yourself.

Have a nice day.

Re:obligatory anti-XML post

[ttfkam]

And now you've combined data and logic which should be kept separate whenever possible.

Your solution is akin to writing a config file that consists of Perl hashs and arrays for inclusion via "require." You're locking into a single language and a wrong keystroke will crash your program (as opposed to simply having the config file parser report an error so that you can gracefully exit and, most important, tell the user what they did wrong.

Re:yet another tool to get the same job done

[Spoing]

Congratulations. Now you understand how the real world works. Not everyone wants to be an administrator/computer geek/Linus. And why should they? Things like Webmin make it easy to configure a machine so the majority of users (so called unwashed masses - heh, the "masses" probably think the same thing about computer geeks) can get can back to what they were doing. Having a better set of config files is a fine idea, especially for sysadmins, but will the majority of regular users want to fiddle with that? No.

Well, I have friends who wouldn't agree with you, though I tend to think of myself as "a bludgeoned and retreating idealist willing to make some compromises".

E smith is your answer.

[Malcontent]

Please check this out.

It's the esmith configuration schema.

They have already solved this problem in a neat way.

Re:XFree 86 Configuration..

[TeknoHog]

My pipedream (gnome specific): Imagine a panel inside gnomecc named gnomecc->Desktop->Display which allowed you to reconfigure X without restarting the session or the X server. It'd take more than just a standardized way of changing the config file to do this.

There is already the xvidtune utility (included in XFree86) that lets you change some of the video parameters on the fly. Therefore what you describe should be quite possible.

Avoiding configuration

[Animats]

Putting a GUI front end on a mess merely results in a less understandable mess. The problem needs to be fixed. Here's how:

• Avoid configuration wherever possible. If something can be discovered, it shouldn't have to be configured. This applies to most pathnames. Searching directory trees is faster than it used to be. Long-term, a database of where stuff is is needed. Note that Microsoft is going that way.
• The PATH approach to program finding has to go. It leads to everything being in a small number of /bin directories. Each installed program needs to be in a separate directory subtree. /bin directories should consist only of links.
• Separate preferences from configuration. Individual user preferences should be disposable; if they're missing, the defaults apply. The old MacOS worked that way.
• There are several categories of programs. Ordinary applications should be installed in a separate directory subtree for each program. The subtrees should be organized something like the Windows approach of "program files/vendor name/app name/". For ordinary applications, no interaction is allowed between apps at install time, and a standard installer should enforce this. That way, installing A can't mess up B.
• Programs that aren't "ordinary applications" should be given some identifying name, so users are warned that they may mess things up. This will discourage unnecessary tweaking during installs. All games, for example, should be ordinary applications. Programs that mess up other programs' installs should require more privileges to install than those that don't.
• Hardware should be recognized via discovery, period. Support for hardware that can't be discovered should be dropped.

We will now hear from all the people who insist that "they have to do wierd thing X because it's (traditional) (k00l) (needed to support their 386 machine)". They're wrong. Someone needs to be a hardass and fix this thing.

No they arn't.

[autopr0n]

Please, that's the bigest lie in all of this. The idea that what you are doing in Unix is "hard." It's not. Once you learn the config files, getting things set up takes just a few seconds. I can ad a new domain to my DNS server in just a minute or two (just copy a file and modify as needed). I can set up a new share in Samba's web-based config quickly and secure it via IP range or windows username (I would assume it would take just a copy/paste/modify in the config files as well).

I didn't have much trouble handling the apache config either for adding Tomcat on windows or scoop on linux either. (in the case of scoop, all I had to do was copy a pre-genererated text file into my apache conf file).

It's not hard, it just takes time to learn all these damn config formats.

A standard system for editing would go a long way to making things easier. People could spend time learning about how things really work rather then how lazy the author of the software was in writing their config parsers.

XML+some schemas for common things would really help. In the case of Scoop I mentioned, I wouldn't have even had to do the useless and idiotic step of copying data one from one file to another manualy... it could be done by Make via a standard XML parsing API, or perhaps a standard config library.

Well.

[autopr0n]

I wanted to use Linux as a server, as it seemed to be the 'thing to do', as well as the fact that run all that cool server code that was only available for Linux, as well as get experience running a Linux box.

Having a Linux box has allowed me to have a mail/DNS server with a very high uptime, as well as a (non-porn :P) scoop site.

I'm not saying it wasn't fun, but it's not something I want to go through all the time... that's why autopr0n.com uses tomcat on windows with an M$ access file rather then on Linux with Postgresql or MySQL or something. That would probably be a better solution (and eventually I'll migrate, or at least move up to a pirated copy of SQLServer :P), but it's just a lot less of a hassle to set those things up on windows.

sendmail conspericy.

[autopr0n]

If you go to sendmail.com you'll see they sell a 'pro' version with a nice GUI. I'm pretty sure they keep the text config complex for a reason :P

LibCFG: A configuration management API

[Moderation abuser]

http://www.yelm.freeserve.co.uk/libcfg/

XML is not the solution to the problem, XML is just a file format, many different file parsers would still be required for XML just as there are for other file formats.

No what's required is a simple to use *API* which performs the configuration management for you. Whether the configuration information is then stored in a colon separated file, XML file, DB database, SQL database or LDAP server is irrelevant, it can still be read, modified, saved by the application.

LibCFG uses flat text files by default but that's only because nobody's written LDAP, SQL, DBM sections for it.

Note, the license for the library is important. For a library like this to become useful, *everything* has to be able to use it, the most expensive commercial software must be able to include and use the code as well as GNU GPL based software.

Re:LibCFG: A configuration management API

[MikeBabcock]

There is a Gnome-endorsed configuration system out there as well with at least XML and DB backends.

XML+Schemas/DTDs+XLST

[autopr0n]

Actually, you wouldn't need two separate XML files. XML already has the capacity to 'describe' itself via DTDs, as well as XML schemas.

Basically they let you write a description of what your XML document contains (indeed, you need those if you want to have a "valid" XML doc). A standard XML Schema could be developed for "Linux Config", and developers could derive from that any special config options they would need.

You could also use XLST to convert from "app space" where all the tags are descriptive to the domain of the problem (<TCPport port="80" /> to <inetd:setport port="80" service="apache"> people could edit it in either form and have the changes roll back between the two. That way you could have a GUI for apache and inetd and make changes in either and automatically have it propagate all over the system, check for conflicts, etc.

Human readable?

[autopr0n]

type this at your command line for me real quick:

cat /etc/sendmail.cf | more

You call that human readable? Hah!

Anyway, humans can read anything a computer can, with practice, including binary. It's just a question of how long it takes them to learn. XML does not take that much time.

Re:Human readable?

[k8to]

sendmail.cf is not designed to be readable
it's generated by m4 macros.
Bad example.

XML is not really readable either, it's nested
tree structures with lots of encumbered syntax

-josh

Well..

[autopr0n]

part of it is machine readablity. You can parse the second a lot easier if you already have an XML parser. That lets you do things like let installs of software modify the config of other things to include them (this would be great with Apache). As well as GUI things. How would a gui know to associate that comment with that option.

Also, you probably wouldn't just use <option /> tags, thats' just bad/lazy XML. Probably more like this

<logon>
&<!- this options specifies a script to run for every user -->
<scriptForAllUsers file="U.bat">
</logon>


Then in your DTD or Shema you would define what it does. programicaly, so that the comment would show up in a GUI as well.

Lets increase the intelligence to posts ratio...

[BasharTeg]

Those of you who are saying that FreeBSD runs only on the Intel platform, you are totally ignorant, and you should proceed to shut your mouth at this point. FreeBSD runs on Alpha, and has either single user or multi-user boots on Sparc, PPC, IA-64, etc.

The number of machine dependant lines in the 4.4BSD kernel is 39,634 out of 202,251 or about 19.5%. Once the hardware specific areas of a platform are abstracted, the rest of the system (for the most part) works properly. (*Broad generalization*)

As for the subject at hand, Darwin IS based on the Mach microkernel, with FreeBSD subsystems and libraries, and NetBSD userland. However, for those of you ignorant of the Mach microkernel, the short story is, it was partially based on 4.2BSD. So as much as ignorant Linux zealots want to try to make some kind of meaningless point that "Darwin is based on Mach not BSD.", from the perspective of the informed, that's the stupidest statement you could possibly make. No matter how you want to look at it, Darwin IS BSD. As several Apple representitives said at this year's BSDCon, there are currently FAR more deployed BSD boxes than Linux boxes. As for the FreeBSD vs NetBSD argument, you have to decide for yourself whether "based on" means "subsystems and libraries", or "userland utilities".

From apple.com:

"FreeBSD is one of the ongoing BSD development efforts and is our primary reference platform for current BSD kernel development."

"we try to keep Darwin as compatible as possible with FreeBSD (our BSD reference platform)"

"We already synchronize our code periodically with NetBSD for most of our user commands, and we will soon be doing the same with FreeBSD for our libraries."

Schemas for every program

[ttfkam]

I disagree. No two programs have the same set of configuration needs. Two web servers can have very different configuration options depending upon their focus. While I agree that DTDs are a pain in the ass (DTD entities == XML preprocessor == pain), using something like XML Schema or RELAX-NG would give all the necessary metadata a configuration manager would need. Far more so than some one size fits all or (only slightly better) five sizes fit all.

If the UI can see that you need parameters A, B, and C, there is sufficient description data for the parameter (relatively simple with judicious use of XML namespaces), and can tell that A is supposed to be an integer from 0-100, B is supposed to me a valid email address, and C is an enumerated list of possible values (all possible with existing XML technology -- no need to reinvent the wheel), you will have your UI while allowing a sufficient amount of individualism by the programmer.

And you don't need pre-defined schemas to make object mappings. There is previous work in this area already for the generation of object definitions from XML metadata. Of course this is of most use to the actual programs. The config manager needs to process the metadata in addition to the instance document (the config file).

Come to think of it, a schema document describing UI manager "hints" might not be such a bad idea. It wouldn't tell it what each config entails, but could provide info on how that is to be presented. Nice idea!

Re: LDAP. LDAP is an interface, nothing more. The LDAP spec says nothing about how the datastore should be implemented. In short, you could have your config data in any form you wanted and, as long as it was consistent and documented, publish the information through an LDAP-compliant gateway process.

** mouth waters at the possibilities **

Re:Wrong, wrong!

[RevCheswollen]

Um, the practice of keeping all configuration in /etc is a recent innovation - attributable directly to the linux community (see LFSS /HFSS).

Lame unices like the dreaded HP-UX still have configuration files scattered randomly through the folder hierarchy, binaries in /etc, non-optional drivers in /opt, and so forth ad nauseum.

Linux is also moving to a standard of /etc/*.conf for individual configuration files, and /etc/*/* subdirectories (like /etc/httpd or /etc/apache) for multi-file configurations. Note the recent change from /etc/conf.modules to /etc/modules.conf, and the increasingly popular ln -s /etc/nfs.conf /etc/exports.

But you're right that XML configuration is overkill and ridiculously verbose for most purposes. An XML-driven STATELESS configurator would be great, and would allow multiple front-ends to cleanly interoperate, but the basic text configuration files should remain as simple as possible.

XML is not the solution.

[Zygo]

I think that people who moan that Unix should be more like Windows or Mac don't really understand Unix, or Windows, or Mac. XML is part of the problem--people who think XML is the solution, or even neutral, are suffering from a specific kind of delusion which I'll cover in more detail later.

Advocates of auto-detection instead of manual configuration I'll deal with right now: Applications in certain problem domains require explicit human-supplied configuration data. This is unavoidable in any application that has to implement some choice of policy, especially security policy. There are some things you can't ever autodetect, so if you can't configure the application, then you need many different applications, each hard-wired for one possible desirable configuration. Certainly a lot of things that should be auto-detected are not, but auto-detection will not eliminate the configuration problem.

Windows and Mac are platforms controlled by a single vendor. That vendor has a _lot_ of influence over how applications are designed for that platform. When the vendor designs and implements the platform, the vendor will (intentionally or otherwise) reward applications with an architecture consistent with the vendor's design, and penalize those that don't. Applications tend to comply with the vendor architecture because the cost of non-compliance is greater than the cost of compliance.

Unix is not a platform controlled by a single vendor. Heck, it's not even a platform; it's a bunch of little standards that were chosen arbitrarily and stitched together. Linux, FreeBSD, SysV, BSD, Solaris, QNX, HP/UX, etc. are not Unix platforms--they are platforms compatible with Unix, implementing parts of the SUS, ANSI, and POSIX standards, with varying degrees of completeness and success. As such, there is no reward for making an application's architecture consistent with the OS vendor's design; contrariwise, there is a compelling reward for an OS vendor to make an OS architecture consistent with a desirable class of application.

The difference is critical for understanding the configuration file nightmares of both kinds of platform. Windows has a really awful configuration mechanism that nobody would ever use if they weren't compelled to; however, certain parts of Windows are only conveniently accessible via this configuration file format, easy (if not reliable) methods to access the format are provided to all developers and available on all end-user machines, and the configuration schema does not define a lot of policy to get in the way of extensions (need new schema? Add a new DLL!). The Mac has a different architecture and mechanism with the same consequences.

On the other hand, applications that run on Unix are, in whole or in part, designed for "other" systems. Applications that were designed and written exclusively on Unix-like platform X eventually get ported to some other Unix-like platform Y--those that don't tend to be eventually replaced by similar applications that do. When this happens, the resulting application is an alien on both platforms. The application must find everything it needs on each target platform, or it must bring with it those things that are missing from some target platforms. This means that the application will have translation or interpretation layers (all C apps have at least one translation layer in the C compiler) and replacements for missing functionality in the target platform.

Now for the delusions. New code written in a language with link-level compatibility with a free, small, fast, easy to use in code, portable, robust configuration mechanism library will have no problems. Note that the status quo is already as free, small, fast, easy to use in code, portable, and robust as it needs to be for applications that already use it, so alternatives which are not all of these do not merit discussion.

However, there are constraints imposed on this hypothetical library by legacy code:

1. It would have to be already installed or available on all Unix-like systems with bindings to all Unix-like languages, which rules out nearly everything invented after 1978. Otherwise, every application (and language!) must bring either a copy of the library or their own code. Strangely enough, "open file," "read text line," and "lexically analyze this string" primitives are implemented in nearly all Unix-like languages, and interactive text file editors are shipped on nearly all Unix-like systems...
2. It would have to be as small and fast as fgets+scanf, or a lex+yacc parser, otherwise it will hurt performance or have a large footprint, and people who care about performance or size won't use it. XML can't do this unless you drop the i18n and character entity and encoding support--at which point it's not really XML any more.
3. It will have to use small ASCII text files for persistent storage, otherwise you have to implement and force everyone to use either your own editing mechanism (aka "a control panel DLL"), or you implement and force everyone to use replacements for Unix filesystem-crawling tools which allow you to selectively access the data (aka "the registry and regedit", or "replacements for cvs, vi, find and grep")--both of which violate the "once and only once" rule since these facilities exist already in Unix and they are used for this purpose.
4. It will have to be used directly by the actual applications--otherwise you violate the "once and only once" rule by having either two configuration files and two applications (the real application, and a configuration application), or two configuration file handling mechanisms in one application (for old-style and new-style config files).

It might be possible, given overwhelming demand, to convince maintainers of significant mature legacy code to adopt the new configuration mechanism, if the XML-based mechanism is more widely ported and better debugged than their own configuration-management code. Good luck, and be prepared to do the work yourself many, many times over until you either change their minds, or they leave the project and you're stuck maintaining the code by default.

Note that other standardization initiatives, like FHS or the LSB, involve relatively superficial changes which application maintainers were probably willing to make anyway. The FHS and LSB ask application maintainers to change the parameters to a few system calls without affecting the architecture of any part of the application or the semantics of its operation. For example, the application may open()s a file, but after the FHS is implemented that file has a different name. Both names are constant strings, the file is still opened for reading, and it still has the same contents with the same syntax and semantics. A one-line change and a recompile brings the application into compliance, and in an emergency, a symlink will make the application work.

Other initiatives, such as DNS and PAM, are usually facilities that are not intrinsic to applications. Host name resolution or user access verification involve resources shared by many applications, and there is a real technical penalty for non-compliance.

Changing the way applications handle configuration is a large architectural change--you're asking people to replace open(), read(), close(), and some code which is intrinsic to the application with new functions that have different semantics, and may even have to be called at different times. Expect resistance.

The only hope of this ever happening in real life is for a Linux distributor to decide to implement it, and contribute the work to achieve it. Said vendor will have to hack and slash their way through hundreds if not thousands of packages to bring them into compliance. Once all of that is done, all the upstream package maintainers will have to integrate the changes--or all the other Linux vendors will have to repeat the hack-and-slash job.

Re:A better config file format than XML

[be-fan]

Hmm, maybe its a matter of preference. I find the XML style easier as it is more word based rather than symbol based. Also, I think the structure of the file is much more readily apparent with XML than with standard UNIX config files. Of course, I'm one of those hierachical thinkers (you could probably describe my thought process in XML...) so it might just be me...