Slashdot Mirror
How to Fix the Unix Configuration Nightmare
jacoplane writes: "There's an interesting article on freshmeat talking how sorting out some kind of standard for configuration could really help Unix systems could be more user friendly. The article points out that since Apple has managed to build a quite usable system on top of NetBSD, it should be doable to do the same for open-source interfaces."
There are situations were I've wondered why they went off and created their own configuration file format just for that one application, but that's not too common. The problem with a universal format is that each application need to do different things. Typing ALL apps to one format would in many cases make configuration changes more difficult.
i.e. instead of changing one thing, you have to make the change for every interface/terminal/etc it is running on. Just an example mind you.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Have a look. webmin
Yes, it's based off of FreeBSD. FreeBSD 3.2-RELEASE IIRC.
yeah, OS X is based on FreeBSD... and yes the person was talking about OS X in the blurb when he mentioned Apple.
and yes OS X is freakin' nice. leave it to Apple to do what no one else has/can.
flames? who cares? look at my email addy.
JavaBeans does everything that this article wants now. It suports introspection, special field editors... Then you can build general purpose editors for any type of user. Couple this with javaspaces, for storage. JavaBeans is a shipping technology, but JavaSpaces is still a bit dodgy last time I looked (6 months ago.) Sun have spent alot of time getting this right, there is no good reason te reinvent it.
I am the NUL and the DEL, the beginning and the end.
As it has been already said in a comment on [fm], the answer is GConf:
it has the ability of configuring apps and has the option of using a DB or XML as a backend.
Start porting!
The complaints about Webmin is that it isn't perfect. It doesn't solve the desire for a universal config mechanism or encourage the editing of files directly. OK. It doesn't. Yet, it exists now and is a lifesaver when using multiple UNIXs. Anyone who wants to know what any Webmin module is doing probably also knows how to use find -cmin, ls -lart, or read the module itself since they tend to be in Perl.
When a universal config comes along, I'll use it and guess what -- it'll work with WebMin.
Webmin might not be the right thing, but it is good enough for a vast number of uses.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
A brilliant article, and a subject which I have tried before (with little success) to broach with various OpenSource developers. Unfortunately the comments demonstrate the lack of understanding that is all too common in the free software world: an inability to distinguish between apis/formats and software implementations; and an unwillingness to move to a more user friendly system.
GConf and Webmin, for example, are not solutions that address the problem posed by the article. Other comments completely lose the plot: this is about a common configuration format which can use metadata to provide for configuring just about any application, it is not about creating an all-encompassing configuration that will give settings to all applications everywhere for all time.
Gconf is half of the solution at best. It provides a library (common abstraction) for accessing the data. But an application still has to have explicit knowledge of the data in order to act on it. A configuration framework describes this knowledge to allow any compliant application to act on it.
Sometimes an example is worth 1000 pictures (and hence about 1000000 words), so consider a simple packet filtering application with an ACCEPT list and a DENY list, which can listen on several interfaces.
Such an application would have a meta-data configuration file (say /etc/conf/meta/filter.cfg) which specifies:
section name=interface count=multiple type=ipv4 ; section name=accept count=multiple type=cidr ; section name=dent count=multiple type=cidr. Now any application can access the packet filter's meta-data file, and the real configuration file (say /etc/conf/filter.cfg), and understand the meaning, to a degree, of the configuration. The types "ipv4" and "cidr" will have to be primitives for the configuration framework.
Thus a general configuration application could present a tree which has three branches (interface, accept, deny), each of which displays a screen with a list. Appropriate entries can then be added, modified or removed, reconfiguring the packet filter without the configuration application having any prior knowledge of the filter's configuration or requirements.
With adequate forethough (for the categorisation and types required) such a system can be extremely powerful. A single configuration library can be used by the application, as well as GUI and CLI configuration clients. You could easily have a command in a script: config pktfltr add -section interfaces -value 1.2.3.4 .
A common library like this can also provide powerful features that a lot of applications lack, like local (user) settings which override global defaults. The library can maintain global, group and user configurations for each application, and getting/setting preferences will take the overlays into account.
Implementing such a system would also provide an ideal opportunity to introduce configuration versioning. CVS (for example, or arch, which includes directory versioning) could be used to maintain various versions of the configuration files in /etc/conf.
Arguments about diversity are largely moot. The data format is independant of the data in any well-written program. .INI files, Bind-style files, XML files and SAMBA files can all represent the same information, but they look different. Some are arguably more efficient in some situations. Even the feared Windows registry has the same functionality ... it just makes the (fatal) error of storing everything in a mostly inseparable datastore (sortof like an /etc/conf directory in which you can't treat the various files separately).
This is a suggestion which needs to be followed through, and supported by the community.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
"Ernest Prabhakar of Apple Computer informed attendees at the USENIX BSD Conference earlier this week that BSD (upon which Mac OS X is based) is now "three times more popular" than Linux on desktop computers."
http://www.osopinion.com/perl/story/16382.html
Other points in the article that scream XML to me:
* A key element would be the configuration format description file. This would list the configuration options for a given piece of software, giving for each one the name, type (boolean, list, string, etc.), options, category (for subsections within the config), and help text (short and long).
This looks very much like XML Schema to me - it can specify all these data types, including enumerations, and a schema-aware XML editor (eg XML Spy in the Wondows world - anyone know the best Linux option?) will prompt you with help-lists of valid elements and attributes, or list of enumerated values. Doesn't do help text by default, but has an expansion mechanism (xsd:appinfo) for adding application-specific to be added to a schema.
* It all needs to be language, distribution, and operating system neutral, so as to avoid turning off any potential software developers who might find it useful.
I don't know of any programming language or O/S that doesn't have an XML Parser (many don't yet have XML Schema support, but if you stay off specifying default values you don't need that at run time). And XML is good for natural languages - UTF 8 or UTF 16 from start.
There aren't many formats that are equally machine and human readable, even fewer that allow document struture and data typing, and still fewer that have open or free implementations on practically all platforms.
Yes, I know this is a comment on freshmeat (I wrote it).. but it pains me to see so much incorrect/incomplete information here that people just believe for lack of experience with OS X.
/etc (which is a symlink to /private/etc):
/etc and puts it in a
/etc style formats
/System/
/Library/Preferences, Network-wide app prefs (only
/Network/Library/Preferences, and User app prefs go
/etc, as nothing there is meant
---
MacOS X's solution to this problem is a couple levels
deep. There are three (?!) systems for configuration:
(1)
It's still there, used for a few things. But not much.
More or less just stuff that runs before everything else,
or stuff that apple doesn't have much control over, like
Apache.
(2) Netinfo database:
basically takes a lot of stuff out of
"registry", this registry is accessible over a network
connection for remote administration and such. You
can also do some funky server/client stuff with this too.
There's a command line interface to it that lets you
read/change 'keys' and also dump out certain keys in
well known
(3) Property Lists (.plist):
This is the way that just about all configurable
information in MacOS X is stored. There are a couple
different kinds of plists that the Core Foundation
libraries understand, but the most prevalent of which is
the XMLified version, the others are either vestigal or
are used primarily for localization of strings. Every
application (well, bundled app) on the system has an
info.plist file that has application settings that the OS
reads from on app startup, but you can also put your
own tags in there. Applications also have the option of
putting their own tags in the Resources subdirectory in
their bundle, which is the preferred way. OS specific
preferences (requires admin privs) go in
Library/Preferences, System-wide app preferences go
in
applicable if you have OS X Server on the network) are
in
in ~/Library/Preferences.
The Core Foundation framework manages more or less
all of the work. They'll find the plist for you, let you
work with the data structures, and serialize/deserialize
it to plist format. It's really quite nice to use. Just
about all of the data structures in Core Foundation are
serializable, even the property list data structures, so
sometimes you'll see property lists with serialized
property lists inside them.
As far as unified configuration interfaces go, there
really isn't a need for any in Mac OS X.
You're on your own with
to be novice-tweaked anyways (In OS X).
There is a GUI NetInfo Manager and command line tools
to configure the stuff in that database (nothing a
novice needs to touch).
Configuring the plists is primarily done inside the
applications that you use (since it's so easy for the
developer to use Core Foundation to read/write
property lists), there are no separate configuration
programs, but there are command line tools for working
with plists. If you're using the developers kit, you get a
little (not very poweful and pretty inefficient) program
to edit plist files graphically, but otherwise you either
do it by hand with a text editor or the command line
tools and risk breaking things or let the apps take care
of it themself.
BTW, all of the Core Foundation stuff is available in two
flavors: C, and Objective C. The C libraries are pretty
straightforward and work in all situations, but if you're
writing an app using the Cocoa framework in Obj C, it's
way easier.. though there are a few little peculiarities
about the Obj C implementations. The bonus is that
you can use both at the same time, and Core
Foundation types / Cocoa instances are interchangable
by some voodoo that Apple did. If the Cocoa
implementation doesn't work right (As with serializing a
plist that has NSNumber Boolean types), you just use
the Core Foundation implementation and a typecast..
voila, one line of code changed and everything is
happy.
First, XML is not a silver bullet. The thing this article screamed was not "XML" but "API". Define the application programming interface for a library, and stop thinking that each application must directly access the native data.
Second, XML Schema is a definition for an XML language, not a document described in a particular XML language. XML is a meta-language, you have to MAKE a dialect using a schema or DTD first. XML Schema could not, for example, include primitive types for IP addresses (very common in config files), or validation constraints for integer values.
Third, XML parsers are no-where near as ubiquotous as you would like to believe. To date there are no (zero, zip, nada, nothing) fully compliant XML parsers. Read the XML spec. carefully and understand just how many IFs and BUTs there are to creating a parser. Xerces and MS's parsers are the closest, but still have a number of problems. Expat-derived parsers happily handle a large subset of XML, but not everything. A vast number of parsers out there ignore the DTD, which means they cannot handle entities (which is a big issue).
Finally, there is nothing intrinsically human-readable about XML. This is common bullshit sprouted by those who haven't actually considered the requirements for human understanding. A simple .ini-style line delimited key=value is the most readable and understandable (from a psychological viewpoint) configuration format.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
These is no technical requirement for dot files. If the entire community decided to standardise on ~/etc, it would happen.
The general theory of /etc, /usr/etc and /usr/local/etc is quite simple. System configuration goes into all of these directories (no user config). Configuration needed at bootup goes into /etc. Configuration needed for OS-default applications goes in /usr/etc, which is available once /usr (containing the OS-default applications) is mounted. Configuration for application you have installed specific to your system (stuff that should be in /usr/local) goes into /usr/local/etc.
The article is not proposing a central configuration file, but a standard for configuration. That would be like saying all application MUST store their configurations in a specific directory, and they must all use the same file format (say XML). Storing all configuration files in the same directory is not the same as storing all configuration in one file.
The benefit of a common location and format is that every app or admin knows where to look for files, and how to interpret them. The structure of each file is identical, so you don't need to know if you use #, //, ' or $ for comments, and whether each line must start with a TAB or end with a ; . What each file says is still different, its just how it says it. Currently trying to work with config files on Linux is like being in a building and trying to talk to people, but everyone is in a different room and talking a different language. Standardize the language and the location, and you only have to content with understanding what is being said.
Incidently, system.dat is not a security problem. In Unix terms that file is r/w by root only, and access only occurs through a kernel-level API. So "cracking" system.dat is no more or less hard than "cracking" shadow.passwd .
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
One other thing--OS X has its own file directory structure--User for usr, for instance, and a few other changes. Functionally, this is FreeBSD, but Apple's directory parallels allow mere mortals to grasp its naming. Doesn't make for smooth going for experienced UNIX heads, but a trip to the Terminal shows that what they need is indeed there. There are just symlinks to the common directories, I believe.
OS X is also one of the very, very few *nix systems that is CASE-INSENSITIVE.
Vos teneo officium eram periculosus ut vos recipero is.
that is UNIX. Take it or leave it.
/etc anyway?
/etc and either see a file or directory named something along the lines of what I'm configuring. Get it up and if I don't know it well, either quickly read the comments or the blah.conf man page... configured... working... I'm done in here for a while until I think of some other cool thing to do.
I agree. I'm reading all this wondering why people are complaining so much. How much time do people spend in
I need to get something set up, I go to
When Win95 went to a single, non-text config file (registry), I instantly thought this was going to be a really bad move. And it proved to be. Registry rot is incredible. Anyone ever notice that if you reinstall Windows about once every year or so, you get a massive boost in performance? I installed 98 the other day (just for D2:LOD, Starcraft and RainbowSix), it boots (PII-300, 512MB) in about 30 seconds and shuts down in 1-2 seconds. Prior to this, 98 was taking an age, *if* it would shut down at all, and I had all sorts of stability problems. BTW, I put the C: on a 700MB partition and used a D: of 2.5GB for the games. I burn the C: to a CDR and when things get bad, just dd it straight back to hda1.
My point is (back to the OSes that are more fun than a Frost Nova up Andariels arse), *I* want to mess with those config files, rather than have some app(s) messing with one really important file.
I like it too.
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
Quick example taken from the above docs:
<?xml version="1.0" encoding="UTF-8"?>t .dtd"><dict>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyLis
<key>Year Of Birth</key>
<integer>1965</integer>
<key>Pets Names</key>
<array/>
<key>Picture</key>
<data>
PEKBpYGlmYFCPA==
</data>
<key>City of Birth</key>
<string>Springfield</string>
<key>Name</key>
<string>John Doe</string>
<key>Kids Names</key>
<array>
<string>John</string>
<string>Kyra</string>
</array>
</dict>
</plist>
Note: The <array/> tag is specifying an empty array.
OS X is based on Darwin and Darwin takes bits and pieces from all three *BSD, Mach, GNU (compiler) and a lot of other projects. E.g. most of the userland stuff is taken from NetBSD, you can read it up in the Darwin FAQ. Or look at the Darwin-contributors page which lists them all in alphabetical order.
So saying "It's based on FreeBSD" is as wrong/as correct as saying "It's based on NetBSD".
But since FreeBSD's J. Hubbard now works for Apple and FreeBSD doesn't run on the range of machines Mac OS X supports yet (but NetBSD does) expect OS X's roots to FreeBSD to be promoted more than its roots to NetBSD... (sigh).
NetInfo sits in places of the same sort of things that NIS, NIS+ and DCE registries can replace: passwd, aliases, exports, fstab, hosts, group, printcap, and stuff like that.
Configuration files are stored in ~/Libraries/Preferences. Apple recommends the use of XML, but there are many preferences files that are in more traditional MacOS resource format. And then there's Mozilla....
So, yes, technically your preferences don't go in dotfiles. And it is nice that they're all in one directory which has an obvious purpose. But each application can still choose its own file format.
Please check this out.
It's the esmith configuration schema.
They have already solved this problem in a neat way.
War is necrophilia.
Those of you who are saying that FreeBSD runs only on the Intel platform, you are totally ignorant, and you should proceed to shut your mouth at this point. FreeBSD runs on Alpha, and has either single user or multi-user boots on Sparc, PPC, IA-64, etc.
The number of machine dependant lines in the 4.4BSD kernel is 39,634 out of 202,251 or about 19.5%. Once the hardware specific areas of a platform are abstracted, the rest of the system (for the most part) works properly. (*Broad generalization*)
As for the subject at hand, Darwin IS based on the Mach microkernel, with FreeBSD subsystems and libraries, and NetBSD userland. However, for those of you ignorant of the Mach microkernel, the short story is, it was partially based on 4.2BSD. So as much as ignorant Linux zealots want to try to make some kind of meaningless point that "Darwin is based on Mach not BSD.", from the perspective of the informed, that's the stupidest statement you could possibly make. No matter how you want to look at it, Darwin IS BSD. As several Apple representitives said at this year's BSDCon, there are currently FAR more deployed BSD boxes than Linux boxes. As for the FreeBSD vs NetBSD argument, you have to decide for yourself whether "based on" means "subsystems and libraries", or "userland utilities".
From apple.com:
"FreeBSD is one of the ongoing BSD development efforts and is our primary reference platform for current BSD kernel development."
"we try to keep Darwin as compatible as possible with FreeBSD (our BSD reference platform)"
"We already synchronize our code periodically with NetBSD for most of our user commands, and we will soon be doing the same with FreeBSD for our libraries."
Um, the practice of keeping all configuration in /etc is a recent innovation - attributable directly to the linux community (see LFSS /HFSS).
/etc, non-optional drivers in /opt, and so forth ad nauseum.
/etc/*.conf for individual configuration files, and /etc/*/* subdirectories (like /etc/httpd or /etc/apache) for multi-file configurations. Note the recent change from /etc/conf.modules to /etc/modules.conf, and the increasingly popular ln -s /etc/nfs.conf /etc/exports.
Lame unices like the dreaded HP-UX still have configuration files scattered randomly through the folder hierarchy, binaries in
Linux is also moving to a standard of
But you're right that XML configuration is overkill and ridiculously verbose for most purposes. An XML-driven STATELESS configurator would be great, and would allow multiple front-ends to cleanly interoperate, but the basic text configuration files should remain as simple as possible.