Slashdot Mirror
What Makes A UNIX System UNIX?
ian asks: "Since there are now so many different flavors of UNIX out there (Linux, Free BSD, AIX, Solaris, AT&T UNIX, etc...) what do they all have in common that lets these all be called UNIX? Programs written for one flavor of UNIX typically cannot be ported to another without considerable effort. The features offered by the different implementations vary widely: some are more secure than others, some cluster better than others, some offer journaling file systems, some are more robust. The differences between the different kinds of UNIX seem to be as great as the differences between any particular implementation and other OSs. Could one port all the standard command line utilities to NT, clone one or two of the popular shells, set up the directory structure in the standard UNIX layout and call it Microsoft UNIX?"
Provided you are working high level, cross-platform APIs ( which should behave the same on all UNIXs ), and you don't try to code to several different APIs [1] then I don't see why it should be so hard. There are differences in the way different UNIXs handle many things [2] but there are toolkits like Qt and glib that take care of the low-level portability stuff and provide their own data-types and functions.
Just for fun, I grepped for #ifdef directives in krn and there were hardly any ( and the ones I saw weren't about portability issues ).
[1] this is why the vim code is so complex -- they are simoultaneously coding for motif, gtk, athena, ncurses, and win32 without using any high level portability tools ( because there's none that are portable across all target platforms ).
[2] An example -- many C++ compilers still have either nonfunctional or incomplete STL implementations. Another example -- some UNIXs ship with different DBMs ( ndbm,dbm,gdbm ).
Well, not totally, but there are so many errors and misconceptions in the question I'm not sure how to answer.
Since there are now so many different flavors of UNIX out there (Linux, Free BSD, AIX, Solaris, AT&T UNIX, etc...) what do they all have in common that lets these all be called UNIX? Programs written for one flavor of UNIX typically cannot be ported to another without considerable effort.
Wrong. Well-written programs can be easily ported from one unix to another. The problem is that many programs are written by people who have a relatively poor understanding of how to write portable code, and no idea how to distinguish the C standard, the POSIX standard, stuff that's not POSIX but almost universally available on Unix and Unix-like systems, and stuff that is specific to a particular implementation.
The features offered by the different implementations vary widely: some are more secure than others, some cluster better than others, some offer journaling file systems, some are more robust.
None of which has anything to with writing portable code. For example, the C interface to the file system (fopen(), fprintf()) and posix interface (open(), read(), write()) make the underlying file system transparent. Of course, if you're writing a tool to manage a cluster, it's going to be implementation specific.
The differences between the different kinds of UNIX seem to be as great as the differences between any particular implementation and other OSs.
Sorry, you've obviously never actually had to write programs that worked on a wide variety of systems. I used to work on a large product that ran on a variety of Unix systems, OpenVMS, and NT. IIRC, the only difference between the Unices was different options to mmap() and dealing with an old version of SunOS that didn't implement POSIX signals quite correctly. (Of course, there were many differences to the system headers, but those are supplied by the vendor, and are there precisely so that you don't have to worry about internal differences.) FWIW, the OpenVMS version was pretty easy, because even the the system calls were totally different, most of the concepts mapped pretty well. NT, on the otherhand, was a complete pain in the ass.
Now obviously, one *can* write code that is specific to a particular Unix implementation. And noone will argue that at the admin level, things are aren't chaos.
Could one port all the standard command line utilities to NT, clone one or two of the popular shells, set up the directory structure in the standard UNIX layout and call it Microsoft UNIX?
No, because you haven't changed the syscall interface. Cygwin comes a lot closer, but was also a lot more work than what you've proposed.
You can find many of the popular unix tools for windows at http://www.gnusoftware.com.
...
You'll find bash, grep, cygwin, emacs
They've got some nice things there, look at geoshell's screenshots, you wouldn't believe it's windows.
Anything that tightly restricts the evolution, expandibility, and scope of two systems by tying them to each other does not "feel right."
You are completely correct in that statement, but it has nothing to do with C or Unix. The evolution, expandibility, and scope of the two systems in question has been in no way restricted. Unix has continued to expand and evolve well beyond the original scope of the C language as it eventually was defined in the ANSI standard. Likewise, C is considered the most portable language ever written, with implementations for just about every platform that has a compiler, and it has continued to evolve and be expanded past the original Unix API. It has even been expanded to include object support in C++, which was in turn used to write BeOS, a fact I'm sure you are no doubt aware of, given your handle.
So, if you have a legitimate complaint, by all means voice it, but otherwise, keep the FUD to yourself, K?
(Note to moderators: Before moderating this down as "Flamebait", check out this guy's posting history. He makes rabid Linux supporters seem tame by comparison.)
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Who gives a rip about the command line utilities - just make NT as stable as Unix, and you can call it Grandma Pearl's Home-Brewed Operating System with Extra Apples, for all I care!
Seriously, while you're at it, you should ask exactly what Windows is. It's in the same boat - there's several flavors of a single OS that really don't have much in common. Windows CE and Windows NT don't share much except a start button, when it comes down to brass tacks.
What's your damage, Heather?
This is just right out of the top of my memory,
so I could be mistaken, but as far as I remember
you have to pay someone (The Open Group?)
for the right to call something Unix.
It involves certification.
I've heard that it needs to have evolved from
the UNIX-codebase, but this doesn't make very
much sense, as it would be impossible to
write a new Unix from scratch, but still conforming to the Posix-standards, and being
source-compatible with the tradional Unix-variants.
Perhaps someone can enlighten me here, but
would it be possible to pay that organization
for certification of Linux?
Not that I think it would matter, because Linux
is now bigger than Unix anyway, and it seems very
important for other Unixvariants to include some
sort of compatibility layer to be able to run
Linuxbinaries.
I did think it mattered 2 years ago though.
Recently having ported software between Linux, FreeBSD and NT, I have developed my own idea of what a ``UNIX'' is. It's simplicity most of all. Everything is a file, and most calls that work on file descriptors work on _all_ file descriptors. You can make a select() call on a file, a pipe, and a socket. This is impossible on NT, because files are HANDLEs, sockets are SOCKETs, HANDLEs from anonymous pipes cannot be used for select() like calls (only named pipes there). Why this diversity in the API ?
Being a long time Unix and Windows developer, I don't see any advantage to the developer to having one ioctl() that works with all devices. While it makes sense on the kernel (one api to export), it is total nonsense for the end developer.
type 'man ioctl' : There is absolute no useful information given. It's a generic function and the developer must explictly know what kind of device he is talking to and somehow find the right parameter values to pass in to make it do what he wants. ioctl documentation can never be complete.
I would argue that it's better to have more functions that are strongly typed and can be documented seperately. If the developer want's abstration, let him build it into his application - don't force it on them.
-- Virtual Windows Project
But don't just go by my word. Go spend a few K on a Windows NT server licence, hardware, and documentation, then play with it. You'll know it's Windows because it feels "right," and because it flashes its monitor a very pretty sky blue to let you know it wants your attention. Surveys say that 60% of everyone's favourite colour is blue.
---
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
Money__ asks: "Since there are now so many different flavors of Windows out there (1.0, 3.1, 95, 98, NT, 2K , etc...) what do they all have in common that lets these all be called Windows? Programs written for one flavor of Windows typically cannot be ported to another without considerable effort. The features offered by the different implementations vary widely: some are more secure than others, some cluster better than others, some offer journaling file systems, some are more robust. The differences between the different kinds of Windows seem to be as great as the differences between any particular implementation and other OSs. Could one port all the standard GUI utilities to GNOME, clone one or two of the popular GUI features, set up the directory structure in the standard Windows layout and call it WINDOWS/UNIX?"
Hmmmmmm
_________________________
I suppose if M$ was to actually provide a near full implementation of a unix shell, filesystem, and command line utilities, it could be argued that NT would be, indeed, a UNIX. The one thing I wonder about is the kernel. UNIX kernels are just that.. UNIX kernels. I'm no kernel hacker by any means, but seems to me for a kernel to be a UNIX kernel, it has to have a certain structure and interface, which makes it a UNIX kernel. SO in that respect, NT could never really be a UNIX.
...though some (including Microsoft) have tried. It may pass POSIX compliance tests, it may run UNIX software, but there are certain differences that will be difficult or impossible to overcome.
First is the security issue. Unix has the concept of a superuser (root). NT has role-based security. "Administrator" is not special other than having more rights than user accounts, including the right to manager user rights.
The key differences are that an NT Administrator cannot open any NTFS file regardless of modes, as a Unix superuser can. Also the Administrator account cannot arbitrarily pose as other users. There is no working setuid() call. (POSIX requires setuid to exist, but it doesn't have to work... it may return EPERM unconditionally. That is just what Microsoft's POSIX subsystem does.)
Second is the executable file format. Most modern OSes have standardized on ELF. NT is one of the holdouts, still using PE (a variant of COFF). Shared libraries on NT (i.e. DLLs) are loaded and relocated by the OS, not in user space as ELF shared objects are. And DLLs have annoying limitations, like requiring data symbols to be imported/exported.
Third, file semantics on NT are tailored for Win32. Ever try to remove/rename a NTFS file while it is open in some process? You can't. On Unix, linking and unlinking of open files is permitted, and many utilities depend on that behavior.
While it may be possible to certify NT as "POSIX-compliant" (or even get Unix branding, I don't know), it will never work truly like a Unix system. There are just too many core differences.
The philosophy is a result of more than twenty years of software development and has grown from the UNIX community
instead of being enforced upon it. It is a defacto-style of software development. The nine major tenets of the UNIX
Philosophy are:
1.small is beautiful
2.make each program do one thing well
3.build a prototype as soon as possible
4.choose portability over efficiency
5.store numerical data in flat files
6.use software leverage to your advantage
7.use shell scripts to increase leverage and portability
8.avoid captive user interfaces
9.make every program a filter
The Ten Lesser Tenets
1.allow the user to tailor the environment
2.make operating system kernels small and lightweight
3.use lower case and keep it short
4.save trees
5.silence is golden
6.think parallel
7.the sum of the parts if greater than the whole
8.look for the ninety percent solution
9.worse is better
10.think hierarchically
I can't believe I didn't see this while scanning the comments--maybe I missed it. A huge part of UNIX is the filesystem. The UNIX filesystem has got to be among the most beautiful device abstraction layers ever built. I don't need to know how many drives are in the machine--or I can add extra drives, change the physical location of data, and keep the same paths and filenames.
/dev/ram and make filesystems in files on my filesystem. Then there are links, especially symlinks. And networked filesystem types that look local to the user (slow with lots of latency, and some race conditions, but local). You can even access the kernel via the filesystem.
I can access my printer as a file, my serial ports as a file, my memory as a file. Heck, I can mkfs
NTFS and VFAT don't have symlinks and the hardware is up-front and ugly (ever try moving an app from C: to D:?). I'm sure there's a lot more, but these things alone are horrible. What happened to device abstraction, that I have to track my hardware when naming files?
-Paul Komarek
Hmmm, you ask a tough question, but here's a go at an answer.
:)) playing field, made of lots of tools that easily allow the user to interact with files, and hence, the computer. Since this is the layer a user spends most of his or her time with, this is what people have come to identify when they say UNIX.
UNIX encompasses more than just a set of API's, and user shells. UNIX has more to do with an OS design philosiphy, than with any particular implimentation. At the outset, UNIX strives to provide an environment made of small utilities that can easily be used together to accomplish a task. This is true from the kernel design, to the typical C API's, out to the actual user environment. From a kernel perspective, most UNIX's have a virtual file system, virtual memory system, and a process swapper. There is much more than this, but these are the three main components. By utilizing features of each of these basic parts, it is possible to implement inter process communication(Pipes, shared memory, etc), security(mostly via the VFS, since most everything in UNIX is seen as a file), and Time sharing(via the process swapper).
The next layer would be the C libraries that most UNIX's come with. While not directly tied to the OS itself, most C libraries will heavily reflect the OS they live on, since they are the common gateway to make sys calls, the main way to call system functions. It is with system calls that a program can interact with files, and access system resources. The OS will handle the scheduling and access control from behind the scenes. The C library is a way to ask for resources.
But, what most people think of when they think of UNIX, is the user environment. Unfortunately, this is missleading. Everything from Windows to OpenVMS, to OS/390 MVS can be made to look and smell like UNIX, but that doesn't make them UNIX.
What the user sees is a simplified(if you can believe that grep, sed, and awk, ad nauseam are simple
The moral of the story is that UNIX really is a "handle" that is used to encompass an approach to OS design. UNIX is a way of viewing how things should be shared, and managed. That is why UNIX runs on almost every platform. UNIX is really a memme that dictates how system resources are controlled, and not one of how a system should be architected from a hardware perspective. UNIX is a way for multiple users to access a single system, in quasi-realtime , interactive fashion. It lays out such things as scheduling, Access Control architecture, and resource presentation. If an OS follows this design philosophy, than that makes it more a UNIX, than what it's user shells look like.
That is why Linux is a UNIX, without having any blessed UNIX C code in it. It's an architecture built on the UNIX philosophy. Just remember to never let a purist hear you call Linux UNIX though. That, unfortunately, is a matter of religion, best left for another discussion.
-Master Switch, one more element in the machine
IMHO the biggest thing all flavors of Unix have in common is the "everything is a file" design. So the answer to your question about porting all sorts of things to NT and calling it MS Unix is no. Everything in NT is an object.
I've been working with Unix now for 20+ years. I've worked in the kernels of Cray Super Computers and been driven batty by a BSDish system running on a no name box for a network logger. The equipment has ranged from a 386 that took 5 minutes to boot Linux to SGI super computer cluster. And there are a few things that they all had in common. These common "features" are, in my not so humble opinion, what makes something "Unix".
For that matter, the user can't tell the difference between an IDE cdrom and a SCSI cdrom.
While API is important, and I wouldn't want to give up my X windows, or bash/tcsh I've run on Unix boxes where those are different or missing. In some cases my tools of choice were missing and I had to bootstrap 'gcc' into place and from there I added the extra tools (sed/awk/tcsh/bash) that made my life comfortable.
Unix is a style. A way of solving problems. We don't think in terms of "monolithic (Office)" programs that do almost what we want, but instead we think in terms of solving a piece of the problem, chipping away till there is nothing left. And in the process we find that we have a dozen new tools that people will use in ways we never considered.
PERL was never intended to be a webserver, but the tool is so powerful that it now is. That and much more. Every tool that Unix has is used in ways that its creator never intended. Microsoft Office will never be more than what it is, What You See Is ALL You Get.
I leave you with this quote then: "Like a precious metal, UNIX is being beaten and molded into whatever forms are most pleasing to the owners." -- Lee A Butler
What is Linux, then? Well, it's "unix-like", despite the FAQ's claim that this term is a trademark violation. Personally, I'm sceptical of the continued validity of the "UNIX" mark (once something enters common usage, it can't be a trademark), but there've been no court challenges, so it stands.
--
Others have already answered the question with reference to POSIX, but I just thought I'd thow in my two Euro as well...
Recently having ported software between Linux, FreeBSD and NT, I have developed my own idea of what a ``UNIX'' is. It's simplicity most of all. Everything is a file, and most calls that work on file descriptors work on _all_ file descriptors. You can make a select() call on a file, a pipe, and a socket. This is impossible on NT, because files are HANDLEs, sockets are SOCKETs, HANDLEs from anonymous pipes cannot be used for select() like calls (only named pipes there). Why this diversity in the API ?
Running a daemon simply requires you to start up the executable with an "&" if it doesn't support forking to background by itself. I have a routine that does this on UNIX as well as on NT. The UNIX code is perhaps 5-10 lines, and the NT code is roughly 150. On NT you must specifically talk to the service control manager, ask it to ask your program to ask it to ask the program to do all kinds of strange things. Horrible, and inflexible to the extreme.
The system calls on UNIX take a minimum necessary number of arguments, and perform some action fairly much as you would expect it to. An average system call on UNIX probably has, say, 3-4 arguments. I haven't counted the average on NT, but I'd guess it's 6-7 arguments there. This is not because NT is more advanced, but because IMO the design of Win32 is flawed.
As an example, to start up a program on UNIX, you'll write something like (dumbed down):
if(!fork())
exec("my_program", arguments, environment)
Two system calls that perform a very clear and simple task each, taking 0 and 3 arguments.
The similar code on NT is something along the lines of a call to CreateProcess() which takes 10 (ten!) arguments. These arguments are: program name, command line and environment as on UNIX. Besides, you'll have to pass arguments such as current directory, process security attributes, thread security attributes, filehandle inheritance flag, creation flags, etc. etc. My favorite is that you have to pass an argument specifying the Window Size (!!) of your new process... I nearly pissed my pants when I saw that.
UNIX system calls are fairly consistent. Usually they will return either a 0 pointer, or the integer -1, if the call fails (depending on whether the call returns a pointer or an integer). On NT the calls return either INVALID_PROCESS_HANDLE, INVALID_FILE_HANDLE, INVALID_SOCKET, FALSE, 0, -1, or whatever the failure-return-code-of-the-day was when some call was last rewritten. Error codes can be retrieved from errno on UNIX, while on NT they come from GetLastError() usually (with some exceptions I've happily forgotten about), unless of course it was a socket call that failed, in which case the errorcode can be retrieved from WSAGetLastError(). Sigh.
The point I'm trying to make here is, that to me UNIX stands for a _fairly_ consistent and well designed API. Win32 is neither well designed nor consistent, and porting command line utilities to NT is not going to make it UNIX, not by a long shot.
The basic cause of the backspace problem is a different ASCII code for the key in DEC terminals (where it's 0x7F) and others such as Sun systems (where it's 0x08). Unix is configurable (using the stty command) to deal with either, so that it didn't really matter too much.
A modern Unix-like[1] system is a blend of many different threads of development, with both Sun and DEC well represented. So, the Linux console follows the DEC convention, while the xterm's in most distributions follow Sun (Debian is the major exception here).
So, in a system with the backspace problem, you have many factors coming together:
And, no from-the-ground-up reconsideration of the system, during which the backspace problem would certainly be fixed.
If that doesn't define Unix, I'm not sure what does.
[1] Yes, I know it's an "abuse of the trademark." The Open Group can stuff it.
LILO boot: linux init=/usr/bin/emacs
You know it's Unix when the backspace key often performs an action other than deleting the character to the left of the cursor.
When (actually if) this problem is fixed, the system will have changed so much from Unix that it probably wouldn't be recognizable to Thompson and Kernighan.
LILO boot: linux init=/usr/bin/emacs
A unix system is like an orgasm. When you have one, you'll know it.
--
Why can't I moderate something "Wrong" or at least "Grossly Misinformed"?
I think the most important thing to realize is that no single thing defines what makes Unix be Unix instead of Just Another Operating System. Sure, there is POSIX, but POSIX doesn't cover everything. Sure, there are the "standard" Unix shell tools, but those can be ported. You can't nail Unix down in any easy definition. Yet, Unix inevitably feels like The Right Thing to those who know it. If you like cliches, Unix reminds me of the old line "I don't know art, but I know what I like."
/, and cannot function without it. It is more then not being able to do anything useful without the programs in the root; it is the fact that the Unix filesystem is a large part of the Unix API.
/bin, configuration data in /etc, devices in /dev, temporary files in /tmp, "user files" in /usr. None of these are mandated by the kernel or the utilities, but they are definitely old friends to a Unix hacker like me.
That being said, I'd like to touch upon a few things that make Unix what it is:
The design of Unix is driven by synthesis. You don't create a specific tool to solve a particular problem; you break it down into smaller, general problems and write general tools to solve them. You then combine those tools to solve the original problem -- but you can continue using them afterwards.
This leads us to The Unix Philosophy. Anything you call "Unix" or "Unix-like" will adhere to it. However, the Unix philosophy is a set of design goals, not a system definition. Something can follow those goals without being Unix. So that doesn't cover everything.
Let's start with the filesystem. As others have said, a key element of Unix is the single filesystem. Unix must have a root filesystem mounted at
Additional filesystems are spliced into this single presentation, not mounted as separate trees. System hardware is abstracted and presented using file system entries. These are things that cannot be done if your OS doesn't support them. Then you have the organization of the files in the Unix filesystem. Programs in
Unix processes also behave in a certain way. Process spawn overhead is low and context switching is fast. Signals and exit codes are used for IPC. fork() and exec() are separate system calls.
Unix treats text as data and data as text. Configuration files are generally human-readable. You can "cat" a binary file without the OS doing end-of-line manipulations. Any particular meaning of a character (^D for EOF, e.g.) comes from the terminal driver, not the I/O mechanism.
Lastly, Unix was implemented in C, and C was designed to implement Unix. Contrast this with other OSes, where the language you're programming in and the system library are generally completely separate things. This synthesis (to borrow from the Mazda ad campaign) "just feels right".
While I'm on the subject, I'd like to address two things that Unix explicitly isn't:
Unix is not a trademark. I'm sure The Open Group doesn't agree with me, but Unix was around before they were, and will continue to be around long after they are gone. They control who can legally put "UNIX" on their product, but that is a matter of layers and money, of which Unix cares about neither.
Unix is not a particular source implementation. There are very Unix-ish things which have not one line of AT&T or BSD code in them, and there are things totally not Unix which contain BSD code. MS-Windows is one of them.
I forget who said it, but if you're looking for one line answers, then this fits best:
"Unix isn't so much an operating system as it is a painstakingly compiled oral history of the hacker culture."
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
- POSIX compliant system API
- Availability of traditional UNIX shell tools
- Ability to run without a windowing system
- Modular design
The question is like asking "What's a car?" BMW, Mercedes, Honda, GM and lots of others fit the mold without there being a 'definitive' car. I think the same applies to UNIX: there's lots of variants, based on the same basic (and time-tested) design. I've done some software porting across different UNIX systems and in my opinion, the differences between these systems were blown out of proportion by the marketing machine from Redmond (and others). There are differnces, but they're manageableAlso, while this is not as true anymore as it was a few years ago, I would say that an important part of the UNIX philosophy is the fact that most configuration parameters (and other useful stuff) are stored in plain text files (yes, Solaris and AIX don't do this anymore, but the basic idea is still there). You could port all the shell utilities to NT only to find out that they're basically useless as everything is stored in some binary file that you can't really process with your trusted text tools. ASCII is portable and accessible; a fact that UNIX really drives (or at least should drive) home ...
It's a well-kept secret that the NT Executive has a single-rooted hierarchy, containing device nodes, named pipes, and filesystem roots. It even supports mount points, sort of, via a concept called reparse points.
Of course none of this does us any good, because the Win32 subsystem does its best to hide whatever elegance lives in the NT kernel. You can get glimpses of it... for example, try opening the file \\.\PhysicalDrive0 for reading (don't write to it, that's your raw disk sectors!).
You get the feeling that NT could have been something better if they didn't have to make it compatible with Win9x.
Oh, and BTW, NTFS on Win2k does symlinks too.