Slashdot Mirror
Tim O'Reilly Says Piracy is Progressive Taxation
Idmat writes "In Tim's latest opus, he reflects on the lessons of his experience as a publisher: (1) Obscurity is a far greater threat to authors and creative artists than piracy, (2) Piracy is progressive taxation; (3) Customers want to do the right thing, if they can; (4)Shoplifting is a bigger threat than piracy; (5) File sharing networks don't threaten book, music, or film publishing. They threaten existing publishers; (6)"Free" is eventually replaced by a higher-quality paid service; and finally, courtesy of Larry Wall, (7)There's more than one way to do it. "
a Taco Snotting for Christmas.
Leave the wife, Taco, please snot me!
1.2 Power Grows on You
It has been said that Unix is not an operating system as much as it is a way of thinking. In The UNIX Programming Environment, Kernighan and Pike write that at the heart of the Unix philosophy "is the idea that the power of a system comes more from the relationships among programs than from the programs themselves."
Most of the nongraphical utility programs that have run under Unix since the beginning, some 30 years ago, share the same user interface. It's a minimal interface, to be sure -- but one that allows programs to be strung together in pipelines to do jobs that no single program could do alone.
Most operating systems -- including modern Unix and Linux systems -- have graphical interfaces that are powerful and a pleasure to use. But none of them are so powerful or exciting to use as classic Unix pipes and filters, and the programming power of the shell.
A new user starts by stringing together simple pipelines and, when they get long enough, saving them for later execution in a file (Section 1.8), alias (Section 29.2), or function (Section 29.11). Gradually, if the user has the right temperament, he gets the idea that the computer can do more of the boring part of many jobs. Perhaps he starts out with a for loop (Section 28.9) to apply the same editing script to a series of files. Conditions and cases soon follow and before long, he finds himself programming.
On most systems, you need to learn consciously how to program. You must take up the study of one or more programming languages and expend a fair amount of concentrated effort before you can do anything productive. Unix, on the other hand, teaches programming imperceptibly -- it is a slow but steady extension of the work you do simply by interacting with the computer.
Before long, you can step outside the bounds of the tools that have already been provided by the designers of the system and solve problems that don't quite fit the mold. This is sometimes called hacking; in other contexts, it is called "engineering." In essence, it is the ability to build a tool when the right one is not already on hand.
No single program, however well thought out, will solve every problem. There is always a special case, a special need, a situation that runs counter to the expected. But Unix is not a single program. It is a collection of hundreds of them, and with these basic tools, a clever or dedicated person can meet just about any computing problem.
Like the fruits of any advanced system, these capabilities don't fall unbidden into the hands of new users. But they are there for the reaching. And over time, even those users who want a system they don't have to think about will gradually reach out for these capabilities. Faced with a choice between an hour spent on a boring, repetitive task and an hour putting together a tool that will do the task in a flash, most of us will choose the latter.
-- TOR
1.4 Communication with Unix
Probably the single most important concept for would-be power users to grasp is that you don't "talk" directly to the Unix operating system. Instead, you talk to a program -- and that program either talks to Unix itself or it talks to another program that talks to Unix. (When we say "talk" here, we mean communication using a keyboard and a mouse.)
There are three general kinds of programs you'll probably "talk" to:
*
The program called the shell (Section 27.1). A shell is a command interpreter. Its main job is to interpret the commands you type and to run the programs you specify in your command lines. By default, the shell reads commands from your tty and arranges for other programs to write their results there. The shell protects Unix from the user (and the user from Unix). It's the main focus of this book (and the rest of this article).
*
An interactive command, running "inside" a tty, that reads what you type directly. These take input directly from the user, without intervention from the shell. The shell's only job is to start them up. A text editor, a mail program, or almost any application program (such as word processing) includes its own command interpreter with its own rules. This book covers a few interactive commands -- such as the vi editor -- but its main focus is the shell and "noninteractive" utilities that the shell coordinates to do what needs doing.
*
A Graphical User Interface (GUI) with a desktop, windows, and so on. On Unix, a GUI is implemented with a set of running programs (all of which talk to Unix for you).
Unix was around long before GUIs were common, and there's no need to use a GUI to use Unix. In fact, Unix started in the days of teletypes, those clattering printing devices used to send telegrams. Unix terminals are still referred to as teletypes or ttys (Section 2.7).
The core of the Unix operating system is referred to as the kernel (Section 1.10). Usually, only programs talk to the kernel (through system calls). Users talk to one of the three previous types of programs, which interprets their commands and either executes them directly or passes them on to other programs. These programs may, in turn, request lower-level services from the kernel.
Let's look at a specific example of using the shell. When you type a command to display files whose four-character filenames start with the letter "m":
??? Section 1.13
% cat m???
it is the shell that finds the filenames, makes a complete list of them, and calls the cat (Section 12.2) command to print the expanded list. The cat command calls on the kernel to find each file on the disk and print its contents as a stream of characters on the display.
Why is this important? First of all, you can choose between several different shells (Section 1.6), each of which may have different rules for interpreting command lines.
Second, the shell has to interpret the command line you type and package it up for the command you are calling. Because the shell reads the command line first, it's important to understand just how the shell changes what it reads.
For example, one basic rule is that the shell uses "whitespace" (spaces or tabs) to separate each "argument" of a command. But sometimes, you want the shell to interpret its arguments differently. For example, if you are calling grep (Section 13.1), a program for searching through files for a matching line of text, you might want to supply an entire phrase as a single argument. The shell lets you do this by quoting (Section 27.12) arguments. For example:
% grep "Power Tools" articles/*
Understanding how the shell interprets the command line, and when to keep it from doing so, can be very important in a lot of special cases, especially when dealing with wildcards (Section 1.13), like the * (asterisk) in the previous example.
You can think of the relationship of the kernel, the shell, and various Unix utilities and applications as looking like Figure 1-1.
Figure 1-1. Relationship of kernel, shell, utilities, and applications
figs/upt3_0101.gif
Figure 1-1 shows that a user can interact with the shell, as well as directly with interactive commands like cat and ls. The shell transfers control to the commands it starts for you -- then those commands may write the output you see. The shell also has some built-in commands (Section 1.9) that run directly within the shell itself. All of the commands shown in Figure 1-1 interact directly with Unix itself.
--TOR and JP
we've progressed where everyone regardless of race, color and creed are equal in the eyes of the law.
If you look through history you'll find this same retarded right-winger argument at every milestone. For one thing, homosexuals are NOT equal in the eyes of the law and even those who are "equal" are subject to the law's interpretation and enforcement, which is often less than ideal.
If people are poor (at least in America) it's their own damn fault. They should work harder and get educations, not look for government handouts.
So poor people should just go out and get a job? Are you hiring? wake up, our economy is far from perfect. Every time a manufacturing plant moves to Mexico, a city is destroyed. Thank god for NAFTA and unregulated trade, eh? You propose we don't need any progressive reforms? Tell that to the people writing software for FREE because there's no other way to compete with Microsoft.
Socialism just doesn't work, look at history.
Uh... you mean communism? Yeah, that didn't go so well, but communism isn't the ultimate goal of social reforms. If you look at history, you'll find that a simplistic black and white approach never works, whether it's to the left or the right.
"Question with boldness even the existence of a god." - Thomas Jefferson
Help us re-take this message board.
http://www.askheartbeat.com/cgibin/ultimatebb.c
1.5 Programs Are Designed to Work Together
As pointed out by Kernighan and Pike in The UNIX Programming Environment, there are a number of principles that distinguish the Unix environment. One key concept is that programs are tools. Like all good tools, they should be specific in function, but usable for many different purposes.
In order for programs to become general-purpose tools, they must be data independent. This means three things:
1.
Within limits, the output of any program should be usable as the input to another.
2.
All of the information needed by a program should be either contained in the data stream passed to it or specified on the command line. A program should not prompt for input or do unnecessary formatting of output. In most cases, this means that Unix programs work with plain text files that don't contain "nonprintable" or "control" characters.
3.
If no arguments are given, a program should read the standard input (usually the terminal keyboard) and write the standard output (usually the terminal screen).
Programs that can be used in this way are often called filters.
One of the most important consequences of these guidelines is that programs can be strung together in "pipelines" in which the output of one program is used as the input of another. A vertical bar (|) represents pipe and means "take the output of the program on the left and feed it into the program on the right."
For example, you can pipe the output of a search program to another program that sorts the output, and then pipe the result to the printer program or redirect it to a file (Section 43.1).
Not all Unix programs work together in this way. An interactive program like the Emacs editor (Section 19.1) generally doesn't read from or write to pipes you'd create on the command line. Instead, once the shell has started Emacs, the editor works independently of the shell (Section 1.4), reading its input and output directly from the terminal. And there are even exceptions to this exception. A program like less (Section 12.3) can read its standard input from a pipe and still interact with you at the keyboard. It does that by reading directly from your tty (Section 2.7).
-- TOR
1.6 There Are Many Shells
With most operating systems, the command intepreter is built in; it is an integral part of the operating system. With Unix, your command interpreter is just another program. Traditionally, a command interpreter is called a "shell," perhaps because it protects you from the underlying kernel -- or because it protects the kernel from you!
In the early 1980s, the most common shells were the Bourne shell (sh) and the C shell (csh). The Bourne shell (Section 3.3) (named after its creator, Steve Bourne) came first. It was excellent for shell programming (Section 1.8). But many Unix users (who were also writing programs in the C language) wanted a more familiar programming syntax -- as well as more features for interactive use. So the C shell came from Berkeley as part of their Unix implementation. Soon (on systems that gave you the choice, at least) csh was much more popular for interactive use than sh. The C shell had a lot of nice features that weren't available in the original Bourne shell, including job control (Section 23.1) and history (Section 30.2). However, it wasn't hard for a shell programmer or an advanced user to push the C shell to its limits.
The Korn shell (also named after its creator, David Korn) arrived in the mid-1980s. The ksh is compatible with the Bourne shell, but has most of the C shell's features plus features like history editing (Section 30.14), often called command-line editing. The Korn shell was available only with a proprietary version of Unix, System V -- but now a public-domain version named pdksh is widely available.
These days, most original C shell users have probably switched to tcsh (pronounced "T-shell"). It has all the features of csh and more -- as well as fewer mis-features and outright bugs.
The "Bourne-again" shell, bash, is from the Free Software Foundation. It's fairly similar to the Korn shell. It has most of the C shell's features, plus command-line editing and a built-in help command. The programming syntax, though, is much more like the original Bourne shell -- and many systems (including Linux) use bash in place of the original Bourne shell (but still call it sh).
The Z shell, zsh, is an interesting hybrid. It tries to be compatible with most features of all the other shells, with compatibility modes and a slew of options that turn off conflicting features. In its soul, though, zsh has a different way of doing some things. It's been accused of feature creep. But zsh users love its flexibility.
There are other shells. If you're a fan of the Bell Labs research operating system named Plan 9 (actually, Plan 9 from Outer Space), you'll be happy to know that its shell, rc, has been ported to Unix. If you program in Tcl, you'll probably be familiar with tclsh, which lets you intermix Unix commands with Tcl commands. (And we can't forget wish , the shell that's a superset of tclsh: it uses Tcl/Tk commands to let you build graphical interfaces as you go.) Least -- but certainly not last -- if you're a minimalist who needs the original sh, a newer shell named ash emulates the late-1980s Bourne shell.
In this book, we try to be as generic as we can. Where we need to get specific, many examples are shown in the style of both the Bourne shell and the C shell -- for instance, we'll often show Bourne-shell functions side-by-side with C-shell aliases. Because bash and ksh can read scripts written for the original Bourne shell, we use original sh syntax to make our shell programming as portable as possible.
Where we talk about "the Bourne shell" or sh, it's usually a safe bet that the information applies to bash and ksh too. In the same way, "the C shell" generally also means tcsh.
--JP and ML
1.7 Which Shell Am I Running?
/etc/passwd
/etc/passwd, you'll need to configure this using Netinfo.
You can usually tell which family your shell belongs to by a character in the prompt it displays. Bourne-type shells, such as bash , usually have $ in the prompt. The C shell uses % (but tcsh users often use >).
If your shell has superuser (Section 1.18) privileges, though, the prompt typically ends with a hash, #.
To check the shell that runs automatically when you log in to Unix, type one of these commands (the second is for systems that use NIS, Sun's Network Information Service, to manage network-wide files):
% grep yourloginname
% ypmatch yourloginname passwd
You should get back the contents of your entry in the system password file. For example:
shelleyp*:1006:1006:Shelley Powers:/usr/home/shelleyp:/usr/local/bin/bash
The fields are separated by colons, and the default shell is usually specified in the last field.
Note that in Mac OS X, passwords are managed and stored in Netinfo by default. To store the passwords in
--TOR and SP
One of the really wonderful things about the shell is that it doesn't just read and execute the commands you type at a prompt. The shell is a complete programming language.
The ease of shell programming is one of the real highlights of Unix for novices. A shell program need be no more than a single complex command line saved in a file -- or a series of commands.
For example, let's say that you occasionally need to convert a Macintosh Microsoft Word file for use on your Unix system. Word lets you save the file in ASCII format. But there's a catch: the Mac uses a carriage return ASCII character 015 to mark the end of each line, while Unix uses a linefeed (ASCII 012). As a result, with Unix, the file looks like one long paragraph, with no end in sight.
That's easy to fix: the Unix tr (Section 21.11) command can convert every occurrence of one character in a file to another:But you're a novice, and you don't want to remember this particular piece of magic. Fine. Save the first part of this command line in a file called mac2unix in your personal bin directory (Section 7.4):Make the file executable with chmod (Section 50.5):Now you can say:But why settle for that? What if you want to convert a bunch of files at once? Easy. The shell includes a general way of referring to arguments passed to a script and a number of looping constructs. The script:
for Section 35.21, $x Section 35.9will convert any number of files with one command, replacing each original with the converted version:As you become more familiar with Unix, it quickly becomes apparent that doing just a little homework can save hours of tedium. This script incorporates only two simple programming constructs: the for loop and variable substitution (Section 35.9, Section 35.3).[1] As a new user with no programming experience, I learned these two constructs by example: I saved a skeleton for loop in a file and simply filled in the blanks with whatever commands I wanted to repeat. Section 35.2 has more about shell programming.
[1] [Tim is keeping this article simple, as an illustration of how easy writing a shell program can be. If you're writing this little script for general use, you can make it work like a filter (Section 1.5) by adding four or five more lines of code: a case (Section 35.10) or if (Section 35.13) statement that tests the number of command-line arguments. With no filename arguments, the script would simply run tr '\015' '\012'. -- JP]
In short, Unix is sometimes difficult because it is so rich and complex. The user who doesn't want to learn the complexity doesn't have to -- the basic housekeeping commands are simple and straightforward. But the user who wants to take the time to investigate the possibilities can uncover a wealth of useful tools.
-- TOR
If you have arrived at Unix via Windows 2000 or some other personal computer operating system, you will notice some big differences. Unix was, is, and always will be a multiuser operating system. It is a multiuser operating system even when you're the only person using it; it is a multiuser operating system even when it is running on a PC with a single keyboard; and this fact has important ramifications for everything that you do.
Why does this make a difference? Well, for one thing, you're never the only one using the system, even when you think you are. Don't bother to look under your desk to see if there's an extra terminal hidden down there. There isn't. But Unix is always doing things "behind your back," running programs of its own, whether you are aware of it or not. The most important of these programs, the kernel, is the heart of the Unix operating system itself. The kernel assigns memory to each of the programs that are running, partitions time fairly so that each program can get its job done, handles all I/O (input/output) operations, and so on. Another important group of programs, called daemons, are the system's "helpers." They run continuously -- or from time to time -- performing small but important tasks like handling mail, running network communications, feeding data to your printer, keeping track of the time, and so on.
Not only are you sharing the computer with the kernel and some mysterious daemons, you're also sharing it with yourself. You can issue the ps x (Section 24.5) command to get a list of all processes running on your system. For example:This output tells us that the user has only three windows open. You may think that they're only running four or five programs, but the computer is actually doing a lot more. (And, to keep this brief, we aren't showing all the lines of output!) The user logged into his Linux system on virtual console (Section 23.12) 2, which shows as tty2 in the TTY column; a lot of programs are running there, including the X Window System (Section 1.22) (which actually runs itself as another user -- root -- so its process isn't listed here). The user is also running Gnome and Enlightenment, which keep track of the workstation's display. Two of the windows are Gnome terminals, which are windows that act like separate terminals; each has its own tty, pts/0 and pts/1. And the list continues.
If you are running a different window system (or no window system at all) or different utility programs, you will see something different. But we guarantee that you're running at least two programs, and quite likely many more. If you want to see everything that's running, including the daemons, type the command ps aux (Berkeley-style ps) or ps -el (for many other flavors of ps). You'll be impressed.
Because there is so much going on at once, Unix requires a different way of thinking. The Unix kernel is a traffic cop that mediates different demands for time, memory, disks, and so on. Not only does the kernel need to run your programs, but it also needs to run the daemons, any programs that other users might want to start, or any programs that you may have scheduled to run automatically, as discussed in Chapter 23. When it runs a program, the kernel allocates a small slice of time -- up to a second -- and lets the program run until that slice is used up or until the program decides to take a rest of its own accord (this is called "sleeping"). At this point, regardless of whether the program is finished, the kernel finds some other program to run. The Unix kernel never takes a vacation: it is always watching over the system.
Once you understand that the kernel is a manager that schedules many different kinds of activity, you understand a lot about how Unix works. For example, if you have used any computer system previously, you know that it's a bad idea to turn the computer off while it is writing something to disk. You will probably destroy the disk, and you could conceivably damage the disk drive. The same is true for Unix -- but with an important complication. Any of the programs that are running can start doing something to the disk at any time. One of the daemons makes a point of accessing the disk drive every 30 seconds or so, just to stay in touch. Therefore, you can't just turn a Unix computer off. You might do all sorts of damage to the system's files -- and not just your own, but conceivably files belonging to many other users. To turn a Unix system off, you must first run a program called shutdown, which kicks everyone off the system, makes sure that a daemon won't try to play with a disk drive when you aren't looking, and runs a program named sync to make sure that the disks have the latest version of everything. Only then is it safe to pull the switch. When you start up a Unix system, it automatically runs a program called fsck, which stands for "filesystem check"; its job is to find out if you shut down the system correctly and try to fix any damage that might have happened if you didn't.
--ML and JP