Slashdot Mirror

BlueCalx- writes: "dotcomma has created a new programming contest: this time, to determine whether or not someone can create a program that can automatically parse an RDF file and generate a haiku based on its headlines or stories. Slashdot users such as 575 have essentially been doing the same thing for months: now, it's time to see if a computer program can do the same thing *g*. After witnessing the success of the AI Bots challenge a few months ago, it'll be interesting to see if a program like this is possible." Anyone who can generate intelligible, germane haiku from headlines without human intervention has my respect -- it's a lot thornier than it sounds.

Effugas asks: "I've been thinking about something off and on for some time now...perhaps, in all of our complaining that the patent office equates 'net' with 'new', we've done a bit of this ourselves? I'm thinking particularly in regards to non-computer related economic trends that look suspiciously like what the computer industry has taught us to expect. To wit: You don't own your apps (ASP's), you can't control your software (UCITA), your music isn't yours (SDMI), your privacy isn't yours, etc. Now look at the real world in areas where tech savviness is on the rise: leased cars, rented houses, long term apartments / condos / duplexes...your employment is at will and can disappear anytime, and your cities seem strangely hostile to you doing anything other than working, sleeping, or spending. Note the lack of any kind of long term commitments, ownerships, investments, or so on... Is there a relationship between tech patterns and what's going on outside? I'd appreciate your comments."

chazR writes: "Researchers from CSIRO were present at the creation of a new volcanic island Once the molten lava stops being thrown 70m into the air, I reckon this would make an ideal offshore site for a server farm. Who's going to hassle you on top of a volcano miles from the nearest land. Getting the OC-192 link in could be a problem though ... " Well, that's only if Australia (not me) actually has dibs on the real estate.

In the first part of Slashdot's 'Making Music with Linux' series, we discussed the possibilities of using Linux as an audio production operating system. While we lamented the lack of a fully-functional audio suite for Linux, we saw the silver lining in the cloud of patience, and witnessed a great number of free sound tools that were well on their way to greatness. In this installment, we talk a little more about high-end audio mastering, low-end sound transport, and using Linux as a tool for sound synthesis. Part II of a series.

Burning a CD under Linux is super-easy, and there are quite a few programs that make it possible under Linux. X-CD-Roast is a popular tool, and is used constantly by Linux enthusiasts to burn their own multi-session content onto a compact disc. The problem is that there is still no Linux equivalent for the huge multi-track mixer you'll find at professional recording studios. If you're hoping to mix down tracks and perform studio-quality takes and 'bounces' of your work, you're going to need an expensive chunk of professional digital audio hardware. Once you've got your finished master, you'll have no problem making as many copies as you want using your Linux machine. Until we have a real-time mixdown utility, the waiting game wins again.

The world of low-bandwidth sound transport is wide open on the Linux platform, except for the glaring exception of not being able to play Windows Media Player sound files on your machine. Although the 'media darling' of sound transport is the mp3 file format, there's no denying the fact that mp3 provides solid sound quality and a small file footprint. Mp3 still isn't the answer to all of our prayers, however. No matter what anyone tells you, mp3 is not CD-quality sound, and tends to boost the mid-range in most tunes.

When you're living with a low-bandwidth target, the sound you're streaming is secondary in importance to stretching that stream over a maximum number of clients with the least amount of lag. RealPlayer suffers from the same mid-range band pass issues as mp3, but in the grand scheme of things, you're a lot more likely to encounter RealPlayer as an option when you're surfing. Producing RealPlayer content is easy to do and easy to host, but the downside is that it's not free. RealProducer will run you $149.95 from RealNetworks.

Joseph Ottinger, Linux-savvy musician, shares his thoughts on streaming audio. "I choose mp3 because of the high quality and decent compression. Real's stuff is nice because it's streaming, but even on a fast connection, that stuff sounds like it's ground up. Microsoft's streaming format is worse, lacking even more of the fundamental sound's depth and clarity. The problem with mp3 is Napster. Napster makes it easy to send and find mp3's. A lot of people trade though, so people rip stuff at low quality just to get their numbers up and their bandwidth down."

Csound is a wonderfully portable and versatile sound synthesis package written entirely in C. Csound uses two files to work its magic; a score file that basically acts as a timed-event trigger file, and an orchestra file to interpret what voices it should use to play the score. This is a classical approach to sound synthesis, and can be used to either generate a sound file or, if your system is fast enough, to send the output directly to a DAC on your system's soundcard. If you like, you can even use a standard MIDI file to act as the score file. Csound has about a bazillion extensions, and nifty gadgets that use it.

For those interested in using their Linux machine as a powerful tool for creating original instruments and sounds, they'll find a friend in Cecilia, a wonderful sound synthesis tool that sits right on top of Csound, without you having to get your hands dirty. Cecilia provides real-time signal processing on sound files, live input, or can work as a software synthesizer on its own. If you're in love with real-time resonance or envelope filters, Cecilia lets you configure the software synth to the limit.

Next time, we discuss Linux sound hardware and support, and we're going acronym-diving! Find out what OSS and ALSA are, and why they've got something to do with those big speakers you bought. We'll also navigate the treacherous waters of musical notation with Linux. If you know of any fantastic audio production programs, please let us know! See you next time...

Jon "maddog" Hall is the head honcho at Linux International. Monday we asked him questions about the future of Linux, his beer preferences, and other burning issues. Today, as our premier interview guest in the last year of the 20th century, maddog answers with his usual wit and candor.

1) What NEW directions do you see Linux going in?
by coreman

There have been lots of articles on what is the future of the current Linux projects... What do you see as the NEW, non-current directions that Linux will embark on in the near future/next century?

maddog:

Holy cow!! Linux is already being used in ubiquitous computing, embedded computing, turnkey systems, desktop systems, server systems, and Beowulf Supercomputers...how many more directions do you think it should turn?

True, I have been vocal about not telling people to use linux as a "highly available" system, due to its current lack of things like a log-based file system, hot-swap hardware, better hardware error detection and other things that a highly available system would need for a 366 day a year uptime.

And true, many people have noted a lack of NUMA memory capabilities, allowing expansion of processors past the single ultra-high-speed memory bus limitations of cost vs performance.

And a lot of different countries would LOVE to have true internationalization and localization done, so just by changing a message catalog (or adding to it) an operating system or application could be localized for a particular culture.

And then there is the perennial lack of applications, device drivers, etc to make Linux a truly viable desktop.

All of these are bad news, but the good news is that each of them is being addressed at "Linux speed".

In the past, and as short a time as ten years ago, these were all seen as lacking in the commercial Unix operating system space. I heard these same issues as reasons to use systems such as MPE, MVS and VMS, and that Unix systems "could never" do these things. Now most of these needs can be met through these same commercial Unix systems. The Linux community can now take the best of these solutions and designs for this work and re-implement them to meet the needs of the Linux community.

Since visualization of solutions is the hardest thing, and architecting a solution is the second hardest thing, if we pick the best implementation and re-implement it, we should have solutions to these (and other) issues very quickly.

2) Linux vs. HURD
by Gurlia

Do you think GNU/HURD might one day take over Linux's place? It certainly has a more modern design, although it is currently still in the works. Do you think it's a plausible alternative to Linux when it is ready for general consumption?

Or does Linux have a drive in the Open Source community that HURD doesn't? Linux seems to have generated a lot of enthusiasm, fandom, (and zealotry?). Could it be this drive that made Linux so successful and the lack thereof make HURD take such a long time to get developed?

(Disclaimer: I am NOT trying to start a flamewar between Linux and HURD supporters.)

maddog:

I think that if the technical goodness of a kernel was the defining issue, then one of the *BSDs would certainly be where Linux is today.

It is hard to say why Linux caught on, and the *BSDs did not, but I definitely think it has to do with the community spirit that the Linux community has managed to garner.

I have known of the FSF for years. I fought to have free software delivered with Digital's Unix systems since 1984. I have tried to donate equipment to the FSF. Unfortunately for reasons dealing with Richard Stallman's ideals of Free Software, and Digital's inability to create a licensing agreement (in those days) that he could sign, I was limited in what I could do. After a while, it was JUST TOO HARD.

With the Linux community I have never run into an issue yet that was JUST TOO HARD. Most of the time the people agree with what has to be done, and that their goals and ideals are much the same as mine.

I will also say that I am REALLY GLAD that Richard Stallman is the way he is, and has the ideals he has, because he continues to show me the path that I SHOULD be going, even though I may only get halfway there. Without his lead, I would have no epitome to reach.

Now, will HURD take over from where Linux is when HURD is ready? Perhaps. If it really is a better-performing kernel than Linux. Remember that a "more modern design" does not guarantee better performance, or even easier maintenance.

I have not looked at the HURD's design, but I understand that it is based on a microkernel, and from this you may get the idea that it is a "more modern design". But OSF's code was based on a microkernel, and so was MkLinux and a variety of other systems that have come out. None of these have shown a performance improvement over what can be done with a well-structured monolithic kernel with loadable device drivers and loadable kernel modules.

Since my customers sort of sit around with stop watches measuring performance in SPEC-marks and SPEC-rates, TPC numbers, etc., it is hard to give up performance for other issues.

Still, if the HURD comes out, and if it is system-call compatible with Linux, there would be a good case for substituting the HURD kernel for the Linux kernel. I make the one stipulation because I think that ISVs are TIRED of porting applications from one system to another, and really want to see ONE binary interface from the Linux/HURD community for each hardware architecture. This is why I think that one of the most important aspects of the Linux community is the Linux Standard Base (LSB) project.

3) Chasing the taillights?
by Wiktor Kochanowski

Linux, and in general the Open Source development model, has been accused in the past of "chasing the taillights" -- of always catching up to features that other commercial programs have, because they are results of vision rather than of a creeping evolution.

Myself, I think there may be something in this view, when I look e.g. at the emerging UI input methods like voice recognition and pen input/handwriting recognition on the client side, and various goodies on the server side.

Do you agree with this? If so, is Linux condemned to always be a few steps behind of the current state of the art of OS design, at least as far as features go?

If not, what examples of vision and features unique to Linux would you provide as examples?

maddog:

You are talking to someone who has been in the computer field for thirty years, so for the most part all I have seen is "chasing the tailights". Sure there have been some innovations in networking, but most of the operating systems have taken a lot of their "innovation" from systems such as the Whirlwind, SAIL, Xerox STAR and "re-implimented" them.

[I am sure I pissed off a lot of people with that last statement, but I purposely made it that way to get people to really take a look at what these older systems have done, and to marvel at what they did so long ago.]

I think that a lot of the features of an "operating system" you are talking about (i.e. input/handwriting, voice recognition, etc.) are things that were developed as layered product projects, and "integrated" into a certain operating system by a certain company we all know as part of "THAT COMPANY's innovations" (DON'T GET ME STARTED ON THAT TOPIC).

As the Linux worldwide market grows, I think you will find that more and more of this innovation will happen on Linux, due to both the Open Source concept and the worldwide virtual groups of minds who will work on it. The difference will be that the Open Source model will show where the innovation actually came from, and not where it was bought from.

4) certification
by Zurk

Regarding the recent community linux certification efforts etc, can we expect to see LI take a part in this? Are we going to get free community certification for Linux? Especially since all PHB's now seem to want certification...

maddog:

Ahem.

It was members of LI that started the LSB effort, and a lot of our members are very active in the pursuit of this standard.

LI members encouraged both Sair and the LPI in their standards efforts, feeling that (particularly in the early stages) two or more open certification efforts would be useful, since the community would decide (in the end) which of the certification efforts was best. The voting on this would be by how many people signed up for that certification, and which certification was judged best by the hiring managers of the certified people.

In the case of hardware certification, LI has been encouraging an emerging distribution and hardware manufacturer neutral certification group which has the goal of determining what the steps should be for certification rather than any set rules of certification itself.

I do not believe that there will be a "Free Certification" simply because there is a lot of boring, mundane work in marking answer forms, administering certificates, etc, but it can be made as inexpensive as possible by making it open, with openly published standards that have to be met.

5) Linux feature growth
by ajs

As I mentioned in a recent article thread, the Linux kernel is braving new waters in several areas which UNIX has traditionally shunned in the kernel (graphics support, http server, game support for network management, etc). These features raise the eyebrows of many people, but is this the way you see operating system design moving in the future? Are we so bound by the dreaded user-mode context switch that we have to plow every service as deep into the kernel as it will go?

Mind you, I'm all for the khttpd idea as a single example, but it seems like the beginning of a trend that will end up making the original Linux kernel look like a wristwatch driver, and leave a lot of low-end users in a bind....

maddog:

I still remember the time we placed the X window server inside the Ultrix kernel. This created a few problems, not the least of which meant that a mistake in code that (with a user-space based kernel) would normally only cause the X-server to crash, the person to be forced off the system, and the login-prompt to re-appear, NOW would cause the whole system to crash due to some type of "kernel panic". We also noted that the X-server (which managed its own memory) would grow without bound, using all of the available kernel memory in a few hours under specific graphics loads. In a user-based X-server, this was (at least) tolerable, but in a kernel-based X-server, it caused the whole system to hang.

All of this was to save a few microseconds of system context switch time and to give better "feel" to the X-server. Then an engineer got almost exactly the same "feel" with the user based X-server by raising the scheduling priority on both the X-server and the window manager.

And I might point out that since that time kernel-based scheduling of lightweight threads has made this type of issue even less of an argument.

My personal belief is that there are certain things that an operating system kernel should do, which is schedule resources among hardware and processes, including memory and CPU time. All other things should be put out in user space. But the last time I wrote kernel code was twenty-five years ago for a PDP-8....

[and speaking of time....as I typed this last part of the answer, the clock on my Linux system turned to:

[maddog@localhost maddog]$ date
Sat Jan 1 00:00:01 EST 2000

...so you can see that Linux is Y2K compliant]

6) How can you afford development?
by joshkerr

I don't understand how Linux can complete in the upper end server market, especially against competitors like Microsoft and Sun.

Microsoft is about to release Windows 2000 datacenter which will allow up to 64gig of ram and 32 processors. How can any one company afford that kind of equipment for the development of Linux?

Do you have any plans to recruit companies like Compaq and Dell so that they are major players in the development efforts of Linux? It seems to me that it would be benificial to have companies like this helping to direct the future development of Linux in terms of large scale applications. I realize that these companies are developing drivers and the such, but that isn't really what I'm talking about..

Apache running on Linux on a machine with 32 processors and 64gig of ram, able to out perform anything MS can throw at it. That is what I'm talking about...

maddog:

First of all, let me point out that Sun has two major divisions, SunSoft and Sun Microsystems. While SunSoft MAY see Linux as a competitor, Sun Microsystems sees it as another operating system to help it sell SPARCS and Intel PCs, which Sun makes. Even SunSoft can look at Linux as helping to maintain the Unix marketplace, and perhaps re-creating the Unix desktop. This will, in the long run, benefit SunSoft.

As to the other large vendors such as SGI, IBM, HP and Compaq, each of these companies have engineers working on internal projects to help Linux xpan the larger types of hardware platforms. Unfortunately, as you get into these very large systems, there are several basic differences that can occur just in the support of NUMA memory alone. Different internal bus structures and architectures might make it very hard for one kernel to be delivered across all these platforms.

At the last Comdex show, in his keynote speech, Linus acknowledged the fact that these larger systems might have a radically different kernel (or kernels) developed for them, but that the kernel programming interfaces would probably remain the same.

Ergo, when you bought your distribution of Linux on a CD-ROM, for certain machines you may have to get your 'boot diskettes" from the people who ship you the machine. Or perhaps certain of the kernel functions might be included in the boot ROMS that come with the machine, and linked into the Linux kernel as it boots.

I know a lot of you think of Compaq as a "Microsoft Shop", but they also sell about 1 Billion USD of Unixware every year, and support 11 different operating systems on their Intel platforms. As long as Linux helps them sell a significant number of hardware platforms, they will make the investment in supporting Linux on them.

7) How about the software no one wants to write?
by moonboy

How about the software no one wants to write? By this, I mean the software that most programmers would consider "boring", yet is truly essential to the further growth of Linux as a desktop and server OS. It's great that we have so many window managers, office suites, browsers, etc. both existing and coming down the pike, but what about the other stuff that's just not as exciting? The stuff you really have to pay people to write? Maybe third party vendors with paid employees are the answer, but will all of those companies want to make their software truly Open Source?

madog:

There are several projects underway which are looking at the funding of "boring" Open Source tasks. Some of them are quite interesting in their approach. One might be to fund scholarships for college students based on how much documentation they write, or have written. Or perhaps making it a co-op assignment. On the other hand, perhaps we have to be more prudent in how we make these "boring" tasks attractive. We all like to listen to the person who wrote the code, but what about listening to the person who wrote the documentation? Perhaps more people would write good documentation, if they were the ones invited to the many conferences and trade events occurring around the world, freeing up the developers to spend more time at home writing code (or even at workshops writing code).

8) Beer?
by Mike Hall

I have had the chance to meet you at several LinuxWorldExpo's and USENIX etc.. At each of these events, you were always present at the parties with a large glass of beer.

My question: What is your favorite beer? and why?

maddog:

I am not a great fan of the darker beers such as porter or stout, although this is not a hard and fast rule. For instance, I do like Guiness draught, and particularly when it comes from Temple Bar in Dublin.

As to the lighter beers and ales, I admit to being a beer snob, and I like few of the "national brands". Anchor Steam was my first "micro-brew", and still retains its unique flavor. Pete's Wicked Ale was a long-time favorite, but I feel my tastes have drifted away from it (or vice versa).

There are a lot of micro-brews that I like, and a lot that I really hate. Do not put any fruit in it, or strange spices, or "non-beer" things (such as hot mexican peppers). And PLEASE don't hand me a beer that you feel has to be improved by sticking a lemon or lime into it. Save it for watering the plants.

9) How you get the nick name?
by Kamelion

What ever happened to get the nick name "maddog"? Must have a pretty interesting story behind that, eh?

maddog:

(SIGH) I have told this story so many times.....but perhaps this will curtail telling it a few thousand times more....

Once upon a time I taught in a small two-year technical college. The Dean of Instruction was a fine gentleman, but we did not see eye-to-eye in teaching students, Often we would have arguments, and often the arguments would get heated. During these arguments, often the entire school would hear both the Dean and my opinions on many topics. And sometimes these arguments would get REALLY heated (like the time the Dean hired and fired me four times on the same day). It was during these times that the arguments were too hot for maddogs and Englishmen. The Dean was British....

Finally, my own question and answer:

Q: Why do you like Linux so much? Why do you spend time evangelizing it?

The computer industry has been very good to me over the past thirty years. I have seen computers go from room-sized monsters to things that would fit on your wrist, or at least in a small pouch on your side. Yet I feel that there are still a lot of answers that have to be found and even tougher problems that have to be solved by users.

I am fond of talking about the applications that I have seen running on Linux as I have traveled around the world. People working on understanding how the Universe works in places like Fermilab or Brookhaven National Labs, people working to find new paints, new sources of energy, and other research projects using Beowulf systems. I am interested in seeing people reduce the cost of embedded systems projects by using a well-written operating system that is scalable and free.

And finally I really enjoy seeing people working in the Health-care space, trying to disseminate information that can help cure diseases such as AIDS or cancer.

As I saw Linux spreading over the planet, and being used in places like Sao Paulo, Brazil, or Korea, or China, I knew that the planet earth had to take every chance to find the next great mind in computer science, and that it was less likely to find this mind in a closed-source environment that had all of computer development funneled through Redmond, Washington. We had to have a mechanism for finding the next "Albert Einstein" of computer science, and I see Linux as a magnifying glass, waiting to help us locate that person.

And so to you, Mr, (or Ms.) Einstein, wherever you are... become involved with Open Source projects, and give the world a hand. It desperately needs you.

md - at the beginning of the new century

Monday: Steve Wozniak. Tuesday: two special "surprise" interview guests.

After last Sunday's story about Microsoft looking for Linux skills, Alfred Perlstein wrote in with the news that talented FreeBSD admins can also find themselves positions with Microsoft, in particular, at Hotmail. The Hotmail guys do seem to have a sense of humour though; witness hostnames like rotate-the-shield-harmonics.hotmail.com.

Great answers to this week's interview questions. Mick Morgan, of the UK's CCTA [Central Computer and Telecommunications Agency] has turned this Q&A session into a truly detailed primer on how to choose the hardware and operating system behind a high-profile Web site - and has dispelled quite a few myths in the process. You'll want to read this interview even if you're not into server mechanics. It contains enough personal insight and wit to be of interest even to Slashdot's least-technical readers. (Click below to see what we mean!)

fprintf asks:
Seems like a simple question, but why Linux? It seems like all the other high powered sites are using BSD of one variant or another.

...and...

Raul Acevedo asks:
In the original Sunday Times article, you are quoted as saying:

"... you can't beat them [Linux on Intel] in the bangs for your buck department. It blows Sun out of the water..."

Could you elaborate on how Linux compares to Solaris? Did you mean that Linux blows Sun out of the water in terms of price/performance (which is obvious since Linux is free), or just in general for your particular needs?

I'd be curious to hear your thoughts on Linux vs. Solaris, not just in terms of price, but overall performance, reliability, maintainability, and ease of use. As a developer, I'm seeing Linux considered as an alternative to Solaris in many places, but there's little factual (or even anecdotal) information comparing the two.

ANSWER:

I'll take these two together since the answers overlap.

In retrospect, I wish I /had/ chosen OpenBSD ;-)

And I would certainly choose OpenBSD over GNU/Linux if I were building a firewall, or an intrusion detection system (based on say, Marcus Ranum's NFR) where packet capture at wire speed was important. (No - that tells you nothing about CCTA's network architecture....)

The choice of GNU/Linux seems to have caused all sorts of interest (witness this interview itself) when a *BSD may not have been so "controversial". Frankly I'm a little surprised at the reaction the choice seems to have generated. After all, we are just talking about web servers here. Many ISPs choose GNU/Linux on Intel for exactly the same reasons I have done - best value for money for the task in hand.

Let's put this into perspective first though - and dispel a few myths which seem to have cropped up in the press. I have emphatically /not/ ditched Solaris in favour of GNU/Linux. I still have 14 operational Solaris boxes running on the network. I have GNU/Linux running on 5 Dell Poweredge 2300s (with half a gig of RAM each - the Times article suffered from poor editing). I also run GNU/Linux on my desktop in the office, on my laptop and desktop machine at home and on a couple of internal servers handling DNS and proxy services for CCTA.

The GNU/Linux choice came about for two reasons:

• - I had operational experience of GNU/Linux on a day to day basis.
• - I was faced with replacing life expired Sun hardware (including a SPARC 1000E and a couple of Sparc 20s) as part of the normal process of hardware/maintenance/upgrade.

Having bought a Redhat distribution for my laptop, whenever I needed a quick solution to problem that would normally involve buying another box (such as building a secondary internal DNS server, or a local SMB server) I could simply recycle an old desktop PC with GNU/linux at practically zero cost.

On the second point. When the usual business planning round came up and I had to make decisions about hardware replacement for some of the older servers, it was obvious GNU/Linux on Intel could be a much cheaper option than simple replacement of the Sun hardware. Consider: a Dual 450MHz Pentium II, with 27 gig of disk, internal DDS3 and CDROM and half a gig of RAM costs less than £5000; a dual 300MHz UltraSPARC 2 with similar configuration costs around three times that. Question. Do I need to spend that kind of money simply to run a Web server? So I ran some tests and concluded that - no I didn't need to spend that kind of money (taxpayers money I should add) and plumped for the GNU/Linux on Intel combination on the purely pragmatic grounds of best value for money for the job in hand.

For the purpose of testing I took as a benchmark the maximum real life hit rate I had ever seen on one of the Solaris servers - around 1.5-2 million hits in a day. (By hit, I mean http GET or POST request). Then I doubled that as working assumption of a realistic maximum load in my environment.

For testing I took a fairly standard, but reasonably specced PC (a single Pentium 450MHz processor, 256Mb ECC SDRAM, single 18Gb LVD 10,000 RPM SCSI disk) and loaded Redhat Linux 5.2 running Apache 1.3.3. (Because that was what I had to hand). Apart from the Web server, I turned off all other daemons. I then loaded that server with a complete copy of my main www.open.gov.uk web.

In order to simulate a real life load, I had to find some way of grabbing a randomised list of URLS from the server which reflected the real world as closely as possible. After some testing with a variety of home spun scripts and commmand line web testers (such as webgrab) it quickly became clear that I would bog down the clients long before I made any real demands on the server. Some searching around and questions of colleagues lead me to http://alumni.caltech.edu/~dank/fixing-overloaded-web-server.html which is a useful site pointing to benchmarks and tools. This pointed me to http_load at http://www.acme.com/software/http_load/ which turned out to be pretty nifty since it runs in a single process. And of course, being OSS, I could tweak the code slightly to match my requirements. Thus armed I built some lists of URLs which were deliberately chosen to represent small text/HTML files, medium sized gif/jpeg files and large PDFs since this is the real life world on the public web servers. In load testing the server I then fired up just three client machines (one SPARC 5 running Solaris and two low end Pentiums running GNU/Linux since that was all I had to hand).

In peak load testing over a sustained 4 hour period I managed to get the server to deliver over 13,000 Mbytes in just under 500,000 HTTP transfers. During that period, CPU utilisation never went above 10%, and was usually around the 5% mark. Disk utilisation was minimal. The network connection rate was much higher than anything I'd seen in real life on the existing external servers (some 500 established connections during snapshots on the load testing period). Also during the test, Apache complained that it had reached the MaxClients setting (then 150) with no adverse effects.

Given that such a reasonably low end server handled most of what I could throw at it in my test environment, I concluded that GNU/Linux on only slightly beefier hardware made eminent sense.

----------

anthonyclark asks:
Do you get many cracker/script kiddie attacks on the various web sites you run?

ANSWER:

Yes ;-)

Any high profile site is going to attract unwelcome visitors. My job is made harder, and more stressful, by such attention - but that is what I am paid for. My friends know that I have nightmares about waking up to find graffiti (which is all it is) on one of my customers sites.

Like any other conscientious sysadmin I take a personal interest in the security of my servers. Naturally I will use all the tools at my disposal to minimise the vulnerabilities. But of course I get unwelcome attention.

A plea to the community if I may. And here I can do no better than quote from Fyodor's article in Phrack Volume 8 issue 54 where he discusses remote OS fingerprinting:

"A worse possibility is someone scanning 500,000 hosts in advance to see what OS is running and what ports are open. Then when someone posts (say) a root hole in Sun's comsat daemon, our little cracker could grep his list for 'UDP/512' and 'Solaris 2.6' and he immediately has pages and pages of rootable boxes. It should be noted that this is SCRIPT KIDDIE behavior. You have demonstrated no skill and nobody is even remotely impressed that you were able to find some vulnerable .edu that had not patched the hole in time. Also, people will be even _less_ impressed if you use your newfound access to deface the department's web site with a self-aggrandizing rant about how damn good you are and how stupid the sysadmins must be."

Sysadmins are not stupid. They are simply usually overworked and have to balance the need to provide services to their customer base with the need to minimise the risks to those services. Attacking public servers (whoever owns them) merely serves to irritate sysadmins, and usually nobody else.

I was not overjoyed to notice comments on /. of the form "whoo, so the Royal Web site has moved to Linux. I've got a rootkit with your name on it" (you know who you are). Consider. I have just moved some high profile web sites to the OS of choice to you readers. You want to see that OS taken seriously. Scribbling graffiti all over such a web site would have all sorts of negative impacts on the perceptions of people who matter.

Besides, you'd upset me.

----------

chromatic asks:
If you could add or change three things about Linux to make your job easier or more enjoyable, what would they be?

ANSWER:

1. The ability to read BUGTRAQ, evaluate the threat, consider vulnerability to that threat and auto patch or upgrade accordingly. It should then email me saying "I'm OK now, you can go back to reading /.".

2. An artificial intelligence based real time log watcher and network daemon which could learn network connect patterns and modify either the stack or the services running accordingly. The system should be capable of real-time blocking (a la portsentry) of "hostile" connects, co-operation with external IDS systems and firewalls, real-time reconfiguration of external security components, real-time alerts to other hosts on the lines of "hey guys, I'm being hit by X, watch it." It should then email me saying "I'm OK now, you can go back to reading /." :-)

3. An ASCII character based version of rogue. I miss it.

----------

Ryandav asks:
What kind of redundancy do you build into the server system for such a large and important site, ie. round-robin style servers or large, beefy superboxes, etc...

ANSWER:

You can see from answer above that I do not use "large, beefy superboxes". Frankly you don't need to to run a Web server. Nor do I use round robin DNS or other load balancing such as CISCO local director. In my experience of the sites I run, I don't need to do so. None of the sites gets hit hard enough to warrant the additional complexity of mirrored, load balanced servers. Our most popular site by far is the Royal Household site. That takes around 2-2.5 million hits per week (though I expect that to go up slightly now). The highest consistent hit rate I have seen is around 1.5-2 million hits per day. Any of the servers I have could cope with that. The redundancy we build in is in having backup hardware ready to run.

----------

wowbagger asks:
To what extent is the Royal Family involved with the site (e.g. content creation)?

ANSWER:

The Royal Family take an active interest in both of the royal web sites (one of which is hosted by the Press Association - www.royalinsight.gov.uk -). This interest includes both the current content of the sites, as well as future developments. The Queen herself launched royal.gov.uk in March 1997.

jd asks:
What's the official reaction to these sites running Linux? Assuming the British Government, and Her Majesty, are aware that their public image on the Internet is being presented via software that is non-traditional and non-commercial, what do they think of it all?

ANSWER:

The priority for the heavily visited royal web site is accessibility, balanced of course by reliability and security. These are the important issues, rather than the nature of the server operating system.

----------

Dicky asks:
What is your background? Are you a techie, an admin person, or an other? Do you use Linux personally? If so, did you come from a Unix, Windows or other background?

ANSWER:

I am a techie (though some of my friends and colleagues are a little less complimentary than that). My background is in Unix sysadmin and network management. I joined CCTA in 1993 from the UK Treasury where I was responsible for their Unix based OA system. Prior to that I was responsible for IT security in the Treasury. I have done some small systems development work in the past on MS/DOS machines (way before windows really took off) and CP/M micros. Most of my early career was in specialist support areas such as statistics, though I did a short stint in policy for a while in the mid to late 80's - didn't like it much.

Yes, I use GNU/Linux personally. It is my preferred platform for home use.

Dicky also asks:
And a related question: What is the primary system around your department?

ANSWER:

Depends what you mean by my department. In my area of responsibility the main systems are all *nix based. But the corporate desktop is NT4.

----------

Brian Knotts asks:
The obvious question: Does the Queen read Slashdot? :-/

ANSWER:

No. The Queen's interest in Internet matters is non-technical, although she sees on her visits to a wide variety of organisations the increasingly imaginative uses for the Internet.

----------

Simon Brooke asks: I've been very pleased lately to see Open.Gov's clear policy statement on the use of open standards. I'm personally involved in working with some large UK companies on their own Web standards policies, and having this to point to has been extremely useful to me. How difficult was it to get buy in to these standards by all the people who 'own' different Government sites, and how difficult is it to enforce?

I notice, for example, that the Scottish Parliament's web site, and my local Council's Web site, do not yet conform. Without wishing to point fingers at specific organisations, is it your intention to cajole all sites within .gov.uk to conform to these standards? Is it appropriate for members of the public to draw administrators of these sites attention to these standards?

ANSWER:

CCTA has long been a standards based organisation. My colleague Neil Pawley is CCTA's representative to W3C. Neil is also lead designer on the open.gov.uk site. Since CCTA is a member of W3C it is entirely appropriate that we should take a lead in using standards set by that organisation. Using HTML4, CSS2 and XHML1 for example on a real life server gives us valuable information on usability issues such as browser compatibility. Much of the feedback we have received has been very positive. On occasion we have had to deviate slightly from the standards where their use causes our public difficulty because of some incompatibility with a particular client setup. That experience itself is very helpful, since it allows us to feed back into the standards making process.

CCTA has an advisory role on best practice in the use of IS/IT in the UK Public sector. We have no authority to mandate particular standards, nor would we seek to do so. If the use of standards is to be effective in any way, it is because the standards themselves make sense in the real world (witness the growth in the use of the TCP/IP protocol set at the expense of the OSI standards).

Simon Brooke adds... Oh, and, by the way, keep up the good work!

We intend to.

Thanks for your interest. It has been educational for me.

-- Mick Morgan

-- end --

Next week: John Vranesevich of AntiOnline.

Computing has overtaken Sci-Fi. The evolution of UltraIntelligent (UI), Artifical Intelligence (AI) machines that are themselves a new species is just a few years away, predicts Dr. Arthur Clarke in his great new essay collection as do others in their writings and research. Today's kids will clearly witness the evolution of a species that's part machine, part human, or both. Humans need to scramble and learn in order to hold their own, says Clarke.

Guess what? They aren't.

"Can Androids Feel Pain?" Dr. John Irving Good of Trinity College, Oxford, asked in an essay published a few years ago.

It's a good question, one that year by year seems less rhetorical, less the stuff of fantasy, and more an ethical and social concern.

Inventor and author Ray Kurzweil projects that computers will match the computational functions of the human brain early in the next century, and that soon afterwards humans and computers will merge to become a new species.

As early as 1891 (in an article in the Atlantic Monthly), scholars and sci-fi writers have been writing about what many have seen as the inevitable fusion of men and machines.

Fantasists have also been drawn to aliens and the Space Age, themes still flourishing in epically popular evocations like "Star Trek" and "Star Wars." But if a new species arrives to dominate the earth, it probably won't come from distant galaxies. We're making it in labs and universities and teenagers bedrooms.

Good believes that humanity's survival depends on building Artificial Intelligence (AI) machines. More intelligent than we are, they'll answer our questions and solve many of our problems.

The great sci-fi novelist and essayist Arthur C. Clarke takes this idea still further in an ultra-brilliant collection, "Greetings, Carbon-Based Bipeds!" just published by St. Martin's Press.

The evolution of UltraIntelligent (UI) machines is imminent, Clarke predicts. Today's kids will witness the evolution of a species that's part machine, part human being and then, eventually, some combination.

"Perhaps 99 per cent of all the men who have ever lived have known only need; they have been driven by necessity and have not been allowed the luxury of choice," Clarke philosophizes. " In the future, this will no longer be true. It maybe the greatest virtue of the UltraIntelligent (UI) machine that it will force us to think about the purpose and meaning of human existence. It will compel us to make some far-reaching and perhaps painful decisions, just as thermonuclear weapons have made us face the realities of war and aggression, after five thousand years of pious jabber."

Clarke imagines AI machines taking over all but the most creative and trivial human work, inserting themselves into the loop between humans, work, creativity and entertainment.

To co-exist with UltraIntelligent (UI) machines and hold our own, Clarke posits, the entire human race, without exception, must reach the literacy level of the average college graduate -within the next 50 years.

"This represents what may be called the minimum survival level; only if we reach it will we have a sporting change of seeing the year 2200," Clarke says.

This also represents something that isn't going to happen. Except for the most technologically advanced countries - those in Scandanavia come to mind - even prosperous industrial societies like those in the United States, Western Europe and parts of Asia haven't begun to make education about new information technologies - or technology itself -- universally available to citizens.

In the United States, primitive politicians and journalists citing safety and moral issues argue for less, not more, access to technology. The only presidential candidate to make the Internet a major political issue is Elizabeth Dole, and she argues for more restrictions on youthful access to sexual imagery. This isn't a country trying to get to the minimum survival level Dr. Clarke writes about.

If Clarke is right, then for the first time we can begin to imagine a future in which the human race is no longer the planet's dominant species.

As he was thousands of years ago, man will again become a fairly rare animal, probably a nomadic one. Towns may still exist in places of unusual beauty or historic importance, but most homes will be self-contained and completely mobile, relocatable to any spot within hours. The continents will have reverted to wilderness; a rich variety of life forms will return.

It becomes clearer daily that we aren't going to be turned into alien pod people or probably even obliterated by the dread weapons we've been building. We are likely instead to simply become dumber, less durable, and les efficient than the computer-based machines we're creating.

A more concrete and hard-headed look at this evolution appears in Steven Levy's "Artificial Life: A Report From the Frontier Where Computers Meet Biology," now in paperback from Vintage. Levy opens his book describing creatures that cruise silently, seeing, reproducing, dying, even cannibalizing themselves for nourishment. The name of the ecosystem he describes is Poly World, located not in some jungle or forest but in the chips and disk drives of a Silicon Graphics Irix Workstation.

Levy calls this new species "a-life," (AL) and he argues that we're fast approaching the point where a-life will surpass our ability to control and shape it. As far back as 1980, he reports, the members of the NASA Self-Replicating Systems (SRS) unit confronted the possibility that artificial life would drive natural life out of existence.

Writes Levy: "The almost innate skepticism about whether it could happen at all, combined with the vague feeling that the entire enterprise has a whiff of the crackpot to it, assures that the alarm over what those scientists [making a-life] are doing will be minimal. The field of artificial life will therefore be policed only by itself, a freedom that could conceivably continue until the artificial-life community ventures beyond the point where the knowledge can be stuffed back into its box. By then it may be too late to deal with the problem by simply turning off the computers."

And what, exactly, are the problems? Will computers become conscious? Will they replicate our personalities and souls? Will they seek to push us and our inadequate and inferior ways aside? Will there be room enough for Us and Them? Will all this God-playing wreak havoc with the nature of human existence, as Mary Shelley warned a couple of hundred years ago?

Scientists, computing and otherwise, are hopelessly divided about the urgency of confronting the implications of a-life. Most don't think UI machines pose great danger to the human race, as long as we can turn them off when we want to.

"But can we?" scientist Norbert Winner asks in Levy's book. "To turn a machine off effectively, we must be in possession of information as to whether the danger point has come. The mere fact that we have made the machine does not guarantee that we shall have the proper information to do this."

Leaders of the artificial life movement are well aware of questions like this. But society at large has paid no attention whatever to the staggering ethical and other issues surrounding the science of artificial life. For most Americans, technology - as presented by a shallow political and media structure - is IPO's and start-ups, software and games, e-auctioning and e-trading, pornography or brain-damaging Net games. But AI threatens to alter human life more than all of them combined.

As much or more than any other social aspect of computing and science, AI, UI and AL suggest a monumental social and cultural story, however currently ignored. They won't be much considered until human beings discover a new life form imminently threatening to dominate the planet, or at least carving out its own space and behavior.

Pop culture, as usual, does a better job of raising these questions than journalism. Clarke's own "200l: A Space Odyssey" took a more malevolent view of computing's ultimate intentions than his non-fiction writing. And the looming conflict between humans and the AI machines they have made was at the heart of the evocative movie "The Matrix," which depicts a cataclysmic battle for survival between the human and mechanical species of the future. In fact, the "Matrix" asks the very question posed by Levy's scientists: will humans be able to turn the things off once they make them?

As the Space Age fizzles and the Digital Age takes shape, the sci-fi futurists and novelists are forgetting the alien invasion scenarios of the last half-century and turning their dark sides towards the evolution of the spiritual machines Kurzweil and others have been writing about.

The evolution of AI-life makes it even clear why the great sci-fi writers - Clarke, Verne, Asimov and Bradbury - have always had such hold on the imaginations of bright people. They weren't imagining the future so much as they were describing it.

El Volio writes "Believe it or not, Worth magazine has a very funny, subtly insightful article on the phenomenon of eBay addiction... " I've witnessed said addiction in people. Its more than just a little disturbing. Its fairly similiar to the Day Trading addiction I've witnessed in roommates too (the worst part is they don't trade, they just reload all day long).

opal_seven writes "Initially I was skeptical of the claims in the press release. But after witnessing the setup in operation, I'm still trying to pick my jaw up off the floor. Read it and (if your not living in Tucson) weep. "

Another Not So Anonymous Coward is pulling hair out over this issue: "I think this may not be the most appropriate forum for this, but I am at my wits end. I have been asking around and presenting this to the top minds I know, and it baffles all of them. So far, not a single person has been able to explain this, even on the Linux newsgroups. I am setting up PPP. And I have a problem. I am not a complete fool, so I know how to set up PPP correctly (Linux RH6.0, if you were wondering). Everything works perfectly, it connects, it pings it traces routes as far as the eye can see. But, if you ask for a http request, pppd gets a brain fart and starts acting like a mad man. I added the argument "record /tmp/ppp.log" to pppd to see what was happening, and I found the answer. Whenever I make a connection, pppd recieves the first 3-4000 characters of the page, I can see it in the log. I ususally get about 1/2 a page, BUT then it coughs. It waits a couple of seconds, and then STARTS OVER. It recieves again the first 3000 characters and hiccups again. So, I can use the web if I want to look at pages with less than 4000 characters, but anything more than that and it just hangs forever calling up the heads of web-pages. Does anybody have a clue as to why this is happening? (I know there is no porblems on the remote side becuase when I use windows 98, everything works peachy. It has to be my shiny new Linux box... grrrr!) " This sounds really odd. I admit that I'm stumped. Do any of you kind folks have any clues?

Veralden writes "Here is a story about Gates and another MS witness contradicting on how their sales data is recorded. According to Gates, they have sales results in digital form and a witness testified that they have paper sales records. MS denies this, but the article seems to take a more objective view. " Here is the story.

I-man writes "Extra extra! DOJ lawyer completely destroys the credibility of yet another Redmond Exec!" Wrap it up people. This is just getting to loony. I'm not sure which "Bill" related trial is more boring right now. They're both pretty darn funny though. Update: 02/23 02:24 by CT : cswiii sent us a nice link to a CNN story about a Six-week break trial. After which we'll finally get some closing.

whimsy writes "A senior Compaq executive credited Microsoft Tuesday with making computers easier to use, more reliable, and less expensive." It interesting to see how Compaq has been scrambling the past few days to not look like a "victim". This relates to John Rose, Compaq official, who's the next witness in DOJ trial currently going on. The article talks a little about the history of Compaq and Microsoft, as well as some of their disagreements.

Several folks have written in to comment that the Feb issue of Wired (7.02) has the "Slashdot Effect" listed in the jargon watch section. An anonymous reader noted that news.com has an article about domain disputes that mentions Slashdot and the whole ajax.org mess from ages back. And finally, Time Digital has an article that refers to " Slashdot Longhairs" (along with rapid apple partisan and java futurist) to describe how biased MSs recent DOJ witness sounds. I don't have long hair. Do you? Thanks to Matthew Rose for that one.

Sean Garagan writes "Well, it looks like MS is going to start using Linux to try and save itself. According to an article at PCWeek, MS will use Linux as an example of why it doesn't have a monopoly when it questions the gov'ts last witness, an MIT Economics prof. " If this is true, then I think it's a nice nail in their coffin- the only real threat to their monopoly is tens of thousands of programmers working for free? Great defense. We're not a monopoly, really.

Last night I witnessed the modern spectacle that is Star Trek:Insurrection. Hit the link below to read my review of the movie. The short summary: I can think of a worse things to watch, but I'd have to think pretty hard (a Full House marathon?) It was crap. Actually, I think crap would have been better than this. The plot (if it can be called that) involves our favorite gang of federation officers going to investigate some craziness on a planet in a part of space known as "The Briar Patch". But don't worry about it, it doesn't matter.

I had high hopes- I thought that last Trek movie was great. I thought Frakes had proven that he could make a good movie. This bomb overrides that thought, and replaces it with a sense of doom and despair that I hadn't thought possible.

The acting was rancid. Star Trek has never been known for quality performances, but this was even bad for Trek. The special effects were cheesy. The vast majority of shots basically scream "SOUND STAGE" at full volume. The plot was terrible, uninteresting, boring, predictable. The "Jokes" were offensively unfunny.

In short, this movie was worse than Star Trek V, which also, was complete crap.

Do not watch this movie. Do not consider seeing this movie. You will hate yourself for seeing this movie. Especially if you are a Trekkie. If you dig Trek, you will hate this movie even more. On a scale of 0 to 10, this gets a -2.

My favorite worst sequences (some spoilers) include one where where Frakes steers the Enterprise with a joystick. Plus every time a ship gets shot, steam shoots out of random places within the control deck, and someone has to go fix it physically. Plus our fearless officers never miss with their phasers. There are terrible shots where Troi and Riker are all flirty. And Worf gets a zit. I've seen worse movies, but I can't really think of any right now.

slana is desperately seeking assistance with the following issue: "Help!!! After three years I am almost MS free, yet each season my Cristmas and Business Card making needs have easily been met by "MS Pub." Hours and days with WP8, GIMP, SANE, SOffice 5.0 have met with limited and convoluted results. I am at what one might call 'an advanced novice at wits end.' Set me free from this seasonal 'Off the Wagon' syndrome!" Can anyone give tips on how to do these things using some of of the software listed? Is there better free software out there for this purpose?

b!X writes "A Federal court in Virginia has struck down Loudoun County's use of censorware in public libraries as unconstitutional. See a press release here and watch the ACLU for the text of the decision. "

Michael Welles, who was actually an expert witness in the case, also sent us the final draft of his deposition. He was asked to eval the effacy of filtering software.

So, whaddya think? Is this good or bad? I think that we need to strike some of this down, but I can also understand the library's position. -Hemos

In a very interesting twist, HP has announced that it will be using techniques it developed for EPIC for use in embedded systems: HP claims that the compilation techniques required by EPIC to make parellelism explicit in critical code paths can be used to create a custom hardware engine with sufficient hardware parallelism to meet the performance requirements of the application. If this were to be done automatically (by a compiler), HP would have a killer product that could explain their motivation for working with Intel on EPIC technology (it was originally HP's idea): HP has been focussing more on Embedded systems recently (witness their recent spat with Sun over their embedded Java) and with the workstation and PC markets becoming saturated this may be a very lucrative move on their part).

In an update to our previous article, AMD has revealed more details about its K7: rather than using RISC operations, the K7 uses MacroOps -- 15 byte bundles of instructions able to encode up to 2 primitive X86 instructions. The instruction decode unit serves up to 3 MacroOps per cycle (6 instructions at most). To reduce stalls, up to 15 MacroOps can be buffered and rescheduled before hitting one of the K7's three integer pipes: apparently this increases the amount of paralellism the K7 can uncover and exploit. The FPU's instruction buffer is even deeper at 44 entries. However the price of all of this is a huge die: 184 sq mm of silicon and the associated costs in decreased yields. Don't expect cheap K7s for a while.