Slashdot Mirror

prostoalex writes "USA Today notices significant rise of Linux in the high-end enterprise environment. Although it doesn't provide obligatory pretty pictures, the paper mentions the projects at Pacific Northwest National Laboratory and NASA. Also if you've missed the New York Times Google article of the day, the expose on John Doerr from Valley's venerable KPCB talks about venture fund investing $12 million in LinuxCare. NYT quote: "That's a freight train I wouldn't want to get in front of," said Mr. Doerr, explaining the importance to having a stake in a Linux-based venture. "Probably get run over.''"

We ran the "Call for questions" Monday, January 13, under the headline, Discuss BIOS and Palladium Issues With an AMIBIOS Rep. Note that Brian Richardson, AMI sales engineer, is a real engineer, not just a salesperson, and is also a staunch Slashdot reader who knows we have low tolerance for PR whitewashes around here. Brian's answers are real, not laundered, and he responded not only to the 10 questions we sent him but also to some he felt deserved answers even though they weren't moderated all the way up. Please note that in much of this interview he is speaking as "Brian Richardson, individual," and that his opinions do not necessarily reflect those of AMI's management. With that said, be prepared to learn a lot about the BIOS business, and how TCPA and Palladium relate (and don't relate) to it.

Preface:

I thought it might be handy for the audience to know who's handling their questions ...

My name is Brian Richardson. I work for American Megatrends, Inc . (AMI). AMI is a privately held company located in Norcross, GA (just north of Atlanta). We employ approximately 400 people worldwide (about 200 in the United States).

I am a "BIOS Sales Engineer", responsible for handling technical issues related to selling and marketing the AMIBIOS8 , our latest BIOS code revision. This includes writing whitepapers, demonstrating products, answering technical sales questions, speaking at industry conferences and handling requests from the press that may require more than a passing knowledge of technology (like this one).

I started at AMI in 1996. I've been in this job for two years. Before that I wrote BIOS code for our notebook team and helped design our Software Quality Automated Testing (SQuAT) system. I also maintain several company intranets and our Bugzilla server, used for tracking bugs during BIOS development.

In spare time, I serve on the board of directors of Tech Corps Georgia. I also managed the Hardware section of linux.com (old articles are archived at linux.omnipotent.net).

This interview covers BIOS in general, but the questions have a heavy slant towards TCPA & Palladium. I'm sure I won't address everybody's TCPA related questions here. AMI has a "TCPA and AMIBIOS8" whitepaper at our website which discusses AMI's implementation. There are also links to other information on TCPA.

To answer some of the more unusual questions that didn't make it into the Top 10:

• You use XOR to clear a register instead of a simple MOV instruction because of the instruction size (XOR uses a two byte opcode, MOV uses three bytes). The savings in space really adds up after a while.

• We haven't finished 1394 boot yet, but we do have USB & USB 2.0 boot support

• I don't know, I've never met Satan ... but I have been to WinHEC

Now on to the questions ...

1) On the Exclusionary Uses of TCPA

by the-banker

Is it (will it be) possible to use TCPA to effectively lock-out certain operating evironments from various services (software, media, etc) solely because the operating environment is not backed by a company, and has no mechanism for paying certification fees and licenses? Specifically, could TCPA be used against free OS's like Free/Open/netBSD and Linux to prevent those users from accessing the same content users of commercial OS's can?

Let me start out by reminding the audience I am not a security expert. I have been reading specs like a madman the past week, expecting such a question from the /. audience. I'm also not a professional TCPAadvocate ... my understanding of TCPA is in relation to what AMIBIOS must do to enable the TPM(a hardware component required by the spec). I'm going to refer toTCPA specifications & FAQ a lot, so verifying my answers will be an exercise left to the reader.

Your question brings up a lot of common issues people seem have with TCPA:

1. What does TCPA do?

2. What does AMIBIOS have to do with TCPA?

3. What is the licensing structure?

4. Can open-source software make use of TCPA?

5. Does this have anything to do with Digital Rights Management (DRM)?

Let's see if Brian can hash his way through these items in some sort of order ...

a) What does TCPA do? TCPA is an industry specification that defines mechanisms for "trusted" client/server interaction ("trust" and "security" are two different things).

TCPA works in a very similar fashion as other key-based security mechanisms (SSH, PGP, SSL). Transmissions are secured by hashing against a key. Keys tend to be very long (128 bits or more), so it is difficult for "bad people" to guess your key. In many mechanisms, the key also serves to identify the user (proof that they are who they say they are). This key is often contained in a file or some sort of removable media, like a smart card.

TCPA adds a few elements to this security scheme:

1. More keys and longer keys (some keys are 160 bits, most are 2048 bits)

2. A crypto-processor to speed key computations

3. Secure key storage on the system mainboard

4. Establish platform "trust". The two excerpts below are taken from the TCPA FAQ:
   12. What do you mean by trust?
   The ability to feel confident that the software environment in a platform is operating as expected. This is done by reliably measuring and reliably reporting (using aliasing) information about the platform.
   
   Another such benefit is improved control of access to data. Previously such access has depended upon authorization or authentication. Now such access can also be linked to the state of the software in the platform. This enables the denial of access to data if rogue software, such as a virus, is introduced into a platform, because such introduction necessarily changes the software state of the platform.

The crypto-processor and key storage are provided by the Trusted Platform Module (TPM). A TCPA enabled system will have a TPM on the motherboard. This TPM can be disabled, as per TCPA specification, if the user wants to opt-out.

One concern is that TCPA is equivalent to a unique identifier on your computer, which causes a large number of privacy concerns. There's a large section of the FAQ (Item #13) that covers this topic:

The solutions support privacy principles in a number of ways:

1. The owner controls personalization.

2. The owner and user control the trust relationship.

3. Provides private object storage and digital signature capability.

4. Private personalization information is never exposed.

5. User keys are encrypted prior to transmission.

6. Supports multiple certificate authorities giving the user choice.

It is also important to know what the solutions are not:

1. They are not global identifiers.

2. They are not personalized before user interaction.

3. They are not fixed functions - it can be disabled permanently.

4. They are not controlled by others (only the owner controls).

b) What does AMIBIOS have to do with TCPA? The TPM requires initialization during BIOS POST. This allows what they refer to as "metrics" to be stored that help establish that the BIOS & OS can be trusted (i.e. haven't been h4x0r3d). Our "TCPA & AMIBIOS8" whitepaper has more information.

c) What is the licensing structure? There isn't one. From the TCPA FAQ:

10. What are the licensing and/or royalty arrangements for the technologies outlined by the TCPA specification?

The TCPA spec is currently set up as a "just-publish" IP model.

d) Can open-source software make use of TCPA? Yes. From the TPM FAQ:

18. Does the TCPA support open source systems?

Yes. The ability to use the TPM functionality is available to all developers of software. An open source project could determine to use TPM functionally today. The concepts of measurement, protected storage and attestation of measurements are fundamental concepts that hold true for any type of OS or application. The platforms that support TCPA today are not limited to only one OS and if open source developers provided applications that used the TPM functionality they would find support.

Remember ... SSH, GPG and SSL aren't any less secure because they're open-source. The whole point of key-based security is that you can't see the data without the key, even if you know the decryption mechanism.

e) TCPA & DRM? This question wasn't directly asked, but it's on everybody's mind ...

TCPA has been connected to proposed legislation that would require "content protection" on most digital media devices (including PCs).

While somebody could write a DRM application using the TPM, they could also write one without it. Non-DRM applications can be developed under TCPA. The example I thought of is an improved VPN for companies that are super-paranoid about their data (think about it ... 2048 bit keys, no hash load on the system CPU, ability to tie accessibility to a unique platform).

Adding TCPA & a TPM to a system doesn't automatically add DRM to a platform. Some application has to tie the TPM to the "media" being "protected". Merely adding TCPA to AMIBIOS doesn't constitute DRM:

Captain: What happen?
Mechanic: Somebody set up us the DRM.
Cats: How are you gentlemen !! All your BIOS are belong to us.

2) Advantage

by TedCheshireAcad

What is the advantage to me, a Linux using consumer, to buying your product over those of your competitors?

First, the short answer: a proven and stable product based on nearly two decades in the PC industry, with support for the latest technology.

Now, the long answer: Let me give a little background on how BIOS gets onto your average motherboard. I know that's not what you asked, but it will explain product design and benefits to the end user.

AMI markets AMIBIOS directly to the motherboard manufacturer, who we see as the actual "BIOS customer". So many of our features are oriented to motherboard manufacturers or BIOS developer. The end result of using our codebase is to produce a stable BIOS for the motherboard manufacturer's customer (that's you, the end user).

You can break these down three major areas:

1. Code structure (ease of development, tools, source management, etc.)

2. Technology support (OS, chipsets, processors, peripherals, etc.)

3. Support after the sale

a) The "BIOS core" is a different code component from silicon support code. The same applies to our technology support modules (ACPI,USB, TCPA, ASF, SMBIOS, APM, etc.). This allows board developers to pick just the code they need for their system. An embedded Linux board for an industrial controller has different BIOS requirements than the typical "white box" motherboard (OS compatibility, supported hardware, power management, etc.).

AMI also developed a custom GUI to make BIOS development easier (Visual eBIOS, or VeB). Believe it or not, most BIOS development happens at the DOS prompt in x86 assembly code. We found it harder to get new engineers comfortable with DOS-based development (DOS is 22 years old, so is the average college graduate). VeB also incorporates source control, so engineers manage the code from the same place they edit the code.

b) Technology support is pretty broad. We have to work on new chipsets, technologies and devices while keeping backwards compatibility for older hardware we'd rather forget about. This involves a lot of work with hardware vendors (Intel, AMD, ServerWorks, nVIDIA, etc.), software companies (Microsoft, RedHat, etc.) and technical specification groups (there's one for most every acronym out there). As you might imagine, there's a lot of testing to make sure all these things play well together.

Technology support also applies to features that don't have cool three letter acronyms. One example of this is "Fast POST" (POST is Power On Self Test, BIOS execution from power-on to OS bootloader). There was customer demand to boot the PC faster. This pressure came from Microsoft for a better overall user experience (yes, the obvious joke is "boot speed doesn't matter when you don't have to reboot so often" ... but I'm taking the high road). So now Fast POST is standard in AMIBIOS8.

c) "Service after the sale" sounds like something you hear in a men's clothing store, but it applies to BIOS as well. Customers expect bugs to be fixed, new features to be added, and a voice on the phone when they can't quite figure out which bit goes where. Some customers develop using our source code (as a licensee), while others use our engineers to create their BIOS (as contractors).

That might have been more of a sales pitch than you were expecting (sorry). There's more product information at the AMIBIOS website.

3) Performance hit

by oliverthered

I assume that data pathways will be signable or encrypted in some way. What performance hit will the [operating system] take when using trusted system? e.g. How much extra data is added to form a signature, what methods are used for signing. and how will this benefit the end-user?

A: I assume this is in reference to TCPA, so I'll use what I know of that spec to answer the question.

Everybody who's used SSH or SCP has experienced computation overhead from data encryption. That's the main reason TCPA has the Trusted Platform Module (TPM). Along with storing keys, it had a dedicated crypto-processor to handle random number generation, hashing and digital signatures. Due to the size of a security key, these hash computations add overhead (overhead == delay).

In TCPA, the hash/generation stuff is offloaded to the TPM. Since this dedicated processor does the work, the main system processor doesn't have to. The TPM is also a function specific processor, meaning it's optimized for security tasks (translation: faster than your general purpose x86 CPU). This is a good thing, since most of the TPM keys are 2048 bits.

If you look at Transmeta's recent security press release, you see the same functionality. Although this story was reported as Transmeta releasing DRM, they are actually providing an integrated crypto-processor in the TM5800. This function-specific processor is accessible through an extension to the x86 instruction set (similar to MMX or 3DNow!). The difference between this & the TPM is how you access the functions.

Sidenote: does any open-source developer want to check if these extensions could be used to improve SSH, SCP or GPG performance?

The signing methods and potential benefits are outlined in the TCPA specification and FAQ.

4) Why are BIOSes closed source?

by mcelrath

Having recently had a lot of trouble with my laptop's BIOS, on an issue that I could most certainly fix if I had access to the code... I started wondering what benefit AMI and other vendors have by keeping BIOS code secret? I can think of none whatsoever.

An open-source TCPA BIOS might go a long way to alleviating the fears of the open source community, since we could see exactly what it is you're forcing on us. And hey, no doubt you'd get a few bug-fixing patches in return for your efforts.

So, is an open-source BIOS a possibility? (TCPA or otherwise)

Just to get this out of the way:

1. AMI isn't forcing anybody to take any product offering, TCPA or otherwise.

2. TCPA doesn't block open-source (see #18 in the TPM FAQ @ trustedpc.org).

3. The TPM Memory Present (MP) driver BIOS uses during POST isn't open-source (it's provided by the TPM manufacturer).

This was the focus of a linux.com article several years back. There's plenty of advantages to open-source, but there are two main reasons for closed source BIOS: Legal Restrictions & Economics.

The creation of an open-source BIOS isn't limited by the BIOS itself, but by the information required to create the BIOS. Let me take a second and explain how the BIOS works at a programming level. This may seem like a tangent, but it helps explain issues faced by open-source BIOS developers (just think of it as Good Eats for BIOS).

There's three major components of any BIOS:

1. Core Routines

2. Silicon Support Routines

3. Board Specific Routines

The core can be equated to the kernel of an operating system, except that it comprises a larger percentage of the codebase (both in functionality and actual code size). This is everything that's generic from one BIOS to the next.

Silicon Support applies to the chips on the board initialized by the BIOS (processor, northbridge, southbridge, I/O, flash). BIOS core routines will call silicon routines when hardware configuration is required. These routines are created according to an API, so swapping any of these code modules doesn't affect the structure of the core.

Board Specific Routines represent the motherboard manufacturer's configuration. If you look at motherboards from two manufacturers that use the exact same silicon components, you might expect the BIOS from one board to work on the other ... but you'd be wrong. The small hardware changes that differentiate Board Vendor A from Board Vendor B have a large impact on the BIOS. PCI Interrupt routing, chipset General Purpose I/O pins and other parts of vendor's "secret sauce" go into this BIOS layer.

"Fine," you say, "but what does this have to do with open-source BIOS?"

I'm sure you've noticed that there's a BIOS ready for a chipset the day it is announced. AMI and other BIOS companies don't just come along the day of the silicon release and slap a BIOS together. We work hand-in-hand with the chipset vendor for months before the release. They send us an alpha board, we boot it ... they send us a beta board, we add more features ... they send us final silicon, we validate it.

Now remember that this hardware isn't public when AMI gets it. AMI has to sign a has to sign a Non-Disclosure Agreement (NDA) to get a development board or advance specifications, which means we can't tell anybody what we know about the product. Vendor-supplied reference code (memory detection, bridge configuration, etc.) is also covered under NDA. AMI also signs NDAs to cover the motherboard manufacturer's confidential information.

So the BIOS that ends up on those motherboards is constructed using information we can't release to any party not covered by NDA. You might be able to understand how this doesn't fit into to the open-source model.

So an open-source BIOS developer has a big dilemma ... they need access to information, but legally can't include it in open-source code. Many chipset vendors provide information after their chipset is released, but not many board vendors hand out schematics. Reverse engineering might reveal this information, but some items controlled by the BIOS can damage the system if not set properly (data corruption, overheating, smoke, flame, etc.) ... so random bit flipping may not be the answer. And nobody wants to get into the legal issues of using disassembled code in place of reverse engineering.

I think the closing statement from the linux.com LinuxBIOS article still applies ... "The real question isn't if an open source BIOS will ever work on a handful of platforms, but if it will ever become viable for mass market across many platforms."

There's another issue that comes into keeping AMIBIOS source code closed (or for that matter anycommercial source code). This has to do with economics.

This is where I change hats from "AMI company representative" to "average techno-Joe". The next few paragraphs are my feelings, not necessarily those of my employer or anybody else on the planet.

I personally like the idea of open-source, and I use a lot of open-source programs at home and work (Mozilla, OpenOffice, RedHat, Mandrake, ClarkConnect, PostNuke, perl, php, Bugzilla). But I also buy and use regular closed-source programs (my DV editing and VCD/DVD authoring tools). The choice isn't whether or not the source is accessible, but if the tool fits my needs.

In either case, those programs are the product of somebody's time (in most cases, a large group of bodies). They're a conglomeration of people's ideas, a manifestation of their talents, and monetary investment (open-source isn't free to develop, somebody bought that computer hardware). Those people, and whatever company funded their efforts, have the choice to distribute their product anyway they choose.

If a company wants to go open-source, then they can't make money selling source or seat licenses. RedHat doesn't make money selling code, they make money selling a code package and support for that package. My company doesn't operate that way ... in the realm of BIOS, money is made licensing source and selling per-board licenses. That's the way every BIOS vendor makes money.

That doesn't mean there's no open-source within AMI (perl/php/PostNuke/apache intranets, Bugzilla bug tracking, ucLinux on our MegaRAC G2 management card). But the choice to go open-source is done product by product, company by company.

In an industry driven by innovation, many companies feel they loose competitive advantage by opening their source ... if everybody has access to their ideas, then why buy their product over another? That mentality may not fit well with open-source, but these inexpensive computers we currently enjoy are the product of market forces. If there was no profit in computing, would Intel and AMD even exist?

Thus ends my personal views ... back to the actual interview ...

5) Technical Explanation of BIOS Settings

by doppleganger871

I have been doing research on BIOS settings for many years, and I have found good articles on what the settings do, and how to tweak them for the best performance/stability mix. But, I would like to know if the BIOS manufacturer itself would be able to provide an in-depth manual of all the BIOS settings, and what exactly they do. All the manuals that come with motherboards are very short on explanations, and I would like to see someone within the company actually explain to us hardware enthusiasts the down 'n dirty, nitty gritty, dirt under the rug, needle in a haystack type of information that we could use to make our computers run the absolute best they can. Because, as we all know, optimizing software and firmware is a lot cheaper than upgrading parts.

A: I wish I had a great answer for this. Despite my verbose nature, there's not enough room in this interview to discuss every setting that is or will be in the BIOS. Some of the basic settings are covered in BIOS setup manuals, and a few websites do a good job of explain the ugly details. The problem is that those "cryptic" options change for every chipset on the market.

We're always looking at product improvements, and that includes documentation. Our setup manual is a generic template, designed for the motherboard customer as a starting point for their manuals. The "chipset specific setup information" is part of a new documentation effort within AMI (we talked about in meetings this week).

Outside of that, optimizing settings for a specific combination of board, memory and processor is still trial and error (tweak, reboot, benchmark, swear ... tweak, reboot, benchmark, swear ...). I don't know if better documentation will change that.

6) "Trusted" computer

by michael

A few related questions:

a) Isn't the goal of "trusted computing" to allow entities other than the owner of the computer to control what the owner does with his/her hardware? For example, "trusted computing" applied to music implies that the music publisher gains control over what the computer owner can do with the music data files. Isn't this the exact opposite of "trust" as that word is normally used - a trusted computer is one that can't be trusted by the computer's owner to perform the tasks asked of it, because other entities have veto power over the computer's actions?

b) Companies like AMI have repeatedly claimed that they aren't part of Palladium. However, isn't it true that without AMI's trusted BIOS (and all the other components necessary to build a "trusted computer"), Palladium wouldn't work? Why does AMI think they shouldn't be held responsible for enabling Palladium and similar schemes?

c) In what way does AMI benefit, financially or otherwise, from introducing a BIOS designed to make the computer it is installed in less useful to the purchaser of the computer? Please avoid saying that this is "optional"; AMI wouldn't create this BIOS if it wasn't intended to be used.

A: Let's take these in order ...

a) The Goal Of Trusted Computing: Despite the fact my company is a TCPA member company, the concept of trusted computing wasn't created by AMI (we're not even a founding member).

As far as the goals of the specification, I'm not the designated defender of TCPA. I'll let theTCPA speak to their own goals. You seem to automatically equate "trust" to DRM, but that's not what I get from reading the specifications and related materials (see part (e) of my answer to the first question).

b) Palladium & AMIBIOS: You are correct in understanding that Palladium will require some amount of BIOS support. The reason we keep saying "we're not a part of Palladium" is because Palladium doesn't exist in the marketplace ... it's a Microsoft initiative being developed under guarded care in a small circle of developers. It's not a public specification like TCPA, so our role in this scheme is unknown. My understanding is that we'll get a specification from Microsoft whenever they're ready to involve the BIOS developers, but I don't know under what terms it will be made public (my Magic 8 Ball says "Ask Again Later").

c) Financial Benefit: Yes, there is a financial benefit to supporting a technology that our customers ask for ... they continue to be our customers. Not every customer has asked for TCPA yet, but enough large customers have asked to make it financially reasonable. Keep in mind that this is just one more feature we offer, which the customer may or may not want to take.

So when a customer (or customers) comes to AMI and says "Our next motherboard will support TCPA, and we need a BIOS module", AMI has two choices:

1. Say yes, develop the code, make the customer happy

2. Say no

If we select option #2 (for whatever reason), our customer has one of two responses:

1. "No problem, we licensed your code ... we'll add the support ourselves."

2. "Too bad, you have a competitor who offers this support ... it was nice doing business with you."

Option B is an obvious downer, because customers give us money. Money can be exchanged for goods and services, like food ... and I find food to be an important part of a nutritious breakfast.

Option A presents another series of problems. Yes, we kept the customer, but now we have a forked version of our code floating around. If only one customer wants this feature, then it's not a big deal. If twenty customers want this feature, then there's twenty code forks. They're still our customers, so they expect support ... and this is a support nightmare.

Our decision to develop a TCPA option was driven by sufficient demand for the technology. We're not the only company in the marketplace offering TCPA. Phoenix, our largest competitor, has been working on TCPA for quite sometime. IBM is already shipping notebooks with TPM hardware (which run Linux, according to LinuxCare Labs). If AMI customers don't ship TCPA, they we spent time developing a feature nobody wanted (it wouldn't be the first time, but that's happens in cutting edge development), but we have customer goodwill because we're responsive to their needs. It's the same in our eyes as developing support for a chipset ... if nobody likes the chipset, then they don't buy the code to support it.

What we have done by choosing TCPA over any number of proprietary security solutions is present an option that isn't closed to third parties. If we enable TCPA on a board and you want to make use of it, read the spec and develop accordingly.

7) Hardware vendors

by cybermace5

Since a BIOS is only part of a motherboard: what steps will hardware vendors have to take, in order to incorporate your BIOS? Will they have to adhere to certain hardware design rules or controls in order to maintain the TCPA? Is there going to be a licensing procedure for hardware manufacturers?

A: Hardware vendors don't have to do much for AMIBIOS to support TCPA. The TCPA code module gets included as an add-on. The hardware manufacturer has to obtain a TPM to place on the motherboard, but that's available from a third party vendor.

The TCPA specification doesn't mandate licensing (see point #10 in the TCPA FAQ). It's not an AMI specification, so it's not our job to check for compliance. Third-party labs will most likely perform platform certification based on TCPA specifications.

8) Windows override

by Forkenhoppen

I have a question; on previous occassions on VIA hardware I've owned, I've noticed that occasionally, Windows will enable a feature even though I have turned it off in the BIOS.

My question is this; if I have TCPA disabled in my BIOS, will Windows drivers abide by this? Or will they still be able to use aspects of the BIOS originally put in place for use by TCPA even though I have it shut off?

What plans are in place to keep a Windows driver from hijacking TCPA-related information for it's own purposes?

A: A lot of that depends on how the motherboard vendor implements the TPM disable option mandated by the TCPA specification.

The TCPA specification has many options for disabling the TPM. It can be a BIOS setup question, jumper or software driven. The first two would be really hard to override in software (unless there's a robotic hand attached to the USB port). The third option could present a software override, but you would have to reboot to have the TPM enabled at power-on to set proper "root of trust" (you can't just turn it on midstream, since a TCPA system is supposed to hash the BIOS & bootloader).

9) TCPA & Palladium

by ignipotentis

Perhaps you can clarify the differences between the two (TCPA & Palladium). After reading up on both of them, i still find that they seem to be pretty much the same, just marketed differently.

A: From the information that's been made public concerning Palladium, I can try to elaborate on this. As I understand it, the major differences are listed below:

• Curtain Memory

• Control of Specification

• Intellectual Property (IP) Rights

The last two points are pretty self explanatory. Palladium it not a public specification, there may be licensing issues. TCPA is a public document created and reviewed by a number of different companies, with no licensing demands.

The first point is technical in nature. Here's how the Microsoft's Palladium FAQ describes "curtain memory":

The ability to wall off and hide pages of main memory so that each "Palladium" application can be assured that it is not modified or observed by any other application or even the operating system

This type of mechanism doesn't exist in TCPA, and would probably require some sort of support at the chipset level (which means it couldn't be implemented using current northbridge hardware). The total system impact isn't known, and it's any body's guess what this does to application development.

10) What do you think about Linux BIOS?

by lanner

At first, I was going to ask you about how you have cooperated, if at all, with the Linux BIOS project. After all, you often have historically cooperated with Microsoft and Novell. What are you doing to help Linux?

But then it occurred to me, if Linux BIOS was successful, it would put AMI out of the BIOS software development business. Linux BIOS is a competitor of AMI.

What is your personal perspective about Linux BIOS, and what does AMI think about it?

A: There's a lot of overlap with question #4 here. But there are two points I'd like to touch on:

1. Cooperation with Microsoft, Novell & Linux

2. Perspective on LinuxBIOS

a) Saying that we "cooperate" with Microsoft and Novell is misleading. AMI creates AMIBIOS for maximum hardware and software compatibility. For years, Microsoft and Novell were the primary OS vendors used by our customers. Microsoft also drives many PC specifications, and the majority of our customers use Microsoft operating systems. Development and testing are focused based on customer demand.

In the past few years, that situation has changed. Novell isn't a major consideration for our customers, but we still test compatibility. Linux is demanded by more customers, and our testing efforts have been increased to match that demand. We test RedHat, SuSe, Mandrake, Xandros, Lindows and FreeBSD by default (along with various beta distros).

Microsoft is still key to our testing and development (we test everything back to Win98). Customers still need that "Designed for Windows" sticker. But Linux is a major focus in our testing and development ... not just because we develop for compatibility, but because our customers ask for it by name.

b) In some areas, people see LinuxBIOS as competition to the other BIOS vendors.

• As far as the source licensing (open vs. closed), see my answer to question #4.

• In features, LinuxBIOS does some things that our BIOS doesn't (mostly in the areas of cluster management) ... AMI has advantages over LinuxBIOS as well (boot from USB/USB2, JPEG graphics as boot logo, broader chipset support, ACPI/APM power management, etc.).

• LinuxBIOS was developed for a specific application, but has broadened ... AMIBIOS aims to offer broad support in many market segments.

• AMIBIOS has been tested against a larger number of system configurations, works with a larger variety of hardware, and has a longer product history.

I'm not sure how others at AMI feel about LinuxBIOS, but all I have to say is "go for it". There's some neat stuff coming out of that project, and it's interesting to see what they've accomplished. Competition in the market is what makes technology improve ... one notch better than the last thing, one step ahead of the next guy.

Thus ends the interview. Thanks to Slashdot for the opportunity, and thanks to the readers for wading through the text.

Andrew Klaassen writes "Layoffs at SGI yesterday hit, among other things, the XFS for Linux project. Project lead Steve Lord writes, "We do intend to keep working on XFS linux, and I do intend to work really hard to get it into the distributions and Alan and Linus's kernels, [but] it will take us a little while to regroup our efforts and to work out our priorities on the project...." He also mentions that LinuxCare will no longer be helping out with funding for the port."

Dave Sifry writes: "Rusty Russell has posted the scripts that render a function call graph of all of the .c files in the Linux kernel. Each file is graphically represented and named, and function calls are graphically represented inside of each source file. The end result is a 180MB vector PostScript file. You can get the source code and then render it yourself, or if you just want to get the big finished poster, send an e-mail to Linuxcare or contact EverythingLinux if you live down under. BTW, it took us about 5 hours to get that file to print at a print shop down at LWE - that much vector PostScript really tested the limits of their big printers!"

Dave Sifry writes: "Rusty Russell has posted the scripts that render a function call graph of all of the .c files in the Linux kernel. Each file is graphically represented and named, and function calls are graphically represented inside of each source file. The end result is a 180MB vector PostScript file. You can get the source code and then render it yourself, or if you just want to get the big finished poster, send an e-mail to Linuxcare or contact EverythingLinux if you live down under. BTW, it took us about 5 hours to get that file to print at a print shop down at LWE - that much vector PostScript really tested the limits of their big printers!"

That much software sucks -- perhaps most of it -- is hard to dispute. Except for the simplest programs, it seems like the price of complexity is a tendency to failure. Commands don't work, user interfaces are neglected to the point of ruin, and components of even the same piece of software often clash with each other. And once you start combining them and try to use more than one application at once, sometimes the best you can hope for is an operating system that neatly segregates the problems so that your word processor doesn't take down your web browser, your IDE or your e-mail client. At least those are desktop applications for individual users, though -- the trouble compounds briskly when the common faults of software manifest in multiuser environments, where one machine going down means a wasted time and frustration for a lot of people at once. In an effort to outline the ways that software could suck less is coding, reading and writing dervish chromatic.

Making Software Suck Less - Processes Once upon a time, a prominent writer and programmer rose to declare "I want software that doesn't suck!" He then explained that certain successful free software projects have similar development features that contribute to software quality. Most of us aren't gifted with the organizational clarity of a Linus, or the brilliant non-orthogonal design of a Larry.

There's hope, though. Improving the ways in which we produce software can dramatically improve the software itself. Extreme Programming suggests that simple habits, acting in concert, produce extremely powerful results. By adapting these techniques to the unique world of free software, we can improve the quality of our programs. We start by restating some common truths about free software development.

Distributed Development Open development, of course, means that anyone with the time and the inclination can look at the code. Developers and dabblers have the opportunity to modify it to their needs. This by no means guarantees that anyone will do so. Making source code available is no silver bullet. Many qualified eyes actively looking for bugs make bugs shallow.

To harness the power of additional developers, you must attract them to the project. A simple design and clear documentation reduce the amount of work needed to come to grips with the system. XP suggests an overall metaphor to describe the system as a whole and to provide good names for the components and subsystems.

Test First The most powerful weapon in your toolbox is comprehensive unit testing. (Unit tests are automated, independent procedures that exercise the smallest possible pieces of functionality individually.) Extreme Programming recommends a test-first approach. Before adding a new feature, the programmer writes a test for the feature and runs all the unit tests. The new test fails, so he writes code to pass the test. When all tests pass, the feature can be checked in. Tests cover everything that could possibly break.

When a bug appears, the programmer writes tests to demonstrate it. After fixing the bug in the code, he runs all tests again. This not only proves that the fix corrects the defect, but it proves that the correction did not break any other features. Besides that, it can prompt the programmer to write additional tests he had not previously considered.

Test-first programming ensures that all features have unit tests. Coders receive immediate feedback -- it's almost like a game. It produces a different mindset, freeing the programmer to concentrate solely on the task at hand. Code becomes easier to write, in the same way that finding the correct piece for a jigsaw puzzle is easier with its neighbors in place. Finally, it gives programmers the confidence to refactor, modifying existing code, by identifying the effects of a change.

Consider providing your test suite with the project. Add a 'test' or 'check' target to the Makefile. Projects designed to run on multiple platforms and distributions can generate better bug reports by including test results. Make it easy for users to report failures.

Simple Solutions First After writing a test, write the simplest possible code that could pass it. Resist the tendency to make things more flexible than you need at the moment. Concentrate only on the task at hand, programming only what you need to pass the test. Don't sacrifice current elegance and simplicity for a feature months down the road.

"It's true that 'simple and tested code means less bugs'.... Good and clear interfaces reduce bugs. Avoiding complexity reduces bugs. Using your brain before you code something up reduces bugs."
-- Linus Torvalds

Good tests free you to change things in the future by identifying the effects of a change. Simple code localizes changes, reducing interconnections. Besides that, your design will change. Adding a feature will take the same amount of time whether you do it now or in the future. Don't spend time and energy overgeneralizing for something six months away when no one will use it in the meantime. Reduce the need for costly and time-consuming rewrites by avoiding extraneous complexity. Have a Plan, Code For Your Users Write a plan for the software. Describe each feature in a paragraph or less, with sufficient detail that people will know when it is complete. Arrange the stories by importance, then tackle them in that order. This allows you to identify the work that will provide the most value to the customer. (In a free software project, the lead developers may be the customers.)

Dividing the project into stories allows delegation, especially in free software projects. Some tasks require an experienced hacker, while others serve as a gentle introduction to the program. Writing stories also provides sane goals and a project roadmap. Choose a few stories for each release. This gives end users a clear view of project progress.

Continuous Design, Refactor With tests in place, you have the freedom to refine your initial design. By using the simplest solution first, you avoid investing time in hard to follow and difficult to maintain code. Your initial design will change. Your plan will change. Expect this and allow it to happen.

"Premature optimization-including premature low-level design--is the root of all evil."
-- Michael Abrash

When you see an opportunity to refactor, do it! Eliminate duplicate code. Relocate common features into a new object or a function. Reduce complexity and interconnections. Don't maintain the same functionality in multiple places. Use simple, well-defined, and consistent interfaces. This, along with your test suite, will allow you to overhaul individual pieces without having to track down holes in far-flung files. Release Often, Release Working Features Commit to regular release dates. This shortens the feedback loop. Users who can count on regular, stable releases with valuable new features will be more likely to use the software. It also provides more frequent entry points for new developers.

Follow your plan to add a few important features to each release. Focus programmer time and effort towards a simple, reachable goal. Allow time for design, testing, documenting, and packaging. The more often you have to do this, the more likely you are to streamline these processes. Resist the urge to add features you cannot complete in time. Instead, break them up into smaller stories and implement the most important parts. As usual, the unit tests help to stabilize the codebase and keep it in a state of stability. After completing a release, take time to modify your stories, shuffling priorities as necessary. Then commit to the next release.

Conclusion If we truly want excellent software, with well-developed features and no messy surprises, we need more discipline in our approach to creating software. In my experiences with these techniques (and in talking to other developers), I have found that even simply migrating to a test-first approach, though painful, has increased my productivity and my confidence. It's scary at times, but immensely satisfying. Taken together, who knows what levels of excellence we can reach?

Cerb writes: "Open Projects sounded most appealing as I was on the prowl to find one of the many IRC networks to link my idle server to. Having been involved with the open source community for quite some time, it seemed to me to be the place to be. It just seemed 'right' to link there. Now I'm simply stuck with these people. They are exactly what I was looking for. But, when I decided to help out, little did I know that I would end up in a project manager position. So here I am, it's 12.24 am in Vienna, and I'm mulling how to phrase this best."

"OPN have thought a lot about the current development of community sites for the open source community. While we believe that sites like sourceforge and the like do a good job in their category, there are not enough project-related news sites.

We have decided that it is time to gather information, ideas and reminders from the open source community as to how they would like to have a site developed, that actually tells them, all in one place, what is going on with people in the community and the projects they work on.

So we're approaching the community, trying to gather as many contacts as possible and start discussing this idea we had during one long frenzied night of IRC brainstorming. Users who think they can help us out with ideas, development time, written content, information about the open source community and the like are asked to please participate and sign up to our mailing list at http://lists.openprojects.net. That same host runs an nntp server gatewayed to the mailing list archives, in the newsgroup opn.discussion. If you feel you are qualifying as a volunteer right away, you can sign up to our volunteer database using http://lists.openprojects.net/db/

Spreading the word is not easy and wording this right is even harder, so I will not try to make this sound any better. I'll just send it off, hoping that it gets posted and many join us. Should you have more specific questions, please join me on IRC. The server is irc.openprojects.net, the channel is #content.

oh yeah... I am dmalloc on IRC and right now I have quite a problem allocating more sanity.

cheers...

On behalf of dmalloc, lilo and the rest of the gang on irc.openprojects.net"

So who's up for this? I'd like to see a by-project community site that goes beyond the few excellent sites over at LinuxCare's Kernel Central, like Kernel Traffic, though I wonder how hard it will be to pull off an organizational task of this size.

Cerb writes: "Open Projects sounded most appealing as I was on the prowl to find one of the many IRC networks to link my idle server to. Having been involved with the open source community for quite some time, it seemed to me to be the place to be. It just seemed 'right' to link there. Now I'm simply stuck with these people. They are exactly what I was looking for. But, when I decided to help out, little did I know that I would end up in a project manager position. So here I am, it's 12.24 am in Vienna, and I'm mulling how to phrase this best."

"OPN have thought a lot about the current development of community sites for the open source community. While we believe that sites like sourceforge and the like do a good job in their category, there are not enough project-related news sites.

We have decided that it is time to gather information, ideas and reminders from the open source community as to how they would like to have a site developed, that actually tells them, all in one place, what is going on with people in the community and the projects they work on.

So we're approaching the community, trying to gather as many contacts as possible and start discussing this idea we had during one long frenzied night of IRC brainstorming. Users who think they can help us out with ideas, development time, written content, information about the open source community and the like are asked to please participate and sign up to our mailing list at http://lists.openprojects.net. That same host runs an nntp server gatewayed to the mailing list archives, in the newsgroup opn.discussion. If you feel you are qualifying as a volunteer right away, you can sign up to our volunteer database using http://lists.openprojects.net/db/

Spreading the word is not easy and wording this right is even harder, so I will not try to make this sound any better. I'll just send it off, hoping that it gets posted and many join us. Should you have more specific questions, please join me on IRC. The server is irc.openprojects.net, the channel is #content.

oh yeah... I am dmalloc on IRC and right now I have quite a problem allocating more sanity.

cheers...

On behalf of dmalloc, lilo and the rest of the gang on irc.openprojects.net"

So who's up for this? I'd like to see a by-project community site that goes beyond the few excellent sites over at LinuxCare's Kernel Central, like Kernel Traffic, though I wonder how hard it will be to pull off an organizational task of this size.

Cerb writes: "Open Projects sounded most appealing as I was on the prowl to find one of the many IRC networks to link my idle server to. Having been involved with the open source community for quite some time, it seemed to me to be the place to be. It just seemed 'right' to link there. Now I'm simply stuck with these people. They are exactly what I was looking for. But, when I decided to help out, little did I know that I would end up in a project manager position. So here I am, it's 12.24 am in Vienna, and I'm mulling how to phrase this best."

"OPN have thought a lot about the current development of community sites for the open source community. While we believe that sites like sourceforge and the like do a good job in their category, there are not enough project-related news sites.

We have decided that it is time to gather information, ideas and reminders from the open source community as to how they would like to have a site developed, that actually tells them, all in one place, what is going on with people in the community and the projects they work on.

So we're approaching the community, trying to gather as many contacts as possible and start discussing this idea we had during one long frenzied night of IRC brainstorming. Users who think they can help us out with ideas, development time, written content, information about the open source community and the like are asked to please participate and sign up to our mailing list at http://lists.openprojects.net. That same host runs an nntp server gatewayed to the mailing list archives, in the newsgroup opn.discussion. If you feel you are qualifying as a volunteer right away, you can sign up to our volunteer database using http://lists.openprojects.net/db/

Spreading the word is not easy and wording this right is even harder, so I will not try to make this sound any better. I'll just send it off, hoping that it gets posted and many join us. Should you have more specific questions, please join me on IRC. The server is irc.openprojects.net, the channel is #content.

oh yeah... I am dmalloc on IRC and right now I have quite a problem allocating more sanity.

cheers...

On behalf of dmalloc, lilo and the rest of the gang on irc.openprojects.net"

So who's up for this? I'd like to see a by-project community site that goes beyond the few excellent sites over at LinuxCare's Kernel Central, like Kernel Traffic, though I wonder how hard it will be to pull off an organizational task of this size.

To all but a handful of a handful, a perfect filesystem would be boring: files would be where you expect them to be, corrupt files would never be an issue, and a power outage would result in nothing more than a few moments of darkness rather than minutes or hours of lost work. Even so -- perhaps a clue that the perfect example is far, far away -- the news on the Linux filesystem front is pretty exciting of late. In a low-key technical session Friday morning at ALS, kernel hacker Daniel Philips announced to the world the minor revolution he's planning -- which could end up replacing Linux's old standby ext2fs (and it's coming replacement ext3fs) with his Tux2 filesystem. Though Tux2 is an ext2 cousin in many ways, it carries at least one crucial improvement: according to Philips, you can literally pull the plug on a system running Tux2 and expect not to lose files or spend minutes watching fsck crawl by.

Phillips, an expatriate Canadian now employed by Berlin-based Innominate AG, claims 25 years of computer programming experience. He's had stints in everything from database design and game programming to embedded controller system development, and in a dual life which may sound familiar to many computer programmers, Phillips worked through music school by hacking Fortran code. With that background, perhaps it's unsurprising that just a few years after first encountering Linux, and a year from joining the ranks of the kernel hackers (there's a +5 informative thread in Zack Brown's excellent Kernel Traffic), he's come up with what could be a sea change in Linux filesystems.

A Filesystem You Can Live With And Pull The Plug On The central point of a journaling file system is that in exhange for a small hit in performance, file integrity is assured by an ingenious mechanism: rather than being written directly (and riskily), filesystem changes are instead first recorded sequentially in a running list -- the journal -- the contents of which are then acted upon in turn. If the system should crash for any reason while a change is not yet accomplished, the recovery time upon reboot is greatly abbreviated, as long as this "edit decision list" remains intact. Journaling file systems are on the way from multiple projects, and rather than being theoretical, wouldn't-it-be-nice daydreaming, at least one is availble right now: the ReiserFS developed by Hans Reiser is even an option at install on some recent Linux distributions.

Why another, then? Wrong question: Tux2 is not a journaling filesystem. Phillips says that Tux2 offers Linux users the chief advantage of a journaling filesystem (namely, keeping files safe in the event of a system crash) but without a journal, and does so more efficiently.

"The big deal is when you compare it to journaling, which is a popular solution, and you see that it's just plain writing less blocks. That's a big savings. It's also not constantly going back to wherever the journal is on the disk to write to the journal, so there's a lot less seeking involved. So those two things together means that it should significantly outperform journaling." Perhaps more importantly, Tux2 is not actually a wholly new filesystem per se; it shares so much in common with ext2 that it is built as a patch to ext2, with the filesystem converted at runtime. How does Tux2 get around keeping a journal to do the things that a journaling filesystem does? Atomic updates are the key. (See also: soft updates) Instead of a journal, Tux2 uses what Phillips terms a "Phase Tree algorithm."

"I originally called it Tree Phase," he says, "and then Alan Cox mentioned it on the Linux kernel list. He called it Phase Tree on the Linux kernel list, and I decided I liked that better." The Phase Tree algorithm is simple at heart, but takes a little while to grasp -- at least it did for me. Happily, Phillips has written a lucid tutorial on his own site. Probably the best explanation is the one found on Phillips' project site: the exerpts which I found most illuminating are these:

All accesses to filesystem data are performed by descending through a filesystem tree starting at its metaroot.

Normally, three filesystem trees exist simultaneously, each with its own metaroot. One is recorded on disk with a complete, consistent tree descending from it. A consistent second tree, the 'recording' tree, in the process of being recorded to disk, descends from a metaroot in memory, and some of its blocks are in dirty buffers. A third tree, the 'branching' tree is in the process of being accessed and updated by filesystem operations, also with its metaroot in memory. The branching tree and is not required to be internally consistent at all times. In particular, some blocks that are free in the branching tree may not be marked as free in its block allocation maps but held on a 'deferred free' ('defree') list instead.

At some point the recording tree will be fully recorded on disk and its metaroot can be written to disk so that it replaces the metaroot of the recorded tree. This causes the filesystem to move atomically between states, as desired. At this point, the recording tree becomes the recorded tree, and the branching trees metaroot is copied to become the new recording tree. This event is called a 'phase transistion' and the interval between two such events is called a 'phase'.

"The problem is, it's not nice to block filesystem transactions. If you're using a KDE desktop or similar, you find your desktop moving in a very jerky way while the blocks are getting written -- no good. That's why we make another tree by copying the metaroot -- that's how we always start, we never start one by going up the tree -- meanwhile this second tree is undisturbed by that and can be written to the disk in peace."

This additional copy allows the user to work without noticing a system slowdown, while the intermediate branch is copied. Thus, there are always three "trees," and in the event of a system crash, recreating the system's correct state is as easy as identifying the latest succesfully written tree. "Each new tree is always incremented higher, so this is easy," Phillips says.

"There are a couple of other places where [Phase Tree] is obviously better than journaling. For instance, removeable media -- your removeable media is usually slowest, and you don't put a journal on it, because if you did, it would be really, really slow. So you put the journal on your hard disk, and the data on the removeable media. As soon as you pull your removeable media out, you have instant corruption, because you've removed yourself from your backup. Phase Tree doesn't do that -- you can just pull out your removeable media and you have something current up to the last tenth of second, quarter of a second."

Sleepless nights and database integrity Phillips' work with Phase Trees began a decade ago, when he implemented a system with similar functionality for a specialized database called Nirvana which he had developed on his own. "I would have implemented this on a Unix filesystem at the time as well, except I didn't have one available."

Was there a Eureka moment in 1989? "Oh yeah. I dimly recall having a a week of sleepless nights, tossing and turning, trying to figure out if it was even possible to do something abot this, and eventually convinced myself that it was. And as I recall, it was quite tricky to get it to a hundred percent state, not 99.99. I could smell the idea in there, but I couldn't find it's actual realizaton for some time. After that, the generalization of its application to a general file system is pretty obvious."

Still, the idea stayed with him until he realized it would be an interesting way to improve the performance of Linux systems.

Like the puzzle with square pieces sliding around a single missing square, only scant disk resources are used to accomplish the extra data's movement because the information is moved incrementally -- in blocks rather than all at once. That means, says Phillips, that "It really adds very little [disk] overhead. Something on the order of 1 percent."

Additionally, it has one more feature which may appeal to the fsck-hater in you: "Really, it's nearly a defragmenter already," Phillips says. "It would be trivial to add that functionality."

The dual advantages of lower overhead and -- most importantly -- a close relation to the ext2 file system should make it an easier transition for most users. Tux2 is actually built as a patch to the ext2 filesystem; standard ext2 filesystems are converted to Tux2 at mount time. According to Phillips, that conversion takes on the order of a tenth of a second per gigabyte on a typical system.

Fly In The Ointment Though Phillips downplays their significance, patent difficulties may lie ahead for Tux2 as well. Network Appliance applied for a patent in the early 90s which covers similar ground -- a few years after Phillips had implemented it in his database.

"What really steams me in this is that their [patent] application came three years after my invention," says Phillips. "I hate to use the word infringe, because that makes me sound like the bad guy -- but it seems as though my [method] doesn't infringe beause it uses a different algorithm. In fact," he says, "I've got two things: I've got prior art, and I've got a better algorithm ... We can fence them in [legally], so their best strategy is to be nice, but they haven't figured that out yet."

"I don't want to suggest that NetApp got the idea from me -- I don't think they did, I think they developed it independently. The only little problem is the chronology of it. I concieved the whole thing, essentially everything that they've written in their patent, so I was kind of upset when I saw it. I would have gone on to do in on a Unix file system at the time, if I'd only had one available. We know it's stupid, but you see people patenting things all the time on the web -- just because it is a business idea that is now being done on the web." The approach that Phillips has to the dispute is to simply keep working. "I don't want it to become a distraction, I just keep doing what I'm doing."

Do penguins have calendars? Phillips says that Hans Reiser has approached him regarding integrating the file protection capabilities of Tux2 with the additional features of ReiserFS. "But it's pretty obvious where the priority has to be," he says, noting that ext2 is the default file system, and isn't going away any time soon. "Ext2 is what everyone has by default, and that's too big to ignore."

Does Phillips anticipate Tux2 becoming the default file system in Linux systems? "Well, who knows what's going to happen?" he laughs. "It could. But you can be sure of one thing, Tux2 will live a fairly long life as an independent patch that people apply, and I will be the 1st to apply it. But sure, of course I'd like that."

With a caution that fits someone whose last job was in embedded controls, Phillips warns against putting Tux2 in too soon: "It has to be proven, it has to be 100 percent. Because that's the whole point of this, is to 100 percent. So I think any bug which is not an ext2 bug already is just not acceptable."

And ultimately, like any other possible low-level change, "It's up to his high penguiness." Besides which, "it's quite clear what the next Linux filesystem standard is going to be. Well, it's my opinion that ext3 is going to be the most popular standard linux filesystem next year. And a couple years after that, well, I certainly will be using tux2 all the time, and we'll see where it goes."

The current status is heavy development: "I want to give it as a Christmas present to myself and start using it in my root system for my own development," says Phillips, "as soon as I port it to [the 2.4 kernel]." Soon after that, the code will be released to the developers on the Tux2 mailing list which Phillips has been assembling, who will work to make a public release in the months that follow, a process which Phillips says will likely take six months to a year.

"There is a prototype for kernel 2.2.13. I'm not going to release it -- I have my reasons for that, and the main reason is that the amount of cleanup to make it presentable to the public is roughly the same as the amount of work I have to do to bring it to [a newer kernel]. Probably if I'd done nothing else but worked on it for a couple of months, I'd be using it now, but I've done a few other things [in those months], like change from an industrial control systems job where they wanted me to do the next version of the control system in Windows NT to a nice linux job where I can hack the kernel."

Does this have anyone else itching for 2.5?

To all but a handful of a handful, a perfect filesystem would be boring: files would be where you expect them to be, corrupt files would never be an issue, and a power outage would result in nothing more than a few moments of darkness rather than minutes or hours of lost work. Even so -- perhaps a clue that the perfect example is far, far away -- the news on the Linux filesystem front is pretty exciting of late. In a low-key technical session Friday morning at ALS, kernel hacker Daniel Philips announced to the world the minor revolution he's planning -- which could end up replacing Linux's old standby ext2fs (and it's coming replacement ext3fs) with his Tux2 filesystem. Though Tux2 is an ext2 cousin in many ways, it carries at least one crucial improvement: according to Philips, you can literally pull the plug on a system running Tux2 and expect not to lose files or spend minutes watching fsck crawl by.

Phillips, an expatriate Canadian now employed by Berlin-based Innominate AG, claims 25 years of computer programming experience. He's had stints in everything from database design and game programming to embedded controller system development, and in a dual life which may sound familiar to many computer programmers, Phillips worked through music school by hacking Fortran code. With that background, perhaps it's unsurprising that just a few years after first encountering Linux, and a year from joining the ranks of the kernel hackers (there's a +5 informative thread in Zack Brown's excellent Kernel Traffic), he's come up with what could be a sea change in Linux filesystems.

A Filesystem You Can Live With And Pull The Plug On The central point of a journaling file system is that in exhange for a small hit in performance, file integrity is assured by an ingenious mechanism: rather than being written directly (and riskily), filesystem changes are instead first recorded sequentially in a running list -- the journal -- the contents of which are then acted upon in turn. If the system should crash for any reason while a change is not yet accomplished, the recovery time upon reboot is greatly abbreviated, as long as this "edit decision list" remains intact. Journaling file systems are on the way from multiple projects, and rather than being theoretical, wouldn't-it-be-nice daydreaming, at least one is availble right now: the ReiserFS developed by Hans Reiser is even an option at install on some recent Linux distributions.

Why another, then? Wrong question: Tux2 is not a journaling filesystem. Phillips says that Tux2 offers Linux users the chief advantage of a journaling filesystem (namely, keeping files safe in the event of a system crash) but without a journal, and does so more efficiently.

"The big deal is when you compare it to journaling, which is a popular solution, and you see that it's just plain writing less blocks. That's a big savings. It's also not constantly going back to wherever the journal is on the disk to write to the journal, so there's a lot less seeking involved. So those two things together means that it should significantly outperform journaling." Perhaps more importantly, Tux2 is not actually a wholly new filesystem per se; it shares so much in common with ext2 that it is built as a patch to ext2, with the filesystem converted at runtime. How does Tux2 get around keeping a journal to do the things that a journaling filesystem does? Atomic updates are the key. (See also: soft updates) Instead of a journal, Tux2 uses what Phillips terms a "Phase Tree algorithm."

"I originally called it Tree Phase," he says, "and then Alan Cox mentioned it on the Linux kernel list. He called it Phase Tree on the Linux kernel list, and I decided I liked that better." The Phase Tree algorithm is simple at heart, but takes a little while to grasp -- at least it did for me. Happily, Phillips has written a lucid tutorial on his own site. Probably the best explanation is the one found on Phillips' project site: the exerpts which I found most illuminating are these:

All accesses to filesystem data are performed by descending through a filesystem tree starting at its metaroot.

Normally, three filesystem trees exist simultaneously, each with its own metaroot. One is recorded on disk with a complete, consistent tree descending from it. A consistent second tree, the 'recording' tree, in the process of being recorded to disk, descends from a metaroot in memory, and some of its blocks are in dirty buffers. A third tree, the 'branching' tree is in the process of being accessed and updated by filesystem operations, also with its metaroot in memory. The branching tree and is not required to be internally consistent at all times. In particular, some blocks that are free in the branching tree may not be marked as free in its block allocation maps but held on a 'deferred free' ('defree') list instead.

At some point the recording tree will be fully recorded on disk and its metaroot can be written to disk so that it replaces the metaroot of the recorded tree. This causes the filesystem to move atomically between states, as desired. At this point, the recording tree becomes the recorded tree, and the branching trees metaroot is copied to become the new recording tree. This event is called a 'phase transistion' and the interval between two such events is called a 'phase'.

"The problem is, it's not nice to block filesystem transactions. If you're using a KDE desktop or similar, you find your desktop moving in a very jerky way while the blocks are getting written -- no good. That's why we make another tree by copying the metaroot -- that's how we always start, we never start one by going up the tree -- meanwhile this second tree is undisturbed by that and can be written to the disk in peace."

This additional copy allows the user to work without noticing a system slowdown, while the intermediate branch is copied. Thus, there are always three "trees," and in the event of a system crash, recreating the system's correct state is as easy as identifying the latest succesfully written tree. "Each new tree is always incremented higher, so this is easy," Phillips says.

"There are a couple of other places where [Phase Tree] is obviously better than journaling. For instance, removeable media -- your removeable media is usually slowest, and you don't put a journal on it, because if you did, it would be really, really slow. So you put the journal on your hard disk, and the data on the removeable media. As soon as you pull your removeable media out, you have instant corruption, because you've removed yourself from your backup. Phase Tree doesn't do that -- you can just pull out your removeable media and you have something current up to the last tenth of second, quarter of a second."

Sleepless nights and database integrity Phillips' work with Phase Trees began a decade ago, when he implemented a system with similar functionality for a specialized database called Nirvana which he had developed on his own. "I would have implemented this on a Unix filesystem at the time as well, except I didn't have one available."

Was there a Eureka moment in 1989? "Oh yeah. I dimly recall having a a week of sleepless nights, tossing and turning, trying to figure out if it was even possible to do something abot this, and eventually convinced myself that it was. And as I recall, it was quite tricky to get it to a hundred percent state, not 99.99. I could smell the idea in there, but I couldn't find it's actual realizaton for some time. After that, the generalization of its application to a general file system is pretty obvious."

Still, the idea stayed with him until he realized it would be an interesting way to improve the performance of Linux systems.

Like the puzzle with square pieces sliding around a single missing square, only scant disk resources are used to accomplish the extra data's movement because the information is moved incrementally -- in blocks rather than all at once. That means, says Phillips, that "It really adds very little [disk] overhead. Something on the order of 1 percent."

Additionally, it has one more feature which may appeal to the fsck-hater in you: "Really, it's nearly a defragmenter already," Phillips says. "It would be trivial to add that functionality."

The dual advantages of lower overhead and -- most importantly -- a close relation to the ext2 file system should make it an easier transition for most users. Tux2 is actually built as a patch to the ext2 filesystem; standard ext2 filesystems are converted to Tux2 at mount time. According to Phillips, that conversion takes on the order of a tenth of a second per gigabyte on a typical system.

Fly In The Ointment Though Phillips downplays their significance, patent difficulties may lie ahead for Tux2 as well. Network Appliance applied for a patent in the early 90s which covers similar ground -- a few years after Phillips had implemented it in his database.

"What really steams me in this is that their [patent] application came three years after my invention," says Phillips. "I hate to use the word infringe, because that makes me sound like the bad guy -- but it seems as though my [method] doesn't infringe beause it uses a different algorithm. In fact," he says, "I've got two things: I've got prior art, and I've got a better algorithm ... We can fence them in [legally], so their best strategy is to be nice, but they haven't figured that out yet."

"I don't want to suggest that NetApp got the idea from me -- I don't think they did, I think they developed it independently. The only little problem is the chronology of it. I concieved the whole thing, essentially everything that they've written in their patent, so I was kind of upset when I saw it. I would have gone on to do in on a Unix file system at the time, if I'd only had one available. We know it's stupid, but you see people patenting things all the time on the web -- just because it is a business idea that is now being done on the web." The approach that Phillips has to the dispute is to simply keep working. "I don't want it to become a distraction, I just keep doing what I'm doing."

Do penguins have calendars? Phillips says that Hans Reiser has approached him regarding integrating the file protection capabilities of Tux2 with the additional features of ReiserFS. "But it's pretty obvious where the priority has to be," he says, noting that ext2 is the default file system, and isn't going away any time soon. "Ext2 is what everyone has by default, and that's too big to ignore."

Does Phillips anticipate Tux2 becoming the default file system in Linux systems? "Well, who knows what's going to happen?" he laughs. "It could. But you can be sure of one thing, Tux2 will live a fairly long life as an independent patch that people apply, and I will be the 1st to apply it. But sure, of course I'd like that."

With a caution that fits someone whose last job was in embedded controls, Phillips warns against putting Tux2 in too soon: "It has to be proven, it has to be 100 percent. Because that's the whole point of this, is to 100 percent. So I think any bug which is not an ext2 bug already is just not acceptable."

And ultimately, like any other possible low-level change, "It's up to his high penguiness." Besides which, "it's quite clear what the next Linux filesystem standard is going to be. Well, it's my opinion that ext3 is going to be the most popular standard linux filesystem next year. And a couple years after that, well, I certainly will be using tux2 all the time, and we'll see where it goes."

The current status is heavy development: "I want to give it as a Christmas present to myself and start using it in my root system for my own development," says Phillips, "as soon as I port it to [the 2.4 kernel]." Soon after that, the code will be released to the developers on the Tux2 mailing list which Phillips has been assembling, who will work to make a public release in the months that follow, a process which Phillips says will likely take six months to a year.

"There is a prototype for kernel 2.2.13. I'm not going to release it -- I have my reasons for that, and the main reason is that the amount of cleanup to make it presentable to the public is roughly the same as the amount of work I have to do to bring it to [a newer kernel]. Probably if I'd done nothing else but worked on it for a couple of months, I'd be using it now, but I've done a few other things [in those months], like change from an industrial control systems job where they wanted me to do the next version of the control system in Windows NT to a nice linux job where I can hack the kernel."

Does this have anyone else itching for 2.5?

BorgiaPope writes: "Jill Tarter of the SETI Institute's Project Phoenix writes a sad, elegiac piece in Slate about the apparent final silence of Pioneer 10, launched in 1972 and now more than 7 billion miles from Earth. For the past five years, SETI scientists at the Arecibo Observatory in Puerto Rico have used the incredibly faint signals from Pioneer 10 to test the functionality of their noise filtering gear. Alas, Tarter reports that Pioneer 10 hasn't been heard from for several days now. The incredibly hardy, long-lived satellite, which long ago surpassed NASA's wildest expectations for its power supplies and other systems, may finally have drifted peacefully into eternal slumber . . . ." I think the Klingons got it.

OrenWolf writes: "Microsoft has threatened to sue the current developers of r/w NTFS support in the Linux Kernel. Details can be found in the current Kernel Traffic post." No, your honor, we aren't a monopoly.

BorgiaPope writes "WAIS creator and Alexa founder Brewster Kahle is interviewed by Feed. Kahle talks about the 30 terabytes of 'net content stored in Alexa's Linux servers, a data store he calls the 'largest library the world has ever known.' Some fascinating observations about how sites move in and out of the top traffic tier. He also claims that the top ten Web sites have the "greatest worldwide concentration of power since the Roman Empire.""

Dave Sifry writes: "I wanted to let you guys know that my response to Tim O'Reilly's recent column about Linuxcare. Things really aren't as bad as some in the press have made it appear. I feel it is important to get a dose of facts into the conversation -- now that we're out of our quiet period and we can talk about what's going on, and all of the neat stuff we've been working on." After all that's been said about LinuxCare, it's good to hear info straight from the top, and that Tim O'Reilly is an active listener. Just remember who's speaking.

Eupolis writes "Reuters reports that Linuxcare has withdrawn its IPO filings, and is now cutting staff to try to keep from running out of money. " As well as the report from Reuters, News.com has an analysis of the situation as well.

sohp writes: "Alan Cox's summary writeup of the things remaining before 2.4 highlights Kernel Traffic #64. It's quite a long list -- I'm not holding my breath for 2.4 " Kernel Traffic is a pretty cool overview thing if you can't handle the burden of actually subscribing to the list itself ...

We've heard rumors that Linuxcare's upcoming IPO has been shelved for now, and that Linuxcare's CEO and CIO have been removed. This comes right after the roadshow was delayed, which now appears to be cancelled. At this time, I haven't heard back from Linuxcare, so haven't heard their side of the story. However, several reliable sources have said that there've been "irregularities" in revenue recognition, something that the linked article also touches on.Update: 04/07 11:22 by H :I've talked with Art Tyde, Founder of LinuxCare. He confirms that the IPO has been delayed - but that they believe an IPO is in their future. He also did confirm that the CEO, Ferdinand Sarrat, has parted ways with Linuxcare, but that the CIO has remained onboard, and that the management team remains committed to the values of the community and what the company was founded on.

Everybody and Their Dog writes, "SGI announced the availability of XFS for linux 2.3.99pre2, via their CVS. Timely in light of the Journaling ReiserFS controversy, and ext3 delays. " A lot of people sent this in -- good to see SGI following through on their promise.

So I'm at Linux Expo Australia, enjoying the conference. You can hit the link below to read assorted random relevant (and irrelevant) notes from the show floor. No, I haven't seen a kangaroo.

After more than 24 hours of airport/airplane limbo and uncountable time-zone changes, we arrived in Sydney. After 20 hours of sleep, I was almost ready to face the world outside of my hotel room.

The Australian LinuxExpo is held in Darling Harbor in Sydney, which is apparently the epicenter of all Australian tourist activity. The exchange rate is nice tho, so if I wanted to to buy boomerangs or didgeridoos or something, it'd be easy. You can't walk 20 feet without stumbling into a tourist-trap store.

The show itself is fairly typical for a Linux conference. Of course, without a Slashdot booth to hide out in, I'm spending time at either the Debian Booth (hooray! Debian is international!) or at the LinuxCare booth: this sucker is two stories. The ground floor is a little stage where Tridge and others preach their word, and upstairs is a PC graveyard with boxes ranging from iBooks, G4s and Ultra 5s to various laptops, all plugged in to the Net. This is of course where I write this.

Many of the usual suspects are here: SGI, Red Hat, Corel, LinuxCare, Pick, Debian. Absent are VA and Andover (both are represented, although in much smaller numbers than other shows and without the overhead of a booth).

The advantage of not having a booth is that I'm not obligated to spend the whole show trapped in a 20x20 square answering FAQs all day ... instead I get to see talks. Raster did his usual show on Enlightenment, and Rusty did a great job on Netfilter (including revealing that I could use perl to write my own packet filters if I was either clinically insane, or just bored).

I also sat on a panel along with several other much more interesting folks (Tridge, Raster, Terpestra, and hosted by Chris DiBona) on 'Preserving the Linux Community.' Spent a lot of time discussing who is the Linux community (answers ranging from "Anyone who boots Linux" to "Anyone who cares about their operating system and runs Linux.") Eventually the discussion turned to the future of Linux, DeCSS, the MPAA, and what the new corporate influences can do to help. Not a bad panel, although not exactly the busiest of auditoriums.

Maddog gave a good speech last night on his predictions for Linux. Lots of good stuff there, too ... he had some interesting comments about Linux's growth and third world countries. People who are waiting for "Permission" that obviously will never arrive to extend Linux to do what they want it and need it to do. Not so sure about all his talk of "Linux Camps." The average hacker don't look so hot in Swim Trunks ;)

So with the conference portion of the show wrapping up, Raster, Hormes, Rusty and other Aussies are gonna take us to an "Authentic Australian Pub" this evening. I suspect that I won't have a solid memory of the evening: if there is one thing I know for sure it's that these Australians like their lager in great quantities ... I'm not sure if my liver can keep up.

I'll be back in the States next week, after what amounts to my first real vacation since starting Slashdot 2.5 years ago. See ya then.

Linuxcare has introduced version 1.2 of their business card-sized rescue disks, which now contain 140 MB of recovery tools, Debian install capabilities, the X Window System, PCMCIA support, and ssh. From the picture they look pretty cool, too. I remember seeing the business card CDs at a COMDEX a couple of years ago, but this is easily the best use I've seen for them, and is a needed improvement over the previous version.

Well we figured it was either Caldera or LinuxCare, but I've seen the Press Release: The latest Linux Company to file for an initial public offering is Caldera Systems. (btw they need to win an award for 'Linux company CEO with name most likely to be confused with an action hero' for Ransom Love) No word on when or shares or anything, but I'm sure all will trickle out in due time. The rumors are raging that Linuxcare is due to file their S1 before the end of the week.

Andrew G. Feinberg writes "Here is a article on the LinuxCare website. " This is a cool interview just because its not dealing with the usual GNU/Open Source/Free Software stuff, but more with code, coding, and lots of other stuff that frankly just isn't political. Enjoy it.

Andrew G. Feinberg writes "Here is a article on the LinuxCare website. " This is a cool interview just because its not dealing with the usual GNU/Open Source/Free Software stuff, but more with code, coding, and lots of other stuff that frankly just isn't political. Enjoy it.

Samba developer and LinuxCare employee, Andrew Tridgell, was interviewed on Australian ABC's influential 7:30 Report national current affairs program. "All we need to do is fire up a 98 box and do a domain log-on," Tridge said. Thanks to Paul "Rusty" Russell for finding this.

Norm writes "Linuxcare is running an informative article about project forking and how the GPL serves to prevent forking in many cases; a good expose on a timely subject, given recent fears about kernel forks, etc.. " Being at Comdex, I've heard a lot of people wondering about why Linux stays together, questions about what the GPL means and how it works. There seems to be a lot of confusion about what different distributions actually mean.

Norm writes "Linuxcare is running an informative article about project forking and how the GPL serves to prevent forking in many cases; a good expose on a timely subject, given recent fears about kernel forks, etc.. " Being at Comdex, I've heard a lot of people wondering about why Linux stays together, questions about what the GPL means and how it works. There seems to be a lot of confusion about what different distributions actually mean.

imagi asks: "I'm interested in finding out about the legal ramifications of publishing benchmarks on Microsoft Products. I've noticed that many contain a clause forbidding you from giving out performance information. Is this legal? As a consultant I like to advise customers on the best solution for their needs, however without being able to backup recommendation with independent benchmarks, I'm having to take Microsofts word for it. After having read some of their previous FUD (for example Linux Myths) their claims ring hollow. My question then: Is this legal to forbid benchmarking and what would happen if I were to set-up an independent website comparing MS SQL Server vs Oracle 8 on Linux?" There are lies, damned lies and, of course, benchmarks. I find it silly that Microsoft could take action against you if you've shared information that they published and released, but I could be wrong. Can anyone shed some light on this?

Norm J writes "Linuxcare is running an interview with Matt Welsh, author of Running Linux, 3rd edition- it's a great read. " Matt continues to display a staggering amount of good humour and intelligence.

Norm J writes "Linuxcare is running an interview with Matt Welsh, author of Running Linux, 3rd edition- it's a great read. " Matt continues to display a staggering amount of good humour and intelligence.

Anonymous Coward writes "August 31, 1999 -- Linuxcare announced today that it had reached an agreement with Sun Microsystems to provide a range of enterprise-class services for Sun's recently acquired StarOffice productivity suite on Linux. " Lotsa activity about StarOffice in the past few days-from being "community-licensed", to being developed into a Web application. Look for more news on this later on today.

levanti writes "Looks like Dell will outsource support of their Linux servers to LinuxCare. Article can be found here. "

LinuxCare is running a Cool Linux Story competition. This week's winner is about Herbert, the server no one knew was a server.

LinuxCare is running a Cool Linux Story competition. This week's winner is about Herbert, the server no one knew was a server.

David Sifry writes "LinuxCare is proud to announce a free bug tracking service for the open source developer community. We'll set up a jitterbug-based tracking system that will help keep track of your project. It's all web-based and easy to use. If you would like to take advantage of this, drop us a line at caretracker@linuxcare.com and we'll set it up for you! More details are on the LinuxCare website. " Is anyone doing something similiar with CVS yet?

David Sifry writes "LinuxCare is proud to announce a free bug tracking service for the open source developer community. We'll set up a jitterbug-based tracking system that will help keep track of your project. It's all web-based and easy to use. If you would like to take advantage of this, drop us a line at caretracker@linuxcare.com and we'll set it up for you! More details are on the LinuxCare website. " Is anyone doing something similiar with CVS yet?

David Sifry writes "LinuxCare is proud to announce a free bug tracking service for the open source developer community. We'll set up a jitterbug-based tracking system that will help keep track of your project. It's all web-based and easy to use. If you would like to take advantage of this, drop us a line at caretracker@linuxcare.com and we'll set it up for you! More details are on the LinuxCare website. " Is anyone doing something similiar with CVS yet?

David Sifry writes "LinuxCare is proud to announce a free bug tracking service for the open source developer community. We'll set up a jitterbug-based tracking system that will help keep track of your project. It's all web-based and easy to use. If you would like to take advantage of this, drop us a line at caretracker@linuxcare.com and we'll set it up for you! More details are on the LinuxCare website. " Is anyone doing something similiar with CVS yet?

David Sifry writes "According to PC World Online, it only takes a Palm Pilot, some software, and 10 seconds to copy certain types of car keys for infrared door locks. Those pilots sure have lots of uses! The full story is here. " I dig the software that lets you use your Palm IIIs irda port as a universal remote. 4" range sucks, but still. I wonder if I could get my laptop to do that.

David Sifry writes "SuSE and LinuxCare just announced a new Corporate Support and Training Program for Linux. The press release can be found here. SuSE also announced a partnership with Muster Learning Architects to provide corporate training programs. "

David Sifry writes "SuSE and LinuxCare just announced a new Corporate Support and Training Program for Linux. The press release can be found here. SuSE also announced a partnership with Muster Learning Architects to provide corporate training programs. "

Chris DiBona writes "LinuxCare is building a very cool company providing Linux support for large enterprise companies. Dave Sifry (who did the open source vpn) and Art Tyde (pres of BALUG) are the ones who are doing it. They're looking for resumes and asked me to spread it around (resumes@linuxcare.com)"