What Does The Future Hold For Linux?

Nailer asks: "With kernel 2.4 in the final stages of bug hunting, and on track for a December release, I thought it might be pertinent to discuss the future of Linux. What now? ReiserFS will apparently be in 2.4.1, but there's very little information about the mid to long term available. Where do you think Linux [the OS, as well as the kernel] will head in the future? Personally, I'd really like to see POSIX ACLs as the default permission system [allowing the fine grained access control that many apps try and implement themselves]. What do you think?"

Comments

A move to XML would be meaningless...

[slothbait]

without a corresponding "standard" to describe the structure the file should have. If you want regularity, we need a well-accepted consensus on the different styles of options that can be used, and syntax for representing them.

If you had standard meanings for different kinds of config options and a standard format, you could get your wish of a single tool (be it command-line, slang, or X) being used to parse and modify all compliant config files. I've considered implementing something like this, myself. The problem, of course, is less in implementing it than in getting all of the disparate projects to accept it.

The only groups that might have the power to start this change are the distros. Red Hat seems reluctant to define standards, as they don't want to be seen as "pushy". Mandrake doesn't have the clout. To me, that leaves Debian and possibly Suse. If Debian came out with a standard, started using it for their tools, and developed a nice, simple interface, I think they could start persuading others to follow suit. It's a good idea, but it will take a great deal of persuasion.

But back to my earlier point point: simply moving to XML buys you nothing but complexity. What you really want is a common ordering and syntax to the different files such that they can be edited by a common tool. That's all well and good, but that can be achieved *without* XML just as easily as it can with. I see no inherit advantage in using XML, except the gain in "buzzword compliance".

--Lenny

Re:A move to XML would be meaningless...

[__donald_ball__]

But back to my earlier point point: simply moving to XML buys you nothing but complexity. What you really want is a common ordering and syntax to the different files such that they can be edited by a common tool. That's all well and good, but that can be achieved *without* XML just as easily as it can with. I see no inherit advantage in using XML, except the gain in "buzzword compliance".

You've evidently never read the XML specification, or tried to use XML anywhere. Here's the deal. XML takes ASCII to the next level. ASCII lets you encode strings (in latin characters) in files in standard fashion. XML lets you encode trees (in arbitrary character sets) in files in a standard fashion. Trees are the natural data structure for most configuration files.

Even more useful stuff - you get validation with DTDs and Schemas. That means each program doesn't necessarily have to check its own configuration files for sanity, it can rely on the parser to catch the syntactical and much of the grammatical errors. Hell, each program doesn't have to write its own parser any more, making it simpler and reducing the possible number of bugs in the system (if all daemons shared a common configuration file parser).

Finally, by using XSLT stylesheets, it's very easy to transform XML files between different formats - giving you a relatively simple upgrade procedure when a daemon changes its configuration file format, and giving you a relatively simple way to convert configuration files between daemons - from postfix to sendmail, say. Think I'm blowing smoke? I already have all of my system data centralized in one XML file from which I generate the various daemon's configuration files. I think this is probably going to be the way this unfolds - userland tools will arise that use XML files to generate the collection of oddities that is the /etc filesystem.

I really don't understand the Linux community's perceived resistance to XML. I think it's because Microsoft was an active participant in the development process, and because many of the early implementations of the XML tools were written in Java. I really wish that attitude would change, because XML is an important standard for taking UNIX to the next level. One of the things that made UNIX great was the proliferation of small command line tools that cooperated by passing ASCII streams around. Imagine how much more powerful that paradigm could be if you they passed trees around!

Major things

[JohnZed]

My ultra-wishlist would include:

• Performance monitoring! This has already been mentioned in this discussion, but it's such a great feature that I had to mention it. Without patching the kernel, it's almost impossible to create a sysadmin tool like PerfMon or a developer's tool like SpeedShop or VTune. And SGI has suggested that they'd be willing to port SpeedShop if the infrastructure were in place. There are a lot of patches floating around that give the basic capabilities (essentially, recording and configuring hardware performance counters), but most are lacking overflow monitoring, which is huge.
• Modern high-performance I/O: On kernel-traffic you can see a recent discussion about select and poll on Linux. They just don't scale well AT ALL. Similarly, zero-copy TCP is very well established as a massive performance gain for network servers, and there's no excuse to fail to use it for calls like sendfile() which could so clearly benefit.
  
• New scheduler: The CITY-Umich scheduler project is pretty interesting, as was the work done by IBM's Java group. Regardless of what solution is used, though, we have to stop making excuses and admit that the current scheduler is highly flawed for systems running hundreds or thousands of threads.
  --JRZ

Re:The Problem With ACL

[Nailer]

The simple but very UNIX user/group/others scheme in contrast promotes a little thinking and planning in advance, and as a result you can do what you want by putting people in different groups and use those.

I don't think rwx permissions are very Unixy at all - in that they're unfortunately popular on Unix, but don't fit in with Unix philosophy. Unix philosophy gives users only the permission they need to do their work on the system. Unix-style systems are by default locked down much more fully than other OSes for regular users.

For root, however, its a different story. Why is is someone logging in as the system account with permission to perform any action on the system for the purposes of adding users?

Run top [or of your a GUI person, KDE System Guard or a similar utility]. Check whose running all those daemons. The answer is root. Why [especially if rwxs is enough]? SSH and CUPS and automount don't need permissions to create new entries in /dev, or to modify /etc/passwd. But if you can exploit them, you have permission to do just that. Crackers heaven.

If rwxs is enough, why do nasty tools like sudo exist, and why do so many apps require setuid root permission? rwxs is not UNix like. Its a kludge that's popular on Unix systems. And it definitely not Unix philosophy.

As long as there is Unix, there will be Linux and

[doublem]

As long as there is Unix, there will be Linux and BSD.

Why? Simple. Sun, IBM and HP all see Linux as a way to gain mind share. Let's look at two hypothetical:

A business builds a small web server and database server using NT or 2000. Their business grows and soon they realize they need more power. Microsoft promises to scale infinitely. A salesman from Sun shows up and offers a Solaris solution. Better stability, scales better, and Oracle is kicking MS-SQL's but in exactly the kind of business they're running. But running Solaris would require retraining of staff, replacing all their software and migrating their data to a new system. Sun looses the sale.

A business builds a small web server and database server using Linux. The business outgrows the memory model of Linux because the same memory optimizations that allow you to have a 4,000 hit per day web server running on a 386 cause some problems when you reach eight processors and 24 gigs of RAM. Solaris walks in and offers a solution that scales better, runs ports of all the software they're using and because they know how to run Linux, Solaris is a small skip and a jump away in IT skills.

The more people who run Linux or BSD on their home systems, the more sales the big Unix vendors will be able to make, because there will be more people who know how to operate the systems. This is exactly why NT has gained so much ground. People know how to run Windows, so they figure a web server running NT can't be that big a leap.

Oddly enough, optimizing the Kernel for massive systems with a plethora of processors and RAM could hurt Linux if the big Unix companies see it as a threat.

www.matthewmiller.net

Re:ACL introduction?

[SuiteSisterMary]

In UNIX, there are three entities that can be given rights to an object; the owner, a group and everybody. Thusly, no more than one group can have control over a an object. In other words, Finance and Marketing cannot be both given write access to a directory; you'd need a third group which had, as members, the first two groups. Access Control Lists start out by stating the default permissions (usually 'nobody can do anything') and then applying exceptions (Bill can read, Lucy can read/write, Doug can read/write/execute.) ACLs also allow for more options; with UNIX, you're stuck with read/write/execute. With an ACL, you can define almost any operation and allow/deny it. Read, write, execute, list contents, create subdirectory, grant rights to others, revoke rights from others, copy files in, copy files out, etc etc. ACLs are a requirement of any level of Trusted Computing above c2, I think. They're very powerful, but also much more difficult to maintain than UNIX style permissions.

The kernel...

[Cmdr. Marille]

Well I guess I would be easier to just talk about Linux, the kernel.
As already mentioned integrating some kind of Journalling Filesystem will maybe the most important task.
May it be XFS, JFS, ext3, ReiserFS, or Tux2, i think this will and must be one of the next addition to the kernel tree(probbably ReiserFS in 2.4.1). There is no big and bright future for Linux if we don't get to a filesystem with better data integrity. I think that in fact the diversity of Jfs's is a great thing because:

It will create competition

There will be FS's optimized for certain tasks(something that is already happening now, look at ReiserFS SQUID Optimization)
The existence of XFS and JFS for Linux already shows that both IBM and SGI are really willing to put their Unix experience into the future of Linux (At least that's what I hope)
I think the involvment of companies like IBM and SGI will maybe the biggest chance for Linux over the next years
There is the Unix experience of two of the biggest players on the market going into the most active Developer community in the world. We can really hope for a bright future

What's next

[clinko]

Dun Dun DUN!!!!

2.4.1
then!
2.4.2 maybe 2.4.3!

It's a joke, don't kill me :)

Re:Incorporate kqueue/kevent from FreeBSD

[edhall]

Linus' comment on this:

I've actually read the BSD kevent stuff, and I think it's classic over-design. It's not easy to see what it's all about, and the whole <kq, ident, filter> tuple crap is just silly. Looks much too complicated.

All due respect to Mr Torvalds, but I'm not sure he understands the intent behind the kqueue interface. It's not just intended to be a poll() on steriods, though it serves that function. A kevent does not need to be related to a file descriptor--for example, signals and the status of other processes can also be monitored with the appropriate filter. Other filters will be added as time goes on. The tuples Linus objects to are actually the simplest way to provide this level of generality through a single interface.

-Ed

My wish list

[Salamander]

Here are some of the ways I'd like to see Linux evolve in the near future:

• Better modularity. There's still way too much interdependency between what should be separate subsystems. Sometimes this is in the form of explicit references to each others' data or functions, but just as often it's a more subtle but still undocumented reliance on "X never does this so I won't even check for it, and God help the guy who comes after me if X ever changes".
• A better VFS layer. Just having one isn't enough because, to be blunt, some folks did a pretty piss-poor job of specifying, implementing, or testing it. This fact is the primary reason 2.4 isn't out yet.
• A mature persistent-module-data interface. We don't need anything as overburdened as the Windows registry or AIX ODM, but we do need something and recent steps in this direction are a very good sign.
• Journaled, soft-update, and atomic-update filesystems. Not one, but several. Let them compete. This is an area where Linux can be the foundation for significant improvements in the state of the art.
• Better testing. We need a serious test suite for each of the major kernel components, and one for the whole system as well. We should be beyond the point where the current near-total lack of unit or regression tests is acceptable.
• Better performance management and monitoring tools. How many of you have used PerfMon on NT? The way it's implemented is kind of crufty, but the flexibility and functionality it provides makes Linux look really bad by comparison. It should be a standard part of Linux kernel or major-subsystem (e.g. database, webserver) development to define and export counters for a general tool like PerfMon to use. A lot of the bickering on linux-kernel about where the bottlenecks are would be neatly settled by a five-minute session with such a tool.
• ACLs? I'm still not 100% convinced we need them, but they are more powerful than the current system and there seems to be a demand. In any case they're likely to become a checklist item for a lot of folks soon.

Lastly, there's one more thing I think Linux needs, but explaining why takes more than should go into a single list item. Linux needs a good forking. Seriously. Competition is good. The cabal - yes, there is one in effect, even if its exact membership is debatable - generally has good ideas and provides incredible value in bringing order to chaos. On the other hand, the Powers That Be sometimes suffer from severe Not Invented Here syndrome, and sometimes they use their bully pulpit to shout down perfectly good ideas that conflict with their own biases (or even projects that would compete with what they want to work on). Several have recently seemed to start believing that they're omniscient, as though merely being a genius wasn't good enough. Linus and the others deserve our gratitude, and our respect, but not worship or unquestioning obedience. They need a wakeup call, in the form of someone defying their wishes and achieving superior results in the process.

This will happen, sooner or later, one way or another. We have two choices:

• Embrace the possibility as a generator of innovation and healthy competition, and as a way to keep everyone honest and humble.
• Let it become a source of chaos and dissension, until someone else eats our lunch for us.

That's it. There are no other choices. I know this will probably not "resonate" with the younger members of the audience, but I would compare the situation to a divorce. The most amicable divorces, where people still remain friends, occur when the people accept the reality and work together on making necessary change go smoothly. The messiest, most destructive divorces happen when people have stayed together long after their interests diverged and they've spent years learning how to hate each other. I don't want Linux to turn into War of the Roses.

Why Not Just Read Kernel Traffic?

[Carnage4Life]

Nailer asks:
With kernel 2.4 in the final stages of bug hunting, and on track for a December release, I thought it might be pertinent to discuss the future of Linux. What now?

If you are truly interested in Linux kernel development and the future of the OS, why not just just subscribe to the Linux Kernel mailing list, browse the archive or read the digests on Kernel traffic?

Slashdot is comprised primarily of Linux users not Linux developers. Questions like this are better sent to mailing lists frequented by the people who make these decisions than to a bunch of armchair critics. This article is similar to asking a bunch of random Windows users where Windows&#153 development should go and expecting a coherrent answer.

Second Law of Blissful Ignorance

Re:foolishness...

[Alomex]

I'm only going to say this once - XML is *a* solution to the trivial problem of syntax;

Syntax is not a trivial problem between advanced applications. Designing a protocol that is expandable, be it for config files or interprocess communication is a pain in the neck. We started using SGML for config files and interprocess communication in 1995. You have no idea how much work we saved by using standard parsers and a protocol that wouldn't break if we added a column to the message...

it does not, however, assault the intractable problems of semantics.

True enough. XML only gets you half-way there. Isn't that still better than nothing? Moreover, since the semantics is intractable, as you well state, the solution is to manually provide syntactic markings through, you guessed it... XML tagging.

After 6 years of adminning systems, I feel more secure with each daemon checking their config files than passing it onto an independent parser daemon.

Reality check. What is likelier to be buggy: a one-off parsing routine or a well established and universally tested parser such as SAX?

We know the answer to that one since the whole open source movement is predicated on it. The publicly available routine will be less buggy.

The world is moving en-masse to XML. Yes, it is overhyped (just as high-level languages, structured programming, OOP and Java were in their time). Yet all of those were clear steps forward in computing.

Same goes with XML. Linux can be either ahead of the curve, or behind it, always destined to be a late copy of a thirty-year old operating system.

Now that Linux is stable and of amazing quality it is time to start looking towards the future and make sure a good operating system becomes hands down the best.

foolishness...

[denshi]

You've evidently never read the XML specification, or tried to use XML anywhere.

So you wade right into it with an ad hominum attack - how nice.

I'm only going to say this once - XML is *a* solution to the trivial problem of syntax; it does not, however, assault the intractable problems of semantics. Your fantasy of trees magically passed between programs with no knowledge of each other falls down when you realize that each program assigns different meanings to each file/tree/whatever.

Other points:
Programs don't have to check their own config files: After 6 years of adminning systems, I feel more secure with each daemon checking their config files than passing it onto an independent parser daemon. Each program gets to test once with the parser code inside the program; if you move parsing out then you move that work onto the sysadmin (or home user) - the world is filled with minor API changes that screw you over. Feel like running regression tests on every piece of software you install?
Programs avoid bugs in their parsers: Parsing text is a solved problem. When was the last bug where Apache couldn't parse its own files? Compare that to XML, which is an evolving standard - even if you never have a bug in the XML parsers, who's to say that total backwards compatibility will be maintained?
When the daemon changes its config file format: Riiiggghht.. I've seen this happen only a handful of times over the hundreds of software packages I've fucked with. Copying config between applications? Best idea in your post, but most of us will still handcheck it, because of Semantic Complexity. Does postfix's 'address' field mean IPv4, and sendmail's 'address' field allow IP or DNS hostname? When you move to IPv6, does postfix break silently? These are issues that sysadmins everywhere deal with on a daily basis - none of it will go away with a common syntax.

I really don't understand the Linux community's perceived resistance to XML. I think it's because Microsoft was an active participant in the development process, and because many of the early implementations of the XML tools were written in Java.

I think it's because XML is not relevant for most of our problems. I use XML for foreign database data exchange. (I also use Java, so I don't understand your reasoning.) Thus far, I don't have any better uses for it, and I'm not hurrying to find any.

I do like your idea about changing streams to structured streams, though. Most apps these days are moving to shared memory in the absence of any such solution.

Standards...

[jackb_guppy]

Stop using flat files for .conf, Use XML as standard configuration format. This includes the make tools, lilo, and the rc.d as well.

With this grand unification:

1) XML parser API could be added so all modules will not have to create thier own.

2) A single config tool could be written so that the options and help information can be entered and checked. Lowering the entry skill level for general users.

Anothe wish is to get the /etc back. Move the conf data for a /.conf directory or some thing.