IBM's JFS & PTh-NG Reaches 1.0

jd writes "IBM's Journaled Filing System becomes the second commercial filing system for Linux to reach the exalted 1.0 status! It also follows close on the heels of another of IBM offering, the PThreads Next Generation project, which also hit 1.0 today." Check out this LWN story on it as well. It's worth noting that this is a free as in open source version - GPLed. There will be another commericial version as well.

Comments

Re:Questions

[Anonymous Coward]

Yes, it's possible ... you can check out Wingate for one piece of software that allows inet sharing. There are probably others as well.

It's a bit slow and expensive, but that's the usual way if you're going to use windows.

Re:beyond 5

[Anonymous Coward]

The Score:6 was not due to a race condition. It was due to someone being moderated to 5, then being moderated down (to 4), then moderated back up to 5. Then, the person who moderated it down posted, cancelling his negative moderation, which brought the score up to 6.

It has since been fixed.

Re:How do these differ from Linux Threads?

[Anonymous Coward]

AFAIK the IBM "Next Generation POSIX Threads" (NGPT) is mainly a more conforming implementation of POSIX Threads than the older Linux Threads.. It's still based on "kernel threads" (i.e. clone()'d processes), but I assume the reference to M:N threading means that it can map a number of user level threads onto a different number of kernel threads in the same way Solaris maps user threads onto OS Light Weight Processes (LWPs).

There's an interesting update to the clone() man page that indicates some of the recent clone() enhancements that were added to allow full POSIX thread implementation:

http://lwn.net/2001/0628/a/man-clone.php3

Re:How do these differ from Linux Threads?

[AndyS]

Ahhh, I've noticed this

But, it's not as bad as you think

I use 2.4.5-acsomethingorother

It has threaded coredumps - which is nice, until of course you get one (well "one" - it dumps every single core seperately for each cloned process)

So you have to manually go through them and bt them - but it does work, it's simply fiddly, and you can't easily check other threads without seperately reading in the cores.

In terms of threaded processes however, gdb is a lot better - indeed, the worst thing we've been bitten by is some of the bugs that it has with C++ and static members in superclasses.

gdb CVS seems competant with threads - I've had it weird out on me occasionally, but generally it's capable of doing the job.

gdb 5.0 isn't capable (at least, not the one debian ship), but the CVS version is a damn site better.

HTH

Re:Comments about Pth compliance, M:N threading.

[Kaz Kylheku]

When they talk about performance, they might, e.g., have in mind a $100,000 multi-proc web application server running hundreds or even thousands of threads in parallel, with several thousand more requests in queue.

Again, using thousands of threads is not going to get the best performance out of it, if it does not actually have thousands of CPU's or network cards or disks. A reasonably small pool of threads is all it takes. You need a thread for each peripheral you ever need to wait for, and each processor you need to keep busy, and then some.

There is no special set of processing principles for a $100,000 server which say that you can stuff in any number of threads. Of course, with a larger address space, memory and greater memory bandwidth, you can be more wasteful and notice it less.

Hahaha!

[Kaz Kylheku]

The threading library LinuxThreads has to play the violin while standing on its head to make threads look POSIX-like. The kernel people look upon POSIX threads as braindamaged, and they are largely right.

In my view, most of the noncompliances that exist actually make LinuxThreads a better library. For example fcntl() locks are owned by threads rather than the ``process''. This is more natural to program with; you don't want a locking request by one thread to just proceed even though it overlaps with an existing lock held by another thread in the same process! Think of the race condition bugs that can cause. Yet that's what POSIX requires. It's evident that POSIX is largely driven by vendors who have operating systems on which it's easier to hack threads this way, as sort of dadoes that are engraved into processes.

Another braindamaged area of POSIX is that all threads share the same current working directory. This is upheld by LinuxThreads using CLONE_FS,
but in principle it doesn't have to be. Again, think of the bugs this can cause! One thread does a chdir(), and the file accesses done by another thread go beserk into another directory.

Then there is the whole problem of security contexts. In POSIX, your effective and real user and group ID's are process-wide; if a thread changes user ID, it changes it for all threads. Like chdir, changes in user ID *need* to be done in a procedural discipline, regardless of what you may think of the idea of having multiple security contexts in one address space. Think you can fork() around the user ID problem? If you fork() in a POSIX threaded process, the child can use only async-safe functions, or the behavior is undefined. If you need to do more things, you must exec() a new image.

On the other hand, there are some reasonably nice behaviors in POSIX that don't work on Linux, like doing a fork() in one thread, but doing the wait() in another. (Workaround: make a fork server thread which handles fork and wait requests on behalf of others).

Comments about Pth compliance, M:N threading.

[Kaz Kylheku]

The way I see it, this IBM project is just reinventing LinuxThreads with a few differences, the biggest being M:N threading. If you look at their list of known issues listed in the release notes, it's about the same as for LinuxThreads: no process shared mutexes, no process identity for threads (except when only one underlying system task is used). These two are the biggest sources of complaints from some LinuxThreads users.

The rest of my comments have to do with M:N, specifically the false claim that M:N provides a performance enhancement for all multithreaded applications.

I don't believe that M:N threading is a good idea It creates issues and complications in the library implementation. The responsibility of scheduling and dispatching is divided between user space and the kernel instead of being done in one place. What M:N threading does is speed up voluntary context switches---context switches that take place within a threading function that is directly or indirectly invoked by the application, such as a synchronization function (pthread_cond_wait, pthread_mutex_lock, etc). Such a function can just call the user space scheduler, which can dispatch a thread without cooperation from the kernel. This is how M:N reduces overhead.

M:N does nothing for involuntary context switches, which have to somehow go through the kernel (for example, a signal is delivered to the process, which swaps context so that returning from the handler will cause a new thread to run). Actually, this kind of context switch can be worse than the ordinary Linux kernel context switch, depending on how it is done. The current task is interrupted to run kernel code (transition 1). Then a handler in user space is dispatched (transition 2). Then the handler returns to the kernel (transition 3) then the kernel passes control back to user space with a new context (transition 4). On the other hand, a native context switch is just two transitions: interrupt the current task, and dispatch the new one. In any case, the kernel is involved in the involuntary context switches of the so-called ``user space'' scheduler, which puts their expense in about the same ballpark as a kernel task switch (within the same address space).

So what about the faster voluntary context switches that M:N provides? Unfortunately, most of the *useful* voluntary context switch points are in the kernel: such as blocking calls that wait for I/O or real-time events. So M:N does nothing for I/O or response to real-time inputs. Dispatching a response to the completion of I/O or a real time input always requires a switch from the kernel to the user process.

M:N also does nothing for compute-intensive multithreading that is done *sanely*. Sure, M:N may speed up a program that performs, say, some operation on a large matrix using 50 threads on two processors, because when these threads synchronize, it can be done using those fast voluntary context switches. But M:N will do nothing for a program that uses two threads over two processors to do the same thing, which is the more sane design.

As a rule, the number of threads in an application should be not much more than the minimum that is required to utilize the various functional units of the hardware (processors and peripherals). Using too many threads just causes wasteful context switches that accomplish nothing, increases the memory access footprint of the application (since each thread has its own private data areas, such as the stack), and causes the scheduler to have to choose from among more threads.

It's not worth trying to speed up brain-damaged applications that make poor use of threads, yet this is exactly what M:N is for.

Re:Comments about Pth compliance, M:N threading.

[ink]

I couldn't disagree more.

M:N does nothing for involuntary context switches, which have to somehow go through the kernel (for example, a signal is delivered to the process, which swaps context so that returning from the handler will cause a new thread to run).

That's the whole point. Threads that do not need kernel scheduling don't have to bother the kernel scheduler. The process is still scheduled by the kernel and all niceties are followed. Take the GUI application with 10 threads that are handling event-driven input. None of them will block. If they all must be scheduled by the kernel, then you have 10 expensive context switches to do relativly trivial things. If they are all scheduled as user threads, then one quanta can take care of all of them in a single switch.

So M:N does nothing for I/O or response to real-time inputs.

Real-time inputs? Yes, user threads do a tremendous amount of good as long as there are no blocking threads involved (which should be scheduled as kernel threads). I/O? You are correct; you shouldn't schedule I/O as userland threads.

M:N also does nothing for compute-intensive multithreading that is done *sanely*. Sure, M:N may speed up a program that performs, say, some operation on a large matrix using 50 threads on two processors, because when these threads synchronize, it can be done using those fast voluntary context switches. But M:N will do nothing for a program that uses two threads over two processors to do the same thing, which is the more sane design.

And yet there are numerous counter-examples that can be thrown. If the programmer doesn't understand what's going on, then all sorts of things could cause performance problems (using large big-oh algorithms being the most obvious). A bad programmer can't be saved anyway -- if a low-level work unit can be split up without blocking, why not schedule it in userland? If (as above) you have 2 threads doing a *lot* of independent work, then it makes sense to schedule them as kernel threads because the time to process masks the context switch. The whole point is that the programmer, not the OS, is in control here.

As a rule, the number of threads in an application should be not much more than the minimum that is required to utilize the various functional units of the hardware (processors and peripherals). Using too many threads just causes wasteful context switches that accomplish nothing, increases the memory access footprint of the application (since each thread has its own private data areas, such as the stack), and causes the scheduler to have to choose from among more threads.

It's not worth trying to speed up brain-damaged applications that make poor use of threads, yet this is exactly what M:N is for.

The problem is, you never know the architecture (CPUs) you're going to end up running on someday. Using 10 GUI userland event threads along with 10 blocking kernel I/O threads allows your application to scale from 1 to many processors; if you use 20 kernel scheduled threads, then it might not perform as well on a single-CPU system as it otherwise would. Which one of the above applications is the real brain-dead one (please stand up).

The wheel is turning but the hamster is dead.

Re:Comments about Pth compliance, M:N threading.

[Phouk]

Hm, careful, after all, they are Big Blue... ;-)

When they talk about performance, they might, e.g., have in mind a $100,000 multi-proc web application server running hundreds or even thousands of threads in parallel, with several thousand more requests in queue.

For such a situation, for example, your arguments don't seem to apply, as far as I understand.

Re:Distros?

[jd]

Hard to say. Red Hat like to use all sorts of experimental stuff, so they're one possibility. On the other hand, I'm going to make a wild guess and say Debian'll be the first distro to provide the packages as an option.

Re:Filing system?

[jd]

Shhhhh! You're giving TiVO ideas!

Re:Isn't it third?

[jd]

Well..... if you want to stretch a point, it's actually at 1.0.1pre.

Re:How do these differ from Linux Threads?

[Lally Singh]

you've got replies telling you that the difference is that userspace threads are mapped into linux threads... The real cost of linux threads is that they're almost real processes, with almost the same costs involved. In reality, all you need is a register context and a stack... Thread creation costs are much higher than absolutely necessary.

--

standards compliancy in Pthread implementation

[VAXGeek]

A good factor in Linux acceptance has been Linus' conformance to the POSIX spec regarding Pthreads. It would be very beneficial if IBM were to stick to the spec exactly, thus making it easy to port large systems from AIX and Solaris, thereby increasing Linux marketshare. It is good to see a large corporation like IBM actually helping in Open Source development and actively contributing time and money to the Linux kernel development efforts.
------------
a funny comment: 1 karma
an insightful comment: 1 karma
a good old-fashioned flame: priceless

Re:gdb and core dumps

[spitzak]

I actually have not much idea what is going on, and am just trying to debug my program without learning details about how gdb and threads are implements. But so far (on a 4.2 machine) I have had good luck debugging mutliple threads with gdb, if I can get the thread to crash while running under gdb. It then seems to produce a stack dump and other information that is correct for the thread

I have had no success with setting break points and getting them to trigger. And I have absolutely no luck using core dumps generated when the program was not run under gdb. This is made more annoying because gdb changes something so that the code that I can crash easily when run directly does not crash when run under gdb. Pretty annoying!

Anybody have an explanation of what is going on?

PS: also don't use "-march=i686" in a multithreaded program. Is this a gcc bug?

Re: I agree 100%

[spitzak]

It is pretty obvious that the "posix standard" was forced through to conform to quick implementations of light-weight threads on existing single-threaded Unix systems.

Linux should ignore stupid standards. All information that is stored in the os should be seperate copies for each thread. There is no reason to share even file descriptors (they should be duped during clone and each process can seek() individually) The only things threads should share is their virtual memory. Light-weight threads can still be implemented by having calls like setuid() force the thread to become a heavy thread.

I would also like to see them get rid of the locks they put around virtually every system call (such as getc()!!!). The overhead of these is insane, and I really really can manage to do this on my own if I really have multiple threads reading from the same file.

Re:How do these differ from Linux Threads?

[ink]

What's wrong with Linux threads? I've been using them for a while. They seem to work for me.Why are the IBM ones so much better?

Linux threads are 1:1 mapped with kernel threads. This is great for I/O intensive operations because they can be independently scheduled by the kernel. If one task is bloking on some thing or other in the kernel, the others can be scheduled just fine. The problem comes with scalability; scheduling a kernel thread is just as expensive as scheduling a process, so if you have many many threads and relatively few processors things can get slow because the task switching/creation overtakes the performance.

The solution to this is to have both kernel threads and userland threads. Things that don't block and/or require other things that block the kernel should be spawned as userland threads (they are about an order of magnitude faster for both scheduling and creating), and things that will block the kernel (network I/O, file I/O, etc.) should be setup as kernel threads. This makes life easier on the scheduler and also makes GUI applications "feel" faster than they currently do. Solaris has been doing this for quite some time already.

The wheel is turning but the hamster is dead.

cheap Linux process creation

[MenTaLguY]

Process creation under Linux (>= 2) is cheaper than on many other systems, however. It's (usually) still a net win.

OTOH, for _kernel level_ threads, you need more than just a register context and stack -- the scheduler ends up needing a fair amount of bookkeeping data anyway.

Linus just decided that there's not much point in making threads a special case ("thread-like" behavior depends on which of several things are shared, specified by flags).

Things get a lot simpler that way, and is probably one reason why thread and process creation are so cheap on Linux.

Even the recently added CLONE_THREAD stuff is a pretty generalist approach -- it just controls whether or not threads (including those considered "processes") share a "thread group" (the id reported by getppid(2) in 2.4).

Re:cheap Linux process creation

[sailesh]

Unfortunately, in a Java server side environment when you have about 300 threads with the same priority of the same process, this policy doesn't work too well. The rub lies in the fact that the scheduler treats essentially treats each of the 300 threads as though they are 300 processes, and applies the algorithm to select the next thread to execute to every process in the list. There was a result shown by some guys from IBM that a rude hack like letting the scheduler to always choose a thread at random ends up performing much better for the server-side Java benchmarks. Of course, this is only a rude hack for pedagogical purposes.

Re:Pansy-ass lightweights...

[khuber]

Does Natalie Portman use JFS or ReiserFS?

-Kevin

Re:FS conversion tools please...

[Psiren]

Solaris has had a similar capability since 7. You just stick the logging option in your vfstab and you're off. Its not vxFS, but its quick and easy, and a lot more robust than no logging at all. It will be nice when Linux users can do the same. Playing with a new fs is all very well if you have the free space and time to implement it. Sticking one word in a config file is a lot easier though ;)

Did SGI GPL their XFS 1.0 announcement?

[Booker]

[punch /tmp]$ diff -u xfs_announce jfs_announce
--- xfs_announce Tue May 01 08:19:51 2001
+++ jfs_announce Thu Jun 28 14:30:02 2001
@@ -1,10 +1,12 @@
-SGI is pleased to announce the 1.0 release of XFS, high-performance
-journaled file system for Linux.
+IBM is pleased to announce the v 1.0.0 release of the open source
+Journaled File System (JFS), a high-performance, and scalable file
+system for Linux.

-http://oss.sgi.com/projects/xfs/
+http://oss.software.ibm.com/jfs

-XFS, widely recognized as the industry-leading high-performance
-filesystem, provides rapid recovery from system crashes and the
-ability to support extremely large disk farms. ... It is a
-mature technology that has been proven on thousands of IRIX
-systems as the default filesystem for all SGI customers.
+JFS is widely recognized as an industry-leading high-performance
+file system, providing rapid recovery from a system power outage or crash and the
+ability to support extremely large disk configurations. The
+open source JFS is based on proven journaled file system technology
+that is available in a variety of operating systems such as AIX and
+OS/2.

;-)

Re:Don't you mean?

[artdodge]

Actually, no; Daniel Frye announced the two releases yesterday at Usenix, and he made a point of talking about "Linux" and "Open Source", and not "GNU/Linux" and "Free Software".

For better or for worse - I also watched the NetBSD people just about go ballistic when he said Linux has been ported to every platform ("that matters" was probably implied).

Re:Questions

[kels]

We've got AT&T @home, which requires special windows only software to run,

Not true, AT&T @Home comes with special Windows software, but the network connection is accessible from any OS, just needs a DHCP client. I set it up on a Debian Sparc box and never used the Windows CD...

Re:Isn't it third?

[jefft]

XFS and JFS are both sold as part of a commercial (as opposed to "open" or "free") operating system. Reiserfs is not.

Re:How do these differ from Linux Threads?

[Kludge]

I read the paper. The paper describes only their own implementation, it does not compare to other implementations. Did you read the paper?

How do these differ from Linux Threads?

[Kludge]

Am I right in saying that Linux threads
are kernel level threads and the IBM/GNU threads are user space threads?

What's wrong with Linux threads? I've been using them for a while. They seem to work for me.Why are the IBM ones so much better?

Since they're both posix threads (pthread_create()) how does one determine which one will be used when both are on one system?

Re:How do these differ from Linux Threads?

[kulderzipken]

The libpthread which is found first by the shared library loader will be used (cf. /etc/ld.conf and LD_LIBRARY_PATH).

Re:How do these differ from Linux Threads?

[randombit]

I had written this out before and then Netscape decided it was time to crash.

What's wrong with Linux threads? I've been using them for a while. They seem to work for me.Why are the IBM ones so much better?

LinuxThreads are OK. The problem is that LinuxThreads uses a 1:1 mapping between kernel processes and the threads, whereas this package uses an N:M setup, which is much more scalable. The thing is that Linux doesn't really scale well to large number of processes (by large I mean around 10K+, to pull a number out of my hat). Since LinuxThreads is 1:1, that means these limitations also apply to LinuxThreads. Of course the IBM threads stuff isn't immune but the way it works could help things out quite a bit (though mostly for larger things).

Also LinuxThreads is kind of buggy. Having not used IBM's stuff I can't say anything one way or another there but hopefully this package improves on that.

Since they're both posix threads (pthread_create()) how does one determine which one will be used when both are on one system?

Generally speaking, you're only going to have one of these packages on the system. And since both (in theory) comply with the POSIX standard, it shouldn't matter which one is actually installed (except for the little issue of binary compatability, of course).

Re:How do these differ from Linux Threads?

[GGardner]

>What's wrong with Linux threads?

Off the top of my head:

Using pthreads makes core dumps unusable

Despite "thread aware" patches to gdb, the debugger still wierds out on my when using pthreaded application

Pthreads mmaps 2Mb for each pthread's stack, meaning that there is a hard limit of around a thousand threads in any 32 bit system

Doesn't look like the new pthreads implementation really fixes any but the last one, though.

Re:How do these differ from Linux Threads?

[Doomdark]

Oh please. I doubt many experience s/w engineers love async operations (I/O or otherwise). Conceptually using blocking threads (like in Java) is _MUCH_ simpler, and therefore generally leads to code that actually works. Async operations means that operations have to be somehow polled if you want any concurrency. And if you don't just do the queuing on your threads.

Re:How do these differ from Linux Threads?

[Diomedes01]

From what I understand, this library greatly increases the POSIX compliance of the threads under Linux. As of now, they are not fully compliant, especially in regards to SMP platforms. BTW, this information is all available if one actually bothers to follow the links and read the available information. Also, this whitepaper may be of some assistance (PostScript).


-------

Re:How do these differ from Linux Threads?

[Diomedes01]

Yes, I did read the paper. I was simply pointing it out as an additional resource. I agree that Linux threads work "fine", but that doesn't mean that they are as good as they could be. I think that the GNU/IBM attempt at threads is meant to simply increase performance and stability, and they appear to have done a great job of it. I'm currently downloading the package and will try to see what the differences are. Sorry if I came off as antagonistic, but I'm sick of people asking questions who haven't even read the material that is linked to. Apparently you did, so accept my apology.


-------

Re:IBM sucks

[scav]

You can really talk...The leader that was democratically elected in Israel by your colleagues and maybe you (however one may never know becuase you posted as an anon coward), killed Palestinians. This eyewitness account from a Dutch doctor living in North Wales (God knows what would happen if he was living in the USA) reveals Ariel Sharon's war crimes in the Sabra and Shatila massacres.

You all go on about IBM who did not order Hitler to kill any Jews and yet you go and kill any Palestinians you can get your hands on. Is it because the Jews are "chosen ones" and the rest of the world are worthless in your eyes?

Re:beyond 5

[jawad]

The only post I ever remember getting above 5 was a (Score 7 : Brilliant).. Of course, that was CmdrTaco himself.

Re:getting it in

[akeru]

Poke around the SGI's XFS website for a little bit and you'll find links to ISO's of a modified version of RedHat's 7.1 installer, that works with them, so that that install is started from teh XFS disk, and continues with all of the rest of the packages from teh regular RH ones.

Don't you mean?

[Elwood P Dowd]

Free as in Free Software? :)

I realize that Free Software is a subset of Open Source(TM), and the statement was not incorrect, but how hard is it to say?
--

IBM Rocks

[augustz]

That is all...

Too many JFS ??

[jamesk]

Will not the inclusion of so many different JFS now make it appear that linux is fragmenting? Yes I know that choice is good, but what about having to choose between so many good opinions?

ReiserFS, XFS and JFS will likely all do an excellent job but with so much overlap in functionality how will the community respond -- how will corporate america choose (more particularly, who will the CIO/CTO/VP of MIS decide to listen to, when fights break out among the tech staff over which JFS to use.

Choice is a great thing, its the choosing that's the killer.

Re:Too many JFS ??

[gmhowell]

I'm not sure why anyone (other than Micro-Soft) would think this would be a sign of forking. Presumably, all will wind up in the kernel, and all will be either compiled in or available as modules for all the major distros. "mount -t auto yadda yadda yadda" should take care of this.

This actually makes Linux stronger, as it provides choice. Got an old system: use ext3. Want speed: use ReiserFS. Want (I don't know. Sorry): use JFS, XFS, fooFS.

This isn't the either/or situation of FAT, FAT-32, and NTFS (not sure if XP will add another FS standard). All versions of the GNU/Linux OS should have the ability to read all of the filesystems.

You do, however, have a point that there is some overlap. I am not a part of any of the devel teams (or any devel teams for that matter) so it would seem, at least from a lay perspective, that having (just for example) Hans, the ext3 group, etc working on and submitting patches for JFS would result in the strongest solution. But there is nothing to preclude this from happening (except, perhaps, for Hans' ego and financial plans. And I don't know how open IBM is. And...)

Anyway, this is fairly early in the development, and the choice of filesystems is hardly as major a concern as the Gnome/KDE schism WRT forking possibilities.

Re:Too many JFS ??

[be-fan]

Multiple filesystems don't add to fragmentation, since they are all interface compatible. When multiple things have the same interface (whether they be software components or cars) multiplicity is called 'choice'. When they have different interfaces (GNOME, KDE, etc) that is 'fragmentation'.

Re:Too many JFS ??

[2Bits]

As far as I'm concerned, too many of a good thing is never a bad thing.

Re:Too many JFS ??

[zoftie]

I don't know, but reiserfs corrupted on me
horribly and I could not find a single disk
without reiserfsck, so I was fscked.
I haven't tried JFS yet, still using ext2.
Yes cash recovery bad, but I don't get mysteriosly
missing files and directories, in place of which
new ones cannot be created. That was 2.4.3 kernel
BTW.

Logging filesystems are have an inch think layer
of complexity, so each variation will inarguably
will offer its own perks and will come with
disadvantages. The more the merrier.
I hope that will not fragment the useshare, of
bugfinders though. Before ext2 was hacked on
by so many people all bugs were out in no time.
Now with fifty FS types in kernel, bugs are
more 'deep'. ESR - "with enough eyes all bugs are shallow"?

As 'bonus' I would praise IBM for Linux push, but
their highend hardware people suck and know shit,
for capabilities of hardware. There was a demo
S/390 @ office, and they the techies did not know
small endian from big endian. Not impressed at
all. Maybe its better closer to IBM motherland
who knows. Until then its sun stuff for us.

Re:Too many JFS ??

[Canonymous Howard]

Linux is fragmented in the same way that Chevy is fragmented.

Chevy sells cars, trucks, vans, minivans and commercial trucks. Not only that, they sell different kinds of cars- the Camero, the Cavalier, the Corvette, the Malabu, the Monte Carlo, the Prism, etc.

Wow, that's a lot of options.

Think there's any danger they'll be wiped out by a company that offers only one type of car, only one type of truck, and only one type of van? You have to admit, it would be less confusing that way... and everybody knows that Camero sales just scavenge sales that would otherwise go to Cavaliers...

Re:fooFS

[gmhowell]

I don't care if you call your filesystem fooFS; please don't call it BarFS.

Okay, time to go home. Took me three reads to get that one. Wish I had thought of it.

+1 Funny

Re:FS conversion tools please...

[gmhowell]

AFAIK (which ain't much), ext3 has this as a design consideration, or at least as part of the implementation.

I was tempted to wait, but got impatient and used ReiserFS. Perhaps when I upgrade the 20GB drive in there, I'll switch.

Re:Questions

[gmhowell]

You can/should be able to replace the WinME box with a Linux box (but the initial setup may require a Win box. My DSL was setup with NT4.0, but now runs on Linux).

A linux box can be made to read PC disks (Dos, Win3.1->NT), Amiga disks, Mac disks, and a bunch of others.

There is zero problem accessing files over the network amongst these systems. Depending on the mix, you can use NFS, SMB, AppleTalk, ftp, etc.

Finding your NIC is the first step. In the past, I would have reccommended linuxnewbie.org to ask that sort of question. Not sure where these days.

BTW, treat your father like the head of a major corporation: do what you have to do. Don't tell him you switched computers on him. Chances are he'll never notice. I take it you are still in school (be it elementary or university. It doesn't matter) and now have some time off for the summer. Presumably your father works. That gives you plenty of time to fiddle with things. Eventually, you'll get it right, and just leave it on.

LVM!Re:Too many JFS ?? - too much is never enough!

[microbob]

Now all we need is the LVM.

varyon MyPornVG

Microbob

Re:Too many JFS ?? - too much is never enough!

[henley]

This actually makes Linux stronger, as it provides choice. Got an old system: use ext3. Want speed: use ReiserFS. Want (I don't know. Sorry): use JFS, XFS, fooFS.

Speaking as an ex-AIX worshipper (I have now been converted to the One True OS and it's apt-get luvin' incarnation), IBM JFS has one MAJOR attraction for me: ease of filesystem modification.

I haven't tried JFS/Linux yet, but after years of expanding filesystems on production systems without outage ("chfs -a size=+blocks /fs/mount/point"), porting this to Linux is all it'll take me to switch. Performance? Pah! nice to have, but if you're reliant on FS performance you need to splurge a few beans on more memory. Ease of administration is the major win with JFS here folks....

Distros?

[jamesc]

Excellent! I wonder which distro will be the first to include JFS as an option? Ordinarily, I'd say SuSE, but they've released 7.2 recently.

Sure, I can (and will) install JFS myself, but I want to see it exposed to lots of people in a distro and stand up under the load before I put it on a production server. Example: SuSE and reiserfs.
--

Re:Distros?

[emir]

i doubt that, suse was first to adopt reiserFS not because they were more cutting edge than mandrake or debian but because they were main sponsors of hans reiser and his company.

Re:FS conversion tools please...

[jamesc]

So, a FS conversion tool would be really nice, like that DOS to NTFS thingy.

Yes, it would be. Don't hold your breath. In long standing *nix tradition, in-place fs conversion tools are very low priority. Instead, you're supposed to convert to the new fs by copying it to a new partition using the usual methods:

• Plug in an extra disk temporarily.
• Back up the original to tape (twice), then mkfs the partition.
• Back it up using the network, then mkfs.
• Back up to a compressed tar ball on some other file system.
• Etc. Use your ingenuity.

--

/Now/ will they port Lotus Notes?

[_Quinn]

So Linux now has an AIX-class FS and threads library... what else do they need?

-_Quinn

Re:/Now/ will they port Lotus Notes?

[OUSpirit]

You can run Notes R5 under Wine (post 01/2001 builds, I believe).
The link below might have some notes to help you get it up and running if you are interested.
http://www.keysolutions.com/NotesFAQ/canclient.htm l

Re:/Now/ will they port Lotus Notes?

[Diomedes01]

Are you referring to the Lotus Notes client or server? I believe that the server (Lotus Domino) has been available for Linux for quite some time. The client, AFAIK, is only available for Windows, so the existence of an AIX-class file system and threads library is completely irrelevant. -Dio


-------

JFS is buggy

[chrysalis]

I just installed JFS to benchmark it. It seems to be a bit buggy : sometimes the keyboard blo



-- Pure FTP server - Upgrade your FTP server to something simple and secure.

Re:Questions

[jguthrie]

BassGuy23 asked:

Can a linux box read a PC floppy or HD? How about one for Mac? Can a redhat box access files from a Mandrake one?

What you are asking is if Linux is compatable with all the partitioning methods and file systems that those kinds of computers use. The answer is "yes". Of course, the PC is well supported because even early Linux systems had to coexist with and use the filesystems for the other PC operating systems installed on the same hardware, but the Macintosh is well supported as well. In fact, recently my office received a Mac ZIP disk containing works in progress from our advertising agency and of all the computers in the office, only those running Linux could read it.

Whether or not a kernel from one distribution (Mandrake) can read the file system used by another (RedHat) depends upon the options used to build the kernel. Some older kernels won't be able to read some file systems. However, most all kernels distributed understand the second extended file system (or ext2fs) as it's been around for a number of years now and is widely used.

BassGuy23 also asked:

Can I get a linux box to access the internet through the Windows network?

On the Internet, there is no such thing as a "Windows network". Everybody uses the same TCP/IP protocols, including Windows computers. Unless the other Windows computers on the network are running special software as well, you shouldn't have a problem with a Linux computer.

Re:LVM

[theEd]

There already is a LVM in Linux (built in 2.4 or higher, patch for other versions). But if you are looking for IBM's LVM then I believe this is what you would be looking for...

http://oss.software.ibm.com/developerworks/opensou rce/evms/index.php

Re:Too many JFS ?? - too much is never enough!

[be-fan]

Actually, XFS is dynamically resizable too. And fast at that.

BTW> Is it just me, or do ugly people and slow systems have something in common...

Pansy-ass lightweights...

[be-fan]

Whenever a potentially interesting tech article comes up on /., all we get is a bunch of pansy-assed posts about licenses or whatnot. This is news for NERDs. Where are the hard-hitting questions? Why hasn't somebody downloaded the damn thing and posted benchmarks comparing the major JFSs? (I'm working on it!) How fast is it? How stable is it? How easy is it to install? How does it work internally? Good grief, you'd think you'd get something meaty on a discussion like this...

Re:Pansy-ass lightweights...

[be-fan]

Actually, if you got your head out of your elitist ass, you'd realize that JFSs are useful to EVERYONE, not just 'leet server admins. BeOS users have had a kick-ass JFS for years now helping keep their MP3 collections safe. For these desktop users, speed matters as well as safety. Thus, a benchmark of JFSs IS valid. If XFS and JFS are both equally safe in the "pull-the-plug" test, then why not use the faster one?

Filing system?

[quartz]

the second commercial filing system for Linux to reach the exalted 1.0 status!

Is that something I would run on my filing cabinet?

Re:Always nice

[Ded Bob]

FreeBSD will have it: fa.freebsd.hackers post.

It will only be used as a stepping stone until a freer version is complete:

- The main point of this port is to have a reasonable native freebsd
pthread implementation till the scheduler activations stuff is ready.

Re:IBM should be praised

[Ded Bob]

Also, they just switched to the MIT license. Cool.

Congrats, JFS team!

[Will the Chill]

Well, we've got ourselves a new filesystem to play with, guys, and IMHO a pretty cool one, at that. I'm a summer intern with the JFS team at IBM, so I'm definitely looking at this subjectively. However, I've been testing it's stability for the past month or so, and have been pleased with the results. (It'll pass RedHat's "cerberus" kernel stressing tests.)

Oh, and there are more features on the way, so check the JFS website every once in a while!

-Will the Chill

Mac Disks

[FyreFiend]

Linux can read HFS formatted disks w/o a problem. Unfortunely it can't read HFS+ (the default with MacOS 8.5 and newer). It's not a problem with floppies, zips, and other removable media because they are still formatted as HFS but hard drives are usually HFS+.

Re:FS conversion tools please...

[Cyberdyne]

So, a FS conversion tool would be really nice, like that DOS to NTFS thingy.

I'm sure the group (IBM, Reiser, SGI, Ext3) that comes up with the first will have a first-mover advantage and more users.

ext3 does this already. All the new features which have been retrofitted to ext2 recently - not just journalling, but directory indexes (to speed up file access using a tree instead of a linear search), tail-merging and a few other nice things - are backwards compatible. To use them, mount the file system with the right driver and mount option; mount under an older driver, and the FS still works, just without journalling etc. You don't even need a conversion utility: it just does it all at mount time!

Re:No More AIX

[supersnail]

I doubt if IBM is trying to dump AIX, it is after all by far the best implemtation of UNIX and is constantly improving.

There are several features such as workload management, cluster management, high availability features which are pretty much unique to AIX and which make running really large systems easier and less nerve racking.

IBMs interest in LINUX is two fold.

1. As LINUX and AIX converge, (LINUX gets JSF, pThreads, DB2 MQSeries etc. -- AIX gets LINUX comaptable libraries, linkers and package management), then IBM gets a super cheap platform for developing software that can run commercially on AIX.

2. Very few people are going to replace a large AIX server with a LINUX one, but they are likely to replace an NT or low end Solaris server with a cheap linux box -- so stuffing thier competitors.

Re:Questions

[emir]

> As someone who can't use Linux (we have a WinME box sharing our cable modem connection), I have often wondered how compatable are such things as different file systems? Can a linux box read a PC floppy or HD?

if you mean windows/dos formated floppy/hd by saying PC floppy than the answer is yes, it can read PC floppy or HD. > How about one for Mac?

not sure about this one , but i think it has support for mac filesystem.

> Can a redhat box access files from a Mandrake one?

yes. only difference between different linux distributions a) location of config files b) some distros are using bsd init (slack ?)instead of sysV init like rest of the the distros c) packaging system d) amount of apps shipped with the distro.

> Can I get a linux box to access the internet through the Windows network?

i'm no expert in this area but i believe you can do that with the help of samba. try looking at samba.org for more info. there is also free samba book published by oreilly on their site. good luck

Re:Isn't it third?

[Galvatron]

Isn't Reiser a hobbyist project? Generally, commercial means funded by a corporation for profit. So XFS would be first, and JFS would be second.

The only "intuitive" interface is the nipple. After that, it's all learned.

Re:Questions

[pi_rules]

Read Win32 and Mac disks? You betcha. Shoot, you can read:

• ADFS -- RiscOS
• Amiga FFS
• Apple Mac
• BFS -- Boot sector thing for SCO UnixWAre
• DOS FAT
• Fat16/32
• NTFS
• EFS -- Pre Iso9660 filesystem for CDROMS
• Ramdisks
• ISO 9660 for CD's. Plus the MS Joliet extensions
• Minix FS -- Nostalgia I suppose.
• FreeVxFS -- main FS in Sco UnixWare the docs say.
• HPFS -- OS/2's filesystem.
• QNX4FS -- for QNX Systems
• ext2 -- most common Linux filesystem
• System V filesystems, for SCoO, Xenix, and Coherent
• UDF -- for DVDs
• UFS -- for SunOS, FreeBSD, NetBSD, OpenBSD, NeXTstep.

And now we're getting ReiserFS, JFS, and XFS from SGI is on the way. Ext3 is out there too but not used all the time.

So yeah, you can read Win32 formatted disks and just about everything else under the sun.

And you don't have to use Windows to use @home. There's plenty of documentation out there on this.

getting it in

[donglekey]

I think this is awsome just like everyone else, but the real win will be when it gets into the standard install of a few distrobutions. Right now ResierFS and ext3 are relativly new to distro's installers. I think having XFS, Resiser and JFS in the install is the real kicker. I want XFS so bad I can taste it, but I suck at linux and don't want to try to get it to work without doing it from the installer.

Re:getting it in

[donglekey]

cool stuff, I will definitly do that

Re:Isn't it third?

[Wesley Felter]

ReiserFS is funded by Namesys, a for-profit corporation AFAIK.

Isn't it third?

[Wesley Felter]

What about ReiserFS and XFS? Or is there some weird meaning to "commercial filing system" that I don't get?

Re:beyond 5

[The Pim]

Third, everyone picks an appropriate (positive, smartass) moderation for this article

Uh, I'm talking about the "wakka wakka" article, not mine. You know, the gut-bustingly funny one.

beyond 5

[The Pim]

for the love of jesus, mod this guy up beyond 5.

A post with Score: 6 has appeared on slashdot before (anyone remember it?). This suggests that there was a race in slashcode; and since slashdot still uses a non-transactional data store, I bet the it's still there. I think everyone gets where I'm going with this.

So, here's the plan: First, someone moderates this back down to 4. Second, everyone with mod points synchronizes their clocks to UTC (apt-get install ntp). Third, everyone picks an appropriate (positive, smartass) moderation for this article and moves their pointer over the "Moderate" button. Fourth, at exactly midnight (00:00:00 UTC, June 29), everyone clicks "Moderate".

Everyone without mod points places bets on how high we can get this sucker.

FS conversion tools please...

[2Bits]

Alright, some open source hacker will tell me to go write it myself, for this kind of request....

A lot of people are using Linux distributions (e.g. RH) that don't come with journaling file system by default (argh....). Sure, we can create a dummy partition, and more the system to it, and create the journaled file system (JFS, ReiserFS, ...), and re-install on that, blah blah blah... it's that simple.

But hey, we got production system out there, and the disk has no space for new parition anymore. And we have patched and configured the system to work exactly like we want (performing, reliable, and secure), and we have no intention to re-install things and migrate. And I have a VAIO 505FX that does not have spare disk space to for use to install journaling FS, and I don't want to go thru all the hassle of re-installing everything.

So, a FS conversion tool would be really nice, like that DOS to NTFS thingy.

I'm sure the group (IBM, Reiser, SGI, Ext3) that comes up with the first will have a first-mover advantage and more users.

Re:FS conversion tools please...

[monkie-donkie]

ext3 does what you ask, it just adds journaling to ext2. you don't have to destroy any data

Re:beyond 5 [OT]

[smack_attack]

You won't get him past 5... I'll tell you why:

1) No one is going to mod him down to try your theory. sorry :(
2) The display limit for moderations is -1 to +5 he will still get the extra karma though.
3) In MySQL, you can't have 2 inserts occur at the exact same moment, there is a queueueu-ing system in place, this may not have been very robust when it happened previously.

So, now you know...

---

Re:Too many JFS ?? - too much is never enough!

[duffbeer703]

Amen brother. AIX rocks.

I wish they would port the AIX LVM. The linux lvm is a load of shit.

What IBM said when PthNG project was proposed:

[JCCyC]

"Make it so."

don't you mean

[sulli]

IBM's JFS & PTh-NG v1.0, GPLd & GZIPd, avl immed? No reason to include any complete English words, after all...

(And you thought slashdot was losing its "news for nerds" flavor!)

Re:Questions

[11223]

XFS is *on the way*? I run three computers with XFS and will be installing a fourth shortly. Take a look at their site - not only are they at 1.0, but there's a very useful install disc for RH7.1!

Always nice

[marcovje]

to companies that market with Linux also actually deliver, and also in a workable way (GPL).

Now I just hope that the GPL won't be too much of a problem to push this into FreeBSD

fooFS

[Cardhore]

I don't care if you call your filesystem fooFS; please don't call it BarFS.

Re:Questions

[rppp01]

I run ATT@Home in my home, and linux is my firewall/router for my home network. I don't seem to recall any 'special software' in order to run it. Of course I only get external equipment (cable hardware) so nothing goes in the box unless I put it there.

Re:JFS may not be the best

[markus_baertschi]

It is known that JFS performance can be limited if you perform a lot of activity which modifies the file-system structure. Everything from creating files, renaming to unpacking archives, etc.

One easy fix is to move the log volume of the filesystem to another disk. This can increase your performance a lot. If both are on the same physical volume, all the seeks to write data and the jfs log will kill your performance.

Markus

JFS may not be the best

[t'mbert]

Okay gurus (and I hope someone sees this)!

I have developed an app in tcl for my client. The target system is AIX, which we work with alot. I'm using Postgres for my database.

Now I developed the app on SuSE linux. Here's the machine specs for both:

• AIX: F70, 2-way, SCSI drives (no SSA), 1GB RAM
• SuSE: PII 333, Standard IDE, 128MB RAM

The tcl code reads stuff from the database, does ALOT of recursive file ops (like rm -r * and cp -r *), and creates a gihungus tar.gz of all those little directories and files.

It takes the VERY EXPENSIVE AIX box about 3 hours to run the process; it takes my $800 linux server less than an hour! Same code. The database is not at fault (runs real fast on AIX). Recursive ops from the shell also cause disk io to peg out, and the box is not at all busy with other things.

SuSE is 6.4, running LVM and ReiserFS. AIX is running AIX 4.3.3 with JFS and LVM.

So what gives? The sys admins are baffled. The bottleneck on the AIX is disk io. The joke around the office is that we'll replace that piece of carp AIX box with a top-of-the-line linux PII!

So is JFS at fault? Is ReiserFS that much faster? Any ideas? Changing databases or languages is not an option, this is a question about FS's, please stick to that.

Re:No More AIX

[fors]

AIX can't even begin to touch Tru64 in the clustering department.

Re:Questions

[tb3]

> Can I get a linux box to access the internet through the Windows network? You don't need to use Samba to get IP Routing, you just need to turn on IP sharing on the Windows box, and configure the Linux box accordingly.

I've got IP routing working between a Windows box with a cable modem and a Mac. Took about fifteen minutes to set up.

"What are we going to do tonight, Bill?"

IBM should be praised

[21mhz]

for another open-source project: International Components for Unicode. They really think of us the people of the whole world. I hope this will obsolete gettext.

Wakka wakka

[return 42]

Don't those idiots at IBM ever listen to Microsoft? Now look what'll happen!

GPLGP
LGPLGPL
GPLGPLGPL
GPLGPLG
PLGPLIBM
GPLGPLG
PLGPLGPLG
PLGPLGP
LGPLG

No More AIX

[locker1776]

IBM is just trying to get Linux as stable as AIX. I think they are going to get rid of AIX in a few years, and they hope Linux will be there for enterprise applications. Good for IBM: Free software development. Good for Users: An open source stable operating system with commercial goodies.