Journaling Flash File System

menthos writes: "It seems like Axis Communications has brought a YAJFS (Yet Another Journaling File System) to Linux. It is however a bit different, in that it is designed for flash memory filesystems, as in Axis's own Web cams. It is GPL, and the JFFS link is here. Maybe this could mean more Linux PDAs?"

Comments

Re:This fs is a bad idea - Wrong on all accounts

[Anonymous Coward]

You're wrong on all points here actually. What you mean are bad reasons for a JFFS, are actually the reasons that makes a flash require a journaling system, and makes it a lot faster and less wearing on the Flash.

First, you're mixing up meta-data journaling with data-journaling. JFFS is log-structured, which means that a write is not first journaled somewhere and then performed. Instead, the write and the data it replaces are written to the end of a log.

Second, NOR-flashes have very large erase-sector sizes (64 kbytes). It is impossible to run a normal filesystem on them, because you cannot simply rewrite a single FS block (normally in the order of 0.5 or 1 kbyte). You would then need to erase and rewrite the entire sector each time which takes like 2 seconds during which you can potentially loose all the blocks in the sector, along with greatly reducing the life-time of the Flash (since you'd need to rewrite the sectors all the time).

The result is that you need to write to the flash sequentially, which implies a log-structured journaling file-system, which is exactly what JFFS is. You only need to erase a sector when all the data in it is redundant. As a bonus, you get wear-levelling since the usage of the Flash is circulated (it's a circular log).

This is all described in the white-paper which we'll release soon.

/One of the JFFS designers

Why a flash jfs?

[Olivier Galibert]

Because flash memory is not standard memory. In particular it has slower access times and does not like to be written to often that much. This makes using caches necessary for good performance (especially on 200+ Mhz handhelds).

But if you use caches, there's always the question of what happens in the case of a power outage (as in, the battery just popped off). Journaling allows to go back quickly to a reasonable state 99.9% of the time (at least). To the point that the user will probably not notice the problem.

Incidentally, a jfs does _not_ protect against disk crashes at all, it is not designed for that. If you want protection against (most) disk crashes, what you need is RAID.

Actually, if you want very reasonable data security without daily backups, especially given the current cost of large disks and comparable-size backup systems, you probably need a jfs+raid combination.

OG.

Wow...

[pb]

People releasing mature, production-quality code to get hacked on are very cool.

Maybe this could get used for floppies or RAM disks as well? Fast mp3-playing PC-like things? Or devfs, to save the state of the darn thing?

Anyhow, I'm sure the community will find nifty uses for it. :)
---
pb Reply or e-mail; don't vaguely moderate.

Standards creation

[narrowhouse]

I think this is one thing about opensource that companies have overlooked. Often the first company to sell alot of units with a particular feature becomes a standard, (Office files for example). When you open source something like this you stand a better chance of other companies doing things your way. Suddenly your technology is the standard and you can sell a ton of the hardware because many manufacturers use it. I'm glad to see that some companies are getting the idea that they are spending money writing software anyway, if they keep it in-house it only works there, if they opensource it can become the way things are done many places.

Re:Question:

[stripes]

I was using their description. I agree that it's a rather dubious one. However their processor is hard to categorise - I don't think I've ever encountered anything quite like it before.

I admit it is their description. I was going to let it slide on that basis alone. But then I figured what the heck. We raise a stink in almost every thread where a media article mis-uses the term "hacking". Why should we let hardware marketers mis-use the term RISC just as badly?

Also for a somewhat similar CPU look at the Motorola 68F3XX series.

Of course the addressing modes are very valuable for the kind of work this chip is intended for.

The addressing modes don't really make life simpler. They just let you do "LOAD ((R1)+R2), R3" rather then "LOAD (R1), R4; ADD R2, R4, R4; LOAD (R4), R3" (assuming target register is the last one). It doesn't save any cycles, even on CPUs where the double indirect plus displacement is faster then the two loads and add, the whole CPU is slower at everything for having the indirect mode!

Go back and read about the horror of the "fast" VAX in the Mashy essay I posted the link to.

Maybe this is why it is hard to categorise - the design (including the instrcution set) is optimsed towards doing I/O and networking manipulation.

The parts of the instruction set that make it CISCy aren't really needed for I/O. I don't object to the in-register byte/bit flopping. I don't object to the integrated Ethernet or serials. I definitely don't object to the integrated net booting.

To be honest I don't even object to the parts that make it CISCy, only that they chose to mis-lead people by saying "RISC" rather then "CISC and damn proud!". It's not the mistake that kills you, it;s the cover-up :-)

100 mips

Thanks. I couldn't remember.

Perhaps not - but it isn't really a classical CISC either. Perhaps someone needs to invent a new acronym ?

Well since it is "standard CISC with a bunch of peripherals stuck inside it to appeal to embedded network apps", I would go with "embedded CISC", or "embedded network CISC".

The Data sheet seems to push the "RISC" angle as an explanation for the low power consumption - and it certainly achieves pretty good results there.

The Motorola ColdFire is a CISC that goes up to around 200Mhz at very low power. The Motorola DragonBall is a CISC that is definitely the low-power brains behind current Palm Pilots.

CISC doesn't mean power hungry. Hell thanks to Intel and AMD CISC doesn't even mean slow-as-a-dog.

The "bottom line" for me is that, on paper, it allows me to build a solution with a hardware cost around half that I can achieve any other way.

Well, that's because it is a kick ass little CISC CPU.

I think perhaps my application is unusual - I don't want any displays or user interface (the customer explicitly wants a "grey box screwed to the wall that they can just forget about") - so many single board computers on the market would just sit there with half the chips never doing anything - but I can't be the only person who needs to do this kind of thing ?

To have totally no UI is a bit unusual. But to have no raster pixel display (LCD or otherwise) isn't uncommon. But most applications are either in such huge volumes that it's better to do a design just for that application (so you can save $0.03 by leaving off an address pin you don't need, or other stuff like that), or such low volume, that they are better off buying a board with parts they don't need.

I think you can normally get the "unneeded" parts down to serial ports by looking at the "BASIC Stamp" market, and the small SBCs that replace high-end stamps.

Or look at Chuck Moore's stuff. Sure it has the NTSC video generator, but are (well, will be, someday) dirt cheap. Oh, and it's definitely a CPU design that I have a hard time pegging as RISC or CISC. So you could win on that one :-) Besides it's so cool to fit the computer inside the mouse.

Re:Question:

[stripes]

They've designed their own custom RISC processor with built-in network support and quite a lot of very neat I/O ( A dream come true for telecomms embedded people).

It's not very RISCy. It looks more like a almost-68000 with delay slot branches (if I remember right). It has a bunch of indirect addressing modes not found on most (or any) RISCs. Of corse it doens't have to be a RISC. It runs pretty quick (50Mhz? 100Mhz?), as you say it has lots of good I/O features and built-in networking.

It is a pretty cool CPU for the right target. But it ain't a RISC.

People who want to know more about a cool embeded CPU with lots of networking and I/O features should definitly checkout their little CISC CPU.

Anyone that thinks it is a RISC chould check out page 17 of the ETRAX 100 Data Sheet, and then maybe the John Mashey "What Is RISC" essay. Enjoy.

Re:Question:

[drix]

Flash memory has a very finite lifespan. You can only get a limited number of read/writes out of a flash chip (something ~ 1 billion IIRC) before it goes kablooey. Thus preventing read/write intensive operations is of the utmost importance. This, obviously, precludes letting fsck thrash your flash chip up.

--

Re:Well... :-)

[Hanno]

The company is Swedish, but they have local offices in Germany and several other countries.

However, I don't quite understand your comment. Here in Germany, the word "Axis" or the German translation "Achse" isn't regarded as fascist vocabulary, unlike other words that you Americans seem to think are ok.

------------------

Re:Axis screwed my master thesis project

[Hanno]

Hmm. The Axis software developer I met at CeBIT told me that I'd need the SDK to do so.

Anyway, dear Anonymous Coward, if you know more about this, please contact me at kontakt@hanno.de.

------------------

Axis screwed my master thesis project

[Hanno]

Ok, this is going to hurt my karma, but let me vent my anger here.

At University, I am working on a protocol extension to Jini to bring Jini to non-Java capable devices, e.g. embedded CPUs. Back when I began, we thought that we should find a cool hardware device to try the new protocol on. The Axis camera is a great product (I knew from several job-related projects) and it is hands-down the coolest device out there that combines an embedded microcontroller, an ethernet connector and a restricted hardware environment. Perfect to demonstrate my project of native Jini support without the need for a Java Virtual Machine.

October last year, long before I actually started working on my thesis, I requested developer information from Axis about their network camera. It took me four requests to get any response from them in the first place. Finally, after writing a polite complaint to Axis HQ Sweden, I got to an overly enthusiastic marketing drone from Axis Germany.

He told me that my project is a great thing, that Axis was very interested in it and that he would do everything he could do to help me.

He just noted that the formerly close-source firmware of the Axis camera will be changed to a port of the Linux kernel very soon and that I should wait for the new Linux version, due out in November.

"Isn't that great?" he told me "you don't even have to sign an NDA, we will provide you with the complete source of the firmware and one or two free demo units. All you have to do to get the demo units is sign a research agreement with us." I have an email from him lying around somewhere where he confirms this offer.

Oh, you can imagine how happy I was. Just a few weeks to get a demo unit with full firmware source and it's a cool product, too!

Well, weeks passed. Months passed. My contact person was always happy, bright and optimistic when I called him and asked about the progress of the product. No matter when I called him, the camera was due to be released "in two or three weeks" and "yes, you will get your demo unit with firmware source from us."

Strange that one time he asked me specifically not to contact the firmware developers at Axis Sweden before the release of the camera, since is "going to cause problems" for him...

Well, the camera was finally released in late Februar for CeBIT. There, I met an actual developer from Axis Sweden. He had never heard of my continuous requests.

"Interesting. It would have been nice to know this a few months back so that we could have looked into this project during our development." Yeah, but I told Axis about it four months ago.

The bottom line: There will be no freely available firmware source of the camera until "the end of the year". The developers' superiour has no interest in my thesis project, so that even with an NDA I will not be allowed to modify the camera's firmware now. (I could wait another year, but hey, one day I do want to finish University...)

I would have been nice to know this four months ago. Of course, because of the continouing promises and full confirmations by Axis Germany, I did not look into alternative hardware.

Anyway, if you're interested, see http://developer.jini.org:80/exc hange/users/hanno/ for more information about my protocol extension. You'll need a Jini developer account.

------------------

Why have a jfs for flash?

[WasterDave]

Absolutely, but you're forgetting what it is in power outages that causes disk crashes.

Filesystems (and ext2fs in particular) keep a certain amount of the state of the 'disk' in memory, leaving the actual physical disk in an incomplete state. If you see what I mean.

The simple solution is to use synchronous writes - the api call doesn't return until the state has been written on disk. This, incidentally, is the default for mount under FreeBSD - you can make FreeBSD a whole lot faster writing to disk by mounting async, but it would be less safe so you have to enable it manually :) But anyway, this isn't a fail safe method, you just have to be a much better shot with the power button.

A jfs means that even if you trip the power half way through the write, the whole thing can be rolled back to a pre-written state, a la database transactions.

A quick aside is the softupdates package under FreeBSD which does something similar to a jfs. Unfortunately I've not got further than the limited licence for this so I cannot comment further.

So, why does this matter for flash? Because flash is embedded. It cannot be allowed to go wrong. Do you really want a fsck button on your digital camera? Oh - OK, errrmm, Aircraft? Exactly.

Corrections by email please ;)

Dave

Re:Why have a jfs for flash?

[toh]

FreeBSD "non-async" mounts differ a bit from Linux's idea of "sync". Linux's async is completely async, as in it writes data and metadata willy-nilly as the opportunity arises. Its sync mount is just the opposite, writing everything synchronously in an anal-retentive manner that's just too slow for most real-world use. FreeBSD's async mount is the same (dangerous), but the default mount is actually "non-async" which offers a sort of best-of-both compromise where metadata is written synchronously (you can also make a UFS filesystem completely synchronous, with roughly the same performance penalty as on Linux).

Softupdates is very cool, but it's not really journalling AFAIK. It's just a very well-thought out extension to the existing FFS "non-async" mode that knows how to ordering metadata writes so that a crash can always be recovered from, and so that it knows which ones it can be lazy about. Thus both performance and reliability are improved, and it accomplishes one of (not all) of the goals of a JFS, but in a fairly different manner. There are still benefits to be had from adopting a real JFS, though (I'm pulling for SGI's XFS). I understand softupdates is also due to be released fully free under the BSD licence, once the author thinks it's ready (it's part of the distribution now, you just have to create the symlinks in indication of your acceptance of the current licence). There's a white paper on softupdates at http://www.ece.cmu.edu/~ganger /papers/CSE-TR-254-95/ that gives some explanation of the existing sync/non-async/async scheme as well.

Well... :-)

[/]

You do have to wonder at a German company named "Axis". Are they deliberately trying to sound fascist or what? You should probably should have gone with a nice American company like "Allied Computing". :-)

Re:Well... :-)

[/]

World War II was fought between the Axis Powers (Germany, Austria, Italy, etc.) against the Allied Powers (US, Great Britain, etc.)

This fs is a bad idea

[acidrain]

Flash has a limited write lifespan. Journalling requires writing to the journal before every modification to the fs. Flash being very slow for writes makes this a major performance hit.
.
fsck doesn't need to make very many writes at all when fixing an insane drive. Recovery requires lots of reads, but the flash reads are fast and the drive is very small. This performance hit is nothing compared to doubling the already slow flash writes.
.
There are filesystems that do not leave the fs in an insane state by making only one modification at a time, and do not have this kind of overhead for regular use. The only case where the journal would give stability with better performance is deleting massive directory trees...

Re:Question:

[choco]

I design Telecomms stuff - much of it embedded. We've just been asked to design a box which takes serial data from PBXs, collates it, caches it and makes it them available over a network. JFFS is one very key part of that - and this announcement could not have come at a better time for me.

Axis have an excellent business model on all of this. They've designed their own custom RISC processor with built-in network support and quite a lot of very neat I/O ( A dream come true for telecomms embedded people). They're offering Linux as the OS and it seems they've taken a long, hard look at Linux and what it might lack for people using their chip. They spotted a gap - so they created the JFFS and released back to the developer community which gave them Linux in the first place.

It seems to me to be a perfect example of how Open source should work and how it can work for everyone's benefit - and how open source can be part of a very rational business stratgey.

So I hope to be an Axis customer very soon.

People who want to know more about the RISC chip should look at :

http://developer.axis.com/about/

How many does we need jfs's does we need?

[pol-pot]

Just wonder if someone could make a few more: 20gb ibm- one partition ibm jfs and one sgi-jfs 8 gb seagate reiser fs (jfs) 20 gb scsi needs a new jfs...

Yes This is the soulution...

(Those who only laughs of jokes and only take serious what is serious has not understood a thing...

Question:

[dragonfly_blue]

Why do we necessarily need to have a journaling file system for flash memory? I had thought that journaling was used as a failsafe mechanism to safeguard against power outages or disk crashes.

It's cool, I just don't know what it's for!

=P