Slashdot Mirror
Journaling Flash File System
menthos writes: "It seems like Axis Communications has brought a YAJFS (Yet Another Journaling File System) to Linux. It is however a bit different, in that it is designed for flash memory filesystems, as in Axis's own Web cams. It is GPL, and the JFFS link is here. Maybe this could mean more Linux PDAs?"
There's also a link on that page for a bluetooth driver for linux. Anyone know of any bluetooth hardware already available? They mention that the driver supports any hardware that implements the HCI UART Transport Layer.
dan
Is it stable enough yet? WHich will be chosen as the successor to ext2? xfs? jfs? ext3? What?
First, you're mixing up meta-data journaling with data-journaling. JFFS is log-structured, which means that a write is not first journaled somewhere and then performed. Instead, the write and the data it replaces are written to the end of a log.
Second, NOR-flashes have very large erase-sector sizes (64 kbytes). It is impossible to run a normal filesystem on them, because you cannot simply rewrite a single FS block (normally in the order of 0.5 or 1 kbyte). You would then need to erase and rewrite the entire sector each time which takes like 2 seconds during which you can potentially loose all the blocks in the sector, along with greatly reducing the life-time of the Flash (since you'd need to rewrite the sectors all the time).
The result is that you need to write to the flash sequentially, which implies a log-structured journaling file-system, which is exactly what JFFS is. You only need to erase a sector when all the data in it is redundant. As a bonus, you get wear-levelling since the usage of the Flash is circulated (it's a circular log).
This is all described in the white-paper which we'll release soon.
Because flash memory is not standard memory. In particular it has slower access times and does not like to be written to often that much. This makes using caches necessary for good performance (especially on 200+ Mhz handhelds).
But if you use caches, there's always the question of what happens in the case of a power outage (as in, the battery just popped off). Journaling allows to go back quickly to a reasonable state 99.9% of the time (at least). To the point that the user will probably not notice the problem.
Incidentally, a jfs does _not_ protect against disk crashes at all, it is not designed for that. If you want protection against (most) disk crashes, what you need is RAID.
Actually, if you want very reasonable data security without daily backups, especially given the current cost of large disks and comparable-size backup systems, you probably need a jfs+raid combination.
OG.
People releasing mature, production-quality code to get hacked on are very cool.
:)
Maybe this could get used for floppies or RAM disks as well? Fast mp3-playing PC-like things? Or devfs, to save the state of the darn thing?
Anyhow, I'm sure the community will find nifty uses for it.
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
I think this is one thing about opensource that companies have overlooked. Often the first company to sell alot of units with a particular feature becomes a standard, (Office files for example). When you open source something like this you stand a better chance of other companies doing things your way. Suddenly your technology is the standard and you can sell a ton of the hardware because many manufacturers use it. I'm glad to see that some companies are getting the idea that they are spending money writing software anyway, if they keep it in-house it only works there, if they opensource it can become the way things are done many places.
Insert pithy comment here.
I admit it is their description. I was going to let it slide on that basis alone. But then I figured what the heck. We raise a stink in almost every thread where a media article mis-uses the term "hacking". Why should we let hardware marketers mis-use the term RISC just as badly?
Also for a somewhat similar CPU look at the Motorola 68F3XX series.
The addressing modes don't really make life simpler. They just let you do "LOAD ((R1)+R2), R3" rather then "LOAD (R1), R4; ADD R2, R4, R4; LOAD (R4), R3" (assuming target register is the last one). It doesn't save any cycles, even on CPUs where the double indirect plus displacement is faster then the two loads and add, the whole CPU is slower at everything for having the indirect mode!
Go back and read about the horror of the "fast" VAX in the Mashy essay I posted the link to.
The parts of the instruction set that make it CISCy aren't really needed for I/O. I don't object to the in-register byte/bit flopping. I don't object to the integrated Ethernet or serials. I definitely don't object to the integrated net booting.
To be honest I don't even object to the parts that make it CISCy, only that they chose to mis-lead people by saying "RISC" rather then "CISC and damn proud!". It's not the mistake that kills you, it;s the cover-up :-)
Thanks. I couldn't remember.
Well since it is "standard CISC with a bunch of peripherals stuck inside it to appeal to embedded network apps", I would go with "embedded CISC", or "embedded network CISC".
The Motorola ColdFire is a CISC that goes up to around 200Mhz at very low power. The Motorola DragonBall is a CISC that is definitely the low-power brains behind current Palm Pilots.
CISC doesn't mean power hungry. Hell thanks to Intel and AMD CISC doesn't even mean slow-as-a-dog.
Well, that's because it is a kick ass little CISC CPU.
To have totally no UI is a bit unusual. But to have no raster pixel display (LCD or otherwise) isn't uncommon. But most applications are either in such huge volumes that it's better to do a design just for that application (so you can save $0.03 by leaving off an address pin you don't need, or other stuff like that), or such low volume, that they are better off buying a board with parts they don't need.
I think you can normally get the "unneeded" parts down to serial ports by looking at the "BASIC Stamp" market, and the small SBCs that replace high-end stamps.
Or look at Chuck Moore's stuff. Sure it has the NTSC video generator, but are (well, will be, someday) dirt cheap. Oh, and it's definitely a CPU design that I have a hard time pegging as RISC or CISC. So you could win on that one :-) Besides it's so cool to fit the computer inside the mouse.
It's not very RISCy. It looks more like a almost-68000 with delay slot branches (if I remember right). It has a bunch of indirect addressing modes not found on most (or any) RISCs. Of corse it doens't have to be a RISC. It runs pretty quick (50Mhz? 100Mhz?), as you say it has lots of good I/O features and built-in networking.
It is a pretty cool CPU for the right target. But it ain't a RISC.
People who want to know more about a cool embeded CPU with lots of networking and I/O features should definitly checkout their little CISC CPU.
Anyone that thinks it is a RISC chould check out page 17 of the ETRAX 100 Data Sheet, and then maybe the John Mashey "What Is RISC" essay. Enjoy.
Flash memory has a very finite lifespan. You can only get a limited number of read/writes out of a flash chip (something ~ 1 billion IIRC) before it goes kablooey. Thus preventing read/write intensive operations is of the utmost importance. This, obviously, precludes letting fsck thrash your flash chip up.
--
I think there is a world market for maybe five personal web logs.
Seems like / and I both have a high Karma already.
FYI: Posting with anonymously gives you a default 0 points, posting with a user name gives you 1 point, posting with a user name and having a high Karma gives you 2.
So no, / and my messages weren't moderated up, 2 was simply their default score.
But yes, it was offtopic. Because of that, I now used the "no score +1 bonus" that is used by default. Back to a score of 1.
------------------
------------------
You may like my a cappella music
However, I don't quite understand your comment. Here in Germany, the word "Axis" or the German translation "Achse" isn't regarded as fascist vocabulary, unlike other words that you Americans seem to think are ok.
------------------
------------------
You may like my a cappella music
Hmm. The Axis software developer I met at CeBIT told me that I'd need the SDK to do so.
Anyway, dear Anonymous Coward, if you know more about this, please contact me at kontakt@hanno.de.
------------------
------------------
You may like my a cappella music
At University, I am working on a protocol extension to Jini to bring Jini to non-Java capable devices, e.g. embedded CPUs. Back when I began, we thought that we should find a cool hardware device to try the new protocol on. The Axis camera is a great product (I knew from several job-related projects) and it is hands-down the coolest device out there that combines an embedded microcontroller, an ethernet connector and a restricted hardware environment. Perfect to demonstrate my project of native Jini support without the need for a Java Virtual Machine.
October last year, long before I actually started working on my thesis, I requested developer information from Axis about their network camera. It took me four requests to get any response from them in the first place. Finally, after writing a polite complaint to Axis HQ Sweden, I got to an overly enthusiastic marketing drone from Axis Germany.
He told me that my project is a great thing, that Axis was very interested in it and that he would do everything he could do to help me.
He just noted that the formerly close-source firmware of the Axis camera will be changed to a port of the Linux kernel very soon and that I should wait for the new Linux version, due out in November.
"Isn't that great?" he told me "you don't even have to sign an NDA, we will provide you with the complete source of the firmware and one or two free demo units. All you have to do to get the demo units is sign a research agreement with us." I have an email from him lying around somewhere where he confirms this offer.
Oh, you can imagine how happy I was. Just a few weeks to get a demo unit with full firmware source and it's a cool product, too!
Well, weeks passed. Months passed. My contact person was always happy, bright and optimistic when I called him and asked about the progress of the product. No matter when I called him, the camera was due to be released "in two or three weeks" and "yes, you will get your demo unit with firmware source from us."
Strange that one time he asked me specifically not to contact the firmware developers at Axis Sweden before the release of the camera, since is "going to cause problems" for him...
Well, the camera was finally released in late Februar for CeBIT. There, I met an actual developer from Axis Sweden. He had never heard of my continuous requests.
"Interesting. It would have been nice to know this a few months back so that we could have looked into this project during our development." Yeah, but I told Axis about it four months ago.
The bottom line: There will be no freely available firmware source of the camera until "the end of the year". The developers' superiour has no interest in my thesis project, so that even with an NDA I will not be allowed to modify the camera's firmware now. (I could wait another year, but hey, one day I do want to finish University...)
I would have been nice to know this four months ago. Of course, because of the continouing promises and full confirmations by Axis Germany, I did not look into alternative hardware.
Anyway, if you're interested, see http://developer.jini.org:80/exc hange/users/hanno/ for more information about my protocol extension. You'll need a Jini developer account.
------------------
------------------
You may like my a cappella music
Those of us who have barely made through your half-assed English and only kinda understood what you meant didn't find it very funny either. Verbal concordance is the kind of thing you learn in Basic English 101. Sign up for a class in the most nearby kindergarten, ASAP.
To the editors: your English is as bad as your Perl. Please go back to grade school.
Absolutely, but you're forgetting what it is in power outages that causes disk crashes.
:) But anyway, this isn't a fail safe method, you just have to be a much better shot with the power button.
;)
Filesystems (and ext2fs in particular) keep a certain amount of the state of the 'disk' in memory, leaving the actual physical disk in an incomplete state. If you see what I mean.
The simple solution is to use synchronous writes - the api call doesn't return until the state has been written on disk. This, incidentally, is the default for mount under FreeBSD - you can make FreeBSD a whole lot faster writing to disk by mounting async, but it would be less safe so you have to enable it manually
A jfs means that even if you trip the power half way through the write, the whole thing can be rolled back to a pre-written state, a la database transactions.
A quick aside is the softupdates package under FreeBSD which does something similar to a jfs. Unfortunately I've not got further than the limited licence for this so I cannot comment further.
So, why does this matter for flash? Because flash is embedded. It cannot be allowed to go wrong. Do you really want a fsck button on your digital camera? Oh - OK, errrmm, Aircraft? Exactly.
Corrections by email please
Dave
I write a blog now, you should be afraid.
Yopy has most certainly a JFS-like solution for its flash memory, but as Yopy itself is not in production yet and there's yet no source to be seen, I don't know for certain.
That's what I find so nice about the JFFS: It's here, and it's here now. And with an "established" JFFS, Linux PDA manufacturers and manufacturers of many other types of embedded devices with flash memory maybe won't have to re-invent the same thing over and over. So you're right, PDA manufacturers don't have to use this and they can use whatever they seem fit, including their own custom solutions, but still I think that this is great because PDA manufacturers won't have to invent some solution of their own.
I don't know if there are other documented journaling flash file systems for Linux out there, but I know that I haven't seen one before.
GNU/Linux. The Freshmaker.
The irony is that I actually checked the "no score +1 bonus" box, but slashdot munged the result and gave me the bonus anyway.
"If one is really a superior person, the fact is likely to leak out without too much assistance" -- John Andrew Holmes
You do have to wonder at a German company named "Axis". Are they deliberately trying to sound fascist or what? You should probably should have gone with a nice American company like "Allied Computing". :-)
"If one is really a superior person, the fact is likely to leak out without too much assistance" -- John Andrew Holmes
Flash has a limited write lifespan. Journalling requires writing to the journal before every modification to the fs. Flash being very slow for writes makes this a major performance hit.
.
fsck doesn't need to make very many writes at all when fixing an insane drive. Recovery requires lots of reads, but the flash reads are fast and the drive is very small. This performance hit is nothing compared to doubling the already slow flash writes.
.
There are filesystems that do not leave the fs in an insane state by making only one modification at a time, and do not have this kind of overhead for regular use. The only case where the journal would give stability with better performance is deleting massive directory trees...
-- http://thegirlorthecar.com funny dating game for guys
>It's not very RISC
I was using their description. I agree that it's a rather dubious one. However their processor is hard to categorise - I don't think I've ever encountered anything quite like it before.
>It has a bunch of indirect addressing modes not found on most (or any) RISCs.
Of course the addressing modes are very valuable for the kind of work this chip is intended for.
Maybe this is why it is hard to categorise - the design (including the instrcution set) is optimsed towards doing I/O and networking manipulation.
>It runs pretty quick (50Mhz? 100Mhz?),
100 mips IIRC
>as you say it has lots of good I/O features and >built-in networking.
Excellent for the kind of thing I need to do.
>But it ain't a RISC
Perhaps not - but it isn't really a classical CISC either. Perhaps someone needs to invent a new acronym ?
The Data sheet seems to push the "RISC" angle as an explanation for the low power consumption - and it certainly achieves pretty good results there.
The "bottom line" for me is that, on paper, it allows me to build a solution with a hardware cost around half that I can achieve any other way.
I think perhaps my application is unusual - I don't want any displays or user interface (the customer explicitly wants a "grey box screwed to the wall that they can just forget about") - so many single board computers on the market would just sit there with half the chips never doing anything - but I can't be the only person who needs to do this kind of thing ?
AJB
I design Telecomms stuff - much of it embedded. We've just been asked to design a box which takes serial data from PBXs, collates it, caches it and makes it them available over a network. JFFS is one very key part of that - and this announcement could not have come at a better time for me.
Axis have an excellent business model on all of this. They've designed their own custom RISC processor with built-in network support and quite a lot of very neat I/O ( A dream come true for telecomms embedded people). They're offering Linux as the OS and it seems they've taken a long, hard look at Linux and what it might lack for people using their chip. They spotted a gap - so they created the JFFS and released back to the developer community which gave them Linux in the first place.
It seems to me to be a perfect example of how Open source should work and how it can work for everyone's benefit - and how open source can be part of a very rational business stratgey.
So I hope to be an Axis customer very soon.
People who want to know more about the RISC chip should look at :
http://developer.axis.com/about/
AJB
Which AIX was that? Default AIX filesystem is JFS (journaling fs) which keep's journal log of all transactions on a separate LV. You must know that journaling doesn't help you in saving your data as RAID and backups do... If disk dies, no journaling will help you. Journaling is about speed, not about safety (so you don't have to fsck all of the disk)
Just wonder if someone could make a few more: 20gb ibm- one partition ibm jfs and one sgi-jfs 8 gb seagate reiser fs (jfs) 20 gb scsi needs a new jfs...
Yes This is the soulution...
(Those who only laughs of jokes and only take serious what is serious has not understood a thing...
It's cool, I just don't know what it's for!
=P
Free music from Jack Merlot.
We just lost an AIX file system at work. Someone pulled the plug on it without shutting the computer down and it appeared not to have a journaling system on it, how come? Does anyone know about AIX journaling?
I love *nix and all, but this really bugs me. You switch windows or mac off, nothing like that happens, so it takes a little longer to boot due to the fs verification but you turn your AIX box and that's it, it's dead. Praise the backups...
You can't handle the truth.