Samsung Creates New File System F2Fs For Linux & Android

sfcrazy writes "Samsung has created a new Linux file system called F2FS. Jaegeuk Kim of Samsung writes on the Linux Kernel Mailing List: F2FS is a new file system carefully designed for the NAND flash memory-based storage devices. We chose a log structure file system approach, but we tried to adapt it to the new form of storage. Also we remedy some known issues of the very old log structured file system, such as snowball effect of wandering tree and high cleaning overhead."

Comments

SSD Drives

[Terry Pearson]

While the primary benefit will initially be for Android devices, this will be great news for solid state drives as well. Great job Samsung!

Re:SSD Drives

[Anonymous Coward]

No. SSDs present themselves to the OS as contiguous block devices. Filesystems intended for bare NAND flash like jffs(2), yaffs, and this new F2Fs would be totally useless for SSDs. They're intended for bare NAND, which SSDs are not.

Re:SSD Drives

[gumpish]

Did you get the money to buy your SSD drive by going to the ATM machine and entering your PIN number?

Re:SSD Drives

[Terry Pearson]

I am not 100% clear on NAND tech, but Samsung had an announcement about a NAND based SSD last week. I believe this is related to the F2FS announcement today. http://www.anandtech.com/show/6329/samsung-releases-tlc-nand-based-840-ssd

Re:SSD Drives

[Anonymous Coward]

No, I stole it.

Re:SSD Drives

[Anonymous Coward]

SSDs are NAND, but they are not bare NAND. They have control circuitry which manages the problems with NAND (e.g. bad blocks), and presents the drive as a contiguous block of good storage.

These filesystems are all for bare NAND, not SSDs, which include NAND, but are not bare NAND.

Re:SSD Drives

[Anonymous Coward]

No, I stole it.

The money for the drive, or the drive itself? I need clarification, you weren't very specific.

Re:SSD Drives

[Anonymous Coward]

The PIN number at your ATM machine so I could get a free SSD drive.

Re:SSD Drives

[Anonymous Coward]

Sorry, I stole the ATM, the money in the ATM, and the SSD in the ATM.

-- Ethanol-fueled

Re:SSD Drives

[Dragonslicer]

To be fair, SSD could also stand for Solid State Disk, in which case SSD drive wouldn't be quite so bad.

Re:SSD Drives

[Dekker3D]

So it's a drive you put solid state disks into?
Not just a pedantic joke.. I actually like that thought. Twenty-first century diskdrives :D

Re:SSD Drives

[gweihir]

This is only for raw NAND chips without management functions. SSDs are all managed.

Re:SSD Drives

[ATMAvatar]

Why bother when you can purchase it online by sending data through your NIC card?

Re:SSD Drives

[bogolisk]

SSDs are NAND, but they are not bare NAND. They have control circuitry which manages the problems with NAND (e.g. bad blocks), and presents the drive as a contiguous block of good storage.

These filesystems are all for bare NAND, not SSDs, which include NAND, but are not bare NAND.

How can this be "Informative", it's plain wrong. f2fs works on top of block devices.

Re:SSD Drives

[evilviper]

Did you get the money to buy your SSD drive by going to the ATM machine and entering your PIN number?
 

Nope, I couldn't... ATM, the ATM isn't on the ATM. No ETA on the ATM getting back on the ATM, ATM.

Now who's complaining?

Re:SSD Drives

[fnj]

Wrong, actually.

Re:SSD Drives

[Dr.Dubious DDQ]

Oh, THIS old joke again? That does it, I'm hunting you down and hacking your machine. Just let me look up your IP protocol address on the DNS server...

Re:SSD Drives

[mrmeval]

Yea right this is from the company that used the shittiest file system on the planet and continues to do so. I'm sure this one won't be a stuttering piece of shit like their other implementations.

Re:SSD Drives

[GNious]

Thinking that would be a USB port...

Re:SSD Drives

[repvik]

Bare NAND is presented as a block device. NAND SSDs are also presented as block devices. That does not imply that they are equal. SSDs have a controller that does remapping on the fly, in many cases on the fly compression, bad block handlling and much more. Bare NAND does not have that layer. That is why the ACs comment should be moderated informative, and you should be moderated "plain wrong".

Re:SSD Drives

[Shapemaker]

Replying to undo a mis-moderation.

Re:SSD Drives

[Patch86]

Why would you need to look up his IP protocol? There are only two of them (4 and 6), and Slashdot doesn't support 6 (I think).

You might want to look up his IP number- but that's not very funny.

Re:SSD Drives

[gumpish]

Disks are usually round.

Keyword: Android

[GeneralTurgidson]

I don't think Samsung would've created this if Android wasn't a huge driver behind their mobile success.

Re:Keyword: Android

[Anonymous Coward]

Well yeah, McDonalds isn't going to going to manufacture couches when it has no benefit to them.

Re:Keyword: Android

[Xtifr]

Sure, and they might not have released it to the public if it weren't for the GPL. On the other hand, they've developed something that looks like it may be very useful, and have released it without batting an eye. They're one of only seven Platinum members of the Linux Foundation. I think it's clear they understand how the ecosystem works, and they're happy to participate. Hard to fault them for that.

And actually, as I understand it, they use Linux for a lot more than just Android devices. They also have embedded Linux in other systems, like TVs.

Re:Keyword: Android

[Anonymous Coward]

Some freezers too. They are actually involved with the development of the ELF end the enlightenment windows manager.

I think Samsung likes Linux a lot

Re:Keyword: Android

[MobileTatsu-NJG]

And actually, as I understand it, they use Linux for a lot more than just Android devices. They also have embedded Linux in other systems, like TVs.

Really? Well that explains why my Samsung TV has such a horrible interface*.

* Kidding. :P

Re:Keyword: Android

[CFD339]

You nailed it. I wish I had mod points today to mod you up.

Re:Keyword: Android

[GT66]

The irony being that a McDonalds couch would likely contain more actual meat than one of their hamburgers.

Re:Keyword: Android

[Xtifr]

I think Samsung likes Linux a lot

Considering that a Platinum membership in the Linux Foundation requires plopping down at least half a million bucks, I suspect you're probably right. :)

Heck, Google only has a Gold membership, and we know they like Linux. Samsung is in elite territory with corporations like IBM and Intel.

Re:Keyword: Android

[Xtifr]

I know you're kidding, but I should point out that Linux is not a requirement for building bad interfaces (though one might claim that it helps). TV engineers in general seem to have some impressive skills at building bad interfaces. My last three TVs all had terrible interfaces, and none of them were Linux-based. :)

Re:Keyword: Android

[AvitarX]

Except, funnily enough, that's my biggest complaint about Samsung electronics in general (their TV remotes are horribly designed, and their digital cameras I've used had really stupid interfaces too).

They always look great spec wise for the money, but the actual product feels just not right (their premium Android phones being an exception I hear, though I've avoided them after my other experiences with their stuff).

Re:Keyword: Android

[MobileTatsu-NJG]

It does make me wish Apple would release a TV so Samsung'd finally get on the ball.

Re:Keyword: Android

[Anonymous Coward]

$99 Apple TV is a much better product to sell. More profit, more useful/transferrable, fewer SKUs, less inventory, lower cost WIP, on and on.

Re:Keyword: Android

[Anonymous Coward]

"The irony being that a McDonalds couch would likely contain more actual meat than one of their hamburgers."

How dare you, Sir? They're 100% cow udders and noses.

Re:Keyword: Android

[Anonymous Coward]

Especially with a typical McDonald's customer sitting on it.

Re:Keyword: Android

[Anonymous Coward]

Why aren't you using the terminal via the serial port? The command line is obviously more useful than the GUI in Linux-based systems.

Re:Keyword: Android

[Anonymous Coward]

Sure, and they might not have released it to the public if it weren't for the GPL. On the other hand, they've developed something that looks like it may be very useful, and have released it without batting an eye. They're one of only seven Platinum members of the Linux Foundation. I think it's clear they understand how the ecosystem works, and they're happy to participate. Hard to fault them for that.

And actually, as I understand it, they use Linux for a lot more than just Android devices. They also have embedded Linux in other systems, like TVs.

But it's a file system. You would probably be better off if they released just a well defined spec and some reference code to get started. Then you would get multiple implementations and better chance of catching bugs.

Really, what matters most is if it's very well defined, and absolutely not in the "source code is the documentation" way. If necessary implementation details live solely in the code and not well documented algorithms and data structures, what's it all worth really.

Anyway, rah rah GPL code is cool or whatever, but handing you a (well?) designed file system gratis is the real favor.

Re:Keyword: Android

[maxdread]

Well considering the vast size of Samsung, they probably do far more work with Linux than Google does as well.

People forget we're talking about a company that not only builds products in pretty much every home electronics category but also ships, CCTV, aircraft (for a while), artillery and automated turrets. None of this counting the bits and pieces they research and build that go into each of those products.

Re:Keyword: Android

[beelsebob]

Actually, it's not for bare nand like you find in android devices... it's for SSDs like their recently released 840 series.

Re:Keyword: Android

[Anonymous Coward]

The "Smart" crap in my otherwise awesome Samsung TV is inconsistent and very hit and miss. I have a full qwerty remote that some of the built in apps don't even take input from (forcing me to use arrows and an on screen keyboard.)

I bought an Apple TV which provides similar but far superior functionality to the apps built into the TV set and I couldn't be happier with it.

If I had known how bad the smart junk in the TV would turn out to be, I would have bought a less expensive Samsung TV (one with similar visual quality but none of the Internet ready features) and just bought the Apple TV from the start.

Re:Keyword: Android

[PRMan]

I agreed with this in the past, but the new Samsung plasma I just bought has a great, easy to use menu. I was able to understand what everything did without looking in a manual.

Re:Keyword: Android

[amiga3D]

I've been buying mostly Samsung electronics for a while now. I find them to be superior for the money. Now I have another reason to buy them.

Re:Keyword: Android

[gQuigs]

Are you sure they weren't Linux based? A lot of the TVs from Sony and are also Linux based.

Just curious as to what your last 3 TVs were...

Re:Keyword: Android

[Anonymous Coward]

It does make me wish Apple would release a TV so Samsung'd finally get on the ball.

I am sure it would really shine. Especially if they design the UI with purple haze as a theme!

Re:Keyword: Android

[AvitarX]

The menu was sane, it was the remote that offended me.

I also have a blue ray player, when I turn it on, it presses play automatically. If a disk is at the start, I have to watch all the unskipable ads before I can hit stop, then menu, to browse to Netflix.

Also, the 2 internet functions are behind an internet explorer icon, neither is a browser (Pandora ands Netflix ).

The tiny buttons with illegible even smaller print in-between them, with 4 inexplicable colored buttons on the remote are the biggest hi wtf though.

Re:Keyword: Android

[Anonymous Coward]

A lot of embedded programming is Linux these days. Car entertainment systems, for example.
And it needs to be Linux. Microsoft flushed their Windows CE down the shitter and programmers can't stand real Embedded OSes anymore.
Sure the UIs all suck, but it is more a problem of letting the hardware design guys loose with photoshop and presenting that as specifications for the software contractors to implement.

Re:Keyword: Android

[21mhz]

My Samsung TV menu keeps freezing randomly if I switch the interface language to Russian.
Say what you will about Linux, but gettext isn't that incompetently programmed. This must be the work of the legions of Samsung's coders.

Re:Keyword: Android

[MobileTatsu-NJG]

Man, I even said 'kidding'.

Re:Keyword: Android

[Anonymous Coward]

"Heck, Google only has a Gold membership, and we know they like Linux. Samsung is in elite territory with corporations like IBM and Intel."

The only reasons Google likes linux is because it's free, they have access to the source and it's a good base to start modifications from.

Linux is just a tool for Google to achieve their business goals, if a better tool comes along, they would use it instead. I guarantee it.

Re:Keyword: Android

[stoatwblr]

If you think TV engineers write bad UIs, you should see the ones rocket scientists cobble together. Seriously, a PhD in astrophysics deoesn't mean there are any programming skills whatsoever, let alone any appreciation of ergonomics.

Re:Keyword: Android

[Anonymous Coward]

Well, yeah. Think of the guy sitting in it.

Better hurry to the Patent Office

[bobthesungeek76036]

Glad to see proof that Samsung does innovate and not steal everything from AAPL like all Apple Fanboys think.

Re:Better hurry to the Patent Office

[Anonymous Coward]

Glad to see proof that Samsung does innovate and not steal everything from AAPL like all Apple Fanboys think.

What are you talking about? This is clearly a copy of Apple's original filesystem concept THAT THEY INVENTED when they created HFS! Why doesn't Samdung ACTUALLY innovate and find a new way to store data on a collection of sectors instead of just copying Apple all the time.

The worst part is that Samdung didn't also copy the MARVELOUS AND CLEARLY CORRECT INVENTION of hiding the filesystem (which Apple invented) from the users. They're so far behind Apple that they can only BLATANTLY STEAL the easy parts!

Re:Better hurry to the Patent Office

[Anonymous Coward]

I think your satire is a little too believable. It actually comes off as more reasonable than many of the pro-Apple arguments in the "Why Steve Jobs is a Saint" threads.

Re:Better hurry to the Patent Office

[Anonymous Coward]

Apple created the LFS, Litigation File System. The unique innovation looks ahead for a user copying a file from one directory to another, blocks the request, and transfers the operation to a county in Texas to be tied up in I/O for years.

Re:Better hurry to the Patent Office

[Anonymous Coward]

agree with you I do.

Re:Better hurry to the Patent Office

[Anonymous Coward]

I don't know whether to laugh or cry at that comment. I only hope youre being sarcastic. Because such lack of intelligence is unheard of.

Remember when this was unthinkable?

[Anonymous Coward]

Commercial hardware companies contributing to open-source and the kernel, I mean.

It’s nice to see that Linux and the open-source philosophy more and more just is generally accepted.

Let's hope it.s because they have seen the advantages of humans working together, helping each other out... and not just for nefarious dog-eat-dog (aka capitalist aka "free market" aka law of the jungle*) purposes.

* Don’t worry. I know they're not supposed to be the same. The point I want to make, is that nowadays it gets all used to describe the same thing.

Re:Remember when this was unthinkable?

[Xtifr]

No, I don't. I remember when it was rare, but not when it was unthinkable. Even if you mean copyleft as opposed to merely open-source (there was and is a lot more reluctance about copyleft), commercial hardware companies were contributing to the GNU project even before the Linux kernel sprang into existence. GCC has always had the backing of hardware companies. The GCC Ada backend was fully funded by commercial companies several years before Linus went public with his experimental kernel.

Heck, some companies even recognized that the GPL protected their own code, even before Linux appeared. The GPL'd versions of Ghostscript existed because Aladdin recognized that the GPL prevented others from taking unfair advantage of their code, while still allowing the community to contribute.

Re:Remember when this was unthinkable?

[Jonner]

This is a good thing, but corporations contributing to Free Software projects has been business as usual for over a decade now. Generally, they do so because they correctly perceive that cooperation is more beneficial to their respective bottom lines than keeping everything secret. Even Oracle, a corporation with a clear history of hostility to Free and Open Source software has supported development of the Btrfs Linux file system for many years. Cooperation between competing corporations is nothing new. Observe how Apple continues to buy essential components like CPUs from Samsung while both continually beat on each other in court.

Re:Remember when this was unthinkable?

[Anonymous Coward]

I do remember when it was unthinkable. When even writing a driver for Linux was like a taboo, never to even dream about.

Not 100% all companies. But by far and wide pretty much all of them.

For samsung flash firmware

[Anonymous Coward]

Maybe they can handle the case when multiple databases
are updated and the flash chip firmware goes into seconds of
Internal processing and reorganization.
(somehow it works worse with other brands of flash)

exactly

[manicpop]

That's the beauty of the open source model. People and businesses contribute things that benefit them directly, but they benefit everyone indirectly. Large companies don't contribute to the Linux kernel to be nice guys, they generally contribute code and patches to benefit their own products and systems. Their contributions benefit everyone, however.

Re:exactly

[Anonymous Coward]

it's the only trickle down model that actually works.

Re:exactly

[ZeroSumHappiness]

Because it's legally required to trickle down perhaps?

Re:exactly

[amiga3D]

Sadly I have no mod points for your insightful post.

Re:exactly

[DragonWriter]

Because it's legally required to trickle down perhaps?

Well, no, because it works in open source even outside of the copyleft world, and its only required in the copyleft world.

Copyleft probably was critical in establishing the benefits of big interests participating in the open source world rather than locking everything up, to be sure, but once it was established there's been quite a lot of stuff that has come down to the public in open source form even when no legal mandate existed.

Re:Samsung Innovating?

[future assassin]

Yah on ways to destroy Apple.

Re:Samsung Innovating?

[MobileTatsu-NJG]

Watch out, Apple.. we have a FILE SYSTEM!!!

And Samsung sends out a big FU

[Anonymous Coward]

to Cupertino and Redmond.

Re:And Samsung sends out a big FU

[NatasRevol]

who shrug and say, hey we already invented filesystems decades ago. With lots of patents around them.

Innovation

[PRMan]

Hope Apple wasn't planning on using this in their iPads...

Re:Innovation

[Anonymous Coward]

They still got a few years to come up with something similar, patent and then enforce it in texan courtrooms...

Re:Innovation

[Dupple]

I'm sure apple weren't. Why would they?

However, the MS lawyers will be working all weekend to find ways of recovering potentially lost licence fees...

Re:Innovation

[Anonymous Coward]

I find it amusing that you're slaggin' on Apple when it's most like Microsoft patents that Samsung is attempting to get away from. But, hey, the cool thing to do to get modded up is to insult Apple so keep at it.

Re:Innovation

[romiz]

Apple is rich enough to skip eMMC based memory for its iDevices, so it does not necessarily need this kind of file system. The NAND or eMMC trade-off is 'Spend (a lot of) money once to write your own FTL, and adapt it for each new chip' or 'Buy a chip with hardware FTL and a standard interface for a higher price".

You can check the tear-downs for all Apple devices: all of them directly use NAND, which makes sense. Apple buys large enough numbers of Flash to have reliable sources, and can invest the money to write a FTL.

Re:Innovation

[CannonballHead]

Apple doesn't use filesystems. ;)

Re:Innovation

[LodCrappo]

So what if they were? They can still invent it later if they want to.

Headline reads F2Fs

[rossdee]

Hands up all those who saw the liower case s at the end of F2Fs and thought that it meant it was a plural of F2F which is presumably a fighter plane made by Grumman

Re:Headline reads F2Fs

[Anonymous Coward]

Bueller? ... Bueller?

It's not for bare NAND

[bogolisk]

No. SSDs present themselves to the OS as contiguous block devices. Filesystems intended for bare NAND flash like jffs(2), yaffs, and this new F2Fs would be totally useless for SSDs. They're intended for bare NAND, which SSDs are not.

You're wrong

f2fs work on top of block devices. f2fs sends TRIM (ATA command) down to the device. Bare NAND flash doesn't grok ATA commands.

Re:It's not for bare NAND

[Anonymous Coward]

If this is the case, then I don't see the point. Filesystems already in use support TRIM.

Re:It's not for bare NAND

[Paul Jakma]

You appear to be correct, judging by the information in the patch (e.g. see the LWN link posted by someone else, later in these comments). (commenting mostly because I accidently modded you as 'troll' when I'd meant to click 'informative' and this is the only way to undo that).

Re:It's not for bare NAND

[romiz]

The problem is that even with a translation layer for block access, flash-based devices have limitations, which means that different usage patterns can dramatically change the performance of the device.

For a (simplified) example, to write a file in ext3, you need to store the new data for the file, but you also need to store other metadata: the location of the data blocks themselves in the inode, the file size in the directory, the journaling data. This means that you have four 'internal block descriptors' open for writing at the same time.

But block descriptors are a limited resource in SSDs, and even more so for low-cost eMMC devices. This means that with only two or three open files with regular writing, you could quite easily lead to some kind of thrashing state, with the device quickly opening and closing descriptors. Since flash memory writing is strongly constrained, this means that a whole block (2 MiB block size is common) containing a descriptor will need to be erased before its next use. As a result, each block only contains little interesting data, and writing only a small amount of data leads to a lot of flash write and erase access. This problem is called write amplification, and reduces both the disk's performance and its durability.

The F2FS design is a log-based design, where all files on the disk share 6 common writing areas, for each kind of stored data, where the information is stored as it arrives. This will have a very positive effect against the write amplification problem, and is an example of how an adapted file system can have a positive impact, even on block-based devices.

Re:It's not for bare NAND

[bogolisk]

If this is the case, then I don't see the point. Filesystems already in use support TRIM.

Just because you send TRIM down it doesn't mean that the device can erase the block. The erase block size in NAND is usually 256KB or larger. Using 256KB as IO block is just crazy, drivers use something like 16KB or 32KB. The filesystem has to be aware of the erase block size so it can send down TRIM command for an aligned and contiguous 256KB block, then the device can go on and erase it.

Anything better than FAT

[epSos-de]

I would argue that anything is better than FAT for the NAND storage. Under FAT for example: Taking a picture with the FAT-formated camera can corrupt non-related files for some reason, if the camera is low on power. Entire music collections can get lost, when the battery in the FAT-formated phone went out during the write process. Anything better than FAT.

Re:Anything better than FAT

[h4rr4r]

If we did not have patents and Windows as the dominate desktop we would not have to deal with FAT in 2012. Either your camera uses that or the vendor is stuck with paying microsoft more or trying to get users to install a driver for a proper filesystem. All those options are pretty bad.

The cool things of this is...

[Anonymous Coward]

...that iApple is forbidden from using/(stealing) this code.

when you use GPL, you're driving with Stalin

[Thud457]

Watch out, Apple.. we have a FILE SYSTEM!!!

You know who else had a filesystem?...

NAND Flash File Systems

[Anonymous Coward]

They have added necessary functionality no one in the Linux world gave a shit about implementing. This was a huge hole in competing with Apple and Microsoft in mobile devices.

Re:Samsung Innovating?

[Anonymous Coward]

So far, they're about $1,000,000,000 in the hole...

I wonder how it is to be used

[erroneus]

My first reaction was "Is this to replace FAT?" Then I read about "log files" and I wonder if this is essentially a file system for more efficient logging. The article and the mailing list message seem to be somewhat empty in that regard. "Hey, there's a new file system..." That's about the size of it. So what's it about?

It would be interesting if there were an improvement to FAT and it somehow ended up as an alternative in consumer devices... but then again, how to get it onto Windows machines? "driver software" I suppose...

No, the more I think about it, the more I believe it's a limited purpose file system intended to improve something about internal record keeping in embedded Linux devices.

Re:I wonder how it is to be used

[Jerome H]

You can find a lot of details in the txt file: https://lwn.net/Articles/518719/

Re:I wonder how it is to be used

[Lordrashmi]

I'm not a filesystem expert, but this is what I believe they are referring too: http://en.wikipedia.org/wiki/Log-structured_file_system

Re:I wonder how it is to be used

[Anonymous Coward]

It is a replacement for the default Linux file system, which doesn't know how to deal with the problems that flash devices, like SD present when used as a SSD.

Re:I wonder how it is to be used

[Anonymous Coward]

http://en.wikipedia.org/wiki/Log-structured_file_system

Re:I wonder how it is to be used

[shutdown -p now]

This cannot replace FAT, since the whole point of FAT is to be interoperable with all those Windows machines out there. For as long as Windows only understands FAT and NTFS on removable devices, any consumer device will use those (and specifically FAT, for certain other reasons) in any of its memory that is directly exposed to be mounted as a block device.

On the other hand, for internal device memory, Android has already moved to a high-level protocol (MTP) to expose that to PCs, so they don't care what file system backs it internally. I haven't checked, but I'd expect that any 4.x device has its internal memory fully in ext4 or other Linux native FS already.

Re:I wonder how it is to be used

[Anonymous Coward]

My first reaction was "Is this to replace FAT?"

Mine was "they're trying to prevent exFAT from getting a foothold". With flash memory in various mobile storage media (tablets, smartphones, flash drives, memory cards) getting as prevalent as it is, interoperability with different OS's becomes very important. To my knowledge, FAT is currently the only file system that works out of the box on all the major systems (Windows, MacOS, Android, Linux, probably Windows Phone). And it's outlived its usefulness. I'm guessing Microsoft has been pushing its own proprietary exFAT as a replacement for a while now. It's already been accepted as the standard for SDXC cards. Samsung may be trying to avoid future licensing costs and/or getting under MS's thumb. I'm glad they're taking this route.

Re:I wonder how it is to be used

[RobbieCrash]

The move to MTP is something I've been speculating has to do with moving away from the FAT patent licensing issues that Microsoft is using to bilk Android manufacturers. I find it super annoying to use, since it isn't treated like a block level device Python won't interact with it, I can't read it in anything like Baobab, so I tend to lose track of what's occupying what space.

Re:I wonder how it is to be used

[shutdown -p now]

The move to MTP is something I've been speculating has to do with moving away from the FAT patent licensing issues that Microsoft is using to bilk Android manufacturers

There may be that angle to it, but I think the main reason is because it removes the requirement for the phone to unmount the device to allow the PC to mount it - this can wreck havoc on any app that has files on the unmounted partitions open, requiring them to be aware of this scenario (and for many apps that pretty much means that they have to shut down - e.g. mapping apps that store their map cache there), and it also means that manufacturers have to carve out two partitions - one for the OS and other non-PC-mountable data, the other one for user data - and decide on how large each one should be, which they often get wrong.

You don't need it to be treated as a block device to get your OS (and Python etc) to access it as a normal filesystem. You just need a filesystem driver that wraps MTP. Since you're on Linux, you can just use mtpfs or go-mtfps with FUSE, and get the best experience of any desktop OS out there.

Re:I wonder how it is to be used

[RobbieCrash]

Thanks for pointing me to mtpfs, which solves Linux at home! But not Windows at work or Windows on laptop. I didn't even think of taking that approach to it,maybe there's a Windows equivalent...

I tried using PyMTP, but I can't get it to work on Windows, so that's my real gripe.

Re:I wonder how it is to be used

[shutdown -p now]

Windows has some built-in support for MTP (at least since Vista), though it's not on FS level - instead, it's hooked up into Shell/Explorer, much like Libraries in 7. So if you're content with Explorer, or some file manager that's using Shell interfaces, then it should just work.

For automation, when you actually want to see it as FS, yeah, it's a mess. Ideally a driver-source-compatible port of FUSE would solve this, and people have made several attempts at FUSE-Win32, but apparently writing Win32 FS drivers is not for the faint of heart, so I don't know of anything stable.

Re:I wonder how it is to be used

[Anonymous Coward]

I think you are confusing a log, with a Log-structured file system or a journaling file system. These are among the types of file systems currently used in Linux.

See http://en.wikipedia.org/wiki/Log-structured_file_system
And http://en.wikipedia.org/wiki/Journaling_file_system

Re:I wonder how it is to be used

[Anonymous Coward]

The move to MTP is something I've been speculating has to do with moving away from the FAT patent licensing issues that Microsoft is using to bilk Android manufacturers. I find it super annoying to use, since it isn't treated like a block level device Python won't interact with it, I can't read it in anything like Baobab, so I tend to lose track of what's occupying what space.

Install a file server on the phone. Game over.
You can upload and download files from a standard ftp client, and those exist on basically all operating systems. No dealing with MTP or other shit like that.

Re:I wonder how it is to be used

[cbhacking]

I'm actually one of those people you mention in the second paragraph. The real problem isn't Win32, it's that the filesystem stack is below Win32. Win32 is a subsystem layer that sits on top of NT, preserving compatibility with Win9x apps and presenting a similar user experience. Direct NT programming is a different beast in many ways, even when you stay in user-mode. Kernel-mode NT is just another level of fun on top of that. It's certainly possible, of course, but the overlap of people who are interested in FUSE and who are sufficiently knowledgeable to write an NT driver which lives in the storage stack but can be accessed like a device driver (i.e. can be accessed when there's no filesystem using it mounted) is... small.

That said, the project I worked on was three undergrads for three months, and by the end of it we had a userspace component that was basically compatible with any FUSE driver you could compile with MinGW, a driver that you could load and call IOCTLs on, and support for maybe four filesystem operations (list directory contents, at least half of stat, open, file and read file). It even didn't usually crash until you tried to unload the driver (or shut down the machine; for obvious reasons we used virtual machines to test on). I still have the code for the various components somewhere; if I find myself in need of a new hobby project at some point (no time soon, but maybe eventually...) I may dig it out and have another go at the project. I really would like to see it succeed.

Re:I wonder how it is to be used

[shutdown -p now]

Was that an open source project / can it still be downloaded somewhere?

Re:I wonder how it is to be used

[21mhz]

Ironically, it was Microsoft who developed MTP and pushed it through the standards committee.
And now they extended it for Windows Phone, so regular MTP support is not enough to work with WP devices. This is perhaps the stupidest move in the whole Windows Phone story.

Re:I wonder how it is to be used

[Anonymous Coward]

All it requires is some drivers for Winbloze surely? I can't believe so many here are discussing the technobabble & not seeing the big picture FFS - the end of MS FAT extortion on Androids woo hoo!

Kudos to Sam for sticking it to the Msman! Now, where do I d/l the drivers for my S2...? :)))

Re:I wonder how it is to be used

[Anonymous Coward]

MTP seems badly-broken. For example, while writing one large file, you cannot also read and readdir: it's one operation at a time. IMHO emulating a USB network interface with AFP / SMB servers on it would have been smarter. Since linux supports multiple routing tables now, this could be done separately from the phone's Internet stack. It could also be done with link-local addresses, which both IPv4 and IPv6 support. There's much less to fix with this approach.

Re:I wonder how it is to be used

[Anonymous Coward]

I wish Android wouldn't be so freakin' stubborn about FS choice on your external/SD-card/internal storage partitions. FAT/FAT32 is a pile of crap, it's terribly slow, outdated, and has that nasty 4GB limit that impedes HD video and experimental filesystem images (for running native Linux and such). I can't believe anyone would still recomment that awful FS but Android basically requires it whether you need its compatibility or not. I formatted my external SD card ext3 and it doesn't work on Android. I can manually mount it on the terminal, but then permissions get screwed up as Android abuses the permission system a ton for sandboxing (not what the permission system was designed for!). NTFS would work better, but the kernel driver is read-only, which sucks. I would love them to get ntfs-3g working so we can use NTFS, have interoperability with Windows and Linux as mass storage, and support large files. Either that, or make a new filesystem and actually push for integration rather than just keep spamming new FS'es that only work on Linux (we have tons of filesystems on Linux, none of which are cross-compatible besides ext2/3/4 due to projects to port it).

MTP/Camera is a terribly dumb way to handle it, as both of those protocols are crap. They're slow, they don't give low-level access, they aren't treated as real drives on most OS'es, they are incredibly limited, and they cannot replace mass storage. Why not give the user a choice (FAT for max compatibility, NTFS for Windows+large files, ext3 for Linux/openness with ext2fsd as an option for Windows).

They don't need it

[Anonymous Coward]

since their hardware/software probably already deals directly with NAND flash chips inside the SOC.

Re:Too late, I already patent my FSF

[jones_supa]

Actually you should say "Samsung-ssi" or "Samsung sunsengnim" in Korean. The -san is a Japanese thing. :)

Re:Too late, I already patent my FSF

[Anonymous Coward]

Um, Samsung is a Korean company. -san is not the right way to refer to them.

Interesting....

[Anonymous Coward]

but how do you make a file system look like an iPhone?

The funny thing of this is...

[Anonymous Coward]

that you think they care.

When they really, really, don't. They solved this issue before the first iPhone came out.

Of COURSE they did

[Anonymous Coward]

and slowly the Linux world catches up. The next thing will be vendors being able to deal with raw NAND in software instead
of having to buy ATA controller chips. The thing is, this is such a HUGE market that portable device development is starting to
directly influence how Linux will evolve. Power management in all of its forms is another big can of worms.

Re:Of COURSE they did

[Anonymous Coward]

In Linux Land, we've had that for years. How's that Windows progress going for you, Mr. Neanderthal?

RFS

[thegarbz]

While the primary benefit will initially be for Android devices, this will be great news for solid state drives as well. Great job Samsung!

Before you go congratulating them on a great job, remember this is the second time they did this. The original attempt was called Robust File System. It was an abortion based on FAT16/32 with a duplicated file allocation table and some sort of journalling hacked on top.

It was claimed to be optimised for NAND devices and all that other good stuff, but the community quickly came to rename it Really Fucking Slow.

This file system was so slow that on the original Galaxy S the kernel would think software locked up while writing to the disk and prompt the user to force close the device. Search for "lagfix" if you're interested in what a disaster this was. There were users world wide trying to find fixes for the slow system performance, and the fix was often in the form of a kernel which supported ext4 or yaffs and a utility which converted the entire /system and /data partitions in the phone to the more common file systems.

I don't have high hopes in Samsung's competence here.

Re:RFS

[SirJorgelOfBorgel]

First thing I thought as well. We'll see how it goes - this one doesn't sound as stupid as the previous one, but anybody who knows Samsung knows that they are very weak in the software performance department ...

(I wrote some of those utilities, btw)

Re:RFS

[Anonymous Coward]

My thoughts exactly. Very very weak. They tend to use their early adopters as beta testers. On the first galaxy i7500, you could seriously improve responsiveness by recompiling the touchscreen module so it didn't log every pointer event.

Back when the first Samsung phone with rfs arrived, I sort of reverse engineered it and made (probably the first?) rudimentary rfs to fat converter. That system simply divided the underlying fat system into lots of moveable little equally sized blocks, with a header directly before each block containing the block number(and much more that I didn't bother to understand). Didn't look very elegant or fast in any possible way...

Re:RFS

[thegarbz]

(I wrote some of those utilities, btw)

Then let me just say thankyou for your hard work. It's quite amazing when communities and hackers can create a better product than the companies who actually have access to the full specifications and sources.

Re:RFS

[david.given]

Before you go congratulating them on a great job, remember this is the second time they did this. The original attempt was called Robust File System. It was an abortion based on FAT16/32 with a duplicated file allocation table and some sort of journalling hacked on top.

*twitches*

Damn you, thegarbz! Damn you to heck! I had nearly managed to forget about RFS today before you reminded me. Now I'm going to have to attempt to wipe my memory with methylated spirits and Friends reruns. Again.

...I have it on dubiously reliable authority that the utter disaster that was RFS was one of the reasons why Google clamped down so hard on what modifications vendors could make to Android and still get the official Google apps. Which is why most modern Android devices are all so similar.

Re:RFS

[stoatwblr]

Nothing to add, except "Thanks for the utilities"

Rehash

[Bozovision]

It looks to me like most the problems they are solving have already been solved. There are already several open source log-structured file systems. This list excludes experimental and similar software from educational institutions:
      - Yaffs - http://www.yaffs.net/ - designed from the ground up for NAND
      - JFFS2 - http://sourceware.org/jffs2/jffs2-html/jffs2-html.html - ditto.
      - NANDFS - http://wiki.freebsd.org/NAND - BSD style licence

Plus there's Ext4 - which is used in Android now - not designed for NAND, but seems to work ok.

This work by Samsung fixes the problems with their previous file system. It's good, but it's not unique. Good PR though.

Re:Rehash

[Anonymous Coward]

and for plain block devices like eMMC/SSD with built-in blackbox FTL, there is DoCoMo's nilfs. DoCoMo is not "weak in the software development department."

There are a few newer NAND filesystems, one of which is ubifs that a lot of SoC vendors seem to be contriibuting to. A key feature is that the filesystem has absorbed partitioning: with "level 1" understanding of the filesystem, appropriate for a boot rom, you can find the ubifs partitions on a device. Then you need "level 2" understanding to see the files inside them. This way, *hopefully*, the filesystem can evolove in not-backwards-compatible ways at level 2, while the older code linked into the bootrom still works well enough at level 1 to read the nvram variables or whatever.

I think there may be others than ubifs, though, maybe better ones. JFFS2 is not really serious because it falls over with normal-sized devices, like >1GByte, which is probably why android uses yaffs.

There are really a lot of things one must support to replace the FTL. For example Erase operations take a long time and can be suspended to do a read/write, then return to the Erase, but if you lose power you'd better remember to finish the Erase. It's worth doing this IMVHO because then you can raise the performance / reliability of the bottom-of-the-barrel parts. Some of them have latency blips or corruption problems when powering down or "sleeping".

Google like leeching from Linux

[21mhz]

Heck, Google only has a Gold membership, and we know they like Linux.

Surely they like to take and modify, but playing along and contributing is another matter.

Re:Google like leeching from Linux

[Patch86]

They donate at least $100,000 to the Linux Foundation a year, if nothing else. Pocket money to Google, maybe. But No small chunk of change to the Linux Foundation. Creating and releasing an extremely popular and novel Linux-based OS has got to count for something, too.

Samsung donates at least $500,000 a year, so they do still win in "we love Linux" top trumps.

Re:Google like leeching from Linux

[21mhz]

Creating and releasing an extremely popular and novel Linux-based OS has got to count for something, too.

Forking the Linux kernel to practically derail driver development for mobile devices counts for something as well.
Here Samsung and Intel also serve as counterweights by running their own Tizen effort, which even sees some automotive use with Genivi.
But the story of "Linux on mobile", between Google and Nokia, is a sad and confused one.

They should use

[partyguerrilla]

ReiserFS instead. I heard it's killer.

Re:They should use

[BatGnat]

too soon

Re:They should use

[PuZZleDucK]

That is comedy gold my man... everyone else obviously does not get it :D and I obviously have no mod points.

Right...

[Anonymous Coward]

because if someone is reeeally good at filesystems for Android, it's Samsung. The Galaxy S had the best IO performance ever.

Orsum

[Anonymous Coward]

A new FS!?

Just what the world needs right now!

captcha=frenzied

Gangnam Style?

[Anonymous Coward]

I would actually support Samsung on this, if they provided a Gangnam Style lock screen app with the horsie dance cartoon character version of Psy. But sadly, if F2FS is anywhere near as bad as RFS, then the song lyrics may be somewhat applicable...

Great new thing from Samsung!

[rleesBSD]

Um, yeah - but will Apple sue me?