Slashdot Mirror
Apple Introduces New File System AFPS With Tons Of 'Solid' Features (apple.com)
On the sidelines of its Worldwide Developer's Conference, Apple also quietly unveiled a new file system dubbed APFS (Apple File System). Here's how the company describes it: HFS+ and its predecessor HFS are more than 30 years old. These file systems were developed in an era of floppy disks and spinning hard drives, where file sizes were calculated in kilobytes or megabytes. Today, solid-state drives store millions of files, accounting for gigabytes or terabytes of data. There is now also a greater importance placed on keeping sensitive information secure and safe from prying eyes. A new file system is needed to meet the current needs of Apple products, and support new technologies for decades to come.Ars Technica dived into the documentation to find that APFS comes with a range of "solid" features including support for 64-bit inode numbering, and improved granularity of object time-stamping. "APFS supports nanosecond time stamp granularity rather than the 1-second time stamp granularity in HFS+." It also supports copy-on-write metadata scheme which aims to ensure that file system commits and writes to the file system journal stay in sync even if "something happens during the write -- like if the system loses power." The new file system offers an improvement over Apple's previous full-disk encryption File Vault application. It also features Snapshots (that lets you throw off a read-only instant of a file system at any given point in time), and Clones. According to the documentation, APFS can create file or directory clones -- and like a proper next-generation file system, it does so instantly, rather than having to wait for data to be copied. From the report: Also interesting is the concept of "space sharing," where multiple volumes can be created out of the same chunk of underlying physical space. This sounds on first glance a lot like enterprise-style thin provisioning, where you can do things like create four 1TB volumes on a single 1TB disk, and each volume grows as space is added to it. You can add physical storage to keep up with the volume's growth without having to resize the logical volume.As the documentation notes, things are in early stage, so it might take a while before AFPS becomes available to general users.
This new filesystem should become stable in about 2028.
or did they slap their logo onto something existing?
Because considering how long file system code takes to mature these days, we could be in for some long wait...
The acronym "AFPS" is incorrectly used in both the summary and the headline. Dylexia, perhaps?
C'mon, it's 2016. Where is compression?
We've had filesystems with those features for a long time. Why re-invent the wheel?
It's a hard job. We're into year fifteen of ZFS and it's just starting to gain some features that make administration of it manageable by non-experts. Give it another five before you want to make it your default on a desktop for grandma. BTRFS will be along five years after that.
If Apple can pull off something similar in a couple years, it will be a major triumph. It's too bad for everybody that Steve got bitchy at Jonathan and the community hasn't had Apple's help as a contributor for the past decade.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I'm glad Apple has introduced this. As of now, the snapshot API and others are not present, but now Apple is on parity with everyone else in the industry.
APFS isn't like ZFS or btrfs, but more like ReFS in the fact that it still requires a logical volume manager. It would be nice if it had RAID, but that is a minor item, compared to just getting rid of HFS+, which just had to be killed.
Some features I like:
The ability to encrypt volumes with multiple volume keys. It looks like it will be similar to Oracle's ZFS on Solaris, but the implementation can be completely different.
Snapshots. Something like zfs send and zfs send -i will be quite useful for backups.
Copy-on-write capability, which is useful for VMs.
Of course, it appears that Apple will be documenting and publishing the FS's specs in 2017, which will be even more useful for compatibility.
All and all, even though there is no RAID 5/RAID-Z, or LVM replacement, this is a heck of a lot better than what OS X/macOS has now.
Sure, it's possible, but what does it mean across an enterprise if you don't have a Stratum 0 clock for everyone?
Compression? why - to compress all of those mp3 and mp4 video files? Or your TXT docs?
I was thinking that a current issue is the crypto-ransom stuff and that a FS needs to version on-demand. Sure everyone is *supposed* to have backups. I don't know what the Mac world is like but most PC folks I know do a file-copy to a USB drive (if they do anything at all). I'm not talking about what smart IT folks do - referring instead to general users.
How many people have a time machine? (and is that good enough?)
Swift is stable. Swift 2 has provided a very solid foundation that thousands upon thousands of developers have used to create many real-world applications with.
Yes, there is Swift 3 in the works. What, are you really suggesting that Apple shouldn't try to improve Swift? Are you saying that they should let it stagnate, just to meet your strange definition of "stable"?
Even when Swift 3 is released it doesn't mean you won't be able to use Swift 2 any longer. If you've got old Swift 2 code, then keep on using the Swift 2 compiler and toolset!
But if you want the benefits that Swift 3 brings, then by all means you'll be able to use it, too. Will you have to make some changes to your existing Swift 2 code to get it to work with Swift 3? Perhaps! But that's a perfectly reasonable cost if you want to make use of newer functionality.
You sound like one of those fools who moans about Python 2 versus Python 3, not realizing that it has actually gone very smoothly. Python 2 users weren't forced to upgrade when they didn't want to, and in fact they'll be able to use Python 2 for years to come! Python 3 was able to bring some serious improvements to the table. Users who wanted to use Python 3 had the freedom to do so. The major libraries all supported Python 3 very quickly, and the only ones that didn't are niche libraries that few users care about.
Maybe you have some personal vendetta against Swift. Maybe you're a Rust supporter who has seen his language flail miserably and then fail miserably, while Swift gets better and better. Regardless of why you don't like Swift, the fact remains that Swift 2 is stable, Swift 3 will soon be stable, and despite all of your bitching and moaning about this "instability" that doesn't exist there will be thousands, if not millions, of developers using Swift to accomplish great things.
With ransomware on the rise, having a filesystem that can take snapshots, perhaps coupled with a version of Time Machine that works on snapshots will help provide some mitigation. If the ransomware doesn't have root, it can't purge snapshots, although it can do mayhem in other places.
I would say Time Machine is OK for an "oh shit" backup for bare metal restores, but I wouldn't really rely on it as my sole way to retrieve data, because I've had instances where TM backups got hopelessly corrupted. I would probably recommend TM + Borg Backup or another utility, ideally a utility that can pull the data, so ransomware cannot get near the backup repository to destroy any existing data. Barring that, there is always Carbonite or Mozy, but one has to be aware of the security implications of dumping to the cloud.
ZFS is under CDDL and would not even need to be "licensed" in the usual sense — it is free for anybody to take. "Too free" for certain zealots, in fact, which is why it was not part of Linux kernel for a while — until the supposed "license incompatibility" myths got debunked.
Even Linux now offers ZFS — Apple would've had a much easier time porting it, because MacOS is already FreeBSD-based and the FreeBSD-project had ZFS available "out of the box" for several major releases spanning many years.
What did Apple find lacking about ZFS, that would justify creating their own, is, indeed, a mystery. Probably, a case of the Not Invented Here Syndrome. Sad...
In Soviet Washington the swamp drains you.
I was hoping Apple would license ZFS or even Veritas Volume Manager/Veritas FS from Symantec.
I thought Veritas was also called Online Journaled File System (OnlineJFS or OJFS). What else is OJFS?
...what has been done on some other platforms. A quick look at the feature list remind me of some of the features which have been available in the NetApp file system for quite some time. And that is not meant in a negative way, Good job and good luck.
How about letting users unplug removable media without having to eject it first like every other OS has had for about a decade.
another closed source OS filesystem is not what the world wants or needs, regardless of just how many "solid" features you think are in it. Take a page from microsoft and spare the countless BSD and Linux ussers mounting flaky FUSE partitions for android developers and MPD partitions for our mp3 players. in a decade or two when you get tired of the filesystem, and inevitably abandon it for whatever white rabbit the board of directors is chasing in the never ending story of capitalism, im sure new versions of APFS will exist for linux and BSD admins such as goddamn APFS, and fucking APFS to torture us long after your soiree is completed..
Good people go to bed earlier.
Aye. Whereas almost all of Microsoft's filesystem advances are hidden in shadow-copies and inaccessible system folders. Or enterprise-version only RAID-like features. We can't even freaking tag files n folders unless they are "media" files.
You have inordinately cheap disk
Because of Apple's tendency to solder the SSD to the mainboard in the Mac Pro and all current MacBook laptops other than the non-Retina MBP, an upgrade requires replacing the whole computer at a substantial cost. Only external storage is "inordinately cheap" on a Mac, and not all laptop use cases make external spinning rust practical.
Sure, you could find lots of value in compression.... and you can get it with file compression utilities.
That's fine, so long as these utilities can let the user mount an archive read-only as a folder and thereby let other applications see the archive's contents as files in as a folder. Does macOS Sierra introduce anything that interferes with OSXFUSE?
If I didn't know any better, it sounds like Apple might be gearing up to offer some sort of in-house virtualization, with this new filesystem laying the foundation for it.
Because I don't see that feature in any of the information.
Mac users probably either do nothing or have an automated incremental backup system ("Time Machine") to an external network drive ("Time Capsule").
.. case sensitive filenames by default? :D
Just wondering. I know HFS+ can have case-sensitivity, but not sure if it is on by default. And some people seem to be discouraging that, based on quick googling.
All I want from Apple when it comes to file systems is for them to unbind Photos from HFS+ . There is no reason an app should be as closely bound to a file system type to the point that the app is unusable on anything other than that specific file system. I'm referring to the storing of photos. Photos will only store photos on an HFS+ file system and will corrupt your data if you try to use anything else.
Will non-Apple OSes be able to mount this new FS? Apple's track record is not great on this; its a shame to see the question not even addressed in the couple of mentions of AFPS I have seen.
HFS may be thirty years old but we still have major headaches transferring files between Macs and other machines. I truly believe that Apple would be better served if they invested in a open filesystem format.
For starters, number 1 we're really logical thinkers and B, we value consistency.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
Long ago, there was a Stacker-like tool for classic Mac OS called TimesTwo that installed itself as a SCSI driver. There were also file-level tools called DiskDoubler, Now Compress, and StuffIt SpaceSaver that intercepted file open calls and decompressed files in the background. Files were written uncompressed, to be compressed later by the "AutoDoubler" background task.
"things are in early stage". Come back in ten years and let us know how you're doing against ext4 then. Hell, it's really hard to beat ext4 as a general purpose filesytem (don't forget: we are talking Desktop and Mobile class systems here) in terms of performance and reliability. Checkout OpenBenchmarking!
In theory, any journaled file system would tolerate surprise disconnection to the same extent it tolerates surprise power loss. The problem is that journaled file systems tend to be either proprietary or copylefted, hindering their wide adoption for removable media across all major desktop operating systems.
OK that was a pretty good one.
Your recent ones have been lagging.
Really, terabyte SSDs? Today's SSDs, in terms of storage capabilities are more like mechanical drives of 20 years ago. Yes, data centers may have large SSDs, but not users. Will average users benefit from this new file system or will things like 64bit pointers on a drive less than a gigabyte simply consume more of the drive for little benefit?
Finally, are not there already file systems available that meet whatever this new need of Apple's is that would not require the recreation of the wheel (or disk)? If the goal is secure Apple a spot in the enterprise, wouldn't a main stream file system like ZFS be more likely to achieve that goal?
Hopefully, they get rid of .DS_Store while they're at it...
ZFS is not recommended for non-ECC RAM. RAM errors can get propagated to disk by application read operations, not just writes.
http://research.cs.wisc.edu/ad...
I back up my MacBook Pro to a FreeBSD box using ZFS with compression and deduplication (and snapshot it periodically, because if TimeMachine detects that your backups are corrupted then the only option is to delete them an redo from start, and it's nice to be able to revert just one backup if the last backup broke something). With lz4 compression, the compression ratio for the ZFS filesystem that I use as a backup target is 2.08x - that's a fairly hefty saving. It's harder to measure how much dedup is saving me, because it's a pool-wide property and I have ripped a load of my DVD collection, which probably doesn't contain many duplicate blocks. Currently the dedup ratio is 1.2x, but given that the TimeMachine volume is around 20% of the total space and contains a lot more redundancy than the rest, I wouldn't be surprised if I'm saving a relatively large amount there too.
I probably get more benefit from dedup on the backup disk (TimeMachine copies entire files, even if only a single block has changed) than I would on the laptop, but I'd expect to see similar benefits from compression on the source and the backup, and that 2x compression ratio looks pretty good...
I am TheRaven on Soylent News
"APFS supports nanosecond time stamp granularity rather than the 1-second time stamp granularity in HFS+.
Damn, 1-nansecond time stamp granularity? A factor of one billion improvement in resolution, that's fairly impressive. I'm not sure it'll be of much use to a lot of people, but I'm all for greater precision/resolution in stuff like this.
Just cruising through this digital world at 33 1/3 rpm...
I don't understand why innovations like those found in BeFS (like rich metadata support) go ignored whenever someone creates a new file system. If you are going to break compatibility, you might as well add in some useful features.
Which is it? APFS or AFPS? For crying out loud.
Thanks for it, i will keep doing! Bitcoins
I was mistaken. I apologize. Thank you for clarifying. So let me correct myself:
In order to understand Apple's intent in leaving transparent compression out of a file system, we'll have to watch for what Apple chooses to solder down in its next round of hardware.
I don't understand why innovations like those found in BeFS (like rich metadata support) go ignored whenever someone creates a new file system. If you are going to break compatibility, you might as well add in some useful features.
That sad truth is that the utility provided by rich metadata support is more than counterbalanced by the headaches it causes whenever you need to copy your files anywhere outside of the filesystem that supports it. (Just ask any .sd2 audio format users about what happened to their audio files when they tried to copy their data to a FAT32-formatted USB key, or email them, or etc).
Because of compatibility/interoperability issues, programs can't rely on putting important data into the resource forks, so the feature ends up going unused. Because the feature ends up going unused, people continue to use filesystems that don't support it. Because people continue to use filesystems that don't support it, app developers can't rely on it to work. And round and round we go, forever stuck in a bleak dystopia where the only supported "metadata" is a three-letter extension to the filename. :(
I don't care if it's 90,000 hectares. That lake was not my doing.
Wow! Now Apple supports sparse files like good old Unix(TM). Did Cupertino write this themselves or did they port something from BSD?
The weird part is that this APFS apparently supports network file sharing with SMB (Microsoft NetBIOS) only, meaning no AFP (AppleTalk), NFS, AFS or the dozen of recent cluster file systems. How about FTP or SFTP?
Seems I will keep waiting for a FS that can do versioning.
I wonder if it is more of an API hindrance than a technical one (as a snapshot already has much of what you need for it).
Why can't money or development time be put towards existing open solutions and fixing them up?
I am using ZFS on my FreeNAS machine and from what I understand BTRFS has some features superior to ZFS (but also some stability issues)
It's really super frustrating as a geek that there isn't a single "be all and end all" file system out there yet. Something definitively powerful, reliable, fast.
I know there's been some improvements over the years to these file systems but damn if progress doesn't seem slow.
Also Apple creating their own? How long until it's reliable? It takes forever to make a filesystem good.
The title misspells APFS, saying AFPS instead.
Yeah, remember how you email in BeOS was just a text file and the file manager showed the "From" fields? Or how you could organize MP3's right in the filemanager? There's tons of uses for that such as organizing digital assets.
Too bad there were no applications for BeOS. Even though I recall an article I can't find saying "BeOS is on the verge of a JBloom for applications!"
Can someone fix the title?
"AFPS" -> "APFS"
What's on that list that NTFS hasn't had forever?
Anyone else notice the transposition in the title (AFPS instead of APFS)?
So use ECC RAM. By an amazing coincidence, the company announcing this software also just happens to also manufacture hardware.
ECC requires a compatible motherboard, CPU and RAM. Strangely higher end Intel CPUs, i5 and i7, tend not to support ECC.
Can you cut and paste files yet?
Wanna buy a shirt?
https://www.redbubble.com/people/stealthfinger/shop?asc=u
The BSD Now show "ZFS in the trenches" talks about how different developers have different opinions on whether you must use ECC RAM with ZFS but all recommended ECC if possible (so that part is not in debate). Around the 56:30 mark of the episode, Josh Paetzel of FreeNAS explains that if the ZFS spacemaps were corrupted due to memory errors you could be in a worse situation compared to less sophisticated filesystems that have an fsck. This is because with a corrupt spacemap ZFS would refuse to let you access any data whereas the fsck would "just" (irreversibly) decopule or delete chunks of data in an attempt to allow access to the rest. It's not a great situation - inaccessible data versus data loss and/or corruption but it is a difference.
Late to the party, but for those asking about ZFS & apple, Adam Leventha (dtrace Sun engineer)l posted an exceptionally informative post about the history of this mess earlier this week:
http://dtrace.org/blogs/ahl/20...