Slashdot Mirror
Apple Introduces New File System AFPS With Tons Of 'Solid' Features (apple.com)
On the sidelines of its Worldwide Developer's Conference, Apple also quietly unveiled a new file system dubbed APFS (Apple File System). Here's how the company describes it: HFS+ and its predecessor HFS are more than 30 years old. These file systems were developed in an era of floppy disks and spinning hard drives, where file sizes were calculated in kilobytes or megabytes. Today, solid-state drives store millions of files, accounting for gigabytes or terabytes of data. There is now also a greater importance placed on keeping sensitive information secure and safe from prying eyes. A new file system is needed to meet the current needs of Apple products, and support new technologies for decades to come.Ars Technica dived into the documentation to find that APFS comes with a range of "solid" features including support for 64-bit inode numbering, and improved granularity of object time-stamping. "APFS supports nanosecond time stamp granularity rather than the 1-second time stamp granularity in HFS+." It also supports copy-on-write metadata scheme which aims to ensure that file system commits and writes to the file system journal stay in sync even if "something happens during the write -- like if the system loses power." The new file system offers an improvement over Apple's previous full-disk encryption File Vault application. It also features Snapshots (that lets you throw off a read-only instant of a file system at any given point in time), and Clones. According to the documentation, APFS can create file or directory clones -- and like a proper next-generation file system, it does so instantly, rather than having to wait for data to be copied. From the report: Also interesting is the concept of "space sharing," where multiple volumes can be created out of the same chunk of underlying physical space. This sounds on first glance a lot like enterprise-style thin provisioning, where you can do things like create four 1TB volumes on a single 1TB disk, and each volume grows as space is added to it. You can add physical storage to keep up with the volume's growth without having to resize the logical volume.As the documentation notes, things are in early stage, so it might take a while before AFPS becomes available to general users.
This new filesystem should become stable in about 2028.
The acronym "AFPS" is incorrectly used in both the summary and the headline. Dylexia, perhaps?
C'mon, it's 2016. Where is compression?
It's a hard job. We're into year fifteen of ZFS and it's just starting to gain some features that make administration of it manageable by non-experts. Give it another five before you want to make it your default on a desktop for grandma. BTRFS will be along five years after that.
If Apple can pull off something similar in a couple years, it will be a major triumph. It's too bad for everybody that Steve got bitchy at Jonathan and the community hasn't had Apple's help as a contributor for the past decade.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I'm glad Apple has introduced this. As of now, the snapshot API and others are not present, but now Apple is on parity with everyone else in the industry.
APFS isn't like ZFS or btrfs, but more like ReFS in the fact that it still requires a logical volume manager. It would be nice if it had RAID, but that is a minor item, compared to just getting rid of HFS+, which just had to be killed.
Some features I like:
The ability to encrypt volumes with multiple volume keys. It looks like it will be similar to Oracle's ZFS on Solaris, but the implementation can be completely different.
Snapshots. Something like zfs send and zfs send -i will be quite useful for backups.
Copy-on-write capability, which is useful for VMs.
Of course, it appears that Apple will be documenting and publishing the FS's specs in 2017, which will be even more useful for compatibility.
All and all, even though there is no RAID 5/RAID-Z, or LVM replacement, this is a heck of a lot better than what OS X/macOS has now.
I was hoping Apple would license ZFS or even Veritas Volume Manager/Veritas FS from Symantec. Heck, even ReFS from MS. However, with all the cash they have, I am happy they are putting out something. I wouldn't expect it to be a default filesystem until 2017, perhaps 2018, as filesystems are something never to be undertaken lightly, but long term, it is crucial to macOS's usefulness, especially as SSDs get larger, and TRIM support is more critical to performance.
Licensing. Apple did flirt with ZFS, but for some reason, and I would guess it was license issues, they decided not to go that route. Using btrfs would bring GPL/BSD licensing issues. So, Apple either had to license something like ReFS from MS, or roll their own.
Compression? why - to compress all of those mp3 and mp4 video files? Or your TXT docs?
I was thinking that a current issue is the crypto-ransom stuff and that a FS needs to version on-demand. Sure everyone is *supposed* to have backups. I don't know what the Mac world is like but most PC folks I know do a file-copy to a USB drive (if they do anything at all). I'm not talking about what smart IT folks do - referring instead to general users.
How many people have a time machine? (and is that good enough?)
Swift is stable. Swift 2 has provided a very solid foundation that thousands upon thousands of developers have used to create many real-world applications with.
Yes, there is Swift 3 in the works. What, are you really suggesting that Apple shouldn't try to improve Swift? Are you saying that they should let it stagnate, just to meet your strange definition of "stable"?
Even when Swift 3 is released it doesn't mean you won't be able to use Swift 2 any longer. If you've got old Swift 2 code, then keep on using the Swift 2 compiler and toolset!
But if you want the benefits that Swift 3 brings, then by all means you'll be able to use it, too. Will you have to make some changes to your existing Swift 2 code to get it to work with Swift 3? Perhaps! But that's a perfectly reasonable cost if you want to make use of newer functionality.
You sound like one of those fools who moans about Python 2 versus Python 3, not realizing that it has actually gone very smoothly. Python 2 users weren't forced to upgrade when they didn't want to, and in fact they'll be able to use Python 2 for years to come! Python 3 was able to bring some serious improvements to the table. Users who wanted to use Python 3 had the freedom to do so. The major libraries all supported Python 3 very quickly, and the only ones that didn't are niche libraries that few users care about.
Maybe you have some personal vendetta against Swift. Maybe you're a Rust supporter who has seen his language flail miserably and then fail miserably, while Swift gets better and better. Regardless of why you don't like Swift, the fact remains that Swift 2 is stable, Swift 3 will soon be stable, and despite all of your bitching and moaning about this "instability" that doesn't exist there will be thousands, if not millions, of developers using Swift to accomplish great things.
With ransomware on the rise, having a filesystem that can take snapshots, perhaps coupled with a version of Time Machine that works on snapshots will help provide some mitigation. If the ransomware doesn't have root, it can't purge snapshots, although it can do mayhem in other places.
I would say Time Machine is OK for an "oh shit" backup for bare metal restores, but I wouldn't really rely on it as my sole way to retrieve data, because I've had instances where TM backups got hopelessly corrupted. I would probably recommend TM + Borg Backup or another utility, ideally a utility that can pull the data, so ransomware cannot get near the backup repository to destroy any existing data. Barring that, there is always Carbonite or Mozy, but one has to be aware of the security implications of dumping to the cloud.
ZFS is under CDDL and would not even need to be "licensed" in the usual sense — it is free for anybody to take. "Too free" for certain zealots, in fact, which is why it was not part of Linux kernel for a while — until the supposed "license incompatibility" myths got debunked.
Even Linux now offers ZFS — Apple would've had a much easier time porting it, because MacOS is already FreeBSD-based and the FreeBSD-project had ZFS available "out of the box" for several major releases spanning many years.
What did Apple find lacking about ZFS, that would justify creating their own, is, indeed, a mystery. Probably, a case of the Not Invented Here Syndrome. Sad...
In Soviet Washington the swamp drains you.
I was hoping Apple would license ZFS or even Veritas Volume Manager/Veritas FS from Symantec.
I thought Veritas was also called Online Journaled File System (OnlineJFS or OJFS). What else is OJFS?
How about letting users unplug removable media without having to eject it first like every other OS has had for about a decade.
I was hoping Apple would license ZFS or even Veritas Volume Manager/Veritas FS from Symantec. Heck, even ReFS from MS. However, with all the cash they have, I am happy they are putting out something. I wouldn't expect it to be a default filesystem until 2017, perhaps 2018, as filesystems are something never to be undertaken lightly, but long term, it is crucial to macOS's usefulness, especially as SSDs get larger, and TRIM support is more critical to performance.
I don't know anything about Veritas FS; but I have looked long and hard at ZFS; but the continued major issues with ZFS on macOS, with Finder integration and more, while getting SLOWLY better (and which would no doubt get better with Apple's access and engineering), still signal that ZFS just isn't ready for Prime-Time on OS X, much to my chagrin.
Would you want to be licensing anything from Oracle today?
Aye. Whereas almost all of Microsoft's filesystem advances are hidden in shadow-copies and inaccessible system folders. Or enterprise-version only RAID-like features. We can't even freaking tag files n folders unless they are "media" files.
You have inordinately cheap disk
Because of Apple's tendency to solder the SSD to the mainboard in the Mac Pro and all current MacBook laptops other than the non-Retina MBP, an upgrade requires replacing the whole computer at a substantial cost. Only external storage is "inordinately cheap" on a Mac, and not all laptop use cases make external spinning rust practical.
Sure, you could find lots of value in compression.... and you can get it with file compression utilities.
That's fine, so long as these utilities can let the user mount an archive read-only as a folder and thereby let other applications see the archive's contents as files in as a folder. Does macOS Sierra introduce anything that interferes with OSXFUSE?
If I didn't know any better, it sounds like Apple might be gearing up to offer some sort of in-house virtualization, with this new filesystem laying the foundation for it.
Licensing. Apple did flirt with ZFS, but for some reason, and I would guess it was license issues, they decided not to go that route. Using btrfs would bring GPL/BSD licensing issues. So, Apple either had to license something like ReFS from MS, or roll their own.
Exactly. And it WAS Licensing in the case of ZFS. They didn't want to have their Filesystem beholden to the likes of Oracle (and I for one, don't blame them a bit!).
.. case sensitive filenames by default? :D
Just wondering. I know HFS+ can have case-sensitivity, but not sure if it is on by default. And some people seem to be discouraging that, based on quick googling.
Will non-Apple OSes be able to mount this new FS? Apple's track record is not great on this; its a shame to see the question not even addressed in the couple of mentions of AFPS I have seen.
HFS may be thirty years old but we still have major headaches transferring files between Macs and other machines. I truly believe that Apple would be better served if they invested in a open filesystem format.
Why would they do that? How else can they move to a "monthly subscription model" for your file system? You'll pay Apple some nominal $9.99 charge per month - can't do that if it's some easy-to-support file system. Gotta be proprietary so your files and data are nearly impossible to move over (the rest of the OS/dev languages will quickly follow with support for nanosecond file times - and will break if that granularity isn't there) and thus Apple achieves the ultimate lock-in. With monthly recurring revenue.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
For starters, number 1 we're really logical thinkers and B, we value consistency.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
Long ago, there was a Stacker-like tool for classic Mac OS called TimesTwo that installed itself as a SCSI driver. There were also file-level tools called DiskDoubler, Now Compress, and StuffIt SpaceSaver that intercepted file open calls and decompressed files in the background. Files were written uncompressed, to be compressed later by the "AutoDoubler" background task.
> but the continued major issues with ZFS on macOS, with Finder integration and more,
You wouldn't happen to have more detailed information by chance please?
> much to my chagrin.
I too lament that fact that ZFS wasn't chosen. I guess this is the typical NIH here by Apple. :-/
It means you can guarantee that each file has a unique 64-bit timestamp by simply assigning a sequential nanosecond timestamp if two files get written at once. It also gives an opportunity to work around UNIX's year 2038 problem (2^31 UTC seconds since 1970) and Apple's year 2040 problem (2^32 UTC seconds since 1904), pushing it out to at least 2262 (2^63 UTC nanoseconds since 1970).
In theory, any journaled file system would tolerate surprise disconnection to the same extent it tolerates surprise power loss. The problem is that journaled file systems tend to be either proprietary or copylefted, hindering their wide adoption for removable media across all major desktop operating systems.
Really, terabyte SSDs? Today's SSDs, in terms of storage capabilities are more like mechanical drives of 20 years ago. Yes, data centers may have large SSDs, but not users. Will average users benefit from this new file system or will things like 64bit pointers on a drive less than a gigabyte simply consume more of the drive for little benefit?
Finally, are not there already file systems available that meet whatever this new need of Apple's is that would not require the recreation of the wheel (or disk)? If the goal is secure Apple a spot in the enterprise, wouldn't a main stream file system like ZFS be more likely to achieve that goal?
ZFS is not recommended for non-ECC RAM. RAM errors can get propagated to disk by application read operations, not just writes.
http://research.cs.wisc.edu/ad...
I back up my MacBook Pro to a FreeBSD box using ZFS with compression and deduplication (and snapshot it periodically, because if TimeMachine detects that your backups are corrupted then the only option is to delete them an redo from start, and it's nice to be able to revert just one backup if the last backup broke something). With lz4 compression, the compression ratio for the ZFS filesystem that I use as a backup target is 2.08x - that's a fairly hefty saving. It's harder to measure how much dedup is saving me, because it's a pool-wide property and I have ripped a load of my DVD collection, which probably doesn't contain many duplicate blocks. Currently the dedup ratio is 1.2x, but given that the TimeMachine volume is around 20% of the total space and contains a lot more redundancy than the rest, I wouldn't be surprised if I'm saving a relatively large amount there too.
I probably get more benefit from dedup on the backup disk (TimeMachine copies entire files, even if only a single block has changed) than I would on the laptop, but I'd expect to see similar benefits from compression on the source and the backup, and that 2x compression ratio looks pretty good...
I am TheRaven on Soylent News
"APFS supports nanosecond time stamp granularity rather than the 1-second time stamp granularity in HFS+.
Damn, 1-nansecond time stamp granularity? A factor of one billion improvement in resolution, that's fairly impressive. I'm not sure it'll be of much use to a lot of people, but I'm all for greater precision/resolution in stuff like this.
Just cruising through this digital world at 33 1/3 rpm...
> but the continued major issues with ZFS on macOS, with Finder integration and more,
You wouldn't happen to have more detailed information by chance please?
> much to my chagrin.
I too lament that fact that ZFS wasn't chosen. I guess this is the typical NIH here by Apple. :-/
Got nothin' to do with NIH. Apple uses LOTS of industry standards and Open Source projects that ALL fall under the "Not Invented Here" category.
Dig down into the Forums on the OpenZFSOnOSX Site to see what issues people are still having these days with ZFS (OpenZFS) under OS X. Last I looked was early this year.
I don't understand why innovations like those found in BeFS (like rich metadata support) go ignored whenever someone creates a new file system. If you are going to break compatibility, you might as well add in some useful features.
Now? It means very little other than effectively guaranteeing a total ordering on files created on a single system. Remember, however, that filesystems typically stay in production for at least 20 years (35 for HFS+ is not that uncommon - UFS is even older) and so a little bit of headroom is always a good idea.
I am TheRaven on Soylent News
I was mistaken. I apologize. Thank you for clarifying. So let me correct myself:
In order to understand Apple's intent in leaving transparent compression out of a file system, we'll have to watch for what Apple chooses to solder down in its next round of hardware.
It will be opensourced in 2017, when it's ready for the world and settled.
Actually, Apple already commited to opening up the volume format:
so now we only have to worry about patent encumbrance? at least that shrinks the problem domain
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I don't understand why innovations like those found in BeFS (like rich metadata support) go ignored whenever someone creates a new file system. If you are going to break compatibility, you might as well add in some useful features.
That sad truth is that the utility provided by rich metadata support is more than counterbalanced by the headaches it causes whenever you need to copy your files anywhere outside of the filesystem that supports it. (Just ask any .sd2 audio format users about what happened to their audio files when they tried to copy their data to a FAT32-formatted USB key, or email them, or etc).
Because of compatibility/interoperability issues, programs can't rely on putting important data into the resource forks, so the feature ends up going unused. Because the feature ends up going unused, people continue to use filesystems that don't support it. Because people continue to use filesystems that don't support it, app developers can't rely on it to work. And round and round we go, forever stuck in a bleak dystopia where the only supported "metadata" is a three-letter extension to the filename. :(
I don't care if it's 90,000 hectares. That lake was not my doing.
I'm going to guess that this uses Apple's "double float" version of the Unix timestamp. Basically, if you're going to use a 64-bit integer to avoid the Y2038 problem, you'll end up with a lot of useless bits. If you convert the same numbers to 64-bit double float, those extra bits become nanosecond resolution (or probably better than your hardware can measure). If for some reason you want to deal with timestamps a few billion years in the past or future, then you can afford to deal with mere seconds-level accuracy. Beyond that, you may have to deal with accuracy of minutes or hours, but what's a few seconds compared to the life of the universe?
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
More than that, ZFS was already under legal action from NetApp, so I have a feeling the licensing squabble wasn't just about money or support, but legal indemnity should Sun / Oracle lose that thing.
Slashdot still doesnâ(TM)t support Unicode after it was added to the HTML standard in 1997.
Seems I will keep waiting for a FS that can do versioning.
I wonder if it is more of an API hindrance than a technical one (as a snapshot already has much of what you need for it).
Also, it's worth noting that Apple will often "reinvent the wheel" when the existing wheels don't quite work they way they'd like. They don't have to maintain compatibility with other vendors or old concepts, which frees them to do what they like.
Why can't money or development time be put towards existing open solutions and fixing them up?
I am using ZFS on my FreeNAS machine and from what I understand BTRFS has some features superior to ZFS (but also some stability issues)
It's really super frustrating as a geek that there isn't a single "be all and end all" file system out there yet. Something definitively powerful, reliable, fast.
I know there's been some improvements over the years to these file systems but damn if progress doesn't seem slow.
Also Apple creating their own? How long until it's reliable? It takes forever to make a filesystem good.
FTP and SFTP are trivial user-space servers. Of course they'll work. OS X doesn't support AFS, AFAIK. I'm not surprised that Apple hasn't added NFS support yet; I hope they will, because there are many situations where SMB doesn't work very well.
As for AFP, I'm not surprised that they haven't added support. AFP is really designed around HFS+. It performs poorly on other filesystems because of things like the searchfs system call not existing or being emulated by brute force. AFP was barely usable on UFS, and aliases didn't work. Supporting AFP would likely require a lot of extra engineering, whereas most of those behaviors (aliases, searching via Spotlight instead of filesystem metadata) are at least partially supported through emulation atop SMB already, so they'd get a lot of that functionality for free.
Check out my sci-fi/humor trilogy at PatriotsBooks.
So use ECC RAM. By an amazing coincidence, the company announcing this software also just happens to also manufacture hardware.
ECC requires a compatible motherboard, CPU and RAM. Strangely higher end Intel CPUs, i5 and i7, tend not to support ECC.
Can you cut and paste files yet?
Wanna buy a shirt?
https://www.redbubble.com/people/stealthfinger/shop?asc=u
The BSD Now show "ZFS in the trenches" talks about how different developers have different opinions on whether you must use ECC RAM with ZFS but all recommended ECC if possible (so that part is not in debate). Around the 56:30 mark of the episode, Josh Paetzel of FreeNAS explains that if the ZFS spacemaps were corrupted due to memory errors you could be in a worse situation compared to less sophisticated filesystems that have an fsck. This is because with a corrupt spacemap ZFS would refuse to let you access any data whereas the fsck would "just" (irreversibly) decopule or delete chunks of data in an attempt to allow access to the rest. It's not a great situation - inaccessible data versus data loss and/or corruption but it is a difference.
Late to the party, but for those asking about ZFS & apple, Adam Leventha (dtrace Sun engineer)l posted an exceptionally informative post about the history of this mess earlier this week:
http://dtrace.org/blogs/ahl/20...
mlts, re ZFS, see my post at the end of this story.