Displaying difference between APFS snapshots Problem

APFS is amazing... and infuriating with its lack of tools available.

What I'm trying to do is simply an equivalent of btrfs send --no-data -p /snapshots/parent /snapshots/child

My investigation


Most of the google-fu returns usages of tmutil


tmutil is able to listlocalsnapshots / deletelocalsnapshots / localsnapshot (create)
tmutil compare &lt;snapshot_path&gt; doesn't accept 2 snapshots
tmutil compare EXISTING_SNAPSHOT returns cryptic &amp; undocumented Must specify a snapshot root when performing a machine comparison.
tmutil calculatedrift &lt;machine_directory&gt; doesn't seem to be related to APFS snapshots
tmutil contains some undocumented options, but nothing related to snapshots
Snapshot created by tmutil have hardcoded com.apple.TimeMachine.YYYY-MM-DD-HHMMSS.local naming format

diskutil contains limited snapshots support


diskutil deleteSnapshot works as intended
diskutil listSnapshots list snapshots with sizes and some metadata (unlike tmutil)
No facility for creating snapshots

Kernel contains fs_snapshot_* syscalls


The documentation is almost non-existent, but a single man page is available 
All usual operations (fs_snapshot_delete / fs_snapshot_list / fs_snapshot_create / fs_snapshot_mount) are supported
GitHub ahl/apfs hosts a small utility which demonstrates the syscalls usage
fs_snapshot_list list only com.apple.TimeMachine.* snapshots
Any write operations (e.g. fs_snapshot_create) end up with application being killed with fs_snapshot_create: Operation not permitted (probably due to documented in the man reason: "require an additional entitlement")

Commercial app, Carbon Copy Cloner, is able to manipulate snapshots


For obvious reasons they don't disclose how they do that ;)



Help? ;)

Is there any method of getting a diff between two snapshots? Any documentation how they can be managed?
The only naïve approach which because of speed is not feasible is running rsync between r/o mounted snapshots... but this is just stupid.
There's no tool available for APFS similar to what you see with btrfs send. The state of APFS is still relatively "early days" when it comes to tools like that. On btrfs it took 3-4 years before they had the btrfs send tool in ordinary Linux - I wouldn't expect Apple to be able to push something out faster than that.

The btrfs tool is very elegant in that it relatively efficiently provides a list of simple commands (such as unlink, rename, write) to transform an existing snapshot (on both the source and destination) into another snapshot that exists on the source, but not yet on the destination.

With APFS you haven't got such a tool to efficiently create the list of diff commands. However, you can use any tool that synchronizes two ordinary folders over a network. It won't be as efficient in generating the list of diffs, but it should be possible to make it just as efficient in applying the diffs (as the btrfs receive command is implemented in user-space). 

A well known tool for that purpose is rsync, but you've already mentioned that you do not want to run rsync because of the speed. Unfortunately there's really no methods speedier - unless you happen to have some kind of special use case where you can make assumptions about the changes between your snapshots.

Regarding Carbon Copy Cloner and the entitlements - I don't think it's a "secret" as such how they work with snapshots. You've found the various system calls for snapshot manipulation (which they use), and you can read the the actual on-disk format specification on Apple's web page. You could use dtrace to inspect the actual system call parameters and order of them that CCC uses.

The sad part is that creating your own programs that do something with snapshots (besides listing them), requires the entitlement "com.apple.private.vfs.snapshot". You can get that entitlement only by talking to Apple's technical support for developers. If you're creating backup software for example, you should fit the criteria for getting it - but the only real way to find out is to contact Apple.