Show HN: Filesystem Watcher
I have personally written a similar tool and I am very curious about how this could be using a near-zero amount of resources while maintaining accuracy. As far as I know, there are two ways to implement this functionality:
1) store an in memory representation of the file system and periodically refresh the in memory state by polling the paths under watch and emitting events when differences are detected
2) hook into the underlying kernel events like kqueue, inotify, fsevents, ReadDirectoryChangesW, etc and report events<p>Option 1 uses a lot of CPU and memory (the map storing the paths being monitored could easily grow to be tens or even hundreds of megabytes if many files are being monitored, which is often the case in large source projects). I have seen tools that use polling with a 100ms interval continuously burn 50% of cpu monitoring a modest sized directory with tens of thousands of files.<p>Option 2 theoretically would use less memory and little to no cpu, but in practice, the story is more complicated. If you are using an inotify or kqueue like api, you will have to store handles for all of the paths that are being monitored, which can take a significant amount of memory. On macos, the file system events are not accurate in the sense that you can&#x27;t trust the type of event. It doesn&#x27;t reliably distinguish between creation and modification events. So if you want to know specifically what kind of event happened, you end up back in case 1 where you have to store an in memory representation of the file system and diff against the in memory representation and the current file system state when you detect an event. For some use cases, you may not care to distinguish between creations and modifications and can get away with a lower memory, but less accurate, solution.<p>In my experience, getting all of this right is much more difficult than it appears at first glance. Good luck to you.