I&#x27;ve seen this repeated a lot, but have not had quite the same experience with &quot;permanent&quot; performance degradation.  Especially if I eventually expand the pool with another vdev.  Not sure about ZFSonLinux, but:<p>1) Having a ZIL helps with this, and in general.
2) ZFS changes strategy depending on how full it is, it spends more time avoiding further fragmentation rather than grabbing the first empty slot.  This hit would go away if you get the free space back up.
3) Finally, there is a way[1] to have ZFS keep all the info it needs in RAM to greatly alleviate the times when it starts hunting harder to prevent more fragmentation.  It looks like the RAM requirements are 32GB&#x2F;1PB... so not too bad IMO.<p>[1] <a href="https:&#x2F;&#x2F;blogs.oracle.com&#x2F;bonwick&#x2F;entry&#x2F;space_maps" rel="nofollow">https:&#x2F;&#x2F;blogs.oracle.com&#x2F;bonwick&#x2F;entry&#x2F;space_maps</a>