I feel like I should comment to avoid the leaving an incomplete thread.
Ultimately, I just used the windowed reading 33 functionality of Rasterio.
I then glued the windowed outputs back together  using np.block(). Probably not the most efficient method but it works well enough for my needs.