I have figured out a solution where I split the larger image into a list of tiles and then run one big loop in which I pass one tile at a time to the model.
Maybe not pretty, but it works pretty well.