I was able to narrow it down to the HSV shift transform causing it. This error below if anyone is curious. The version of albumentations I was previously using (0.4.3) did not have tracebacks for it and just skipped the channel it had a problem with ( I think). The latest version (1.0.0) is much more verbose with a traceback clearly showing shift_hsv was causing it.
Thanks for the help @ptrblck
return F."shift_hsv"(image, hue_shift, sat_shift, val_shift) File, line 55, in wrapped_function result = result.reshape(shape) 
ValueError: cannot reshape array of size 1769472 into shape (768,768,4)