I assume you are running out or memory or are you really seeing a MemoryError?
In the former case, could you try to lower the number of kernels in your UNet and check the memory usage?
The OOM shouldn’t be related to loading the data as numpy arrays.
PS: based on the target shape, I assume you are dealing with a multi-label classification, i.e. each pixel might correspond to multiple classes?