Using the method above yielded:
X = torch.from_numpy(data).clone() | TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.
    X = torch.from_numpy(data).clone()   <-- error here
    X = X / 255.0 # scale pixel values to [0, 1]
    y = torch.Tensor([i[1] for i in data])

edit: I have also changed my model back to taking in a picture of width: 640 and height: 800 since it seems you can certainly train on any dimensions you want as long as you eval at the same dimensions.
x = torch.randn(800, 640).view(-1, 1, 800, 640)