Answering my own question.
The fix is easy. First, the load is CPU bound indeed. To move the computation to GPU (as we want to) we can change the code above to introduce batching on Dataset level by specifying sampler in DataLoader or implement batching by ourselves.
I’ve followed the second approach (due to the fact that fastai DataLoader doesn’t offer sampler option). The code is below (it drops last non full batch but I have aligned data):
class Dataset(torch.utils.data.Dataset):
    def __init__(self, df, bs):
        self.l = len(df)
        df_numpy = df.to_numpy()
        self.x = torch.from_numpy(df_numpy[:,0:2]).cuda()
        self.target = torch.from_numpy(df_numpy[:,2:3]).float().cuda()
        self.bs = bs
        
    def __getitem__(self, i):
        return (self.x[self.bs*i:self.bs*i+self.bs,:], self.target[self.bs*i:self.bs*i+self.bs,:])
    
    def __len__(self): return int(self.l / self.bs)