From the look of it. I think you are trying to train your network to learn the best inputs for XGBClassifier from some features.
But from this line
clf.fit(x.detach().cpu().numpy(),targets.detach().cpu().numpy())

Are you trying to make x to be the same as target?