I had a similar problem when converting data read from pandas to PyTorch.
I was able to solve it using pd.to_numeric(...)
Hence, for the code at the top, the fix could be:
x_data = pd.to_numeric(xy.iloc[6:-1,:-1].values)
y_data = pd.to_numeric(xy.iloc[:6,:-1].values)