Also, is it correct that this will be faster if the categorical one-hot inputs are sparse tensors? I have 10-50 categorical variables, each has about 10 categories, and about 50 more continuous variables. I suspect that with such a low number of categories, that dense is faster.