Hello,
Firstly, it actually depends on how this data is correlated, and understanding that would be the first step incorporating this data in your model. You may want to introduce some data in the later layers of your model and sometimes you can pass the data to your model all at once.
Secondly, you might want to extract features from your data, and doing so from categorical data is tough, especially because one hot encoded representations of such data would result in sparse tensors, and applying normal layers of such data would yield in meaningless results. I generally apply one of the two strategies, One is such that you would use an embedding layer for one hot encoded representations of each categorical field and then concatenate them.  Another way is to use sparse convolutions and sparse linear layers. Nvidia has a package called Minkowski Engine which has sparse implementations of the convolutional operations. You might want to go through the theory first to see which fits your case better.
To load the text data, load them as the pretrained vector representations of the words. Is it a standard dataset with which you are working ?