To avoid loading all the data into memory at once you can try to use tf.keras.utils.Sequence and create a data generator. It will load the data into RAM in batches as the training goes.
Here is an example of how it is done with images (Image segmentation with a U-Net-like architecture) but you can rewrite it to query the data base and transform the original sparse data into OHE vectors for 40 items just for the necessary batch.
To increase the accuracy you can experiment with more complicated architectures. Here are examples for structured data: Structured Data