When you want to approximate a blackbox function, you need to have access to its inputs and outputs. If you don’t, the problem of



 adent:

a function that is close to the original (unknown) function that created the 20K dataset.


is ill defined. For instance, is the function "i => i-th string in your data" a desired function? Surely it outputs some pretty damn good results.
That is why I suggested looking into generative models, i.e. models that describes how a dataset could be generated, with the goal that the resulting models should be able to generalize. One example of such model is a deep function that maps standard Normal random values N(0, 1) to the type of data you want to model.



 adent:

So it would not make sense to use a portion of the 20K strings as the real data set to be trained on, with a generator that attempts to mimic that real data set


It depends. It makes sense in VAE case, but makes less sense in GAN case.



 adent:

The idea was to use an MDN’s probability distribution of the output to generate a dataset that would be compared to the original 20K strings with a cost function based on the predicted distribution compared with the original 20K strings/values.


MDN’s number of mixture components scales with #outputs. The data you have has 88 components and is quite multimodal (approximate to uniform even as it is supposed to be a pseudo random function). So MDN is likely bad here.