I’m not sure what might create the issue and we would need an executable code snippet to debug further. 
Could you try to use random data, stick to your current pipeline as close as possible, and check if this would reproduce the error?