If you can isolate the inputs that cause CTC or gradients to go wrong (before the inputs become NaN) I’d be most interested. So far no-one did that and it almost seems like for most people seeing NaNs the source is somewhere else.
Best regards
Thomas