Right, you might need to consider what the meaning of your training data is. At the moment, you are passing in a batch of 128 examples, where each example has 13 channels but only a length of 1. I assume the TDNNLayer is something like a 1D convolution, so in this case it cannot compute an output when the filter size length (5) is greater than the input size (1). The fundamental issue is that the sequence length must be increased to use the layer.