By skimming through the section for CNN-2 and based on your explanations, I guess the input shape should be [batch_size, 1, 5, 50], as the kernels seem to operate 2-dimensional.
Unfortunately, I cannot find any architecture description for CNN-2, but it seems the kernel size should be [2, 3] (based on Figure 3).
The authors mention a CAFFE implementation. Were you able to find it, as it would make it easier to recreate the model?