Not sure about why this ( reshaping before and after CNN) works, but at least tensorflow 1.x has an identical implementation: https://github.com/tensorflow/tensorflow/blob/05ab6a2afa2959410d48aab2336cea9dc1e2c13e/tensorflow/python/keras/layers/wrappers.py#L237 58