Based on the notebook it seems the author create CNNs containing 7 conv-bn blocks and a final linear layer, while your current model seems to be quite different.
Am I missing the “right” model implementation or is the posted code not supposed to replicate the Keras model?