Thank you for your post.
I have noticed,

If I use the original transfer learning script (from tutorial), my model seems to be learning I will post the script in the OP after this post. I am not sure what I am doing wrong in my revision

As for you post, thank you! If I wanted to train at different parts of the model at
different rates, do I create multiple optimizers?