Part of the problem was the code requires TF > 2.0.
The only difference I see is that the user told me he got it to work by adjusting the comment tags as such:
#sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True, log_device_placement=True))
#keras.backend.set_session(sess)

Now the GPU works.
I also changed:
from keras.optimizers import adam
to
from keras.optimizers import adam_v2
and
optimizer=adam(learning_rate=0.001)
to
optimizer=adam_v2.Adam(learning_rate=0.001)
Before this the logfile blew up to 6 GB with entries like:
2021-08-19 05:08:41.796216: I tensorflow/core/framework/op_kernel.cc:1287] No device-specific kernels found for NodeDef '{{node _SOURCE}}'Will fall back to a default kernel.
2021-08-19 05:08:41.796223: I tensorflow/core/framework/op_kernel.cc:1287] No device-specific kernels found for NodeDef '{{node _SOURCE}}'Will fall back to a default kernel.
2021-08-19 05:08:41.796232: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node _SINK}} = NoOp[]()
2021-08-19 05:08:41.796238: I tensorflow/core/framework/op_kernel.cc:1287] No device-specific kernels found for NodeDef '{{node _SINK}}'Will fall back to a default kernel.
2021-08-19 05:08:41.796245: I tensorflow/core/framework/op_kernel.cc:1287] No device-specific kernels found for NodeDef '{{node _SINK}}'Will fall back to a default kernel.
2021-08-19 05:08:41.796255: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/mod}} = FloorMod[T=DT_INT32, _class=["loc:@loss/dense_1_loss/mean_squared_error/Mean"]](training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/add, training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/Size)
2021-08-19 05:08:41.796283: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/add}} = AddV2[T=DT_INT32, _class=["loc:@loss/dense_1_loss/mean_squared_error/Mean"]](loss/dense_1_loss/mean_squared_error/Mean/reduction_indices, training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/Size)
2021-08-19 05:08:41.796303: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/Size}} = Const[_class=["loc:@loss/dense_1_loss/mean_squared_error/Mean"], dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: 2>]()
2021-08-19 05:08:41.796319: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node loss/dense_1_loss/mean_squared_error/Mean/reduction_indices}} = Const[dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: -1>]()
2021-08-19 05:08:41.796335: I tensorflow/core/framework/op_kernel.cc:1487] Instantiating kernel for node: {{node _send_training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/mod_0}} = _Send[T=DT_INT32, client_terminated=true, recv_device="/device:CPU:0", send_device="/device:CPU:0", send_device_incarnation=-6529568560417163830, tensor_name="training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/mod:0"](training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/mod)
2021-08-19 05:08:41.796357: I tensorflow/core/common_runtime/executor.cc:1717] Process node: 0 step -1 {{node _SOURCE}} = NoOp[]() device: /device:CPU:0
2021-08-19 05:08:41.796368: I tensorflow/core/common_runtime/executor.cc:1717] Process node: 4 step -1 {{node training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/Size}} = Const[_class=["loc:@loss/dense_1_loss/mean_squared_error/Mean"], dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: 2>]() device: /device:CPU:0
2021-08-19 05:08:41.796378: I tensorflow/core/common_runtime/executor.cc:1717] Process node: 5 step -1 {{node loss/dense_1_loss/mean_squared_error/Mean/reduction_indices}} = Const[dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: -1>]() device: /device:CPU:0
2021-08-19 05:08:41.796390: I tensorflow/core/common_runtime/executor.cc:1717] Process node: 3 step -1 {{node training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/add}} = AddV2[T=DT_INT32, _class=["loc:@loss/dense_1_loss/mean_squared_error/Mean"]](loss/dense_1_loss/mean_squared_error/Mean/reduction_indices, training/Adam/gradients/loss/dense_1_loss/mean_squared_error/Mean_grad/Size) device: /device:CPU:0

Anyways seems to be good now perhaps this will help someone down the line.