I adjusted the LR to 0.0001 and 0.00001 respectively. 1e-5 shows some improvement, I will let it run for some more epochs.
1e-4:
Epoch 1 -- loss 23092.178817, RMS error 0.796679, R² score -0.764875 
Epoch 2 -- loss 29207.602612, RMS error 0.786404, R² score -0.489875 
Epoch 3 -- loss 30938.809148, RMS error 0.781764, R² score -0.447401 
Epoch 4 -- loss 36056.395415, RMS error 0.783350, R² score -0.420444 
Epoch 5 -- loss 38970.840979, RMS error 0.779231, R² score -0.424409 
Epoch 6 -- loss 38725.751165, RMS error 0.781632, R² score -0.482981 
Epoch 7 -- loss 53787.240595, RMS error 0.780569, R² score -0.483686 
Epoch 8 -- loss 67411.768857, RMS error 0.783626, R² score -0.524781 
Epoch 9 -- loss 55415.253039, RMS error 0.782485, R² score -0.469684 
Epoch 10 -- loss 156964.512527, RMS error 0.796239, R² score -1.015255 
Epoch 11 -- loss 154036.751858, RMS error 0.797862, R² score -1.176710 
Epoch 12 -- loss 136232.229137, RMS error 0.799174, R² score -1.175171 
Epoch 13 -- loss 54970.134525, RMS error 0.780720, R² score -0.520048 
Epoch 14 -- loss 77784.615496, RMS error 0.785365, R² score -0.509746 
Epoch 15 -- loss 174066.987240, RMS error 0.798963, R² score -1.201877 
Epoch 16 -- loss 129737.510334, RMS error 0.790259, R² score -0.897164 
Epoch 17 -- loss 123497.888881, RMS error 0.791715, R² score -0.918957 
Epoch 18 -- loss 51374.109922, RMS error 0.780750, R² score -0.554175 
Epoch 19 -- loss 61803.601078, RMS error 0.782392, R² score -0.486701 
Epoch 20 -- loss 100403.409419, RMS error 0.785900, R² score -0.647144 

1e-5:
Epoch 1 -- loss 176985.034656, RMS error 0.811731, R² score -1.415746 
Epoch 2 -- loss 160285.464975, RMS error 0.810098, R² score -1.328036 
Epoch 3 -- loss 91367.396519, RMS error 0.793799, R² score -0.854228 
Epoch 4 -- loss 15116.144004, RMS error 0.782176, R² score -0.526900 
Epoch 5 -- loss 4085.806400, RMS error 0.780666, R² score -0.492350 
Epoch 6 -- loss 4382.456566, RMS error 0.781868, R² score -0.519753 
Epoch 7 -- loss 7855.515105, RMS error 0.783714, R² score -0.562923 
Epoch 8 -- loss 6698.984660, RMS error 0.782674, R² score -0.540443 
Epoch 9 -- loss 4957.788071, RMS error 0.781994, R² score -0.524670 
Epoch 10 -- loss 4914.508950, RMS error 0.782255, R² score -0.525899 
Epoch 11 -- loss 5427.704976, RMS error 0.781978, R² score -0.517649 
Epoch 12 -- loss 4490.562387, RMS error 0.781281, R² score -0.501221 
Epoch 13 -- loss 3955.365406, RMS error 0.780896, R² score -0.490551 
Epoch 14 -- loss 5101.983363, RMS error 0.781299, R² score -0.496392 
Epoch 15 -- loss 4485.993145, RMS error 0.780489, R² score -0.477653 
Epoch 16 -- loss 3481.978184, RMS error 0.780190, R² score -0.460933 
Epoch 17 -- loss 3528.847296, RMS error 0.779919, R² score -0.458042 
Epoch 18 -- loss 3748.922880, RMS error 0.779473, R² score -0.451958 
Epoch 19 -- loss 3778.684385, RMS error 0.779273, R² score -0.442150 
Epoch 20 -- loss 3355.956672, RMS error 0.778943, R² score -0.435794 

My initial thought was to simplify the autoencoder architecture, and have just 1 hidden layer before it goes to the latent layer.
The loss there was still volatile, but RMS and R² showed some improvement, however, I guess it was just learning alot slower, hence the ‘nice learning curve’.