<p>After 100 steps learning rate will switch from 1e-2 to 1e-24.</p>
<p>What exactly is the problem here? Are you confusing steps with epochs?</p>

CC BY-SA 4.0
2021-05-16T09:57:12.997Z
Are you sure? I've tested this. With 1e-24 learning is completly stalled. However, after 100 steps learning just continues, implying that we are still learning with 1e-2. I'm glad you think it's correct but It does not seem to work
nammerkage
CC BY-SA 4.0
2021-05-16T09:50:16.483Z
2021-05-16T09:50:16.483Z
spb
<p>I'm trying to manipulate the learning rate with tf PiecewiseConstantDecay.
I can easily check if the algorithm switches learning rate values, because one rate is extremely low 1e-20 !!
However, NO setting of the &quot;boundaries&quot; causes the algorithm to switch learning rate... What am I doing wrong?</p>
<pre><code>step = tf.Variable(0, trainable=False)
boundaries = [100]
values = [1e2, 1e-20]

schedule = tf.optimizers.schedules.PiecewiseConstantDecay(boundaries, values)
lr = 1e-4 * schedule(step)

optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
model.compile(optimizer=optimizer, loss='mean_squared_error')

history = model.fit(x = x_train, y = y_train, validation_data=(x_val, y_val), epochs=100,batch_size=32)
</code></pre>
<p>Would love your input.</p>

<neural-networks><machine-learning>
Tensforflow schedule - does not change boundaries
nammerkage
# Tensforflow schedule - does not change boundaries

(Asked by: username_0 on 2021-05-16)

I'm trying to manipulate the learning rate with tf PiecewiseConstantDecay.
I can easily check if the algorithm switches learning rate values, because one rate is extremely low 1e-20 !!
However, NO setting of the "boundaries" causes the algorithm to switch learning rate... What am I doing wrong?
step = tf.Variable(0, trainable=False)
boundaries = [100]
values = [1e2, 1e-20]

schedule = tf.optimizers.schedules.PiecewiseConstantDecay(boundaries, values)
lr = 1e-4 * schedule(step)

optimizer = tf.keras.optimizers.Adam(learning_rate=lr)
model.compile(optimizer=optimizer, loss='mean_squared_error')

history = model.fit(x = x_train, y = y_train, validation_data=(x_val, y_val), epochs=100,batch_size=32)

Would love your input.

**Tags:** neural-networks, machine-learning


---

## 1 answers

### Answer by username_1 on 2021-05-16. Score: 1

After 100 steps learning rate will switch from 1e-2 to 1e-24.

What exactly is the problem here? Are you confusing steps with epochs?

> **Comments:**
> **username_0**: Are you sure? I've tested this. With 1e-24 learning is completly stalled. However, after 100 steps learning just continues, implying that we are still learning with 1e-2. I'm glad you think it's correct but It does not seem to work  

---

