Machine Learning, Optimization, Gradient Descent, Reinforcement Learning.
function for RELAX. REBAR makes the mistakes of evaluating f(y)=(σ_λ(z)−t)² directly for its surrogate. This expression is 0 at σ_λ(z)=0.5 and large at 0 or 1. This roughly tells the algorithm that “values of θ around 0.5 are bad and values around 0 or 1 are good.” But this is simply not true!