Very neat result by @jasondeanlee.bsky.social at arxiv.org/pdf/2411.17668
Opens a new frontier to push Gradient Descent!

They show special steps can give GD an anytime convergence rate of 1/T^1.03.
Very curious if this will eventually match the conjectured limit of 1/T^1.27 for final iterate rates.