<p>How about:</p>
<ul>
<li>speed of convergence</li>
<li>stability/variance w.r.t to the initial random seed (or other sources of variance like learning rate)</li>
<li>presence/number of saddle points in your loss landscape</li>
</ul>

CC BY-SA 4.0
2022-06-16T13:29:57.713Z
2022-06-16T13:29:57.713Z
ImotVoksim
<p>I'm comparing 2 optimization algorithms for deep neural nets through visualizing the loss landscape. The visualization method is described <a href="https://arxiv.org/abs/1712.09913" rel="nofollow noreferrer">here</a>.</p>
<p>Besides the qualitative observation that how trajectory moves w.r.t. the loss level-sets, are there any quantitive measures to compare the two methods?</p>
<p><a href="https://i.sstatic.net/28MPDm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/28MPDm.png" alt="enter image description here" /></a>
<a href="https://i.sstatic.net/4rbdzm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4rbdzm.png" alt="enter image description here" /></a></p>

<neural-networks><loss>
Visualizing the loss landscape in deep NN to compare optimization methods
Blade
# Visualizing the loss landscape in deep NN to compare optimization methods

(Asked by: username_0 on 2022-06-14)

I'm comparing 2 optimization algorithms for deep neural nets through visualizing the loss landscape. The visualization method is described [here](https://arxiv.org/abs/1712.09913).

Besides the qualitative observation that how trajectory moves w.r.t. the loss level-sets, are there any quantitive measures to compare the two methods?

[![enter image description here](https://i.sstatic.net/28MPDm.png)](https://i.sstatic.net/28MPDm.png)
[![enter image description here](https://i.sstatic.net/4rbdzm.png)](https://i.sstatic.net/4rbdzm.png)

**Tags:** neural-networks, loss


---

## 1 answers

### Answer by username_1 on 2022-06-16. Score: 1

How about:

* speed of convergence
* stability/variance w.r.t to the initial random seed (or other sources of variance like learning rate)
* presence/number of saddle points in your loss landscape


---

