title: How to solve nan loss?
body: <h1>Problem</h1>
  
  <p>I'm running a Deep Neural Network on the MNIST where the loss defined as follow:</p>
  
  <p><code>cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, label))</code></p>
  
  <p>The program seems to run correctly until I get a nan loss in the 10000+ th minibatch. Sometimes, the program runs correctly until it finished. I think <code>tf.nn.softmax_cross_entropy_with_logits</code> is giving me this error.
  This is strange, because the code just contains <code>mul</code> and <code>add</code> operations.</p>
  
  <h1>Possible Solution</h1>
  
  <p>Maybe I can use:</p>
  
  <pre><code>if cost == "nan":
    optimizer = an empty optimizer 
  else:
    ...
    optimizer = real optimizer
  </code></pre>
  
  <p>But I cannot find the type of <code>nan</code>. How can I check a variable is <code>nan</code> or not?</p>
  
  <p>How else can I solve this problem?</p>