<h3 dir="auto">System information</h3>
<p dir="auto">See attached <a href="https://github.com/tensorflow/tensorflow/files/3584174/tf_env.txt">tf_env.txt</a> generated by <code class="notranslate">tf_env_collect.sh</code>.</p>
<p dir="auto">tf.version.VERSION <strong>1.14.0</strong><br>
tf.version.GIT_VERSION unknown</p>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">We have been successfully training Tensorflow models and loading them into Java project for inference. We have been training models using Tensorflow [1.8.0 - 1.10.0] and loading them into a gradle project compiled with 1.8.0. It worked.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="dependencies {

    compile(
            &quot;org.tensorflow:tensorflow:&quot; + &quot;1.8.0&quot;,
            &quot;com.google.guava:guava:&quot; + guavaVersion,
            &quot;com.opencsv:opencsv:&quot; + openCSVVersion,
    )"><pre class="notranslate"><code class="notranslate">dependencies {

    compile(
            "org.tensorflow:tensorflow:" + "1.8.0",
            "com.google.guava:guava:" + guavaVersion,
            "com.opencsv:opencsv:" + openCSVVersion,
    )
</code></pre></div>
<p dir="auto">Recently we have retrained the models using Tensorflow <strong>1.14.0</strong>. After replacing the older models and raising the gradle version number to 1.14.0</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="dependencies {

    compile(
            &quot;org.tensorflow:tensorflow:&quot; + &quot;1.14.0&quot;,
            &quot;com.google.guava:guava:&quot; + guavaVersion,
            &quot;com.opencsv:opencsv:&quot; + openCSVVersion,
    )"><pre class="notranslate"><code class="notranslate">dependencies {

    compile(
            "org.tensorflow:tensorflow:" + "1.14.0",
            "com.google.guava:guava:" + guavaVersion,
            "com.opencsv:opencsv:" + openCSVVersion,
    )
</code></pre></div>
<p dir="auto">one of the tests that attempts to load the model started throwing the following exception below:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="java.lang.UnsatisfiedLinkError: /tmp/tensorflow_native_libraries-1567776737834-0/libtensorflow_jni.so: /tmp/tensorflow_native_libraries-1567776737834-0/libtensorflow_jni.so: undefined symbol: _ZN11tensorflow10FileSystem20RecursivelyCreateDirERKSs

	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at org.tensorflow.NativeLibrary.load(NativeLibrary.java:101)
	at org.tensorflow.TensorFlow.init(TensorFlow.java:66)
	at org.tensorflow.TensorFlow.&lt;clinit&gt;(TensorFlow.java:70)
	at org.tensorflow.SavedModelBundle.&lt;clinit&gt;(SavedModelBundle.java:170)"><pre class="notranslate"><code class="notranslate">java.lang.UnsatisfiedLinkError: /tmp/tensorflow_native_libraries-1567776737834-0/libtensorflow_jni.so: /tmp/tensorflow_native_libraries-1567776737834-0/libtensorflow_jni.so: undefined symbol: _ZN11tensorflow10FileSystem20RecursivelyCreateDirERKSs

	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at org.tensorflow.NativeLibrary.load(NativeLibrary.java:101)
	at org.tensorflow.TensorFlow.init(TensorFlow.java:66)
	at org.tensorflow.TensorFlow.&lt;clinit&gt;(TensorFlow.java:70)
	at org.tensorflow.SavedModelBundle.&lt;clinit&gt;(SavedModelBundle.java:170)
</code></pre></div>
<p dir="auto">This does not look like a bug issue but it would be nice if you could provide some tips on how to fix this on our end. Thank you.</p>
<p dir="auto">I trying to add summary and checkpoint to my distributed tensorflow training with custom data</p>
<pre class="notranslate">summary_hook = tf.train.SummarySaverHook(save_secs=600,output_dir=FLAGS.log_dir,summary_op=summary_op)
checkpoint_hook = tf.train.CheckpointSaverHook(save_steps=test_timing,checkpoint_dir=FLAGS.log_dir,saver=saver)
with tf.train.MonitoredTrainingSession(server.target,is_chief=is_chief,hooks=[sync_replicas_hook,summary_hook,checkpoint_hook],config=sess_config) as sess:</pre>
<p dir="auto">with above I can't run the sess with error of below</p>
<pre class="notranslate">2017-08-24 19:38:31.250556: W tensorflow/core/framework/op_kernel.cc:1148] Invalid argument: Shape [-1,13] has negative dimensions
2017-08-24 19:38:31.250633: E tensorflow/core/common_runtime/executor.cc:644] Executor failed to create kernel. Invalid argument: Shape [-1,13] has negative dimensions
      [[Node: mfcc_input = Placeholder[dtype=DT_FLOAT, shape=[?,13], _device="/job:worker/replica:0/task:0/gpu:0"]()]]
2017-08-24 19:38:31.251688: W tensorflow/core/framework/op_kernel.cc:1148] Invalid argument: Shape [-1,13] has negative dimensions
2017-08-24 19:38:31.251734: E tensorflow/core/common_runtime/executor.cc:644] Executor failed to create kernel. Invalid argument: Shape [-1,13] has negative dimensions
      [[Node: mfcc_input = Placeholder[dtype=DT_FLOAT, shape=[?,13], _device="/job:worker/replica:0/task:0/gpu:0"]()]]
2017-08-24 19:38:31.253478: W tensorflow/core/framework/op_kernel.cc:1148] Invalid argument: Shape [-1,3] has negative dimensions
2017-08-24 19:38:31.253522: E tensorflow/core/common_runtime/executor.cc:644] Executor failed to create kernel. Invalid argument: Shape [-1,3] has negative dimensions
      [[Node: labels = Placeholder[dtype=DT_INT32, shape=[?,3], _device="/job:worker/replica:0/task:0/gpu:0"]()]]</pre>
<p dir="auto">how ever if I run without the summary and checkpoint hook it just work fine....<br>
I want to specify which timing and which directory to view my summary and save the checkpoint.<br>
Does anyone have an idea how to solve this problem?</p>