<p dir="auto">I am using tranform_graph to quantize darkflow's tiny-yolo-v2 *.pb<br>
<a href="https://github.com/thtrieu/darkflow">https://github.com/thtrieu/darkflow</a></p>
<p dir="auto">found several issues</p>
<ol dir="auto">
<li>
<p dir="auto">when i use video to do inference (or images) the performance drops to very low with gpu enable<br>
from <strong>5.xx fps to 1.xx fps</strong>, and i can feel the images inference speed drops, by input with lots of pictures.</p>
</li>
<li>
<p dir="auto">after use tranform_graph's quantization, the mAP drops from <strong>68% to 13%</strong> (with my 500 pics selcted from pascal_voc2012)<br>
the function i used list below</p>
</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="../tensorflow-src/tensorflow/bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=_151875_12_28_02_36_51_tiny-yolo-v2.pb  --inputs='input' --outputs='output' --out_graph=_151875_12_28_02_36_51_tiny-yolo-v2.quantize_node.pb --transforms='remove_nodes(op=Identity, op=CheckNumerics)
  fold_constants(ignore_errors=true)
  fold_batch_norms
  fold_old_batch_norms
  quantize_weights
  quantize_nodes
  strip_unused_nodes
  merge_duplicate_nodes
  sort_by_execution_order'"><pre class="notranslate"><code class="notranslate">../tensorflow-src/tensorflow/bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=_151875_12_28_02_36_51_tiny-yolo-v2.pb  --inputs='input' --outputs='output' --out_graph=_151875_12_28_02_36_51_tiny-yolo-v2.quantize_node.pb --transforms='remove_nodes(op=Identity, op=CheckNumerics)
  fold_constants(ignore_errors=true)
  fold_batch_norms
  fold_old_batch_norms
  quantize_weights
  quantize_nodes
  strip_unused_nodes
  merge_duplicate_nodes
  sort_by_execution_order'
</code></pre></div>
<ol start="3" dir="auto">
<li>
<p dir="auto">after quantize mAP drops, i use "insert_logging" &amp; "freeze_requantization_ranges"<br>
tried to bring back mAP with that pascal_voc selected 500 pictures<br>
but the mAP drops to <strong>0%.</strong></p>
</li>
<li>
<p dir="auto">i wrote a script try to fix the logging file<br>
I go thorough logging with single image, one by one<br>
if the mAP goes up, i merge with the last highest mAP ...<br>
until the last image<br>
so comes out with a valid logging file, which bring back the mAP to <strong>62%</strong></p>
</li>
</ol>
<p dir="auto">now the questions are</p>
<ol dir="auto">
<li>is that possible to quantize yolo-tiny-v2?<br>
because the mAP and performance drops so much (even thou the mAP can be adjust back)</li>
<li>are there any other neural network models that you suggest me to use?<br>
my target is to create some quantized small models for running on ic chips</li>
<li>does that means logging and freeze_requantization_ranges might not suitable for tiny-yolo-v2?</li>
</ol>
<p dir="auto">Thanks for your time for reading this</p>
<p dir="auto">== cat /etc/issue ===============================================<br>
Linux 8fbb96ca119d 4.15.0-34-generic <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="115956927" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/37" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/37/hovercard" href="https://github.com/tensorflow/tensorflow/issues/37">#37</a>~16.04.1-Ubuntu SMP Tue Aug 28 10:44:06 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux<br>
VERSION="16.04.5 LTS (Xenial Xerus)"<br>
VERSION_ID="16.04"<br>
VERSION_CODENAME=xenial</p>
<p dir="auto">== are we in docker =============================================<br>
Yes</p>
<p dir="auto">== compiler =====================================================<br>
c++ (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609<br>
Copyright (C) 2015 Free Software Foundation, Inc.<br>
This is free software; see the source for copying conditions.  There is NO<br>
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.</p>
<p dir="auto">== uname -a =====================================================<br>
Linux 8fbb96ca119d 4.15.0-34-generic <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="115956927" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/37" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/37/hovercard" href="https://github.com/tensorflow/tensorflow/issues/37">#37</a>~16.04.1-Ubuntu SMP Tue Aug 28 10:44:06 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux</p>
<p dir="auto">== check pips ===================================================<br>
numpy                 1.15.4<br>
protobuf              3.6.1<br>
tensorflow-gpu        1.12.0</p>
<p dir="auto">== check for virtualenv =========================================<br>
False</p>
<p dir="auto">== tensorflow import ============================================<br>
tf.VERSION = 1.12.0<br>
tf.GIT_VERSION = v1.12.0-0-ga6d8ffae09<br>
tf.COMPILER_VERSION = v1.12.0-0-ga6d8ffae09<br>
Sanity check: array([1], dtype=int32)</p>
<p dir="auto">== env ==========================================================<br>
LD_LIBRARY_PATH /usr/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64<br>
DYLD_LIBRARY_PATH is unset</p>
<p dir="auto">== nvidia-smi ===================================================<br>
Sat Jan 26 05:56:00 2019<br>
+-----------------------------------------------------------------------------+<br>
| NVIDIA-SMI 390.87                 Driver Version: 390.87                    |<br>
|-------------------------------+----------------------+----------------------+<br>
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |<br>
|===============================+======================+======================|<br>
|   0  Quadro GV100        Off  | 00000000:18:00.0 Off |                  Off |<br>
| 40%   54C    P0    50W / 250W |      1MiB / 32508MiB |      0%      Default |<br>
+-------------------------------+----------------------+----------------------+<br>
|   1  Quadro GV100        Off  | 00000000:3B:00.0 Off |                  Off |<br>
| 40%   54C    P0    50W / 250W |      0MiB / 32508MiB |      5%      Default |<br>
+-------------------------------+----------------------+----------------------+</p>
<p dir="auto">+-----------------------------------------------------------------------------+<br>
| Processes:                                                       GPU Memory |<br>
|  GPU       PID   Type   Process name                             Usage      |<br>
|=============================================================================|<br>
|  No running processes found                                                 |<br>
+-----------------------------------------------------------------------------+</p>
<p dir="auto">== cuda libs  ===================================================<br>
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart_static.a<br>
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176</p>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code</strong>: yes</li>
<li><strong>OS Platform and Distribution</strong>: Linux Fedora 16.04</li>
<li><strong>TensorFlow installed from</strong>: binary</li>
<li><strong>TensorFlow version</strong>: v1.4.0-19-ga52c8d9, 1.4.1</li>
<li><strong>Python version</strong>: 2.7</li>
<li><strong>Bazel version</strong>: N/A</li>
<li><strong>CUDA/cuDNN version</strong>: cuda_8.0.61,  cudnnv5</li>
<li><strong>GPU model and memory</strong>: GeForce GTX TITAN X ,   12207MiB</li>
<li><strong>Exact command to reproduce</strong>:</li>
</ul>
<p dir="auto">We have a computer cluster that some of the machines have GPU and others don't. I have installed tensorflow-gpu-1.4 from wheel file in a virtualenv in a folder on the file-server which means that it is accessible on all the machines in the cluster.<br>
My program is a distributed software which means that some of the tasks are done on all the cluster nodes. (data generation and configuration) and the machine learning part is only done on the machines with GPU. I activate the aforementioned virtualenv before running the servers on nodes of the cluster so all the nodes are running inside the same virtual environment.<br>
On the machines that have a GPU when I import tensorflow everything works fine, but when I import the tensorflow on the machines that do not have the gpu (and Cuda is not installed on them) I get following error:</p>
<blockquote>
<p dir="auto">In [1]: import tensorflow as tf<br>
ImportError                               Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 import tensorflow as tf</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/<strong>init</strong>.py in ()<br>
22<br>
23 # pylint: disable=wildcard-import<br>
---&gt; 24 from tensorflow.python import *<br>
25 # pylint: enable=wildcard-import<br>
26</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/<strong>init</strong>.py in ()<br>
47 import numpy as np<br>
48<br>
---&gt; 49 from tensorflow.python import pywrap_tensorflow<br>
50<br>
51 # Protocol buffers</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py in ()<br>
70 for some common reasons and solutions.  Include the entire stack trace<br>
71 above this error message when asking for help.""" % traceback.format_exc()<br>
---&gt; 72   raise ImportError(msg)<br>
73<br>
74 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long</p>
<p dir="auto">ImportError: Traceback (most recent call last):<br>
File "virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <br>
from tensorflow.python.pywrap_tensorflow_internal import *<br>
File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <br>
_pywrap_tensorflow_internal = swig_import_helper()<br>
File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper<br>
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)<br>
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory</p>
<p dir="auto">Failed to load the native TensorFlow runtime.</p>
<p dir="auto">See <a href="https://www.tensorflow.org/install/install_sources#common_installation_problems" rel="nofollow">https://www.tensorflow.org/install/install_sources#common_installation_problems</a></p>
<p dir="auto">for some common reasons and solutions.  Include the entire stack trace<br>
above this error message when asking for help.</p>
</blockquote>
<p dir="auto">I am aware that tensorflow-gpu is statically linked to the Cuda libraries and I installed a local version of Cuda using the runfile in a folder on the file-server (accessible to all the nodes) and added its path to the LD_LIBRARY_PATH and PATH and now importing tf gives me the following error:</p>
<blockquote>
<p dir="auto">2018-02-15 01:41:58.519693: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU</p>
</blockquote>
<blockquote>
<p dir="auto">supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA<br>
2018-02-15 01:41:58.519972: E tensorflow/stream_executor/cuda/cuda_driver.cc:406] failed call to cuInit: CUresult(-1)<br>
2018-02-15 01:41:58.520007: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:158] retrieving CUDA diagnostic information for host: maserati<br>
2018-02-15 01:41:58.520017: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: maserati<br>
2018-02-15 01:41:58.520140: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: Not found: was unable to find libcuda.so DSO loaded into this program<br>
2018-02-15 01:41:58.520167: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:369] driver version file contents: """NVRM version: NVIDIA UNIX x86_64 Kernel Module  384.111  Tue Dec 19 23:51:45 PST 2017<br>
GCC version:  gcc version 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC)<br>
"""<br>
2018-02-15 01:41:58.520191: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 384.111.0</p>
</blockquote>
<p dir="auto">And when i run a hello world script it gives me the following error (which clearly means it cant run anything on gpu because of previous error):</p>
<blockquote>
<p dir="auto">In [5]: with tf.device("/GPU:0"):<br>
...:     hello = tf.constant('Hello, TensorFlow!')<br>
...:     sess = tf.Session()<br>
...:     print(sess.run(hello))<br>
...:<br>
InvalidArgumentError                      Traceback (most recent call last)<br>
 in ()<br>
2     hello = tf.constant('Hello, TensorFlow!')<br>
3     sess = tf.Session()<br>
----&gt; 4     print(sess.run(hello))<br>
5</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict, options, run_metadata)<br>
887     try:<br>
888       result = self._run(None, fetches, feed_dict, options_ptr,<br>
--&gt; 889                          run_metadata_ptr)<br>
890       if run_metadata:<br>
891         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _run(self, handle, fetches, feed_dict, options, run_metadata)<br>
1118     if final_fetches or final_targets or (handle and feed_dict_tensor):<br>
1119       results = self._do_run(handle, final_targets, final_fetches,<br>
-&gt; 1120                              feed_dict_tensor, options, run_metadata)<br>
1121     else:<br>
1122       results = []</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)<br>
1315     if handle is None:<br>
1316       return self._do_call(_run_fn, self._session, feeds, fetches, targets,<br>
-&gt; 1317                            options, run_metadata)<br>
1318     else:<br>
1319       return self._do_call(_prun_fn, self._session, handle, feeds, fetches)</p>
<p dir="auto">/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_call(self, fn, *args)<br>
1334         except KeyError:<br>
1335           pass<br>
-&gt; 1336       raise type(e)(node_def, op, message)<br>
1337<br>
1338   def _extend_graph(self):</p>
<p dir="auto">InvalidArgumentError: Cannot assign a device for operation 'Const_1': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.<br>
[[Node: Const_1 = Const<a href="">dtype=DT_STRING, value=Tensor&lt;type: string shape: [] values: Hello, TensorFlow!&gt;, _device="/device:GPU:0"</a>]]</p>
<p dir="auto">Caused by op u'Const_1', defined at:<br>
File "/virtualenv/bin/ipython", line 11, in <br>
sys.exit(start_ipython())<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/<strong>init</strong>.py", line 119, in start_ipython<br>
return launch_new_instance(argv=argv, **kwargs)<br>
File "/virtualenv/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance<br>
app.start()<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 355, in start<br>
self.shell.mainloop()<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 493, in mainloop<br>
self.interact()<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 484, in interact<br>
self.run_cell(code, store_history=True)<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2718, in run_cell<br>
interactivity=interactivity, compiler=compiler, result=result)<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes<br>
if self.run_code(code, result):<br>
File "/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code<br>
exec(code_obj, self.user_global_ns, self.user_ns)<br>
File "", line 2, in <br>
hello = tf.constant('Hello, TensorFlow!')<br>
File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant<br>
name=name).outputs[0]<br>
File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op<br>
op_def=op_def)<br>
File "/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in <strong>init</strong><br>
self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access</p>
<p dir="auto">InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'Const_1': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.<br>
[[Node: Const_1 = Const<a href="">dtype=DT_STRING, value=Tensor&lt;type: string shape: [] values: Hello, TensorFlow!&gt;, _device="/device:GPU:0"</a>]]</p>
</blockquote>
<p dir="auto">I dont wan't/ can't install cuda on non-gpu machines is there any work around for this issue?</p>