<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): RHEL7 3.10.0-862.11.6.el7.x86_64</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: -</li>
<li>TensorFlow installed from (source or binary): conda</li>
<li>TensorFlow version (use command below): 1.11.0</li>
<li>Python version: Python 3.6.6 :: Anaconda, Inc.</li>
<li>Bazel version (if compiling from source): -</li>
<li>GCC/Compiler version (if compiling from source): -</li>
<li>CUDA/cuDNN version: -</li>
<li>GPU model and memory: -</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong></p>
<p dir="auto">When creating a custom <code class="notranslate">tf.keras.Model</code> with custom <code class="notranslate">build()</code> function, the variable name scopes seem not to propagate correctly in <code class="notranslate">tf.keras.Dense</code> layers.</p>
<p dir="auto"><strong>Describe the expected behavior</strong></p>
<p dir="auto">Variables of inner <code class="notranslate">tf.keras.Dense</code> layers should have different names, such as:<br>
<code class="notranslate">dense/kernel:0</code><br>
<code class="notranslate">dense/bias:0</code><br>
<code class="notranslate">dense_1/kernel:0</code><br>
<code class="notranslate">dense_1/bias:0</code></p>
<p dir="auto">Right now they have the same name:<br>
<code class="notranslate">kernel:0</code><br>
<code class="notranslate">bias:0</code><br>
<code class="notranslate">kernel:0</code><br>
<code class="notranslate">bias:0</code></p>
<p dir="auto">This is problematic for saving / loading parameters.</p>
<p dir="auto"><strong>Code to reproduce the issue</strong><br>
Provide a reproducible test case that is the bare minimum necessary to generate the problem.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
import os

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

tf.enable_eager_execution()
tf.executing_eagerly()


class MyModel(tf.keras.Model):
    def __init__(self):
        super().__init__(name='mymodel')
        
        self.f = tf.keras.layers.Dense(units=10)
        self.g = tf.keras.layers.Dense(units=10)

#         Workaround
#         self.f = tf.keras.Sequential([tf.keras.layers.Dense(units=10)])
#         self.g = tf.keras.Sequential([tf.keras.layers.Dense(units=10)])

    def build(self, input_shapes):
        self.f.build(input_shapes[0])
        self.g.build(input_shapes[1])
        self.built = True
    
    def call(self, x, y):
        return self.f(x) + self.g(y)


model = MyModel()
model.build([(None, 5), (None, 3)])

for v in model.variables:
    print(v.name)"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
import os

os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

tf.enable_eager_execution()
tf.executing_eagerly()


class MyModel(tf.keras.Model):
    def __init__(self):
        super().__init__(name='mymodel')
        
        self.f = tf.keras.layers.Dense(units=10)
        self.g = tf.keras.layers.Dense(units=10)

#         Workaround
#         self.f = tf.keras.Sequential([tf.keras.layers.Dense(units=10)])
#         self.g = tf.keras.Sequential([tf.keras.layers.Dense(units=10)])

    def build(self, input_shapes):
        self.f.build(input_shapes[0])
        self.g.build(input_shapes[1])
        self.built = True
    
    def call(self, x, y):
        return self.f(x) + self.g(y)


model = MyModel()
model.build([(None, 5), (None, 3)])

for v in model.variables:
    print(v.name)
</code></pre></div>
<p dir="auto"><strong>Other info / logs</strong><br>
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.</p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Deep Learning VM<br>
Version: m10<br>
Based on: Debian GNU/Linux 9.5 (stretch) (GNU/Linux 4.9.0-8-amd64 x86_64\n)</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:</li>
<li>TensorFlow installed from (source or binary): deep-learning image</li>
<li>TensorFlow version (use command below): 1.11</li>
<li>Python version: 2.7</li>
<li>Bazel version (if compiling from source):</li>
<li>GCC/Compiler version (if compiling from source):</li>
<li>CUDA/cuDNN version:</li>
<li>GPU model and memory: N/A</li>
</ul>
<p dir="auto">You can collect some of this information using our environment capture <a href="https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh">script</a><br>
You can also obtain the TensorFlow version with<br>
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"</p>
<p dir="auto"><strong>Describe the current behavior</strong><br>
Running deep model and some wide linear models. Inference performance is very bad. 2-4x slower relative to running inference without MKL<br>
<strong>Describe the expected behavior</strong><br>
Performance should actually improve with intel mkl instruction set .</p>
<p dir="auto"><strong>Code to reproduce the issue</strong><br>
code for deep and wide linear model. or logistic regression example code from tensorflow example</p>
<p dir="auto"><strong>Other info / logs</strong><br>
When running using google deep learning image version M9 on gpu machine  (image : tf-latest-cu92, version M9) . Note : the inference is only running on cpu as i turn off the visibility for cuda devices, So the tensorflow code runs only runs on cpu. The image family says they are intel optimized packages but when i rung the benchmarks with verbosity on , i do not observe any mkl related stuff.</p>
<p dir="auto">I start another deep learning image (tf-latest-cpu , version M10): Running exact same code on this machine with environment variable (export MKL_VERBOSE=1): I can observe a lot of openMP thread settings , KMP_xxx settings and mkl instructions logged with some timing information. I didn't observe any such thing in the M9 gpu image , even though in both place when i execute command   i observe following logs:<br>
M9 gpu image<br>
Numpy + Intel(R) MKL: THREADING LAYER: (null)<br>
Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime<br>
Numpy + Intel(R) MKL: preloading libiomp5.so runtime<br>
MKL_VERBOSE Intel(R) MKL 2019.0 Product build 20180829 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 2 (Intel(R) AVX2) enabled processors, Lnx 2.20GHz lp64 intel_thread<br>
MKL_VERBOSE SDOT(2,0x55fd25117d40,1,0x55fd25117d40,1) 1.61ms CNR:OFF Dyn:1 FastMM:1 TID:0  NThr:16<br>
1.11.0</p>
<p dir="auto">M10 cpu image :<br>
Numpy + Intel(R) MKL: THREADING LAYER: (null)<br>
Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime<br>
Numpy + Intel(R) MKL: preloading libiomp5.so runtime</p>
<p dir="auto">User settings:</p>
<p dir="auto">KMP_AFFINITY=granularity=fine,verbose,compact,1,0<br>
KMP_BLOCKTIME=0<br>
KMP_SETTINGS=1<br>
OMP_NUM_THREADS=32</p>
<p dir="auto">Effective settings:</p>
<p dir="auto">KMP_ABORT_DELAY=0<br>
KMP_ADAPTIVE_LOCK_PROPS='1,1024'<br>
KMP_ALIGN_ALLOC=64<br>
KMP_ALL_THREADPRIVATE=128<br>
KMP_ATOMIC_MODE=2<br>
KMP_BLOCKTIME=0<br>
KMP_CPUINFO_FILE: value is not defined<br>
KMP_DETERMINISTIC_REDUCTION=false<br>
KMP_DEVICE_THREAD_LIMIT=2147483647<br>
KMP_DISP_HAND_THREAD=false<br>
KMP_DISP_NUM_BUFFERS=7<br>
KMP_DUPLICATE_LIB_OK=false<br>
KMP_FORCE_REDUCTION: value is not defined<br>
KMP_FOREIGN_THREADS_THREADPRIVATE=true<br>
KMP_FORKJOIN_BARRIER='2,2'<br>
KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'<br>
KMP_FORKJOIN_FRAMES=true<br>
KMP_FORKJOIN_FRAMES_MODE=3<br>
KMP_GTID_MODE=3<br>
KMP_HANDLE_SIGNALS=false<br>
KMP_HOT_TEAMS_MAX_LEVEL=1<br>
KMP_HOT_TEAMS_MODE=0<br>
KMP_INIT_AT_FORK=true<br>
KMP_INIT_WAIT=2048<br>
KMP_ITT_PREPARE_DELAY=0<br>
KMP_LIBRARY=throughput<br>
KMP_LOCK_KIND=queuing<br>
KMP_MALLOC_POOL_INCR=1M<br>
KMP_NEXT_WAIT=1024<br>
KMP_NUM_LOCKS_IN_BLOCK=1<br>
KMP_PLAIN_BARRIER='2,2'<br>
KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'<br>
KMP_REDUCTION_BARRIER='1,1'<br>
KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'<br>
KMP_SCHEDULE='static,balanced;guided,iterative'<br>
KMP_SETTINGS=true<br>
KMP_SPIN_BACKOFF_PARAMS='4096,100'<br>
KMP_STACKOFFSET=64<br>
KMP_STACKPAD=0<br>
KMP_STACKSIZE=4M<br>
KMP_STORAGE_MAP=false<br>
KMP_TASKING=2<br>
KMP_TASKLOOP_MIN_TASKS=0<br>
KMP_TASK_STEALING_CONSTRAINT=1<br>
KMP_TEAMS_THREAD_LIMIT=32<br>
KMP_TOPOLOGY_METHOD=all<br>
KMP_USER_LEVEL_MWAIT=false<br>
KMP_VERSION=false<br>
KMP_WARNINGS=true<br>
OMP_AFFINITY_FORMAT='OMP: pid %P tid %T thread %n bound to OS proc set {%a}'<br>
OMP_ALLOCATOR=omp_default_mem_alloc<br>
OMP_CANCELLATION=false<br>
OMP_DEBUG=disabled<br>
OMP_DEFAULT_DEVICE=0<br>
OMP_DISPLAY_AFFINITY=false<br>
OMP_DISPLAY_ENV=false<br>
OMP_DYNAMIC=false<br>
OMP_MAX_ACTIVE_LEVELS=2147483647<br>
OMP_MAX_TASK_PRIORITY=0<br>
OMP_NESTED=false<br>
OMP_NUM_THREADS='32'<br>
OMP_PLACES: value is not defined<br>
OMP_PROC_BIND='intel'<br>
OMP_SCHEDULE='static'<br>
OMP_STACKSIZE=4M<br>
OMP_TARGET_OFFLOAD=DEFAULT<br>
OMP_THREAD_LIMIT=2147483647<br>
OMP_TOOL=enabled<br>
OMP_TOOL_LIBRARIES: value is not defined<br>
OMP_WAIT_POLICY=PASSIVE<br>
KMP_AFFINITY='verbose,warnings,respect,granularity=fine,compact,1,0'</p>
<p dir="auto">OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116869296" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/212" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/212/hovercard" href="https://github.com/tensorflow/tensorflow/issues/212">#212</a>: KMP_AFFINITY: decoding x2APIC ids.<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116841232" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/210" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/210/hovercard" href="https://github.com/tensorflow/tensorflow/issues/210">#210</a>: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116433852" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/154" data-hovercard-type="pull_request" data-hovercard-url="/tensorflow/tensorflow/pull/154/hovercard" href="https://github.com/tensorflow/tensorflow/pull/154">#154</a>: KMP_AFFINITY: Initial OS proc set respected: 0-31<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116441460" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/156" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/156/hovercard" href="https://github.com/tensorflow/tensorflow/issues/156">#156</a>: KMP_AFFINITY: 32 available OS procs<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116444691" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/157" data-hovercard-type="pull_request" data-hovercard-url="/tensorflow/tensorflow/pull/157/hovercard" href="https://github.com/tensorflow/tensorflow/pull/157">#157</a>: KMP_AFFINITY: Uniform topology<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116597495" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/179" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/179/hovercard" href="https://github.com/tensorflow/tensorflow/issues/179">#179</a>: KMP_AFFINITY: 1 packages x 16 cores/pkg x 2 threads/core (16 total cores)<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116895438" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/214" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/214/hovercard" href="https://github.com/tensorflow/tensorflow/issues/214">#214</a>: KMP_AFFINITY: OS proc to physical thread map:<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 0 maps to package 0 core 0 thread 0<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 16 maps to package 0 core 0 thread 1<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 1 maps to package 0 core 1 thread 0<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 17 maps to package 0 core 1 thread 1<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 2 maps to package 0 core 2 thread 0<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 18 maps to package 0 core 2 thread 1<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 3 maps to package 0 core 3 thread 0<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 19 maps to package 0 core 3 thread 1<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 4 maps to package 0 core 4 thread 0<br>
OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="116516798" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/171" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/171/hovercard" href="https://github.com/tensorflow/tensorflow/issues/171">#171</a>: KMP_AFFINITY: OS proc 20 maps to package 0 core 4 thread 1</p>
<p dir="auto">OMP: Info <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="117261381" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/250" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/250/hovercard" href="https://github.com/tensorflow/tensorflow/issues/250">#250</a>: KMP_AFFINITY: pid 8331 tid 8331 thread 0 bound to OS proc set 0<br>
MKL_VERBOSE Intel(R) MKL 2019.0 Product build 20180829 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 2 (Intel(R) AVX2) enabled processors, Lnx 2.20GHz lp64 intel_thread<br>
MKL_VERBOSE SDOT(2,0x5622b7736500,1,0x5622b7736500,1) 2.54ms CNR:OFF Dyn:1 FastMM:1 TID:0  NThr:16<br>
1.11.0</p>
<p dir="auto">So i assume intel mkl is being used in M10 image where as mkl is not being used in the M9 image (Note: i have turned off visibility for cuda devices so only cpu inference is being compared) . I observe 2-4x performance degradation with intel mkl.<br>
The mkl suggested flags are appropriate:<br>
KMP_AFFINITY=granularity=fine,verbose,compact,1,0<br>
KMP_BLOCKTIME=0<br>
KMP_SETTINGS=1<br>
OMP_NUM_THREADS=32</p>
<p dir="auto">Any ideas on how to debug the root cause and get the maximum performance for my models.</p>