<h5 dir="auto">Detailed description</h5>
<p dir="auto">Intel's Inference Engine supports asynchronous forward passes using multiple inference requests. Let users enable async. networks inference.</p>
<h4 dir="auto">Initial proposal</h4>
<ul dir="auto">
<li>Requires adding a new method <code class="notranslate">Net::enableAsync(bool)</code> to specify if we want to execute network asynchronously.</li>
<li><code class="notranslate">Net::setInput</code> is used to set input data in the internal inputs queue and <code class="notranslate">Net::forward</code> to retrieve outputs in the same order.</li>
<li><code class="notranslate">Net::forward</code> returns an empty Mat in case of output is not ready =&gt; User need to check that output is not empty.</li>
</ul>
<p dir="auto">Considered scenarios of Net in async mode:</p>
<h5 dir="auto">Standard loop</h5>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="net.enableAsync(1);

net.setInput(inp1);
out1 = net.forward();

net.setInput(inp2);
out = net.forward();  // out is output for inp1 if out1 is empty. Otherwise it's output for inp2
..."><pre class="notranslate"><code class="notranslate">net.enableAsync(1);

net.setInput(inp1);
out1 = net.forward();

net.setInput(inp2);
out = net.forward();  // out is output for inp1 if out1 is empty. Otherwise it's output for inp2
...
</code></pre></div>
<p dir="auto">This way user should store inputs and pop them by ready outputs (i.e. render detections).</p>
<h5 dir="auto">Prefetch</h5>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="net.enableAsync(1);

net.setInput(inp1);
net.setInput(inp2);
...

out1 = net.forward();
out = net.forward();  // out is output for inp1 if out1 is empty. Otherwise it's output for inp2"><pre class="notranslate"><code class="notranslate">net.enableAsync(1);

net.setInput(inp1);
net.setInput(inp2);
...

out1 = net.forward();
out = net.forward();  // out is output for inp1 if out1 is empty. Otherwise it's output for inp2
</code></pre></div>
<h5 dir="auto">Syncing</h5>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="net.enableAsync(1);

net.setInput(inp1);

out = net.forward();  // assume out is empty here (not ready)
out = net.forward();  // assume out is not ready again

net.enableAsync(0);
out = net.forward();  // wait here until output is ready"><pre class="notranslate"><code class="notranslate">net.enableAsync(1);

net.setInput(inp1);

out = net.forward();  // assume out is empty here (not ready)
out = net.forward();  // assume out is not ready again

net.enableAsync(0);
out = net.forward();  // wait here until output is ready
</code></pre></div>
<p dir="auto">Syncing let users control maximum number of asynchronous requests. In example, if number of fetched inputs equals to specified maximum number of requests, we disable async mode and wait for the output of the first input in queue. Note that it doen't disable asynchronous computations at that time but specifies that next <code class="notranslate">Net::forward</code> invocation should return output.</p>
<h5 dir="auto">Steps to reproduce</h5>
<h5 dir="auto">System information (version)</h5>

<ul dir="auto">
<li>OpenCV =&gt; OpenCV online documentation</li>
<li>Operating System / Platform =&gt; Not applicable</li>
<li>Compiler =&gt; Python</li>
</ul>
<h5 dir="auto">Detailed description</h5>
<p dir="auto">The Python code for Otsu's binarization gven in the online documentation for OpenCV Image Thresholding at<br>
<a href="https://docs.opencv.org/3.4/d7/d4d/tutorial_py_thresholding.html" rel="nofollow">https://docs.opencv.org/3.4/d7/d4d/tutorial_py_thresholding.html</a><br>
requires the following corrections:</p>
<ol dir="auto">
<li>
<p dir="auto">Histogram normalization should be done by dividing by the sum, not max:<br>
hist_norm = hist.ravel()/hist.sum()</p>
</li>
<li>
<p dir="auto">The computation of mean and variances can generate the division-by-zero run-time error when either q1 or q2 is zero.  To avoid this, we must include the statement<br>
if  q1 &lt; 1.e-6  or  q2  &lt; 1.e-6:  continue</p>
</li>
</ol>
<h5 dir="auto">Corrected code</h5>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="img = cv.imread('noisy2.png',0)
blur = cv.GaussianBlur(img,(5,5),0)

hist = cv.calcHist([blur],[0],None,[256],[0,256])
hist_norm = hist.ravel()/hist.sum()
Q = hist_norm.cumsum()
bins = np.arange(256)
fn_min = np.inf
thresh = -1
for i in xrange(1,256):
    p1,p2 = np.hsplit(hist_norm,[i]) # probabilities
    q1,q2 = Q[i],Q[255]-Q[i] # cum sum of classes
    if  q1 &lt; 1.e-6  or  q2  &lt; 1.e-6:  continue
    b1,b2 = np.hsplit(bins,[i]) # weights

    m1,m2 = np.sum(p1*b1)/q1, np.sum(p2*b2)/q2
    v1,v2 = np.sum(((b1-m1)**2)*p1)/q1,np.sum(((b2-m2)**2)*p2)/q2

    fn = v1*q1 + v2*q2
    if fn &lt; fn_min:
        fn_min = fn
        thresh = i

ret, otsu = cv.threshold(blur,0,255,cv.THRESH_BINARY+cv.THRESH_OTSU)
print( &quot;{} {}&quot;.format(thresh,ret) )"><pre class="notranslate"><code class="notranslate">img = cv.imread('noisy2.png',0)
blur = cv.GaussianBlur(img,(5,5),0)

hist = cv.calcHist([blur],[0],None,[256],[0,256])
hist_norm = hist.ravel()/hist.sum()
Q = hist_norm.cumsum()
bins = np.arange(256)
fn_min = np.inf
thresh = -1
for i in xrange(1,256):
    p1,p2 = np.hsplit(hist_norm,[i]) # probabilities
    q1,q2 = Q[i],Q[255]-Q[i] # cum sum of classes
    if  q1 &lt; 1.e-6  or  q2  &lt; 1.e-6:  continue
    b1,b2 = np.hsplit(bins,[i]) # weights

    m1,m2 = np.sum(p1*b1)/q1, np.sum(p2*b2)/q2
    v1,v2 = np.sum(((b1-m1)**2)*p1)/q1,np.sum(((b2-m2)**2)*p2)/q2

    fn = v1*q1 + v2*q2
    if fn &lt; fn_min:
        fn_min = fn
        thresh = i

ret, otsu = cv.threshold(blur,0,255,cv.THRESH_BINARY+cv.THRESH_OTSU)
print( "{} {}".format(thresh,ret) )
</code></pre></div>