<p dir="auto">Attention models get more popular. In many of my experiments with attention, I end up using Softmax over multiple dimensions.</p>
<p dir="auto">tf.nn.softmax() supports only one dimension. It would be nice to expand the API with another argument, softmax_dim, for softmax over multiple dimension, say over 2 (images) or 3 (CT-scans, videos)</p>
<p dir="auto">I;d like to here what you think? DO more people end up writing softmax code themselves?</p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: iPhone 7 simulator</li>
<li>TensorFlow installed from (source or binary): source</li>
<li>TensorFlow version (use command below): master</li>
<li>Python version: 2.7</li>
<li>Bazel version (if compiling from source): 0.15.0</li>
<li>GCC/Compiler version (if compiling from source): None</li>
<li>CUDA/cuDNN version: None</li>
<li>GPU model and memory: None</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong><br>
I convert the TF model to TFLite model by tflite_convert command with float inference_type, and input_shapes set to 1,W,H,3; when I run the model with input shape W*H, <strong>it works well</strong>; However, when I <strong>change</strong> the input shapes to different values, it would give error like "tensorflow/contrib/lite/kernels/kernel_util.cc:125 d1 == d2 || d1 == 1 || d2 == 1 was not true; Node number xx (ADD) failed to prepare" for tf.add op. Similar errors appear for other ops like CONCAT and RESHAPE. When I set breakpoints in tflite codes, it seems to always check/use the tensor shapes when converting the model, even the input shape has changed.</p>
<p dir="auto">Part of the codes are like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="int input = interpreter-&gt;inputs()[0];
    interpreter-&gt;ResizeInputTensor(input, new_input_sizes);
    if (interpreter-&gt;AllocateTensors() != kTfLiteOk) {
        NSLog(@&quot;Failed to allocate tensors for model.&quot;);
        exit(-1);
    }"><pre class="notranslate"><code class="notranslate">int input = interpreter-&gt;inputs()[0];
    interpreter-&gt;ResizeInputTensor(input, new_input_sizes);
    if (interpreter-&gt;AllocateTensors() != kTfLiteOk) {
        NSLog(@"Failed to allocate tensors for model.");
        exit(-1);
    }
</code></pre></div>