Stack Exchange
Question about conv2D in Android NN: the output shape limit
When I was trying the android nn code, found these in Conv2D.cpp:
tflite::Dims<4> im2colDim;                                                    \
im2colDim.sizes[3] = (int)getSizeOfDimension(outputShape, 0);                 \
im2colDim.sizes[2] = (int)getSizeOfDimension(outputShape, 1);                 \
im2colDim.sizes[1] = (int)getSizeOfDimension(outputShape, 2);                 \
im2colDim.sizes[0] = (int)inDepth * filterHeight * filterWidth;               \
                                                                              \
im2colDim.strides[0] = 1;                                                     \
for (int i = 1; i < 4; i++) {                                                 \
    im2colDim.strides[i] = im2colDim.strides[i - 1] * im2colDim.sizes[i - 1]; \
}                                                                             \
                                                                              \
Type* im2colData = nullptr;                                                   \
uint64_t im2colByteSize = sizeof(Type);                                       \
std::unique_ptr<Type[]> im2colGuard;                                          \
for (int i = 0; i < 4; i++) {                                                 \
    im2colByteSize *= im2colDim.sizes[i];                                     \
}                                                                             \
/* http://b/77982879, tflite::optimized_ops::Conv uses int for offsets */     \
if (im2colByteSize >= 0x7fffffff) {                                           \
    LOG(ERROR) << "Conv size is too large, not enough memory";                \
    return false;                                                             \
}                                                                             \
if (im2colByteSize <= kStaticBufferSize) {                                    \
    im2colData = reinterpret_cast<Type*>(static_scratch_buffer);              \
} else {                                                                      \
    im2colData = new (std::nothrow) Type[im2colByteSize / sizeof(Type)];      \
    if (im2colData == nullptr) {                                              \
        LOG(ERROR) << "Conv size is too large, not enough memory";            \
        return false;                                                         \
    }                                                                         \
    im2colGuard.reset(im2colData);                                            \
}

I wonder why the byteSize has to be checked?
code link:
https://android.googlesource.com/platform/frameworks/ml/+/refs/heads/master/nn/common/operations/Conv2D.cpp
around line 163
thanks guys

The two checks on the total size of the tensor in bytes (im2colByteSize) are to check if:

The size of the buffer is higher than the MAX positive number that can be expressed with a signed int (this is to prevent SEGV caused by tflite::optimized_ops::Conv using a signed int to express the offset)

The size is small enough to avoid to allocate memory and use the static_scratch_buffer


