Hello, I recently learned that gradients and backprop for grouped convolution is not supported on CPU, as discussed in the following github threads:

  

      github.com/keras-team/keras
  

  
    
  
	  
  

  
    
      group conv2d can't backprop properly 1
    

    
      
        opened 
        
          
        
        Nov 28, 2021
      
      

        
          closed 
        
          
        
        Dec 1, 2021
      
        

      
        
          
          breadbread1984
        
      
    

    
        
          type:support
        
        
          stat:awaiting response
        
    
  


  
    Please go to TF Forum for help and support:

https://discuss.tensorflow.org/ta…g/keras

If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with the documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

**Here's why we have that policy:**.

Keras developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

**System information**.
- Have I written custom code (as opposed to using a stock example script provided in Keras): no
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 20.04
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): 2.7.0
- Python version: 3.8.0
- Bazel version (if compiling from source): n/a
- GPU model and memory: n/a
- Exact command to reproduce: n/a

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with:
python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

v2.7.0-rc1-69-gc256c071bb2 2.7.0

**Describe the problem**.

Describe the problem clearly here. Be sure to convey here why it's a bug in Keras or why the requested feature is needed.

**Describe the current behavior**.

Group tf.keras.layers.Conv2D can forward propagate without problem but can't backward propagate properly.

**Describe the expected behavior**.

Group Conv2D should back propagate properly. 

**[Contributing](https://github.com/keras-team/keras/blob/master/CONTRIBUTING.md)**.

- Do you want to contribute a PR? (yes/no): no
- If yes, please read [this page](https://github.com/keras-team/keras/blob/master/CONTRIBUTING.md) for instructions
- Briefly describe your candidate solution(if contributing):

**Standalone code to reproduce the issue**.

Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.

```python
import tensorflow as tf;

inputs = tf.random.normal(shape = (4,224,224,256));
conv2d = tf.keras.layers.Conv2D(256, (3,3), groups = 4, padding = 'same');
with tf.GradientTape() as tape:
  outputs = conv2d(inputs);
# NOTE: error occurs here
grads = tape.gradient(outputs, conv2d.trainable_variables);
```

**Source code / logs**.

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

```shell
Traceback (most recent call last):
  File "test.py", line 27, in <module>
    grads = tape.gradient(outputs, conv2d.trainable_weights);
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py", line 1084, in gradient
    flat_grad = imperative_grad.imperative_grad(
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/eager/imperative_grad.py", line 71, in imperative_grad
    return pywrap_tfe.TFE_Py_TapeGradient(
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py", line 159, in _gradient_function
    return grad_fn(mock_op, *out_grads)
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/ops/nn_grad.py", line 581, in _Conv2DGrad
    gen_nn_ops.conv2d_backprop_input(
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1247, in conv2d_backprop_input
    _ops.raise_from_not_ok_status(e, name)
  File "/home/xieyi/.local/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 7107, in raise_from_not_ok_status
    raise core._status_to_exception(e) from None  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: Computed input depth 256 doesn't match filter input depth 64 [Op:Conv2DBackpropInput]

```
  

  

  
    
    
  

  


  

      github.com/tensorflow/tensorflow
  

  
    
  
	  
  

  
    
      Grouped convolutions generate seriously obscure errors on CPU
    

    
      
        opened 
        
          
        
        Sep 3, 2021
      
      


      
        
          
          frgfm
        
      
    

    
        
          stat:awaiting tensorflower
        
        
          type:bug
        
        
          comp:ops
        
        
          TF 2.5
        
    
  


  
    Hello there :wave: 

Today I ran into a cumbersome error that only happens whe…n running on CPU instead of GPUs. I tracked the source of the error to grouped convolutions and managed to make a reproducible minimal snippet. I happened to suspect that it was because of grouped convolutions since I ran into some problems a few days ago with those using SavedModels but it's pure luck. 

It would be good to improve the error message or even get this fixed if possible :pray: 

Happy to help provided some directions!

**System information**
- Have I written custom code: yes, the code snippet
- OS Platform and Distribution: Linux Ubuntu 20.04
- TensorFlow installed from: binary, via pip
- TensorFlow version: 2.5.0
- Python version: 3.8
- CUDA/cuDNN version: CUDA 11.4 (cuDNN 8.2.0)
- GPU model and memory: NVIDIA GeForce RTX 2070 with Max-Q Design

**Describe the current behavior**

As of now, running the snippet further down below throws an error on CPU but not on GPU.

**Describe the expected behavior**

Simple:
- having a better error (pointing the lack of support of grouped convolutions on CPU)
- or even better, if that could get fixed :) 

**Standalone code to reproduce the issue**

```python
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential

samples = tf.zeros((1, 256, 256, 3), dtype=tf.float32)
model = Sequential([layers.Conv2D(18, padding='same', kernel_size=3, groups=1), layers.GlobalAveragePooling2D(), layers.Dense(1)])
trouble_model = Sequential([layers.Conv2D(18, padding='same', kernel_size=3, groups=3), layers.GlobalAveragePooling2D(), layers.Dense(1)])

# Backprop on classic model
with tf.GradientTape() as tape:
    out = model(samples, training=True)
grads = tape.gradient(out, model.trainable_weights)

# Now with grouped conv
with tf.GradientTape() as tape:
    out = trouble_model(samples, training=True)
grads = tape.gradient(out, trouble_model.trainable_weights)
```

which runs successfully on GPU but on CPU throws the following:
```
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-1-e03a8706f9a2> in <module>
     19 with tf.GradientTape() as tape:
     20     out = trouble_model(samples, training=True)
---> 21 grads = tape.gradient(out, trouble_model.trainable_weights)

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
   1072                           for x in nest.flatten(output_gradients)]
   1073 
-> 1074     flat_grad = imperative_grad.imperative_grad(
   1075         self._tape,
   1076         flat_targets,

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, sources_raw, unconnected_gradients)
     69         "Unknown value for unconnected_gradients: %r" % unconnected_gradients)
     70 
---> 71   return pywrap_tfe.TFE_Py_TapeGradient(
     72       tape._tape,  # pylint: disable=protected-access
     73       target,

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/backprop.py in _gradient_function(op_name, attr_tuple, num_inputs, inputs, outputs, out_grads, skip_input_indices, forward_pass_name_scope)
    157       gradient_name_scope += forward_pass_name_scope + "/"
    158     with ops.name_scope(gradient_name_scope):
--> 159       return grad_fn(mock_op, *out_grads)
    160   else:
    161     return grad_fn(mock_op, *out_grads)

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/nn_grad.py in _Conv2DGrad(op, grad)
    579   # in Eager mode.
    580   return [
--> 581       gen_nn_ops.conv2d_backprop_input(
    582           shape_0,
    583           op.inputs[1],

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/gen_nn_ops.py in conv2d_backprop_input(input_sizes, filter, out_backprop, strides, padding, use_cudnn_on_gpu, explicit_paddings, data_format, dilations, name)
   1245       return _result
   1246     except _core._NotOkStatusException as e:
-> 1247       _ops.raise_from_not_ok_status(e, name)
   1248     except _core._FallbackException:
   1249       pass

~/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6895   message = e.message + (" name: " + name if name is not None else "")
   6896   # pylint: disable=protected-access
-> 6897   six.raise_from(core._status_to_exception(e.code, message), None)
   6898   # pylint: enable=protected-access
   6899 

~/miniconda3/lib/python3.8/site-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: Computed input depth 3 doesn't match filter input depth 1 [Op:Conv2DBackpropInput]
```
  

  

  
    
    
  

  


Nothing on the documentation for convolution with grouping indicated I would run into this issue, and the error message that came up when I attempted to run the optimizer (copied below) was very unhelpful for diagnosing the source of the issue.
InvalidArgumentError: Computed input depth 32 doesn't match filter input depth 4 [Op:Conv2DBackpropInput]

I did not expect to see a backprop operation for Conv2D when my model was only using Conv1D layers, and I only figured out the issue was related to grouping because I was very familiar with the parameters of the model I was running.
I believe adding the following features would make it much easier for future developers to diagnose the issue:

A more informative error message when you attempt to run backprop on a grouped convolution layer on CPU
Possibly a warning when you load a grouped convolution layer on CPU at all
A note in the documentation for convolution layers that gradients for grouped convolution is only supported on GPU

Thank you.