Here’s a simple test script I wrote which executes a single conv layer.
I had to drop back to PyTorch v0.3 so I no longer see which backend is active, but I haven’t yet found a configuration where grouped convolution is markedly faster than ordinary convolution. e.g.:
$ python3 group_conv.py --batch=256 --img-size=50 --in-channels=256 --out-channels=256 --kernel-size=3 --groups=True
---------------  ---------------  ---------------  ---------------  ---------------  ---------------
Name                    CPU time        CUDA time            Calls        CPU total       CUDA total
---------------  ---------------  ---------------  ---------------  ---------------  ---------------
ConvForward             77.271us      16688.654us                1         77.271us      16688.654us

$ python3 group_conv.py --batch=256 --img-size=50 --in-channels=256 --out-channels=256 --kernel-size=3 --groups=False
---------------  ---------------  ---------------  ---------------  ---------------  ---------------
Name                    CPU time        CUDA time            Calls        CPU total       CUDA total
---------------  ---------------  ---------------  ---------------  ---------------  ---------------
ConvForward             72.029us      15207.838us                1         72.029us      15207.838us


  
      gist.github.com
  
  
    https://gist.github.com/db434/74fcfcf49d52aa08c2335ee916bac9f4 49
group_conv.py
import argparse
import os
import subprocess
import sys
import torch

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--batch", type=int, default=256)
    parser.add_argument("--img-size", type=int, default=32)
This file has been truncated. show original