I am executing resnet50_trainer example on MNIST dataset:
The following command works:
python resnet50_trainer.py --train_data ~/mnist_train_lmdb --num_gpus 4 --batch_size 64
but adding num_channels leads to error:
python resnet50_trainer.py --train_data ~/mnist_train_lmdb --num_gpus 4 --batch_size 64 --num_channels 1
...
...
INFO:ResNe(X)t_trainer:Starting epoch 0/1
[E net_async_base.cc:377] [enforce fail at conv_op_cudnn.cc:555] filter.dim32(1) == C / group_. 1 vs 3
Error from operator:
input: "gpu_1/data" input: "gpu_1/conv1_w" output: "gpu_1/conv1" name: "" type: "Conv" arg { name: "kernel" i: 7 } arg { name: "order" s: "NCHW" } arg { name: "enable_tensor_core" i: 0 } arg { name: "stride" i: 2 } arg { name: "pad" i: 3 } arg { name: "exhaustive_search" i: 1 } arg { name: "ws_nbytes_limit" i: 67108864 } device_option { device_type: 1 device_id: 1 } engine: "CUDNN"frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*)```