Hi,
Is there a way to get the same result with an adam optimizer before and after gradient accumulation ?
the apex error i am getting,
…
Compiling cuda extensions with
nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:04_Central_Daylight_Time_2018
Cuda compilation tools, release 10.0, V10.0.130
from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0/bin
…
multi_tensor_sgd_kernel.cu
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1379): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1437): note: see reference to class template instantiation ‘ska::flat_hash_map<K,V,H,E,A>’ being compiled
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1383): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1391): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1473): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1510): note: see reference to class template instantiation ‘ska::flat_hash_set<T,H,E,A>’ being compiled
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1478): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1482): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1486): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
c:/users/rafi/anaconda3/envs/myenv2/lib/site-packages/torch/include\c10/util/flat_hash_map.h(1490): error C3203: ‘templated_iterator’: unspecialized class template can’t be used as a template argument for template parameter ‘_Ty1’, expected a real type
error: command ‘C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\nvcc.exe’ failed with exit status 2
Running setup.py install for apex … error