Hi all, hope you’re well. I’m running a script with pytorch lightning and keep getting the below Segmentation Fault error. I really have no idea what’s going on/how to address it - I imported faulthandler to get a better sense of what’s causing the issue and that output is pasted below. Would appreciate any help on getting this to work.
Fatal Python error: Segmentation fault
Current thread 0x00007f08d3c82740 (most recent call first):
File “”, line 228 in _call_with_frames_removed
File “”, line 1173 in create_module
File “”, line 565 in module_from_spec
File “”, line 666 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/google/protobuf/descriptor.py”, line 47 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py”, line 7 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/tensorflow/python/eager/context.py”, line 32 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/tensorflow/python/init.py”, line 41 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 972 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/tensorflow/init.py”, line 41 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/huggingface_hub/keras_mixin.py”, line 19 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/huggingface_hub/init.py”, line 37 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/transformers/file_utils.py”, line 51 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/transformers/dependency_versions_check.py”, line 36 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/transformers/init.py”, line 43 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/torchmetrics/functional/text/bert.py”, line 28 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/torchmetrics/functional/init.py”, line 68 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “”, line 228 in _call_with_frames_removed
File “”, line 1058 in _handle_fromlist
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/torchmetrics/init.py”, line 14 in 
File “”, line 228 in _call_with_frames_removed
File “”, line 850 in exec_module
File “”, line 680 in _load_unlocked
File “”, line 986 in _find_and_load_unlocked
File “”, line 1007 in _find_and_load
File “/wynton/protected/home/ichs/dmandair/anaconda3/envs/ONC_EXP/lib/python3.9/site-packages/pytorch_lightning/utilities/types.py”, line 25 in 
…
test.sh: line 6:  9413 Segmentation fault      python end_to_end_attention.py