Thanks for the update.
Given that you are using 1.8.1 and a 1080Ti (sm_61), I think you are hitting this issue 1, which is already fixed in later releases. Could you update libtorch and rerun your workload?