No, the caffe2 bits are built in libcaffe2_...so. The library splitting was introduced to avoid the relocation issue 1 introduced by the large library size during linking due to the size increase in CUDA, cuDNN etc.
The libtorch_cuda_cpp.so contains symbols for cuDNN, NCCL etc. Also, if I delete it, I’m unable to import torch and get:
ImportError: libtorch_cuda_cpp.so: cannot open shared object file: No such file or directory

so unsure what your workflow is.
TL;DR: don’t delete it, as this lib is needed.