  In this paper, we propose TensorFHE, an FHE acceleration solution based on
GPGPU for real applications on encrypted data. TensorFHE utilizes Tensor Core
Units (TCUs) to boost the computation of Number Theoretic Transform (NTT),
which is the part of FHE with highest time-cost. Moreover, TensorFHE focuses on
performing as many FHE operations as possible in a certain time period rather
than reducing the latency of one operation. Based on such an idea, TensorFHE
introduces operation-level batching to fully utilize the data parallelism in
GPGPU. We experimentally prove that it is possible to achieve comparable
performance with GPGPU as with state-of-the-art ASIC accelerators. TensorFHE
performs 913 KOPS and 88 KOPS for NTT and HMULT (key FHE kernels) within NVIDIA
A100 GPGPU, which is 2.61x faster than state-of-the-art FHE implementation on
GPGPU; Moreover, TensorFHE provides comparable performance to the ASIC FHE
accelerators, which makes it even 2.9x faster than the F1+ with a specific
workload. Such a pure software acceleration based on commercial hardware with
high performance can open up usage of state-of-the-art FHE algorithms for a
broad set of applications in real systems.

Hardware Architecture;Cryptography and Security