Face detection, get face bounding box and landmarks. Here’s the SOTA Papers and Codes 4.
(Use landmarks) Aligned face is very beneficial for improve the performance of face recognition.
Here’s the Megaface Rank. 4

Insightface 2, which include RetinaFace for face detection and ArcFace for face recognition, use MxNet instead of PyTorch, but you can find some third-party re-implementation in its README file.
Dataset quality is more important than network structure.
For faster inference: MobileNet v3 1, EfficientNet and IdleBlock 1 are worth trying.

Addition:

The main idea is to enhance intraclass compactness and inter-class discrepancy.
More focused on intraclass compactness: ShpereFace, CosFace and ArcFace 1 are a series.
More focused on inter-class discrepancy: UniformFace 3 is one of the recent works.