@seyonec It's a great help to experiment with ChemBERTa.
BTW, There are several models that handle SMILES in the model repository. Can you kindly recommend the one with the best performance in handling hERG dataset? https://paperswithcode.com/dataset/herg