hey @david-waterworth, looking at the paper’s github repo (link), it seems that they just changed the embedding layer of the BERT implementation in transformers to accept character embeddings: character-bert/character_bert.py at 0519a6e22a2912bf0fdbbd49bd562cc2e5410bc7 · helboukkouri/character-bert · GitHub 1
you could try the same approach and see if that works!