I wouldn’t say it is very small…
It has 2 topics x 2 branches (for text & section) x 3 folds (trained by cross-validation), every branch has the same sequence of linear layers with dropouts:
__init__(self, emb_in=768, h0=512, h1=256, h2=128, h3=64, out=32)
then concatenation and averaging…
In total 6.8M parameters