  Deep generative modeling for biological sequences presents a unique challenge
in reconciling the bias-variance trade-off between explicit biological insight
and model flexibility. The deep manifold sampler was recently proposed as a
means to iteratively sample variable-length protein sequences by exploiting the
gradients from a function predictor. We introduce an alternative approach to
this guided sampling procedure, multi-segment preserving sampling, that enables
the direct inclusion of domain-specific knowledge by designating preserved and
non-preserved segments along the input sequence, thereby restricting variation
to only select regions. We present its effectiveness in the context of antibody
design by training two models: a deep manifold sampler and a GPT-2 language
model on nearly six million heavy chain sequences annotated with the IGHV1-18
gene. During sampling, we restrict variation to only the
complementarity-determining region 3 (CDR3) of the input. We obtain log
probability scores from a GPT-2 model for each sampled CDR3 and demonstrate
that multi-segment preserving sampling generates reasonable designs while
maintaining the desired, preserved regions.

Machine Learning;Biomolecules