2. Related Work
2.1. Finetuning Neural Networks
One way to finetune a neural network is to directly continue
training it with the additional training data. But this approach
can lead to overfitting, mode collapse, and catastrophic for-
getting. Extensive research has focused on developing fine-
tuning strategies that avoid such issues.
HyperNetwork is an approach that originated in the Natural
Language Processing (NLP) community [25], with the aim
of training a small recurrent neural network to influence the
weights of a larger one. It has been applied to image gener-
ation with generative adversarial networks (GANs) [4, 18].
Heathen et al. [26] and Kurumuz [43] implement HyperNet-
works for Stable Diffusion [72] to change the artistic style
of its output images.
Adapter methods are widely used in NLP for customiz-
ing a pretrained transformer model to other tasks by em-
bedding new module layers into it [30, 84]. In computer
vision, adapters are used for incremental learning [74] and
AddingConditionalControltoText-to-ImageDiffusionModels