I can see that you are changing the input size of linear here. How would one go about dynamically changing the output size instead? Would it just be changing the last 1 for 0?
model.weight = nn.Parameter(torch.cat((model.weight, torch.randn(10, 1)), 1))

to
model.weight = nn.Parameter(torch.cat((model.weight, torch.randn(10, 1)), 0))