Hi Bruno!
I have both a conceptual comment and a couple of technical comments.
On the conceptual side:



 CasellaJr:

Now, I want to combine (sum, or other operations) these weights.


I have concerns about combining weights like this from two
independently-trained models.  (I have heard reports that this
works or can be made to work, but I’m skeptical.)
The issue is that there is a lot of arbitrariness and redundancy
in the weights of a neural-network model.  If I take two identical
models, but give them different (but equivalent) initializations,
and train them on the same training data (but probably batched
up into different (but equivalent) random batches), there is no
reason for “weight-17” in model A to have the same value as
“weight-17” in model B.
Training model A takes it down some particular path along the
“loss-surface” to some reasonable location in weight-configuration
space.  But because of the redundancy and randomness, training
model B takes it down some entirely different (but comparably
worthwhile) path to some entirely different location (that is comparably
good in making predictions).
Speaking somewhat figuratively, let’s say that, by happenstance,
the weights on the “left-hand” side of model A learn to recognize
straight edges, while weights on the “right-hand” side learn curved
edges.  But now assume that, by happenstance, this is reversed in
model B, so the left side learns curved edges and the right, straight
edges.
So weight-17 in model A means straight, but weight-17 in model B
means curved.  These two weights mean two different things, so it
doesn’t make sense to add, or average, or otherwise combine then
together.
On the technical side:


model3.parameters = modelMNIST.parameters()+modelSVHN.parameters()

If I do in this way, I obtain this error:
SyntaxError: can't assign to function call


.parameters is a method (“member function”) of Module, while
.parameters() is the result of invoking that method (calling that
function).  You are trying to assign something to the function as the
error message indicates.
But furthermore, the result of calling .parameters() is also not
something you can assign to.  It’s a “generator” – something you
can really just iterate over.
So you could simultaneously iterate over the result of calling
.parameters() on your three models.
But …


And in this way:
model3.block_1[0].weight = modelMNIST.block_1[0].weight + modelSVHN.block_1[0].weight

I get this error:
TypeError: cannot assign 'torch.cuda.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)



A Parameter is not a Tensor – it’s a wrapper for a Tensor that
has some added functionality.  When you add two Parameters
together, you don’t get a new Parameter – you get a new Tensor
that is the sum of the two wrapped Tensors.  If you assign a Tensor
to a (reference to a) Parameter, you don’t get a (reference to a)
Parameter – you get a (reference to a) Tensor (and a Module
is smart enough not to let you do this).
You would need to do something like this:
l1 = torch.nn.Linear (2, 3)
l2 = torch.nn.Linear (2, 3)
l3 = torch.nn.Linear (2, 3)
with torch.no_grad():
    l1.weight.copy_ (l2.weight + l3.weight)

Best.
K. Frank