To answer myself, I partially solved the problem for by splitting the value into a buffer with value 0 and a python constant with the user-defined value, and then summing them in the forward pass. Hopefully the JIT is smart enough to see that they are both constant, and it propagates them when required, but I haven’t verified it yet.
There is still has a requirement on strict=False though, since loading a pretrained model to which any of these layers have been added will complain about the missing buffer.
It would be nice to see a proper API for this use case, especially give the fact that constant values could be easily optimized by the JIT.
Alessandro