Subtracting a constant from your input won’t help much, as e.g. a relu might kill it after the first layer.
Maybe you could change the bias of the last linear layer to create a specific offset, but I doubt you will be able to create exactly 0.1 without zeroing the weights.
May I ask what your use case is, that you need a specific initial value?