If you pass in an ordered dict, the ordering will be preserved:
nn.ModuleDict(OrderedDict({ 
   'b': nn.Linear(2,4), 
   'a': nn.Linear(4,8) 
} ))

The crux is that Python 2 does not preserve order in dict (and for early Python 3.x it’s an implementation detail), so in order to have a deterministic dict->OrderedDict conversion, the key are sorted.
Alternatively, you can also pass in an iterator {...}.items() - looks funny, probably deserves an explanatory comment in the code, but works.
Best regards
Thomas