Create the MoCo model and load the weights using:
for k in list(state_dict.keys()):
    # retain only encoder_q up to before the embedding layer
    if k.startswith('module.encoder_q') and not k.startswith('module.encoder_q.fc'):
        # remove prefix
        k_no_prefix = k[len("module."):]
        state_dict[k_no_prefix] = state_dict[k]   # leave encoder_q in the param name
        # copy state from the query encoder into a new parameter for the key encoder
        state_dict[k_no_prefix.replace('encoder_q', 'encoder_k')] = state_dict[k]
    del state_dict[k]

msg = moco_model.load_state_dict(state_dict, strict=False)

This should give you the MoCo model with initialized query and key encoders (their parameters are copies of each other to begin with). The MLP and queue will still be randomly initialized; you’ll have to train them on your dataset.