I think the DM is hinting that there might well be molestation on the way back, so it might be in your best interest to already be with us unless you want to sit out of the game for another week while combat happens.