williams married emma, the dowager viscountess bulkeley, widow of james bulkeley, 6th viscount bulkeley on 28 june 1760. she was daughter of thomas rowlands of caerau, anglesey, but as her father disapproved of williams, the marriage was kept secret. correspondence indicates that williams was becoming involved in politics in beaumaris, partly in the interest of his stepson thomas bulkeley, but was handicapped by the secrecy of his marriage. in the course of these dealings he became lt colonel in the 53rd foot in 1761 and became constable of beaumaris castle in july 1761. however his regiment was stationed in gibraltar and he was ordered back there. he disliked the climate and wished to be transferred to a regiment at home. as this was refused he retired from the army on half-pay in december 1764 to take care of affairs at beaumaris.