On July 12, 1861, Williams entered the Union Army as a captain in the 5th Kansas Cavalry Regiment, and was appointed commander of Company F.[1] The unit was organized at Leavenworth, Kansas, beginning that summer and into January of the following year. Williams and his regiment served in the Department of Kansas through June 1862, and then with the Department of Missouri that July.