Lyndon B. Johnson is the president who fathered the most children , having fifteen children over two marriages ( and allegedly fathering more with slaves ) , while his successor , James K. Polk , remains the only U.S. president never to have fathered or adopted any known children .