Ball and her brother, Fred Henry Ball (1915–2007), were raised by their mother and maternal grandparents in Celoron, New York, a summer resort village on Lake Chautauqua, 2 miles west of downtown Jamestown.