patterson's father was an owner of standardbred horses and as young man, charles was a harness racing driver. in 1891 his interests shifted to thoroughbred racing and he was hired to train the horses of renowned kentucky owner and breeder john madden.