Upon his retirement he moved his family first to New Zealand and later to Melbourne, Australia. His son Lee Wai-ho from his second marriage became a professor after his graduation in the University of Melbourne, while his daughter, Lee Yuk-yee was graduated with a Master degree.