`` my mother never spoke to me about that time , but joanne once explained that our father threatened to take both of us and prevent our mother from ever seeing us again if she did n't agree to his terms . ''