ray had told alan later , in a long conversation over glasses of brandy at the doctor 's castle view home , that the symptoms often came and went in cases where the tumor was located high on the stem of the brain .