he could n't remain absent from the ballroom too long without the guests wondering where their host had disappeared to , and since his mother would be the one to suffer most from any potential rumors , he had no choice but to do as she asked .