they had n't been able to find him a replacement with just one day 's notice , which was one of the reasons james had been so nervous about letting me come back to the house without him , even in the middle of the day .