the small print reported that the queen insisted that smithy come to the lunch at buckingham palace to which he had not been invited by wilson who was shamed into asking smith over to tea at no 10 that afternoon .