with his fortune and influence , he stirred up the political scene and publicly spoke and charged queen victoria herself with building and maintaining a wall of systematic racism and hatred toward the people of scotland and ireland .