On New Year's Day in 1776, Dunmore gave orders to burn waterfront buildings in Norfolk from which patriot troops were firing on his ships. However, the fire spread. The city burned, and with it any hope that Dunmore's loyalists could return to Virginia.[13] Dunmore retreated to New York.