This pilot clinical trial studies how well durvalumab before surgery works in treating patients with oral cavity or oropharynx cancer. Monoclonal antibodies, such as durvalumab, may interfere with the ability of tumor cells to grow and spread.