This phase II trial studies how well nivolumab works in treating patients with anal canal cancer that has not responded to previous treatment (refractory) and has spread to other places in the body (metastatic). Monoclonal antibodies, such as nivolumab, may block tumor growth in different ways by targeting certain cells.