Solved by douglasrizzo in post #2 
              
              
                This was solved here: Documentation confusing on whether SSD and RetinaNet count background as class object · Issue #4106 · pytorch/vision · GitHub