I believe I do now have it working.
The key was to just remove all the additional labels from the label.pbtxt file. Then change the number of classes in the configuration file to 1.
I had tried this before and it didn’t work. I suspect the reason was that some of my train images had pictures in it which did not contain the single class I am detecting for, and I suspect this caused it to not work as intended.
I ensured that all images had at least one instance of my class by filtering for it in fifty one.
Very surprised there was not more information on what I would expect is a very common use case. But glad it is resolved.