Is there a way to detect caching servers? I need to find out if there are OSX/iOS caching servers on a bunch of different networks - and put in a caching server where needed. But since the caching service on a Mac server is pretty black box, there isn't much information available on how it works.

I know that if I have access to a server, it's trivial to check if caching is on in the server software. I'm looking for, say, a port to scan, or a flag that gets set on client devices' App Stores that can be checked, or something in firewall logs to look for. There is just so little information out there I can't say exactly where to start.

I have remote access to at least a headless Mac Mini on each network.
Not directly before Sierra on macOS.
On Sierra and later, run the AssetCacheLocatorUtil from any unix shell (in terminal or other app)
We have even found a way to tell after the fact whether an update came from the cache or from Apple on the client side by looking at the logs instead of sniffing network connections and traffic.

Here are the Sierra and later details.
Query Apple's update infrastructure and check if any local caching points for updates, app downloads, and/or iCloud data are present and it also checks for connectivity and service
AssetCacheLocatorUtil

The manual page is quite well written and covers a lot of the details. Below are some observations from before this tool was around to help.
The servers listen on somewhat randomized ports - my two caching servers are currently listening on ports 49558 and 51858
I suppose you could port scan all the servers and try to deduce which ones are using OS X server in general and then exhaustively probe all the higher ports on the presumption that a single higher port open might be a caching server.
You can request a server to tell you the port it's running:
sudo serveradmin fullstatus caching | grep Port

The clients check in with Apple directly, Apple notices the public IP address from which the requites came. It then makes two checks:

what servers have checked in and are running from that same public IP address
are there PRS records in public DNS for the public IP address the client has used and if another caching server could serve that IP address.

It then responds to the client to request the asset from the server(s) that are local to the IP. If that caching server doesn't have the asset, it requests it and passes it on to the client. If the client doesn't get it's assets, there is a retry mechanism to get the asset directly from Apple's CDN so you don't always get things from the caching servers in my experience.
You also might be able to make a list of servers that have remote access enabled - listening on port 311/asip-webadmin or by collecting the status of each server periodically:
sudo serveradmin settings info | grep enableRemoteAdministration