When using a USB-C monitor with power delivery, does constantly having it plugged in damage the battery? By my assumption, the USB-C port on the monitor will charge the device even on full charge. Is this correct? I believe this would damage the battery over long periods of time.
No, having it constantly plugged in won't damage the battery.

The MacBook contains power management circuitry that ensures that the battery is not "overcharged".

So having a USB-C monitor that charges your MacBook plugged in at all times is no different from having an ordinary charger plugged in at all times.