I am trying to convert xml into JSON but I get this error

xml.parsers.expat.ExpatError: reference to invalid character number: line 84, column 19

because that node has &#4
There are multiple nodes like this.
I am using xmltodict library to do this.
Repairing broken XML (or any other broken files, e.g. Excel files or PDF files) is always best done by fixing the software that produced the broken data in the first place. Anyone generating XML is doing so for a reason and should be prepared to fix the bugs in their code; and if they aren't prepared to fix bugs in their code, you should ask yourself whether it's a good idea to continue depending on them as a supplier.
If you do have to attempt a repair yourself, the first thing to remember is that the data is not XML, so XML tools are no use to you; you need to get in at a lower level (sometimes even the binary level).
Sometimes a simple regular expression replace will do the job: here, for example, you could try replacing &#x4; by <?hex 4?> at which point you have well-formed XML and can start using XML tools to process it. But to do this you always need a good knowledge of the exact nature of the corruptions in your data. For example, this particular replacement won't work if the bad data is in an attribute.
How to remove invalid character from xml in python