Methods, systems, and apparatuses are provided for implementing a semantically and visually interpretable medical diagnosis network. An example method enables performance of medical diagnosis by a diagnostic system and comprises steps of receiving input comprising at least one of (i) a medical image and (ii) audio or text content, and inputting the received input into a deep learning framework of the diagnostic system. The example method further comprises steps of generating, by the deep learning framework, attention data regarding the received input, the attention data comprising pixel-level localized attention regions within the medical image, and generating, by the deep learning framework, a diagnostic report comprising (a) a diagnosis drawn based on the received input, the attention data, and corresponding audio or text content and (b) a medical justification of the diagnosis provided in a natural language format. Corresponding systems and apparatuses are contemplated herein.
METHODS AND APPARATUSES FOR IMPLEMENTING A SEMANTICALLY AND VISUALLY INTERPRETABLE MEDICAL DIAGNOSIS NETWORK