  Echocardiography is widely used to clinical practice for diagnosis and
treatment, e.g., on the common congenital heart defects. The traditional manual
manipulation is error-prone due to the staff shortage, excess workload, and
less experience, leading to the urgent requirement of an automated
computer-aided reporting system to lighten the workload of ultrasonologists
considerably and assist them in decision making. Despite some recent successful
attempts in automatical medical report generation, they are trapped in the
ultrasound report generation, which involves unstructured-view images and
topic-related descriptions. To this end, we investigate the task of the
unstructured-view topic-related ultrasound report generation, and propose a
novel factored attention and embedding model (termed FAE-Gen). The proposed
FAE-Gen mainly consists of two modules, i.e., view-guided factored attention
and topic-oriented factored embedding, which 1) capture the homogeneous and
heterogeneous morphological characteristic across different views, and 2)
generate the descriptions with different syntactic patterns and different
emphatic contents for different topics. Experimental evaluations are conducted
on a to-be-released large-scale clinical cardiovascular ultrasound dataset
(CardUltData). Both quantitative comparisons and qualitative analysis
demonstrate the effectiveness and the superiority of FAE-Gen over seven
commonly-used metrics.

Computer Vision and Pattern Recognition