  Real-time defect detection is crucial in laser-directed energy deposition
(L-DED) additive manufacturing (AM). Traditional in-situ monitoring approach
utilizes a single sensor (i.e., acoustic, visual, or thermal sensor) to capture
the complex process dynamic behaviors, which is insufficient for defect
detection with high accuracy and robustness. This paper proposes a novel
multimodal sensor fusion method for real-time location-dependent defect
detection in the robotic L-DED process. The multimodal fusion sources include a
microphone sensor capturing the laser-material interaction sound and a visible
spectrum CCD camera capturing the coaxial melt pool images. A hybrid
convolutional neural network (CNN) is proposed to fuse acoustic and visual
data. The key novelty in this study is that the traditional manual feature
extraction procedures are no longer required, and the raw melt pool images and
acoustic signals are fused directly by the hybrid CNN model, which achieved the
highest defect prediction accuracy (98.5 %) without the thermal sensing
modality. Moreover, unlike previous region-based quality prediction, the
proposed hybrid CNN can detect the onset of defect occurrences. The defect
prediction outcomes are synchronized and registered with in-situ acquired robot
tool-center-point (TCP) data, which enables localized defect identification.
The proposed multimodal sensor fusion method offers a robust solution for
in-situ defect detection.

Image and Video Processing;Audio and Speech Processing;Signal Processing