The present teaching relates to method, system, medium, and implementations for adaptive dialogue. One or more images of a dialogue scene are received that capture surrounding of a dialogue between a user and a machine conducted based on a dialogue policy. One or more objects present in the scene are detected from the images based on a first artificial neural network. Spatial relationships among the one or more objects are detected from the images based on a second artificial neural network. Scene modeling information of the scene is then generated based on the one or more objects and the spatial relationships and is to be used for adaptive dialogue.
SYSTEM AND METHOD FOR ADAPTIVE DIALOGUE VIA SCENE MODELING USING COMBINATIONAL NEURAL NETWORKS