An end-to-end control method is disclosed. The method comprises the following steps of when a controller controls a device, collecting sensor data from an external environment of the controlled deviceand/or the controlled device itself by a sensor, and recording control data generated by the control of the controller in real time; taking the sensor data and the control data which are temporally associated as paired data and inputting to a robot; and training the robot through using the paired data, wherein the trained robot makes a judgment according to the sensor data and controls the device, which includes the following steps of inputting sensor data collected by the sensor into the robot; determining whether the control data corresponding to the sensor data can be determined accordingto the sensor data by the robot; if the control data can be determined, controlling the device according to the control data corresponding to the sensor data by the robot; and if the robot can not determine the control data, controlling the device by the controller.
End-to-end control method and control system based on natural intelligence