A method of controlling one or more robotic devices, such as autonomous vehicles using neural networks, to perform a task. A first neural network determines information about environmental conditions relating to the task. Second neural network 206 and the information about the environmental conditions are used to help control the one or more robotic devices when carrying out the task. The information may include a logical state describing the environmental conditions. The second neural network may determine actions to help control the devices when carried out. The environmental conditions may include unexpected events. The neural networks may generate one or more loss values using which they are trained. The environmental conditions may be determined based on sensor data obtained by environmental sensors 214 such as cameras, navigation sensors, GPS sensors, RADAR sensors, LIDAR sensors, or inertial sensors.
Robotic control using deep learning