Systems, methods, tangible non-transitory computer-readable media, and devices for operating an autonomous vehicle are provided. For example, the disclosed technology can include receiving sensor data and map data. The sensor data can include information associated with an environment detected by sensors of a vehicle. The map data can include information associated with traffic signals in the environment. Further, an input representation can be generated based on the sensor data and the map data. The input representation can include regions of interest associated with images of the traffic signals. States of the traffic signals in the environment can be determined, based on the input representation and a machine-learned model. Traffic signal state data that includes a determinative state of the traffic signals can be generated based on the states of the traffic signals.
Traffic signal state classification for autonomous vehicles