Systems and methods for detecting objects are provided. In one example, a computer-implemented method includes receiving sensor data from one or more sensors configured to generate sensor data. The method includes inputting the sensor data to a machine-learned model that generates a class prediction and an instance prediction for each of a plurality of portions of the sensor data. The instance prediction includes an energy value based on a distance to at least one object boundary. The machine learned model can be trained to generate a common energy value to represent the at least one object boundary. The method includes generating as outputs of the machine-learned model, an instance prediction and a class prediction corresponding to each of the plurality of portions of the sensor data. The method includes generating one or more object segments based at least in part on the instance predictions and the class predictions.
Semantic and instance segmentation