Techniques are disclosed for 3D object reconstruction using photometric mesh representations. A decoder is pretrained to transform points sampled from 2D patches of representative objects into 3D polygonal meshes. An image frame of the object is fed into an encoder to get an initial latent code vector. For each frame and camera pair from a sequence, a polygonal mesh is rendered at the given viewpoints using an object mesh generation neural network trained to transform a set of two-dimensional data points representing the first object into the polygon mesh, the set of 2D data points representing colour pixels in at least two images of the first object, the at least two images having different camera poses,. The mesh is optimized over the set of latent feature vectors by creating a virtual viewpoint, rasterized to obtain a depth map. The 3D mesh projections are aligned by projecting the coordinates corresponding to the polygonal face vertices of the rasterized mesh to both selected viewpoints. The photometric error is determined from RGB pixel intensities sampled from both frames. Gradients from the photometric error are backpropagated into the vertices of the assigned polygonal indices by relating the barycentric coordinates of each image to update the latent code vector.
3D object reconstruction using photometric mesh representation