We investigate two-stage graph pre-training strategies to improve BART's structure(Knowledge Graph) awareness.