If you only want to use the decoder of BART, you can do so by simply using the BartDecoder class 2. So your code could look something like:
from transformers.models.bart.modeling_bart import BartDecoder
model = BartDecoder.from_pretrained("facebook/bart-base")

But note that BART is a seq2seq (encoder-decoder) model, it has been pre-trained in an encoder-decoder set-up, so the best results will probably be obtained by using both the encoder and decoder. But of course you can still use only the decoder if you want