@StackSmasher,
consider you have the tensor inputs_embeds which I believe will be in the shape of (batch_size, seq_length, dim), or If you have a hidden_state in the shape of (batch_size, dim) just unsqueeze(dim=1) it to become (batch_size,1,dim)
Then smoothly u can achieve the desired output by:

create an attention mask (batch_size, seq_len)

create decoder_input_ids (batch_size, 1)


tracing these two parameters, leads to two functions below:
attention mask you are already familiar with it.
  

      github.com
  

  
    huggingface/transformers/blob/d5ff69fce92bb1aab9273d674e762a8eddcb2e3f/src/transformers/generation_utils.py#L394-L403 1



      
      
          def _prepare_attention_mask_for_generation(
              self, input_ids: torch.Tensor, pad_token_id: int, eos_token_id: int
          ) -> torch.LongTensor:
              is_pad_token_in_inputs_ids = (pad_token_id is not None) and (pad_token_id in input_ids)
              is_pad_token_not_equal_to_eos_token_id = (eos_token_id is None) or (
                  (eos_token_id is not None) and (pad_token_id != eos_token_id)
              )
              if is_pad_token_in_inputs_ids and is_pad_token_not_equal_to_eos_token_id:
                  return input_ids.ne(pad_token_id).long()
              return input_ids.new_ones(input_ids.shape, dtype=torch.long)
      
    
  


  

  
    
    
  

  


decoder_input_ids it’s ones too (multiplied by config.decoder_start_token_ids).
  

      github.com
  

  
    huggingface/transformers/blob/d5ff69fce92bb1aab9273d674e762a8eddcb2e3f/src/transformers/generation_utils.py#L419-L426



      
      
          def _prepare_decoder_input_ids_for_generation(
              self, input_ids: torch.LongTensor, decoder_start_token_id: int = None, bos_token_id: int = None
          ) -> torch.LongTensor:
              decoder_start_token_id = self._get_decoder_start_token_id(decoder_start_token_id, bos_token_id)
              decoder_input_ids = (
                  torch.ones((input_ids.shape[0], 1), dtype=torch.long, device=input_ids.device) * decoder_start_token_id
              )
              return decoder_input_ids
      
    
  


  

  
    
    
  

  


wrapping it all:
generator = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
inputs_embeds = # a 3D tensor, [batch, seq_length, dim]
attention_mask = torch.ones(inputs_embeds.shape[:2], dtype=torch.long)
decoder_input_ids = torch.ones((inputs_embeds.shape[0], 1), dtype=torch.long)*generator.config.decoder_start_token_id
output_ids = generator.generate(attention_mask=attention_mask, decoder_input_ids=decoder_input_ids, inputs_embeds=inputs_embeds, max_length=100,num_beams=4)