"Bidirectional attention"

by olivierdehaene HF staff - opened Jul 1

Jul 1

Hello,

It seems that compared to the 7B variant, this model has bi-directional attention turned off. Is this normal?
See this line where is_causal is set to True in this variant.
VS
This line where is_causal is set to False in the 7B variant.

zyznull

Alibaba-NLP org Jul 1

Thanks for your careful observation! The bi-directional attention is indeed used on this model. The code has already been updated now.

olivierdehaene changed discussion status to closed Jul 2

olivierdehaene changed discussion status to open Sep 19

olivierdehaene

Sep 19

@zyznull , it seems that you didn't properly update the code. is_causal is still set to True by default in the model forward which is the main entrypoint for Transformers and SentenceTransformers.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment