prdev
/

gpt2-differential-linear-attention

diff_attention_lm

Model card Files Files and versions Community

gpt2-differential-linear-attention / README.md

prdev's picture

Update README.md

8b8c197 verified 3 months ago

|

history blame contribute delete

223 Bytes

metadata

library_name: transformers
tags: []

Model Card for Model ID

A version of GPT-2 trained with a differential linear transformer. See https://prdev.substack.com/p/differential-linear-attention?r=2hyugh for details.