content.raw: no ai slop posted here today i just feel like posting what i did for today
content.type: text
content.value: no ai slop posted here today i just feel like posting what i did for today
content.type: new_line
content.type: new_line
content.raw: wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.
content.type: text
content.value: wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.
content.type: new_line
content.type: new_line
content.raw: code @ 
content.type: text
content.value: code @ 
content.href: https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
content.raw: https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
content.type: link
rawContent: no ai slop posted here today i just feel like posting what i did for today
  
  wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.
  
  code @ https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
reactions.reaction: 👍
reactions.users: John6666 iproskurina