Peter Hatvani's picture

Peter Hatvani PRO

RabidUmarell

AI & ML interests

Toxicity in Hungarian

Recent Activity

updated a dataset 24 days ago
RabidUmarell/hureddit-toxicity-setfit
published a dataset 24 days ago
RabidUmarell/hureddit-toxicity-setfit
updated a model about 1 month ago
RabidUmarell/hubert-embedding-setfit-toxic
View all activity

Organizations

Hungarian Research Centre for Linguistics's profile picture

RabidUmarell's activity

reacted to wassemgtk's post with 👀 about 1 month ago
view post
Post
2872
I’ve been diving into the iRoPE architecture from Llama 4—a game-changer for long-context models! It interleaves local attention (with RoPE) for short contexts and global attention (with inference-time temp scaling) for long-range reasoning, aiming for infinite context. I’m going to try writing iRoPE—who wants to help?

Code: https://github.com/wassemgtk/iRoPE-try/blob/main/iRoPE.ipynb
  • 1 reply
·
New activity in RabidUmarell/vicc-korpusz 3 months ago