Running
on
T4
350
๐ฅ
alkinun
AtAndDev
AI & ML interests
LLMs, Alignment, Merging, Unsloth, DPO, SFT, ORPO, SPIN..
Recent Activity
liked
a Space
about 9 hours ago
stevengrove/YOLO-World
reacted
to
as-cle-bert's
post
with โค๏ธ
about 9 hours ago
Hi HuggingFacers!๐คถ๐ผ
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ https://huggingface.co/spaces/as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- https://huggingface.co/sentence-transformers/LaBSE
- https://huggingface.co/prithivida/Splade_PP_en_v1
- https://huggingface.co/facebook/dinov2-large
And datasets:
- https://huggingface.co/datasets/Karbo31881/Pokemon_images
- https://huggingface.co/datasets/wanghaofan/pokemon-wiki-captions
- https://huggingface.co/datasets/TheFusion21/PokemonCards
Have fun!๐
liked
a model
about 9 hours ago
deepseek-ai/DeepSeek-V3-Base
Organizations
AtAndDev's activity
liked
a
Space
about 9 hours ago
reacted to
as-cle-bert's
post with โค๏ธ
about 9 hours ago
Post
821
Hi HuggingFacers!๐คถ๐ผ
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- sentence-transformers/LaBSE
- prithivida/Splade_PP_en_v1
- facebook/dinov2-large
And datasets:
- Karbo31881/Pokemon_images
- wanghaofan/pokemon-wiki-captions
- TheFusion21/PokemonCards
Have fun!๐
As my last 2024 project, I've dropped a Discord Bot that knows a lot about Pokemons๐ฆ
GitHub ๐ https://github.com/AstraBert/Pokemon-Bot
Demo Space ๐ as-cle-bert/pokemon-bot
The bot integrates:
- Chat features (Cohere's Command-R) with RAG functionalities (hybrid search and reranking with Qdrant) and chat memory (managed through PostgreSQL) to produce information about Pokemons
- Image-based search to identify Pokemons from their images (via Qdrant)
- Card package random extraction and description
HuggingFace๐ค, as usual, plays the most important role in the application stack, with the following models:
- sentence-transformers/LaBSE
- prithivida/Splade_PP_en_v1
- facebook/dinov2-large
And datasets:
- Karbo31881/Pokemon_images
- wanghaofan/pokemon-wiki-captions
- TheFusion21/PokemonCards
Have fun!๐
liked
a
model
about 9 hours ago
reacted to
sayakpaul's
post with ๐๐ฅ
about 9 hours ago
Post
2814
Commits speak louder than words ๐คช
* 4 new video models
* Multiple image models, including SANA & Flux Control
* New quantizers -> GGUF & TorchAO
* New training scripts
Enjoy this holiday-special Diffusers release ๐ค
Notes: https://github.com/huggingface/diffusers/releases/tag/v0.32.0
* 4 new video models
* Multiple image models, including SANA & Flux Control
* New quantizers -> GGUF & TorchAO
* New training scripts
Enjoy this holiday-special Diffusers release ๐ค
Notes: https://github.com/huggingface/diffusers/releases/tag/v0.32.0
liked
2
Spaces
about 10 hours ago
liked
a
model
4 days ago
liked
a
Space
4 days ago
reacted to
InferenceIllusionist's
post with ๐ฅ
4 days ago
Post
1861
MilkDropLM-32b-v0.3: Unlocking Next-Gen Visuals โจ
Stoked to release the latest iteration of our MilkDropLM project! This new release is based on the powerful Qwen2.5-Coder-32B-Instruct model using the same great dataset that powered our 7b model.
What's new?
- Genome Unlocked: Deeper understanding of preset relationships for more accurate and creative generations.
- Preset Revival: Breathe new life into old presets with our upgraded model!
- Loop-B-Gone: Say goodbye to pesky loops and hello to smooth generation.
- Natural Chats: Engage in more natural sounding conversations with our LLM than ever before.
Released under Apache 2.0, because sharing is caring!
Try it out: InferenceIllusionist/MilkDropLM-32b-v0.3
Shoutout to @superwatermelon for his invaluable insights and collab, and to all those courageous members in the community that have tested and provided feedback before!
Stoked to release the latest iteration of our MilkDropLM project! This new release is based on the powerful Qwen2.5-Coder-32B-Instruct model using the same great dataset that powered our 7b model.
What's new?
- Genome Unlocked: Deeper understanding of preset relationships for more accurate and creative generations.
- Preset Revival: Breathe new life into old presets with our upgraded model!
- Loop-B-Gone: Say goodbye to pesky loops and hello to smooth generation.
- Natural Chats: Engage in more natural sounding conversations with our LLM than ever before.
Released under Apache 2.0, because sharing is caring!
Try it out: InferenceIllusionist/MilkDropLM-32b-v0.3
Shoutout to @superwatermelon for his invaluable insights and collab, and to all those courageous members in the community that have tested and provided feedback before!
reacted to
suayptalha's
post with โค๏ธ๐๐ฅ
4 days ago
Post
1556
๐ FastLlama Series is Live!
๐ฆพ Experience faster, lighter, and smarter language models! The new FastLlama makes Meta's LLaMA models work with smaller file sizes, lower system requirements, and higher performance. The model supports 8 languages, including English, German, and Spanish.
๐ค Built on the LLaMA 3.2-1B-Instruct model, fine-tuned with Hugging Face's SmolTalk and MetaMathQA-50k datasets, and powered by LoRA (Low-Rank Adaptation) for groundbreaking mathematical reasoning.
๐ป Its compact size makes it versatile for a wide range of applications!
๐ฌ Chat with the model:
๐ Chat Link: suayptalha/Chat-with-FastLlama
๐ Model Link: suayptalha/FastLlama-3.2-1B-Instruct
๐ฆพ Experience faster, lighter, and smarter language models! The new FastLlama makes Meta's LLaMA models work with smaller file sizes, lower system requirements, and higher performance. The model supports 8 languages, including English, German, and Spanish.
๐ค Built on the LLaMA 3.2-1B-Instruct model, fine-tuned with Hugging Face's SmolTalk and MetaMathQA-50k datasets, and powered by LoRA (Low-Rank Adaptation) for groundbreaking mathematical reasoning.
๐ป Its compact size makes it versatile for a wide range of applications!
๐ฌ Chat with the model:
๐ Chat Link: suayptalha/Chat-with-FastLlama
๐ Model Link: suayptalha/FastLlama-3.2-1B-Instruct
reacted to
anton-l's
post with ๐ฅ๐
5 days ago
Post
1973
Introducing ๐๐
๐ข๐ง๐๐๐๐ญ๐ก: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
๐ ๏ธ carefully extracting math data from Common Crawl;
๐ iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! ๐
Weโre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
๐ ๏ธ carefully extracting math data from Common Crawl;
๐ iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! ๐
Weโre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
reacted to
etemiz's
post with ๐๐ง ๐
5 days ago
Post
2268
As more synthetic datasets are made, we move slowly away from human alignment.
- 4 replies
upvoted
a
paper
5 days ago
reacted to
m-ric's
post with ๐
5 days ago
Post
2029
๐๐ฎ๐ ๐ ๐ข๐ง๐ ๐
๐๐๐ ๐ซ๐๐ฅ๐๐๐ฌ๐๐ฌ ๐๐ข๐๐จ๐ญ๐ซ๐จ๐ง, ๐ ๐ฆ๐ข๐๐ซ๐จ๐ฌ๐๐จ๐ฉ๐ข๐ ๐ฅ๐ข๐ ๐ญ๐ก๐๐ญ ๐ฌ๐จ๐ฅ๐ฏ๐๐ฌ ๐๐๐ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ ๐๐ ๐ฉ๐๐ซ๐๐ฅ๐ฅ๐๐ฅ๐ข๐ณ๐๐ญ๐ข๐จ๐ง ๐ฅณ
๐ฐ๏ธ Llama-3.1-405B took 39 million GPU-hours to train, i.e. about 4.5 thousand years.
๐ด๐ป If they had needed all this time, we would have GPU stories from the time of Pharaoh ๐: "Alas, Lord of Two Lands, the shipment of counting-stones arriving from Cathay was lost to pirates, this shall delay the building of your computing temple by many moons "
๐ ๏ธ But instead, they just parallelized the training on 24k H100s, which made it take just a few months.
This required parallelizing across 4 dimensions: data, tensor, context, pipeline.
And it is infamously hard to do, making for bloated code repos that hold together only by magic.
๐ค ๐๐๐ ๐ป๐ผ๐ ๐๐ฒ ๐ฑ๐ผ๐ป'๐ ๐ป๐ฒ๐ฒ๐ฑ ๐ต๐๐ด๐ฒ ๐ฟ๐ฒ๐ฝ๐ผ๐ ๐ฎ๐ป๐๐บ๐ผ๐ฟ๐ฒ! Instead of building mega-training codes, Hugging Face colleagues cooked in the other direction, towards tiny 4D parallelism libs. A team has built Nanotron, already widely used in industry.
And now a team releases Picotron, a radical approach to code 4D Parallelism in just a few hundred lines of code, a real engineering prowess, making it much easier to understand what's actually happening!
โก ๐๐'๐ ๐๐ถ๐ป๐, ๐๐ฒ๐ ๐ฝ๐ผ๐๐ฒ๐ฟ๐ณ๐๐น:
Counting in MFU (Model FLOPs Utilization, how much the model actually uses all the compute potential), this lib reaches ~50% on SmolLM-1.7B model with 8 H100 GPUs, which is really close to what huge libs would reach. (Caution: the team is leading further benchmarks to verify this)
Go take a look ๐ https://github.com/huggingface/picotron/tree/main/picotron
๐ฐ๏ธ Llama-3.1-405B took 39 million GPU-hours to train, i.e. about 4.5 thousand years.
๐ด๐ป If they had needed all this time, we would have GPU stories from the time of Pharaoh ๐: "Alas, Lord of Two Lands, the shipment of counting-stones arriving from Cathay was lost to pirates, this shall delay the building of your computing temple by many moons "
๐ ๏ธ But instead, they just parallelized the training on 24k H100s, which made it take just a few months.
This required parallelizing across 4 dimensions: data, tensor, context, pipeline.
And it is infamously hard to do, making for bloated code repos that hold together only by magic.
๐ค ๐๐๐ ๐ป๐ผ๐ ๐๐ฒ ๐ฑ๐ผ๐ป'๐ ๐ป๐ฒ๐ฒ๐ฑ ๐ต๐๐ด๐ฒ ๐ฟ๐ฒ๐ฝ๐ผ๐ ๐ฎ๐ป๐๐บ๐ผ๐ฟ๐ฒ! Instead of building mega-training codes, Hugging Face colleagues cooked in the other direction, towards tiny 4D parallelism libs. A team has built Nanotron, already widely used in industry.
And now a team releases Picotron, a radical approach to code 4D Parallelism in just a few hundred lines of code, a real engineering prowess, making it much easier to understand what's actually happening!
โก ๐๐'๐ ๐๐ถ๐ป๐, ๐๐ฒ๐ ๐ฝ๐ผ๐๐ฒ๐ฟ๐ณ๐๐น:
Counting in MFU (Model FLOPs Utilization, how much the model actually uses all the compute potential), this lib reaches ~50% on SmolLM-1.7B model with 8 H100 GPUs, which is really close to what huge libs would reach. (Caution: the team is leading further benchmarks to verify this)
Go take a look ๐ https://github.com/huggingface/picotron/tree/main/picotron
- 1 reply