AI & ML interests

Making funny and goofy LM's and AI's

Recent Activity

FlameF0XΒ  updated a Space 8 days ago
GoofyLM/GoofyLM
FlameF0XΒ  updated a model 8 days ago
GoofyLM/N2-Nemo
FlameF0XΒ  updated a collection 8 days ago
Nx
View all activity

FlameF0XΒ 
posted an update 2 days ago
view post
Post
116
Hello there world! I am happy to announce that you now can fine-tune FlameF0X/SnowflakeCore-G1-Tiny , the code for that is in the model card.

I aslo lost the training log 😐
FlameF0XΒ 
posted an update 6 days ago
view post
Post
1195
Hello! I am sad to say but fine-tuning FlameF0X/SnowflakeCore-G1-Tiny is complicated and the instruct version would need to wait some time.
  • 2 replies
Β·
FlameF0XΒ 
posted an update 18 days ago
FlameF0XΒ 
posted an update 19 days ago
view post
Post
250
SnowflakeCore-G1 Update:
Got it running and training! Context window is currently set to 2048 tokens.
Training is active and stable. Will share results once I have some metrics to report.
  • 2 replies
Β·
FlameF0XΒ 
posted an update 20 days ago
view post
Post
1929
SnowflakeCore-G1 development update: We're building a 24-layer transformer with 32K context and 1024 embedding dimensions - pretty ambitious! Even running at batch_size=1 with heavy gradient accumulation, we're hitting memory walls at 300GB RAM. Scaling up to ~1TB will take some time, but the architecture is looking promising. Thanks for following along with the journey! πŸ˜…
  • 1 reply
Β·
FlameF0XΒ 
updated a Space 25 days ago
FlameF0XΒ 
posted an update 27 days ago
view post
Post
1149
Hello there!
I just find out that all the SnowflakeCore-G0 series are Mask Language Models instead of LLM's.
The development of SnowflakeCore-G0-Releas-3 would be delayed even more.

Edit: I officially end the development of SnowflakeCore-G0 and start the development of SnowflakeCore-G1 what SHOULD be the text generator.

Edit-2: After some evaluation of the code, the models are actual Text Generator. So the development of G0 will continue.
FlameF0XΒ 
posted an update 29 days ago
view post
Post
1369
Hi everyone!
The release of https://huggingface.co/FlameF0X/SnowflakeCore-G0-Release-3-1B is currently delayed due to hardware limitationsβ€”I'm currently lacking the compute resources needed to complete training. I'm exploring options and will keep you updated on any progress.
Thank you for your patience and support!