Smol Community

community

AI & ML interests

The SmolTuners group is a community dedicated to the development of small-scale Large Language Models (LLMs) using consumer-grade GPUs.

Recent Activity

SmolTuners's activity

KnutJaegersbergĀ 
posted an update 2 days ago
view post
Post
1690
Evolution and The Knightian Blindspot of Machine Learning


The paper discusses machine learning's limitations in addressing Knightian Uncertainty (KU), highlighting the fragility of models like reinforcement learning (RL) in unpredictable, open-world environments. KU refers to uncertainty that can't be quantified or predicted, a challenge that RL fails to handle due to its reliance on fixed data distributions and limited formalisms.


### Key Approaches:

1. **Artificial Life (ALife):** Simulating diverse, evolving systems to generate adaptability, mimicking biological evolution's robustness to unpredictable environments.

2. **Open-Endedness:** Creating AI systems capable of continuous innovation and adaptation, drawing inspiration from human creativity and scientific discovery.

3. **Revising RL Formalisms:** Modifying reinforcement learning (RL) models to handle dynamic, open-world environments by integrating more flexible assumptions and evolutionary strategies.

These approaches aim to address MLā€™s limitations in real-world uncertainty and move toward more adaptive, general intelligence.

https://arxiv.org/abs/2501.13075
Delta-VectorĀ 
posted an update 3 days ago
KnutJaegersbergĀ 
posted an update 4 days ago
view post
Post
1987
Artificial Kuramoto Oscillatory Neurons

Artificial Kuramoto Oscillatory Neurons (AKOrN) differ from traditional artificial neurons by oscillating, rather than just turning on or off. Each neuron is represented by a rotating vector on a sphere, influenced by its connections to other neurons. This behavior is based on the Kuramoto model, which describes how oscillators (like neurons) tend to synchronize, similar to pendulums swinging in unison.

Key points:

Oscillating Neurons: Each AKOrNā€™s rotation is influenced by its connections, and they try to synchronize or oppose each other.
Synchronization: When neurons synchronize, they "bind," allowing the network to represent complex concepts (e.g., "a blue square toy") by compressing information.
Updating Mechanism: Neurons update their rotations based on connected neurons, input stimuli, and their natural frequency, using a Kuramoto update formula.
Network Structure: AKOrNs can be used in various network layers, with iterative blocks combining Kuramoto layers and feature extraction modules.
Reasoning: This model can perform reasoning tasks, like solving Sudoku puzzles, by adjusting neuron interactions.
Advantages: AKOrNs offer robust feature binding, reasoning capabilities, resistance to adversarial data, and well-calibrated uncertainty estimation.
In summary, AKOrN's oscillatory neurons and synchronization mechanisms enable the network to learn, reason, and handle complex tasks like image classification and object discovery with enhanced robustness and flexibility.

yt
https://www.youtube.com/watch?v=i3fRf6fb9ZM
paper
https://arxiv.org/html/2410.13821v1
  • 2 replies
Ā·
KnutJaegersbergĀ 
posted an update 5 days ago
KnutJaegersbergĀ 
posted an update 9 days ago
KnutJaegersbergĀ 
posted an update 10 days ago
view post
Post
1751
Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI

It's an interesting paper that argues "new approaches are required that can reliably solve a wide variety of problems without existing skills."
"It is therefore hoped that the benchmark outlined in this article contributes to further exploration of this direction of research and incentivises the development of new AGI approaches that focus on intelligence rather than skills."

https://arxiv.org/abs/2501.07458
s3nhĀ 
in SmolTuners/README 15 days ago

Gh organization

8
#3 opened about 1 month ago by
s3nh
AnA202Ā 
in SmolTuners/README 15 days ago

Gh organization

8
#3 opened about 1 month ago by
s3nh
KnutJaegersbergĀ 
posted an update 16 days ago

Gh organization

8
#3 opened about 1 month ago by
s3nh
CaioXapelaumĀ 
in SmolTuners/README 26 days ago

Gh organization

8
#3 opened about 1 month ago by
s3nh
s3nhĀ 
updated a Space about 1 month ago
Delta-VectorĀ 
in SmolTuners/README about 1 month ago

Gh organization

8
#3 opened about 1 month ago by
s3nh
s3nhĀ 
in SmolTuners/README about 1 month ago

Optimizers

#2 opened about 1 month ago by
s3nh

Datasets

3
#1 opened about 1 month ago by
s3nh
Delta-VectorĀ 
in SmolTuners/README about 1 month ago

Datasets

3
#1 opened about 1 month ago by
s3nh
KnutJaegersbergĀ 
posted an update about 1 month ago
s3nhĀ 
posted an update about 1 month ago
view post
Post
1821
Welcome back,

Small Language Models Enthusiasts and GPU Poor oss enjoyers lets connect.
Just created an organization which main target is to have fun with smaller models tuneable on consumer range GPUs, feel free to join and lets have some fun, much love ;3

https://huggingface.co/SmolTuners
Ā·
KnutJaegersbergĀ 
posted an update about 2 months ago
KnutJaegersbergĀ 
posted an update 2 months ago