lmz/moshi-swift
Updated
Laurent Mazare
lmz
AI & ML interests
None yet
Recent Activity
updated
a model
10 days ago
lmz/moshi-swift
liked
a Space
23 days ago
freddyaboulton/talk-to-moshi
liked
a model
about 1 month ago
kyutai/mimi
Organizations
lmz's activity
updated
a
model
10 days ago
liked
a
Space
23 days ago
liked
a
model
about 1 month ago
reacted to
reach-vb's
post with 🔥
3 months ago
Post
2842
Less than two days ago Kyutai Labs open sourced Moshi - an ~7.6B on-device Speech to Speech foundation model and Mimi - SoTA streaming speech codec! 🔥
The release includes:
1. Moshiko & Moshika - Moshi finetuned on synthetic data (CC-BY license) ( kyutai/moshi-v01-release-66eaeaf3302bef6bd9ad7acd)
2. Mimi - Streaiming Audio Codec, processes 24 kHz audio, down to a 12.5 Hz representation with a bandwidth of 1.1 kbps (CC-BY license) ( kyutai/mimi)
3. Model checkpoints & Inference codebase written in Rust (Candle), PyTorch & MLX (Apache license) (https://github.com/kyutai-labs/moshi)
How does Moshi work?
1. Moshi processes two audio streams: one for itself and one for the user, with the user's stream coming from audio input and Moshi's stream generated by the model.
2. Along with these audio streams, Moshi predicts text tokens for its speech, enhancing its generation quality.
3. The model uses a small Depth Transformer for codebook dependencies and a large 7B parameter Temporal Transformer for temporal dependencies.
4. The theoretical latency is 160ms, with a practical latency of around 200ms on an L4 GPU.
Model size & inference:
Moshiko/ka are 7.69B param models
bf16 ~16GB VRAM
8-bit ~8GB VRAM
4-bit ~4GB VRAM
You can run inference via Candle 🦀, PyTorch and MLX - based on your hardware.
The Kyutai team, @adefossez @lmz and team are cracked AF, they're bringing some serious firepower to the open source/ science AI scene, looking forward to what's next! 🐐
The release includes:
1. Moshiko & Moshika - Moshi finetuned on synthetic data (CC-BY license) ( kyutai/moshi-v01-release-66eaeaf3302bef6bd9ad7acd)
2. Mimi - Streaiming Audio Codec, processes 24 kHz audio, down to a 12.5 Hz representation with a bandwidth of 1.1 kbps (CC-BY license) ( kyutai/mimi)
3. Model checkpoints & Inference codebase written in Rust (Candle), PyTorch & MLX (Apache license) (https://github.com/kyutai-labs/moshi)
How does Moshi work?
1. Moshi processes two audio streams: one for itself and one for the user, with the user's stream coming from audio input and Moshi's stream generated by the model.
2. Along with these audio streams, Moshi predicts text tokens for its speech, enhancing its generation quality.
3. The model uses a small Depth Transformer for codebook dependencies and a large 7B parameter Temporal Transformer for temporal dependencies.
4. The theoretical latency is 160ms, with a practical latency of around 200ms on an L4 GPU.
Model size & inference:
Moshiko/ka are 7.69B param models
bf16 ~16GB VRAM
8-bit ~8GB VRAM
4-bit ~4GB VRAM
You can run inference via Candle 🦀, PyTorch and MLX - based on your hardware.
The Kyutai team, @adefossez @lmz and team are cracked AF, they're bringing some serious firepower to the open source/ science AI scene, looking forward to what's next! 🐐
- 1 reply
updated
a
Space
3 months ago
updated
a
model
5 months ago
updated
a
model
7 months ago
updated
a
model
9 months ago
updated
2
models
10 months ago
updated
2
models
11 months ago
yolo-v3.cfg cannot be parsed successfully.
1
#1 opened 12 months ago
by
Steve0010
fix model reference
1
#6 opened 12 months ago
by
radames
updated
a
Space
12 months ago
Update phi-2_0.json
#8 opened about 1 year ago
by
radames
updated
a
model
about 1 year ago
Upload phi-2_0.json
#7 opened about 1 year ago
by
radames
phi2-quantized-files
2
#6 opened about 1 year ago
by
radames
updated
a
model
about 1 year ago