🎯 Phonemic Transcription Leaderboard

Welcome to the Phonemic Transcription Leaderboard! This simple leaderboard helps you track and compare the performance of different speech-to-phoneme model. Feel free to use it for your own hugging face leaderboards!

✨ Features

📊 Interactive leaderboard with real-time sorting
🔄 Easy model submission system
📈 Automatic evaluation of submitted models
📱 Responsive design that works on all devices

🎯 What This Project Does

This leaderboard tracks two key metrics for phonemic transcription models:

PER (Phoneme Error Rate): How accurately your model converts speech to phonemes
PWED (Phoneme Weighted Edit Distance): A more nuanced metric that considers phonemic features

Models are evaluated on the TIMIT speech corpus, a gold standard in speech recognition research.

🚀 Getting Started

Prerequisites

Python 3.10
Git
A love for speech recognition! 🎤

Quick Installation

Clone this repository:

git clone [your-repo-url]
cd phonemic-leaderboard

Set up your environment:

# Create a virtual environment with Python 3.10
python3.10 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Install the required dependencies
pip install -r requirements.txt

Launch the leaderboard:

# Run the application
uvicorn app:app --host 0.0.0.0 --port 7860

Visit http://localhost:7860 in your browser and see the magic! ✨

🎮 Using the Leaderboard

Submitting a Model

Go to the "Submit Model" tab
Enter your model details:
- Model name (e.g., "wav2vec2-phoneme-wizard")
- Submission name (e.g., "MyAwesomeModel v1.0")
- GitHub/Kaggle/HuggingFace URL (optional)
Click Submit and watch your model climb the ranks! 🚀

Checking Model Status

Navigate to the "Model Status" tab
Enter your model name or task ID
Get real-time updates on your model's evaluation progress

📊 Understanding the Results

The leaderboard shows:

Model names and submission details
PER and PWED scores (lower is better!)
Links to model repositories
Submission dates

Sort by either metric to see who's leading the pack!

🛠️ Technical Details

Built with Gradio for a smooth UI experience
Runs on a basic compute plan (16GB RAM, 2vCPUs) for easy reproducibility
Evaluation can take several hours - perfect time to grab a coffee ☕

🤝 Contributing

Want to make this leaderboard even better? We'd love your help! Here are some ways you can contribute:

Add new evaluation metrics
Improve the UI design
Enhance documentation
Submit bug fixes
Add new features

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🌟 Acknowledgments

Thanks to the TIMIT speech corpus for providing evaluation data
Shoutout to the panphon library for PWED calculations
Built with love by Koel Labs 💙

🆘 Need Help?

Got questions? Found a bug? Want to contribute? Open an issue or reach out to us! We're here to help make speech recognition evaluation fun and accessible for everyone!

Remember: Every great model deserves its moment to shine! 🌟

Happy Transcribing! 🎤✨