DEVELOPMENT.md

Development

Design Decisions

We specifically opt for a single-space leaderboard for simplicity. We solve the issue of keeping the gradio UI interactive while models are evaluating by using background tasks instead of a separate space.

Setup

Prerequisites

• Python 3.10
• Git
• A love for speech recognition! 🎤

Quick Installation

1. Clone this repository:

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/spaces/KoelLabs/IPA-Transcription-EN
cd IPA-Transcription-EN

2. Set up your environment and download data:

. ./scripts/install.sh

3. Launch the leaderboard in development mode (auto-reloads on code changes):

. ./scripts/run-dev.sh

4. Visit http://localhost:7860 in your browser and see the magic! ✨

Adding/Removing Dependencies

0. Activate the virtual environment with . ./venv/bin/activate
1. Add the dependency to requirements.txt (or remove it)
2. Make sure you have no unused dependencies with pipx run deptry .
3. Run pip install -r requirements.txt
4. Freeze the dependencies with pip freeze > requirements_lock.txt

Run without reloading

. ./scripts/run-prod.sh

File Structure

The two most imporant files are app/app.py for the main gradio UI and app/tasks.py for the background tasks that evaluate models.

IPA-Transcription-EN/
├── README.md                   # General information about the leaderboard
├── CONTRIBUTING.md             # Contribution guidelines
├── DEVELOPMENT.md              # Development setup and design decisions
├── requirements.txt            # Python dependencies
├── requirements_lock.txt       # Locked dependencies
├── scripts                     # Helper scripts    
│   ├── install.sh              # Install dependencies and download data
│   └── run-dev.sh              # Run the leaderboard in development mode
├── venv                        # Virtual environment
├── app/                        # All application code lives here
│   ├── data/                   # Phoneme transcription datasets
│   ├── queue/                  # Stores leaderboard state and task status
│   |   ├── tasks.json          # Task queue
│   |   ├── results.json        # Detailed evaluation results
│   |   └── leaderboard.json    # Compact results for leaderboard display
│   ├── app.py                  # Main Gradio UI
│   ├── tasks.py                # Background tasks for model evaluation
│   ├── data.py                 # Data loading and processing
│   ├── inference.py            # Model inference
│   └── phone_metrics.py        # Evaluation metrics
└── img/                        # Images for README and other documentation