major refactor

DEPLOYMENT.md

@@ -0,0 +1,66 @@
+# Deployment Guide for Hugging Face Spaces
+
+This repository is structured for deployment on Hugging Face Spaces using Docker.
+
+## Repository Structure
+
+```
+icelandic-llm-leaderboard-hf/
+├── README.md                 # HF Spaces metadata and description
+├── Dockerfile               # Multi-stage Docker build
+├── LICENSE                  # Apache 2.0 license
+├── .gitignore              # Git ignore patterns
+├── DEPLOYMENT.md           # This file
+├── backend/                # FastAPI backend
+│   ├── app/               # Application code
+│   │   ├── api/          # API routes
+│   │   ├── config/       # Configuration
+│   │   ├── core/         # Core functionality
+│   │   └── services/     # Business logic
+│   ├── original_src/     # Original Icelandic leaderboard logic
+│   └── pyproject.toml    # Python dependencies
+└── frontend/               # React frontend
+    ├── build/             # Production build artifacts
+    ├── src/              # Source code
+    ├── public/           # Static assets
+    ├── package.json      # Node.js dependencies
+    └── server.js         # Express production server
+```
+
+## HF Spaces Configuration
+
+The README.md contains the required HF Spaces metadata:
+- SDK: docker
+- OAuth: enabled for HF authentication
+- Tags: leaderboard, icelandic, language evaluation
+- License: Apache 2.0
+
+## Deployment Process
+
+1. **Upload to HF Spaces**: Upload this entire directory structure to your HF Space
+2. **Environment Variables**: Set HF_TOKEN in your Space settings if needed
+3. **Build**: HF Spaces will automatically build using the Dockerfile
+4. **Access**: Your leaderboard will be available at your Space URL
+
+## Architecture
+
+- **Frontend**: React SPA served by Express on port 7860
+- **Backend**: FastAPI server on port 7861
+- **Proxy**: Express proxies `/api/*` requests to FastAPI
+- **Data**: Pulls from HF repositories (mideind/icelandic-llm-leaderboard-*)
+
+## Key Features
+
+- Real-time leaderboard with Icelandic benchmarks
+- Interactive filtering and search
+- Model comparison and pinning
+- Responsive design with dark/light themes
+- Automatic data synchronization from HF repositories
+
+## Environment Variables
+
+- `HF_TOKEN`: Hugging Face API token (optional, can use HF OAuth)
+- `PORT`: Frontend server port (default: 7860)
+- `INTERNAL_API_PORT`: Backend server port (default: 7861)
+
+The application will automatically use HF OAuth for authentication when deployed on HF Spaces.

Dockerfile_

@@ -0,0 +1,62 @@
+# Build frontend
+FROM node:18 as frontend-build
+WORKDIR /app
+COPY frontend/package*.json ./
+RUN npm install
+COPY frontend/ ./
+
+RUN npm run build
+
+# Build backend
+FROM python:3.12-slim
+WORKDIR /app
+
+# Create non-root user
+RUN useradd -m -u 1000 user
+
+# Install poetry
+RUN pip install poetry
+
+# Create and configure cache directory
+RUN mkdir -p /app/.cache && \
+    chown -R user:user /app
+
+# Copy and install backend dependencies
+COPY backend/pyproject.toml backend/poetry.lock* ./
+RUN poetry config virtualenvs.create false \
+    && poetry install --no-interaction --no-ansi --no-root --only main
+
+# Copy backend code
+COPY backend/ .
+
+# Install Node.js and npm
+RUN apt-get update && apt-get install -y \
+    curl \
+    netcat-openbsd \
+    && curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
+    && apt-get install -y nodejs \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy frontend server and build
+COPY --from=frontend-build /app/build ./frontend/build
+COPY --from=frontend-build /app/package*.json ./frontend/
+COPY --from=frontend-build /app/server.js ./frontend/
+
+# Install frontend production dependencies
+WORKDIR /app/frontend
+RUN npm install --production
+WORKDIR /app
+
+# Environment variables
+ENV HF_HOME=/app/.cache \
+    HF_DATASETS_CACHE=/app/.cache \
+    INTERNAL_API_PORT=7861 \
+    PORT=7860 \
+    NODE_ENV=production
+
+# Note: HF_TOKEN should be provided at runtime, not build time
+USER user
+EXPOSE 7860
+
+# Start both servers with wait-for
+CMD ["sh", "-c", "uvicorn app.asgi:app --host 0.0.0.0 --port 7861 & while ! nc -z localhost 7861; do sleep 1; done && cd frontend && npm run serve"]

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship covered by this License,
+      whether in Source or Object form, made available under the License,
+      as indicated by a copyright notice that is included in or attached
+      to the work. (Such copyright notice may also be included in a file
+      accompanying the work.)
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based upon (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and derivative works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control
+      systems, and issue tracking systems that are managed by, or on behalf
+      of, the Licensor for the purpose of discussing and improving the Work,
+      but excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to use, reproduce, modify, display, perform,
+      sublicense, and distribute the Work and in such Derivative Works
+      in Source and Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, trademark, patent,
+          attribution and other notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright notice and/or license for Your
+      own additions to the Work, and may provide additional or different
+      license terms and conditions for use, reproduction, or distribution
+      of Your additions to the Work, or for any such Derivative Works as a
+      whole, provided Your use, reproduction, and distribution of the
+      Work otherwise complies with the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Support. When redistributing the Work or
+      Derivative Works thereof, You may choose to offer, and charge a fee
+      for, acceptance of support, warranty, indemnity, or other liability
+      obligations and/or rights consistent with this License. However, in
+      accepting such obligations, You may act only on Your own behalf and
+      on Your sole responsibility, not on behalf of any other Contributor,
+      and only if You agree to indemnify, defend, and hold each Contributor
+      harmless for any liability incurred by, or claims asserted against,
+      such Contributor by reason of your accepting any such warranty or
+      support.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same page as the copyright notice for easier identification within
+      third-party archives.
+
+   Copyright [2024] [Mideind ehf.]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,10 +1,70 @@
 ---
-title: Icelandic Llm Leaderboard Hf
-emoji: 😻
-colorFrom: green
-colorTo: indigo
+title: Icelandic LLM Leaderboard
+emoji: 🇮🇸
+colorFrom: blue
+colorTo: green
 sdk: docker
-pinned: false
+hf_oauth: true
+pinned: true
+license: apache-2.0
+tags:
+- leaderboard
+- modality:text
+- submission:automatic
+- test:public
+- language:icelandic
+- eval:language
+short_description: Track, rank and evaluate LLMs on Icelandic language tasks
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Icelandic LLM Leaderboard 🇮🇸
+
+A comprehensive leaderboard for evaluating Large Language Models (LLMs) on Icelandic language tasks. This leaderboard tracks model performance across various Icelandic benchmarks including WinoGrande-IS, GED, Inflection, Belebele-IS, ARC-Challenge-IS, and WikiQA-IS.
+
+## Features
+
+- 📊 Interactive table with advanced sorting and filtering
+- 🔍 Semantic model search with regex support
+- 📌 Pin models for easy comparison
+- 📱 Responsive and modern React interface
+- 🎨 Dark/Light mode support
+- ⚡️ Optimized performance with virtualization
+- 🇮🇸 Specialized for Icelandic language evaluation
+
+## Benchmarks
+
+### Core Icelandic Tasks
+- **WinoGrande-IS (3-shot)**: Icelandic common sense reasoning
+- **GED**: Grammatical error detection in Icelandic
+- **Inflection (1-shot)**: Icelandic morphological inflection
+- **Belebele-IS**: Icelandic reading comprehension
+- **ARC-Challenge-IS**: Icelandic science questions
+- **WikiQA-IS**: Icelandic question answering
+
+## Architecture
+
+The leaderboard uses a modern React frontend with a FastAPI backend, containerized with Docker for seamless deployment on Hugging Face Spaces.
+
+### Frontend (React)
+- Material-UI components
+- TanStack Table for advanced data handling
+- Real-time filtering and search capabilities
+
+### Backend (FastAPI)  
+- Integration with Hugging Face repositories
+- Automatic data synchronization
+- RESTful API endpoints
+
+## Data Sources
+
+The leaderboard pulls evaluation results from:
+- **Results Repository**: `mideind/icelandic-llm-leaderboard-results`
+- **Requests Repository**: `mideind/icelandic-llm-leaderboard-requests`
+
+## Contributing
+
+To submit a model for evaluation, please follow the submission guidelines in the leaderboard interface.
+
+## License
+
+Apache 2.0 License - see LICENSE file for details.

backend/app/__init__.py

@@ -0,0 +1 @@
+# Icelandic LLM Leaderboard Backend

backend/app/api/__init__.py

@@ -0,0 +1,3 @@
+from .router import router
+
+__all__ = ["router"]

backend/app/api/endpoints/__init__.py

@@ -0,0 +1,3 @@
+from .leaderboard import router as leaderboard_router
+
+__all__ = ["leaderboard_router"]

backend/app/api/endpoints/leaderboard.py

@@ -0,0 +1,49 @@
+from fastapi import APIRouter
+from typing import List, Dict, Any
+import logging
+
+from app.services.leaderboard import IcelandicLeaderboardService
+from app.core.fastapi_cache import cached, build_cache_key
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+leaderboard_service = IcelandicLeaderboardService()
+
+def leaderboard_key_builder(func, namespace: str = "icelandic_leaderboard", **kwargs):
+    """Build cache key for Icelandic leaderboard data"""
+    key_type = "raw" if func.__name__ == "get_leaderboard" else "formatted"
+    key = build_cache_key(namespace, key_type)
+    logger.debug(f"Built Icelandic leaderboard cache key: {key}")
+    return key
+
+@router.get("")
+@cached(expire=300, key_builder=leaderboard_key_builder)
+async def get_leaderboard() -> List[Dict[str, Any]]:
+    """
+    Get raw Icelandic leaderboard data
+    Response will be automatically GZIP compressed if size > 500 bytes
+    """
+    try:
+        logger.info("Fetching raw Icelandic leaderboard data")
+        data = await leaderboard_service.fetch_raw_data()
+        logger.info(f"Retrieved {len(data)} Icelandic leaderboard entries")
+        return data
+    except Exception as e:
+        logger.error(f"Failed to fetch raw Icelandic leaderboard data: {e}")
+        raise
+
+@router.get("/formatted")
+@cached(expire=300, key_builder=leaderboard_key_builder)
+async def get_formatted_leaderboard() -> List[Dict[str, Any]]:
+    """
+    Get formatted Icelandic leaderboard data with restructured objects
+    Response will be automatically GZIP compressed if size > 500 bytes
+    """
+    try:
+        logger.info("Fetching formatted Icelandic leaderboard data")
+        data = await leaderboard_service.get_formatted_data()
+        logger.info(f"Retrieved {len(data)} formatted Icelandic entries")
+        return data
+    except Exception as e:
+        logger.error(f"Failed to fetch formatted Icelandic leaderboard data: {e}")
+        raise

backend/app/api/router.py

@@ -0,0 +1,7 @@
+from fastapi import APIRouter
+from app.api.endpoints import leaderboard_router
+
+router = APIRouter()
+
+# Include all endpoint routers
+router.include_router(leaderboard_router, prefix="/leaderboard", tags=["leaderboard"])

backend/app/asgi.py

@@ -0,0 +1,111 @@
+"""
+ASGI entry point for the Icelandic LLM Leaderboard API.
+"""
+import os
+import logging
+import logging.config
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.middleware.gzip import GZipMiddleware
+
+from app.api.router import router
+from app.core.fastapi_cache import setup_cache
+from app.config import hf_config
+
+# Configure logging
+LOGGING_CONFIG = {
+    "version": 1,
+    "disable_existing_loggers": True,
+    "formatters": {
+        "default": {
+            "format": "%(name)s - %(levelname)s - %(message)s",
+        }
+    },
+    "handlers": {
+        "default": {
+            "formatter": "default",
+            "class": "logging.StreamHandler",
+            "stream": "ext://sys.stdout",
+        }
+    },
+    "loggers": {
+        "uvicorn": {
+            "handlers": ["default"],
+            "level": "WARNING",
+            "propagate": False,
+        },
+        "uvicorn.error": {
+            "level": "WARNING", 
+            "handlers": ["default"],
+            "propagate": False,
+        },
+        "uvicorn.access": {
+            "handlers": ["default"],
+            "level": "WARNING",
+            "propagate": False,
+        },
+        "app": {
+            "handlers": ["default"],
+            "level": "INFO",
+            "propagate": False,
+        }
+    },
+    "root": {
+        "handlers": ["default"],
+        "level": "INFO",
+    }
+}
+
+# Apply logging configuration
+logging.config.dictConfig(LOGGING_CONFIG)
+logger = logging.getLogger("app")
+
+# Create FastAPI application
+app = FastAPI(
+    title="Icelandic LLM Leaderboard",
+    version="1.0.0",
+    docs_url="/docs",
+)
+
+# Add CORS middleware
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# Add GZIP compression
+app.add_middleware(GZipMiddleware, minimum_size=500)
+
+# Include API router
+app.include_router(router, prefix="/api")
+
+@app.on_event("startup")
+async def startup_event():
+    """Initialize services on startup"""
+    logger.info("🇮🇸 ICELANDIC LLM LEADERBOARD STARTING UP")
+    
+    # Log HF configuration
+    logger.info(f"Organization: {hf_config.HF_ORGANIZATION}")
+    logger.info(f"Token Status: {'Present' if hf_config.HF_TOKEN else 'Missing'}")
+    logger.info(f"Using repositories:")
+    logger.info(f"  - Queue: {hf_config.QUEUE_REPO}")
+    logger.info(f"  - Results: {hf_config.RESULTS_REPO}")
+    
+    # Setup cache
+    setup_cache()
+    logger.info("FastAPI Cache initialized")
+    
+    logger.info("🚀 Icelandic LLM Leaderboard ready!")
+
+@app.get("/")
+async def root():
+    """Root endpoint"""
+    return {"message": "Icelandic LLM Leaderboard API", "status": "running"}
+
+@app.get("/health")
+async def health_check():
+    """Health check endpoint"""
+    return {"status": "healthy"}

backend/app/config/__init__.py

@@ -0,0 +1,21 @@
+from .hf_config import (
+    HF_TOKEN,
+    HF_ORGANIZATION,
+    REPO_ID,
+    QUEUE_REPO,
+    RESULTS_REPO,
+    EVAL_REQUESTS_PATH,
+    EVAL_RESULTS_PATH,
+    API
+)
+
+__all__ = [
+    "HF_TOKEN",
+    "HF_ORGANIZATION", 
+    "REPO_ID",
+    "QUEUE_REPO",
+    "RESULTS_REPO",
+    "EVAL_REQUESTS_PATH",
+    "EVAL_RESULTS_PATH",
+    "API"
+]

backend/app/config/hf_config.py

@@ -0,0 +1,41 @@
+import os
+from pathlib import Path
+from huggingface_hub import HfApi
+
+# Load environment variables from .env file
+try:
+    from dotenv import load_dotenv
+    # Look for .env file in the project root
+    env_path = Path(__file__).parent.parent.parent / ".env"
+    if env_path.exists():
+        load_dotenv(env_path)
+        print(f"Loaded .env from: {env_path}")
+    else:
+        # Try loading from current directory
+        load_dotenv()
+        print("Loaded .env from current directory")
+except ImportError:
+    print("python-dotenv not available, using system environment only")
+
+# Configuration for Icelandic LLM Leaderboard
+HF_TOKEN = os.environ.get("HF_TOKEN")
+HF_ORGANIZATION = "mideind"
+
+# Debug: Print token status (first 10 chars only for security)
+if HF_TOKEN:
+    print(f"HF_TOKEN loaded: {HF_TOKEN[:10]}...")
+else:
+    print("HF_TOKEN not found in environment")
+
+# Repository configuration
+REPO_ID = f"{HF_ORGANIZATION}/icelandic-llm-leaderboard"
+QUEUE_REPO = f"{HF_ORGANIZATION}/icelandic-llm-leaderboard-requests"
+RESULTS_REPO = f"{HF_ORGANIZATION}/icelandic-llm-leaderboard-results"
+
+# Local cache paths
+HF_HOME = os.getenv("HF_HOME", ".")
+EVAL_REQUESTS_PATH = os.path.join(HF_HOME, "eval-queue")
+EVAL_RESULTS_PATH = os.path.join(HF_HOME, "eval-results")
+
+# API instance
+API = HfApi(token=HF_TOKEN)

backend/app/core/cache.py

@@ -0,0 +1,33 @@
+import os
+from pathlib import Path
+from datetime import timedelta
+
+class CacheConfig:
+    def __init__(self):
+        self.base_path = Path(os.getenv("HF_HOME", "."))
+        self.cache_ttl = timedelta(minutes=5)  # 5 minute cache TTL
+        
+    def get_cache_path(self, cache_type: str = "datasets") -> Path:
+        """Get cache path for different cache types"""
+        cache_path = self.base_path / "cache" / cache_type
+        cache_path.mkdir(parents=True, exist_ok=True)
+        return cache_path
+    
+    def flush_cache(self, cache_type: str = None):
+        """Flush specific cache or all caches"""
+        if cache_type:
+            cache_path = self.get_cache_path(cache_type)
+            for file in cache_path.glob("*"):
+                if file.is_file():
+                    file.unlink()
+        else:
+            cache_base = self.base_path / "cache"
+            if cache_base.exists():
+                for cache_dir in cache_base.iterdir():
+                    if cache_dir.is_dir():
+                        for file in cache_dir.glob("*"):
+                            if file.is_file():
+                                file.unlink()
+
+# Global cache configuration
+cache_config = CacheConfig()

backend/app/core/fastapi_cache.py

@@ -0,0 +1,63 @@
+import asyncio
+import json
+import hashlib
+from typing import Any, Callable, Dict, Optional
+from functools import wraps
+import logging
+
+logger = logging.getLogger(__name__)
+
+# Simple in-memory cache
+_cache: Dict[str, Any] = {}
+_cache_lock = asyncio.Lock()
+
+def build_cache_key(namespace: str, *args, **kwargs) -> str:
+    """Build a cache key from namespace and parameters"""
+    key_data = f"{namespace}:{args}:{sorted(kwargs.items())}"
+    return hashlib.md5(key_data.encode()).hexdigest()
+
+def cached(expire: int = 300, key_builder: Optional[Callable] = None):
+    """
+    Cache decorator for FastAPI endpoints
+    
+    Args:
+        expire: Cache expiration time in seconds
+        key_builder: Function to build cache key
+    """
+    def decorator(func: Callable) -> Callable:
+        @wraps(func)
+        async def wrapper(*args, **kwargs):
+            # Build cache key
+            if key_builder:
+                cache_key = key_builder(func, **kwargs)
+            else:
+                cache_key = build_cache_key(func.__name__, *args, **kwargs)
+            
+            # Check cache
+            async with _cache_lock:
+                if cache_key in _cache:
+                    cached_data, timestamp = _cache[cache_key]
+                    import time
+                    if time.time() - timestamp < expire:
+                        logger.debug(f"Cache hit for key: {cache_key}")
+                        return cached_data
+                    else:
+                        # Expired, remove from cache
+                        del _cache[cache_key]
+            
+            # Cache miss, execute function
+            logger.debug(f"Cache miss for key: {cache_key}")
+            result = await func(*args, **kwargs)
+            
+            # Store in cache
+            async with _cache_lock:
+                import time
+                _cache[cache_key] = (result, time.time())
+            
+            return result
+        return wrapper
+    return decorator
+
+def setup_cache():
+    """Setup cache configuration"""
+    logger.info("FastAPI cache initialized with in-memory backend")

backend/app/services/__init__.py

@@ -0,0 +1,3 @@
+from .leaderboard import IcelandicLeaderboardService
+
+__all__ = ["IcelandicLeaderboardService"]

backend/app/services/leaderboard.py

@@ -0,0 +1,277 @@
+import os
+import json
+import logging
+from typing import List, Dict, Any
+from pathlib import Path
+from huggingface_hub import snapshot_download
+from fastapi import HTTPException
+
+from app.config import (
+    QUEUE_REPO,
+    RESULTS_REPO,
+    EVAL_REQUESTS_PATH,
+    EVAL_RESULTS_PATH,
+    HF_TOKEN
+)
+from app.core.cache import cache_config
+
+logger = logging.getLogger(__name__)
+
+# Import original processing logic
+import sys
+import os
+
+# Add the original Icelandic leaderboard source to Python path
+original_src_path = os.path.join(os.path.dirname(__file__), '..', '..', 'original_src')
+if original_src_path not in sys.path:
+    sys.path.insert(0, original_src_path)
+
+# Also add the parent directory so imports like 'src.display.utils' work
+backend_path = os.path.join(os.path.dirname(__file__), '..', '..')
+if backend_path not in sys.path:
+    sys.path.insert(0, backend_path)
+
+try:
+    from leaderboard.read_evals import get_raw_eval_results
+    from populate import get_leaderboard_df
+    from display.utils import COLS, BENCHMARK_COLS, Tasks
+except ImportError as e:
+    # Fallback for development without mounted volume
+    logger.warning(f"Could not import original modules: {e}")
+    # Define minimal fallbacks
+    COLS = ["Model", "Average ⬆️", "Type", "Precision", "Architecture", "Hub License", "Hub ❤️", "#Params (B)", "Available on the hub", "Model sha"]
+    BENCHMARK_COLS = ["WinoGrande-IS (3-shot)", "GED", "Inflection (1-shot)", "Belebele (IS)", "ARC-Challenge-IS", "WikiQA-IS"]
+    
+    class MockTask:
+        def __init__(self, name, col_name):
+            self.name = name
+            self.col_name = col_name
+    
+    class Tasks:
+        task0 = MockTask("winogrande_is", "WinoGrande-IS (3-shot)")
+        task1 = MockTask("ged", "GED") 
+        task2 = MockTask("inflection", "Inflection (1-shot)")
+        task5 = MockTask("belebele_is", "Belebele (IS)")
+        task6 = MockTask("arc_challenge_is", "ARC-Challenge-IS")
+        task7 = MockTask("wiki_qa_is", "WikiQA-IS")
+
+class IcelandicLeaderboardService:
+    def __init__(self):
+        self.results_path = EVAL_RESULTS_PATH
+        self.requests_path = EVAL_REQUESTS_PATH
+        
+    async def _ensure_data_available(self):
+        """Ensure evaluation data is available locally"""
+        try:
+            # Download results if not exists or empty
+            if not os.path.exists(self.results_path) or not os.listdir(self.results_path):
+                logger.info(f"Downloading results to {self.results_path}")
+                snapshot_download(
+                    repo_id=RESULTS_REPO,
+                    local_dir=self.results_path,
+                    repo_type="dataset",
+                    token=HF_TOKEN,
+                    tqdm_class=None,
+                    etag_timeout=30
+                )
+            
+            # Download requests if not exists or empty  
+            if not os.path.exists(self.requests_path) or not os.listdir(self.requests_path):
+                logger.info(f"Downloading requests to {self.requests_path}")
+                snapshot_download(
+                    repo_id=QUEUE_REPO,
+                    local_dir=self.requests_path,
+                    repo_type="dataset",
+                    token=HF_TOKEN,
+                    tqdm_class=None,
+                    etag_timeout=30
+                )
+                
+        except Exception as e:
+            logger.error(f"Failed to download data: {e}")
+            raise HTTPException(status_code=500, detail=f"Failed to download data: {str(e)}")
+
+    async def fetch_raw_data(self) -> List[Dict[str, Any]]:
+        """Fetch raw leaderboard data using original Icelandic processing logic"""
+        try:
+            await self._ensure_data_available()
+            
+            logger.info("Processing Icelandic leaderboard data")
+            
+            # Try to use original processing logic if available
+            try:
+                raw_data, df = get_leaderboard_df(
+                    self.results_path, 
+                    self.requests_path, 
+                    COLS, 
+                    BENCHMARK_COLS
+                )
+                
+                # Convert DataFrame to list of dictionaries
+                data = df.to_dict('records')
+                
+                logger.info(f"Processed {len(data)} Icelandic leaderboard entries")
+                return data
+                
+            except NameError:
+                # Fallback: return mock data for testing
+                logger.warning("Using mock data - original processing modules not available")
+                return self._generate_mock_data()
+                
+        except Exception as e:
+            logger.error(f"Failed to fetch Icelandic leaderboard data: {e}")
+            raise HTTPException(status_code=500, detail=str(e))
+    
+    def _generate_mock_data(self) -> List[Dict[str, Any]]:
+        """Generate mock data for testing when original modules aren't available"""
+        return [
+            {
+                "Model": "test-model/icelandic-gpt-7b",
+                "Average ⬆️": 85.5,
+                "Type": "fine-tuned",
+                "T": "🔶",
+                "Precision": "bfloat16",
+                "Architecture": "LlamaForCausalLM",
+                "Hub License": "apache-2.0",
+                "Hub ❤️": 42,
+                "#Params (B)": 7.0,
+                "Available on the hub": True,
+                "Model sha": "abc123def456",
+                "WinoGrande-IS (3-shot)": 78.5,
+                "GED": 92.3,
+                "Inflection (1-shot)": 85.1,
+                "Belebele (IS)": 80.7,
+                "ARC-Challenge-IS": 76.2,
+                "WikiQA-IS": 89.4
+            },
+            {
+                "Model": "test-model/icelandic-llama-13b",
+                "Average ⬆️": 88.2,
+                "Type": "instruction-tuned",
+                "T": "⭕",
+                "Precision": "float16",
+                "Architecture": "LlamaForCausalLM", 
+                "Hub License": "mit",
+                "Hub ❤️": 156,
+                "#Params (B)": 13.0,
+                "Available on the hub": True,
+                "Model sha": "def456abc789",
+                "WinoGrande-IS (3-shot)": 82.1,
+                "GED": 94.8,
+                "Inflection (1-shot)": 87.9,
+                "Belebele (IS)": 85.3,
+                "ARC-Challenge-IS": 79.8,
+                "WikiQA-IS": 91.2
+            }
+        ]
+
+    async def get_formatted_data(self) -> List[Dict[str, Any]]:
+        """Get formatted leaderboard data compatible with React frontend"""
+        try:
+            raw_data = await self.fetch_raw_data()
+            formatted_data = []
+            
+            for item in raw_data:
+                try:
+                    formatted_item = await self.transform_data(item)
+                    formatted_data.append(formatted_item)
+                except Exception as e:
+                    logger.error(f"Failed to format entry: {e}")
+                    continue
+                    
+            logger.info(f"Formatted {len(formatted_data)} entries for frontend")
+            return formatted_data
+            
+        except Exception as e:
+            logger.error(f"Failed to format leaderboard data: {e}")
+            raise HTTPException(status_code=500, detail=str(e))
+
+    async def transform_data(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """Transform Icelandic leaderboard data into format expected by React frontend"""
+        
+        # Create unique ID and clean model name
+        raw_model_name = data.get("Model", "Unknown")
+        
+        # Extract clean model name from HTML if present
+        if '<a target="_blank" href=' in raw_model_name:
+            # Parse HTML to extract clean model name
+            import re
+            match = re.search(r'>([^<]+)</a>', raw_model_name)
+            model_name = match.group(1) if match else raw_model_name
+        else:
+            model_name = raw_model_name
+            
+        precision = data.get("Precision", "Unknown") 
+        revision = data.get("Model sha", "Unknown")
+        unique_id = f"{model_name}_{precision}_{revision}"
+        
+        # Map Icelandic tasks to evaluations format
+        evaluations = {}
+        task_mapping = {
+            "WinoGrande-IS (3-shot)": "winogrande_is",
+            "GED": "ged", 
+            "Inflection (1-shot)": "inflection",
+            "Belebele (IS)": "belebele_is",
+            "ARC-Challenge-IS": "arc_challenge_is",
+            "WikiQA-IS": "wiki_qa_is"
+        }
+        
+        for task_display_name, task_key in task_mapping.items():
+            if task_display_name in data:
+                evaluations[task_key] = {
+                    "name": task_display_name,
+                    "value": data.get(task_display_name, 0),
+                    "normalized_score": data.get(task_display_name, 0)
+                }
+        
+        # Extract model type and clean it
+        model_type_symbol = data.get("T", "")
+        model_type_name = data.get("Type", "Unknown")
+        
+        # Map Icelandic model types to frontend format
+        type_mapping = {
+            "pretrained": "pretrained",
+            "fine-tuned": "fine-tuned", 
+            "instruction-tuned": "instruction-tuned",
+            "RL-tuned": "RL-tuned"
+        }
+        
+        clean_model_type = type_mapping.get(model_type_name, model_type_name)
+        
+        features = {
+            "is_not_available_on_hub": not data.get("Available on the hub", True),
+            "is_merged": False,  # Not tracked in Icelandic leaderboard
+            "is_moe": False,     # Not tracked in Icelandic leaderboard
+            "is_flagged": False, # Not tracked in Icelandic leaderboard
+            "is_official_provider": False  # Not tracked in Icelandic leaderboard
+        }
+        
+        metadata = {
+            "upload_date": None,  # Not available in Icelandic data
+            "submission_date": None,  # Not available in Icelandic data
+            "generation": None,  # Not available in Icelandic data
+            "base_model": None,  # Not available in Icelandic data
+            "hub_license": data.get("Hub License", ""),
+            "hub_hearts": data.get("Hub ❤️", 0),
+            "params_billions": data.get("#Params (B)", 0),
+            "co2_cost": 0  # Not tracked in Icelandic leaderboard
+        }
+        
+        transformed_data = {
+            "id": unique_id,
+            "model": {
+                "name": model_name,
+                "sha": revision,
+                "precision": precision,
+                "type": clean_model_type,
+                "weight_type": None,  # Not available in Icelandic data
+                "architecture": data.get("Architecture", "Unknown"),
+                "average_score": data.get("Average ⬆️", 0),
+                "has_chat_template": False  # Not tracked in Icelandic leaderboard
+            },
+            "evaluations": evaluations,
+            "features": features,
+            "metadata": metadata
+        }
+        
+        return transformed_data

backend/original_src/about.py

@@ -0,0 +1,75 @@
+from dataclasses import dataclass
+from enum import Enum
+
+@dataclass
+class Task:
+    benchmark: str
+    metric: str
+    col_name: str
+
+
+# Select your tasks here
+# ---------------------------------------------------
+class Tasks(Enum):
+    # task_key in the json file, metric_key in the json file, name to display in the leaderboard 
+    task0 = Task("icelandic_winogrande_stringmatch", "exact_match,get-answer", "WinoGrande-IS (3-shot)")
+    task1 = Task("icelandic_sentences_ged_stringmatch", "exact_match,get-answer", "GED")
+    task2 = Task("icelandic_inflection_all", "exact_match,get-answer", "Inflection (1-shot)")
+    task5 = Task("icelandic_belebele", "exact_match,get-answer", "Belebele (IS)")
+    task6 = Task("icelandic_arc_challenge", "exact_match,get-answer", "ARC-Challenge-IS")
+    task7 = Task("icelandic_wiki_qa", "lm_judge_score,get-answer", "WikiQA-IS")
+
+# ---------------------------------------------------
+
+
+
+# Your leaderboard name
+TITLE = """<h1 align="center" id="space-title">Icelandic LLM leaderboard</h1>"""
+
+# What does your leaderboard evaluate?
+INTRODUCTION_TEXT = """
+"""
+
+# Which evaluations are you running? how can people reproduce what you have?
+LLM_BENCHMARKS_TEXT = f"""
+## New submissions
+Do you want your model to be included on the leaderboard? Open a discussion on this repository with the details of your model and we will get back to you.
+
+## Benchmark tasks
+The Icelandic LLM leaderboard evaluates models on several tasks. All of them are set up as generation tasks, where the model's output is compared to the expected output.
+This means that models that have not been instruction fine-tuned might perform poorly on these tasks.
+
+The following tasks are evaluated:
+
+### WinoGrande-IS
+The Icelandic WinoGrande task is a human-translated and localized version of the ~1000 test set examples in the WinoGrande task in English.
+Each example consists of a sentence with a blank, and two answer choices for the blank. The task is to choose the correct answer choice using coreference resolution.
+The benchmark is designed to test the model's ability to use knowledge and common sense reasoning in Icelandic. For this benchmark, we use 3-shot evaluation.
+The Icelandic WinoGrande dataset is described in more detail in the IceBERT paper (https://aclanthology.org/2022.lrec-1.464.pdf).
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic-winogrande
+
+### GED
+This is a benchmark for binary sentence-level Icelandic grammatical error detection, adapted from the Icelandic Error Corpus (IEC) and contains 200 examples.
+Each example consists of a sentence that may contain one or more grammatical errors, and the task is to predict whether the sentence contains an error.
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic-sentences-gec
+
+### Inflection benchmark
+The inflection benchmark tests models' ability to generate inflected forms of 300 Icelandic adjective-noun pairs for all four cases, singular and plural.
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic-inflection-all-flat
+
+### Belebele (IS)
+This is the Icelandic subset (900 examples) of the Belebele benchmark, a multiple-choice reading comprehension task. The task is to answer questions about a given passage.
+- Link to dataset: https://huggingface.co/datasets/facebook/belebele
+
+### ARC-Challenge-IS
+A machine-translated version of the ARC-Challenge multiple-choice question-answering dataset. For this benchmark, we use the test set which contains 1.23k examples.
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic-arc-challenge
+
+### WikiQA-IS
+The Icelandic WikiQA dataset is a collection of 1.9k question-answer pairs from the Icelandic Wikipedia, meant to evaluate models' knowledge of Icelandic culture and history. 
+They were collected by making GPT-4o generate questions and anwswers
+given Icelandic Wikipedia articles as context. All examples were then manually verified and corrected where necessary. For evaluation, we prompt GPT-4o to 
+compare the generated answer to the original answer for semantic similarity and rate the answer on the following scale: (0, "poor"), (1, "fair"), (2, "excellent").
+- Link to dataset: https://huggingface.co/datasets/mideind/icelandic_wiki_qa
+"""
+