readme

.github/workflows/sync_to_hf_space.yml

@@ -0,0 +1,20 @@
+name: Sync to Hugging Face Space
+on:
+  push:
+    branches: [main]
+jobs:
+  sync-to-hub:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+          lfs: true
+      - name: Push to Hugging Face Space
+        env:
+          HF_TOKEN: ${{ secrets.HF_TOKEN }}
+        run: |
+          git config --global user.email "[email protected]"
+          git config --global user.name "Your Name"
+          git remote add space https://huggingface.co/spaces/Coool2/Final_Assignment_Template
+          git push -f https://Coool2:[email protected]/spaces/Coool2/Final_Assignment_Template main

README.md

@@ -1,15 +1,177 @@
----
-title: Template Final Assignment
-emoji: 🕵🏻‍♂️
-colorFrom: indigo
-colorTo: indigo
-sdk: gradio
-sdk_version: 5.25.2
-app_file: app.py
-pinned: false
-hf_oauth: true
-# optional, default duration is 8 hours/480 minutes. Max duration is 30 days/43200 minutes.
-hf_oauth_expiration_minutes: 480
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# 🤖 Advanced GAIA Agents Challenge Solution
+
+A comprehensive solution for the [Hugging Face Agents Course Unit 4 GAIA Challenge](https://huggingface.co/learn/agents-course/unit4/hands-on), featuring advanced multimodal AI agents with dynamic RAG capabilities, quantized models for Kaggle compatibility, and both synchronous/asynchronous execution modes.
+
+## 🌟 Features
+
+### 🧠 Dual Agent Architecture
+- **Agent 1 (LlamaIndex)**: Advanced multimodal agent with dynamic knowledge base and hybrid reranking
+- **Agent 2 (Smolagents)**: Gemini-powered agent with BM25 retrieval and observability
+
+### Features for Agent 1
+### 🎯 Multimodal Capabilities
+- **BAAI Visualized Embedding**: BGE-M3 based multimodal embeddings running on cuda:1
+- **Pixtral 12B Quantized**: FP8/4-bit quantized vision-language model for resource-constrained environments
+- **Hybrid Retrieval**: Text + visual content processing with ColPali and SentenceTransformer reranking
+
+### ⚡ Execution Modes
+- **Asynchronous Mode**: Concurrent question processing for maximum speed
+- **Kaggle Compatibility**: Optimized for resource-constrained environments
+
+### 🔍 Advanced RAG System
+- **Dynamic Knowledge Base**: Automatically updated with web search results
+- **Multimodal Parsing**: Handles text, images, PDFs, audio, and video files
+- **Smart Reranking**: Hybrid approach combining text and visual rerankers
+
+## 🏗️ Architecture
+
+```
+┌─────────────────────────────────────────────┐
+│                  APP                        │
+│            (Async/Sync Modes)               │
+└─────────────────┬───────────────────────────┘
+                  │
+         ┌────────┴────────┐
+         │                 │
+    ┌────▼────┐       ┌────▼────┐
+    │Agent 1  │       │Agent 2  │
+    │LlamaIdx │       │Smolagent│
+    └────┬────┘       └────┬────┘
+         │                 │
+    ┌────▼────┐       ┌────▼────┐
+    │Dynamic  │       │BM25 +   │
+    │RAG +    │       │Langfuse │
+    │Hybrid   │       │Observ.  │
+    │Rerank   │       │         │
+    └─────────┘       └─────────┘
+```
+
+## 🚀 Quick Start
+
+### Prerequisites
+
+### Installation
+
+1. **Clone the repository**:
+```bash
+git clone https://github.com/yourusername/gaia-agents-challenge
+cd gaia-agents-challenge
+```
+
+2. **Install FlagEmbedding with visual support**:
+```bash
+git clone https://github.com/FlagOpen/FlagEmbedding.git
+cd FlagEmbedding/research/visual_bge
+pip install -e .
+cd ../../..
+```
+
+3. **Install additional dependencies**:
+#### For Agent 1: 
+```bash
+pip install -r requirements.txt
+```
+#### For Agent 2: 
+```bash
+pip install -r requirements2.txt
+```
+
+
+4. **Set environment variables**:
+```bash
+export GOOGLE_API_KEY="your_gemini_api_key"
+export HUGGINGFACEHUB_API_TOKEN="your_hf_token"
+export LANGFUSE_PUBLIC_KEY="your_langfuse_public_key"  # Optional
+export LANGFUSE_SECRET_KEY="your_langfuse_secret_key"  # Optional
+```
+
+### Usage
+
+```bash
+# LlamaIndex Agent
+python agent.py
+
+# Smolagents Agent
+python agent2.py
+```
+
+## 📁 Project Structure
+
+```
+├── agent.py                 # LlamaIndex-based agent with dynamic RAG
+├── agent2.py               # Smolagents-based agent with observability
+├── appasync.py             # Original async Gradio interface
+├── app.py                  # Original sync Gradio interface
+├── custom_models.py        # Custom model implementations
+├── requirements.txt        # Python dependencies
+├── README.md              # This file
+```
+
+## 🧪 Testing
+
+### Run Individual Components
+```bash
+# Test BAAI embedding
+python -c "from custom_models import BaaiMultimodalEmbedding; print('BAAI OK')"
+
+# Test Pixtral quantized
+python -c "from custom_models import PixtralQuantizedLLM; print('Pixtral OK')"
+
+# Test agents
+python agent.py
+python agent2.py
+```
+
+### Run GAIA Evaluation
+```bash
+# Through the web interface
+python app.py
+
+# Or programmatically
+python -c "
+from agent2 import GAIAAgent
+agent = GAIAAgent()
+result = agent.solve_gaia_question({'Question': 'Test question', 'task_id': 'test'})
+print(result)
+"
+```
+
+## ��� Customization
+
+### Adding New Models
+1. Create a new class in `custom_models.py`
+2. Implement the required interfaces
+3. Update the agent configuration
+
+### Modifying RAG Behavior
+- Edit `DynamicQueryEngineManager` in `agent.py`
+- Adjust reranking strategies in `HybridReranker`
+- Configure search parameters in `enhanced_web_search_tool`
+
+### UI Customization
+- Modify `app_unified.py` for interface changes
+- Add new execution modes
+- Integrate additional observability tools
+
+## 🐛 Troubleshooting
+
+### Common Issues
+
+#### Model Loading Failures
+- Check internet connectivity for model downloads
+- Verify HuggingFace token permissions
+- Clear model cache: `rm -rf ~/.cache/huggingface/`
+
+#### Visual BGE Import Errors
+```bash
+# Ensure proper installation
+cd FlagEmbedding/research/visual_bge
+pip install -e .
+```
+
+## 🔗 References
+
+- [GAIA Benchmark](https://huggingface.co/datasets/gaia-benchmark/GAIA)
+- [LlamaIndex](https://github.com/run-llama/llama_index)
+- [BGE Models](https://github.com/FlagOpen/FlagEmbedding)
+- [Gradio](https://github.com/gradio-app/gradio)