| title: Turing-Test-Prompt-Competition | |
| app_file: eval.py | |
| sdk: gradio | |
| sdk_version: 4.44.0 | |
| # Turing-Test-Prompt-Competition | |
| This project implements a chatbot using vLLM for inference and Streamlit for the user interface. | |
| ## Setup and Deployment | |
| ### vLLM Deployment | |
| To deploy vLLM: | |
| 1. Download the LLaMA model: | |
| ``` | |
| modal run download_llama.py | |
| ``` | |
| 2. Deploy the vLLM inference service: | |
| ``` | |
| modal deploy vllm_inference.py | |
| ``` | |
| ### Running the Chatbot | |
| To run the chatbot locally: | |
| 1. Start the Streamlit app: | |
| ``` | |
| streamlit run chatbot.py | |
| ``` | |
| 2. To make the chatbot accessible over the internet, use ngrok: | |
| ``` | |
| ngrok http 8501 | |
| ``` | |
| ## Project Structure | |
| - `download_llama.py`: Script to download the LLaMA model | |
| - `vllm_inference.py`: vLLM inference service | |
| - `chatbot.py`: Streamlit-based chatbot interface | |
| ## License | |
| This project is licensed under the GNU Affero General Public License v3.0. See the [LICENSE](LICENSE) file for details. | |
| ## Contributing | |
| Contributions are welcome! Please feel free to submit a Pull Request. | |
| ## Support | |
| If you encounter any problems or have any questions, please open an issue in this repository. |