File size: 1,179 Bytes
469e264 45a9357 469e264 45a9357 469e264 45a9357 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
---
title: Turing-Test-Prompt-Competition
app_file: eval.py
sdk: gradio
sdk_version: 4.44.0
---
# Turing-Test-Prompt-Competition
This project implements a chatbot using vLLM for inference and Streamlit for the user interface.
## Setup and Deployment
### vLLM Deployment
To deploy vLLM:
1. Download the LLaMA model:
```
modal run download_llama.py
```
2. Deploy the vLLM inference service:
```
modal deploy vllm_inference.py
```
### Running the Chatbot
To run the chatbot locally:
1. Start the Streamlit app:
```
streamlit run chatbot.py
```
2. To make the chatbot accessible over the internet, use ngrok:
```
ngrok http 8501
```
## Project Structure
- `download_llama.py`: Script to download the LLaMA model
- `vllm_inference.py`: vLLM inference service
- `chatbot.py`: Streamlit-based chatbot interface
## License
This project is licensed under the GNU Affero General Public License v3.0. See the [LICENSE](LICENSE) file for details.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## Support
If you encounter any problems or have any questions, please open an issue in this repository. |