TheDrummer

/

Cydonia-24B-v4

like 34

Dear @TheDrummer ,

I am writing to you today to inquire about your services for a custom LLM fine-tuning project. I've been following your work on fine-tuning large language models with great interest. Your expertise in this domain is highly regarded, and I have a potential project for which I believe your skills would be an ideal fit.

I am looking to fine-tune a model in the 20-40 billion parameter range on approximately 50-100MB of proprietary text data. I would strongly prefer to utilize a base model that is known to have been trained on a broad and relatively unfiltered dataset without any significant safety alignment, with a specific requirement for maximizing the intelligence-to-(GPTslop+censorship) ratio. I am particularly interested in working with base models rather than instruction-tuned variants, as I need the foundational capabilities without additional alignment layers.

The intended application is for a single-turn specialist model with a maximum context length of 4,000 tokens. The model will be deployed for specialized tasks that require both technical accuracy and nuanced understanding within my particular domain. The project would require both standard supervised fine-tuning and modern preference optimization techniques such as DPO or whatever methodology you currently consider most effective for balancing accepted and rejected response patterns. A critical requirement for this project is avoiding catastrophic forgetting, as maintaining the model's existing knowledge base while adapting it to my specific domain is essential.

Could you please provide me with a cost estimate for this work, along with your expected timeline for completion? I would also appreciate any insights you might have regarding the optimal base model selection for achieving these objectives. I am prepared to discuss the project requirements in greater detail and can provide additional specifications as needed. Please let me know your availability for a more detailed consultation.

I look forward to hearing from you and potentially working together on this project.

Best regards,

Charles McSneed