YAML Metadata Warning: The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

๐Ÿ‡บ๐Ÿ‡ฆ uk-gender-swapper-aya-101

This model is a PEFT-fine-tuned version of the aya-101 large language model, adapted for Ukrainian gender swapping. It is designed to generate gender-swapped versions of input sentences, making it useful for tasks involving gender bias mitigation, inclusive language generation, or synthetic dataset creation.

Model Type: Aya-101 (12.9B) + PEFT (LoRA)
Language: Ukrainian ๐Ÿ‡บ๐Ÿ‡ฆ
Task: Gender Swapping (Text-to-Text Generation)


๐Ÿ’ก Use Cases

  • Bias analysis and mitigation in NLP models
  • Synthetic data generation for gender-balanced corpora
  • Exploration of linguistic gender structures in Ukrainian
  • Educational and research purposes related to gender and language

๐Ÿง  Model Details

  • Base Model: CohereForAI/aya-101
  • Fine-tuning Method: Parameter-Efficient Fine-Tuning (PEFT) with LoRA
  • Training Data: A custom parallel dataset of Ukrainian sentences with gender-swapped equivalents (e.g., male-to-female and female-to-male subject transformations)
  • Tokenizer: Same as Aya-101 (LLaMA tokenizer)

๐Ÿ”ง How to Use

You can use this model with the peft library and transformers. Hereโ€™s an example using the merged adapter:

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel, PeftConfig

adapter_repo_id = "linndfors/uk-sent-gender-swapper_aya-101"

config = PeftConfig.from_pretrained(adapter_repo_id)
base_model_id = config.base_model_name_or_path
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

base_model = AutoModelForSeq2SeqLM.from_pretrained(base_model_id)

base_model.resize_token_embeddings(len(tokenizer))
inference_model = PeftModel.from_pretrained(base_model, adapter_repo_id)

prefix = "ะŸะตั€ะตั„ั€ะฐะทัƒะน ั†ะต ั€ะตั‡ะตะฝะฝั, ะทะผั–ะฝะธะฒัˆะธ ะณะตะฝะดะตั€ะฝั– ััƒั‚ะฝะพัั‚ั– ะฝะฐ ะฟั€ะพั‚ะธะปะตะถะฝั– (ั‡ะพะปะพะฒั–ั‡ะธะน <-> ะถั–ะฝะพั‡ะธะน): "

def generate_swapped_response(input_text, inference_model, tokenizer): 
    inputs = tokenizer(prefix + input_text, return_tensors="pt")

    with torch.no_grad():
        outputs = inference_model.generate(**inputs, max_new_tokens=100, temperature=0.1)

    generated = tokenizer.decode(outputs[0], skip_special_tokens=True)

    return generated

input = "ะ’ั–ะฝ ะฟั€ะฐั†ัŽั” ะฒะธะบะปะฐะดะฐั‡ะตะผ ะฒ ัƒะฝั–ะฒะตั€ัะธั‚ะตั‚ั–."
gender-swapped_output = generate_swapped_response(input, inference_model, tokenizer)

๐Ÿ“Š Example

Input:
ะ’ั–ะฝ ะฟั€ะฐั†ัŽั” ะฒะธะบะปะฐะดะฐั‡ะตะผ ะฒ ัƒะฝั–ะฒะตั€ัะธั‚ะตั‚ั–.
Output:
ะ’ะพะฝะฐ ะฟั€ะฐั†ัŽั” ะฒะธะบะปะฐะดะฐั‡ะบะพัŽ ะฒ ัƒะฝั–ะฒะตั€ัะธั‚ะตั‚ั–.


๐Ÿ—๏ธ Training Details

  • Adapter Type: LoRA (rank=32)
  • PEFT Library: peft
  • Hardware: Trained on Colab Pro+ (A100)
  • Epochs: 15
  • Batch Size: 4
  • Optimizer: adamw_torch
  • Learning Rate: 5e-5

๐Ÿ“ Files and Artifacts

  • adapter_config.json โ€“ PEFT configuration
  • adapter_model.bin โ€“ Fine-tuned LoRA weights
  • README.md โ€“ Description
  • tokenizer_config.json, tokenizer.model, etc. โ€“ Tokenizer files from the base model

โš ๏ธ Limitations & Warnings

  • May not generalize well to complex or idiomatic language.
  • Designed specifically for binary gender swaps in Ukrainian.
  • Not suitable for tasks requiring preservation of named entities in all cases.

๐Ÿ“œ Citation

If you use this model in your work, please cite the original Aya-101 paper and consider crediting this repository:

@misc{linndfors2024ukgenderswapper,
  title={uk-gender-swapper-aya-101},
  author={Olha Nahurna},
  year={2024},
  howpublished={\url{https://huggingface.co/linndfors/uk-gender-swapper-aya-101}}
}

๐Ÿ™ค Questions or Feedback?

Feel free to open an issue or reach out via the Hugging Face discussion tab.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for linndfors/uk-gender-swapper-aya-101

Base model

CohereLabs/aya-101
Finetuned
(5)
this model