<p></p><p>Dataset_Maker.ipynb</p><p>Dataset_Maker.ipynb_</p><p>📊 Dataset Maker by Hollowstrawberry</p><p>This is based on the work of Kohya-ss and Linaqruf. Thank you!</p><p></p><p>⭕ Disclaimer</p><p>The purpose of this document is to research bleeding-edge technologies in the field of machine learning inference.</p><p>Please read and follow the Google Colab guidelines and its Terms of Service.</p><p></p><p>🚩 Start Here</p><p></p><p>[6]</p><p>24 s</p><p>1️⃣ Setup</p><p>This cell will load some requirements and create the necessary folders in your Google Drive.</p><p></p><p>Your project name can't contain spaces but it can contain a single / to make a subfolder in your dataset.</p><p></p><p>project_name:</p><p>Timeskip_Asta</p><p>The folder structure doesn't matter and is purely for comfort. Make sure to always pick the same one. I like organizing by project.</p><p></p><p>folder_structure:</p><p></p><p>Organize by category (MyDrive/lora_training/datasets/project_name)</p><p>📂 Connecting to Google Drive...</p><p>Mounted at /content/drive</p><p>✅ Project Timeskip_Asta is ready!</p><p>GitHub	🇬🇧 English	🇪🇸 Spanish</p><p>🏠 Homepage	GitHub		</p><p>📊 Dataset Maker	GitHub	Open in Colab	Abrir en Colab</p><p>⭐ Lora Trainer	GitHub	Open in Colab	Abrir en Colab</p><p>🌟 XL Lora Trainer	GitHub	Open in Colab	</p><p></p><p>[ ]</p><p>2️⃣ Scrape images from Gelbooru</p><p>We will grab images from the popular anime gallery Gelbooru. Images are sorted by tags, including poses, scenes, character traits, character names, artists, etc.</p><p></p><p>If you instead want to use your own images, upload them to your Google Drive's Loras/project_name/dataset folder.</p><p>If you instead want to download screencaps of anime episodes, try this other colab by another person. It's more complicated though.</p><p>Up to 1000 images may be downloaded by this step in just one minute. Don't abuse it.</p><p></p><p>Your target tags should include the relevant tags for your character/concept/artstyle, and exclude undesired tags (for example, explicit images may affect learning). Separate words with underscores, separate tags with spaces, and use - to exclude a tag. You can also include a minimum score: score:&gt;10</p><p></p><p>tags:</p><p>1girl -sex -bdsm -loli -greyscale -monochrome hatsune_miku</p><p></p><p>[ ]</p><p>3️⃣ Curate your images</p><p>We will find duplicate images with the FiftyOne AI, and mark them with delete.</p><p></p><p>Then, an interactive area will appear below this cell that lets you visualize all your images and manually mark with delete to the ones you don't like.</p><p></p><p>If the interactive area appears blank for over a minute, try enabling cookies and removing tracking protection for the Google Colab website, as they may break it. Regardless, you can save your changes by sending Enter in the input box above the interactive area.</p><p></p><p>This is how similar 2 images must be to be marked for deletion. I recommend 0.97 to 0.99:</p><p></p><p>similarity_threshold:</p><p>0.985</p><p></p><p>[7]</p><p>2 min</p><p>4️⃣ Tag your images</p><p>We will be using AI to automatically tag your images, specifically Waifu Diffusion in the case of anime and BLIP in the case of photos. Giving tags/captions to your images allows for much better training. This process should take a couple minutes.</p><p></p><p>method:</p><p></p><p>Anime tags</p><p>Anime: The threshold is the minimum level of confidence the tagger must have in order to include a tag. Lower threshold = More tags. Recommended 0.35 to 0.5</p><p></p><p>tag_threshold:</p><p></p><p> </p><p>0.35</p><p>blacklist_tags:</p><p>bangs, breasts, multicolored hair, two-tone hair, gradient hair, virtual youtuber, parody, style parody, official alternate costume, official alternate hairstyle, official alternate hair length, alternate costume, alternate hairstyle, alternate hair length, alternate hair color</p><p>Photos: The minimum and maximum length of tokens/words in each caption.</p><p></p><p>caption_min:</p><p>10</p><p>caption_max:</p><p>75</p><p>📊 Tagging complete. Here are the top 50 tags in your dataset:</p><p>1boy, muscular, pectorals, abs, headband, spiked hair, muscular male, belt, green eyes , grey hair, bara, large pectorals, nipples, white hair, thighs, thick thighs, brown belt, penis, erection, testicles, white background, uncensored, topless male, torn clothes, black headband, holding weapon, holding sword, boots, stomach, nude, scar</p>
Public
SD 1.5
https://civitai.com/api/download/models/806077
true
https://civitai.com/api/download/models/806077
99C8DA58
7309FB575B
73A8BA4E574A
A227503DDDFE26050E2818133C74C8A09972DB20B4886C78CAFAA6B3C56D5B5B
0B8E7ED6
7309FB575BC866BF819E1F4C5335A90A95D007099F7E042721D528022C33BA92
SafeTensor
Timeskip_Asta-10.safetensors
No Pickle imports
Success
2024-09-04T17:51:36.523Z
Model
Success
UfC[FWxuK-R-NOjZR5bII^NHxCs.xDxZaxoJ
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/5e06e5e9-af51-404f-86a1-fa0d71ce2947/width=450/27754560.jpeg
v1.0
2024-09-04T17:50:44.784Z
Muscle
Timeskip Asta
anime character muscular male man muscle nsfw