<p>This is a continuation of the research here <a target="_blank" rel="ugc" href="https://civitai.com/articles/6792">https://civitai.com/articles/6792</a> using my own dataset and approach.<br /><br />I'm working on creating an updated guide to flux captioning and this is part of the work towards that will cover more nuanced approaches to training.<br /><br />But for now, for people who are interested I'll go into a brief description of the direction I'm heading and the kinds of results I'm finding.<br /><br />First of all there seems to be a misconception between which part of the dataset is most important. Many people would claim that training without any captions, or on short triggers is the most effective methodology and that the only real method to improve this is via stacking more high quality images.</p><p><br />But actually what I'm finding is that this is in itself leading moreso to overfitting, especially when used with repeats.</p><p></p><p>This idea all started a few months ago, with SDXL, when WarAnakin spoke to me about the results of some of his research in the area. He'd found through sheer trial and error across multiple attempts that one of the leading causes of overfitting was actually not having images that are too similar, but actually even having a single image with the same or close enough captions to another image in the set - even in the case where such images are vastly different - was causing much more direct and noticeable impacts on results.</p><p></p><p>Since then I've been experimenting with various different approaches with SDXL and found that despite SDXL being a BITCH to actually train, compared to SD1.5, results were significantly improved and fit much better with the existing data already present in the models.</p><p></p><p>In simplistic terms, instead of using repeats in the classical sense, instead of using caption dropout and caption shuffling in a totally randomized manner, the idea is that we use tools, like chat GPT, to expand the number of variations of captions per image. Creating copies of those images to pair with the new variant captions.</p><p></p><p>Having a subset of copies of the images without any captions, a set with just trigger words, A set captioned with tags and a set captioned with natural language descriptions.<br /><br />In this case, this particular experiment has taken that one step further.</p><p></p><p>What would happen if we restricted the dataset down to only 20 unique images and were to create ~30 different captions per image instead of 3?</p><p></p><p>And then we find ourselves here. After 20 epochs, I went through to test which ones were the most prompt adherent, and maintained the style most closely and found that a combination of Epochs 7 + 16 seemed to produce the best results.</p><p></p><p>However, there is a slight caveat here. While the results have been promising, the experiment is far from over AND I've been conducting it using CIVITAI's online trainer. Therefore, though I can train locally, for the purposes of continuing the research in a slightly more controlled manner, I've placed this resource into early access for a small one-time buzz exchange until a goal is met and the research is able to be continued.<br /><br />The next steps would involve expanding the dataset to cover more subjects, perhaps going from 20 to 40 images in total, but this time covering more people, animals and objects.</p>
Public
Flux.1 S
https://civitai.com/api/download/models/768699
true
https://civitai.com/api/download/models/768699
5B162786
B569E4EFB9
28C3216D9439
3972DBB4DDC338BEF75174D5C79C3B4941189B9C0CA369EE8BC6316325CCB89C
8F773F87
B569E4EFB970CA13A319AC48EA86209555953BBF409D10E2A75722A7F0E7077D
SafeTensor
WoolyFluxTry3-000007.safetensors
No Pickle imports
Success
2024-08-25T23:32:41.061Z
Model
Success
https://civitai.com/api/download/models/768699?type=Model&format=Diffusers
13D458EE
5096A3CC60
44CEA4A07907E742252705583C26875FCB7D0CC0A289939D730CF04EC5183E11
CD2D9D3A
5096A3CC60204CBF8D96F54E1D76B31DCEC96B61A22238F2192DC9D2CE29051D
Diffusers
loras.zip
No Pickle imports
Success
2024-08-25T23:35:41.822Z
Model
Success
https://civitai.com/api/download/models/768699?type=Training%20Data
72576CB1
06C7C2C8B4
BBCBB30C36E6B8E8934CEF5DE275FAB2CF44B7D11EC1697A0EBF71FF9667306C
CFA87617
06C7C2C8B4B299EB8E2223A1EC7B717F061C405EBB2303FE92C53D0914A923CB
Other
764830_training_data.zip
No Pickle imports
Success
2024-08-25T23:35:48.889Z
Training Data
Success
UGEWH%fy8wj2r*WFngn$aAW;adaym*oft6Sv
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/2fddcef2-308c-4550-85a6-c8ed7af8450a/width=450/26148427.jpeg
UTJHdQn#M_WV*0ayoeWCo~bHjGjZ-OxZ$fbb
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/d5029c0c-4330-4b54-9d63-5d62c6143ce4/width=450/26148420.jpeg
UyHVefocoIbIx|WCt2j[gOazniazoeofoIWC
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/edaab4af-cfe2-49b3-86f7-2d83ac66e59e/width=450/26148410.jpeg
UGLN}XoL4T%NF$R*s+ofyZM|f5bG_N%3xaIU
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/899504ce-4c0b-47af-947a-10ccf1435494/width=450/26148413.jpeg
UFHeXqWB0K%2~qkDW9aeENV@MxShITxa%1Na
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/2d9fdc68-6835-4a1f-8858-f6349de28015/width=450/26148417.jpeg
UbEgBgi_Dif,u6aeMxa|t8WXRjn$skkDxYWB
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/ca39906a-a51a-4beb-afbc-aeff8c84b447/width=450/26148408.jpeg
UWLNxrVXROtQ7jRingV@IuS5adRQ$LWVt6kB
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/a6ecf119-aede-4de9-be7b-8098094b1343/width=450/26148419.jpeg
UMH_Phjs4nj]~WayMxoLI:WCIpWBIAoL%2WB
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/cd4e2f6b-d9b8-49f2-a1c2-fad11006fdc4/width=450/26148421.jpeg
ULLX9}WU?]%2%%fjnMjbEKazMdax_3oLIVS1
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/b4bfc182-1043-4f6b-80b7-2f660987d0f2/width=450/26148423.jpeg
U9M$Cxwa01j^}]of,tw]02bbIDoe]hxZ=@oe
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/7dfb7953-e25b-4c4c-bdd8-2f627b670817/width=450/26148425.jpeg
V0.8 Alpha
2024-08-25T23:55:03.834Z
Woolen wooly felted
WoolyFlux Alpha
wool style