Body: <p>I'm trying to generate some synthetic images of humans, that are &quot;good enough&quot; to be used to fine-tune a stable diffusion model, but i'm not sure if this is possible.</p>
  <p>I have experimented with some methods, but none gave satisfactory results. I tried using stable diffusion itself and some hacks which can be found <a href="https://mythicalai.substack.com/p/how-to-create-consistent-characters" rel="nofollow noreferrer">here</a> , but the outputs had minor differences that might degrade the quality of the model. I also tried using GANs, specifically InterFaceGAN, to modify the pose and smile of the characters, but the images were 95% similar and too low quality and for training.</p>
  <p>I wonder if there is a better method to do so. Is it even feasible?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T15:34:32.383Z
Comments.Text: Can you explain the scope of what you want to fine tune and generate? Some synthetic image approaches may lack certain features, but those may not be necessary for your project. E.g. what variations, if any, do you need for backgrounds, lighting, framing, poses/activity, character diversity, clothing and accessories? Is there any rendering styles you need (photographic, illustration, cartoon?) Are any of those fixed, but would be specific to your project (e.g. they must all be wearing historical costumes)?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T15:47:04.540Z
Comments.Text: Are you trying to create a LoRA (or similar) for some recurring characters? So one constraint is "all the same character" for any specific set?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T16:02:58.357Z
Comments.Text: Hello @NeilSlater Im trying to fine-tune an sd model on human images, except that I don't want to use real training data for legal reasons, im looking for synthetic images that are neutral (neutral facial expressions, background, and clothing and wearing a simple bikini would be my perfect training data, and of course to look as realistic as possible. I actually got some consistent results using this [model](https://replicate.com/lucataco/realistic-vision-v5.1) using a very detailed prompt, but I am afraid there might be subtle differences that cannot detected by eye but sd wil detect them.
Comments.UserDisplayName: interesting
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T16:08:10.597Z
Comments.Text: You still aren't saying what the scope is of your fine tuning? E.g. Single character, or preferred image style, or better general human anatomy? These would each require different training data sets, that might suit different approaches to generation. I'm getting the sense that you want to have fine-tuned images for some named characters? E.g. you want to describe "Bob", have enough training examples of "Bob" to fine-tune in Stable Diffusion and then be able to generate more images of "Bob" doing stuff?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T16:42:27.323Z
Comments.Text: The perfect flow would be generating a random real human with random physical appearance if the random human is approved by some hypothetical user, i want to be able to generate a set of consistent and similar images that can be used to train SD on that random human.
Comments.UserDisplayName: interesting
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-29T20:55:23.900Z
Comments.Text: So you want a series of random characters. But for training purposes, multiple images of each character?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-08-30T13:08:41.200Z
Comments.Text: yes @NeilSlater exactly to give a follow up in my progress, i found this experiments that does so using textual inversion https://github.com/BelieveDiffusion/tutorials/blob/main/consistent_character_embedding/README.md but it includes a little of manual selection however im trying to make all fully automated
Comments.UserDisplayName: interesting
Tags: <generative-adversarial-networks><generative-model><image-generation><diffusion-models>
Title: How to generate quality synthetic images of human subjects to be used for training stable diffusion
UserDisplayName: interesting
ThreadText: # How to generate quality synthetic images of human subjects to be used for training stable diffusion
  
  (Asked by: username_0 on 2023-08-27)
  
  I'm trying to generate some synthetic images of humans, that are "good enough" to be used to fine-tune a stable diffusion model, but i'm not sure if this is possible.
  
  I have experimented with some methods, but none gave satisfactory results. I tried using stable diffusion itself and some hacks which can be found [here](https://mythicalai.substack.com/p/how-to-create-consistent-characters) , but the outputs had minor differences that might degrade the quality of the model. I also tried using GANs, specifically InterFaceGAN, to modify the pose and smile of the characters, but the images were 95% similar and too low quality and for training.
  
  I wonder if there is a better method to do so. Is it even feasible?
  
  **Tags:** generative-adversarial-networks, generative-model, image-generation, diffusion-models
  
  > **Comments:**
  > **username_1**: Can you explain the scope of what you want to fine tune and generate? Some synthetic image approaches may lack certain features, but those may not be necessary for your project. E.g. what variations, if any, do you need for backgrounds, lighting, framing, poses/activity, character diversity, clothing and accessories? Is there any rendering styles you need (photographic, illustration, cartoon?) Are any of those fixed, but would be specific to your project (e.g. they must all be wearing historical costumes)?  
  > **username_1**: Are you trying to create a LoRA (or similar) for some recurring characters? So one constraint is "all the same character" for any specific set?  
  > **username_0**: Hello @username_1 Im trying to fine-tune an sd model on human images, except that I don't want to use real training data for legal reasons, im looking for synthetic images that are neutral (neutral facial expressions, background, and clothing and wearing a simple bikini would be my perfect training data, and of course to look as realistic as possible. I actually got some consistent results using this [model](https://replicate.com/lucataco/realistic-vision-v5.1) using a very detailed prompt, but I am afraid there might be subtle differences that cannot detected by eye but sd wil detect them.  
  > **username_1**: You still aren't saying what the scope is of your fine tuning? E.g. Single character, or preferred image style, or better general human anatomy? These would each require different training data sets, that might suit different approaches to generation. I'm getting the sense that you want to have fine-tuned images for some named characters? E.g. you want to describe "Bob", have enough training examples of "Bob" to fine-tune in Stable Diffusion and then be able to generate more images of "Bob" doing stuff?  
  > **username_0**: The perfect flow would be generating a random real human with random physical appearance if the random human is approved by some hypothetical user, i want to be able to generate a set of consistent and similar images that can be used to train SD on that random human.  
  > **username_1**: So you want a series of random characters. But for training purposes, multiple images of each character?  
  > **username_0**: yes @username_1 exactly to give a follow up in my progress, i found this experiments that does so using textual inversion https://github.com/BelieveDiffusion/tutorials/blob/main/consistent_character_embedding/README.md but it includes a little of manual selection however im trying to make all fully automated  
  
  ---
  
  ## 0 answers
  
  