alkzar90/ppaine-landscape

Dreambooth Hackaton 23': How can we use a text-to-image generative model to explore the cinematographic appeal of Torres del Paine 🇨🇱?

Torres del Paine National Park is a national park encompassing mountains, glaciers, lakes, and rivers in southern Chilean Patagonia. It is also part of the End of the World Route, a tourist scenic route. Wikipedia

Snowscat's' Photo, Unsplash

• Reddit post #1: Dreambooth Hackaton: How can we use a text-to-image model to explore the cinematographic appeal of Torres del Paine 🇨🇱?
• Reddit post #2: Animal statues at Torres del Paine, ppain landscape model; Dreambooth + StableDiffusion

Description

DreamBooth model for the ppaine concept trained by alkzar90 on the alkzar90/torres-del-paine dataset.

This is a Stable Diffusion model fine-tuned on the ppaine concept with DreamBooth. It can be used by modifying the instance_prompt: a photo of ppaine landscape

This model was created as part of the DreamBooth Hackathon 🔥. Visit the organisation page for instructions on how to take part!

This is a Stable Diffusion model fine-tuned on landscape images for the landscape theme.

Cinematographics rendering & Object/Artifacts insertion

Figure 1: Cinematographics renderings and object/artifacts insertions in the Chilean Torres del Paine national park. Text prompts for generated images up-to-down rows and left-to-right; (i) "The ppaine landscape in the middle earth, cinematic light, lord of the ring style, epic", (ii) "The ppaine landscape in the middle earth, a visible dragon skeleton bones, cinematic light, lord of the ring style, epic", (iii) "A long branches forest in the ppaine landscape, mountain peaks at the background, cinematic light, realistic, lord of the ring style, epic", (iv) "A futuristic jeep riding in ppaine landscape, cinematic light, technology, (v) "A futuristic tensor airship flying over the ppaine landscape at night, NIKON-Z-FX", (vi) "A huge tensor bridge in the ppaine landscape, cinematic light, majestic, architecture".

Object/Artifact insertions

Figure 2-a: Animal statues in the Chilean Torres del Paine national park. Text prompts for the image: "A photo of an ancient stone condor statue in the ppaine landscape, michaelangelo, majestic, NIKON-Z-FX, 28mm",

Figure 2-b: Animal statues in the Chilean Torres del Paine national park. Text prompts for the image: "A photo of a marble huemul statue in the ppaine landscape, majestic, michaelangelo, NIKON-Z-FX, 28mm",

Figure 2-c: Structures in the Chilean Torres del Paine national park. Text prompts for the image: "A panoramic photo of ppaine landscape with an Eiffel in the middle, NIKON-Z-FX, realistic, cinematic light",

Director's eye view

What does the director's cut concept mean? The definition by the Merriam-Webster dictionary is: "a version of a motion picture that is edited according to the director's wishes and that usually includes scenes cut from the version created for general distribution".

Figure 3: Illustration of the director cuts of the Chilean Torres del Paine national park, in Wes Anderson's eyes. Text prompts for generated images left-to-right; (i) "The ppaine landscape, Wes Anderson style, cinematic light", (ii) "The ppaine landscape with a small house in the middle, Wes Anderson style, fish eye", (iii) "The ppaine landscape with a small house in the middle, Wes Anderson style, fish eye".

Artistic Style Transfer

One way to monitor the fine-tuning process is to look at the model capabilities for transferring well-known artistic styles into the Torres del Paine landscape.

Figure 4: Artistic renderings of the Chilean Torres del Paine national park in the style of famous painters. Text prompts for generated images up-to-down rows and left-to-right; (i) "A painting of the ppaine landscape, Vincent Van Gogh style", (ii) "A painting of the ppaine landscape, Michelangelo style", (iii) "A painting of the ppaine landscape, Botero style", (iv) "A painting of the ppaine landscape, Pierre-Auguste Renoir style", (v) "A painting of the ppaine landscape, Leonardo Da Vinci style", (vi) "A painting of the ppaine landscape, Rembrandt style".

Usage

from diffusers import StableDiffusionPipeline

pipeline = StableDiffusionPipeline.from_pretrained('alkzar90/ppaine-landscape')
image = pipeline().images[0]
image

References

• DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (Ruiz et al. 2022)
• High-Resolution Image Synthesis with Latent Diffusion Models (Rombach et al., 2022 )
• Training Stable Diffusion with Dreambooth using 🧨 Diffusers (Post)
• Hugging Face DreamBooth Hackathon

Thanks to John Whitaker and Lewis Tunstall

Thanks to John Whitaker and Lewis Tunstall for writing out and describing the initial hackathon parameters at https://huggingface.co/dreambooth-hackathon.