Translation from landscape paintings to figurative watercolors using U-GAT-IT
“Subtle Bodies” is a series of AI-generated artworks that mimic my figurative watercolors. The figures were generated from my landscape paintings using image-to-image translation, thus finding the subtle body within the landscape.
My previous work with image-to-image translation involved modifying textures and art attributes, but lacked form modification. My paper on this work was selected for poster presentation in the NeurIPS Machine Learning for Creativity and Design Workshop in 2018. Those works were shown in a solo show in Santa Fe December 2018 and online at http://www.aiartonline.com/art/holly-grimm/.
By using a new framework, U-GAT-IT, the model was able to make geometric changes to the form, such that the original landscape was unrecognizable in the new image. U-GAT-IT by J. Kim et al (2019), is a modification of a GAN (Generative Adversarial Network) for image-to-image translation on shapes as well as textures within the same model. It adds another layer from a CNN which weights the importance of each feature map.
I trained U-GAT-IT on two datasets: my abstract landscape paintings and watercolor life drawings. The resulting model was able to “extract” a figure from a landscape painting and apply a watercolor style.
AI Disclaimer: Although these images were created by an AI, the above text was wholly human sourced and does not necessarily reflect the AI’s opinion.
Here are six samples of input paintings and output figurative watercolors:
| Magma Plume | Jungle Theorem | Loretto Morning | Rose Garden | SF Train Station | Perseus Rumble |
|---|---|---|---|---|---|
 |  |  |  |  |  |
 |  |  |  |  |  |
Full results:
