Subtle Bodies - September 2019

Translation from landscape paintings to figurative watercolors using U-GAT-IT

U-GAT-IT by J. Kim et al, is a modification of a GAN (Generative Adversarial Network) which does image to image translation on shapes as well as textures within the same model by adding an additional layer from a CNN which weights the importance of each feature map.

I trained U-GAT-IT on two datasets: my abstract landscape paintings and watercolor life drawings. The resulting model was able to "extract" a figure from a landscape painting and apply a watercolor style.

Here are six samples of input paintings and output figurative watercolors:

Magma Plume
magma plume
magma plume 305001
Jungle Theorem
Jungle Theorem
power surge 305001
Loretto Morning
Loretto Morning
power surge 305001
Rose Garden
Rose Garden
power surge 305001
SF Train Station
SF Train Station
power surge 305001
Perseus Rumble
Perseus Rumble
perseus rumble 305001

 

Full results:


My previous work with image to image translation used CycleGAN to modify textures and art attributes; the results lacked form modification. My paper on this work was selected for poster presentation in the NeurIPS Machine Learning for Creativity and Design Workshop in 2018. Those works were shown in a solo show in Santa Fe December 2018 and onliine at http://www.aiartonline.com/art/holly-grimm/.