A new smart drawing app that uses deep learning can convert simple sketches and doodles into photo-realistic landscape artworks in the style of famous artists.

GauGAN

The “GauGAN” app from Nvidia, which is a play on the name of French post-Impressionist painter Paul Gauguin, is described as a “smart paintbrush” that works through the interplay of two generative adversarial networks, a generator and a discriminator, powered by deep learning.

How Does It Work?

When running the app, users can draw a simple sketch/doodle outline of a landscape in an on-screen grid/segmentation map. Users can label each segment (e.g. with sea, sky, trees etc). Using its deep learning training on a million images, GauGAN can then fill in labelled areas with photo-realistic images to create detailed artworks.

In creating the pictures, the generator network of GauGAN creates images that it presents to the discriminator. The discriminator, which is that part that has been trained on the one million real images coaches the generator with pixel-by-pixel feedback on how to improve the realism of its synthetic images, thereby enabling GauGAN to arrive at a stunning final image.

Not Just Landscapes

GauGAN can also add features such as buildings roads and people, as well as style filters.  Some filters enable users to produce an original artwork in the style of a famous artist or change the lighting of an artwork e.g. from day to night.

Like A Colouring Book Picture…

Nvidia’s blog describes it as being “like a colouring book picture that describes where a tree is, where the sun is, where the sky is,” and then “the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colours, based on what it has learned about real images”.

What Does This Mean For Your Business?

The GauGAN app is a tool that can offer time and cost-saving benefits, and new creative benefits to those who need to create virtual worlds as part of their work e.g. games developers, architects, urban planners and landscape designers.  The app offers them the chance to generate better prototype ideas and make rapid changes to synthetic scenes. This could prove to be an effective and time-saving tool when it comes to taking simple brainstormed ideas to the more detailed stage quickly.

The GauGAN app may also prove to be an interesting new, experimental tool for artists and graphic designers.