Electronicdesign 28075 Nvidia Rtx Promo

With GauGAN, AI Painting Will Never Be the Same

Aug. 1, 2019
NVIDIA’s RTX platforms can do more than ray tracing when they’re implemented with the open-source GauGAN technology.

The RTX family developed by NVIDIA is designed to deliver high-performance graphics, including ray-tracing support. The technology is showing up in laptops like the Lenovo Legion Y740 Laptop Studio Edition in addition to PCs and add-in cards that were on display at this year’s SIGGRAPH.

Another NVIDIA highlight at SIGGRAPH was the number of applications that now take advantage of the RTX technology. They use the NVIDIA Studio Driver, which provides optimizations for popular creative applications including OTOY Octane Render, Blender, Autodesk Arnold, Maxon Cinema 4D, Substance Painter by Adobe, and Magix VEGAS Pro. The driver adds 30-bit color support for OpenGL applications such as Adobe Photoshop and Premiere. It allows artists to work with high-dynamic-range (HDR) photos and videos with full accuracy without encountering the banding typical with 24-bit color systems.

One of the other pieces of software receiving the spotlight was NVIDIA’s GauGAN (Fig. 1). It’s an open-source drawing project that lets anyone create photorealistic images or videos. The split-screen interface lets a person draw blobs on the left side while the right shows the photorealistic image.

1. GauGAN generates a photorealistic image (right) from a rough drawing (left).

GauGAN uses generated-adversarial-networks (GAN) technology to perform this magic. It leverages deep-neural-network (DNN) machine-learning (ML) models to let someone create everything from a seascape to a mountain forest. Essentially, it works like an image-editing program where a brush can take many forms, including one that generates patterns. The difference is that GauGAN’s pallet consists of photorealistic items that are merged with the rest of the image in a natural fashion.   

Hollywood artists are taking to GauGAN like bees to honey. It allows them to create backgrounds quickly to which additional artifacts are added (Fig. 2). Thus, they’re able to concentrate on content; the software running on NVIDIA GPUs can handle the addition of elements and merge them with the existing image. The system isn’t designed to build “deep fakes,” since it’s building up the background from scratch.

2. VFX artists love GauGAN because it can quickly create backgrounds, enabling the rapid addition of artifacts.

The system was trained with real-world images. It actually uses model ML models. In particular, there’s a generator and discriminator. The first creates images while the second “coaches” the generator to develop a photorealistic image.

NVIDIA doesn’t plan on making GauGAN into a product, but the technology is likely to show up in image- and video-editing tools in the future. The system needs the type of performance that the RTX platforms provide so that editing can be done in real time. The company’s interactive SIGGRAPH demo runs on a TITAN RTX GPU.

Sponsored Recommendations

Near- and Far-Field Measurements

April 16, 2024
In this comprehensive application note, we delve into the methods of measuring the transmission (or reception) pattern, a key determinant of antenna gain, using a vector network...

DigiKey Factory Tomorrow Season 3: Sustainable Manufacturing

April 16, 2024
Industry 4.0 is helping manufacturers develop and integrate technologies such as AI, edge computing and connectivity for the factories of tomorrow. Learn more at DigiKey today...

Connectivity – The Backbone of Sustainable Automation

April 16, 2024
Advanced interfaces for signals, data, and electrical power are essential. They help save resources and costs when networking production equipment.

Empowered by Cutting-Edge Automation Technology: The Sustainable Journey

April 16, 2024
Advanced automation is key to efficient production and is a powerful tool for optimizing infrastructure and processes in terms of sustainability.


To join the conversation, and become an exclusive member of Electronic Design, create an account today!