NVIDIA_RTX_Promo.png

With GauGAN, AI Painting Will Never Be the Same

Aug. 1, 2019
NVIDIA’s RTX platforms can do more than ray tracing when they’re implemented with the open-source GauGAN technology.

The RTX family developed by NVIDIA is designed to deliver high-performance graphics, including ray-tracing support. The technology is showing up in laptops like the Lenovo Legion Y740 Laptop Studio Edition in addition to PCs and add-in cards that were on display at this year’s SIGGRAPH.

Another NVIDIA highlight at SIGGRAPH was the number of applications that now take advantage of the RTX technology. They use the NVIDIA Studio Driver, which provides optimizations for popular creative applications including OTOY Octane Render, Blender, Autodesk Arnold, Maxon Cinema 4D, Substance Painter by Adobe, and Magix VEGAS Pro. The driver adds 30-bit color support for OpenGL applications such as Adobe Photoshop and Premiere. It allows artists to work with high-dynamic-range (HDR) photos and videos with full accuracy without encountering the banding typical with 24-bit color systems.

One of the other pieces of software receiving the spotlight was NVIDIA’s GauGAN (Fig. 1). It’s an open-source drawing project that lets anyone create photorealistic images or videos. The split-screen interface lets a person draw blobs on the left side while the right shows the photorealistic image.

1. GauGAN generates a photorealistic image (right) from a rough drawing (left).

GauGAN uses generated-adversarial-networks (GAN) technology to perform this magic. It leverages deep-neural-network (DNN) machine-learning (ML) models to let someone create everything from a seascape to a mountain forest. Essentially, it works like an image-editing program where a brush can take many forms, including one that generates patterns. The difference is that GauGAN’s pallet consists of photorealistic items that are merged with the rest of the image in a natural fashion.   

Hollywood artists are taking to GauGAN like bees to honey. It allows them to create backgrounds quickly to which additional artifacts are added (Fig. 2). Thus, they’re able to concentrate on content; the software running on NVIDIA GPUs can handle the addition of elements and merge them with the existing image. The system isn’t designed to build “deep fakes,” since it’s building up the background from scratch.

2. VFX artists love GauGAN because it can quickly create backgrounds, enabling the rapid addition of artifacts.

The system was trained with real-world images. It actually uses model ML models. In particular, there’s a generator and discriminator. The first creates images while the second “coaches” the generator to develop a photorealistic image.

NVIDIA doesn’t plan on making GauGAN into a product, but the technology is likely to show up in image- and video-editing tools in the future. The system needs the type of performance that the RTX platforms provide so that editing can be done in real time. The company’s interactive SIGGRAPH demo runs on a TITAN RTX GPU.

About the Author

William G. Wong | Senior Content Director - Electronic Design and Microwaves & RF

I am Editor of Electronic Design focusing on embedded, software, and systems. As Senior Content Director, I also manage Microwaves & RF and I work with a great team of editors to provide engineers, programmers, developers and technical managers with interesting and useful articles and videos on a regular basis. Check out our free newsletters to see the latest content.

You can send press releases for new products for possible coverage on the website. I am also interested in receiving contributed articles for publishing on our website. Use our template and send to me along with a signed release form. 

Check out my blog, AltEmbedded on Electronic Design, as well as his latest articles on this site that are listed below. 

You can visit my social media via these links:

I earned a Bachelor of Electrical Engineering at the Georgia Institute of Technology and a Masters in Computer Science from Rutgers University. I still do a bit of programming using everything from C and C++ to Rust and Ada/SPARK. I do a bit of PHP programming for Drupal websites. I have posted a few Drupal modules.  

I still get a hand on software and electronic hardware. Some of this can be found on our Kit Close-Up video series. You can also see me on many of our TechXchange Talk videos. I am interested in a range of projects from robotics to artificial intelligence. 

Sponsored Recommendations

Comments

To join the conversation, and become an exclusive member of Electronic Design, create an account today!