NVIDIA GTC 2022 : The most important AI event this Fall |

Welcome to our coverage of the most anticipated AI event of the season, NVIDIA’s GPU Technology Conference. Every six months, NVIDIA hosts the hugely popular GTC event which brings together experts in AI from many industries and academia. In addition, GTC is also the prime event where NVIDIA announces new products, technologies and software.

We covered the Spring GTC event in great detail in March, where we covered the launch of the latest H100 GPU for data center as well as the Jetson AGX Orin board for embedded applications.

As with all our coverage, while we try to bring the gist of most important sessions to you, there is nothing like being in the conference yourself and interacting with experts and industry leaders yourself. This is why we strongly recommend that you register for FREE and attend GTC yourself. You can win a RTX 3080 Ti GPU

The first day of GTC started off with multiple in-depth full day workshops on deep learning, computer vision, as well as sessions by industry experts sharing their journey into the field of AI.

Here are some of the most relevant sessions:

5 Paths to a Career in AI [SE41225]
Building custom synthetic data generation workflow [A41329]
How CUDA programming works [A41101]
Building simulation ready USD assets [A41339]
Accelerating Inference with Triton Inference Server [DLIT41270]
Honorary mentions
Summary

1. 5 Paths to a Career in AI [SE41225]

In this interactive event, five panelists from industry and academia shared the 5 unique ways by which they ended up with a career in AI. The panelists were:

Prof. Jay Lee: Founding Director of Industrial AI center at University of Cincinnati and an expert in AI for manufacturing.
Laura Leal-Taxie: Professor at Technical University Munich and Chief Scientist at Argo AI, a startup working on autonomous driving
Justin Taylor: Vice President of AI at Lockheed Martin, a US Defense contractor
Carrie Gotch: developing 3D creative tools at Adobe
Chelsea Summer: NVIDIA’s Healthcare AI startups Lead

The session was conducted in a Q&A style.

Q. What skills are necessary to break into the field?

Chelsea started off by commenting that curiosity and critical thinking, asking questions and willingness to fail are the most important qualities one needs to get into AI. It is important to note that innovation does not come without failure. Other panelists remarked that the ability to communicate across teams and connect technology to problems to identify solutions is really important as well.

Q. What role have internships played in your career?

Carrie recounted how her internship at NASA helped her identify her passion. Prof. Jay Lee remarked that internships help you understand the difference between academia and the real-world. The 3 D’s of applying AI to real world problems are: Data, Discipline and Domain. Data to learn from, Discipline to identify which models and methods work best, and Domain knowledge to understand the problem and evaluate the value created by the AI solution. Internships give you a chance to go hunt for data yourself rather than have data delivered to you as an academic benchmark dataset. In the process you also discover that if your data is low-quality, it is useful but not usable. These differences can only be learnt in a practical internship.

Q. What are some industry advancements from the time you started?

Prof. Laura remarked that conferences in her field of computer vision have grown by 10x since she started. Skepticism has been replaced by optimism. Justin remarked that safety and security domains have changed enormously since he started in this industry about 20 years ago. Computing hardware has advanced massively and advanced algorithms can be deployed to more products. Carrie gave the example of seeing heads up displays for pilots under development at NASA during her internship and now HUDs for AR and consumer applications are around the corner. Similar advances in facial recognition have demonstrated the quick application and productization of research ideas into commercial settings.

Q. What do you think about finding mentors and getting out of your comfort zone?

Chelsea gave an analogy which resonated with the other panelists. She likened a professional network to a rack of different spices in your kitchen cabinet. You can pull up any bottle at any time as needed to enhance your career. Just like one wouldn’t cook all their meals with just one spice, one should diversify their network with expertise from different fields since you never know what expertise you might need when. Other panelists remarked that your network is your net worth and it is therefore essential to expand your network purposefully.

Q. How did you get into your area of expertise?

Laura shared that she got into computer vision when she went to Boston for her Master’s thesis. She also shared how she learnt the skills learnt from cell tracking in microscope images to tracking cars and pedestrians for autonomous driving.

Jay shared that his initial foray into industrial AI was in the 1980s when the US was going through a major change in its automotive manufacturing industry. He shared how he learnt about various problems on the assembly line of cars, airplanes and even semiconductor metrology. This led to him identifying the common problems in varied sectors and he decided to switch to academia and founded a center for Industrial AI at the University of Cincinnati.

Other panelists also shared their journey at various points during the conversation and their experiences reinforced the idea that there is no ‘one correct path’ to get into AI. Instead, one should start with whatever domain they are interested in and explore how AI connects to their domain of interest. If you would like to follow their advice, OpenCV has curated some of the best courses on Deep Learning and computer vision, which you can take to accelerate your own unique journey into your domain of interest.

2. Building custom synthetic data generation workflow [A41329]

As you probably know, NVIDIA has some of the most sophisticated simulation platforms for self driving (called DRIVE Sim) and robotics (called Issaac Sim). In this session, Bhumin Pathak presented a general framework for generating synthetic data in other domains with a custom workflow. The framework that enables this is called “Omniverse Replicator”.

The process of generating synthetic data is divided into 5 steps:

Identifying personas or 3D assets required for data generation. This also includes third party assets created by yourself or other 3D artists.
The second step is using the 3D assets to generate a scene.
In the third step, replicator API can be used to randomize scene properties to procedurally generate different variations of scenes.
With this done, you can use the auto-annotation tools within the replicator to get bounding boxes, segmentation masks etc from procedurally generated data.
Finally, any AI model of your choice can be generated with the synthetic data.

More details about the Omniverse Replicator can be found here.

3. How CUDA programming works [A41101]

We have recently published a short series on CUDA programming for deep learning, which you can find here and here. In this session, Stephen Jones from NVIDIA shared the three key aspects to keep in mind when writing CUDA programs to run on GPUs. Taking the example of classifying images of flowers with a neural network, Stephen touched upon the key aspects of:

Asynchronous execution model of CUDA,
CUDA occupancy and how it can be thread limited, register limited or memory limited, and
Coalesced memory access patterns

In particular, he explained how CUDA splits the workload into threads, blocks, warps and schedules the warps on streaming multiprocessors (SMs). Since an SM can be thread, register or shared memory limited, one needs to think of the resource allocation right at the time of algorithm design. Finally, he showed that accessing memory linearly (in sequence) is much more efficient than accessing randomly. As a stark example, an A100 GPU which has the performance of 27 iPhone 13 Pro phones can appear slower than an iPhone 6S if the memory access is done randomly rather than sequentially.

If you do not understand terms like warps, SMs, blocks etc. we strongly recommend reading our series on demystifying GPU architectures. Stephen’s talk will make a lot more sense after reading those posts.

4. Building simulation ready USD assets [A41339]

Synthetic data and digital twining are keywords in industrial AI. Just like images are the basic data type in computer vision and text sentences are the basic data type in natural language processing, the most basic data type in simulation is USD or Universal Scene Description. USD is a file format to describe 3D objects, assets or entire scenes. USD was developed by Pixar and subsequently open sourced. Industries have taken up USD as their preferred format for describing objects and worlds and NVIDIA’s Omniverse uses USD files as input to understand the scene and objects to be simulated. In this talk, engineers from NVIDIA shared some of the best practices for creating simulation ready 3D assets for Omniverse as well as some future work they plan to do.

The crux of their recommendation is ‘modularity’. They recommend structuring your world and objects into hierarchical and modular files with cross references, rather than one large USD file. This enables various sub-parts of the scene to be individually modified and updated. The same recommendation goes for material descriptions in MDL. However, from their talk it appears that this is early days for USD in simulation and NVIDIA is trying to create a new standard called SimReady to denote USD files which can be used for meaningful dynamic simulations and will also encompass the Universal Robot Description Format (URDF), which is widely used in Robotics.

5. Accelerating Inference with Triton Inference Server [DLIT41270]

This was a hands-on free training lab for attendees of GTC, where Adam from NVIDIA provided a tutorial on the Triton inference server in the context of MLOps. As the need for inference grows and data distribution shifts over the life cycle of an application, Triton helps deploy your PyTorch, TensorFlow or ONNX model with TensorRT acceleration and packages your application into a neat docker container which especially helps with cloud deployments.

An explanation of the Prometheus tool for measuring GPU usage was provided as well as tips on asynchronous execution requests were shared which would help maximize throughput (at the expense of some latency).

6. Honorary mentions

There are some sessions which are very similar to the sessions we covered in the spring GTC. While we will not cover them here again, we would very much recommend you watch these sessions as they are excellent resources for improving your understanding of AI.

AI models made simple with NVIDIA TAO toolkit [A41172]
Introduction to “Learning Deep Learning” [DLIT41269]
Introduction to AI in Data Center [DLIT41304]
Introduction to Autonomous Vehicles [A41086]

7. Summary

In this first day post of GTC 2022 Fall edition, we have shared how experts and industry leaders in industrial AI got started and traced their rewarding career journeys to the top. We have also covered the basics of CUDA programming, ways to generate custom synthetic data for training machine learning models, the current and future prospects of USD format for simulation in robotics and digital twinning, as well as a unified approach to deploying trained AI models for inference across various types of computing nodes with Triton inference server.

GTC is just getting started. The most important of this conference will be the keynote by NVIDIA CEO Jensen Huang on Day 2, a few hours after this summary is published. We are excited for the really big announcements and cool things that Jensen will share in his keynote. To make sure you don’t miss out on those announcements, be sure to register for GTC if you haven’t already done so.