An overview of the topics to be covered, from the progress of generative AI to the compute requirements for training and inference.
Generative AI creates novel content, moving beyond prediction to unlock new use cases in coding, content creation, and customer experience.
NVIDIA Inference Microservices (NIMs) and blueprints provide pre-packaged, optimized containers to quickly deploy models for tasks like retrieval-augmented generation (RAG).
Building a production-ready model involves a multi-stage process including data curation, distributed training, alignment, optimized inference, and implementing guardrails.
Training massive models requires splitting them across thousands of GPUs using tensor, pipeline, and data parallelism to manage compute and communication.
Training large models like Llama requires millions of GPU hours, while inference for a single large model can demand a full multi-GPU server.
Effective multi-node training depends on high-speed interconnects like NVLink and network architectures designed to minimize communication latency between GPUs.
NVIDIA's DGX Cloud Lepton is a marketplace connecting developers to a global network of cloud partners for scalable, on-demand GPU compute.
22:07Ankit Patel
32:19Kevin Klues
23:01Adolf Hohl
31:25Patrick Koss
35:16Ankit Patel
24:26Thomas Schmidt
28:38Tomislav Tipurić
32:27Christian Liebel
.png?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
BWI GmbH
Conrad Electronic SE