Anshul Jindal

LLMOps-driven fine-tuning, evaluation, and inference with NVIDIA NIM & NeMo Microservices

What if deploying custom LLMs was fully automated? Learn to build a repeatable, end-to-end pipeline from fine-tuning to inference with NVIDIA NeMo and NIM.

LLMOps-driven fine-tuning, evaluation, and inference with NVIDIA NIM & NeMo Microservices
#1about 6 minutes

Understanding the GenAI lifecycle and its operational challenges

The continuous cycle of data processing, model customization, and deployment for GenAI applications creates production complexities like a lack of standardized CI/CD and versioning.

#2about 2 minutes

Breaking down the structured stages of an LLMOps pipeline

An effective LLMOps process moves a model from an experimental proof-of-concept through evaluation, pre-production testing, and finally to a production environment.

#3about 4 minutes

Introducing the NVIDIA NeMo microservices and ecosystem tools

NVIDIA provides a suite of tools including NeMo Curator, Customizer, Evaluator, and NIM, which integrate with ecosystem components like Argo Workflows and Argo CD for a complete LLMOps solution.

#4about 4 minutes

Using NeMo Customizer and Evaluator for model adaptation

NeMo Customizer and Evaluator simplify model adaptation through API requests that trigger fine-tuning on custom datasets and benchmark the resulting model's performance.

#5about 3 minutes

Deploying and scaling models with NVIDIA NIM on Kubernetes

NVIDIA NIM packages models into optimized inference containers that can be deployed and auto-scaled on Kubernetes using the NIM operator, with support for multiple fine-tuned adapters.

#6about 4 minutes

Automating complex LLM workflows with Argo Workflows

Argo Workflows enables the creation of automated, multi-step pipelines by stitching together containerized tasks for data processing, model customization, evaluation, and deployment.

#7about 3 minutes

Implementing a GitOps approach for end-to-end LLMOps

Using Git as the single source of truth, Argo CD automates the deployment and management of all LLMOps components, including microservices and workflows, onto Kubernetes clusters.

#8about 3 minutes

Demonstrating the automated LLMOps pipeline in action

A practical demonstration shows how Argo CD manages deployed services and how a data scientist can launch a complete fine-tuning workflow through the Argo Workflows UI, with results tracked in MLflow.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Defining LLMOps and understanding its core benefits
03:02 MIN

Defining LLMOps and understanding its core benefits

From Traction to Production: Maturing your LLMOps step by step

Implementing a CI/CD pipeline for your NLP model
06:54 MIN

Implementing a CI/CD pipeline for your NLP model

Multilingual NLP pipeline up and running from scratch

The convergence of ML and DevOps in MLOps
01:33 MIN

The convergence of ML and DevOps in MLOps

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment

What MLOps is and the engineering challenges it solves
04:56 MIN

What MLOps is and the engineering challenges it solves

MLOps - What’s the deal behind it?

Introducing NVIDIA NIM for simplified LLM deployment
03:30 MIN

Introducing NVIDIA NIM for simplified LLM deployment

Efficient deployment and inference of GPU-accelerated LLMs​

Q&A: MLOps tools for building CI/CD pipelines
02:49 MIN

Q&A: MLOps tools for building CI/CD pipelines

Data Science in Retail

Understanding the role and challenges of MLOps
03:08 MIN

Understanding the role and challenges of MLOps

The Road to MLOps: How Verivox Transitioned to AWS

Understanding the core principles and lifecycle of MLOps
02:50 MIN

Understanding the core principles and lifecycle of MLOps

MLOps on Kubernetes: Exploring Argo Workflows

Featured Partners

Related Videos

See all videos
DevOps for AI: running LLMs in production with Kubernetes and KubeFlow
34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Efficient deployment and inference of GPU-accelerated LLMs​
23:01

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

Adding knowledge to open-source LLMs
25:35

Adding knowledge to open-source LLMs

Sergio Perez & Harshita Seth

From Traction to Production: Maturing your LLMOps step by step
41:45

From Traction to Production: Maturing your LLMOps step by step

Maxim Salnikov

MLOps on Kubernetes: Exploring Argo Workflows
31:02

MLOps on Kubernetes: Exploring Argo Workflows

Hauke Brammer

MLOps - What’s the deal behind it?
29:24

MLOps - What’s the deal behind it?

Nico Axtmann

Self-Hosted LLMs: From Zero to Inference
30:04

Self-Hosted LLMs: From Zero to Inference

Roberto Carratalá & Cedric Clyburn

DevOps for Machine Learning
44:40

DevOps for Machine Learning

Hauke Brammer

Related Articles

View all articles
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?
Benedikt Bischof
MLOps And AI Driven Development
Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will learn:How the concept of AI became an academic field and ...
MLOps And AI Driven Development
Luis Minvielle
What Are Large Language Models?
Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of GitHub Copilot know the feeling: this new kind of te...
What Are Large Language Models?

From learning to earning

Jobs that call for the skills explored in this talk.