When is retrieval-augmented generation not enough? Learn the multi-stage process for deeply embedding new knowledge into an open-source LLM.
#1about 4 minutes
Understanding the LLM training pipeline and knowledge gaps
LLMs are trained through pre-training and alignment, but require new knowledge to stay current, adapt to specific domains, and acquire new skills.
#2about 5 minutes
Adding domain knowledge with continued pre-training
Continued pre-training adapts a foundation model to a specific domain by training it further on specialized, unlabeled data using self-supervised learning.
#3about 6 minutes
Developing skills and reasoning with supervised fine-tuning
Supervised fine-tuning uses instruction-based datasets to teach models specific tasks, chat capabilities, and complex reasoning through techniques like chain of thought.
#4about 8 minutes
Aligning models with human preferences using reinforcement learning
Preference alignment refines model behavior using reinforcement learning, evolving from complex RLHF with reward models to simpler methods like DPO.
#5about 2 minutes
Using frameworks like NeMo RL to simplify model alignment
Frameworks like the open-source NeMo RL abstract away the complexity of implementing advanced alignment algorithms like reinforcement learning.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
02:26 MIN
Understanding the core capabilities of large language models
Data Privacy in LLMs: Challenges and Best Practices
01:12 MIN
Introducing InstructLab for accessible LLM fine-tuning
Unlocking the Power of AI: Accessible Language Model Tuning for All
08:05 MIN
How large language models are trained
Inside the Mind of an LLM
05:18 MIN
Addressing the core challenges of large language models
Accelerating GenAI Development: Harnessing Astra DB Vector Store and Langflow for LLM-Powered Apps
02:55 MIN
Addressing the key challenges of large language models
Large Language Models ❤️ Knowledge Graphs
03:42 MIN
Using large language models as a learning tool
Google Gemini: Open Source and Deep Thinking Models - Sam Witteveen
02:21 MIN
The training process of large language models
Google Gemini: Open Source and Deep Thinking Models - Sam Witteveen
What Are Large Language Models?Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of GitHub Copilot know the feeling: this new kind of te...
Benedikt Bischof
MLops – Deploying, Maintaining And Evolving Machine Learning Models in ProductionWelcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Bas Geerdink who gave advice on MLOps.About the speaker:Bas is a programmer, scientist, and IT manager. At ING, he is responsible for the Fast...
Krissy Davis
The Best Large Language Models on The MarketLarge language models are sophisticated programs that enable machines to comprehend and generate human-like text. They have been the foundation of natural language processing for almost a decade. Although generative AI has only recently gained popula...
Daniel Cranney
Panel Discussion: Responsible AI in Practice - Real-World Examples and ChallengesIntroductionIn the ever-evolving landscape of artificial intelligence, the concept of "responsible AI" has emerged as a cornerstone for ethical and practical AI implementation. During the WWC24 Panel discussion, three eminent experts—Mina, Bjorn Brin...
From learning to earning
Jobs that call for the skills explored in this talk.
27:11
31:50
29:20
30:04
58:00
29:11
34:21
28:38.png?w=240&auto=compress,format)
.gif?w=240&auto=compress,format)
.png?w=240&auto=compress,format)
