The current AI landscape is fragmented with many tools, making it complex to build, scale, and maintain applications with features like RAG and agents.
Llama Stack, an open-source project from Meta, provides a standardized, modular framework to simplify AI development with a single API for various components.
Llama Stack abstracts away differences between local and remote LLMs and integrates safety shields to filter harmful inputs and outputs.
Llama Stack organizes the complex RAG process into three distinct, swappable layers for vector embeddings, retrieval, and agentic workflows.
Llama Stack simplifies agent creation by integrating tools, orchestration, and reasoning models through the standardized Model Context Protocol (MCP).
Llama Stack provides out-of-the-box telemetry using OpenTelemetry, enabling developers to trace multi-step agent workflows with tools like Jaeger.
This live demo showcases running Llama Stack locally to perform inference, block unsafe prompts, use an agent to check the weather, and inspect traces in Jaeger.
Llama Stack enables a seamless transition from a local development setup to a scalable production environment on Kubernetes by maintaining a consistent API.
A complex agent interacts with multiple MCP servers to query a CRM, analyze customer data, send Slack notifications, and generate a PDF report.
34:21Aarno Aukia
30:04Roberto Carratalá & Cedric Clyburn
26:47Alejandro Saucedo
30:01Mario Fusco
32:24Bruno Meseguer & Markus Eisele
23:33Oliver Will
31:53Timo Salm
30:38Alex Soto
Jobs that call for the skills explored in this talk.
