Cloud-based AI faces challenges like required internet connectivity, data privacy risks, and high costs, creating a need for local alternatives.
Web LLM enables running multi-gigabyte language models like Llama 3 directly in the browser for offline use, despite initial download and initialization times.
In-browser AI performance is accelerated by combining WebAssembly for efficient computation and the new WebGPU API for direct access to the system's GPU.
The Web Neural Network (WebNN) API provides access to dedicated Neural Processing Units (NPUs) for even faster, more efficient on-device model inference.
The experimental Prompt API addresses the issue of redundant model downloads by allowing websites to access a single, shared OS-level model like Gemini Nano.
A demonstration shows how the Prompt API can use a local model to accurately extract structured data from unstructured text, highlighting its practical application.
WebSD brings text-to-image generation to the browser by running Stable Diffusion models locally using WebGPU, enabling creative AI tasks without cloud dependency.
Local AI models offer superior privacy, offline availability, and low cost, but come with trade-offs like lower quality, high system requirements, and slower performance.
While cloud-based models are currently superior, the trend towards more compact open-source models and OS-integrated AI suggests a growing role for local AI in specialized web applications.
30:36Christian Liebel
30:13Maxim Salnikov
27:23Nico Martin
33:51Thomas Steiner
25:17Scott Hanselman
22:07Ankit Patel
25:47Nathaniel Okenwa
29:52Anshul Jindal & Martin Piercy
Jobs that call for the skills explored in this talk.
Conrad Electronic SE
RE-INvent Retail GmbH
webLyzard
Vienna, Austria