Generative AI on mobile enables powerful, local-first applications like group chat summarization and high-quality audio generation without an internet connection.
The Arm CPU offers scalability, security, and an "optimize once, deploy everywhere" model, making it ideal for high-performance, low-latency AI applications.
A wide range of open-source frameworks, each with unique strengths, are available for deploying AI models on Arm-powered mobile devices.
The KleidiAI library provides highly optimized, low-level routines that integrate directly into popular AI frameworks to ensure the best performance on Arm CPUs.
The AudioGen pipeline runs locally by combining multiple models and processing steps, requiring data type flexibility like FP32 and FP16 for optimal quality.
Generative AI enables smart speakers to run entirely locally, combining speech-to-text, LLM, and text-to-speech models for a private user experience.
The Scalable Matrix Extension 2 (SME2) for Armv9 CPUs uses the Matrix Outer Product Accumulate (MPA) instruction to dramatically accelerate matrix multiplication.
SME2 delivers over six times better performance for key generative AI models like Gemma and Whisper, enabling real-time text summarization and audio generation.
With SME2 support coming to Android, developers using AI frameworks with KleidiAI integration will automatically receive significant performance boosts without any code changes.
19:39Andrew Waafa
17:18Andrew Wafaa
50:12Sam Witteveen
30:36Christian Liebel
32:27Christian Liebel
27:23Nico Martin
35:16Ankit Patel
29:52Anshul Jindal & Martin Piercy
.png?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
Apple Firmenprofil
Analog Devices