The talk introduces a project that combines Azure AI, drones, and mixed reality to create immersive, interactive experiences.
An overview of the key technologies used, including the Unity game engine, Meta Quest 3 for mixed reality, and Azure AI Foundry for intelligence.
Azure AI Foundry is presented as a centralized platform for accessing various AI models, developer tools, and services to build generative AI applications.
A live demonstration shows how to create a project, deploy a GPT-4o model, and use the chat playground to test the AI before integration.
The process of building the mixed reality application involves using the Meta SDK in Unity for camera rigs, controller systems, and crucial hand tracking capabilities.
The Tello drone's UDP-based communication protocol is implemented by feeding its technical architecture to GitHub Copilot, which generates the necessary C# code.
A live demonstration showcases the AI assistant "Anna" in mixed reality, featuring natural language conversation, image recognition, and function calling to manipulate virtual objects.
The final demo shows a real Tello drone being controlled directly through hand movements tracked by the Meta Quest 3 headset, including takeoff and flips.
31:50Zaid Zaim
38:28Zaid Zaim
23:33Oliver Will
1:09:49Simi Olabisi
26:35Ricardo
1:12:04
23:03Bastian Schütz
24:24Damir
Jobs that call for the skills explored in this talk.
Devoteam
Conrad Electronic SE