Zaid Zaim

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

What if your next AI assistant was a character you could meet in VR? Learn to build an immersive copilot with Unity, .NET, and Azure OpenAI.

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
#1about 3 minutes

Introducing the concept of an immersive AI copilot

The talk introduces building an interactive AI character inspired by Disney's WALL-E using immersive technologies and generative AI.

#2about 3 minutes

Using Microsoft Mesh for collaborative XR experiences

Microsoft Mesh integrates into Microsoft Teams to provide a platform for immersive meetings, trainings, and social events.

#3about 1 minute

The core technologies for building an immersive copilot

The project is built using three main components: the Unity game engine, the Mesh Toolkit for deployment, and Azure OpenAI for intelligence.

#4about 5 minutes

Integrating Azure OpenAI service with Unity via REST API

A C# script in Unity calls the Azure OpenAI REST API to handle communication, parse the JSON response, and manage text-to-speech output.

#5about 2 minutes

Using the Mesh Toolkit for cross-platform deployment

The Mesh Toolkit provides shaders, visual scripting, and an optimizer to prepare and deploy the Unity environment to the Mesh platform.

#6about 6 minutes

Live demo of the AI copilot on a HoloLens device

A live demonstration shows interaction with the AI character "Anna" using voice commands on a Microsoft HoloLens.

#7about 6 minutes

Live demo in a shared Microsoft Mesh environment

The AI copilot is demonstrated within a shared, multi-user Microsoft Mesh space, showing collaborative interaction with the character.

#8about 6 minutes

Integrating immersive spaces directly into Microsoft Teams

The final demo showcases how users can join a Microsoft Mesh immersive space directly from a standard Microsoft Teams meeting.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Live demo of the immersive AI assistant Anna
03:03 MIN

Live demo of the immersive AI assistant Anna

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Fusing mixed reality, AI, and drones for magical experiences
03:32 MIN

Fusing mixed reality, AI, and drones for magical experiences

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Exploring the core technology stack for the project
01:51 MIN

Exploring the core technology stack for the project

How Mixed Reality, Azure AI and Drones turned me into a Magician?

A practical walkthrough of the Azure AI Foundry playground
04:24 MIN

A practical walkthrough of the Azure AI Foundry playground

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Creating an intelligent cockpit experience with AI and AR
02:06 MIN

Creating an intelligent cockpit experience with AI and AR

Software defines the vehicle: Why customers and developers will love cars even more

Getting started with the Azure AI Foundry platform
01:27 MIN

Getting started with the Azure AI Foundry platform

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Live demo of a cross-platform XR interaction toolkit
03:35 MIN

Live demo of a cross-platform XR interaction toolkit

Apple Vision Pro: Proven Development Methods Meet the Latest Technology

Exploring the tools for drone and MR development
02:41 MIN

Exploring the tools for drone and MR development

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Featured Partners

Related Videos

See all videos
How Mixed Reality, Azure AI and Drones turned me into a Magician?
35:26

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Zaid Zaim

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
38:28

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

Livecoding with AI
31:39

Livecoding with AI

Rainer Stropek

Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact
23:33

Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact

Oliver Will

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More
1:09:49

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More

Simi Olabisi

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney
1:12:04

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Best practices: Building Enterprise Applications that leverage GenAI
24:24

Best practices: Building Enterprise Applications that leverage GenAI

Damir

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure
26:35

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure

Ricardo

Related Articles

View all articles
Chris Heilmann
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
Join Scott Hanselman at WWC24 to explore AI's role as a superhero or supervillain. Scott shares his 32 years of experience in software engineering, discusses AI myths, ethical dilemmas, and tech advancements. Engage with his live demos and insights o...
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
Chris Heilmann
Exploring AI: Opportunities and Risks for Developers
In today's rapidly evolving tech landscape, the integration of Artificial Intelligence (AI) in development presents both exciting opportunities and notable risks. This dynamic was the focus of a recent panel discussion featuring industry experts Kent...
Exploring AI: Opportunities and Risks for Developers

From learning to earning

Jobs that call for the skills explored in this talk.