The concept of vision for websites is redefined from simply seeing to the ability to deduce, understand, and act on information.
A live demonstration showcases an e-commerce store where users can search for products using both text queries and by uploading images.
Multimodality enables search queries to use multiple media types like text, images, and audio to capture more context and improve user interaction.
Multimodal interfaces provide more natural and context-aware interactions, moving beyond simple keyword searches to a more intuitive experience.
Embedding models encapsulate information into numerical representations (vectors), unlike generative models which create new data.
Vector search operates by converting a query into an embedding and finding the closest, most semantically similar items in a multidimensional space.
Different data types like text, images, and audio are processed by specific encoders and plotted into a single, unified vector space for cross-modal queries.
A code walkthrough demonstrates how to build a text-to-image search feature using a Next.js frontend and a Weaviate backend with a `nearText` query.
The code for an image-to-image search is explained, showing how a base64 image is sent to the backend to perform a `nearImage` vector search.
Beyond e-commerce, multimodal vision can be applied to creative use cases like movie recommenders, educational tools, and map navigation.
1:12:04
1:03:08
21:35Eliran Natan
43:53Sasha Shynkevich
41:40Drishti Jain
41:43Alexander Lichter
44:43Nikita Dubko
27:23Nico Martin
.webp?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
Speech Processing Solutions
Vienna, Austria
infomax websolutions GmbH
Grassau, Germany
doinstruct Software GmbH
Berlin, Germany
Zebresel - Deine Agentur für d igitale Medien.