Tracing the rapid advancement of natural language processing from early models like Word2Vec to the powerful generative AI we see today.
Vector embeddings turn words and sentences into numerical arrays, allowing computers to understand semantic relationships through mathematical operations.
The distance between vectors in an embedding space represents semantic similarity, enabling operations like finding related concepts or answering questions.
Elasticsearch serves as a vector database to store document embeddings and integrates with models from sources like Hugging Face for inference.
The Elasticsearch Python client enables complex, multi-field queries with boolean logic to filter data based on precise criteria before adding semantic layers.
An inference pipeline can automatically apply a sentiment analysis model to all documents, adding a new field to enable filtering by positive or negative tone.
By converting all text into vectors using an embedding model, you can perform a k-NN search to find the most semantically relevant results for a query.
Hybrid search combines the power of semantic vector search with traditional keyword filters and exclusions to create highly relevant and precise results.
The speaker answers audience questions about ensuring relevance, handling out-of-vocabulary terms, updating data sources, and debugging model outputs.
56:46Jodie Burchell
57:52Rainer Stropek
1:05:44Chris Heilmann, Daniel Cranney, Raphael De Lio & Developer Advocate at Redis
26:05Christian Weyer
34:34Daniel Madalitso Phiri
1:00:56Mary Grygleski
58:00Vijay Krishan Gupta & Gauravdeep Singh Lotey
24:42Martin O'Hanlon
.png?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
AUTO1 Group SE
Berlin, Germany
Understanding Recruitment Group
Barcelona, Spain