A recommendation system for substitute products is needed across multiple touchpoints to prevent lost sales from out-of-stock items.
The previous system, based on the Jaccard coefficient, produced low-quality recommendations, particularly for new or unpopular items.
The Prod2Vec algorithm, adapted from Word2Vec, learns product relationships by analyzing co-occurrence within user session context windows.
Incorporating product metadata like category and brand into the model (Meta-Prod2Vec) significantly improves recommendation quality for long-tail items.
The production system uses dbt for data transformation, a Vertex AI pipeline for model training, and Elasticsearch for efficient vector similarity search.
Offline metrics like NDCG confirmed model quality, while mirror traffic analysis showed a 45% increase in product recommendation coverage.
Using TensorFlow's Embedding Projector tool reveals how the model groups similar products into distinct clusters in a high-dimensional space.
Key project takeaways include using simple business-logic baselines for benchmarking and automating exploratory data analysis within the ML pipeline itself.
The project was completed in nine months by a cross-functional team of data engineers, data scientists, and software developers.
58:21Julian Joseph
35:16Ankit Patel
37:24David Mosen
22:41Christoph Fassbach & Daniel Rohr
59:39Julian Joseph
21:01Jan Schweiger
48:01Jan Zawadzki
29:24Nico Axtmann
.gif?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
Understanding Recruitment Group
Barcelona, Spain
Aws Emea Sarl