An initial strategy of collecting massive amounts of data led to high costs, complex queries, and AI models that produced incorrect results.
The data mesh philosophy makes data producers responsible for their data, creating clear ownership from domain systems to domain datasets.
A true data product must have six key qualities: ownership, documentation, data quality checks, architecture, a data contract, and security.
Classifying data products into three tiers based on business impact helps focus governance efforts on the most critical assets first.
Data contracts are formal agreements between data producers and consumers that define responsibilities, SLAs, and procedures for handling issues with critical data.
Implementing data governance is not just a technical fix but a long-term cultural shift that requires changing the entire company's mindset around data.
Implementing governance dramatically reduced the number of tracked interactions, queries, and dashboards while improving data completeness and analyst productivity.
With a governed data foundation in place, AI can be successfully used for semantic validation, anomaly detection, and powering a natural language query Slack bot.
The key takeaway is that more data often leads to more confusion, and implementing strong data governance is the only way to create clarity and value.
28:29Taqi Jaffri, Kapil Gupta & Farooq Sheikh and Tomislav Tipurić
19:44Dominik Schneider
27:53Marin Niehues
31:15Thomas Wollmann, Gerrit Einhoff, Kara Sprague & Alexandra Wudel
30:28Alexandra Wudel, Madalina Florean & Laura Moritz
28:19Jeff Hausmann, Jasmin Kaiser, Bernd Datler & Sonja Alvarez
29:34Dr. Saskia Meier-Andrae, Stefan Bär, Daniel Gebler & Keno Dreßel
26:43Kadir Mourat
Jobs that call for the skills explored in this talk.
TRUSTEQ GmbH