Session + Live Q&A
Data Mesh: An Architectural Deep Dive
Data Mesh is a paradigm shift in how we imagine and build big data management solutions, and most importantly is a shift in how we form our teams around these solutions and govern them. Data Mesh departs from half a century old assumptions on how we need to manage analytical data to be useful. It shifts analytical data architecture from monolithic - such as lake and warehouse - to distributed; data ownership from centralized teams to decentralized owners; from data as an asset to hoard to a product to delight its users; from manual and centralized governance to a federated and computational one; and shifts from data platforms as a utility layer to a set of experience planes. This talk introduces Data Mesh foundations at the high level and deep dives into a proposed logical architecture to bring the paradigm to life. It introduces the architecture of new concepts such as data products - the quantum of Data Mesh architecture - and its affordances, as well as the planes of the data platform in support of computational governance and distribution. If you are a critical thinker with an architectural eye and keen to be part of the next generation solutions to democratize and decentralize big data, this talk is for you.
Speaker
Zhamak Dehghani
Director of Emerging Technologies @thoughtworks & Creator of the Data Mesh concept
Zhamak Dehghani works with ThoughtWorks as the director of emerging technologies in North America, with a focus on distributed systems and big data architecture, with a deep passion for decentralized technology solutions - the foundations for democratization: data mesh, decentralized trust and...
Read moreFind Zhamak Dehghani at:
From the same track
Robust Foundation for Data Pipelines at Scale - Lessons From Netflix
Thursday May 27 / 10:10AM EDT
At Netflix, Data/ML pipelines are widely used and have become central for the business. A very wide scenario presents diverse use cases that go beyond recommendations, predictions and data transformations. As big data and ML gains presence and becomes more impactful, the scalability and stability...
Jun He
Sr. Software Engineer in the Big Data Orchestration Team @Netflix
Harrington Joseph
Sr. Software Engineer @Netflix Data Platform Orchestration Team
Data Mesh in the Real World: Lessons Learnt From the Financial Markets
Thursday May 27 / 09:10AM EDT
CMC Markets, a FTSE 250 financial services company, has been running a successful trading platform for more than 30 years and is now undertaking a broad and ambitious transformation to take advantage of new technologies and ways of working. One vital aspect of a successful transformation to a...
Tareq Abedrabbo
Core Data Principal Engineer @CMCMarkets
Data Pipelines & Data Mesh: Where We Are and How the Future Looks Like
Thursday May 27 / 12:10PM EDT
In the panel, experts with different backgrounds will discuss the current challenges for building Modern Data Pipelines and applying Data Mesh in the real world. We will also discuss how the future looks like in terms of new techniques, architectures, and tools to make effective data-based projects.
Zhamak Dehghani
Director of Emerging Technologies @thoughtworks & Creator of the Data Mesh concept
Tareq Abedrabbo
Core Data Principal Engineer @CMCMarkets
Jacek Laskowski
IT freelancer, Java Champion & Author of "The Internals Of"