A Software Engineer’s Guide to Reliable Data Pipelines

Important, customer-facing applications increasingly rely on data —which means that in order for the application to be reliable, the data needs to be reliable as well. For many software engineers, however, data reliability is a new concept. Where do you start, what tools do you need, and how do you manage the data powering your application?

Egor Gryaznov, the co-founder and CTO of Bigeye and was previously one of the first data engineers at Uber. Egor will draw from his experience supporting thousands of internal users and mission-critical workloads at Uber to provide an actionable guide to data pipeline reliability for software engineers.

In this presentation, you will learn:

• How to approach building data pipelines for a data application
• What tools you will encounter in development and what you need to know about each
• How to ensure reliable data pipelines for your application
• How to manage the data that your application creates