From Batch to Streams: Building Value from Data In-Motion

Most things in life are a constant flow of events. Falling in love, getting a mortgage, and becoming a parent are good examples. However, something usually goes unnoticed: most events ultimately lead to another, and there is an inherent correlation between them. You just need to look closer and continuously.  

The power of looking at things closer and continuously comes when you realize that your building block of analysis is no longer silos of data but rather streams of data. Currently, systems are designed to capture data and store it — leaving the analysis part for later. This batch-oriented thinking leads to business processes that can't be linked together without extra integration efforts such as ETL and conciliation systems.  

Adopting streaming data architectures requires a fundamental shift in how you see data; and, more importantly — how do you capture, store, and analyze them while they are in motion. FIS companies are beginning to realize the importance of data-in-motion in the enterprise world and how their business processes need to be designed to be in their liquid form — as they are in constant change to adapt to new business needs.

This talk will discuss the risks of designing siloed-based systems and how streaming data architectures can become a solution to address competitiveness and mitigate the problem of business opportunities that happened unseen. It will explain the journey to achieve this within your development teams in the most disruptive form and highlight the changes that may disrupt their business but — are necessary to pave the way for how FIS companies can focus on building value and better customer experiences.

Key Takeaways:

1. Understand the benefits of streaming-based systems for financial industries.
2. Comprehend the journey to move from batch to streams.
3. Illustrate the technical challenges that development teams face.
4. Understand what open source technologies exist and their trade-offs.
5. Highlight common "shortcuts and traps" that must be avoided.