Analyzing Big Data On The Fly

Analyzing Big Data On The Fly

Friday, 4:15pm - 5:05pm

Data is all around us. Whether it's log files generated from mobile and web applications, in-game player activity, or the deluge of information from trading floors, social apps, or geo-spacial monitoring. Collecting, analyzing, and responding to that data helps you stay in tune with your customers, business, and market trends. Analyzing your data while it's fresh allows you to react much quicker than the traditional end-of-day batch jobs we're often used to.


In this session, we’ll provide an overview of the key scenarios and business use cases suitable for real-time processing, and how developers are using AWS Kinesis to shift from a traditional batch-oriented approach to a continual real-time data processing model. We’ll explore the key concepts, APIs and features of the service in the context of building a Kinesis-enabled application for real-time processing. We’ll walk through a candidate use case in detail, starting from creating an appropriate Kinesis stream for the use case, configuring data producers to push data into Kinesis, and downstream applications that read from Kinesis to perform real-time processing.

Shawn.Gandhi's picture
Shawn is a systems architect based in New York City who works with large-scale enterprises designing solutions in the cloud and is the co-creater of the HPC/Cloud Computing Workgroup in New York. Prior to joining AWS, he worked on Wall Street building bespoke solutions for many of the world's largest investment banks, hedge funds, and exchanges. Originally from Canada, Shawn studied both hardware and software engineering and is currently focused on real-time data streaming and processing. @shawnagram