Modern Big Data Systems

At any scale of data today "big" is no longer the problem but instead the start of a solution. Landscape innovations in storage, scaling and access are all expanding the capabilities of data systems to suit your needs. In this track, expert practitioners will show you how they've chosen to solve data problems in new and effective ways from NoSQL to NewSQL.

Day of week:

Friday

Location:

Grand Ballroom - Salon D

Host:

Andrew Hart

Presentations

The Next Wave of SQL-on-Hadoop: The Hadoop Data Warehouse

10:40am - 11:30am

By: Marcel Kornacker
Wrote and solves problems with the tool of choice for SQL on hadoop

Apache Hadoop now increasingly serves as complementary technology for cost-efficient data loading and cleaning to support the enterprise data warehouse (EDW), supporting the EDW’s role in enabling interactive analysis and reporting on relational data. However, thanks to recent advances in the Hadoop ecosystem that expand the range of EDW-equivalent analytic capabilities entirely in open source software, it is now also possible for Hadoop-based enterprise data hubs to serve as an EDW for native Big Data. Thus, costly processes for moving that data into the traditional EDW just for the purpose of analysis are no longer required.

In this session, attendees will hear how one user in the financial services area, which has rolled out Impala to 45 production nodes to date, is using that approach (based on HDFS, Parquet, and Impala) to reduce processing time from hours to seconds and to consolidate unstructured data from different sources such as web applications, non-traditional external data sets, card transactions, and analytical reports to get a single view of all its data.

Let Me Graph That For You: Building a Graph Database Application

1:45pm - 2:35pm

By: Ian Robinson
Wrote the book and leading database on Graph databases.

Graphs are one of the best abstractions we have for modeling connectedness. Graph databases, in turn, are one of the best tools we have for modeling, storing and querying complex, densely-connected data. Today, graphs and graph databases are helping solve some of the world's most challenging data problems, in domains as diverse as search, social networking, recommendations, datacenter management, logistics, entitlements and authorization, route finding, network monitoring, and fraud analysis.

In this session we'll take a peek inside the graphista's toolbox. We'll look at some of the tools and techniques you can use to graph your world, experiment with graph data, and apply it in your own applications. With a focus on test-driven, incremental and iterative development, I'll show how to build a flexible graph model to express your domain, and the graph queries to satisfy your application's needs.

Analyzing Big Data On The Fly

4:15pm - 5:05pm

By: Shawn Gandhi
Solution Architect at Amazon Web Services

Data is all around us. Whether it's log files generated from mobile and web applications, in-game player activity, or the deluge of information from trading floors, social apps, or geo-spacial monitoring. Collecting, analyzing, and responding to that data helps you stay in tune with your customers, business, and market trends. Analyzing your data while it's fresh allows you to react much quicker than the traditional end-of-day batch jobs we're often used to.

In this session, we’ll provide an overview of the key scenarios and business use cases suitable for real-time processing, and how developers are using AWS Kinesis to shift from a traditional batch-oriented approach to a continual real-time data processing model. We’ll explore the key concepts, APIs and features of the service in the context of building a Kinesis-enabled application for real-time processing. We’ll walk through a candidate use case in detail, starting from creating an appropriate Kinesis stream for the use case, configuring data producers to push data into Kinesis, and downstream applications that read from Kinesis to perform real-time processing.

Presentations

Big Data Open Space

5:30pm - 6:20pm

How WebMD maintains operational flexibility with NoSQL

3:00pm - 3:50pm

By: Rajeev Borborah , Matthew Wilson
WebMD Sr. Director of Engineering-- WebMD Director of Web Ops

In this session, Rajeev Borborah, WebMD Sr. Director of Engineering and Matt Wilson, WebMD Director of Web Operations, will highlight the growing role of NoSQL at the company. With over 160 applications behind WebMD properties, the engineering and operations teams utilize all types of backends – Oracle, MySQL, MSSQL, MongoDB, and Couchbase to name just a few. These applications read and write data to caches, relational databases, key / value stores, and document databases. In order to support its growing operations, while maintaining operational flexibility and ease of use, the WebMD team is broadening its use of NoSQL. This session will focus on the NoSQL caching and key / value storage implementation behind a few of the WebMD applications: Physician Finder, Symptom Checker and WebMD Runtime. Rajeev and Matt will discuss the NoSQL architecture, including ramp up, benefits, and roadmap.

The Game of Big Data: Scalable, Reliable Analytics Infrastructure at KIXEYE

11:55am - 12:45pm

By: Randy Shoup
CTO at KIXEYE, former Dir. Google App Engine and eBay Chief Engineer

As a maker of real-time strategy games for web and mobile, KIXEYE's business depends on deep insights into how players play our games. By analyzing player behavior in a rich and flexible way, we are able to better target our efforts around user acquisition, game balance, player retention, and game monetization. By storing and analyzing data in standard ways, our data scientists are better able to take learnings from one game and apply them to another.

This session will describe KIXEYE's newly-minted modern analytics infrastructure soup-to-nuts, from Kafka queues through Hadoop 2 to Hive and Redshift. It will outline our efforts around power and flexibility, experimentation and iteration, componentization and testability, and reliability and replayability. It will further share our lessons learned in building, testing, operating, and enhancing this mission-critical piece of our infrastructure.