Design Patterns for Large Scale Data Movement

Design Patterns for Large Scale Data Movement

Wednesday, 1:45pm - 2:35pm

There is no “best” way to distribute data within applications. If you ask an experienced architect for advice you’ll get a litany of followup questions before they’re willing to make recommendations: LAN or WAN? Large blocks of data or small? How many updates per second? They’ll also want to know how “real-time” you need it to be (i.e. what kind of latency you can tolerate), how many recipients each update needs to go to and what level of delivery or transaction guarantee you need. They’ll want to understand your network environment – private, public, cloud, etc. – and any requirements related to archival, regulatory compliance, uptime and disaster recovery. Each of those factors affects the protocols you may choose and manner of implementation decisions.


This session will describe popular data movement design patterns, discuss the myriad of deployment considerations that impact manageability and high availability, and illustrate the impact of such decisions with real world examples.

Ken.Overton's picture
As a consultant with Lab49 Ken's work as developer and architect has run the gamut of financial applications from Pricing and Analytics to High Frequency Trading, for small prop trading desks and some of the highest profile SDP's. He is a subject matter expert in Trading Systems Architecture, Complex Event Processing, Service Oriented Architecture, and Event Driven Architecture. He is a fan of any language with a Hindley-Milner type system and the Detroit Red Wings.