Design Patterns for Large Scale Data Movement

There is no “best” way to distribute data within applications. If you ask an experienced architect for advice you’ll get a litany of followup questions before they’re willing to make recommendations: LAN or WAN? Large blocks of data or small? How many updates per second? They’ll also want to know how “real-time” you need it to be (i.e. what kind of latency you can tolerate), how many recipients each update needs to go to and what level of delivery or transaction guarantee you need. They’ll want to understand your network environment – private, public, cloud, etc. – and any requirements related to archival, regulatory compliance, uptime and disaster recovery. Each of those factors affects the protocols you may choose and manner of implementation decisions.

This session will describe popular data movement design patterns, discuss the myriad of deployment considerations that impact manageability and high availability, and illustrate the impact of such decisions with real world examples.