Speaker: Mosharaf Chowdhury UC Berkeley
Time: 2013-08-16 14:10-2013-08-16 15:10
Venue: FIT-1-222

Abstract:

Many applications do not constrain the destinations of their network transfers. New opportunities emerge when such transfers contribute a large amount of network bytes. By choosing the endpoints to avoid congested links, completion times of these transfers as well as that of others without similar flexibility can be improved. In this paper, we focus on leveraging the flexibility in replica placement during writes to distributed file systems (DFSes), which account for almost half of all cross-rack traffic in data-intensive clusters. The replicas of a DFS write can be placed in any subset of machines as long as they are in multiple fault domains and ensure a balanced use of storage throughout the cluster.
We study DFS interactions with the cluster network, analyze optimizations for replica placement, and propose Sinbad - a system that identifies imbalance and adapts replica destinations to navigate around congested links. Experiments on EC2 and trace-driven simulations show that block writes complete 1.3X (respectively, 1.58X) faster as the network becomes more balanced. As a collateral benefit, end-to-end completion times of data-intensive jobs improve as well. Sinbad does so with little impact on the long-term storage balance.

Short Bio:

Mosharaf Chowdhury(http://www.mosharaf.com/) is a Ph.D. candidate in the AMPLab at UC Berkeley, working with Prof. Ion Stoica. His research looks at the intersection of systems and networking with specific focus on coflows for datacenter and cloud computing frameworks, and the results have appeared in top-tier venues like SIGCOMM, NSDI, and INFOCOM. He is a winner of the NSDI Best Paper Award (2012), a Facebook Fellowship (2012), an Outstanding Teaching Assistant Award (2009), and a Cheriton Scholarship (2009). Mosharaf received his Master's from the University of Waterloo working with Prof. Raouf Boutaba on network virtualization and his Bachelor's from BUET, Bangladesh.