Stream Processing Systems:

FP4S, a novel fragment-based parallel state recovery mechanism that can handle many simultaneous failures for a large number of concurrently running stream applications. The novelty of FP4S is that we organize all the application’s operators into a distributed hash table (DHT) based consistent ring to associate each operator with a unique set of neighbors. Then we divide each operator’s in-memory state into many fragments and periodically save them in each node’s neighbors, ensuring that different sets of available fragments can reconstruct lost state in parallel. This approach makes this failure recovery mechanism extremely scalable, and allows it to tolerate many simultaneous operator failures. We apply FP4S on Apache Storm and evaluate it using large-scale real-world experiments, which demonstrate its scalability, efficiency, and fast failure recovery features. When compared to the state-of-the-art solutions (Apache Storm), FP4S reduces 37.8% latency of state recovery and saves more than half of the hardware costs. It can scale to many simultaneous failures and successfully recover the states when up to 66.6% of states fail or get lost.

See more…


SpamHunter, which can utilize spam correlations from distributed data sources to enhance the performance of large-scale social spam detection. It identifies the correlated social spam from various distributed servers/sources through DHT-based hierarchical functional trees. These functional trees act as bridges among data servers/sources to aggregate, exchange, and communicate the updated and newly emerging social spam with each other. Furthermore, by processing the online social logs instantly, it allows online streaming data to be processed in a distributed manner, which reduces the online detection latency and avoids the inefficiency of outdated spam posts. Our experimental results with real-world social logs demonstrate that SpamHunter reaches 95% F1 score in the spam detection, achieves high efficiency in scaling to a large amount of data servers with low latency.

See more…


Oases, a system that implements a new decentralized overlay-based architecture to support social spam detection in online and distributed manner, with fetching online social data from multiple distributed web servers and updating the classifier with the aggregation of new social spam features to catch the latest spam. Experiments with an implementation of Oases have shown an efficient processing with more than 90% accuracy in online spam detection and attractive load balancing in scaling with hundreds of agents and millions of social posts.

See more…


GOVERNOR, a controller that factors the anticipated checkpointing costs into the backpressure mechanism in stream processing systems, and thus reduces the processing delay caused by checkpointing proactively, lowers the risk of system instability and improves overall throughput. Experiments with an implementation of GOVERNOR in Apache Spark Streaming have shown an overall performance improvement of up to 26% for representative streaming operators and real-world workloads, with negligible overhead.

See more…


ELF, a system that implements a new decentralized “many masters/many workers” architecture to support high-speed iterative, batch, and streaming processing jobs all together, and to offer runtime job feedback and cross-job coordination on large clusters. An ELF prototype implemented and evaluated for a larger scale configuration demonstrates high per-node throughput, sub-second job latency, and sub-second ability to adjust the actions of jobs being run.

See more…


Project Hoover, a middleware that provides the elasticity to horizontally scale-up or scale-down the real-time data processing systems according to current rates of input events and desired event processing latencies. Experiments with an implementation of Project Hoover in Apache Flume have shown Project Hoover can dynamically and continuously monitor Apache Flume’s performance, and right-size the number of Flume collectors according to different log production rates.

See more…


Datacenters and Cloud Computing:

RBay, an integrated information plane that enables secure and scalable resource sharing between geographically distributed datacenters, by building a decentralized ‘hierarchical aggregation tree’ to seamlessly aggregate spare resources from distributed datacenters and attach each datacenter ‘admin-customized’ handlers to customize the resource sharing policies. An experimental evaluation on eight real-world geo-distributed sites demonstrates RBay’s rapid response to composite queries, as well as its extensible, scalable, and lightweight nature.

See more…


v-Bundle, a system that enables weighted fair resource sharing across diverse applications belonging to one tenant, by letting tenants control their own scheduling and automatically shuffle free resources using DHTs. Experimental evaluations show that v-Bundle can scale well to thousands of hosts and VMs, provide customers with better bandwidth utilization and improved application’s quality of service (QoS) through borrowing extra bandwidth when needed, at no additional cost in terms of the total resources allocated to the customer.

See more…


System Virtualization:

DocMan, a toolset that adopts a black-box approach to discover container ensembles and collect information about intra-ensemble container interactions. It uses a combination of techniques such as distance identification and hierarchical clustering. The experimental results demonstrate that DocMan enables optimized containers placement to reduce the stress on bi-section bandwidth of the datacenter’s network. The method can detect container ensembles at low cost and with 92% accuracy and significantly improve performance for multi-tier applications under the best of circumstances.

See more…


Net-Cohort, a set of lightweight, “black-box” techniques that detect dependencies between collaborative VMs which jointly provide a service or accomplish a task, so as to enable smart VM placements and migrations. An implementation of Net-Cohort on a Xen-based system with 15 hosts and 225 VMs shows that its methods can detect VM ensembles at low cost and with about 90.0% accuracy.

See more…


Magnet, a multi-ring based scheme for optimizing power usage in virtualized datacenters using live VM migrations. The nodes are organized into a multi-layer ring-based overlay for transferring load from outer rings to inner rings. Experimental measurements show that the new scheme can reduce the power consumption by 74.8% over base at most with certain adjustably acceptable overhead. The effectiveness and performance insights are also analytically verified.

See more…


Middleware in Distributed Systems:

PKTown, a Peer-to-Peer middleware embedded into Multiplayer PVP Online Games (e.g., StarCraft, DOTA) that locates nodes satisfying target latency constraints. PKTown captures all packets generated by Star Craft. We apply application layer multicast (ALM) to transmit the broadcast packets through overlay network. PKTown extends the LAN game experiences to players belong to different LAN. No change is made on binary code of Star Craft.

We are thankful to the following federal agencies and Institutes for supporting our research efforts: