Scheduling large jobs by abstraction refinement

T.A. Henzinger, V. Singh, T. Wies, D. Zufferey, in:, ACM, 2011, pp. 329–342.

Download
No fulltext has been uploaded. References only!

Conference Paper | Published | English
Department
Abstract
The static scheduling problem often arises as a fundamental problem in real-time systems and grid computing. We consider the problem of statically scheduling a large job expressed as a task graph on a large number of computing nodes, such as a data center. This paper solves the large-scale static scheduling problem using abstraction refinement, a technique commonly used in formal verification to efficiently solve computationally hard problems. A scheduler based on abstraction refinement first attempts to solve the scheduling problem with abstract representations of the job and the computing resources. As abstract representations are generally small, the scheduling can be done reasonably fast. If the obtained schedule does not meet specified quality conditions (like data center utilization or schedule makespan) then the scheduler refines the job and data center abstractions and, again solves the scheduling problem. We develop different schedulers based on abstraction refinement. We implemented these schedulers and used them to schedule task graphs from various computing domains on simulated data centers with realistic topologies. We compared the speed of scheduling and the quality of the produced schedules with our abstraction refinement schedulers against a baseline scheduler that does not use any abstraction. We conclude that abstraction refinement techniques give a significant speed-up compared to traditional static scheduling heuristics, at a reasonable cost in the quality of the produced schedules. We further used our static schedulers in an actual system that we deployed on Amazon EC2 and compared it against the Hadoop dynamic scheduler for large MapReduce jobs. Our experiments indicate that there is great potential for static scheduling techniques.
Publishing Year
Date Published
2011-04-10
Page
329 - 342
Conference
EuroSys
Conference Location
Salzburg, Austria
Conference Date
2011-04-10 – 2011-04-13
IST-REx-ID

Cite this

Henzinger TA, Singh V, Wies T, Zufferey D. Scheduling large jobs by abstraction refinement. In: ACM; 2011:329-342. doi:10.1145/1966445.1966476
Henzinger, T. A., Singh, V., Wies, T., & Zufferey, D. (2011). Scheduling large jobs by abstraction refinement (pp. 329–342). Presented at the EuroSys, Salzburg, Austria: ACM. https://doi.org/10.1145/1966445.1966476
Henzinger, Thomas A, Vasu Singh, Thomas Wies, and Damien Zufferey. “Scheduling Large Jobs by Abstraction Refinement,” 329–42. ACM, 2011. https://doi.org/10.1145/1966445.1966476.
T. A. Henzinger, V. Singh, T. Wies, and D. Zufferey, “Scheduling large jobs by abstraction refinement,” presented at the EuroSys, Salzburg, Austria, 2011, pp. 329–342.
Henzinger TA, Singh V, Wies T, Zufferey D. 2011. Scheduling large jobs by abstraction refinement. EuroSys 329–342.
Henzinger, Thomas A., et al. Scheduling Large Jobs by Abstraction Refinement. ACM, 2011, pp. 329–42, doi:10.1145/1966445.1966476.

Export

Marked Publications

Open Data IST Research Explorer

Search this title in

Google Scholar