Repository: Freie Universit├Ąt Berlin, Math Department

A mathematical programming approach for resource allocation of data analysis workflows on heterogeneous clusters

Mohammadi, Somayeh and PourKarimi, Latif and Droop, Felix and De Mecquenem, Ninon and Leser, Ulf and Reinert, Knut (2023) A mathematical programming approach for resource allocation of data analysis workflows on heterogeneous clusters. The Journal of Supercomputing, 79 (17). pp. 19019-19048. ISSN 0920-8542

Full text not available from this repository.

Official URL: https://doi.org/10.1007/s11227-023-05325-w

Abstract

Scientific communities are motivated to schedule their large-scale data analysis workflows in heterogeneous cluster environments because of privacy and financial issues. In such environments containing considerably diverse resources, efficient resource allocation approaches are essential for reaching high performance. Accordingly, this research addresses the scheduling problem of workflows with bag-of-task form to minimize total runtime (makespan). To this aim, we develop a mixed-integer linear programming model (MILP). The proposed model contains binary decision variables determining which tasks should be assigned to which nodes. Also, it contains linear constraints to fulfill the tasks requirements such as memory and scheduling policy. Comparative results show that our approach outperforms related approaches in most cases. As part of the post-optimality analysis, some secondary preferences are imposed on the proposed model to obtain the most preferred optimal solution. We analyze the relaxation of the makespan in the hope of significantly reducing the number of consumed nodes.

Item Type:Article
Subjects:Mathematical and Computer Sciences > Computer Science
Divisions:Department of Mathematics and Computer Science > Institute of Computer Science > Algorithmic Bioinformatics Group
ID Code:3140
Deposited By: Anja Kasseckert
Deposited On:18 Apr 2024 10:36
Last Modified:18 Apr 2024 10:36

Repository Staff Only: item control page