PB³Opt: Profile-based biased Bayesian optimization to select computing clusters on the cloud

TAS Camacho and VM do Rosario and OO Napoli and E Borin, CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 35 (2023).

DOI: 10.1002/cpe.7540

Given the wide variety of cloud computing resources for creating high- performance computer clusters and their complex performance relationship with applications, finding the optimal, or near-optimal, cluster is a complex problem. As a result, several approaches have been proposed to find the optimal, or near-optimal, cluster for a given high-performance computing workload, while reducing the search cost. Among the approaches found in the literature, Bayesian optimization is one of the most known and applied. However, it is still possible to increase its performance by integrating it with historical data related to workload behavior. In this context, we suggest the PB3Opt approach, which introduces a bias in the Bayesian optimization expected improvement acquisition function. The new acquisition function uses the ranking of computer clusters of previously explored workloads that have the same behavior as the workload being optimized. Our experimental results show that PB3Opt classifies the behavior of workloads in groups so that the average- ranking has 88.7% similarity with the ranking of the workload. With this, PB3Opt finds, for almost 95% of workloads, a solution that is less than or equal to 1.2 x worse than the optimal computer cluster. In addition, the PB3Opt works well when combined with the paramount iterations technique and is capable of reducing the search cost significantly.

Return to Publications page