Abstract
Numerous organisations have embraced Cloud Computing as a result of its resource provisioning flexibility and on-demand pricing structures. Entire clusters of machines can now be provided dynamically to fulfil user’s processing demands. By relocating processes to the Cloud, customers expect to lower the cost associated with establishing and maintaining a compute cluster without jeopardising service quality. Due to their large-scale nature, scheduling algorithms are essential for successfully automating their execution in distributed systems, hence facilitating and speeding up scientific collaborations.
Cloud Computing, on the other hand, has introduced scheduling and management challenges that users of traditional resource pooling models, such as grid and cluster computing, have never seen before. To begin, the prices for resource utilisation vary dynamically and are determined by the type and duration of resources consumed; this prohibits users from acquisitively accumulating as many resources as possible. Secondly, the Cloud Computing marketplace provides a diverse range of on-demand resources with varying levels of performance. Due to the range of resources available, it might be challenging for users to build a cluster that is suited for their applications. As a result, users face difficulties ensuring the desired level of service while running Cloud-based services.
This thesis examines novel resource scheduling strategies for scientific workflows in Cloud-based infrastructure. The key problems posed by the multiCloud, heterogeneous instances, and dynamic resource model are capable of impacting quality of service requirements described in terms of execution time, user-defined deadline and resource utilisation. In order to mitigate this impact and contribute to the advancement in the field of Cloud Computing, this work has made following significant contributions: A deadline-constrained resource scheduling algorithm was proposed for scientific workflows across multiple Clouds. The proposed algorithm (APSO) is an extension of the PSO algorithm which is an evolutionary algorithm. APSO uses a random single point mutation operator and a two-point crossover operator based on genetic algorithm (GA) technique to optimise computation and data transfer cost. Performance evaluation with the state-of-the-art evolutionary algorithms show that APSO outperforms existing evolutionary algorithms. To obtain the desired performance of a group of task applications in Cloud Computing while minimising the incurred monetary cost, a deadline constrained cost minimisation (DCCM) strategy was proposed. In DCCM, tasks were grouped based on their scheduling deadline constraints and data dependencies. DCCM focuses on meeting the user-defined deadline by sub-dividing tasks into different levels based on their priorities. Simulation results showed that DCCM achieved higher success rate when compared to the state-of-the-art approaches. To address resource utilisation in Clouds, Random Forest algorithm which is a supervised machine learning strategy was proposed for dynamic resource provisioning. The proposed technique was compared with other decision tree algorithms including XGBoost, Ridge and Lasso machine learning algorithms as well as Deep and Reinforcement learning algorithms. Experimental results showed that Random Forest achieved a higher prediction accuracy for CPU and memory utilisation when compared to other supervised learning and deep and reinforcement algorithms proposed in literature.