Commercial clouds are quickly becoming the goto platform for hosting on-demand and dynamically scalable services for scientific analyses and computation. Dynamic provisioning of resources is a critical element when utilising the cloud for executing large-scale and complex scientific analyses. In particular, hosting and managing data intensive applications on the cloud raises new challenges in terms of workflow scheduling. In this paper, we introduce a new Budget Aware Trickling (BAT) algorithm that addresses eScience workflow scheduling in the cloud. Our main focus in this paper is on data intensive applications that appear in scientific domains dealing with a large amount of data. The BAT algorithm builds upon the concept of Constrained Critical Paths (CCP) to execute a set of tasks on the same instance to lower the cost of communication and data movement between instances. Our BAT algorithm distributes budget based on the dependency structure inherent in workflows and we show that it yields 30% reduction in makespan while maintaining consistent success rate.
['Vahid Arabnejad', 'Kris Bubendorfer', 'Bryan Ng']
A Budget-Aware Algorithm for Scheduling Scientific Workflows in Cloud