Georgia Institute of Technology College of Sciences

We consider a service system where incoming tasks are instantaneously assigned to one out of many heterogeneous server pools. All the tasks sharing a server pool are executed in parallel and the execution times do not depend on the class of the server pool or the number of tasks currently contending for service. However, associated with each server pool is a utility function that does depend on the class of the server pool and the number of tasks currently sharing it. These features are characteristic of streaming and online gaming services, where the duration of tasks is mainly determined by the application but congestion can have a strong impact on the quality-of-service (e.g., video resolution and smoothness). We derive an upper bound for the mean overall utility in steady-state and introduce two load balancing policies that achieve this upper bound in a large-scale regime. Also, the transient and stationary behavior of these asymptotically optimal load balancing policies is characterized in the same large-scale regime.