Module tower::balance::p2c

This module implements the “Power of Two Random Choices” load balancing algorithm.

It is a simple but robust technique for spreading load across services with only inexact load measurements. As its name implies, whenever a request comes in, it samples two ready services at random, and issues the request to whichever service is less loaded. How loaded a service is is determined by the return value of Load.

As described in the Finagle Guide:

The algorithm randomly picks two services from the set of ready endpoints and selects the least loaded of the two. By repeatedly using this strategy, we can expect a manageable upper bound on the maximum load of any server.

The maximum load variance between any two servers is bound by ln(ln(n)) where n is the number of servers in the cluster.

The balance service and layer implementations rely on service discovery to provide the underlying set of services to balance requests across. This happens through the Discover trait, which is essentially a Stream that indicates when services become available or go away. If you have a fixed set of services, consider using ServiceList.

Since the load balancer needs to perform random choices, the constructors in this module usually come in two forms: one that uses randomness provided by the operating system, and one that lets you specify the random seed to use. Usually the former is what you’ll want, though the latter may come in handy for reproducability or to reduce reliance on the operating system.

Structs

• Balance
  Efficiently distributes requests across an arbitrary number of services.
• MakeBalance
  Constructs load balancers over dynamic service sets produced by a wrapped “inner” service.
• MakeBalanceLayer
  Construct load balancers (Balance) over dynamic service sets (Discover) produced by the “inner” service in response to requests coming from the “outer” service.
• MakeFuture
  A Balance in the making.