Module load_balancer

A transport that tries to distribute requests over multiple upstreams.

It is assumed that the upstreams have similar performance. use the super::redundant transport to forward requests to the best upstream out of upstreams that may have quite different performance.

Basic mode of operation

Associated with every upstream configured is optionally a burst length and burst interval. Burst length deviced by burst interval gives a queries per second (QPS) value. This be use to limit the rate and especially the bursts that reach upstream servers. Once the burst length has been reach, the upstream receives no new requests until the burst interval has completed.

For each upstream the object maintains an estimated response time. with the configuration value slow_rt_factor, the group of upstream that have not exceeded their burst length are divided into a ‘fast’ and a ‘slow’ group. The slow group are those upstream that have an estimated response time that is higher than slow_rt_factor times the lowest estimated response time. Slow upstream are considered only when all fast upstream failed to provide a suitable response.

Within the group of fast upstreams, the ones with the lower queue length are preferred. This tries to give each of the fast upstreams an equal number of outstanding requests.

Within a group of fast upstreams with the same queue length, the one with the lowest estimated response time is preferred.

Probing

Upstream with high estimated response times may be get any traffic and therefore the estimated response time may remain high. Probing is intended to solve that problem. Using a random number generator, occasionally an upstream is selected for probing. If the selected upstream currently has a non-zero queue then probing is not needed and no probe will happen. Otherwise, the upstream to be probed is selected first with an estimated response time equal to the lowest one. If the probed upstream does not provide a response within that time, the otherwise best upstream also gets the request. If the probes upstream provides a suitable response before the next upstream then its estimated will be updated.

Structs

Config

User configuration variables.

ConnConfig

Configuration variables for each upstream.

Connection

This type represents a transport connection.

Transport

Type that actually implements the connection.