Autoscaling · Cloud and Reliability

Why you need it

The intuition

desired = ceil( current_replicas × current_metric / target_metric )

Definition

Autoscaling

Automatically changing the number of copies of a service based on a metric such as CPU or requests per second.

Target utilisation

The level of a metric the autoscaler tries to keep each copy at, such as 50% CPU.

Flapping

Scaling up and down repeatedly because of small changes. A tolerance or a cool-down prevents it.

Examples

1
2
3
4
5
6
7
8

import math

def desired(current, cpu, target):
    return math.ceil(current * cpu / target)

print(desired(4, 80, 50))
print(desired(4, 50, 50))
print(desired(10, 12, 50))

7
4
3

1
2
3
4
5
6
7
8
9
10

import math

def desired(current, cpu, target):
    return math.ceil(current * cpu / target)

def bounded(current, cpu, target, lo, hi):
    return max(lo, min(hi, desired(current, cpu, target)))

print(bounded(4, 300, 50, 2, 10))
print(bounded(4, 5, 50, 2, 10))

10
2

Common mistakes

Quick quiz

1
2
3
4
5
6

import math

def desired(current, cpu, target):
    return math.ceil(current * cpu / target)

print(desired(3, 90, 60))

Practice

3 exercises in three steps: predict what a program prints, change a program to meet a new requirement, then write one from scratch. Your code runs in the browser and is checked the moment you submit.

1. PredictA busy morning

   Type exactly what this prints.

2. ModifyStop the flapping

   This autoscaler adds or removes a pod on every tiny change. Add the tolerance: if cpu / target is within 10% of 1 (from 0.9 to 1.1, inclusive), keep current unchanged.

3. WriteReplay a day

   Write replay(start, readings, target, lo, hi) that returns the replica count after each CPU reading. For each reading:

   1. ratio = cpu / target; if it's from 0.9 to 1.1, keep the current count
   2. otherwise ceil(current × ratio)
   3. then clamp to lo…hi

   replay(2, [50, 90, 300, 10], 50, 2, 8)   # [2, 4, 8, 2]

Terms in this lesson