Skip to content

FAQ: Resource reservation profiles

What changes now?

Starting immediately, any newly created MachineDeployments will be configured to use the metakube-latest profile.

What changes in January 2025?

Starting on 01st January 2025, Machines belonging to MachineDeployments that don't specify any profile will use the metakube-2025-01 profile. Existing Nodes are not affected.

What is the motivation behind the change?

Some MetaKube customers experienced stability issues when their workloads exhausted the memory on their Nodes. This is because Kubelet was the victim of a system OOM kill since not enough memory was reserved for its cgroup.

How does the new profile differ from the current behavior?

The new profile changes three aspects of Kubelet's configuration:

  • MaxPods

    The Node's Pod limit is scaled appropriately for the offered flavors based on their available memory. The numbers reflect the upper end of typical use and should be sufficient for the vast majority of existing setups.

  • Reserved CPU

    For most common Node flavors Kubelet will now reserve less CPU than before.

  • Reserved memory

    Kubelet but more so the container runtime, containerd, use a significant amount of memory as the number of Pods per Node grows. Our current configuration did not reserve enough memory to account for this on Nodes with many Pods or memory-hungry workloads.

As a result, Nodes will usually have between 200Mi and 500Mi less memory available for Pods. It's still possible to over-commit Nodes with memory (see Memory resource management), but it's more likely for a system OOM kill to happen scoped to the Pod's cgroup.

Where does the formula of the new profile come from?

We conducted thorough experiments simulating many different scenarios.

The tests showed a strong linear correlation between the number of Pods and both the CPU and memory usage of Kubelet and containerd. The formulas capture a safe boundary for the vast majority of workloads while keeping as much of the machine's resources available to the user.

Can I opt in before January 2025?

Yes. Set the profile of your existing MachineDeployment to metakube-2025-01 or metakube-latest. The latter will always match the default profile should we introduce new ones in the future.

I didn't experience instability before. Can I keep the old behavior?

While the current behavior will continue to be available with the metakube-legacy profile, we strongly recommend using the new profile. We can otherwise not ensure the stability of the Nodes.

We consider the metakube-legacy profile deprecated and may remove it in the future.

Can I tune the profile to suit my workloads?

We offer granular control over the individual resources if you have special requirements. However, we cannot guarantee any degree of stability or support.

Can I change the profile through the API?

No. It's currently only possible to change the profile by changing the MachineDeployment directly, for example with kubectl.