# MachineDeployments

## Clients

You can manage as the MachineDeployments in your cluster with different clients.

> [!NOTE]
>
> One cluster may have multiple MachineDeployments to represent different groups of nodes.

> [!NOTE]
>
> **Info**
>
> The Terraform provider is our **recommended** way to manage MachineDeployments as the configuration lives in code and can be automated.

```terraform
resource "metakube_node_deployment" "nodes" {
  cluster_id = metakube_cluster.cluster.id
  spec {
    replicas = var.node_replicas
    template {
      cloud {
        openstack {
          flavor                       = ""
          image                        = data.openstack_images_image_v2.image.name
          use_floating_ip              = false
        }
      }
      operating_system {
        ubuntu {}
      }
      versions {
        kubelet = metakube_cluster.cluster.version
      }
    }
  }
}
```

Complete examples can be found in the [Terraform provider MetaKube examples](<https://github.com/syseleven/terraform-provider-metakube/tree/master/examples>).

Our [How-To guide](<https://documentation.syseleven.de/en/prior-docs/metakube-core/how-to/manage-node-deployments/index.md>) shows how to manage MachineDeployment in the MetaKube UI.

> [!NOTE]
>
> **Info**
>
> Using kubectl (or any other Kubernetes client) does not use the MetaKube API and relies on Kubernetes authorization instead. It exposes the resources directly, which can give you more granular control.

A manifest for a MachineDeployment could look like this:

```yaml
apiVersion: cluster.k8s.io/v1alpha1
kind: MachineDeployment
metadata:
name: nodes
namespace: kube-system
spec:
replicas: 2
selector:
    matchLabels:
    group: nodes
template:
    metadata:
    labels:
        group: nodes
    spec:
    versions:
        kubelet: 1.30.3
    providerSpec:
        value:
        cloudProvider: openstack
        cloudProviderSpec:
            availabilityZone: dbl1
            flavor: m1.small
            floatingIpPool: ext-net
            image: Ubuntu Noble 24.04 (2024-08-01)
            network: metakube-mmxfcmgjkx
            region: dbl
            securityGroups:
            - metakube-mmxfcmgjkx
            serverGroupID: fe9dcd44-f08a-492a-9e64-4e38e1d51573
            subnet: 2969ae04-a429-4c1b-83e3-2f76fc3a342a
        operatingSystem: ubuntu
        operatingSystemSpec: {}
```

## Node lifecycle

### Cluster API

MetaKube models the lifecycle of Nodes with the Kubernetes [Cluster API](<https://github.com/kubernetes-sigs/cluster-api>) which defines custom resources that are `MachineDeployment`, `MachineSet` &amp; `Machine`.

### Provisioning

When a new Machine is created (for example during [rolling update](<#rolling-update>)), MetaKube will:

1. **Create OpenStack resources** (port, server, floating IP)
2. Wait until Node comes up and becomes Ready

   1. Server boots and gets initialization config from the OpenStack metadata service
   2. Server runs initialization code and installs Kubelet
   3. Kubelet gets a client certificate through the CSR
   4. **Kubelet registers the Node for the first time**
   5. OpenStack cloud controller manager initializes the Node metadata (for example addresses)
   6. Critical DaemonSet Pods (CNI) start running
   7. MetaKube removes the `node.syseleven.de/not-ready:NoSchedule` taint and marks the Node as Ready

### Deletion

MetaKube may delete Machines as part of:

- Rolling updates
- Scaling down of MachineDeployments
- Deletion of MachineDeployments

To safely deprovision a Kubernetes Node, MetaKube will:

1. Mark Machine object for deletion
2. Start [draining](<https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/>) the Node, honoring potential PodDisruptionBudgets
3. Clean up resources in cloud
4. Delete the Machine object

This makes it safe to delete a Machine (`kubectl -n kube-system delete machine my-node-1234`) to gracefully replace a particular Kubernetes Node.

> [!NOTE]
>
> **Info**
>
> In case a server corresponding to a Kubernetes Node is deleted directly in OpenStack, the OpenStack cloud controller manager (node controller) will delete the Node object and MetaKube will clean up the corresponding Machine and create a replacement.

### Rolling Update

Any change to a MachineDeployment's [machine template](<#machinetemplate>) will trigger a rolling update of the nodes.

MetaKube will gradually scale up the new MachineSet and wait until the Nodes join the cluster and become Ready before scaling down the old MachineSet.

You can manually trigger a rollover with this `kubectl` command:

```bash
kubectl -n kube-system patch machinedeployment $machinedeployment -p '{"spec": {"template": {"metadata": {"annotations": {"kubectl.kubernetes.io/restartedAt": "'$(date +"%Y-%m-%dT%T.%3N%z")'"}}}}}' --type merge
```

## Configuration

> [!NOTE]
>
> **Info**
>
> Not all these options are available in all clients (UI, Terraform). They may instead use default values.

### MachineDeployment

| Field | Type | Explanation |
| --- | --- | --- |
| `spec.replicas` |  | MetaKube will maintain this number of nodes for this MachineDeployment.<br>See also [autoscaling](<https://documentation.syseleven.de/en/prior-docs/metakube-core/concepts/nodes/autoscaling/index.md>) as an alternative to a static replica count. |
| `spec.strategy.rollingUpdate.maxSurge` | `integer` or `percentage` | During a rolling update, MetaKube may start provisioning this many more new nodes than `spec.replicas`. |
| `spec.strategy.rollingUpdate.maxUnavailable` | `integer` or `percentage` | During a rolling update, MetaKube may start removing this many nodes below `spec.replicas` before new nodes become Ready. |
| `spec.template` | MachineTemplate | Template for the machines that get created.<br>**Note:** *Any change to the machine template will trigger a rolling update!* |

### MachineTemplate

| Field | Type | Explanation |
| --- | --- | --- |
| `metadata.labels` | `map(string)` | Labels that are added to the nodes |
| `spec.versions.kubelet` | `string` (SemVer) | The version of Kubelet. Should ideally be kept the same as the control plane version.<br>Changing this will replace existing nodes with new ones.<br><br>You can [upgrade](<https://documentation.syseleven.de/en/prior-docs/metakube-core/how-to/upgrade-a-cluster/index.md>) the control plane and Kubelet versions independently, as long as it doesn't violate the Kubernetes version skew policy.<br>See [supported Kubernetes versions](<https://documentation.syseleven.de/en/prior-docs/metakube-core/concepts/supported-kubernetes-versions/index.md>) for what Kubelet versions are supported for a given control plane version. |
| `spec.taints` | `list(taint)`, see [docs](<https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#concepts>) | Taints that are added to the nodes |
| `spec.providerSpec.value.cloudProviderSpec` | CloudProviderSpec | Cloud provider specific configuration |
| `spec.providerSpec.value.operatingSystem` | `string` | Distro of the operating system. Either `ubuntu` or `flatcar`. |
| `spec.providerSpec.value.operatingSystemSpec` | OperatingSystemSpec | Operating system specific configuration |
| `spec.providerSpec.value.sshPublicKeys` | `list(string)` | Authorized SSH public keys *initially* installed on the machine<br>Once initialized, the keys are managed by the `ssh-key-agent` DaemonSet Pod. |

### CloudProviderSpec (OpenStack)

| Field | Type | Explanation |
| --- | --- | --- |
| `flavor` | `string` | **Name** of Flavor to use for the OpenStack server<br>See [OpenStack Cloud documentation](<https://documentation.syseleven.de/en/prior-docs/openstack-cloud/reference/compute/#flavors>) for a list of available Flavors and [Allocatable resources](<https://documentation.syseleven.de/en/prior-docs/metakube-core/concepts/nodes/resources/#allocatable-resources>) for limitations. |
| `floatingIpPool` | `string` | **Name** of external network that floating IPs for servers would be allocated from |
| `image` | `string` | OpenStack operating system image **name** or **ID** |
| `rootDiskSizeGB` | `number` | Capacity of root disk in GB<br>*Not available for localstorage flavors.*<br>**Note**: *The replacement disk will incur the standard fee for network storage.* |
| `securityGroups` | `list(string)` | **Names** of security groups that are associated with the server port |
| `serverGroupID` | `string` | **ID** of the server group the server belongs to<br>Defaults to cluster-wide shared server group, see [topology](<https://documentation.syseleven.de/en/prior-docs/metakube-core/concepts/nodes/topology/#node-placement>) |
| `subnet` | `string` | **ID** of subnet the server port is allocated in |
| `tags` | `list(string)` | Additional tags for the server |

### OperatingSystemSpec (Ubuntu)

| Field | Type | Explanation |
| --- | --- | --- |
| `distUpgradeOnBoot` | `bool` | Whether to upgrade packages and reboot (if necessary) on first boot |
