Skip to content

Use Local Storage

Deployment

See Local Path Provisioner for deploying the local storage nodes.

Deploy the PXC operator Building Block with the following configuration for the database cluster:

values-pxc-db-*.yaml:

pxc:
  persistence:
    enabled: true
    storageClass: local-path
  nodeSelector:
    pxc-cluster: "true"
  tolerations:
    - key: pxc-cluster
      operator: Exists
      effect: NoSchedule

Troubleshooting

Recovering A Local-Storage Cluster From A Failing Node

Given that a worker node is unrecoverable, the database cluster needs to be recovered manually. This is due to the used local-storage PVC having an affinity to the worker node hence keeping the pod in "Pending" state.

#
# Be sure that the worker node is actually gone!
#

# Get the PVC name of the pod and delete it
kubectl get pods -n syseleven-pxc-operator -l app.kubernetes.io/part-of=percona-xtradb-cluster,app.kubernetes.io/component=pxc --field-selector status.phase=Pending -o "jsonpath={.items[0].spec.volumes[?(@.name=='datadir')].persistentVolumeClaim.claimName}{'\n'}"
kubectl delete pvc -n syseleven-pxc-operator datadir-pxc-db-pxc-X --wait=false

# Get the pod name and delete it. This will trigger it to request a new PVC on a available worker node
# This might be needed a second time if the PVC isn't deleted fast enough by k8s
kubectl get pods -n syseleven-pxc-operator -l app.kubernetes.io/part-of=percona-xtradb-cluster,app.kubernetes.io/component=pxc --field-selector status.phase=Pending -o name | xargs kubectl delete -n syseleven-pxc-operator

# The cluster state will switch back to 'Ready' after pod successfully joined the cluster and sync the data.
kubectl get pxc -n syseleven-pxc-operator

Recovering A Local-Storage Cluster Which Lost Quorum

When a cluster loses its quorum, it must be restored manually. This can happen when multiple worker nodes fail.

Please follow the official guide.