Production-Ready Setup
Scenario
In this example you will create:
- 1 x administrative account
- 2 x user accounts
- 2 x vhosts
- 2 x permissions (granting users access to vhosts)
- 2 x queue policies (enabling redundancy for queues)
Goals
- make RabbitMQ configuration declarative and immutable
- make RabbitMQ queues redundant by mirroring them
Required Configuration
These variables must be defined and populated as GitLab CI variables:
RABBITMQ_ADMIN_PRODUCTION_PASSWDRABBITMQ_PRODUCTION_ERLANGCOOKIE
Define a variable for each user account. Example:
RABBITMQ_USER1_PRODUCTION_PASSWORDRABBITMQ_USER2_PRODUCTION_PASSWORD
You must define these GitLab CI variables in the respective environment you want to use them in. (for example development, staging, production).
See RabbitMQ guide how to create the CI variables.
Deployment
Create a general helm configuration file in the RabbitMQ Building Block directory of your control repository.
values-rabbitmq.yaml:
# best practice values
auth:
# default username for admin account
username: admin
# this feature must be disabled because it is incompatible with importing a RabbitMQ JSON configuration
securePassword: false
# prefix the k8s-secret containing the RabbitMQ configuration with the Helm Release name
extraSecretsPrependReleaseName: true
For each of your environments (for example development, staging, production) create a helm configuration file containing your customizable RabbitMQ parameters.
Please note that the following JSON structure contains RabbitMQ-specific configuration.
Special characters (for example ".") must be escaped adequately (for example "\.").
The following examples use the production environment:
values-rabbitmq-production.yaml:
# define your RabbitMQ users here
users:
user1:
username: my-user1
user2:
username: my-user2
# instruct RabbitMQ to load a customized JSON configuration
loadDefinition:
existingSecret: rabbitmq-load-definition
enabled: true
# k8s-secret to store a customized JSON configuration
# extend this JSON structure to match your needs
# keep in mind that any JSON syntax error will make the configuration import fail!
extraSecrets:
load-definition:
load_definition.json: |
{
"users": [
{
"name": "{{'{{'}} .Values.auth.username {{'}}'}}",
"password": "{{'{{'}} .Values.auth.password {{'}}'}}",
"tags": "administrator"
},
{
"name": "{{'{{'}} .Values.users.user1.username {{'}}'}}",
"password": "{{'{{'}} .Values.users.user1.password {{'}}'}}",
"tags": "management"
},
{
"name": "{{'{{'}} .Values.users.user2.username {{'}}'}}",
"password": "{{'{{'}} .Values.users.user2.password {{'}}'}}",
"tags": "management"
}
],
"vhosts": [
{
"name": "/my-vhost1"
},
{
"name": "/my-vhost2"
}
],
"permissions": [
{
"configure": ".*",
"read": ".*",
"user": "{{'{{'}} .Values.users.user1.username {{'}}'}}",
"vhost": "/my-vhost1",
"write": ".*"
},
{
"configure": ".*",
"read": "export\\.example\\..*",
"user": "{{'{{'}} .Values.users.user2.username {{'}}'}}",
"vhost": "/my-vhost2",
"write": "import\\.example\\..*"
}
],
"policies": [
{
"name": "ha-all",
"pattern": ".*",
"vhost": "/my-vhost1",
"definition": {
"ha-mode": "all",
"ha-sync-mode": "automatic"
}
},
{
"name": "ha-all",
"pattern": ".*",
"vhost": "/my-vhost2",
"definition": {
"ha-mode": "all",
"ha-sync-mode": "automatic"
}
}
]
}
For each of your environments create a helm configuration template containing secrets and password variables:
values-rabbitmq-production.yaml.gotmpl:
# read passwords and secrets from a GitLab CI variables
auth:
password: {{'{{'}} requiredEnv "RABBITMQ_ADMIN_PRODUCTION_PASSWD" {{'}}'}}
erlangCookie: {{'{{'}} requiredEnv "RABBITMQ_PRODUCTION_ERLANGCOOKIE" {{'}}'}}
users:
user1:
password: {{'{{'}} env "RABBITMQ_USER1_PRODUCTION_PASSWORD"{{'}}'}}
user2:
password: {{'{{'}} env "RABBITMQ_USER2_PRODUCTION_PASSWORD"{{'}}'}}
loadDefinition:
existingSecret: {{'{{'}} env "RELEASE_NAME_RABBITMQ" | default "rabbitmq" {{'}}'}}-load-definition
Notice
This is a declarative way of configuring RabbitMQ. Settings which are imported through a JSON load-definition file can only ADD or UPDATE configuration parameters.
It is not possible to DELETE configuration values by removing them from the JSON load-definition above and re-import it again.
If you really need to delete a particular setting then it must be removed through the RabbitMQ Web-UI and all changes need to be reflected in the load-definition JSON script afterwards.
Note
Maintaining a RabbitMQ configuration through both a JSON file and the RabbitMQ Web-UI in parallel will lead to an inconsistent state!
Verify
To verify if your configuration is active in RabbitMQ you can execute these Kubernetes commands:
- Verify if the Kubernetes secret has been created and contains the JSON structure:
kubectl -n syseleven-rabbitmq get secret rabbitmq-load-definition -o jsonpath='{.data.load_definition\.json}' | base64 -d
- Verify from within a RabbitMQ pod: This will also output the JSON file if it was mounted successfully by the pod.
kubectl -n syseleven-rabbitmq exec -it rabbitmq-0 -- cat /app/load_definition.json
- Print the content of the RabbitMQ config file:
kubectl -n syseleven-rabbitmq exec -it rabbitmq-0 -- cat /etc/rabbitmq/rabbitmq.conf
- Verify successful configuration with RabbitMQ command line client:
kubectl -n syseleven-rabbitmq exec -it rabbitmq-0 -- bash
Within the RabbitMQ pod execute these statements: