HCP Vault Secrets Centralized secrets lifecycle management for developers. Learn More
Vault
Database credentials

Database root credential rotation

  • 7min
  • |
  • Vault

Vault's database secrets engine provides a centralized workflow for managing credentials for various database systems. By leveraging this, every service instance gets a unique set of database credentials instead of sharing one. Having those credentials tied directly to each service instance and live only for the life of the service, any abnormal access pattern can be mapped to a specific service instance and its credential can be revoked immediately.

This reduces the manual tasks performed by the database administrator and makes the database access more efficient and secure.

In the dynamic secrets tutorial, you configured Vault to generate dynamic credentials for a PostgreSQL database. In this tutorial, you will learn how to rotate credentials used by Vault, known as the root user.

If you are not familar with how to configure Vault for dynamic credentials, follow the database secrets engine tutorial before you begin.

Scenario

Because Vault is managing the database credentials on behalf of the database administrator, it must also be given a set of highly privileged credentials which can grant and revoke access to the database system.

HashiCups needs to ensure the credentials used by Vault are secure, and do not violate the governance, risk and compliance (GRC) policies surrounding that data stored in the database.

Oliver with the operations team will use Vault's /database/rotate-root/:name API endpoint to rotate the root credentials stored for the database connection. DB Root Credentials

Best Practice

Use this feature to rotate the root credentials immediately after the initial configuration of each database.

Prerequisites

This lab was tested on macOS using an x86_64 based and Apple silicon-based processors. You may also run this tutorial by clicking the Start interactive lab button.

To perform the tasks described in this tutorial, you need to have:

Policy requirements

Note

For the purpose of this tutorial, you can use your root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

To perform all tasks demonstrated in this tutorial, your policy must include the following permissions:

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the database secrets engine and create roles
path "database/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

If you are not familiar with policies, complete the policies tutorial.

Set up the lab

  1. Clone the learn-vault-dynamic-credentials repository.

    $ git clone git@github.com:hashicorp-education/learn-vault-dynamic-credentials.git

  2. Change into the learn-vault-dynamic-credentials directory.

    $ cd learn-vault-dynamic-credentials

  3. Deploy the Vault and PostgreSQL containers.

    $ terraform -chdir=vault-dynamic-creds-docker/ init && \
  terraform -chdir=vault-dynamic-creds-docker/ apply -auto-approve

    Example output:

    Initializing the backend...
Initializing provider plugins...
- Finding kreuzwerker/docker versions matching "3.0.2"...
- Installing kreuzwerker/docker v3.0.2...
...snip...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

POSTGRES_URL = "export TF_VAR_POSTGRES_URL=172.17.0.2:5432"
POSTGRES_URL_HOST = "export 127.0.0.1:5432"
VAULT_ADDR = "export VAULT_ADDR=http://127.0.0.1:8200"
VAULT_TOKEN = "export VAULT_TOKEN=root"

  4. Copy the export command from the Terraform output and export the environment variables.

    Example:

    $ export VAULT_ADDR=http://127.0.0.1:8200 \
   VAULT_TOKEN=root \
   TF_VAR_POSTGRES_URL=172.17.0.2:5432

  5. Verify the PostgreSQL and Vault containers have started.

    $ docker ps -f name=learn --format "table {{.Names}}\t{{.Status}}"
NAMES            STATUS
learn-postgres   Up 4 minutes
learn-vault      Up 4 minutes

    Vault and PostgreSQL are running. Vault connects to PostgreSQL over the Docker bridge network.

  6. Apply the PostgreSQL configuration used in the dynamic secrets tutorial.

    $ terraform -chdir=vault-dynamic-creds-postgres/ init && \
  terraform -chdir=vault-dynamic-creds-postgres/ apply -auto-approve

    Example output:

    Initializing the backend...
Initializing provider plugins...
- Finding cyrilgdn/postgresql versions matching "1.25.0"...
- Installing cyrilgdn/postgresql v1.25.0...
- Installed cyrilgdn/postgresql v1.25.0 (self-signed, key ID 418F268A88A6D481)
...snip...
Plan: 2 to add, 0 to change, 0 to destroy.
postgresql_role.ro: Creating...
postgresql_grant.readonly_tables: Creating...
postgresql_role.ro: Creation complete after 0s [id=ro]
postgresql_grant.readonly_tables: Creation complete after 0s [id=ro_postgres_public_table]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

  7. Apply the Vault configuration used in the dynamic secrets tutorial.

    $ terraform -chdir=vault-dynamic-creds-vault/ init && \
  terraform -chdir=vault-dynamic-creds-vault/ apply -auto-approve

    Example output:

    Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/vault versions matching "4.5.0"...
...snip...
vault_database_secrets_mount.database: Creating...
vault_database_secrets_mount.database: Creation complete after 0s [id=database]
vault_database_secret_backend_role.readonly: Creating...
vault_database_secret_backend_role.readonly: Creation complete after 0s [id=database/roles/readonly]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

    Vault and PostgreSQL are running and configured. You are ready to proceed with the tutorial.

Rotate the root credentials

Vault allows you to rotate the initial root database credentials used when configuring the database secrets engine.

Tip

Once you rotate the root credential, only the Vault knows the new root password. This is the same for all root database credentials given to Vault. A good practice is to create a separate superuser dedicated to Vault which is not used for other purposes.

  1. Rotate the root credentials for the PostgreSQL database connection.

    $ vault write -force database/rotate-root/postgres

  2. Verify that the root credential was rotated by attempting to log into PostgreSQL and enter the password rootpassword.

    Entering the initial password (rootpassword) fails since you rotated the password with the rotate-root command.

    $ psql -h localhost -p 5432 -U root postgres
Password for user root:

    Example output:

    psql -h localhost -p 5432 -U root
Password for user root:
psql: error: connection to server at "localhost" (::1), port 5432 failed: FATAL:  password authentication failed for user "root"

    You can invoke the database/rotate-root/:name endpoint periodically to secure the root credential.

Verify the configuration

Request credentials from the readonly role to verify that the database secrets engine dynamically generates credentials as expected.

  1. Read credentials from the readonly database role.

    $ CREDS=$(vault read -format=json database/creds/readonly) && echo $CREDS

{
  "request_id": "c72c9777-b35a-a8e9-81e2-83c5f2de5e37",
  "lease_id": "database/creds/readonly/7Gj6xNxZBRgQo881rGRB7Ueu",
  "lease_duration": 3600,
  "renewable": true,
  "data": {
    "password": "Ek783iXA-iwzWnewVgON",
    "username": "v-token-readonly-4ibY2gRaYfYaW8qMXAE3-1737564539"
  },
  "warnings": null,
  "mount_type": "database"
}

    The command outputs the results in JSON and stores the output in the CREDS variable.

  2. Verify you can connect to the database using the Vault generated credentials.

    $ psql -h localhost -p 5432 \
       -U $(echo $CREDS | jq -r .data.username) postgres

  3. Enter the password generated by Vault when prompted.

    Example output:

    Password for user v-token-readonly-x7v65y1xuprzxv9vpt80-1525378873:

  4. Verify that you can run commands with the PostgreSQL role provided.

    $ \du
Role name                                        |                         Attributes
-------------------------------------------------+------------------------------------------------------------
ro                                               | No inheritance, Cannot login
                                                 | Password valid until infinity
root                                             | Superuser, Create role, Create DB, Replication, Bypass RLS
v-token-readonly-x6q809467q98yp4yx4z4-1525378026 | Password valid until 2018-05-03 21:07:11+00

  5. Exit the PostgreSQL prompt.

    $ \q

    You successfully rotated the root user credentials used by Vault and used the Vault generated credentials to successfully connected to your PostgreSQL server.

Clean up

  1. Delete the Terraform state files for the PostgreSQL configuration.

    Note

    This step is only required because the root user password was rotated during the tutorial. In a production environment, you would use a dedicated superuser for Vault, and a dedicated user for Terraform.

    $ rm -rf vault-dynamic-creds-postgres/terraform.tfstate*

  2. Destroy the remaining Terraform resources.

    $ terraform -chdir=vault-dynamic-creds-vault/ destroy -auto-approve && \
terraform -chdir=vault-dynamic-creds-docker/ destroy -auto-approve

  3. Unset the environment variables.

    $ unset VAULT_ADDR VAULT_TOKEN TF_VAR_POSTGRES_URL

Summary

In this tutorial, you learned how to manage the privileged root user credentials used by Vault to generate dynamic credentials. After rotating the root credentials, you requested credentials from Vault by reading the readonly to verify that the database secrets engine dynamically generates credentials as expected.

Help and reference

Previous
Next