Securely Manage Secrets with HashiCorp Vault

Image for post
Image for post

By Hitesh Jethva, Alibaba Cloud Community Blog author.

Vault is a free and open-source tool from HashiCorp that can be used for securely storing and accessing secrets. Vault stores and tightly controls access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API. Vault provides an interface to any secret and recording a detailed audit log. You can keep your database credentials, API keys for external services, credentials in the vault. Vault supports multiple storage backends including consul, local disk and cloud storage. Vault enables developers and security professionals to deploy applications in zero-trust environments across public and private data centers.

In this tutorial, we will learn how to manage secretes with HashiCorp Vault on an Alibaba Cloud Elastic Compute Service (ECS) instance with Ubuntu 16.04 installed.

Prerequisites

  • A newly created Alibaba Cloud ECS instance with Ubuntu 16.04 installed.
  • A root password is set up to your instance.

For reference, check out create a new ECS instance and connect to your instance. Next, once you are logged into your Ubuntu 16.04 instance, you’ll need to run the apt-get update -y command to update your base system with the latest available packages.

Install Vault

wget https://releases.hashicorp.com/vault/0.11.4/vault_0.11.4_linux_amd64.zip
wget https://releases.hashicorp.com/vault/0.11.4/vault_0.11.4_SHA256SUMS

Next, check the integrity of the downloaded file with the following command:

grep linux_amd64 vault_*_SHA256SUMS | sha256sum -c -

If everything is fine. You should see the following output:

vault_0.11.4_linux_amd64.zip: OK

Next, extract the downloaded file and copy the extracted binary file to the /usr/local/bin directory so it can accessible from your shell.

unzip vault_0.11.4_linux_amd64.zip
cp vault /usr/local/bin/

Next, you will need to set a Linux capability flag on the binary. You can do this with the following command:

setcap cap_ipc_lock=+ep /usr/local/bin/vault

Configure Vault

useradd -r -d /opt/vault -s /bin/nologin vault

Next, give proper ownership to the /opt/vault directory with the following command:

install -o vault -g vault -m 750 -d /opt/vault

Next, you will need to create a Vault configuration file for storing encrypted secrets in /opt/vault file and listening connections via HTTP.

To do so, create a /etc/vault.hcl file:

nano /etc/vault.hcl

Add the following lines:

backend "file" {
path = "/opt/vault"
}
listener "tcp" {
tls_disable = 1
}

Save and close the file. Then, give proper permissions with the following command:

chown vault:vault /etc/vault.hcl 
chmod 640 /etc/vault.hcl

Next, you will need to add your domain name entry in /etc/hosts file to direct requests to Vault to localhost.

You can do this with the nano /etc/hosts command. Then, add the following line:

127.0.0.1 alibabatest.com

Save and close the file, when you are finished.

Create the Vault System Startup File

nano /etc/systemd/system/vault.service

Add the following lines:

[Unit]
Description=Managing Vault Daemon
After=network.target
ConditionFileNotEmpty=/etc/vault.hcl
[Service]
User=vault
Group=vault
ExecStart=/usr/local/bin/vault server -config=/etc/vault.hcl
ExecReload=/usr/local/bin/kill --signal HUP $MAINPID
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
Capabilities=CAP_IPC_LOCK+ep
SecureBits=keep-caps
NoNewPrivileges=yes
KillSignal=SIGINT
[Install]
WantedBy=multi-user.target

Save and close the file. Then, start Vault service and enable it to start on boot time with the following command:

systemctl start vault
systemctl enable vault

You can check the status of Vault service with the systemctl status vault command. Your output will look like:

● vault.service - a tool for managing secrets
Loaded: loaded (/etc/systemd/system/vault.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2018-11-10 11:56:17 IST; 30s ago
Docs: https://vaultproject.io/docs/
Main PID: 1964 (vault)
CGroup: /system.slice/vault.service
└─1964 /usr/local/bin/vault server -config=/etc/vault.hcl
Nov 10 11:56:21 Node2 vault[1964]: ==> Vault server configuration:
Nov 10 11:56:21 Node2 vault[1964]: Cgo: disabled
Nov 10 11:56:21 Node2 vault[1964]: Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_durati
Nov 10 11:56:21 Node2 vault[1964]: Log Level: (not set)
Nov 10 11:56:21 Node2 vault[1964]: Mlock: supported: true, enabled: true
Nov 10 11:56:21 Node2 vault[1964]: Storage: file
Nov 10 11:56:21 Node2 vault[1964]: Version: Vault v0.11.4
Nov 10 11:56:21 Node2 vault[1964]: Version Sha: 612120e76de651ef669c9af5e77b27a749b0dba3
Nov 10 11:56:21 Node2 vault[1964]: ==> Vault server started! Log data will stream in below:
Nov 10 11:56:21 Node2 vault[1964]: 2018-11-10T11:56:20.401+0530 [WARN] no `api_addr` value specified in config or in VAULT_API_ADDR; falling b

You can also verify the Vault version with the vault –version command. The output will look like this:

Vault v0.11.4 ('612120e76de651ef669c9af5e77b27a749b0dba3')

Initialize Vault

export VAULT_ADDR=http://alibabatest.com:8200

Next, check the vault is in an uninitialized state by running the vault status command. The resulting output will be:

Error checking seal status: Error making API request.URL: GET https://alibabatest.com:8200/v1/sys/seal-status
Code: 400. Errors:
* server is not yet initialized

Next, initialize the Vault using the vault init -key-shares=3 -key-threshold=2 command. The output will look like this:

WARNING! The "vault init" command is deprecated. Please use "vault operator
init" instead. This command will be removed in Vault 0.12.
Unseal Key 1: GxaoGqVbDRlpDbeZJcf23rUeFzo0XprfhqJ2oGgykcwK
Unseal Key 2: mfhRgTXxTb6jMn3hwEuBEsp7TGbnv0U38qnjqJ4I4Cnc
Unseal Key 3: 0YCai3UWTsHLLhUL8vK9qNVPQ5zNEdCdp+MYi4OvDhtN
Initial Root Token: 5L6ArACs6QTJ4xtz9bVrXsFRVault initialized with 3 key shares and a key threshold of 2. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 2 of these keys to unseal it
before it can start servicing requests.
Vault does not store the generated master key. Without at least 2 key to
reconstruct the master key, Vault will remain permanently sealed!
It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.

It is recommended to save each unseal token and the initial root token in a secure place.

Now, vault is initiated but sealed. So you will need to unseal Vault using the newly created unseal tokens. You will need at least two unseal keys in order to make the service become available and ready to use.

You can unseal it by running the following command:

vault operator unseal

Enter your first unseal token and press Enter:

Unseal Key (will be hidden):Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 1
Unseal Nonce: 6dsa453s-1z34-hj87-1g7j-g89976j69f564

The above output indicates that the unsealing is in progress, but still requires one more unsealing key before Vault is ready for use. Now, run the unseal command again:

vault operator unseal

Enter your second unseal token and press Enter:

Unseal Key (will be hidden): 
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 3
Threshold 2
Unseal Progress 1/2
Unseal Nonce ad6c44d0-a32e-e1e3-a80f-e4e72f0a0d4e
Version 0.11.4
HA Enabled false

Vault is now unsealed and ready to use. You can check the status of Vault with the vault status command. Your output will look like this:

Key             Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 3
Threshold 2
Version 0.11.4
Cluster Name vault-cluster-972270a2
Cluster ID aade8afc-d7a6-70d6-bf53-5809b9bf96f5
HA Enabled false

Test Vault

root_token=5L6ArACs6QTJ4xtz9bVrXsFR

Next, write a value to a Vault with the following command:

VAULT_TOKEN=$root_token vault write secret/message value=testing

The output will look like this:

Success! Data written to: secret/message

Next, you will need to create a policy file with the nano policy.hcl command. And also add the following lines:

path "secret/message" {
capabilities = ["read"]
}

Save and close the file. Then, write this policy to Vault with the following command:

VAULT_TOKEN=$root_token vault policy write message-readonly policy.hcl

The output is as follows:

Success! Uploaded policy: message-readonly

Now, create a token with the rights specified in the policy with the following command:

VAULT_TOKEN=$root_token vault token create -policy="message-readonly"

You should see the following output:

Key                  Value
--- -----
token e3EsX0o8rHjl1MR0OmJIei01
token_accessor 8po3dqy4V4goDqkcmLhgrfzm
token_duration 768h
token_renewable true
token_policies ["default" "message-readonly"]
identity_policies []
policies ["default" "message-readonly"]

Next, save the token value from the above output to an environment variable with the following command:

app_token=e3EsX0o8rHjl1MR0OmJIei01

You can now access the data stored in the path secret/message with the following command:

VAULT_TOKEN=$app_token vault read secret/message

The output will look like this:

Key                 Value
--- -----
refresh_interval 768h
value testing

Now, try to listing secrets in Vault with the following command:

VAULT_TOKEN=$app_token vault list secret

You can see that unprivileged token cannot perform other operations:

Error listing secret/: Error making API request.URL: GET http://alibabatest.com:8200/v1/secret?list=true
Code: 403. Errors:
* 1 error occurred:
* permission denied

Original Source

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store