There are many resources (e.g., ) explaining how to use Vault, but none of them goes into the details of setting it up, especially alongside Consul and docker-compose.
I’m not going into the details of Vault and Consul in this blog post, but, for anyone not familiar with the concepts, let’s just say they are open source tools created by Hashicorp for managing secrets, and for simplifying service discovery, respectively.
As a security professional, I often find myself performing assessments of different systems, regardless if they are web/mobile applications, or entire infrastructures. Working in a team, one of the issues we often face is how to share credentials securely among the team members. Credentials managers like KeePass are awesome, but they haven’t been designed for collaboration, and those databases are painful to share and keep up-to-date between all the team members.
That’s where Consul comes handy: ideally we would like to quickly spin up a new instance for every assessment, so to handle password management across the team.
Here is the idea:
we want to spin up a vault server;
which in turn uses consul as a backend storage;
and, since we are lazy (and we don’t want to keep messing with the command line), we want to interface with the vault server with a handy web interface (vault-ui);
all automagically managed by docker-compose.
After a couple of afternoons spent delving into the documentation of the different services, I came up with the following setup:
Let’s start by dissecting the docker-compose file:
First of all, we define a consul service using the consul:latest image provided by Docker Hub. We then expose ports 8500 and 8300. We also specify 2 volumes: config for any configuration file we might need, and /data to provide persistent storage that can survive the container (I specified the local folder ./_data/consul, but you can make it point to a folder of your choosing). Finally, we start the agent in -server (not debug!) mode, specifying the container’s /data folder as the directory where to store the data (this mirrors what we defined in the volumes section).
Second service is the vault server, based on the vault image provided by Docker Hub. We provide some links to the consul service, from which it is dependant, then we expose port 8200. We then have to instruct to use the volumes defined for the consul service. Finally, we start the server passing the configuration stored in the vault.hcl file.
Third service is the webui, based on the jenriquez/vault-ui image. For this service we just expose port 8000 and provide links to the vault server.
Final service is the backup one, based on the Dockerfile defined in the backup/ folder (and shown below). We specify a volume mapped to the local ./_data/backup and provide a link to the consul service.
Now that we have everything ready, let’s start by bootstrapping our setup with docker-compose up:
The 4 services are up and running, but we still need to initialize and unseal our vault. I scripted this in the setup.sh file, which will:
Initialize the vault, and save the root and unseal keys in the keys.txt file
Unseal the vault with the keys provided
Authenticate to the server using the vault’s root token
Enable username/password authentication, and create a user to be used by the webui (in this case: “webui/webui”)
Create an authentication token to be used by the backup service (backup_token)
List the secret backends and add a new backend for our assessment, with a dummy entry server1_ad
After running this script we should have your vault unsealed, a set of credentials (“webui/webui”) that can be used to login in the webui, and an authentication token to be used by the backup service.
Once done, we can use docker-compose down to stop the services, while all our secrets will be stored in the _data/consul folder:
Next time docker-compose is started, we will only have to unseal the vault, with the unseal.sh script:
We could stop here and manage our secrets via the command line, or we could streamline the process a little bit more.
Just open a browser and point it to http://127.0.0.1:8000. You should be presented with a login page. Insert the credentials and you’ll be able to manage your vault through a convenient web interface.
Backup & Cleanup
At the end of the engagement, we might want to backup our secrets, and remove any leftovers file.
The backup service, based on the consul-backup script, will store the backup on the volume we specified in the docker-compose.yml file (_data/backup in this case).
Finally, the clean.sh script can be used to remove any data stored by the scripts or Consul in the _data folder (remember to move any backup file first!)
The setup described in this blog post should be enough to bring anyone up and running with Vault, but it could still be improved.
For example, I have disabled TLS. To re-enable it, just put the server’s certificate in the config folder and uncomment the relevant lines already put in the config\vault.hcl configuration file.
1. Start services: docker-compose up 2. Init vault: ./_scripts/setup.sh 3. When done: docker-compose down
1. Start services: docker-compose up 2. Unseal vault: _scripts/unseal.sh
1. Start services: docker-compose up 2. Run backup: _scripts/backup.sh