In my homelab I have a couple private git repositories containing some sensitive data. I decided to finally clean this up and install a Vault server where I can safely put my secrets, api-keys, credentials,… The Vault is installed on a VM using the official documentation process.

Vault secret operator (VSO)

Hashicorp created an official operator that can be integrated with a kubernetes cluster. This operator makes it really easy to utilize secrets stored on the Vault server.

Initial steps

I started off by creating a new “secret engine” and some secrets in Vault:

# Create secret engine
vault secrets enable -path=homelab-kube kv-v2

# Create secret
vault kv put kvv2/homelab-kube/testing username="web-user" password="secretPass"

I also need a policy that will be used later on:

# Create policy
vault policy write vso-policy - <<EOF
path "homelab-kube/*" {
   capabilities = ["read"]
}
EOF

Next up I need to get some needed credentials from kubernetes and use them to configure the auth method:

# Populate env variables
TOKEN_REVIEW_JWT=$(kubectl get secret vault-auth --output='go-template={{ .data.token }}' | base64 --decode)
KUBE_CA_CERT=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --decode)
KUBE_HOST=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.server}')

# Enable auth method kube
vault auth enable -path=vso kubernetes

# Write config
vault write auth/vso/config \  
      token_reviewer_jwt="$TOKEN_REVIEW_JWT" \  
      kubernetes_host="$KUBE_HOST" \  
      kubernetes_ca_cert="$KUBE_CA_CERT" \  
      disable_issuer_verification=true

Now I just need a role to read the secrets:

# Creating
vault write auth/vso/role/vso-role \
      bound_service_account_names=vault-auth \
      bound_service_account_namespaces=default \
      policies=vso-policy \
      ttl=24h

# Checking
vault list auth/vso/role  
vault read auth/vso/role/vso-role

Installation

Now that the initial steps are ready and the connection between Vault and kubernetes is setup, I can install the operator using helm:

# Install operator
helm repo add hashicorp https://helm.releases.hashicorp.com  
helm install vault-secrets-operator hashicorp/vault-secrets-operator

After that I need some resources in the kubernetes cluster:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: vault-auth
---
apiVersion: v1
kind: Secret
metadata:
  namespace: default
  name: vault-auth
  annotations:
    kubernetes.io/service-account.name: vault-auth
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  namespace: default
  name: role-tokenreview-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
  - kind: ServiceAccount
    name: vault-auth
    namespace: default

This will create a dedicated service account with a secret and the right binding.

And ofcourse it still needs to be applied:

kubectl apply -f resources.yaml

Next up, we will use the newly created CRD’s that we installed using helm:

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
  namespace: default
  name: vault-connection
spec:
  # address to the Vault server.
  address: http://xxx:8200
  skipTLSVerify: true
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
  namespace: default
  name: vault-auth
spec:
  vaultConnectionRef: vault-connection
  method: kubernetes
  mount: vso
  kubernetes:
    role: vso-role
    serviceAccount: vault-auth

Also apply this one:

kubectl apply -f crd.yaml

Verification

To check if this all worked, I can try and receive the secret I created in the beginning using a VaultStaticSecret:

---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  namespace: default
  name: vault-static-secret
spec:
  vaultAuthRef: vault-auth
  mount: homelab-kube
  type: kv-v2
  path: testing
  refreshAfter: 10s
  destination:
    create: true
    name: vso-handled

Now apply this and try to receive the created secret:

# Create secret
kubectl apply -f vss.yaml

# Check secret
kubectl get secret vso-handled -o jsonpath='{.data._raw}' | base64 --decode

{"data":{"password":"secretPass","username":"web-user"}...

Succes! I can now store secrets in my Vault and retrieve them inside kubernetes using the VSO.