Initial public release v1.0.7
- Streamlined README focused on quick start - Complete examples for all major use cases - Decision tree for choosing right pattern - Comprehensive troubleshooting guide
This commit is contained in:
112
INDEX.md
Normal file
112
INDEX.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# ManagedSecret Operator - Documentation Index
|
||||
|
||||
Quick navigation for the ManagedSecret Operator documentation and examples.
|
||||
|
||||
## 📖 Main Documentation
|
||||
|
||||
**[README.md](./README.md)** - Start here!
|
||||
- Installation guide
|
||||
- Quick start
|
||||
- Basic usage examples
|
||||
- Troubleshooting
|
||||
- Best practices
|
||||
|
||||
## 📚 Examples
|
||||
|
||||
**[examples/EXAMPLES-SUMMARY.md](./examples/EXAMPLES-SUMMARY.md)** - Overview of all examples
|
||||
- Use case descriptions
|
||||
- Decision tree for choosing the right pattern
|
||||
- Quick comparison table
|
||||
|
||||
### Individual Examples
|
||||
|
||||
1. **[example-1-no-rotation.yaml](./examples/example-1-no-rotation.yaml)**
|
||||
- Simple API key without rotation
|
||||
- Perfect for: Third-party API keys, static service identifiers
|
||||
|
||||
2. **[example-2-registry-rotation-simple.yaml](./examples/example-2-registry-rotation-simple.yaml)**
|
||||
- Container registry with basic rotation
|
||||
- Perfect for: Docker Registry, Harbor, web servers with htpasswd
|
||||
|
||||
3. **[example-3-postgres-rotation-previous.yaml](./examples/example-3-postgres-rotation-previous.yaml)**
|
||||
- PostgreSQL with zero-downtime rotation
|
||||
- Perfect for: PostgreSQL, any service needing old password to authenticate
|
||||
|
||||
4. **[example-4-minio-rotation-previous.yaml](./examples/example-4-minio-rotation-previous.yaml)**
|
||||
- MinIO object storage with rotation
|
||||
- Perfect for: MinIO, services not auto-reloading credentials
|
||||
|
||||
5. **[example-5-mysql-rotation-previous.yaml](./examples/example-5-mysql-rotation-previous.yaml)**
|
||||
- MySQL with multiple rotation strategies
|
||||
- Perfect for: MySQL, MariaDB, MySQL-compatible databases
|
||||
|
||||
6. **[example-6-comprehensive-multi-service.yaml](./examples/example-6-comprehensive-multi-service.yaml)**
|
||||
- Complete application stack
|
||||
- Perfect for: Understanding how different strategies work together
|
||||
|
||||
## 🎯 Quick Links by Use Case
|
||||
|
||||
### I need to store a secret that never changes
|
||||
→ [Example 1: No Rotation](./examples/example-1-no-rotation.yaml)
|
||||
|
||||
### I need to rotate credentials automatically
|
||||
→ [Example 2: Simple Rotation](./examples/example-2-registry-rotation-simple.yaml)
|
||||
|
||||
### My service needs the old password to change passwords
|
||||
→ [Example 3: PostgreSQL](./examples/example-3-postgres-rotation-previous.yaml)
|
||||
→ [Example 4: MinIO](./examples/example-4-minio-rotation-previous.yaml)
|
||||
→ [Example 5: MySQL](./examples/example-5-mysql-rotation-previous.yaml)
|
||||
|
||||
### I have a complex multi-service application
|
||||
→ [Example 6: Multi-Service](./examples/example-6-comprehensive-multi-service.yaml)
|
||||
|
||||
## 🚀 Getting Started Path
|
||||
|
||||
1. Read [README.md](./README.md) - Installation & Quick Start
|
||||
2. Review [examples/EXAMPLES-SUMMARY.md](./examples/EXAMPLES-SUMMARY.md) - Understand patterns
|
||||
3. Pick the example closest to your use case
|
||||
4. Adapt and deploy!
|
||||
|
||||
## 🔧 Common Tasks
|
||||
|
||||
### Install the operator
|
||||
→ [README.md - Installation Steps](./README.md#installation-steps)
|
||||
|
||||
### Create your first secret
|
||||
→ [README.md - Usage](./README.md#usage)
|
||||
|
||||
### Understand rotation
|
||||
→ [README.md - Secret Rotation](./README.md#secret-rotation)
|
||||
|
||||
### Troubleshoot issues
|
||||
→ [README.md - Troubleshooting](./README.md#troubleshooting)
|
||||
|
||||
### Set up for production
|
||||
→ [README.md - Best Practices](./README.md#best-practices)
|
||||
|
||||
## 📦 Repository Structure
|
||||
|
||||
```
|
||||
managedsecret-operator/
|
||||
├── README.md # Main documentation
|
||||
├── INDEX.md # This file
|
||||
└── examples/
|
||||
├── EXAMPLES-SUMMARY.md # Examples overview
|
||||
├── example-1-no-rotation.yaml
|
||||
├── example-2-registry-rotation-simple.yaml
|
||||
├── example-3-postgres-rotation-previous.yaml
|
||||
├── example-4-minio-rotation-previous.yaml
|
||||
├── example-5-mysql-rotation-previous.yaml
|
||||
└── example-6-comprehensive-multi-service.yaml
|
||||
```
|
||||
|
||||
## 📞 Need Help?
|
||||
|
||||
1. Check [README.md - Troubleshooting](./README.md#troubleshooting)
|
||||
2. Review the examples for similar use cases
|
||||
3. Contact your administrator for registry/Vault setup issues
|
||||
|
||||
---
|
||||
|
||||
**Version:** 1.0.7
|
||||
**Registry:** registry.c5ai.ch/charts/managedsecret-operator
|
||||
711
README-PUBLIC.md
Normal file
711
README-PUBLIC.md
Normal file
@@ -0,0 +1,711 @@
|
||||
# ManagedSecret Operator
|
||||
|
||||
A Kubernetes operator for declarative secret management with automated rotation, drift detection, and zero-downtime password changes.
|
||||
|
||||
## Overview
|
||||
|
||||
The ManagedSecret operator provides:
|
||||
|
||||
- 🔐 **Generates secrets** with configurable password policies
|
||||
- 🏦 **Stores in Vault/OpenBao** as the authoritative source
|
||||
- 🔄 **Syncs to Kubernetes Secrets** for application consumption
|
||||
- 🔍 **Detects and fixes drift** automatically (Vault → K8s)
|
||||
- 🔁 **Rotates secrets** on a configurable schedule
|
||||
- ⏱️ **Preserves previous versions** during rotation for zero-downtime password changes
|
||||
|
||||
### Key Principle: Vault/OpenBao is ALWAYS the source of truth
|
||||
|
||||
- Changes in Vault → Synced to K8s automatically
|
||||
- Changes in K8s → Overwritten by Vault values
|
||||
- Drift detection runs every 1 minute
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
> **Prerequisites:** Contact your administrator to receive:
|
||||
> - Registry username and password for `registry.c5ai.ch`
|
||||
> - Confirmation that OpenBao/Vault is configured for your cluster
|
||||
|
||||
```bash
|
||||
# 1. Set your credentials (provided by administrator)
|
||||
export REGISTRY_USER="<provided-by-admin>"
|
||||
export REGISTRY_PASS="<provided-by-admin>"
|
||||
|
||||
# 2. Install Reloader (for automatic pod restarts during rotation)
|
||||
kubectl apply -f https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml
|
||||
|
||||
# 3. Login to Helm registry
|
||||
helm registry login registry.c5ai.ch -u $REGISTRY_USER -p $REGISTRY_PASS
|
||||
|
||||
# 4. Create imagePullSecret
|
||||
kubectl create namespace managedsecret-operator-system
|
||||
kubectl create secret docker-registry registry-pull-secret \
|
||||
--docker-server=registry.c5ai.ch \
|
||||
--docker-username=$REGISTRY_USER \
|
||||
--docker-password=$REGISTRY_PASS \
|
||||
--namespace managedsecret-operator-system
|
||||
|
||||
# 5. Install operator via Helm
|
||||
helm install managedsecret-operator oci://registry.c5ai.ch/charts/managedsecret-operator \
|
||||
--version 1.0.7 \
|
||||
--namespace managedsecret-operator-system \
|
||||
--set imagePullSecrets[0].name=registry-pull-secret
|
||||
|
||||
# 6. Verify installation
|
||||
kubectl get pods -n managedsecret-operator-system
|
||||
```
|
||||
|
||||
That's it! Now check the [examples](./examples/) folder for comprehensive usage scenarios.
|
||||
|
||||
---
|
||||
|
||||
## 📦 What You Get
|
||||
|
||||
### Helm Chart from registry.c5ai.ch
|
||||
|
||||
The operator is distributed as a Helm chart:
|
||||
|
||||
- ✅ **No building required** - Just `helm install`
|
||||
- ✅ **Versioned releases** - Use specific versions or upgrade easily
|
||||
- ✅ **Pre-configured** - Sensible defaults, customize via `values.yaml`
|
||||
- ✅ **Private & secure** - Hosted on your infrastructure
|
||||
|
||||
### What's Deployed
|
||||
|
||||
```
|
||||
managedsecret-operator-system/
|
||||
├── CustomResourceDefinition (ManagedSecret)
|
||||
├── ServiceAccount (controller-manager)
|
||||
├── Role & RoleBinding (RBAC)
|
||||
├── ClusterRole & ClusterRoleBinding
|
||||
├── Deployment (controller-manager)
|
||||
└── ConfigMap (operator configuration)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### On Your Side (User)
|
||||
- ✅ Kubernetes cluster (1.24+)
|
||||
- ✅ kubectl configured and working
|
||||
- ✅ Helm 3.x installed
|
||||
- ✅ Cluster admin permissions
|
||||
|
||||
### Provided by Administrator
|
||||
- ✅ **Registry credentials** for `registry.c5ai.ch`
|
||||
- ✅ **OpenBao/Vault configuration** for your cluster
|
||||
- Vault address (e.g., `http://openbao.openbao.svc.cluster.local:8200`)
|
||||
- Kubernetes auth configured
|
||||
- Vault role name (typically `managedsecret-operator`)
|
||||
|
||||
### Optional (Recommended)
|
||||
- ✅ **Reloader by Stakater** (for automatic pod restarts on rotation)
|
||||
|
||||
---
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### Step 1: Install Reloader (Recommended)
|
||||
|
||||
Reloader automatically restarts pods when secrets change:
|
||||
|
||||
```bash
|
||||
# Direct install (simplest)
|
||||
kubectl apply -f https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml
|
||||
|
||||
# Via Helm
|
||||
helm repo add stakater https://stakater.github.io/stakater-charts
|
||||
helm install reloader stakater/reloader \
|
||||
--namespace reloader \
|
||||
--create-namespace
|
||||
```
|
||||
|
||||
### Step 2: Authenticate to Registry
|
||||
|
||||
```bash
|
||||
# Set credentials from administrator
|
||||
export REGISTRY_USER="<your-username>"
|
||||
export REGISTRY_PASS="<your-password>"
|
||||
|
||||
# Login to Helm registry
|
||||
helm registry login registry.c5ai.ch -u $REGISTRY_USER -p $REGISTRY_PASS
|
||||
|
||||
# Create namespace
|
||||
kubectl create namespace managedsecret-operator-system
|
||||
|
||||
# Create imagePullSecret
|
||||
kubectl create secret docker-registry registry-pull-secret \
|
||||
--docker-server=registry.c5ai.ch \
|
||||
--docker-username=$REGISTRY_USER \
|
||||
--docker-password=$REGISTRY_PASS \
|
||||
--namespace managedsecret-operator-system
|
||||
```
|
||||
|
||||
### Step 3: Install the Operator
|
||||
|
||||
```bash
|
||||
# Install with default configuration
|
||||
helm install managedsecret-operator \
|
||||
oci://registry.c5ai.ch/charts/managedsecret-operator \
|
||||
--version 1.0.7 \
|
||||
--namespace managedsecret-operator-system \
|
||||
--set imagePullSecrets[0].name=registry-pull-secret
|
||||
|
||||
# Or with custom values
|
||||
cat > values.yaml <<EOF
|
||||
imagePullSecrets:
|
||||
- name: registry-pull-secret
|
||||
|
||||
replicaCount: 1
|
||||
|
||||
resources:
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
EOF
|
||||
|
||||
helm install managedsecret-operator \
|
||||
oci://registry.c5ai.ch/charts/managedsecret-operator \
|
||||
--version 1.0.7 \
|
||||
--namespace managedsecret-operator-system \
|
||||
-f values.yaml
|
||||
```
|
||||
|
||||
### Step 4: Verify Installation
|
||||
|
||||
```bash
|
||||
# Check operator is running
|
||||
kubectl get pods -n managedsecret-operator-system
|
||||
|
||||
# Check CRD is installed
|
||||
kubectl get crd managedsecrets.secrets.c5ai.ch
|
||||
|
||||
# View operator logs
|
||||
kubectl logs -n managedsecret-operator-system \
|
||||
-l control-plane=controller-manager --tail=50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📖 Usage
|
||||
|
||||
### Basic Example: Simple API Key (No Rotation)
|
||||
|
||||
```yaml
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: api-keys
|
||||
namespace: my-app
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: my-app/api-keys
|
||||
|
||||
fields:
|
||||
- name: api-key
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 64
|
||||
minDigits: 10
|
||||
symbolCharacters: "" # Alphanumeric only
|
||||
|
||||
- name: api-endpoint
|
||||
type: static
|
||||
value: "https://api.example.com/v1"
|
||||
|
||||
destination:
|
||||
name: api-secret
|
||||
type: Opaque
|
||||
|
||||
rotation:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
Apply it:
|
||||
```bash
|
||||
kubectl apply -f managedsecret.yaml
|
||||
```
|
||||
|
||||
The operator will:
|
||||
1. Generate a 64-character API key
|
||||
2. Store it in Vault at `secret/data/my-app/api-keys`
|
||||
3. Create Kubernetes Secret `api-secret` in namespace `my-app`
|
||||
|
||||
Use it in your pod:
|
||||
```yaml
|
||||
env:
|
||||
- name: API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-secret
|
||||
key: api-key
|
||||
```
|
||||
|
||||
### Rotation Example: PostgreSQL with Previous Version
|
||||
|
||||
For services that need the old password to authenticate before changing to the new password:
|
||||
|
||||
```yaml
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: postgres-credentials
|
||||
namespace: database
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: database/postgres
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "app_user"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 40
|
||||
minDigits: 8
|
||||
minSymbols: 8
|
||||
|
||||
destination:
|
||||
name: postgres-secret
|
||||
type: Opaque
|
||||
keepPreviousVersion: true # Creates postgres-secret-previous
|
||||
previousVersionTTL: 1h # Cleanup after 1 hour
|
||||
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 2160h # 90 days
|
||||
rotateGeneratedOnly: true
|
||||
```
|
||||
|
||||
When rotation happens:
|
||||
1. New password is generated
|
||||
2. `postgres-secret` is updated with new password
|
||||
3. `postgres-secret-previous` is created with old password
|
||||
4. Your CronJob uses old password to authenticate and set new password
|
||||
5. After 1 hour, `-previous` secret is automatically cleaned up
|
||||
|
||||
**Important:** You'll need a CronJob to handle the password change. See [examples/example-3-postgres-rotation-previous.yaml](./examples/example-3-postgres-rotation-previous.yaml) for a complete implementation.
|
||||
|
||||
---
|
||||
|
||||
## 📚 Examples
|
||||
|
||||
The `examples/` folder contains comprehensive scenarios:
|
||||
|
||||
| Example | Use Case | Description |
|
||||
|---------|----------|-------------|
|
||||
| **[Example 1](./examples/example-1-no-rotation.yaml)** | API Keys | Long-lived credentials without rotation |
|
||||
| **[Example 2](./examples/example-2-registry-rotation-simple.yaml)** | Container Registry | Simple rotation with Reloader |
|
||||
| **[Example 3](./examples/example-3-postgres-rotation-previous.yaml)** | PostgreSQL | Rotation with previous version (needs old password) |
|
||||
| **[Example 4](./examples/example-4-minio-rotation-previous.yaml)** | MinIO | Object storage with admin credentials rotation |
|
||||
| **[Example 5](./examples/example-5-mysql-rotation-previous.yaml)** | MySQL | Database rotation with multiple strategies |
|
||||
| **[Example 6](./examples/example-6-comprehensive-multi-service.yaml)** | Full Stack | Complete application with mixed strategies |
|
||||
|
||||
**See [examples/EXAMPLES-SUMMARY.md](./examples/EXAMPLES-SUMMARY.md) for detailed explanations and decision trees.**
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Secret Rotation
|
||||
|
||||
### How Rotation Works
|
||||
|
||||
```
|
||||
Day 0: Initial setup
|
||||
- ManagedSecret creates secret with generated password
|
||||
- Stored in Vault and synced to K8s
|
||||
|
||||
Day 90: Rotation triggered (schedule: 2160h)
|
||||
1. Operator generates new password
|
||||
2. Updates Vault
|
||||
3. If keepPreviousVersion=true:
|
||||
- Creates <secret-name>-previous with old values
|
||||
4. Updates K8s Secret with new values
|
||||
5. If Reloader is enabled:
|
||||
- Reloader detects change → Restarts pods
|
||||
6. After previousVersionTTL:
|
||||
- Operator deletes -previous secret
|
||||
```
|
||||
|
||||
### When to Use `keepPreviousVersion`
|
||||
|
||||
Use `keepPreviousVersion: true` when your service needs the **old password to authenticate** before changing to the new password:
|
||||
|
||||
- ✅ PostgreSQL, MySQL, MariaDB (`ALTER USER` requires authentication)
|
||||
- ✅ MinIO (`mc admin` needs old credentials)
|
||||
- ✅ LDAP, Active Directory
|
||||
- ❌ Container registries (htpasswd regenerated fresh)
|
||||
- ❌ Redis (CONFIG SET doesn't need old password)
|
||||
- ❌ Web services with startup scripts
|
||||
|
||||
### Forcing Rotation
|
||||
|
||||
```bash
|
||||
# Trigger immediate rotation
|
||||
kubectl annotate managedsecret <name> -n <namespace> \
|
||||
reconcile="$(date +%s)" --overwrite
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Drift Detection
|
||||
|
||||
The operator automatically detects and fixes drift every 60 seconds:
|
||||
|
||||
**Scenario 1: Secret deleted in Kubernetes**
|
||||
```bash
|
||||
kubectl delete secret my-secret -n my-app
|
||||
# Operator recreates it from Vault within 60 seconds
|
||||
```
|
||||
|
||||
**Scenario 2: Secret modified in Vault**
|
||||
```bash
|
||||
# Update value in Vault
|
||||
vault kv put secret/my-app/keys password=newvalue
|
||||
# Operator syncs to K8s within 60 seconds
|
||||
```
|
||||
|
||||
**Scenario 3: Secret manually edited in K8s**
|
||||
```bash
|
||||
kubectl edit secret my-secret -n my-app
|
||||
# Changes are overwritten by Vault values within 60 seconds
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Operator Not Starting
|
||||
|
||||
```bash
|
||||
# Check pod status
|
||||
kubectl get pods -n managedsecret-operator-system
|
||||
|
||||
# View logs
|
||||
kubectl logs -n managedsecret-operator-system \
|
||||
-l control-plane=controller-manager --tail=100
|
||||
|
||||
# Common issues:
|
||||
# - ImagePullBackOff: Check imagePullSecret is created correctly
|
||||
# - CrashLoopBackOff: Check Vault connectivity
|
||||
```
|
||||
|
||||
### Secret Not Created
|
||||
|
||||
```bash
|
||||
# Check ManagedSecret status
|
||||
kubectl get managedsecret <name> -n <namespace> -o yaml
|
||||
|
||||
# Look for conditions/events
|
||||
kubectl describe managedsecret <name> -n <namespace>
|
||||
|
||||
# Check operator logs
|
||||
kubectl logs -n managedsecret-operator-system \
|
||||
-l control-plane=controller-manager | grep <name>
|
||||
```
|
||||
|
||||
### Vault Authentication Failing
|
||||
|
||||
```bash
|
||||
# Verify ServiceAccount exists
|
||||
kubectl get sa controller-manager -n managedsecret-operator-system
|
||||
|
||||
# Check Vault role configuration
|
||||
vault read auth/kubernetes/role/managedsecret-operator
|
||||
|
||||
# Test authentication manually
|
||||
kubectl exec -it -n managedsecret-operator-system \
|
||||
deployment/managedsecret-operator-controller-manager -- sh
|
||||
# Inside pod: Check /var/run/secrets/kubernetes.io/serviceaccount/token exists
|
||||
```
|
||||
|
||||
### Rotation Not Working
|
||||
|
||||
```bash
|
||||
# Check ManagedSecret schedule
|
||||
kubectl get managedsecret <name> -n <namespace> -o yaml | grep schedule
|
||||
|
||||
# Force rotation to test
|
||||
kubectl annotate managedsecret <name> -n <namespace> \
|
||||
reconcile="$(date +%s)" --overwrite
|
||||
|
||||
# Check if Reloader is installed (if using rotation)
|
||||
kubectl get pods -n reloader
|
||||
```
|
||||
|
||||
### Reloader Not Restarting Pods
|
||||
|
||||
```bash
|
||||
# Verify Reloader is running
|
||||
kubectl get pods -n reloader
|
||||
|
||||
# Check pod has correct annotation
|
||||
kubectl get deployment <name> -n <namespace> -o yaml | grep reloader
|
||||
|
||||
# View Reloader logs
|
||||
kubectl logs -n reloader -l app=reloader
|
||||
|
||||
# Required annotation on pod template:
|
||||
# annotations:
|
||||
# reloader.stakater.com/auto: "true"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Best Practices
|
||||
|
||||
### 1. Always Use Reloader for Rotating Secrets
|
||||
|
||||
Without Reloader, pods won't pick up new credentials:
|
||||
```yaml
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
reloader.stakater.com/auto: "true"
|
||||
```
|
||||
|
||||
### 2. Set Appropriate Rotation Schedules
|
||||
|
||||
- **High Security** (passwords, admin credentials): 30-90 days
|
||||
- **Medium Security** (application credentials): 90-180 days
|
||||
- **Low Security** (read-only API keys): 180+ days or disabled
|
||||
|
||||
### 3. Use `keepPreviousVersion` Correctly
|
||||
|
||||
Only use it when the service needs the old password to authenticate:
|
||||
```yaml
|
||||
destination:
|
||||
keepPreviousVersion: true # Only if old password is needed
|
||||
previousVersionTTL: 1h # Adjust based on your workflow
|
||||
```
|
||||
|
||||
### 4. Test Rotation Before Production
|
||||
|
||||
```bash
|
||||
# Create test ManagedSecret with short rotation
|
||||
spec:
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 5m # Test rotation every 5 minutes
|
||||
|
||||
# Watch the rotation
|
||||
kubectl get secrets -n <namespace> -w
|
||||
|
||||
# Verify application handles rotation
|
||||
kubectl logs -n <namespace> <pod-name>
|
||||
```
|
||||
|
||||
### 5. Monitor Rotation Jobs
|
||||
|
||||
For services using `keepPreviousVersion`, monitor your CronJobs:
|
||||
```bash
|
||||
# Check job status
|
||||
kubectl get jobs -n <namespace>
|
||||
|
||||
# View job logs
|
||||
kubectl logs -n <namespace> -l job-name=<rotation-job>
|
||||
|
||||
# Set up alerts for failed jobs
|
||||
```
|
||||
|
||||
### 6. Backup Vault Regularly
|
||||
|
||||
Vault is your source of truth:
|
||||
```bash
|
||||
# Example: Backup with Velero
|
||||
velero backup create vault-backup \
|
||||
--include-namespaces openbao \
|
||||
--snapshot-volumes
|
||||
```
|
||||
|
||||
### 7. Version Control Your ManagedSecrets
|
||||
|
||||
Store ManagedSecret manifests in Git:
|
||||
```yaml
|
||||
# managedsecrets/
|
||||
# ├── production/
|
||||
# │ ├── postgres-credentials.yaml
|
||||
# │ └── api-keys.yaml
|
||||
# └── staging/
|
||||
# ├── postgres-credentials.yaml
|
||||
# └── api-keys.yaml
|
||||
```
|
||||
|
||||
### 8. Use GitOps for Deployment
|
||||
|
||||
Deploy with ArgoCD or Flux:
|
||||
```yaml
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: managedsecrets
|
||||
spec:
|
||||
source:
|
||||
path: managedsecrets/production
|
||||
destination:
|
||||
namespace: default
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Considerations
|
||||
|
||||
### Production Checklist
|
||||
|
||||
- [ ] OpenBao/Vault is properly secured (TLS, audit logging, backups)
|
||||
- [ ] RBAC configured with least-privilege
|
||||
- [ ] Network policies in place
|
||||
- [ ] Monitoring and alerting configured
|
||||
- [ ] Disaster recovery plan documented
|
||||
- [ ] Registry credentials stored securely (not in Git)
|
||||
|
||||
### Credentials Management
|
||||
|
||||
The operator needs:
|
||||
1. Access to Vault (via Kubernetes ServiceAccount auth)
|
||||
2. Access to pull its image (via imagePullSecret)
|
||||
|
||||
Both use Kubernetes-native methods. **Never hardcode credentials in manifests!**
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support
|
||||
|
||||
### Getting Help
|
||||
|
||||
1. **Check documentation**:
|
||||
- This README
|
||||
- [examples/EXAMPLES-SUMMARY.md](./examples/EXAMPLES-SUMMARY.md)
|
||||
- Individual example files
|
||||
|
||||
2. **Check operator logs**:
|
||||
```bash
|
||||
kubectl logs -n managedsecret-operator-system \
|
||||
-l control-plane=controller-manager --tail=100
|
||||
```
|
||||
|
||||
3. **Contact your administrator** if:
|
||||
- You need registry credentials
|
||||
- Vault is not configured for your cluster
|
||||
- Operator authentication issues
|
||||
|
||||
### For Administrators
|
||||
|
||||
#### Information to Provide Users
|
||||
|
||||
**Registry Credentials:**
|
||||
```
|
||||
Registry: registry.c5ai.ch
|
||||
Username: <username>
|
||||
Password: <password>
|
||||
Chart: oci://registry.c5ai.ch/charts/managedsecret-operator
|
||||
Version: 1.0.7
|
||||
```
|
||||
|
||||
**Vault Configuration:**
|
||||
```
|
||||
Address: http://openbao.openbao.svc.cluster.local:8200
|
||||
Auth Method: kubernetes
|
||||
Role: managedsecret-operator
|
||||
KV Mount: secret
|
||||
KV Version: v2
|
||||
```
|
||||
|
||||
#### Prerequisites for New Cluster
|
||||
|
||||
```bash
|
||||
# 1. Enable Kubernetes auth
|
||||
bao auth enable kubernetes
|
||||
bao write auth/kubernetes/config \
|
||||
kubernetes_host="https://kubernetes.default.svc:443"
|
||||
|
||||
# 2. Create policy
|
||||
bao policy write managedsecret-operator - <<'EOF'
|
||||
path "secret/data/*" {
|
||||
capabilities = ["create", "read", "update", "delete", "list"]
|
||||
}
|
||||
path "secret/metadata/*" {
|
||||
capabilities = ["list", "read", "delete"]
|
||||
}
|
||||
EOF
|
||||
|
||||
# 3. Create role
|
||||
bao write auth/kubernetes/role/managedsecret-operator \
|
||||
bound_service_account_names=controller-manager \
|
||||
bound_service_account_namespaces=managedsecret-operator-system \
|
||||
policies=managedsecret-operator \
|
||||
ttl=1h
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Upgrading
|
||||
|
||||
### Upgrade Operator Version
|
||||
|
||||
```bash
|
||||
# Check current version
|
||||
helm list -n managedsecret-operator-system
|
||||
|
||||
# Upgrade to new version
|
||||
helm upgrade managedsecret-operator \
|
||||
oci://registry.c5ai.ch/charts/managedsecret-operator \
|
||||
--version 1.0.8 \
|
||||
--namespace managedsecret-operator-system \
|
||||
--reuse-values
|
||||
```
|
||||
|
||||
### Migration Notes
|
||||
|
||||
When upgrading from earlier versions:
|
||||
- CRD updates are handled automatically
|
||||
- Existing ManagedSecrets continue to work
|
||||
- No manual migration required
|
||||
- Check changelog for breaking changes
|
||||
|
||||
---
|
||||
|
||||
## 📝 Changelog
|
||||
|
||||
### Version 1.0.7
|
||||
- ✨ Added `keepPreviousVersion` for zero-downtime rotation
|
||||
- ✨ Added `previousVersionTTL` for automatic cleanup
|
||||
- 📦 Helm chart deployment support
|
||||
- 📚 Comprehensive examples and documentation
|
||||
- 🐛 Bug fixes and improvements
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Proprietary Use License
|
||||
|
||||
Permission is granted to use this software for any purpose, including commercial use, without charge.
|
||||
|
||||
Modification, adaptation, or creation of derivative works is prohibited.
|
||||
Redistribution of modified versions is prohibited.
|
||||
|
||||
---
|
||||
|
||||
**Version:** 1.0.7
|
||||
**Registry:** registry.c5ai.ch/charts/managedsecret-operator
|
||||
**Minimum Kubernetes:** 1.24+
|
||||
**Compatible with:** OpenBao, HashiCorp Vault
|
||||
316
examples/EXAMPLES-SUMMARY.md
Normal file
316
examples/EXAMPLES-SUMMARY.md
Normal file
@@ -0,0 +1,316 @@
|
||||
# ManagedSecret Operator - Examples Summary
|
||||
|
||||
This directory contains comprehensive examples for the ManagedSecret Operator, demonstrating various rotation strategies and use cases.
|
||||
|
||||
## Files Overview
|
||||
|
||||
### Documentation
|
||||
- **README.md** - Complete installation and usage guide with new features
|
||||
- Added `keepPreviousVersion` and `previousVersionTTL` documentation
|
||||
- Added Reloader integration guide
|
||||
- Updated best practices and architecture diagrams
|
||||
|
||||
### Example Files
|
||||
|
||||
#### Example 1: No Rotation (Simple API Key)
|
||||
**File:** `example-1-no-rotation.yaml`
|
||||
|
||||
**Use Case:** Long-lived API keys that don't need rotation
|
||||
|
||||
**Key Features:**
|
||||
- Generated once and kept indefinitely
|
||||
- No Reloader needed (secret never changes)
|
||||
- Perfect for external service API keys
|
||||
|
||||
**When to Use:**
|
||||
- Third-party API keys (Stripe, SendGrid, etc.)
|
||||
- Service identifiers
|
||||
- Static configuration values
|
||||
|
||||
---
|
||||
|
||||
#### Example 2: Simple Rotation with Reloader (Container Registry)
|
||||
**File:** `example-2-registry-rotation-simple.yaml`
|
||||
|
||||
**Use Case:** Docker/Container registry with HTTP basic auth
|
||||
|
||||
**Key Features:**
|
||||
- Password rotates every 90 days
|
||||
- InitContainer regenerates htpasswd on pod restart
|
||||
- Reloader automatically restarts pods
|
||||
- **No `keepPreviousVersion` needed** - htpasswd is regenerated fresh
|
||||
|
||||
**Rotation Flow:**
|
||||
1. ManagedSecret generates new password → Updates Vault → Updates K8s Secret
|
||||
2. Reloader detects Secret change → Triggers pod restart
|
||||
3. InitContainer runs → Reads new password → Generates fresh htpasswd
|
||||
4. Registry starts with new htpasswd
|
||||
|
||||
**When to Use:**
|
||||
- Container registries (Docker Registry, Harbor)
|
||||
- Web servers with htpasswd (Apache, Nginx)
|
||||
- Any service where credentials are regenerated at startup
|
||||
|
||||
---
|
||||
|
||||
#### Example 3: Complex Rotation with Previous Password (PostgreSQL)
|
||||
**File:** `example-3-postgres-rotation-previous.yaml`
|
||||
|
||||
**Use Case:** PostgreSQL user password that requires authentication to change
|
||||
|
||||
**Key Features:**
|
||||
- Password rotates every 90 days
|
||||
- `keepPreviousVersion: true` creates `-previous` secret
|
||||
- CronJob uses old password to authenticate, sets new password
|
||||
- `previousVersionTTL: 1h` cleans up after grace period
|
||||
|
||||
**Rotation Flow:**
|
||||
1. ManagedSecret creates both `postgres-secret` and `postgres-secret-previous`
|
||||
2. CronJob (every 5 min) detects rotation
|
||||
3. Authenticates with OLD password: `PGPASSWORD=$OLD_PASSWORD psql`
|
||||
4. Changes password: `ALTER USER app_user PASSWORD '$NEW_PASSWORD'`
|
||||
5. Verifies new password works
|
||||
6. After 1 hour, `-previous` secret is deleted
|
||||
|
||||
**When to Use:**
|
||||
- PostgreSQL, MySQL, MariaDB
|
||||
- Any database requiring authentication to change passwords
|
||||
- Services with `ALTER USER` or similar commands
|
||||
|
||||
---
|
||||
|
||||
#### Example 4: MinIO Admin Password Rotation
|
||||
**File:** `example-4-minio-rotation-previous.yaml`
|
||||
|
||||
**Use Case:** MinIO admin credentials with rotation
|
||||
|
||||
**Key Features:**
|
||||
- Password rotates every 60 days
|
||||
- Uses MinIO Client (mc) to change password
|
||||
- `keepPreviousVersion: true` with 2h TTL
|
||||
- CronJob handles rotation via `mc admin user password`
|
||||
|
||||
**Special Considerations:**
|
||||
- MinIO does NOT reload credentials from environment variables
|
||||
- Manual pod restart required after rotation
|
||||
- Consider using a sidecar to monitor and restart
|
||||
|
||||
**Rotation Flow:**
|
||||
1. ManagedSecret creates both secrets
|
||||
2. CronJob authenticates with old credentials: `mc alias set oldminio`
|
||||
3. Changes password: `mc admin user password myminio admin $NEW_PASSWORD`
|
||||
4. Verifies new credentials
|
||||
5. **Important:** MinIO pods must be restarted manually
|
||||
|
||||
**When to Use:**
|
||||
- MinIO object storage
|
||||
- Any service that doesn't auto-reload credentials
|
||||
- Services requiring separate admin tools for password changes
|
||||
|
||||
---
|
||||
|
||||
#### Example 5: MySQL User Password Rotation
|
||||
**File:** `example-5-mysql-rotation-previous.yaml`
|
||||
|
||||
**Use Case:** MySQL application user with automated rotation
|
||||
|
||||
**Key Features:**
|
||||
- Password rotates every 90 days
|
||||
- Multiple password change methods (ALTER USER, SET PASSWORD)
|
||||
- Root credentials as fallback
|
||||
- `keepPreviousVersion: true` with 1h TTL
|
||||
|
||||
**Rotation Flow:**
|
||||
1. ManagedSecret creates both secrets
|
||||
2. CronJob tries multiple approaches:
|
||||
- Method 1: `ALTER USER` as root (preferred)
|
||||
- Method 2: `SET PASSWORD` as root
|
||||
- Method 3: `SET PASSWORD` as user with old password
|
||||
3. Flushes privileges
|
||||
4. Verifies new password
|
||||
|
||||
**When to Use:**
|
||||
- MySQL 5.7+, MySQL 8.0
|
||||
- MariaDB
|
||||
- Any MySQL-compatible database
|
||||
|
||||
---
|
||||
|
||||
#### Example 6: Comprehensive Multi-Service Deployment
|
||||
**File:** `example-6-comprehensive-multi-service.yaml`
|
||||
|
||||
**Use Case:** Complete application stack showing all strategies together
|
||||
|
||||
**Components:**
|
||||
1. **PostgreSQL** - Rotates with previous version (90 days)
|
||||
2. **Redis** - Rotates without previous version (30 days)
|
||||
3. **API Keys** - No rotation (managed externally)
|
||||
4. **TLS Certificates** - No rotation (cert-manager)
|
||||
5. **JWT Secret** - Simple rotation with Reloader (180 days)
|
||||
|
||||
**Demonstrates:**
|
||||
- Multiple secrets with different rotation schedules
|
||||
- Mixed rotation strategies in one application
|
||||
- How Reloader restarts pods for all credential changes
|
||||
- Real-world production setup
|
||||
|
||||
**When to Use:**
|
||||
- As a template for complex applications
|
||||
- To understand how different strategies work together
|
||||
- For learning all features in one example
|
||||
|
||||
---
|
||||
|
||||
## Quick Decision Tree
|
||||
|
||||
### Does your service need the OLD password to set the NEW password?
|
||||
|
||||
**YES** (PostgreSQL, MySQL, MinIO, LDAP):
|
||||
- Use `keepPreviousVersion: true`
|
||||
- Set appropriate `previousVersionTTL`
|
||||
- Create CronJob to handle rotation
|
||||
- See Examples 3, 4, or 5
|
||||
|
||||
**NO** (Registry, Redis, Web servers):
|
||||
- Use simple rotation with Reloader
|
||||
- InitContainer or startup script regenerates credentials
|
||||
- No CronJob needed
|
||||
- See Example 2
|
||||
|
||||
**NOT ROTATING** (API keys, Certificates):
|
||||
- Set `rotation.enabled: false`
|
||||
- No Reloader annotation needed
|
||||
- See Example 1
|
||||
|
||||
---
|
||||
|
||||
## Rotation Schedule Recommendations
|
||||
|
||||
- **High Security** (User passwords, admin credentials): 30-90 days
|
||||
- **Medium Security** (Application credentials, service accounts): 90-180 days
|
||||
- **Low Security** (Read-only API keys, development): 180+ days or disabled
|
||||
- **Previous Version TTL**:
|
||||
- Automated rotation: 1h
|
||||
- Manual intervention: 2-4h
|
||||
- Complex workflows: 24h
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites for All Examples
|
||||
|
||||
1. **OpenBao/Vault** deployed and initialized
|
||||
2. **ManagedSecret Operator** installed
|
||||
3. **Reloader** installed (for rotation examples)
|
||||
4. **cert-manager** (optional, for TLS examples)
|
||||
|
||||
## Installation Order
|
||||
|
||||
```bash
|
||||
# 1. Install OpenBao
|
||||
kubectl create namespace openbao
|
||||
helm install openbao openbao/openbao -n openbao
|
||||
|
||||
# 2. Install ManagedSecret Operator
|
||||
kubectl apply -f managedsecret-operator/
|
||||
|
||||
# 3. Install Reloader
|
||||
kubectl apply -f https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml
|
||||
|
||||
# 4. Deploy your application with examples
|
||||
kubectl apply -f example-X-your-usecase.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Rotation
|
||||
|
||||
### Force Immediate Rotation
|
||||
```bash
|
||||
# Trigger rotation for any ManagedSecret
|
||||
kubectl annotate managedsecret <name> -n <namespace> \
|
||||
reconcile="$(date +%s)" --overwrite
|
||||
```
|
||||
|
||||
### Watch the Flow
|
||||
```bash
|
||||
# Terminal 1: Watch secrets
|
||||
kubectl get secrets -n <namespace> -w
|
||||
|
||||
# Terminal 2: Watch pods
|
||||
kubectl get pods -n <namespace> -w
|
||||
|
||||
# Terminal 3: Watch jobs (for CronJobs)
|
||||
kubectl get jobs -n <namespace> -w
|
||||
|
||||
# Terminal 4: Reloader logs
|
||||
kubectl logs -n reloader -l app=reloader -f
|
||||
```
|
||||
|
||||
### Verify Rotation Completed
|
||||
```bash
|
||||
# Check if -previous secret exists (should only exist during grace period)
|
||||
kubectl get secret <name>-previous -n <namespace>
|
||||
|
||||
# View rotation job logs
|
||||
kubectl logs -n <namespace> -l job-name=<rotation-job>
|
||||
|
||||
# Test connection with new credentials
|
||||
kubectl exec -it -n <namespace> <pod> -- <test-command>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### Issue: Pod not restarting after rotation
|
||||
**Solution:** Check Reloader is installed and annotation is correct
|
||||
```bash
|
||||
kubectl get pods -n reloader
|
||||
kubectl get deployment <app> -o yaml | grep reloader
|
||||
```
|
||||
|
||||
### Issue: Rotation job failing
|
||||
**Solution:** Check if previous secret exists and credentials work
|
||||
```bash
|
||||
kubectl get secret <name>-previous -n <namespace>
|
||||
kubectl logs -n <namespace> -l job-name=<rotation-job>
|
||||
```
|
||||
|
||||
### Issue: Application still using old credentials
|
||||
**Solution:** Verify pod actually restarted after secret change
|
||||
```bash
|
||||
kubectl get pods -n <namespace> -o wide
|
||||
# Check pod start time vs secret update time
|
||||
```
|
||||
|
||||
### Issue: Previous secret not cleaned up
|
||||
**Solution:** Check operator logs for cleanup job
|
||||
```bash
|
||||
kubectl logs -n managedsecret-operator-system -l control-plane=controller-manager | grep cleanup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices Summary
|
||||
|
||||
1. ✅ **Always use Reloader** with rotating secrets
|
||||
2. ✅ **Use `keepPreviousVersion`** for services requiring old password
|
||||
3. ✅ **Set realistic `previousVersionTTL`** based on your workflow
|
||||
4. ✅ **Test rotation** before production deployment
|
||||
5. ✅ **Monitor rotation jobs** for failures
|
||||
6. ✅ **Document rotation procedures** for your team
|
||||
7. ✅ **Use appropriate rotation schedules** based on security requirements
|
||||
8. ✅ **Backup Vault regularly** - it's your source of truth
|
||||
9. ✅ **Never modify K8s Secrets manually** - always use ManagedSecret or Vault
|
||||
10. ✅ **Set up alerts** for failed rotations
|
||||
|
||||
---
|
||||
|
||||
## Support and Contributing
|
||||
|
||||
For issues or questions about these examples:
|
||||
- Check the main README.md for troubleshooting
|
||||
- Review operator logs for detailed error messages
|
||||
- Test in development environment before production
|
||||
|
||||
Remember: Vault/OpenBao is ALWAYS the source of truth. All changes should go through ManagedSecret resources or directly in Vault, never by manually editing Kubernetes Secrets.
|
||||
122
examples/example-1-no-rotation.yaml
Normal file
122
examples/example-1-no-rotation.yaml
Normal file
@@ -0,0 +1,122 @@
|
||||
# Example 1: Simple API Key (No Rotation)
|
||||
#
|
||||
# Use case: Long-lived API key that doesn't need rotation
|
||||
#
|
||||
# Characteristics:
|
||||
# - Generated once
|
||||
# - No rotation schedule
|
||||
# - No Reloader needed (static secret)
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: api-service
|
||||
|
||||
---
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: api-keys
|
||||
namespace: api-service
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: api/service/keys
|
||||
|
||||
fields:
|
||||
# Static service identifier
|
||||
- name: service-id
|
||||
type: static
|
||||
value: "api-service-prod"
|
||||
|
||||
# Generated API key (long, alphanumeric)
|
||||
- name: api-key
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 64
|
||||
minDigits: 10
|
||||
minSymbols: 0
|
||||
minLowercase: 20
|
||||
minUppercase: 20
|
||||
symbolCharacters: "" # No symbols, just alphanumeric
|
||||
allowRepeat: false
|
||||
|
||||
# Static API endpoint
|
||||
- name: api-endpoint
|
||||
type: static
|
||||
value: "https://api.external-service.com/v1"
|
||||
|
||||
destination:
|
||||
name: api-secret
|
||||
type: Opaque
|
||||
|
||||
# No rotation - generate once and keep
|
||||
rotation:
|
||||
enabled: false
|
||||
|
||||
---
|
||||
# Example Deployment using the API key
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: api-consumer
|
||||
namespace: api-service
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: api-consumer
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: api-consumer
|
||||
# No Reloader annotation needed - secret doesn't change
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: your-app:latest
|
||||
env:
|
||||
- name: API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-secret
|
||||
key: api-key
|
||||
- name: API_ENDPOINT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-secret
|
||||
key: api-endpoint
|
||||
- name: SERVICE_ID
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-secret
|
||||
key: service-id
|
||||
|
||||
---
|
||||
# Example: Using the secret in a ConfigMap template
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: api-config
|
||||
namespace: api-service
|
||||
data:
|
||||
config.yaml: |
|
||||
service:
|
||||
name: api-consumer
|
||||
# API credentials loaded from secret via environment variables
|
||||
# See deployment above for how to inject API_KEY
|
||||
|
||||
endpoints:
|
||||
external:
|
||||
timeout: 30s
|
||||
retry: 3
|
||||
|
||||
---
|
||||
# How to retrieve the API key for external use
|
||||
# kubectl get secret api-secret -n api-service -o jsonpath='{.data.api-key}' | base64 -d
|
||||
287
examples/example-2-registry-rotation-simple.yaml
Normal file
287
examples/example-2-registry-rotation-simple.yaml
Normal file
@@ -0,0 +1,287 @@
|
||||
# Example 2: Container Registry with Rotation + Reloader
|
||||
#
|
||||
# Use case: Docker registry with basic auth that rotates every 90 days
|
||||
#
|
||||
# Characteristics:
|
||||
# - Password rotates every 90 days
|
||||
# - Registry doesn't need old password to accept new one
|
||||
# - InitContainer regenerates htpasswd on pod restart
|
||||
# - Reloader automatically restarts pod when secret changes
|
||||
# - No keepPreviousVersion needed
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: registry
|
||||
|
||||
---
|
||||
# ManagedSecret generates and rotates registry credentials
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: registry-auth
|
||||
namespace: registry
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: registry/auth
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "admin"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 32
|
||||
minDigits: 5
|
||||
minSymbols: 5
|
||||
minLowercase: 5
|
||||
minUppercase: 5
|
||||
symbolCharacters: "!@#$%^&*"
|
||||
allowRepeat: false
|
||||
|
||||
destination:
|
||||
name: registry-credentials
|
||||
type: Opaque
|
||||
# No need for previous version - htpasswd is regenerated fresh
|
||||
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 2160h # 90 days
|
||||
rotateGeneratedOnly: true # Only rotate password, keep username
|
||||
|
||||
---
|
||||
# PersistentVolumeClaim for registry storage
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: registry-pvc
|
||||
namespace: registry
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
storageClassName: longhorn
|
||||
|
||||
---
|
||||
# Registry Deployment with Reloader integration
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: registry
|
||||
namespace: registry
|
||||
labels:
|
||||
app: registry
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: registry
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: registry
|
||||
annotations:
|
||||
# Reloader watches registry-credentials and restarts pod on change
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
fsGroup: 65534
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
initContainers:
|
||||
# Generate htpasswd file from current credentials at pod startup
|
||||
- name: generate-htpasswd
|
||||
image: httpd:2.4-alpine
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
USERNAME=$(cat /credentials/username)
|
||||
PASSWORD=$(cat /credentials/password)
|
||||
htpasswd -Bbn "$USERNAME" "$PASSWORD" > /auth/htpasswd
|
||||
chmod 644 /auth/htpasswd
|
||||
echo "htpasswd generated with user: $USERNAME"
|
||||
volumeMounts:
|
||||
- name: credentials
|
||||
mountPath: /credentials
|
||||
readOnly: true
|
||||
- name: auth
|
||||
mountPath: /auth
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
containers:
|
||||
- name: registry
|
||||
image: registry:2
|
||||
ports:
|
||||
- containerPort: 5000
|
||||
name: http
|
||||
env:
|
||||
- name: REGISTRY_STORAGE_DELETE_ENABLED
|
||||
value: "true"
|
||||
- name: REGISTRY_HTTP_ADDR
|
||||
value: "0.0.0.0:5000"
|
||||
# Enable HTTP basic auth
|
||||
- name: REGISTRY_AUTH
|
||||
value: "htpasswd"
|
||||
- name: REGISTRY_AUTH_HTPASSWD_REALM
|
||||
value: "Registry Realm"
|
||||
- name: REGISTRY_AUTH_HTPASSWD_PATH
|
||||
value: "/auth/htpasswd"
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
volumeMounts:
|
||||
- name: registry-storage
|
||||
mountPath: /var/lib/registry
|
||||
- name: auth
|
||||
mountPath: /auth
|
||||
readOnly: true
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 5000
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 5000
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
volumes:
|
||||
- name: registry-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: registry-pvc
|
||||
- name: credentials
|
||||
secret:
|
||||
secretName: registry-credentials
|
||||
- name: auth
|
||||
emptyDir: {}
|
||||
|
||||
---
|
||||
# Service for internal access
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: registry
|
||||
namespace: registry
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: registry
|
||||
ports:
|
||||
- port: 5000
|
||||
targetPort: 5000
|
||||
protocol: TCP
|
||||
name: http
|
||||
|
||||
---
|
||||
# Ingress for external access with TLS
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: registry
|
||||
namespace: registry
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: letsencrypt-production
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: "0"
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- registry.c5ai.ch
|
||||
secretName: registry-ingress-tls
|
||||
rules:
|
||||
- host: registry.c5ai.ch
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: registry
|
||||
port:
|
||||
number: 5000
|
||||
|
||||
---
|
||||
# How the rotation flow works:
|
||||
#
|
||||
# 1. After 90 days, ManagedSecret operator:
|
||||
# - Generates new password
|
||||
# - Updates Vault
|
||||
# - Updates registry-credentials Secret
|
||||
#
|
||||
# 2. Reloader detects Secret change:
|
||||
# - Sees resourceVersion changed
|
||||
# - Triggers rolling restart of registry Deployment
|
||||
#
|
||||
# 3. Pod restart with new credentials:
|
||||
# - InitContainer runs
|
||||
# - Reads NEW password from registry-credentials
|
||||
# - Generates fresh htpasswd file
|
||||
# - Registry container starts with new htpasswd
|
||||
#
|
||||
# 4. Result:
|
||||
# - Old clients with old password: FAIL (401 Unauthorized)
|
||||
# - New clients with new password: SUCCESS
|
||||
# - Downtime: ~1-2 minutes during rolling restart
|
||||
|
||||
---
|
||||
# Testing commands:
|
||||
#
|
||||
# # Get current credentials
|
||||
# kubectl get secret registry-credentials -n registry -o jsonpath='{.data.username}' | base64 -d
|
||||
# kubectl get secret registry-credentials -n registry -o jsonpath='{.data.password}' | base64 -d
|
||||
#
|
||||
# # Test without auth (should fail)
|
||||
# curl -i https://registry.c5ai.ch/v2/
|
||||
#
|
||||
# # Test with auth (should succeed)
|
||||
# REGISTRY_USER=$(kubectl get secret registry-credentials -n registry -o jsonpath='{.data.username}' | base64 -d)
|
||||
# REGISTRY_PASS=$(kubectl get secret registry-credentials -n registry -o jsonpath='{.data.password}' | base64 -d)
|
||||
# curl -u $REGISTRY_USER:$REGISTRY_PASS https://registry.c5ai.ch/v2/_catalog
|
||||
#
|
||||
# # Docker login
|
||||
# docker login registry.c5ai.ch -u $REGISTRY_USER -p $REGISTRY_PASS
|
||||
#
|
||||
# # Force rotation test
|
||||
# kubectl annotate managedsecret registry-auth -n registry reconcile="$(date +%s)" --overwrite
|
||||
# kubectl get pods -n registry -w # Watch pod restart
|
||||
369
examples/example-3-postgres-rotation-previous.yaml
Normal file
369
examples/example-3-postgres-rotation-previous.yaml
Normal file
@@ -0,0 +1,369 @@
|
||||
# Example 3: PostgreSQL Password Rotation with Previous Version
|
||||
#
|
||||
# Use case: PostgreSQL user password that rotates every 90 days
|
||||
#
|
||||
# Characteristics:
|
||||
# - Password rotates every 90 days
|
||||
# - PostgreSQL REQUIRES old password to authenticate before changing
|
||||
# - keepPreviousVersion creates postgres-secret-previous
|
||||
# - CronJob uses old password to authenticate, sets new password
|
||||
# - After 1 hour grace period, old secret is cleaned up
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: database
|
||||
|
||||
---
|
||||
# ManagedSecret with previous version support
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: postgres-app-user
|
||||
namespace: database
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: postgres/app-user
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "app_user"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 40
|
||||
minDigits: 8
|
||||
minSymbols: 8
|
||||
minLowercase: 8
|
||||
minUppercase: 8
|
||||
symbolCharacters: "!@#$%^&*()"
|
||||
allowRepeat: false
|
||||
|
||||
- name: host
|
||||
type: static
|
||||
value: "postgres.database.svc.cluster.local"
|
||||
|
||||
- name: port
|
||||
type: static
|
||||
value: "5432"
|
||||
|
||||
- name: database
|
||||
type: static
|
||||
value: "app_db"
|
||||
|
||||
destination:
|
||||
name: postgres-secret
|
||||
type: Opaque
|
||||
# Keep previous version for authentication during rotation
|
||||
keepPreviousVersion: true
|
||||
previousVersionTTL: 1h # Clean up after 1 hour
|
||||
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 2160h # 90 days
|
||||
rotateGeneratedOnly: true
|
||||
|
||||
---
|
||||
# ServiceAccount for password rotation job
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: postgres-rotator
|
||||
namespace: database
|
||||
|
||||
---
|
||||
# Role to allow reading secrets
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: postgres-rotator
|
||||
namespace: database
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list"]
|
||||
resourceNames: ["postgres-secret", "postgres-secret-previous"]
|
||||
|
||||
---
|
||||
# RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: postgres-rotator
|
||||
namespace: database
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: postgres-rotator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: postgres-rotator
|
||||
namespace: database
|
||||
|
||||
---
|
||||
# CronJob to detect rotation and update PostgreSQL password
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: postgres-password-rotator
|
||||
namespace: database
|
||||
spec:
|
||||
# Run every 5 minutes to check for rotation
|
||||
schedule: "*/5 * * * *"
|
||||
successfulJobsHistoryLimit: 3
|
||||
failedJobsHistoryLimit: 3
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
# Reloader can also trigger this job when secret changes
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
serviceAccountName: postgres-rotator
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: rotate-password
|
||||
image: postgres:15-alpine
|
||||
env:
|
||||
# Current credentials
|
||||
- name: PGUSER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: username
|
||||
- name: NEW_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: password
|
||||
- name: PGHOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: host
|
||||
- name: PGPORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: port
|
||||
- name: PGDATABASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: database
|
||||
|
||||
# Previous credentials (for authentication)
|
||||
- name: OLD_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret-previous
|
||||
key: password
|
||||
optional: true # Might not exist if no rotation yet
|
||||
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
|
||||
echo "=========================================="
|
||||
echo "PostgreSQL Password Rotation Check"
|
||||
echo "=========================================="
|
||||
echo "User: $PGUSER"
|
||||
echo "Host: $PGHOST:$PGPORT"
|
||||
echo "Database: $PGDATABASE"
|
||||
echo ""
|
||||
|
||||
# Check if previous password exists (rotation happened)
|
||||
if [ -z "$OLD_PASSWORD" ]; then
|
||||
echo "No previous password found - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if passwords are different
|
||||
if [ "$OLD_PASSWORD" = "$NEW_PASSWORD" ]; then
|
||||
echo "Passwords are identical - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Rotation detected! Old and new passwords differ."
|
||||
echo ""
|
||||
|
||||
# Test if current password in PostgreSQL is the old one
|
||||
echo "Testing authentication with OLD password..."
|
||||
if PGPASSWORD=$OLD_PASSWORD psql -c "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ Old password still active in PostgreSQL"
|
||||
echo ""
|
||||
echo "Updating to NEW password..."
|
||||
|
||||
# Authenticate with OLD password, set NEW password
|
||||
if PGPASSWORD=$OLD_PASSWORD psql -c "ALTER USER $PGUSER PASSWORD '$NEW_PASSWORD';"; then
|
||||
echo "✓ Password updated successfully!"
|
||||
echo ""
|
||||
|
||||
# Verify new password works
|
||||
echo "Verifying NEW password..."
|
||||
if PGPASSWORD=$NEW_PASSWORD psql -c "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ New password verified and working!"
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Rotation completed successfully"
|
||||
echo "=========================================="
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: New password doesn't work!"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "✗ ERROR: Failed to update password"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "Old password doesn't work - checking if new password already active..."
|
||||
if PGPASSWORD=$NEW_PASSWORD psql -c "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ New password is already active in PostgreSQL"
|
||||
echo "Rotation was already completed"
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: Neither old nor new password works!"
|
||||
echo "Manual intervention required"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
---
|
||||
# Example application Deployment using the credentials
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: app
|
||||
namespace: database
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: postgres-client
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: postgres-client
|
||||
annotations:
|
||||
# Reloader restarts app when credentials change
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: postgres:15-alpine
|
||||
env:
|
||||
- name: PGUSER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: username
|
||||
- name: PGPASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: password
|
||||
- name: PGHOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: host
|
||||
- name: PGPORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: port
|
||||
- name: PGDATABASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-secret
|
||||
key: database
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
# Your application code here
|
||||
while true; do
|
||||
psql -c "SELECT NOW();" || echo "Connection failed"
|
||||
sleep 30
|
||||
done
|
||||
|
||||
---
|
||||
# How the rotation flow works:
|
||||
#
|
||||
# Day 0: Initial setup
|
||||
# - ManagedSecret creates postgres-secret with generated password
|
||||
# - No postgres-secret-previous exists yet
|
||||
# - CronJob runs every 5 minutes, finds no previous password, does nothing
|
||||
#
|
||||
# Day 90: Rotation triggered
|
||||
# 1. ManagedSecret operator:
|
||||
# - Generates NEW password
|
||||
# - Updates Vault with new password
|
||||
# - Creates postgres-secret-previous with OLD password
|
||||
# - Updates postgres-secret with NEW password
|
||||
#
|
||||
# 2. Within 5 minutes, CronJob runs:
|
||||
# - Detects postgres-secret-previous exists
|
||||
# - Compares OLD_PASSWORD vs NEW_PASSWORD (different!)
|
||||
# - Authenticates to PostgreSQL with OLD_PASSWORD
|
||||
# - Runs: ALTER USER app_user PASSWORD 'new_password'
|
||||
# - Verifies new password works
|
||||
# - Job succeeds
|
||||
#
|
||||
# 3. After 1 hour (previousVersionTTL):
|
||||
# - ManagedSecret operator deletes postgres-secret-previous
|
||||
# - CronJob future runs find no previous password, do nothing
|
||||
#
|
||||
# 4. Application pods:
|
||||
# - Reloader detects postgres-secret change
|
||||
# - Triggers rolling restart
|
||||
# - Pods reconnect with NEW password
|
||||
# - ~1 minute downtime per pod during restart
|
||||
|
||||
---
|
||||
# Manual rotation testing:
|
||||
#
|
||||
# # Check current state
|
||||
# kubectl get secret postgres-secret -n database -o yaml
|
||||
# kubectl get secret postgres-secret-previous -n database -o yaml # Might not exist
|
||||
#
|
||||
# # Force rotation
|
||||
# kubectl annotate managedsecret postgres-app-user -n database reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Watch the rotation
|
||||
# kubectl get secrets -n database -w
|
||||
#
|
||||
# # Check CronJob executes
|
||||
# kubectl get jobs -n database -w
|
||||
#
|
||||
# # View rotation logs
|
||||
# kubectl logs -n database -l job-name=postgres-password-rotator-XXXXX
|
||||
#
|
||||
# # Verify password in PostgreSQL matches new secret
|
||||
# NEW_PASS=$(kubectl get secret postgres-secret -n database -o jsonpath='{.data.password}' | base64 -d)
|
||||
# kubectl exec -it -n database deployment/app -- psql -c "SELECT 1;"
|
||||
|
||||
---
|
||||
# Troubleshooting:
|
||||
#
|
||||
# If rotation fails:
|
||||
# 1. Check CronJob logs for errors
|
||||
# 2. Verify PostgreSQL is accessible from CronJob pod
|
||||
# 3. Check if old password still works in PostgreSQL
|
||||
# 4. Manually update if needed:
|
||||
# kubectl exec -it postgres-0 -n database -- psql -U postgres
|
||||
# ALTER USER app_user PASSWORD 'password_from_secret';
|
||||
430
examples/example-4-minio-rotation-previous.yaml
Normal file
430
examples/example-4-minio-rotation-previous.yaml
Normal file
@@ -0,0 +1,430 @@
|
||||
# Example 4: MinIO Admin Password Rotation with Previous Version
|
||||
#
|
||||
# Use case: MinIO admin credentials that rotate every 60 days
|
||||
#
|
||||
# Characteristics:
|
||||
# - Password rotates every 60 days
|
||||
# - MinIO Client (mc) needs old credentials to change password
|
||||
# - keepPreviousVersion creates minio-admin-secret-previous
|
||||
# - CronJob uses old credentials to authenticate, sets new password
|
||||
# - After 2 hour grace period, old secret is cleaned up
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: minio
|
||||
|
||||
---
|
||||
# ManagedSecret for MinIO admin credentials
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: minio-admin
|
||||
namespace: minio
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: minio/admin
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "admin"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 32
|
||||
minDigits: 5
|
||||
minSymbols: 5
|
||||
minLowercase: 5
|
||||
minUppercase: 5
|
||||
symbolCharacters: "!@#$%^&*"
|
||||
allowRepeat: false
|
||||
|
||||
- name: endpoint
|
||||
type: static
|
||||
value: "http://minio.minio.svc.cluster.local:9000"
|
||||
|
||||
destination:
|
||||
name: minio-admin-secret
|
||||
type: Opaque
|
||||
# Keep previous version for authentication during rotation
|
||||
keepPreviousVersion: true
|
||||
previousVersionTTL: 2h # Longer grace period for MinIO
|
||||
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 1440h # 60 days
|
||||
rotateGeneratedOnly: true
|
||||
|
||||
---
|
||||
# ServiceAccount for password rotation job
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: minio-rotator
|
||||
namespace: minio
|
||||
|
||||
---
|
||||
# Role to allow reading secrets
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: minio-rotator
|
||||
namespace: minio
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list"]
|
||||
resourceNames: ["minio-admin-secret", "minio-admin-secret-previous"]
|
||||
|
||||
---
|
||||
# RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: minio-rotator
|
||||
namespace: minio
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: minio-rotator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: minio-rotator
|
||||
namespace: minio
|
||||
|
||||
---
|
||||
# CronJob to detect rotation and update MinIO password
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: minio-password-rotator
|
||||
namespace: minio
|
||||
spec:
|
||||
# Run every 10 minutes to check for rotation
|
||||
schedule: "*/10 * * * *"
|
||||
successfulJobsHistoryLimit: 3
|
||||
failedJobsHistoryLimit: 3
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
serviceAccountName: minio-rotator
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: rotate-password
|
||||
image: minio/mc:latest
|
||||
env:
|
||||
# Current credentials
|
||||
- name: MC_HOST_myminio
|
||||
value: "" # Will be set in script
|
||||
- name: USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret
|
||||
key: username
|
||||
- name: NEW_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret
|
||||
key: password
|
||||
- name: ENDPOINT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret
|
||||
key: endpoint
|
||||
|
||||
# Previous credentials (for authentication)
|
||||
- name: OLD_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret-previous
|
||||
key: username
|
||||
optional: true
|
||||
- name: OLD_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret-previous
|
||||
key: password
|
||||
optional: true
|
||||
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
|
||||
echo "=========================================="
|
||||
echo "MinIO Password Rotation Check"
|
||||
echo "=========================================="
|
||||
echo "User: $USERNAME"
|
||||
echo "Endpoint: $ENDPOINT"
|
||||
echo ""
|
||||
|
||||
# Check if previous password exists (rotation happened)
|
||||
if [ -z "$OLD_PASSWORD" ]; then
|
||||
echo "No previous password found - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if passwords are different
|
||||
if [ "$OLD_PASSWORD" = "$NEW_PASSWORD" ]; then
|
||||
echo "Passwords are identical - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Rotation detected! Old and new passwords differ."
|
||||
echo ""
|
||||
|
||||
# Test if current password in MinIO is the old one
|
||||
echo "Testing authentication with OLD credentials..."
|
||||
if mc alias set oldminio $ENDPOINT $OLD_USERNAME $OLD_PASSWORD > /dev/null 2>&1; then
|
||||
echo "✓ Old credentials still active in MinIO"
|
||||
echo ""
|
||||
|
||||
# Test admin access
|
||||
if mc admin info oldminio > /dev/null 2>&1; then
|
||||
echo "✓ Admin access confirmed with old credentials"
|
||||
echo ""
|
||||
echo "Updating to NEW password..."
|
||||
|
||||
# Change password using old credentials
|
||||
if mc admin user password oldminio $USERNAME $NEW_PASSWORD; then
|
||||
echo "✓ Password updated successfully!"
|
||||
echo ""
|
||||
|
||||
# Verify new password works
|
||||
echo "Verifying NEW password..."
|
||||
if mc alias set newminio $ENDPOINT $USERNAME $NEW_PASSWORD > /dev/null 2>&1; then
|
||||
if mc admin info newminio > /dev/null 2>&1; then
|
||||
echo "✓ New password verified and working!"
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Rotation completed successfully"
|
||||
echo "=========================================="
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: New credentials don't have admin access!"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "✗ ERROR: New password doesn't work!"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "✗ ERROR: Failed to update password"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "✗ ERROR: No admin access with old credentials"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "Old credentials don't work - checking if new credentials already active..."
|
||||
if mc alias set newminio $ENDPOINT $USERNAME $NEW_PASSWORD > /dev/null 2>&1; then
|
||||
if mc admin info newminio > /dev/null 2>&1; then
|
||||
echo "✓ New credentials are already active in MinIO"
|
||||
echo "Rotation was already completed"
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: New credentials exist but no admin access"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "✗ ERROR: Neither old nor new credentials work!"
|
||||
echo "Manual intervention required"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
---
|
||||
# Example: MinIO StatefulSet (simplified)
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: minio
|
||||
namespace: minio
|
||||
spec:
|
||||
serviceName: minio
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: minio
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: minio
|
||||
annotations:
|
||||
# Note: MinIO doesn't auto-reload credentials from env vars
|
||||
# You may need to restart pods manually or use a sidecar
|
||||
reloader.stakater.com/auto: "false"
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
image: minio/minio:latest
|
||||
args:
|
||||
- server
|
||||
- /data
|
||||
- --console-address
|
||||
- ":9001"
|
||||
env:
|
||||
- name: MINIO_ROOT_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret
|
||||
key: username
|
||||
- name: MINIO_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: minio-admin-secret
|
||||
key: password
|
||||
ports:
|
||||
- containerPort: 9000
|
||||
name: api
|
||||
- containerPort: 9001
|
||||
name: console
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
|
||||
---
|
||||
# Service for MinIO
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio
|
||||
namespace: minio
|
||||
spec:
|
||||
selector:
|
||||
app: minio
|
||||
ports:
|
||||
- port: 9000
|
||||
targetPort: 9000
|
||||
name: api
|
||||
- port: 9001
|
||||
targetPort: 9001
|
||||
name: console
|
||||
|
||||
---
|
||||
# How the rotation flow works:
|
||||
#
|
||||
# Day 0: Initial setup
|
||||
# - ManagedSecret creates minio-admin-secret with generated password
|
||||
# - MinIO starts with these credentials
|
||||
# - No minio-admin-secret-previous exists yet
|
||||
# - CronJob runs every 10 minutes, finds no previous password, does nothing
|
||||
#
|
||||
# Day 60: Rotation triggered
|
||||
# 1. ManagedSecret operator:
|
||||
# - Generates NEW password
|
||||
# - Updates Vault with new password
|
||||
# - Creates minio-admin-secret-previous with OLD password
|
||||
# - Updates minio-admin-secret with NEW password
|
||||
#
|
||||
# 2. Within 10 minutes, CronJob runs:
|
||||
# - Detects minio-admin-secret-previous exists
|
||||
# - Compares OLD_PASSWORD vs NEW_PASSWORD (different!)
|
||||
# - Authenticates to MinIO with OLD credentials
|
||||
# - Runs: mc admin user password myminio admin new_password
|
||||
# - Verifies new password works
|
||||
# - Job succeeds
|
||||
#
|
||||
# 3. After 2 hours (previousVersionTTL):
|
||||
# - ManagedSecret operator deletes minio-admin-secret-previous
|
||||
# - CronJob future runs find no previous password, do nothing
|
||||
#
|
||||
# 4. MinIO pods:
|
||||
# - IMPORTANT: MinIO does NOT reload credentials from env vars
|
||||
# - You need to manually restart MinIO pods after rotation
|
||||
# - Or use a sidecar to monitor and trigger restart
|
||||
# - Consider NOT using reloader.stakater.com/auto for MinIO
|
||||
|
||||
---
|
||||
# Alternative: Sidecar to handle MinIO restart
|
||||
# Add this to the MinIO StatefulSet if you want automatic restarts
|
||||
---
|
||||
# sidecar:
|
||||
# - name: credential-watcher
|
||||
# image: bitnami/kubectl:latest
|
||||
# command:
|
||||
# - sh
|
||||
# - -c
|
||||
# - |
|
||||
# LAST_VERSION=""
|
||||
# while true; do
|
||||
# CURRENT_VERSION=$(kubectl get secret minio-admin-secret -n minio -o jsonpath='{.metadata.resourceVersion}')
|
||||
# if [ -n "$LAST_VERSION" ] && [ "$LAST_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
# echo "Credentials changed! Signaling MinIO to restart..."
|
||||
# # Send SIGTERM to MinIO process to trigger graceful shutdown
|
||||
# killall minio
|
||||
# fi
|
||||
# LAST_VERSION=$CURRENT_VERSION
|
||||
# sleep 30
|
||||
# done
|
||||
|
||||
---
|
||||
# Manual rotation testing:
|
||||
#
|
||||
# # Check current state
|
||||
# kubectl get secret minio-admin-secret -n minio -o yaml
|
||||
# kubectl get secret minio-admin-secret-previous -n minio -o yaml
|
||||
#
|
||||
# # Force rotation
|
||||
# kubectl annotate managedsecret minio-admin -n minio reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Watch the rotation
|
||||
# kubectl get secrets -n minio -w
|
||||
#
|
||||
# # Check CronJob executes
|
||||
# kubectl get jobs -n minio -w
|
||||
#
|
||||
# # View rotation logs
|
||||
# kubectl logs -n minio -l job-name=minio-password-rotator-XXXXX
|
||||
#
|
||||
# # Test MinIO access with new credentials
|
||||
# NEW_USER=$(kubectl get secret minio-admin-secret -n minio -o jsonpath='{.data.username}' | base64 -d)
|
||||
# NEW_PASS=$(kubectl get secret minio-admin-secret -n minio -o jsonpath='{.data.password}' | base64 -d)
|
||||
# mc alias set testminio http://minio.minio.svc.cluster.local:9000 $NEW_USER $NEW_PASS
|
||||
# mc admin info testminio
|
||||
|
||||
---
|
||||
# Important Notes:
|
||||
#
|
||||
# 1. MinIO credential reload:
|
||||
# - Unlike PostgreSQL, MinIO does NOT automatically reload credentials
|
||||
# - After rotation, you must restart MinIO pods
|
||||
# - Consider using a sidecar or manual restart workflow
|
||||
#
|
||||
# 2. High Availability:
|
||||
# - For HA MinIO clusters, all nodes share the same root credentials
|
||||
# - Rotation updates credentials cluster-wide
|
||||
# - Rolling restart required for all nodes
|
||||
#
|
||||
# 3. Grace Period:
|
||||
# - 2 hour previousVersionTTL allows time for manual intervention
|
||||
# - Adjust based on your operational procedures
|
||||
#
|
||||
# 4. Monitoring:
|
||||
# - Monitor CronJob success/failure
|
||||
# - Alert on rotation failures
|
||||
# - Track when MinIO pods were last restarted
|
||||
566
examples/example-5-mysql-rotation-previous.yaml
Normal file
566
examples/example-5-mysql-rotation-previous.yaml
Normal file
@@ -0,0 +1,566 @@
|
||||
# Example 5: MySQL User Password Rotation with Previous Version
|
||||
#
|
||||
# Use case: MySQL application user password that rotates every 90 days
|
||||
#
|
||||
# Characteristics:
|
||||
# - Password rotates every 90 days
|
||||
# - MySQL requires old password to authenticate before changing (when using SET PASSWORD)
|
||||
# - keepPreviousVersion creates mysql-app-user-previous
|
||||
# - CronJob uses old password to authenticate, sets new password
|
||||
# - Application pods restart automatically via Reloader
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: mysql
|
||||
|
||||
---
|
||||
# ManagedSecret for MySQL application user
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: mysql-app-user
|
||||
namespace: mysql
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: mysql/app-user
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "app_user"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 32
|
||||
minDigits: 6
|
||||
minSymbols: 6
|
||||
minLowercase: 8
|
||||
minUppercase: 8
|
||||
# MySQL doesn't allow some special chars in passwords
|
||||
symbolCharacters: "!@#$%^&*"
|
||||
allowRepeat: false
|
||||
|
||||
- name: host
|
||||
type: static
|
||||
value: "mysql.mysql.svc.cluster.local"
|
||||
|
||||
- name: port
|
||||
type: static
|
||||
value: "3306"
|
||||
|
||||
- name: database
|
||||
type: static
|
||||
value: "app_db"
|
||||
|
||||
- name: connection-string
|
||||
type: static
|
||||
value: "mysql://app_user@mysql.mysql.svc.cluster.local:3306/app_db"
|
||||
|
||||
destination:
|
||||
name: mysql-app-secret
|
||||
type: Opaque
|
||||
# Keep previous version for authentication during rotation
|
||||
keepPreviousVersion: true
|
||||
previousVersionTTL: 1h
|
||||
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 2160h # 90 days
|
||||
rotateGeneratedOnly: true
|
||||
|
||||
---
|
||||
# Separate ManagedSecret for MySQL root credentials (no rotation)
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: mysql-root
|
||||
namespace: mysql
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: mysql/root
|
||||
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "root"
|
||||
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 40
|
||||
minDigits: 8
|
||||
minSymbols: 8
|
||||
|
||||
destination:
|
||||
name: mysql-root-secret
|
||||
type: Opaque
|
||||
|
||||
# Root password doesn't rotate automatically
|
||||
rotation:
|
||||
enabled: false
|
||||
|
||||
---
|
||||
# ServiceAccount for password rotation job
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: mysql-rotator
|
||||
namespace: mysql
|
||||
|
||||
---
|
||||
# Role to allow reading secrets
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: mysql-rotator
|
||||
namespace: mysql
|
||||
rules:
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list"]
|
||||
resourceNames:
|
||||
- "mysql-app-secret"
|
||||
- "mysql-app-secret-previous"
|
||||
- "mysql-root-secret"
|
||||
|
||||
---
|
||||
# RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: mysql-rotator
|
||||
namespace: mysql
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: mysql-rotator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: mysql-rotator
|
||||
namespace: mysql
|
||||
|
||||
---
|
||||
# CronJob to detect rotation and update MySQL password
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: mysql-password-rotator
|
||||
namespace: mysql
|
||||
spec:
|
||||
# Run every 5 minutes to check for rotation
|
||||
schedule: "*/5 * * * *"
|
||||
successfulJobsHistoryLimit: 3
|
||||
failedJobsHistoryLimit: 3
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
serviceAccountName: mysql-rotator
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: rotate-password
|
||||
image: mysql:8.0
|
||||
env:
|
||||
# Current credentials
|
||||
- name: MYSQL_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: username
|
||||
- name: NEW_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: password
|
||||
- name: MYSQL_HOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: host
|
||||
- name: MYSQL_PORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: port
|
||||
- name: MYSQL_DATABASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: database
|
||||
|
||||
# Previous credentials (for authentication)
|
||||
- name: OLD_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret-previous
|
||||
key: password
|
||||
optional: true
|
||||
|
||||
# Root credentials for fallback
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-root-secret
|
||||
key: password
|
||||
|
||||
command:
|
||||
- bash
|
||||
- -c
|
||||
- |
|
||||
set -e
|
||||
|
||||
echo "=========================================="
|
||||
echo "MySQL Password Rotation Check"
|
||||
echo "=========================================="
|
||||
echo "User: $MYSQL_USER"
|
||||
echo "Host: $MYSQL_HOST:$MYSQL_PORT"
|
||||
echo "Database: $MYSQL_DATABASE"
|
||||
echo ""
|
||||
|
||||
# Check if previous password exists (rotation happened)
|
||||
if [ -z "$OLD_PASSWORD" ]; then
|
||||
echo "No previous password found - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if passwords are different
|
||||
if [ "$OLD_PASSWORD" = "$NEW_PASSWORD" ]; then
|
||||
echo "Passwords are identical - no rotation needed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Rotation detected! Old and new passwords differ."
|
||||
echo ""
|
||||
|
||||
# Test if current password in MySQL is the old one
|
||||
echo "Testing authentication with OLD password..."
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -u"$MYSQL_USER" -p"$OLD_PASSWORD" -e "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ Old password still active in MySQL"
|
||||
echo ""
|
||||
echo "Updating to NEW password..."
|
||||
|
||||
# Method 1: Use ALTER USER (MySQL 5.7.6+, doesn't require old password)
|
||||
# Connect as root to change password
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -uroot -p"$MYSQL_ROOT_PASSWORD" \
|
||||
-e "ALTER USER '$MYSQL_USER'@'%' IDENTIFIED BY '$NEW_PASSWORD';" 2>/dev/null; then
|
||||
echo "✓ Password updated using ALTER USER (as root)"
|
||||
elif mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -uroot -p"$MYSQL_ROOT_PASSWORD" \
|
||||
-e "SET PASSWORD FOR '$MYSQL_USER'@'%' = PASSWORD('$NEW_PASSWORD');" 2>/dev/null; then
|
||||
echo "✓ Password updated using SET PASSWORD (as root)"
|
||||
else
|
||||
# Method 2: Use SET PASSWORD as the user (requires old password connection)
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -u"$MYSQL_USER" -p"$OLD_PASSWORD" \
|
||||
-e "SET PASSWORD = PASSWORD('$NEW_PASSWORD');" 2>/dev/null; then
|
||||
echo "✓ Password updated using SET PASSWORD (as user)"
|
||||
else
|
||||
echo "✗ ERROR: Failed to update password with any method"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# Verify new password works
|
||||
echo "Verifying NEW password..."
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -u"$MYSQL_USER" -p"$NEW_PASSWORD" -e "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ New password verified and working!"
|
||||
echo ""
|
||||
echo "Flushing privileges..."
|
||||
mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -uroot -p"$MYSQL_ROOT_PASSWORD" -e "FLUSH PRIVILEGES;" || echo "Warning: Could not flush privileges"
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Rotation completed successfully"
|
||||
echo "=========================================="
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: New password doesn't work!"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "Old password doesn't work - checking if new password already active..."
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -u"$MYSQL_USER" -p"$NEW_PASSWORD" -e "SELECT 1;" > /dev/null 2>&1; then
|
||||
echo "✓ New password is already active in MySQL"
|
||||
echo "Rotation was already completed"
|
||||
exit 0
|
||||
else
|
||||
echo "✗ ERROR: Neither old nor new password works!"
|
||||
echo "Manual intervention required"
|
||||
echo ""
|
||||
echo "Attempting to reset using root credentials..."
|
||||
if mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -uroot -p"$MYSQL_ROOT_PASSWORD" \
|
||||
-e "ALTER USER '$MYSQL_USER'@'%' IDENTIFIED BY '$NEW_PASSWORD';"; then
|
||||
echo "✓ Password reset successfully using root"
|
||||
exit 0
|
||||
else
|
||||
echo "✗ Failed to reset password"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
---
|
||||
# Example MySQL StatefulSet (simplified)
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: mysql
|
||||
namespace: mysql
|
||||
spec:
|
||||
serviceName: mysql
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mysql
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mysql
|
||||
spec:
|
||||
containers:
|
||||
- name: mysql
|
||||
image: mysql:8.0
|
||||
env:
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-root-secret
|
||||
key: password
|
||||
- name: MYSQL_DATABASE
|
||||
value: "app_db"
|
||||
ports:
|
||||
- containerPort: 3306
|
||||
name: mysql
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/mysql
|
||||
livenessProbe:
|
||||
exec:
|
||||
command:
|
||||
- mysqladmin
|
||||
- ping
|
||||
- -h
|
||||
- localhost
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
exec:
|
||||
command:
|
||||
- mysql
|
||||
- -h
|
||||
- localhost
|
||||
- -e
|
||||
- SELECT 1
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 2
|
||||
# InitContainer to create app user on first run
|
||||
initContainers:
|
||||
- name: init-db-user
|
||||
image: mysql:8.0
|
||||
env:
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-root-secret
|
||||
key: password
|
||||
- name: APP_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: username
|
||||
- name: APP_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: password
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
# Wait for MySQL to be ready
|
||||
until mysql -h mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e "SELECT 1" > /dev/null 2>&1; do
|
||||
echo "Waiting for MySQL..."
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Create user if not exists
|
||||
mysql -h mysql -uroot -p"$MYSQL_ROOT_PASSWORD" <<EOF
|
||||
CREATE USER IF NOT EXISTS '$APP_USER'@'%' IDENTIFIED BY '$APP_PASSWORD';
|
||||
GRANT ALL PRIVILEGES ON app_db.* TO '$APP_USER'@'%';
|
||||
FLUSH PRIVILEGES;
|
||||
EOF
|
||||
|
||||
echo "User $APP_USER created/updated"
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
|
||||
---
|
||||
# Service for MySQL
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: mysql
|
||||
namespace: mysql
|
||||
spec:
|
||||
selector:
|
||||
app: mysql
|
||||
ports:
|
||||
- port: 3306
|
||||
targetPort: 3306
|
||||
name: mysql
|
||||
clusterIP: None # Headless service
|
||||
|
||||
---
|
||||
# Example application using MySQL
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: app
|
||||
namespace: mysql
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mysql-client
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mysql-client
|
||||
annotations:
|
||||
# Reloader restarts app when credentials change
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: mysql:8.0
|
||||
env:
|
||||
- name: MYSQL_HOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: host
|
||||
- name: MYSQL_PORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: port
|
||||
- name: MYSQL_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: username
|
||||
- name: MYSQL_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: password
|
||||
- name: MYSQL_DATABASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: mysql-app-secret
|
||||
key: database
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
# Your application code here
|
||||
while true; do
|
||||
mysql -h"$MYSQL_HOST" -P"$MYSQL_PORT" -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" \
|
||||
-D"$MYSQL_DATABASE" -e "SELECT NOW();" || echo "Connection failed"
|
||||
sleep 30
|
||||
done
|
||||
|
||||
---
|
||||
# How the rotation flow works:
|
||||
#
|
||||
# Initial Setup:
|
||||
# - mysql-app-secret created with initial password
|
||||
# - InitContainer creates MySQL user with this password
|
||||
# - Application connects successfully
|
||||
#
|
||||
# Day 90: Rotation
|
||||
# 1. ManagedSecret operator:
|
||||
# - Generates NEW password
|
||||
# - Creates mysql-app-secret-previous (OLD password)
|
||||
# - Updates mysql-app-secret (NEW password)
|
||||
#
|
||||
# 2. CronJob (within 5 minutes):
|
||||
# - Detects previous secret exists
|
||||
# - Authenticates to MySQL with OLD password (or as root)
|
||||
# - Runs ALTER USER to set NEW password
|
||||
# - Verifies NEW password works
|
||||
#
|
||||
# 3. Reloader:
|
||||
# - Detects mysql-app-secret change
|
||||
# - Restarts application pods
|
||||
# - Pods reconnect with NEW password
|
||||
#
|
||||
# 4. Cleanup (after 1 hour):
|
||||
# - mysql-app-secret-previous deleted automatically
|
||||
|
||||
---
|
||||
# Testing commands:
|
||||
#
|
||||
# # Check current credentials
|
||||
# kubectl get secret mysql-app-secret -n mysql -o yaml
|
||||
#
|
||||
# # Test connection with current credentials
|
||||
# MYSQL_PASS=$(kubectl get secret mysql-app-secret -n mysql -o jsonpath='{.data.password}' | base64 -d)
|
||||
# kubectl run -it --rm mysql-test --image=mysql:8.0 --restart=Never -- \
|
||||
# mysql -h mysql.mysql.svc.cluster.local -u app_user -p"$MYSQL_PASS" -e "SELECT NOW();"
|
||||
#
|
||||
# # Force rotation
|
||||
# kubectl annotate managedsecret mysql-app-user -n mysql reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Watch rotation
|
||||
# kubectl get secrets -n mysql -w
|
||||
# kubectl get jobs -n mysql -w
|
||||
#
|
||||
# # View rotation logs
|
||||
# kubectl logs -n mysql -l job-name=mysql-password-rotator-XXXXX
|
||||
|
||||
---
|
||||
# Important MySQL-specific notes:
|
||||
#
|
||||
# 1. User Host Specification:
|
||||
# - '%' allows connection from any host
|
||||
# - Consider restricting to specific pods/namespaces
|
||||
# - Example: 'app_user'@'%.mysql.svc.cluster.local'
|
||||
#
|
||||
# 2. Authentication Methods:
|
||||
# - MySQL 8.0+ uses caching_sha2_password by default
|
||||
# - Older apps may need mysql_native_password
|
||||
# - Specify during user creation if needed
|
||||
#
|
||||
# 3. Privilege Management:
|
||||
# - Rotation only changes password, not privileges
|
||||
# - Manage privileges separately via root access
|
||||
#
|
||||
# 4. Connection Pooling:
|
||||
# - Application connection pools need to be refreshed
|
||||
# - Reloader restart handles this automatically
|
||||
# - Consider graceful shutdown for long-running transactions
|
||||
572
examples/example-6-comprehensive-multi-service.yaml
Normal file
572
examples/example-6-comprehensive-multi-service.yaml
Normal file
@@ -0,0 +1,572 @@
|
||||
# Example 6: Comprehensive Multi-Service Deployment
|
||||
#
|
||||
# This example demonstrates a complete application stack with:
|
||||
# - PostgreSQL (rotating with previous version)
|
||||
# - Redis (rotating without previous version)
|
||||
# - External API key (no rotation)
|
||||
# - TLS certificates (no rotation)
|
||||
# - Application credentials (rotating with Reloader)
|
||||
#
|
||||
# Shows different rotation strategies working together
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: myapp
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# 1. PostgreSQL Credentials - Rotates with previous version support
|
||||
# ============================================================================
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: postgres-credentials
|
||||
namespace: myapp
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: myapp/postgres
|
||||
fields:
|
||||
- name: username
|
||||
type: static
|
||||
value: "myapp_user"
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 32
|
||||
minDigits: 6
|
||||
minSymbols: 6
|
||||
- name: host
|
||||
type: static
|
||||
value: "postgres.database.svc.cluster.local"
|
||||
- name: port
|
||||
type: static
|
||||
value: "5432"
|
||||
- name: database
|
||||
type: static
|
||||
value: "myapp_db"
|
||||
destination:
|
||||
name: postgres-creds
|
||||
type: Opaque
|
||||
keepPreviousVersion: true # Needs old password for rotation
|
||||
previousVersionTTL: 1h
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 2160h # 90 days
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# 2. Redis Password - Rotates without previous version (CONFIG SET)
|
||||
# ============================================================================
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: redis-credentials
|
||||
namespace: myapp
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: myapp/redis
|
||||
fields:
|
||||
- name: password
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 40
|
||||
minDigits: 8
|
||||
minSymbols: 0 # Redis doesn't like special chars
|
||||
symbolCharacters: ""
|
||||
- name: host
|
||||
type: static
|
||||
value: "redis.myapp.svc.cluster.local"
|
||||
- name: port
|
||||
type: static
|
||||
value: "6379"
|
||||
destination:
|
||||
name: redis-creds
|
||||
type: Opaque
|
||||
# No previous version needed - Redis CONFIG SET doesn't require auth
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 720h # 30 days
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# 3. External API Keys - No rotation
|
||||
# ============================================================================
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: external-api-keys
|
||||
namespace: myapp
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: myapp/api-keys
|
||||
fields:
|
||||
- name: stripe-api-key
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 64
|
||||
minDigits: 0
|
||||
minSymbols: 0
|
||||
- name: sendgrid-api-key
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 64
|
||||
minDigits: 0
|
||||
minSymbols: 0
|
||||
destination:
|
||||
name: api-keys
|
||||
type: Opaque
|
||||
rotation:
|
||||
enabled: false # Managed externally
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# 4. TLS Certificate - No rotation (managed by cert-manager)
|
||||
# ============================================================================
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: app-tls-cert
|
||||
namespace: myapp
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: myapp/tls
|
||||
fields:
|
||||
- name: tls.crt
|
||||
type: static
|
||||
value: |
|
||||
-----BEGIN CERTIFICATE-----
|
||||
# Your certificate here
|
||||
-----END CERTIFICATE-----
|
||||
- name: tls.key
|
||||
type: static
|
||||
value: |
|
||||
-----BEGIN PRIVATE KEY-----
|
||||
# Your private key here
|
||||
-----END PRIVATE KEY-----
|
||||
destination:
|
||||
name: app-tls
|
||||
type: kubernetes.io/tls
|
||||
rotation:
|
||||
enabled: false
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# 5. Application JWT Secret - Rotates with Reloader
|
||||
# ============================================================================
|
||||
apiVersion: secrets.c5ai.ch/v1alpha1
|
||||
kind: ManagedSecret
|
||||
metadata:
|
||||
name: app-jwt-secret
|
||||
namespace: myapp
|
||||
spec:
|
||||
vault:
|
||||
address: "http://openbao.openbao.svc.cluster.local:8200"
|
||||
authMethod: kubernetes
|
||||
role: managedsecret-operator
|
||||
kvVersion: v2
|
||||
mount: secret
|
||||
path: myapp/jwt
|
||||
fields:
|
||||
- name: jwt-secret
|
||||
type: generated
|
||||
generator:
|
||||
type: password
|
||||
length: 64
|
||||
minDigits: 16
|
||||
minSymbols: 0
|
||||
symbolCharacters: ""
|
||||
destination:
|
||||
name: jwt-secret
|
||||
type: Opaque
|
||||
rotation:
|
||||
enabled: true
|
||||
schedule: 4320h # 180 days
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# PostgreSQL Password Rotation Job
|
||||
# ============================================================================
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: postgres-password-rotator
|
||||
namespace: myapp
|
||||
spec:
|
||||
schedule: "*/5 * * * *"
|
||||
jobTemplate:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: rotate
|
||||
image: postgres:15-alpine
|
||||
env:
|
||||
- name: PGUSER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: username
|
||||
- name: NEW_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: password
|
||||
- name: OLD_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds-previous
|
||||
key: password
|
||||
optional: true
|
||||
- name: PGHOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: host
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
if [ -n "$OLD_PASSWORD" ] && [ "$OLD_PASSWORD" != "$NEW_PASSWORD" ]; then
|
||||
echo "Rotating PostgreSQL password..."
|
||||
PGPASSWORD=$OLD_PASSWORD psql -c "ALTER USER $PGUSER PASSWORD '$NEW_PASSWORD';"
|
||||
echo "Rotation complete!"
|
||||
fi
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# Redis Password Rotation Job
|
||||
# ============================================================================
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: redis-password-rotator
|
||||
namespace: myapp
|
||||
spec:
|
||||
schedule: "*/5 * * * *"
|
||||
jobTemplate:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: rotate
|
||||
image: redis:7-alpine
|
||||
env:
|
||||
- name: REDIS_HOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: host
|
||||
- name: REDIS_PORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: port
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: password
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- |
|
||||
# Redis CONFIG SET doesn't require current password
|
||||
redis-cli -h $REDIS_HOST -p $REDIS_PORT CONFIG SET requirepass "$REDIS_PASSWORD"
|
||||
echo "Redis password updated"
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# Main Application Deployment
|
||||
# ============================================================================
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: myapp
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: myapp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: myapp
|
||||
annotations:
|
||||
# Reloader restarts pods when ANY of these secrets change
|
||||
reloader.stakater.com/auto: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: myapp:latest
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
env:
|
||||
# PostgreSQL connection
|
||||
- name: POSTGRES_HOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: host
|
||||
- name: POSTGRES_PORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: port
|
||||
- name: POSTGRES_DB
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: database
|
||||
- name: POSTGRES_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: username
|
||||
- name: POSTGRES_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: postgres-creds
|
||||
key: password
|
||||
|
||||
# Redis connection
|
||||
- name: REDIS_HOST
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: host
|
||||
- name: REDIS_PORT
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: port
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-creds
|
||||
key: password
|
||||
|
||||
# API Keys
|
||||
- name: STRIPE_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-keys
|
||||
key: stripe-api-key
|
||||
- name: SENDGRID_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: api-keys
|
||||
key: sendgrid-api-key
|
||||
|
||||
# JWT Secret
|
||||
- name: JWT_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: jwt-secret
|
||||
key: jwt-secret
|
||||
|
||||
volumeMounts:
|
||||
- name: tls
|
||||
mountPath: /etc/tls
|
||||
readOnly: true
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8080
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /ready
|
||||
port: 8080
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
volumes:
|
||||
- name: tls
|
||||
secret:
|
||||
secretName: app-tls
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# Service
|
||||
# ============================================================================
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: myapp
|
||||
spec:
|
||||
selector:
|
||||
app: myapp
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 8080
|
||||
name: http
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# Ingress with TLS
|
||||
# ============================================================================
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: myapp
|
||||
annotations:
|
||||
cert-manager.io/cluster-issuer: letsencrypt-production
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- myapp.c5ai.ch
|
||||
secretName: myapp-tls-ingress
|
||||
rules:
|
||||
- host: myapp.c5ai.ch
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: myapp
|
||||
port:
|
||||
number: 80
|
||||
|
||||
---
|
||||
# ============================================================================
|
||||
# Summary of Rotation Strategies
|
||||
# ============================================================================
|
||||
#
|
||||
# 1. PostgreSQL (postgres-creds):
|
||||
# - Rotates every 90 days
|
||||
# - keepPreviousVersion: true
|
||||
# - CronJob uses old password to authenticate and change
|
||||
# - Reloader restarts app pods after rotation
|
||||
#
|
||||
# 2. Redis (redis-creds):
|
||||
# - Rotates every 30 days
|
||||
# - No previous version needed (CONFIG SET doesn't require auth)
|
||||
# - CronJob updates password directly
|
||||
# - Reloader restarts app pods after rotation
|
||||
#
|
||||
# 3. API Keys (api-keys):
|
||||
# - No automatic rotation
|
||||
# - Generated once, managed externally
|
||||
# - Update via kubectl or Vault UI when provider rotates keys
|
||||
#
|
||||
# 4. TLS Certificates (app-tls):
|
||||
# - No rotation
|
||||
# - Managed by cert-manager or external process
|
||||
# - Static content stored in Vault
|
||||
#
|
||||
# 5. JWT Secret (jwt-secret):
|
||||
# - Rotates every 180 days
|
||||
# - Simple rotation - just restart with new secret
|
||||
# - Reloader handles pod restart
|
||||
#
|
||||
# ============================================================================
|
||||
# Timeline Example: What Happens on Day 90
|
||||
# ============================================================================
|
||||
#
|
||||
# T+0: ManagedSecret operator triggers rotation
|
||||
# - Generates new passwords for postgres-creds and redis-creds
|
||||
# - Updates Vault
|
||||
# - Creates postgres-creds-previous
|
||||
# - Updates postgres-creds
|
||||
# - Updates redis-creds (no previous needed)
|
||||
#
|
||||
# T+1min: CronJobs detect changes
|
||||
# - Postgres rotator: Uses old password to set new password in DB
|
||||
# - Redis rotator: Sets new password via CONFIG SET
|
||||
#
|
||||
# T+2min: Reloader detects secret changes
|
||||
# - Triggers rolling restart of myapp Deployment
|
||||
# - Pods start with new credentials
|
||||
#
|
||||
# T+1hour: Cleanup
|
||||
# - postgres-creds-previous automatically deleted
|
||||
# - Old credentials no longer accessible
|
||||
#
|
||||
# ============================================================================
|
||||
# Monitoring Commands
|
||||
# ============================================================================
|
||||
#
|
||||
# # Watch all secrets
|
||||
# kubectl get secrets -n myapp -w
|
||||
#
|
||||
# # Check rotation status
|
||||
# kubectl get managedsecrets -n myapp
|
||||
#
|
||||
# # View rotation logs
|
||||
# kubectl logs -n myapp -l job-name=postgres-password-rotator-XXXXX
|
||||
# kubectl logs -n myapp -l job-name=redis-password-rotator-XXXXX
|
||||
#
|
||||
# # Check Reloader activity
|
||||
# kubectl logs -n reloader -l app=reloader | grep myapp
|
||||
#
|
||||
# # Verify app pods restarted
|
||||
# kubectl get pods -n myapp -o wide
|
||||
#
|
||||
# # Test connections with new credentials
|
||||
# kubectl exec -it -n myapp deployment/myapp -- sh
|
||||
# # psql -h $POSTGRES_HOST -U $POSTGRES_USER -d $POSTGRES_DB
|
||||
# # redis-cli -h $REDIS_HOST -a $REDIS_PASSWORD ping
|
||||
#
|
||||
# ============================================================================
|
||||
# Force Rotation for Testing
|
||||
# ============================================================================
|
||||
#
|
||||
# # Force PostgreSQL rotation
|
||||
# kubectl annotate managedsecret postgres-credentials -n myapp \
|
||||
# reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Force Redis rotation
|
||||
# kubectl annotate managedsecret redis-credentials -n myapp \
|
||||
# reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Force all rotations
|
||||
# kubectl annotate managedsecrets -n myapp --all \
|
||||
# reconcile="$(date +%s)" --overwrite
|
||||
#
|
||||
# # Watch the cascade of events
|
||||
# kubectl get events -n myapp --sort-by='.lastTimestamp' -w
|
||||
Reference in New Issue
Block a user