Kustomize: Complete Kubernetes Configuration Management Guide
```html
Kustomize: Complete Kubernetes Configuration Management Guide
```
Introduction to Kustomize
Kustomize is a standalone tool for Kubernetes configuration customization that uses a template-free approach. It's now built directly into kubectl (since v1.14) and follows the Kubernetes philosophy of declarative configuration management.
Core Philosophy:
- Template-free: No new syntax to learn, uses pure YAML
- Declarative: Describes desired state, not processes
- Overlay-based: Layers configurations for different environments
- Git-friendly: Works well with Git-based workflows
- Native: Integrated directly into kubectl
Why Use Kustomize?
Problems Solved by Kustomize:
| Problem | Traditional Solution | Kustomize Solution |
|---|---|---|
| Environment-specific configurations | Copy-paste YAML files | Overlay patching |
| Secret/ConfigMap management | Manual updates | Generators and transformers |
| Multiple environments | Multiple copies of files | Single source of truth |
| Common configurations | Duplicated code | Base configurations |
Benefits:
- No duplication: Single source of truth for manifests
- Easy maintenance: Changes in one place affect all environments
- GitOps ready: Perfect for Git-based workflows
- Kubernetes native: Uses standard Kubernetes YAML
- Simple learning curve: Minimal new concepts
Basic Concepts
1. Base
A directory containing standard Kubernetes manifests with a kustomization.yaml file. This is your source of truth.
2. Overlay
A directory that modifies the base configuration for a specific environment (dev, staging, prod).
3. Kustomization File
The kustomization.yaml file defines how resources should be customized.
4. Patch
A file or inline specification that modifies specific fields in resources.
5. Generator
Creates resources (like ConfigMaps or Secrets) from files or literals.
6. Transformer
Modifies resources after generation (like adding labels or changing names).
Project Structure:
├── base/
│ ├── kustomization.yaml
│ ├── deployment.yaml
│ └── service.yaml
├── overlays/
│ ├── dev/
│ │ ├── kustomization.yaml
│ │ └── patch.yaml
│ ├── staging/
│ │ ├── kustomization.yaml
│ │ └── patch.yaml
│ └── production/
│ ├── kustomization.yaml
│ └── patch.yaml
Installation & Setup
Install Kustomize Standalone:
# macOS with Homebrew
brew install kustomize
# Linux
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
sudo mv kustomize /usr/local/bin/
# Windows with Chocolatey
choco install kustomize
# Verify installation
kustomize version
# Built-in kubectl (v1.14+)
kubectl kustomize --help
IDE/Editor Support:
# VS Code extensions
- Kubernetes
- YAML
- Kustomize
# IntelliJ/IDEA plugins
- Kubernetes
- Kustomize Support
# Create aliases for convenience
alias kk="kubectl kustomize"
alias kkapply="kubectl apply -k"
Basic Usage
Simple Base Configuration:
# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-deployment
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: app
image: myapp:latest
ports:
- containerPort: 8080
env:
- name: ENVIRONMENT
value: "base"
# base/service.yaml
apiVersion: v1
kind: Service
metadata:
name: app-service
spec:
selector:
app: myapp
ports:
- port: 80
targetPort: 8080
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
commonLabels:
app: myapp
managed-by: kustomize
# Build and view
kustomize build base/
# or with kubectl
kubectl kustomize base/
Creating an Overlay:
# overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Reference the base
bases:
- ../../base
# Override namespace
namespace: dev-namespace
# Add name prefix/suffix
namePrefix: dev-
nameSuffix: -v1
# Change replica count
patchesStrategicMerge:
- deployment-patch.yaml
# Add configMap
configMapGenerator:
- name: app-config
files:
- config.properties
# Add secret
secretGenerator:
- name: app-secret
literals:
- username=admin
- password=secret
# Build dev overlay
kustomize build overlays/dev/
# Apply directly
kubectl apply -k overlays/dev/
Advanced Features
Strategic Merge Patches:
# overlays/dev/deployment-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app-deployment
spec:
replicas: 3 # Override from base (2 -> 3)
template:
spec:
containers:
- name: app
env:
- name: ENVIRONMENT
value: "development"
- name: DEBUG
value: "true"
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
# Using JSON Patch
patchesJson6902:
- target:
group: apps
version: v1
kind: Deployment
name: app-deployment
patch: |-
- op: replace
path: /spec/replicas
value: 5
- op: add
path: /spec/template/spec/containers/0/env/-
value:
name: LOG_LEVEL
value: debug
Generators:
# ConfigMap Generator from files
configMapGenerator:
- name: app-config
files:
- configs/database.properties
- configs/application.properties
envs:
- .env
literals:
- JAVA_OPTS=-Xmx512m
- SPRING_PROFILES_ACTIVE=dev
# Secret Generator
secretGenerator:
- name: db-secret
literals:
- db-password=supersecret
- api-key=abcdef123456
type: Opaque
# Generated files look like this (app-config-k25hdh4d5t):
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config-k25hdh4d5t
data:
database.properties: |
db.url=jdbc:postgresql://localhost:5432/mydb
db.username=admin
application.properties: |
server.port=8080
logging.level.root=INFO
JAVA_OPTS: -Xmx512m
SPRING_PROFILES_ACTIVE: dev
Transformers:
# transformers.yaml
apiVersion: builtin
kind: LabelTransformer
metadata:
name: not-important
labels:
environment: production
tier: backend
fieldSpecs:
- path: metadata/labels
create: true
# images.yaml - Image transformer
apiVersion: builtin
kind: ImageTagTransformer
metadata:
name: not-important
imageTag:
name: myapp
newTag: v2.0.0
# kustomization.yaml with transformers
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
transformers:
- transformers.yaml
- images.yaml
# Common annotations
commonAnnotations:
maintained-by: devops-team
git-revision: abc123
deployed-at: "2024-01-18"
# Name transformations
namePrefix: prod-
nameSuffix: -v2
Variables (vars) - Replacing text:
# vars.yaml
apiVersion: builtin
kind: Var
metadata:
name: appImage
objref:
apiVersion: apps/v1
kind: Deployment
name: app-deployment
fieldref:
fieldpath: spec.template.spec.containers[0].image
# kustomization.yaml with vars
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
vars:
- name: APP_IMAGE
objref:
kind: Deployment
name: app-deployment
apiVersion: apps/v1
fieldref:
fieldpath: spec.template.spec.containers[0].image
# Using vars in other resources
configMapGenerator:
- name: app-config
literals:
- image=$APP_IMAGE
- version=v1.0.0
Real-World Examples
Multi-Environment Setup:
# Project structure
kubernetes/
├── base/
│ ├── namespace.yaml
│ ├── deployment.yaml
│ ├── service.yaml
│ ├── ingress.yaml
│ ├── configmap.yaml
│ └── kustomization.yaml
├── components/
│ ├── monitoring/
│ │ ├── service-monitor.yaml
│ │ └── kustomization.yaml
│ └── autoscaling/
│ ├── hpa.yaml
│ └── kustomization.yaml
└── overlays/
├── dev/
│ ├── kustomization.yaml
│ └── patches/
│ ├── deployment-patch.yaml
│ └── config-patch.yaml
├── staging/
│ ├── kustomization.yaml
│ └── patches/
│ ├── deployment-patch.yaml
│ └── ingress-patch.yaml
└── production/
├── kustomization.yaml
├── patches/
│ ├── deployment-patch.yaml
│ ├── hpa-patch.yaml
│ └── pdb-patch.yaml
└── secrets/
├── db-secret.yaml
└── api-secret.yaml
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: myapp
resources:
- namespace.yaml
- deployment.yaml
- service.yaml
- ingress.yaml
- configmap.yaml
- ../components/monitoring
- ../components/autoscaling
commonLabels:
app: myapp
part-of: myapp-system
# overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../base
namespace: production
namePrefix: prod-
replicas:
- name: app-deployment
count: 5
images:
- name: myapp
newName: myregistry.azurecr.io/myapp
newTag: v1.2.3
patchesStrategicMerge:
- patches/deployment-patch.yaml
- patches/hpa-patch.yaml
- patches/pdb-patch.yaml
secretGenerator:
- name: db-secret
files:
- secrets/db-password.txt
type: Opaque
configMapGenerator:
- name: app-config
envs:
- .env.production
transformers:
- ../../transformers/common-transformer.yaml
Database Application with Configurations:
# base/postgres/
├── kustomization.yaml
├── deployment.yaml
├── service.yaml
├── configmap.yaml
└── pvc.yaml
# base/postgres/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml
- configmap.yaml
- pvc.yaml
configMapGenerator:
- name: postgres-config
files:
- postgresql.conf
- pg_hba.conf
# overlays/dev/postgres/
├── kustomization.yaml
├── patches/
│ └── deployment-patch.yaml
└── secrets/
└── postgres-secret.yaml
# overlays/dev/postgres/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../../base/postgres
namespace: dev-database
patchesStrategicMerge:
- patches/deployment-patch.yaml
secretGenerator:
- name: postgres-secret
literals:
- POSTGRES_PASSWORD=devpassword123
- POSTGRES_USER=devuser
# overlays/production/postgres/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../../base/postgres
namespace: production-database
patchesStrategicMerge:
- patches/deployment-patch.yaml
- patches/ha-patch.yaml
secretGenerator:
- name: postgres-secret
files:
- secrets/postgres-password.txt
- secrets/postgres-user.txt
type: Opaque
GitOps with Kustomize
Flux v2 with Kustomize:
# Flux Kustomization for GitOps
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: app-production
namespace: flux-system
spec:
interval: 5m
path: ./kubernetes/overlays/production
prune: true
sourceRef:
kind: GitRepository
name: flux-system
validation: client
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: app-deployment
namespace: production
- apiVersion: v1
kind: Service
name: app-service
namespace: production
# ArgoCD Application with Kustomize
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp-production
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/myorg/myapp.git
targetRevision: HEAD
path: kubernetes/overlays/production
kustomize:
namePrefix: prod-
images:
- myapp:v1.2.3
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
Git Repository Structure for GitOps:
gitops-repository/
├── apps/
│ ├── myapp/
│ │ ├── base/
│ │ │ ├── kustomization.yaml
│ │ │ ├── deployment.yaml
│ │ │ └── service.yaml
│ │ └── overlays/
│ │ ├── dev/
│ │ │ └── kustomization.yaml
│ │ ├── staging/
│ │ │ └── kustomization.yaml
│ │ └── production/
│ │ └── kustomization.yaml
│ └── database/
│ ├── base/
│ └── overlays/
├── clusters/
│ ├── dev-cluster/
│ │ └── kustomization.yaml
│ ├── staging-cluster/
│ │ └── kustomization.yaml
│ └── production-cluster/
│ └── kustomization.yaml
└── infrastructure/
├── monitoring/
├── networking/
└── storage/
# clusters/production-cluster/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../apps/myapp/overlays/production
- ../../apps/database/overlays/production
- ../../infrastructure/monitoring
- ../../infrastructure/networking
- ../../infrastructure/storage
commonLabels:
cluster: production
environment: prod
CI/CD Integration
GitHub Actions Workflow:
# .github/workflows/deploy.yml
name: Deploy with Kustomize
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test-kustomize:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Validate base configuration
run: |
kustomize build kubernetes/base/ --load-restrictor LoadRestrictionsNone
- name: Validate overlays
run: |
for env in dev staging production; do
echo "Validating $env overlay..."
kustomize build kubernetes/overlays/$env/ --load-restrictor LoadRestrictionsNone
done
deploy-dev:
needs: test-kustomize
if: github.ref == 'refs/heads/develop'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Configure kubectl
uses: azure/setup-kubectl@v3
with:
version: 'v1.26.0'
- name: Deploy to dev
run: |
kubectl apply -k kubernetes/overlays/dev/
env:
KUBECONFIG: ${{ secrets.KUBECONFIG_DEV }}
deploy-production:
needs: test-kustomize
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Update image tag in kustomization
run: |
cd kubernetes/overlays/production
kustomize edit set image myapp=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
- name: Preview changes
run: |
kustomize build kubernetes/overlays/production/ --load-restrictor LoadRestrictionsNone
- name: Deploy to production
run: |
kubectl apply -k kubernetes/overlays/production/
env:
KUBECONFIG: ${{ secrets.KUBECONFIG_PROD }}
Jenkins Pipeline:
// Jenkinsfile
pipeline {
agent any
environment {
KUSTOMIZE_VERSION = '4.5.7'
KUBECONFIG = credentials('kubeconfig')
}
stages {
stage('Setup') {
steps {
sh '''
# Install kustomize
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
sudo mv kustomize /usr/local/bin/
'''
}
}
stage('Validate') {
steps {
sh '''
# Validate all configurations
kustomize build kubernetes/base/ --load-restrictor LoadRestrictionsNone
# Validate specific environment
ENVIRONMENT="${ENVIRONMENT:-dev}"
kustomize build kubernetes/overlays/$ENVIRONMENT/ --load-restrictor LoadRestrictionsNone
'''
}
}
stage('Build Overlay') {
steps {
script {
// Update image tag
sh '''
cd kubernetes/overlays/$ENVIRONMENT
kustomize edit set image myapp=$REGISTRY/$IMAGE_NAME:$BUILD_TAG
'''
}
}
}
stage('Deploy') {
steps {
sh '''
# Apply using kustomize
kubectl apply -k kubernetes/overlays/$ENVIRONMENT/
# Wait for rollout
kubectl rollout status deployment/myapp-deployment -n $NAMESPACE --timeout=300s
'''
}
}
stage('Smoke Test') {
steps {
sh '''
# Test deployment
kubectl run test --rm -i --restart=Never \
--image=alpine/curl:latest \
-- curl -f http://myapp-service/
'''
}
}
}
}
Best Practices
Organization Structure:
- Single base per application: One source of truth
- Environment-specific overlays: Separate overlays for dev/staging/prod
- Component-based structure: Group related resources together
- Common configurations: Use components for shared configurations
Naming Conventions:
# Good naming examples
- base/myapp/
- overlays/dev/myapp/
- overlays/production/myapp/
- components/database/
- components/monitoring/
# Resource naming
namePrefix: dev- # for dev environment
nameSuffix: -v1 # for versioning
# Label conventions
commonLabels:
app: myapp
environment: ${ENVIRONMENT}
version: ${VERSION}
managed-by: kustomize
Security Practices:
- Never commit secrets: Use secretGenerator with literals/files
- Use .gitignore: Exclude generated files and secrets
- Validate configurations: Use
kustomize build --load-restrictor LoadRestrictionsNone - Review generated manifests: Always review before applying
Performance Optimization:
- Use strategic merge patches: Instead of JSON patches when possible
- Limit patch files: Keep patches focused and minimal
- Cache base builds: In CI/CD pipelines
- Use components: For reusable configurations
Comparison with Alternatives
| Tool | Approach | Strengths | Weaknesses | Best For |
|---|---|---|---|---|
| Kustomize | Template-free, overlay-based | Kubernetes native, simple, built into kubectl | Limited templating capabilities | Multi-environment management, GitOps |
| Helm | Templating engine with Go templates | Powerful templating, package management | Complex templates, steep learning curve | Application packaging, distribution |
| Jsonnet | Data templating language | Very powerful, programmatic | Complex, not Kubernetes-specific | Complex configurations, multi-cloud |
| cdk8s | Programming languages (TypeScript, Python, etc.) | Type-safe, familiar languages | Requires programming knowledge | Developers, complex logic |
When to Use Kustomize:
- Managing multiple environments (dev/staging/prod)
- Simple customization needs
- GitOps workflows
- Teams already familiar with Kubernetes YAML
- Need native kubectl integration
When Not to Use Kustomize:
- Complex templating needs
- Application packaging for distribution
- Need conditional logic in templates
- Complex variable substitution
Comments
Post a Comment