Kubernetes in ConnectSoft Templates¶
Purpose & Overview¶
Kubernetes in ConnectSoft Templates provides container orchestration capabilities for deploying, scaling, and managing applications in production environments. Kubernetes enables automated deployment, scaling, load balancing, self-healing, and service discovery for containerized applications across all ConnectSoft templates and solutions.
Kubernetes provides:
- Container Orchestration: Automatic container lifecycle management
- Scaling: Horizontal and vertical scaling based on demand
- Service Discovery: Built-in DNS-based service discovery
- Load Balancing: Automatic load distribution across pods
- High Availability: Pod replication and failover
- Self-Healing: Automatic restart of failed containers
- Rolling Updates: Zero-downtime deployments
- Resource Management: CPU and memory limits and requests
- Configuration Management: ConfigMaps and Secrets for configuration
- Network Policies: Security and network isolation
Kubernetes Philosophy
Kubernetes treats containers as first-class citizens, providing a robust platform for deploying and managing applications at scale. ConnectSoft Templates provide production-ready Kubernetes manifests that follow best practices for security, reliability, and observability.
Architecture Overview¶
Kubernetes Components¶
Kubernetes Cluster
├── Control Plane
│ ├── API Server
│ ├── etcd
│ ├── Scheduler
│ └── Controller Manager
├── Worker Nodes
│ ├── Kubelet
│ ├── kube-proxy
│ └── Container Runtime
└── Application Workloads
├── Deployments
├── Services
├── Ingress
├── ConfigMaps
└── Secrets
Application Deployment Architecture¶
Kubernetes Namespace
├── Deployment (Application)
│ ├── ReplicaSet
│ │ └── Pods (3 replicas)
│ │ └── Container
│ │ ├── Application
│ │ ├── Health Checks
│ │ └── Resource Limits
├── Service (ClusterIP)
│ └── Load Balancer
├── Ingress
│ └── External Access
├── ConfigMap
│ └── Application Configuration
└── Secret
└── Sensitive Data
Deployment Flow¶
Container Image (Registry)
↓
Deployment Manifest
↓
Kubernetes API Server
↓
Scheduler (Assigns Pods to Nodes)
↓
Kubelet (Creates Pods)
↓
Container Runtime (Runs Containers)
↓
Service (Exposes Pods)
↓
Ingress (External Access)
Deployment Manifest¶
Basic Deployment¶
File: application-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: application-deployment
labels:
app: application-service
spec:
replicas: 1
selector:
matchLabels:
app: application-service
template:
metadata:
labels:
app: application-service
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
containers:
- name: application-container
image: "registry.example.com/application:1.0.0"
imagePullPolicy: Always
env:
- name: ASPNETCORE_ENVIRONMENT
value: "Production"
- name: ASPNETCORE_URLS
value: "http://+:8081;https://+:7279"
ports:
- name: http
containerPort: 8081
protocol: TCP
- name: https
containerPort: 7279
protocol: TCP
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "256Mi"
cpu: "500m"
livenessProbe:
httpGet:
path: /live
port: 8081
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /ready
port: 8081
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
volumeMounts:
- name: config-volume
mountPath: /app/config
readOnly: true
volumes:
- name: config-volume
configMap:
name: app-config
Key Deployment Components¶
Metadata:
- name: Deployment name (must be unique in namespace)
- labels: Labels for resource selection and organization
Spec:
- replicas: Number of pod replicas (default: 1)
- selector: Labels to match pods managed by this deployment
- template: Pod template specification
Pod Template:
- securityContext: Security settings (run as non-root user)
- containers: Container specifications
- volumes: Volumes to mount in pods
Container Configuration¶
Image:
Image Pull Policies:
- Always: Always pull image (latest)
- IfNotPresent: Pull only if not present locally
- Never: Never pull (use local only)
Environment Variables:
env:
- name: ASPNETCORE_ENVIRONMENT
value: "Production"
- name: ASPNETCORE_URLS
value: "http://+:8081;https://+:7279"
- name: ConnectionStrings__DefaultConnection
valueFrom:
secretKeyRef:
name: microservice-secrets
key: ConnectionStrings__DefaultConnection
Ports:
ports:
- name: http
containerPort: 8081
protocol: TCP
- name: https
containerPort: 7279
protocol: TCP
Health Checks¶
Liveness Probe¶
Purpose: Determines if container is running. If probe fails, Kubernetes restarts the container.
livenessProbe:
httpGet:
path: /live
port: 8081
scheme: HTTP
initialDelaySeconds: 30 # Wait 30s before first probe
periodSeconds: 10 # Check every 10s
timeoutSeconds: 5 # Timeout after 5s
failureThreshold: 3 # Restart after 3 failures
successThreshold: 1 # 1 success = healthy
Configuration Guidelines:
- initialDelaySeconds: Should be greater than application startup time
- periodSeconds: Frequency of checks (10-30 seconds typical)
- failureThreshold: Number of failures before restart (3-5 typical)
Readiness Probe¶
Purpose: Determines if container is ready to serve traffic. If probe fails, Kubernetes removes pod from service endpoints.
readinessProbe:
httpGet:
path: /ready
port: 8081
scheme: HTTP
initialDelaySeconds: 10 # Wait 10s before first probe
periodSeconds: 5 # Check every 5s
timeoutSeconds: 3 # Timeout after 3s
failureThreshold: 3 # Mark unready after 3 failures
successThreshold: 1 # 1 success = ready
Difference from Liveness: - Liveness: Restarts container if unhealthy - Readiness: Removes from load balancer if not ready
Startup Probe¶
Purpose: Indicates that container has started. Useful for slow-starting applications.
startupProbe:
httpGet:
path: /health
port: 8081
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 30 # Allow up to 150s for startup (30 * 5s)
successThreshold: 1
When to Use: - Applications with long startup times - Prevents premature liveness probe failures - Allows graceful initialization
See Startup and Warmup for detailed startup warmup patterns and mechanisms.
Resource Management¶
Resource Requests and Limits¶
Requests: Minimum resources guaranteed to container
Limits: Maximum resources allowed for container
Resource Units¶
Memory:
- Mi: Mebibytes (1024^2 bytes)
- Gi: Gibibytes (1024^3 bytes)
- M: Megabytes (1000^2 bytes)
- G: Gigabytes (1000^3 bytes)
CPU:
- m: Millicores (1000m = 1 core)
- 250m = 0.25 cores
- 500m = 0.5 cores
- 1000m = 1 core
Resource Sizing Guidelines¶
Small Workload:
Medium Workload:
Large Workload:
Resource Quotas¶
Namespace Resource Quota:
apiVersion: v1
kind: ResourceQuota
metadata:
name: application-quota
spec:
hard:
requests.cpu: "4"
requests.memory: "8Gi"
limits.cpu: "8"
limits.memory: "16Gi"
Service¶
ClusterIP Service¶
Purpose: Exposes pods internally within cluster.
apiVersion: v1
kind: Service
metadata:
name: application-service
labels:
app: application-service
spec:
type: ClusterIP
selector:
app: application-service
ports:
- name: http
port: 80
targetPort: 8081
protocol: TCP
- name: https
port: 443
targetPort: 7279
protocol: TCP
Service Discovery:
- DNS name: application-service.namespace.svc.cluster.local
- Short name: application-service
- Namespace-scoped: application-service.namespace
Service Types¶
ClusterIP (default): - Internal cluster access only - No external access
NodePort:
spec:
type: NodePort
ports:
- port: 80
targetPort: 8081
nodePort: 30080 # Accessible on <NodeIP>:30080
LoadBalancer:
ExternalName:
Ingress¶
Basic Ingress¶
Purpose: Exposes HTTP/HTTPS routes from outside cluster to services.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: application-ingress
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
tls:
- hosts:
- application.example.com
secretName: application-tls
rules:
- host: application.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: application-service
port:
number: 80
Ingress Controllers¶
NGINX Ingress Controller:
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/rate-limit: "100"
Traefik:
annotations:
traefik.ingress.kubernetes.io/rule-type: PathPrefix
traefik.ingress.kubernetes.io/ssl-redirect: "true"
TLS/HTTPS¶
TLS Configuration:
Certificate Management: - cert-manager: Automatic certificate provisioning from Let's Encrypt - Manual: Create TLS secret with certificate
cert-manager Integration:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: application-tls
spec:
secretName: application-tls
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- application.example.com
ConfigMap¶
Creating ConfigMap¶
From Literal Values:
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
ASPNETCORE_ENVIRONMENT: "Production"
Kestrel__Endpoints__Http__Url: "http://+:8081"
From File:
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
data:
appsettings.json: |
{
"Kestrel": {
"Endpoints": {
"Http": {
"Url": "http://+:8081"
}
}
}
}
From Multiple Files:
kubectl create configmap app-config \
--from-file=appsettings.json \
--from-file=appsettings.Production.json
Using ConfigMap¶
As Environment Variables:
As Individual Environment Variables:
spec:
containers:
- name: microservice
env:
- name: ASPNETCORE_ENVIRONMENT
valueFrom:
configMapKeyRef:
name: app-config
key: ASPNETCORE_ENVIRONMENT
As Volume Mount:
spec:
containers:
- name: microservice
volumeMounts:
- name: config-volume
mountPath: /app/config
volumes:
- name: config-volume
configMap:
name: app-config
Secrets¶
Creating Secrets¶
From Literal Values:
kubectl create secret generic microservice-secrets \
--from-literal=ConnectionStrings__DefaultConnection="Server=...;Database=...;..."
From File:
YAML Manifest:
apiVersion: v1
kind: Secret
metadata:
name: microservice-secrets
type: Opaque
stringData:
ConnectionStrings__DefaultConnection: "Server=...;Database=...;..."
ApiKey: "secret-api-key"
⚠️ Important: Never commit secrets to source control. Use sealed secrets or external secret managers.
Using Secrets¶
As Environment Variables:
As Individual Environment Variables:
spec:
containers:
- name: microservice
env:
- name: ConnectionStrings__DefaultConnection
valueFrom:
secretKeyRef:
name: microservice-secrets
key: ConnectionStrings__DefaultConnection
As Volume Mount:
spec:
containers:
- name: microservice
volumeMounts:
- name: secrets-volume
mountPath: /app/secrets
readOnly: true
volumes:
- name: secrets-volume
secret:
secretName: microservice-secrets
External Secret Management¶
Azure Key Vault (via External Secrets Operator):
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: microservice-secrets
spec:
secretStoreRef:
name: azure-keyvault
kind: SecretStore
target:
name: microservice-secrets
data:
- secretKey: ConnectionStrings__DefaultConnection
remoteRef:
key: microservice-connection-string
HashiCorp Vault:
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: microservice-secrets
spec:
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: microservice-secrets
data:
- secretKey: ApiKey
remoteRef:
key: secret/data/microservice
property: api_key
Scaling¶
Manual Scaling¶
Scale Deployment:
Update Replicas in Manifest:
Horizontal Pod Autoscaler (HPA)¶
CPU-Based Scaling:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: application-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: application-deployment
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 50
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 60
- type: Pods
value: 2
periodSeconds: 60
selectPolicy: Max
Custom Metrics Scaling:
metrics:
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "100"
Vertical Pod Autoscaler (VPA)¶
VPA Configuration:
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: application-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: application-deployment
updatePolicy:
updateMode: "Auto"
resourcePolicy:
containerPolicies:
- containerName: application-container
minAllowed:
cpu: "100m"
memory: "64Mi"
maxAllowed:
cpu: "2000m"
memory: "4Gi"
Rolling Updates¶
Update Strategy¶
Rolling Update (default):
Recreate:
Rolling Update Parameters¶
maxSurge: Maximum number of pods that can be created above desired replicasmaxUnavailable: Maximum number of pods that can be unavailable during update
Example: - Replicas: 3 - maxSurge: 1 - maxUnavailable: 0 - During update: 4 pods (3 old + 1 new), then 3 pods (all new)
Deployment Rollout¶
View Rollout Status:
Rollout History:
Rollback:
kubectl rollout undo deployment/application-deployment
kubectl rollout undo deployment/application-deployment --to-revision=2
Blue-Green Deployment¶
Create Blue Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: application-blue
spec:
replicas: 3
selector:
matchLabels:
app: application
version: blue
template:
metadata:
labels:
app: application
version: blue
Create Green Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: application-green
spec:
replicas: 3
selector:
matchLabels:
app: application
version: green
template:
metadata:
labels:
app: application
version: green
Switch Service:
apiVersion: v1
kind: Service
metadata:
name: application-service
spec:
selector:
app: application
version: green # Switch from blue to green
Security¶
Pod Security Context¶
Container Security Context:
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
seccompProfile:
type: RuntimeDefault
containers:
- name: microservice
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
Pod Security Standards¶
Pod Security Admission:
apiVersion: v1
kind: Namespace
metadata:
name: application
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted
Security Levels:
- privileged: No restrictions
- baseline: Prevents known privilege escalations
- restricted: Most restrictive (recommended)
Network Policies¶
Ingress Network Policy:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: application-network-policy
spec:
podSelector:
matchLabels:
app: application-service
policyTypes:
- Ingress
- Egress
ingress:
- from:
- namespaceSelector:
matchLabels:
name: frontend
- podSelector:
matchLabels:
app: api-gateway
ports:
- protocol: TCP
port: 8081
egress:
- to:
- podSelector:
matchLabels:
app: database
ports:
- protocol: TCP
port: 1433
Service Account¶
Service Account:
Use in Deployment:
Observability¶
Logging¶
View Pod Logs:
Log Aggregation: - Fluentd / Fluent Bit: Collect logs from pods - Elasticsearch: Store logs - Kibana: Visualize logs
Metrics¶
Prometheus Metrics:
ServiceMonitor (Prometheus Operator):
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: application-metrics
spec:
selector:
matchLabels:
app: application-service
endpoints:
- port: http
path: /metrics
Tracing¶
OpenTelemetry Integration: - Sidecar: Deploy OpenTelemetry Collector as sidecar - DaemonSet: Deploy OpenTelemetry Collector as DaemonSet - Application: Direct export to OpenTelemetry endpoint
CI/CD Integration¶
Deployment Pipeline¶
GitHub Actions:
name: Deploy to Kubernetes
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up kubectl
uses: azure/setup-kubectl@v3
- name: Configure kubectl
run: |
echo "${{ secrets.KUBECONFIG }}" | base64 -d > kubeconfig
export KUBECONFIG=kubeconfig
- name: Deploy to Kubernetes
run: |
kubectl apply -f Deployment/kubernetes/
kubectl rollout status deployment/application-deployment
Azure DevOps:
- task: Kubernetes@1
displayName: 'Deploy to Kubernetes'
inputs:
connectionType: 'Kubernetes Service Connection'
kubernetesServiceEndpoint: 'my-k8s-connection'
namespace: 'default'
command: 'apply'
arguments: '-f Deployment/kubernetes/'
GitOps with ArgoCD¶
ArgoCD Application:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: application
spec:
project: default
source:
repoURL: https://github.com/org/repo
targetRevision: main
path: Deployment/kubernetes
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
Best Practices¶
Do's¶
-
Use Resource Limits
-
Implement Health Checks
-
Use Non-Root User
-
Version Images
-
Use Secrets for Sensitive Data
Don'ts¶
-
Don't Use Latest Tag
-
Don't Store Secrets in ConfigMap
-
Don't Run as Root
-
Don't Skip Resource Limits
Troubleshooting¶
Pod Not Starting¶
Check Pod Status:
Common Issues: - Image pull errors: Check image name and registry credentials - CrashLoopBackOff: Check application logs and startup probes - Pending: Check resource requests vs available resources
Service Not Accessible¶
Check Service:
Common Issues: - No endpoints: Pods not matching service selector - Port mismatch: Service port doesn't match container port - Network policy: Network policy blocking traffic
Configuration Issues¶
Check ConfigMap/Secret:
kubectl get configmap <configmap-name> -o yaml
kubectl get secret <secret-name> -o yaml
kubectl exec <pod-name> -- env | grep <env-var>
Summary¶
Kubernetes in ConnectSoft Templates provides:
- ✅ Deployment Management: Automated container deployment and lifecycle
- ✅ Scaling: Horizontal and vertical scaling with HPA/VPA
- ✅ Service Discovery: DNS-based service discovery
- ✅ Load Balancing: Automatic load distribution
- ✅ Health Checks: Liveness, readiness, and startup probes
- ✅ Configuration: ConfigMaps and Secrets for configuration management
- ✅ Security: Pod security contexts, network policies, service accounts
- ✅ Rolling Updates: Zero-downtime deployments
- ✅ Observability: Logging, metrics, and tracing integration
- ✅ CI/CD Integration: Automated deployment pipelines
By following these patterns, teams can:
- Deploy Reliably: Automated deployments with health checks and rollbacks
- Scale Automatically: Respond to demand with autoscaling
- Maintain Security: Follow security best practices and standards
- Monitor Effectively: Integrate with observability tools
- Operate Efficiently: Use Kubernetes features for operational excellence
Kubernetes provides a robust, production-ready platform for deploying and managing applications at scale, enabling teams to build reliable, scalable, and maintainable distributed systems across all ConnectSoft templates and solutions.
Related Documentation¶
- Startup and Warmup: Startup warmup patterns and Kubernetes startup probe integration
- Health Checks: Health check configuration and Kubernetes probe integration
- Containerization: Container deployment strategies
- Docker Compose: Local development with Docker Compose
- Infrastructure as Code: Infrastructure provisioning patterns