Skip to content

Docker Compose in ConnectSoft Microservice Template

Purpose & Overview

Docker Compose in the ConnectSoft Microservice Template provides a comprehensive multi-container orchestration solution for local development, testing, and CI/CD environments. Docker Compose enables developers to define and run complete application stacks with all dependencies (databases, messaging, observability) in a single command, ensuring consistent environments across the development lifecycle.

Docker Compose in the template provides:

  • Multi-Container Orchestration: Manage application and all dependencies together
  • Service Dependencies: Automatic dependency resolution and startup ordering
  • Network Isolation: Dedicated network for service communication
  • Volume Management: Persistent data storage for databases and certificates
  • Environment Configuration: Environment-specific overrides and configurations
  • Local Development: Simplified local development setup
  • CI/CD Integration: Automated testing and deployment workflows
  • Visual Studio Integration: Native Visual Studio support for Docker Compose projects

Docker Compose Philosophy

Docker Compose provides a developer-friendly way to run the entire microservice stack locally. All services, dependencies, and observability tools are orchestrated together, enabling developers to work with a production-like environment on their local machines. The template supports multiple compose files for different scenarios (development, deployment, CI/CD), ensuring flexibility while maintaining consistency.

Architecture Overview

Docker Compose Stack Architecture

Docker Compose Stack
├── Application Service
│   └── ConnectSoft.MicroserviceTemplate.Application
├── Database Services
│   ├── SQL Server (NHibernate)
│   ├── MongoDB (MongoDB Persistence)
│   └── Redis (Caching, SignalR Backplane)
├── Messaging Services
│   └── RabbitMQ (MassTransit)
├── Observability Services
│   ├── OpenTelemetry Collector
│   ├── Prometheus (Metrics)
│   ├── Grafana (Visualization)
│   ├── Jaeger (Tracing)
│   ├── Elasticsearch (Logs)
│   └── Kibana (Log Analysis)
├── Certificate Service
│   └── Certificate Generator (HTTPS)
└── MCP Inspector (Optional)
    └── Model Context Protocol Inspector

Service Dependencies

Application Service
    ├── depends_on: sql
    ├── depends_on: redis
    ├── depends_on: mongo
    ├── depends_on: rabbitmq
    ├── depends_on: otel-collector
    ├── depends_on: prometheus
    ├── depends_on: grafana
    ├── depends_on: elasticsearch
    ├── depends_on: kibana
    └── depends_on: certgen

Observability Services
    ├── otel-collector → prometheus
    ├── otel-collector → jaeger
    ├── otel-collector → elasticsearch
    ├── grafana → prometheus
    ├── grafana → otel-collector
    ├── grafana → jaeger
    ├── grafana → elasticsearch
    └── kibana → elasticsearch

Network Architecture

Docker Network (backend)
├── Application Container
│   └── connectsoft.microservicetemplate.application
├── Database Containers
│   ├── sql (1433)
│   ├── mongo (27017)
│   └── redis (6379)
├── Messaging Container
│   └── rabbitmq (5672, 15672)
├── Observability Containers
│   ├── otel-collector (4317, 4318, 8888)
│   ├── prometheus (9090)
│   ├── grafana (3000)
│   ├── jaeger (14250, 16686)
│   ├── elasticsearch (9200)
│   └── kibana (5601)
└── Certificate Generator
    └── certgen

Project Structure

Docker Compose Project

The template includes a dedicated Docker Compose project:

ConnectSoft.MicroserviceTemplate.DockerCompose/
├── docker-compose.yml              # Main compose file
├── docker-compose.override.yml     # Development overrides
├── ConnectSoft.MicroserviceTemplate.DockerCompose.dcproj  # Visual Studio project
└── launchSettings.json             # Launch settings

Deployment Docker Compose Files

Additional compose files for deployment scenarios:

Deployment/docker-compose/
├── docker-compose.yml              # Deployment compose file
└── docker-compose-ci.yml           # CI/CD compose file

Main Docker Compose File

docker-compose.yml

The main Docker Compose file defines all services and their configurations:

services:
  # Application Service
  connectsoft.microservicetemplate.application:
    image: ${DOCKER_REGISTRY-}connectsoft.microservicetemplate.application
    container_name: connectsoft.microservicetemplate.application
    build:
      context: .
      dockerfile: ../ConnectSoft.MicroserviceTemplate.Application/Dockerfile
    depends_on:
      - sql
      - redis
      - mongo
      - otel-collector
      - rabbitmq
      - prometheus
      - grafana
      - elasticsearch
      - kibana
    networks:
      - backend

  # Certificate Generator
  certgen:
    image: mcr.microsoft.com/dotnet/sdk:8.0
    container_name: certgen
    entrypoint: ["bash","-lc","dotnet dev-certs https -ep /out/aspnetapp.pfx -p 123456"]
    volumes:
      - httpscerts:/out
    restart: "no"

  # SQL Server
  sql:
    image: "mcr.microsoft.com/mssql/server:2022-latest"
    container_name: sql
    ports:
      - "1433:1433"
    environment:
      - ACCEPT_EULA=y
      - MSSQL_SA_PASSWORD=ConnectSoft123!
      - MSSQL_MEMORY_LIMIT_MB=4096
      - MSSQL_PID=Developer
    networks:
      - backend

  # Redis
  redis:
    image: redis
    container_name: redis
    ports:
      - "60003:60002"
    networks:
      - backend

  # MongoDB
  mongo:
    image: mongo:latest
    container_name: mongo
    ports:
      - "60001:27017"
    networks:
      - backend

  # RabbitMQ
  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    command: ["bash","-lc","rabbitmq-plugins enable --offline rabbitmq_prometheus && docker-entrypoint.sh rabbitmq-server"]
    ports:
      - "5672:5672"      # AMQP port
      - "15672:15672"    # Management UI
      - "15692:15692"    # Prometheus metrics
    environment:
      - RABBITMQ_DEFAULT_USER=dimatest
      - RABBITMQ_DEFAULT_PASS=testDima34
      - RABBITMQ_DEFAULT_VHOST=myhost
    networks:
      - backend

  # OpenTelemetry Collector
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    container_name: otel-collector
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ../Scripts/otel-collector/otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "8888:8888"   # Prometheus metrics
      - "8889:8889"   # Prometheus exporter metrics
      - "13133:13133" # Health check
      - "4317:4317"   # OTLP gRPC receiver
      - "4318:4318"   # OTLP HTTP receiver
      - "55679:55679" # zpages extension
    depends_on:
      - prometheus
      - jaeger
      - elasticsearch
    networks:
      - backend

  # Jaeger (Tracing)
  jaeger:
    image: jaegertracing/jaeger:2.10.0
    container_name: "jaeger"
    environment:
      - COLLECTOR_OTLP_ENABLED=true
    ports:
      - "16686:16686"   # Jaeger UI
      - "14250:14250"   # gRPC
      - "14268:14268"   # HTTP collector
    networks:
      - backend

  # Prometheus (Metrics)
  prometheus:
    container_name: prometheus
    build:
      context: ../Scripts/prometheus
    ports:
      - "9091:9090"
    networks:
      - backend

  # Grafana (Visualization)
  grafana:
    container_name: grafana
    build:
      context: ../Scripts/grafana
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
      - GF_AUTH_DISABLE_LOGIN_FORM=true
    depends_on:
      - prometheus
      - otel-collector
      - jaeger
      - elasticsearch
    ports:
      - 3000:3000
    networks:
      - backend

  # Elasticsearch (Logs)
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1
    container_name: elasticsearch
    hostname: elasticsearch
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - xpack.security.enabled=false
    ports:
      - "9200:9200"
    networks:
      - backend
    volumes:
      - esdata:/usr/share/elasticsearch/data

  # Kibana (Log Analysis)
  kibana:
    image: docker.elastic.co/kibana/kibana:8.16.1
    container_name: kibana
    hostname: kibana
    ports:
      - "5601:5601"
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
      - XPACK_SECURITY_ENABLED=false
    depends_on:
      - elasticsearch
    networks:
      - backend
    volumes:
      - esconfig:/usr/share/elasticsearch/config

  # MCP Inspector (Optional)
  mcp-inspector:
    image: node:20-alpine
    container_name: mcp-inspector
    working_dir: /work
    command: >
      sh -lc "CLIENT_PORT=6274 SERVER_PORT=6277 SERVER_HOST=0.0.0.0 npx --yes @modelcontextprotocol/inspector"
    ports:
      - "6274:6274"   # Inspector UI
      - "6277:6277"   # Inspector proxy
    depends_on:
      - connectsoft.microservicetemplate.application
    networks:
      - backend

networks:
  backend:
    driver: bridge
    name: connectsoft.microservicetemplate.application-network

volumes:
  esdata:
    driver: local
  esconfig:
    driver: local
  httpscerts: {}

Development Override File

docker-compose.override.yml

The override file provides development-specific configurations:

version: '3.8'

services:
  connectsoft.microservicetemplate.application:
    environment:
      - ASPNETCORE_ENVIRONMENT=Docker
      - ASPNETCORE_URLS=http://+:8081;https://+:7279
      - ASPNETCORE_Kestrel__Certificates__Default__Password=123456
      - ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
    # Wait until the cert is present, then start the app
    entrypoint:
      - /bin/sh
      - -lc
      - |
        until [ -f /https/aspnetapp.pfx ]; do
          echo "waiting for /https/aspnetapp.pfx...";
          sleep 0.5;
        done;
        exec dotnet ConnectSoft.MicroserviceTemplate.Application.dll
    ports:
      - "8081:8081"  # HTTP port
      - "7279:7279"  # HTTPS port
    healthcheck:
      test: ["CMD-SHELL", "curl -f https://127.0.0.1:7279/alive || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 20s
    depends_on:
      - certgen
    volumes:
      - httpscerts:/https:ro

Key Features: - Environment Variables: Sets ASPNETCORE_ENVIRONMENT=Docker - Port Mapping: Exposes HTTP (8081) and HTTPS (7279) ports - Certificate Wait: Waits for certificate to be generated before starting - Health Check: Configures container health check using /alive endpoint - Volume Mount: Mounts certificate volume as read-only

Service Configuration

Application Service

Configuration: - Image: Built from application Dockerfile - Network: Connected to backend network - Dependencies: All infrastructure services - Ports: HTTP (8081) and HTTPS (7279) - Health Check: Monitors /alive endpoint

Service Discovery: Services communicate using Docker Compose service names: - sql - SQL Server - mongo - MongoDB - redis - Redis - rabbitmq - RabbitMQ - otel-collector - OpenTelemetry Collector

Database Services

SQL Server: - Port: 1433 (exposed to host) - Memory: 4GB limit - Password: ConnectSoft123! - Connection: Server=sql,1433;Database=...

MongoDB: - Port: 60001 (host) → 27017 (container) - Connection: mongodb://mongo:27017/...

Redis: - Port: 60003 (host) → 60002 (container) - Connection: redis:60002

Messaging Service

RabbitMQ: - AMQP Port: 5672 - Management UI: 15672 (http://localhost:15672) - Prometheus Metrics: 15692 - Credentials: dimatest / testDima34 - Virtual Host: myhost - Connection: amqp://dimatest:testDima34@rabbitmq:5672/myhost

Observability Services

OpenTelemetry Collector: - OTLP gRPC: 4317 - OTLP HTTP: 4318 - Prometheus Metrics: 8888 - Health Check: 13133 - Configuration: Mounted from Scripts/otel-collector/otel-collector-config.yaml

Prometheus: - Port: 9091 (host) → 9090 (container) - UI: http://localhost:9091 - Configuration: Built from Scripts/prometheus

Grafana: - Port: 3000 - UI: http://localhost:3000 - Authentication: Anonymous enabled (development only) - Configuration: Built from Scripts/grafana

Jaeger: - UI: http://localhost:16686 - gRPC: 14250 - HTTP Collector: 14268

Elasticsearch: - Port: 9200 - Data Volume: esdata (persistent) - Security: Disabled (development only)

Kibana: - Port: 5601 - UI: http://localhost:5601 - Configuration Volume: esconfig

Certificate Generation

Certificate Generator Service

The certgen service generates HTTPS certificates for the application:

certgen:
  image: mcr.microsoft.com/dotnet/sdk:8.0
  container_name: certgen
  entrypoint: ["bash","-lc","dotnet dev-certs https -ep /out/aspnetapp.pfx -p 123456"]
  volumes:
    - httpscerts:/out
  restart: "no"

Process: 1. Certificate generator runs first 2. Generates aspnetapp.pfx with password 123456 3. Stores certificate in httpscerts volume 4. Application service waits for certificate before starting 5. Certificate mounted as read-only in application container

Certificate Usage: - Path: /https/aspnetapp.pfx - Password: 123456 - Configuration: Set via ASPNETCORE_Kestrel__Certificates__Default__Path

Networking

Network Configuration

Backend Network: - Driver: Bridge - Name: connectsoft.microservicetemplate.application-network - Isolation: All services in same network can communicate - DNS: Service names resolve to container IPs

Service Communication: - Services use service names as hostnames - Ports are exposed within the network - No need for host port mapping for inter-service communication

Example Connections: 

// SQL Server connection
var connectionString = "Server=sql,1433;Database=MyDb;User Id=sa;Password=ConnectSoft123!;";

// MongoDB connection
var mongoUrl = "mongodb://mongo:27017/MyDatabase";

// Redis connection
var redisConnection = "redis:60002";

// RabbitMQ connection
var rabbitMqUrl = "amqp://dimatest:testDima34@rabbitmq:5672/myhost";

Volumes

Volume Configuration

Persistent Volumes: - esdata: Elasticsearch data (persistent across restarts) - esconfig: Elasticsearch configuration - httpscerts: HTTPS certificates (shared between certgen and application)

Volume Usage: 

volumes:
  esdata:
    driver: local
  esconfig:
    driver: local
  httpscerts: {}

Volume Management: 

# List volumes
docker volume ls

# Inspect volume
docker volume inspect connectsoft_microservicetemplate_application-network_esdata

# Remove volumes (when cleaning up)
docker-compose down -v

Environment Configuration

Environment Variables

Application Environment: - ASPNETCORE_ENVIRONMENT=Docker - ASPNETCORE_URLS=http://+:8081;https://+:7279 - ASPNETCORE_Kestrel__Certificates__Default__Password=123456 - ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx

Database Environment: - SQL Server: MSSQL_SA_PASSWORD=ConnectSoft123! - RabbitMQ: RABBITMQ_DEFAULT_USER=dimatest, RABBITMQ_DEFAULT_PASS=testDima34

Observability Environment: - Grafana: GF_AUTH_ANONYMOUS_ENABLED=true - Elasticsearch: xpack.security.enabled=false

Configuration Files

appsettings.Docker.json: Loaded when ASPNETCORE_ENVIRONMENT=Docker:

{
  "Microservice": {
    "MicroserviceName": "ConnectSoft.MicroserviceTemplate",
    "StartupWarmupSeconds": 20
  },
  "ConnectionStrings": {
    "DefaultConnection": "Server=sql,1433;Database=MyDb;User Id=sa;Password=ConnectSoft123!;TrustServerCertificate=True;",
    "MongoDb": "mongodb://mongo:27017/MyDatabase",
    "Redis": "redis:60002"
  },
  "RabbitMQ": {
    "HostName": "rabbitmq",
    "Port": 5672,
    "UserName": "dimatest",
    "Password": "testDima34",
    "VirtualHost": "myhost"
  },
  "OpenTelemetry": {
    "OtlpEndpoint": "http://otel-collector:4317"
  }
}

Usage

Starting the Stack

Basic Start: 

# Navigate to Docker Compose project directory
cd ConnectSoft.MicroserviceTemplate.DockerCompose

# Start all services
docker-compose up

# Start in detached mode
docker-compose up -d

# Start specific services
docker-compose up sql redis mongo

With Override File: 

# Override file is automatically included
docker-compose up

# Explicitly specify files
docker-compose -f docker-compose.yml -f docker-compose.override.yml up

Stopping the Stack

# Stop all services
docker-compose down

# Stop and remove volumes
docker-compose down -v

# Stop specific services
docker-compose stop sql redis

Viewing Logs

# View all logs
docker-compose logs

# View logs for specific service
docker-compose logs connectsoft.microservicetemplate.application

# Follow logs
docker-compose logs -f

# View last 100 lines
docker-compose logs --tail=100

Building Images

# Build all images
docker-compose build

# Build specific service
docker-compose build connectsoft.microservicetemplate.application

# Build without cache
docker-compose build --no-cache

# Build and start
docker-compose up --build

Health Checks

# Check service status
docker-compose ps

# Check service health
docker inspect connectsoft.microservicetemplate.application | grep Health

# Test health endpoint
curl https://localhost:7279/alive

Visual Studio Integration

Docker Compose Project

The .dcproj file enables Visual Studio integration:

<Project ToolsVersion="15.0" Sdk="Microsoft.Docker.Sdk">
  <PropertyGroup Label="Globals">
    <DockerTargetOS>Linux</DockerTargetOS>
    <DockerLaunchAction>LaunchBrowser</DockerLaunchAction>
    <DockerServiceUrl>{Scheme}://localhost:{ServicePort}</DockerServiceUrl>
    <DockerServiceName>connectsoft.microservicetemplate.application</DockerServiceName>
  </PropertyGroup>
</Project>

Features: - F5 Launch: Start entire stack with F5 - Debugging: Attach debugger to application container - Hot Reload: Automatic rebuild on code changes - Service Management: Start/stop services from Visual Studio

Launch Configuration

launchSettings.json: 

{
  "profiles": {
    "Docker Compose": {
      "commandName": "DockerCompose",
      "composeLaunchAction": "LaunchBrowser",
      "composeLaunchUrl": "{Scheme}://localhost:{ServicePort}",
      "commandVersion": "1.0",
      "serviceActions": {
        "connectsoft.microservicetemplate.application": "StartDebugging"
      }
    }
  }
}

Deployment Docker Compose Files

Deployment Compose File

Deployment/docker-compose/docker-compose.yml: Simplified compose file for deployment scenarios:

version: '3.8'

networks:
  metrics:
    name: ConnectSoft.MicroserviceTemplate-network

services:
  prometheus:
    build:
      context: ./scripts/prometheus
    ports:
      - 9091:9090
    networks:
      - metrics

  grafana:
    build:
      context: ./scripts/grafana
    depends_on:
      - prometheus
    ports:
      - 3001:3001
    networks:
      - metrics

  otel-collector:
    image: otel/opentelemetry-collector:0.97.0
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./scripts/otel-collector/otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "8888:8888"
      - "8889:8889"
      - "13133:13133"
      - "4317:4317"
      - "55679:55679"
    networks:
      - metrics

CI/CD Compose File

Deployment/docker-compose/docker-compose-ci.yml: Minimal compose file for CI/CD pipelines:

version: '3.8'

services:
  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    restart: on-failure
    ports:
      - "15672:15672"
      - "5672:5672"

Best Practices

Do's

  1. Use Override Files for Development 

    # ✅ GOOD - Separate override file
# docker-compose.override.yml for development

  2. Define Service Dependencies 

    # ✅ GOOD - Explicit dependencies
depends_on:
  - sql
  - redis
  - mongo

  3. Use Named Volumes for Persistent Data 

    # ✅ GOOD - Named volumes
volumes:
  - esdata:/usr/share/elasticsearch/data

  4. Configure Health Checks 

    # ✅ GOOD - Health checks
healthcheck:
  test: ["CMD-SHELL", "curl -f https://127.0.0.1:7279/alive || exit 1"]
  interval: 10s

  5. Use Environment Variables for Configuration 

    # ✅ GOOD - Environment variables
environment:
  - ASPNETCORE_ENVIRONMENT=Docker

  6. Set Resource Limits 

    # ✅ GOOD - Resource limits
deploy:
  resources:
    limits:
      memory: 4G

Don'ts

  1. Don't Expose Unnecessary Ports 

    # ❌ BAD - Exposing all ports
ports:
  - "1433:1433"
  - "27017:27017"
  - "6379:6379"

# ✅ GOOD - Only expose needed ports
ports:
  - "8081:8081"  # Application only

  2. Don't Store Secrets in Compose Files 

    # ❌ BAD - Secrets in plain text
environment:
  - PASSWORD=MySecret123!

# ✅ GOOD - Use environment files or secrets
env_file:
  - .env

  3. Don't Use latest Tags in Production 

    # ❌ BAD - Latest tag
image: mongo:latest

# ✅ GOOD - Specific version
image: mongo:7.0

  4. Don't Skip Health Checks 

    # ❌ BAD - No health check
services:
  app:
    image: myapp

# ✅ GOOD - Health check configured
services:
  app:
    image: myapp
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]

Troubleshooting

Issue: Services Not Starting

Symptoms: Services fail to start or crash immediately.

Solutions: - Check logs: docker-compose logs <service-name> - Verify dependencies are started: docker-compose ps - Check resource limits: docker stats - Verify port conflicts: netstat -an | grep <port> - Check disk space: docker system df

Issue: Certificate Not Found

Symptoms: Application fails with certificate error.

Solutions: - Verify certgen service completed: docker-compose logs certgen - Check certificate volume: docker volume inspect <volume-name> - Verify certificate path in environment variables - Check entrypoint wait logic

Issue: Services Can't Connect

Symptoms: Services can't reach each other.

Solutions: - Verify network: docker network inspect <network-name> - Check service names match exactly - Verify ports are correct - Check firewall rules - Ensure services are on same network

Issue: Port Already in Use

Symptoms: Port binding errors.

Solutions: - Find process using port: netstat -ano | findstr :8081 - Change port mapping in compose file - Stop conflicting services - Use different ports for different environments

Summary

Docker Compose in the ConnectSoft Microservice Template provides:

  • ✅ Multi-Container Orchestration: Complete application stack in one command
  • ✅ Service Dependencies: Automatic dependency management
  • ✅ Network Isolation: Secure service communication
  • ✅ Volume Management: Persistent data storage
  • ✅ Environment Configuration: Environment-specific overrides
  • ✅ Local Development: Simplified local development setup
  • ✅ Visual Studio Integration: Native IDE support
  • ✅ CI/CD Support: Deployment and testing workflows

By using Docker Compose, teams can:

  • Run Complete Stack Locally: All services with one command
  • Ensure Consistency: Same environment across developers
  • Simplify Development: No manual service setup required
  • Test Integration: Test with real dependencies
  • Enable CI/CD: Automated testing and deployment
  • Debug Easily: View logs and debug services together
  • Manage Dependencies: Automatic dependency resolution

Docker Compose is the foundation for local development, enabling developers to work with a production-like environment on their local machines while maintaining simplicity and consistency.