CI/CD Pipelines in ConnectSoft Microservice Template¶
Purpose & Overview¶
CI/CD Pipelines automate the build, test, and deployment processes for the ConnectSoft Microservice Template, ensuring consistent, reliable, and repeatable delivery of software. The template uses Azure DevOps Pipelines (YAML) to implement comprehensive Continuous Integration (CI) and Continuous Deployment (CD) workflows.
Why CI/CD Pipelines?¶
CI/CD pipelines provide several critical benefits:
- Automation: Eliminates manual build and deployment steps
- Consistency: Ensures every build follows the same process
- Quality Gates: Automated testing and validation at each stage
- Fast Feedback: Rapid detection of integration issues
- Deployment Confidence: Automated, repeatable deployments
- Traceability: Complete audit trail of builds and deployments
- Parallel Execution: Multiple pipelines for different components
- Infrastructure as Code: Pipeline definitions versioned with code
CI/CD Philosophy
CI/CD pipelines are the automation backbone of modern software delivery. They transform manual, error-prone processes into reliable, repeatable workflows that run on every commit, providing fast feedback and ensuring quality throughout the development lifecycle.
Architecture Overview¶
Pipeline Structure¶
The template uses multiple specialized pipelines:
Main Pipeline (azure-pipelines.yml)
├── CI Stage
│ ├── Lint & Build
│ ├── Unit Tests
│ ├── Integration Tests
│ ├── Architecture Tests
│ ├── Code Coverage
│ └── Artifact Publishing
├── Docker Build Stage (conditional)
│ └── Build & Push Container Image
└── CD Stages
├── On-Prem Deployment (IIS)
└── Dev Environment Deployment
Template Pipeline (azure-pipelines-template.yml)
└── Package & Publish NuGet Template
Documentation Pipeline (azure-pipelines-documentation.yml)
└── Build & Deploy MkDocs Site
Model-Specific Pipelines
├── Service Model (azure-pipelines-service-model.yml)
├── Actor Model (azure-pipelines-actor-model.yml)
├── Messaging Model (azure-pipelines-messaging-model.yml)
└── Infrastructure Model (azure-pipelines-infrastructure-model.yml)
Pipeline Flow¶
Source Code Commit
↓
Trigger Pipeline
↓
CI Stage
├── Lint Code
├── Restore Packages
├── Build Solution
├── Run Tests
│ ├── Unit Tests
│ ├── Integration Tests
│ ├── Architecture Tests
│ └── BDD Tests
├── Code Coverage Analysis
├── Generate Reports
└── Publish Artifacts
↓
Docker Build Stage (if enabled)
├── Build Docker Image
└── Push to Container Registry
↓
CD Stages
├── Manual Approval
└── Deploy to Environment
Main CI/CD Pipeline¶
Pipeline Configuration¶
File: azure-pipelines.yml
Trigger Configuration:
trigger:
branches:
include:
- master
paths:
exclude:
- Docs
- Images
- reports
- azure-pipelines-documentation.yml
- mkdocs.yml
Variables:
variables:
solution: '**/*.slnx'
filteredSolution: '**/*.slnf'
exactSolution: 'ConnectSoft.MicroserviceTemplate.slnx'
runSettingsFileName: 'ConnectSoft.MicroserviceTemplate.Docker.runsettings'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
dotnetVersion: '9.x'
restoreVstsFeed: 'e4c108b4-7989-4d22-93d6-391b77a39552/1889adca-ccb6-4ece-aa22-cad1ae4a35f3'
codeCoverageThreshold: 75
useLocalSqlDb: true
useLocalRedis: true
useLocalMongoDb: true
useLocalRabbitMQ: false
artifactName: 'ConnectSoft.MicroserviceTemplate-drop'
BQC.ForceNewBaseline: true
dockerRegistryServiceConnection: '72b24827-7d11-4ab3-ac84-cf5822a97919'
imageRepository: 'connectsoft'
containerRegistry: 'connectsofttestregistry.azurecr.io'
CI Stage: Build and Test¶
Stage Configuration:
- stage: 'CI_Stage'
displayName: 'Build And Test ConnectSoft.MicroserviceTemplate Application'
pool:
vmImage: 'ubuntu-latest'
Container Services:
The pipeline uses container services for dependencies:
services:
redis: redis
mssql: mssql
mongodb: mongodb
rabbitmq: rabbitmq
otel-collector: otel-collector
seq: seq
postgres: postgres
mysql: mysql
Container Definitions:
containers:
- container: redis
image: redis
options: --name redis
ports:
- 6379:6379
- container: mssql
image: mcr.microsoft.com/mssql/server:2022-latest
options: --name mssql --hostname mssql
ports:
- 1433:1433
env:
SA_PASSWORD: "Password@123"
ACCEPT_EULA: "Y"
MSSQL_PID: "Express"
- container: mongodb
image: mongo
options: --name mongodb
ports:
- 27017:27017
- container: rabbitmq
image: rabbitmq:3-management
options: --name rabbitmq
ports:
- 5672:5672
- 15672:15672
env:
RABBITMQ_DEFAULT_USER: dimatest
RABBITMQ_DEFAULT_PASS: testDima34
RABBITMQ_DEFAULT_VHOST: myhost
Build Steps¶
1. Install .NET SDK:
- task: UseDotNet@2
displayName: 'Install .NET Core SDK - version $(dotnetVersion)'
inputs:
version: $(dotnetVersion)
2. Lint and Clean:
- template: build/lint-microservice-steps.yaml@templates
parameters:
solution: $(solution)
exactSolution: $(exactSolution)
restoreVstsFeed: $(restoreVstsFeed)
isNugetAuthenticateEnabled: true
isRemoveDockerComposeEnabled: true
dockerComposeProjectName: 'ConnectSoft.MicroserviceTemplate.DockerCompose/ConnectSoft.MicroserviceTemplate.DockerCompose.dcproj'
isRemovePythonDiagramsEnabled: true
pythonDiagramsProjectName: 'ConnectSoft.MicroserviceTemplate.DiagramAsCodeModel/ConnectSoft.MicroserviceTemplate.DiagramAsCodeModel.pyproj'
3. Build Solution:
- template: build/build-microservice-steps.yaml@templates
parameters:
solution: $(solution)
exactSolution: $(exactSolution)
restoreVstsFeed: $(restoreVstsFeed)
buildConfiguration: $(buildConfiguration)
4. Run Tests:
- template: test/test-microservice-steps.yaml@templates
parameters:
solution: $(solution)
exactSolution: $(exactSolution)
runSettingsFileName: $(runSettingsFileName)
buildConfiguration: $(buildConfiguration)
codeCoverageThreshold: ${{ variables.codeCoverageThreshold }}
reportsFolder: reports
5. Publish Artifacts:
- template: publish/publish-microservice-steps.yaml@templates
parameters:
solution: $(solution)
exactSolution: $(exactSolution)
artifactName: $(artifactName)
buildConfiguration: $(buildConfiguration)
6. Generate Reports:
- template: generate/generate-microservice-reports-steps.yaml@templates
parameters:
reportsFolder: reports
userName: 'Dmitry Khaymov'
userEmail: 'dmitry.khaymov@gmail.com'
Docker Build Stage¶
Conditional Docker Build:
#if (Docker)
- job: 'Microservice_Build_Push_Container'
dependsOn: 'Microservice_Lint_Build_Test_Publish'
pool:
vmImage: 'ubuntu-latest'
steps:
- template: build/build-and-push-microservice-docker-steps.yaml@templates
parameters:
dockerFile: 'ConnectSoft.MicroserviceTemplate.Application/Dockerfile'
imageName: 'connectsoft.microservicetemplate.application'
dockerRegistryServiceConnection: $(dockerRegistryServiceConnection)
containerRegistry: $(containerRegistry)
imageRepository: $(imageRepository)
#endif
Deployment Stages¶
On-Prem Deployment Stage¶
Stage Configuration:
- stage: 'CD_OnPrem_Stage'
displayName: 'Deploy ConnectSoft.MicroserviceTemplate Application'
dependsOn: 'CI_Stage'
condition: succeeded()
pool: Default
variables:
- group: 'ConnectSoft.MicroserviceTemplate.TestVariablesLibrary'
Manual Approval Job:
- job: waitForApproval
displayName: Wait for approval
pool: server
timeoutInMinutes: 4320
steps:
- task: ManualValidation@0
timeoutInMinutes: 4320
inputs:
notifyUsers: $(notifyUsersList)
instructions: 'Please review the build configuration of ConnectSoft.MicroserviceTemplate application and resume IIS release to Tests'
onTimeout: 'reject'
IIS Deployment Job:
- job: 'Microservice_Deploy_To_IIS'
displayName: 'Deploy ConnectSoft.MicroserviceTemplate.Application To IIS - Tests'
dependsOn: waitForApproval
condition: succeeded()
steps:
- template: deploy/deploy-microservice-to-iis.yaml@templates
parameters:
artifactName: $(artifactName)
microserviceName: 'ConnectSoft.MicroserviceTemplate'
applicationPoolName: 'ConnectSoft.MicroserviceTemplate.Application'
applicationName: 'ConnectSoft.MicroserviceTemplate.Application'
parentWebsiteName: 'Default Web Site'
Dev Environment Deployment¶
Deployment Job:
- deployment: DeployDevJob
displayName: 'Deploy to Dev job'
environment: 'Dev'
strategy:
runOnce:
deploy:
steps:
- template: deploy/deploy-microservice-to-iis.yaml@templates
parameters:
artifactName: $(artifactName)
microserviceName: 'ConnectSoft.MicroserviceTemplate'
applicationPoolName: 'ConnectSoft.MicroserviceTemplate.Application'
applicationName: 'ConnectSoft.MicroserviceTemplate.Application'
parentWebsiteName: 'Default Web Site'
Template Packaging Pipeline¶
Pipeline Configuration¶
File: azure-pipelines-template.yml
Purpose: Package and publish the microservice template as a NuGet package to Azure Artifacts.
Versioning:
Build Number:
Build Steps¶
1. Install .NET SDK:
- task: UseDotNet@2
displayName: 'Install .NET Core SDK'
inputs:
version: '9.x'
includePreviewVersions: true
2. Install NuGet CLI:
3. Authenticate NuGet:
4. Pack Template:
- script: |
nuget pack ConnectSoft.MicroserviceTemplate.nuspec -NoDefaultExcludes -OutputDirectory $(nupkgOutputPath) -Properties Configuration=Release;Version=$(Build.BuildNumber)
displayName: 'Pack ConnectSoft.MicroserviceTemplate.nuspec'
5. Push to Azure Artifacts:
- task: DotNetCoreCLI@2
displayName: Push ConnectSoft.MicroserviceTemplate.Installer to Azure Artifacts
inputs:
command: push
packagesToPush: '$(nupkgOutputPath)/*.nupkg'
publishVstsFeed: $(publishVstsFeed)
6. Publish Artifacts:
- task: PublishBuildArtifacts@1
displayName: Publish .nupkg Files
inputs:
PathtoPublish: $(nupkgOutputPath)
ArtifactName: $(artifactName)
publishLocation: 'Container'
Documentation Pipeline¶
Pipeline Configuration¶
File: azure-pipelines-documentation.yml
Trigger:
trigger:
branches:
include:
- master
paths:
include:
- README.md
- Docs/*
- Images/**
- azure-pipelines-documentation.yml
- mkdocs.yml
Build Documentation Site¶
Stage Configuration:
- stage: 'BUILD_DOCS_SITE'
displayName: 'Build Documentation Site for ConnectSoft.MicroserviceTemplate'
pool:
vmImage: 'ubuntu-latest'
Build Steps:
- template: generate/generate-microservice-documentation-steps.yaml@templates
parameters:
artifactName: $(artifactName)
siteDirectory: $(siteDirectory)
- task: NodeTool@0
displayName: 'Setup Node.js LTS'
inputs:
versionSpec: '18.x'
- script: |
npm install -g log4brains
displayName: 'Install log4brains'
- script: |
npm run adr:build
displayName: 'Build ADRs with log4brains'
- script: |
mkdir -p $(siteDirectory)/adr
cp -r .log4brains/out/* $(siteDirectory)/adr/
echo "ADR files merged into site/adr:"
ls -la $(siteDirectory)/adr/
displayName: 'Merge ADRs into MkDocs site'
- task: PublishPipelineArtifact@1
inputs:
targetPath: $(siteDirectory)
artifactName: $(artifactName)
publishLocation: 'pipeline'
displayName: 'Publish merged site artifact'
Deploy Documentation Site¶
Deployment Stage:
- stage: 'DEPLOY_DOCS_SITE_ONPREM'
displayName: 'Deploy Documentation Site for ConnectSoft.MicroserviceTemplate'
dependsOn: 'BUILD_DOCS_SITE'
condition: succeeded()
pool: Default
Deployment Steps:
- template: deploy/deploy-static-site-to-iis.yaml@templates
parameters:
artifactName: $(artifactName)
microserviceName: 'ConnectSoft.MicroserviceTemplate'
applicationPoolName: 'ConnectSoft.MicroserviceTemplate.Docs'
applicationName: 'ConnectSoft.MicroserviceTemplate.Docs'
parentWebsiteName: 'Default Web Site'
Model-Specific Pipelines¶
Service Model Pipeline¶
File: azure-pipelines-service-model.yml
Purpose: Build, pack, and publish the Service Model NuGet package.
Trigger:
trigger:
branches:
include:
- master
paths:
include:
- ConnectSoft.MicroserviceTemplate.ServiceModel
- azure-pipelines-service-model.yml
Build Steps:
- task: DotNetCoreCLI@2
displayName: 'dotnet build $(project)'
inputs:
command: 'build'
vstsFeed: $(restoreVstsFeed)
projects: $(project)
arguments: --configuration $(buildConfiguration)
- task: DotNetCoreCLI@2
displayName: 'dotnet pack - create nuget package for $(project)'
inputs:
command: pack
packagesToPack: $(project)
packDirectory: '$(build.artifactStagingDirectory)'
nobuild: true
versioningScheme: byBuildNumber
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
- task: DotNetCoreCLI@2
displayName: 'dotnet push package to azure devops artifacts for $(project)'
inputs:
command: push
publishVstsFeed: $(publishVstsFeed)
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
Actor Model Pipeline¶
File: azure-pipelines-actor-model.yml
Similar structure to Service Model pipeline, triggered by changes to ConnectSoft.MicroserviceTemplate.ActorModel.
Messaging Model Pipeline¶
File: azure-pipelines-messaging-model.yml
Similar structure to Service Model pipeline, triggered by changes to ConnectSoft.MicroserviceTemplate.MessagingModel.
Infrastructure Model Pipeline¶
File: azure-pipelines-infrastructure-model.yml
Similar structure to Service Model pipeline, triggered by changes to ConnectSoft.MicroserviceTemplate.InfrastructureModel.
Test Configuration¶
Run Settings¶
File: ConnectSoft.MicroserviceTemplate.Docker.runsettings
Key Configuration:
<RunConfiguration>
<ResultsDirectory>.\TestResults</ResultsDirectory>
<MaxCpuCount>0</MaxCpuCount>
<TestSessionTimeout>600000</TestSessionTimeout>
<TreatNoTestsAsError>false</TreatNoTestsAsError>
<DisableParallelization>true</DisableParallelization>
<EnvironmentVariables>
<Variable name="ASPNETCORE_ENVIRONMENT" value="Development.Docker" />
</EnvironmentVariables>
</RunConfiguration>
Code Coverage Configuration¶
Module Paths:
<ModulePaths>
<Include>
<ModulePath>.*\\ConnectSoft\.MicroserviceTemplate\.dll$</ModulePath>
<ModulePath>.*\\ConnectSoft\.MicroserviceTemplate\.[^\\]*\.dll$</ModulePath>
<ModulePath>.*\\ConnectSoft\.MicroserviceTemplate\.[^\\]*\.exe</ModulePath>
</Include>
</ModulePaths>
Excluded Attributes:
<Attributes>
<Exclude>
<Attribute>^System.Diagnostics.DebuggerHiddenAttribute$</Attribute>
<Attribute>^System.Diagnostics.DebuggerNonUserCodeAttribute$</Attribute>
<Attribute>^System.Runtime.CompilerServices.CompilerGeneratedAttribute$</Attribute>
<Attribute>^System.CodeDom.Compiler.GeneratedCodeAttribute$</Attribute>
<Attribute>^System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute$</Attribute>
<Attribute>^NUnit.Framework.TestFixtureAttribute$</Attribute>
<Attribute>^Xunit.FactAttribute$</Attribute>
<Attribute>^Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute$</Attribute>
</Exclude>
</Attributes>
Test Execution¶
Test Command:
- task: DotNetCoreCLI@2
displayName: 'Run tests'
inputs:
command: 'test'
projects: '**/*Tests/**/*.csproj'
arguments: '--configuration Release --no-build --verbosity normal --settings $(runSettingsFileName) --collect:"XPlat Code Coverage"'
Test Categories:
- Unit Tests: Standard unit tests
- Integration Tests: Integration tests with external dependencies
- Architecture Tests: Architecture validation tests
- BDD Tests: Behavior-driven development tests (Reqnroll)
Code Coverage¶
Coverage Threshold¶
Configuration:
Enforcement:
- Build Quality Checks (BQC) enforce coverage threshold
BQC.ForceNewBaseline: trueallows resetting baseline after major refactoring- Coverage reports generated and published as artifacts
Coverage Reports¶
Report Generation:
- template: generate/generate-microservice-reports-steps.yaml@templates
parameters:
reportsFolder: reports
userName: 'Dmitry Khaymov'
userEmail: 'dmitry.khaymov@gmail.com'
Report Types:
- Code Coverage: Line and branch coverage
- Test Results: Test execution results
- Build Summary: Build artifacts and metrics
Central Package Management Integration¶
Locked Mode Restore¶
Restore Command:
- task: DotNetCoreCLI@2
displayName: 'Restore packages'
inputs:
command: 'restore'
projects: '**/*.sln'
arguments: '--locked-mode'
Benefits:
- Ensures exact versions from
Directory.Packages.props - Prevents version drift
- Reproducible builds
- Consistent dependency resolution
See Central Package Management for details.
Pipeline Templates¶
Template Repository¶
Repository Reference:
resources:
repositories:
- repository: templates
type: git
name: ConnectSoft/ConnectSoft.AzurePipelines
Template Usage:
- template: build/lint-microservice-steps.yaml@templates
- template: build/build-microservice-steps.yaml@templates
- template: test/test-microservice-steps.yaml@templates
- template: publish/publish-microservice-steps.yaml@templates
- template: generate/generate-microservice-reports-steps.yaml@templates
- template: deploy/deploy-microservice-to-iis.yaml@templates
- template: deploy/deploy-static-site-to-iis.yaml@templates
- template: build/build-and-push-microservice-docker-steps.yaml@templates
Benefits:
- Reusability: Shared templates across projects
- Consistency: Standardized build and deployment processes
- Maintainability: Update templates once, affects all pipelines
- Versioning: Template versions can be pinned
Best Practices¶
Do's¶
-
Use Template Repository
-
Pin SDK Versions
-
Use Locked Mode for Restore
-
Configure Code Coverage Threshold
-
Use Container Services for Dependencies
-
Path-Based Triggers
Don'ts¶
-
Don't Hardcode Secrets
-
Don't Skip Tests
-
Don't Ignore Test Failures
-
Don't Use Unversioned Images
-
Don't Deploy Without Approval
Troubleshooting¶
Issue: Pipeline Not Triggering¶
Symptoms: Pipeline doesn't run on commit.
Solutions: 1. Verify branch is in trigger configuration 2. Check path filters (exclude patterns) 3. Verify YAML file is in repository root 4. Check pipeline is enabled in Azure DevOps 5. Verify branch protection rules
Issue: Tests Failing in CI¶
Symptoms: Tests pass locally but fail in pipeline.
Solutions: 1. Check container services are running 2. Verify environment variables are set 3. Check test timeout settings 4. Verify run settings file is correct 5. Check for test parallelization issues 6. Verify database connections and credentials
Issue: Code Coverage Below Threshold¶
Symptoms: Build fails due to low code coverage.
Solutions:
1. Increase test coverage
2. Review excluded code paths
3. Check coverage threshold setting
4. Use BQC.ForceNewBaseline: true to reset baseline (after refactoring)
5. Review coverage report for gaps
Issue: Docker Build Fails¶
Symptoms: Container build fails in pipeline.
Solutions: 1. Verify Dockerfile syntax 2. Check base image availability 3. Verify Docker registry credentials 4. Check Docker build context 5. Review Docker build logs
Issue: Deployment Fails¶
Symptoms: Deployment stage fails.
Solutions: 1. Verify deployment permissions 2. Check variable groups are accessible 3. Verify target environment exists 4. Check IIS configuration 5. Review deployment logs
Pipeline Variables and Secrets¶
Variable Groups¶
Usage:
Benefits: - Centralized variable management - Environment-specific values - Secure secret storage - Reusable across pipelines
Secrets Management¶
Best Practices: - Use Azure Key Vault for secrets - Reference secrets via variable groups - Never commit secrets to source control - Rotate secrets regularly - Use managed identities when possible
Monitoring and Notifications¶
Notifications¶
Email Notifications:
Manual Validation:
- task: ManualValidation@0
inputs:
notifyUsers: $(notifyUsersList)
instructions: 'Please review the build configuration...'
Build Badges¶
Azure DevOps Badges:
- Build status badge
- Test results badge
- Code coverage badge
- Deployment status badge
Integration: - Add badges to README.md - Display in documentation - Show in project dashboards
Summary¶
CI/CD pipelines in the ConnectSoft Microservice Template provide:
- ✅ Automated Builds: Consistent, repeatable builds on every commit
- ✅ Comprehensive Testing: Unit, integration, architecture, and BDD tests
- ✅ Code Quality: Code coverage enforcement and quality gates
- ✅ Container Support: Docker image building and publishing
- ✅ Multi-Environment Deployment: On-prem and cloud deployment support
- ✅ Template Packaging: Automated NuGet package creation and publishing
- ✅ Documentation Publishing: Automated documentation site generation and deployment
- ✅ Model-Specific Pipelines: Separate pipelines for different components
- ✅ Central Package Management: Locked mode restore for reproducible builds
- ✅ Manual Approvals: Controlled deployment with approval gates
By following these patterns, teams can:
- Automate Delivery: Reduce manual steps and errors
- Ensure Quality: Automated testing and quality gates
- Deploy Confidently: Consistent, repeatable deployments
- Maintain Traceability: Complete audit trail of builds and deployments
- Scale Efficiently: Reusable templates and modular pipelines
- Monitor Progress: Build status, test results, and deployment tracking
The CI/CD pipeline infrastructure enables teams to deliver software reliably, consistently, and confidently while maintaining quality and traceability throughout the development lifecycle.