Skip to content

Nightly Template Validation Pipeline

Overview

The nightly template validation pipeline lives in ConnectSoft.AzurePipelines as:

  • nightly-validate-template-installers.yml

Its goal is to continuously validate template installer packages from Azure Artifacts by installing them with dotnet new, generating solutions for multiple variations, and running build plus unit tests.

Scope

The pipeline currently validates these template installer packages from feed ConnectSoft:

  • ConnectSoft.LibraryTemplate.Installer
  • ConnectSoft.ApiLibraryTemplate.Installer
  • ConnectSoft.BaseTemplate.Installer
  • ConnectSoft.AISkillsLibraryTemplate.Installer
  • ConnectSoft.ApiGatewayTemplate.Installer
  • ConnectSoft.DocumentationTemplate.Installer
  • ConnectSoft.Blazor.ComponentLibraryTemplate.Installer
  • ConnectSoft.IdentityTemplate.Installer
  • ConnectSoft.HealthChecksAggregatorTemplate.Installer

ConnectSoft.PlatformTemplate is intentionally excluded for now because it is not currently exposed as a template installer package.

High-Level Flow

  1. Run on a nightly schedule.
  2. Resolve latest installer versions from Azure Artifacts (simple latest logic).
  3. Download .nupkg installers and publish as a pipeline artifact.
  4. Run matrix jobs in parallel:
  5. install template from local .nupkg
  6. scaffold solution/project variations
  7. dotnet restore
  8. dotnet build -c Release
  9. unit-test-focused dotnet test
  10. Publish test results and publish generated scaffolds on failures for troubleshooting.

Design Decisions

  • No docs repo checkout in this pipeline: documentation repositories are not checked out and are not committed from this pipeline. Documentation is maintained directly in docs repositories.
  • Simple latest package selection: latest package version is resolved from feed metadata (isLatest) without additional release-view filtering.
  • Matrix strategy: each {template, variation} is isolated to keep failures localized and maximize parallelism.
  • Unit-test-first validation: integration/e2e/acceptance tests are excluded by default to keep nightly runtime stable and predictable.
  • dotnet new exit code 104: means a post-action was cancelled (for example declining a run-script post-action such as opening TODO.html). The pipeline treats 104 as success when a .sln/.slnx exists under the scaffold output, per the exit code reference.
  • Non-interactive post-action prompts: stdin is fed with plain n lines via cmd.exe so PowerShell does not inject a UTF-8 BOM (which would produce Invalid input "n").

Operational Notes

  • If a template variation fails, inspect:
  • TRX test results (if present)
  • scaffold artifact published for the failed matrix job
  • To add new templates:
  • add package id to download list
  • add short name and variation entries in the matrix
  • keep variations representative but minimal