Template Overlays Specification

This document provides a detailed technical specification for ConnectSoft's template overlay system, covering its architecture, application process, versioning, and compatibility management. It serves as the authoritative contract for how overlays are designed, implemented, and consumed.

Document Purpose

This specification provides:

• Complete technical contract for overlay system architecture
• Overlay application process and composition rules
• Versioning scheme for base and specialized templates
• Compatibility management and declaration format
• Recipe system for template composition
• Authoritative source for template developers

Overlay System Architecture

An overlay is a set of code, documentation, and metadata that extends or modifies a base template. Overlays enable modularity and composition, allowing specialized functionalities to be added without altering the core base template.

What is an Overlay?

An overlay is a delta package containing:

• Code: New source files, modifications to existing base files (via markers), or additional projects.
• Documentation: New markdown files, or sections to be merged into existing base documentation.
• Metadata: Extensions to template.json, ide.host.json, dotnetcli.host.json via .extend.json files.

Overlay Types

Common overlay types correspond to domain-specific functionalities or architectural concerns:

• Identity Overlay: Adds user management, authentication, and authorization logic.
• Auth Server Overlay: Configures an OpenIddict/IdentityServer-based authorization server.
• Audit Overlay: Integrates audit trail capabilities.
• Worker Overlay: Adds background worker services (e.g., MassTransit consumers, scheduled jobs).
• Persistence Overlays: Adds specific ORM/ODM configurations (e.g., NHibernate, MongoDB).

Overlay Composition Order and Precedence

Overlays are applied in a defined order, typically specified in a "recipe." The order determines precedence for conflicting modifications (later overlays override earlier ones).

flowchart TD
    BASE[Base Template Artifact]
    OVERLAY_IDENTITY[Identity Overlay Artifact]
    OVERLAY_WORKER[Worker Overlay Artifact]

    RECIPE[Recipe:<br/>Base + Identity + Worker]

    BASE -->|Apply| OVERLAY_IDENTITY
    OVERLAY_IDENTITY -->|Apply| OVERLAY_WORKER
    OVERLAY_WORKER --> FINAL[Final Composed Template]

    subgraph OverlayCompositionFlow[Overlay Composition Flow]
        BASE
        OVERLAY_IDENTITY
        OVERLAY_WORKER
        FINAL
    end

Hold "Alt" / "Option" to enable pan & zoom

Overlay Application Process

The template generator (or a local pack-template.ps1 script) applies overlays through a multi-step process.

File Addition

• New files and folders from the overlay are copied into the working template directory.
• Conflicts with existing files are resolved based on precedence rules (typically, overlay files overwrite base files unless explicitly configured otherwise).

File Patching

• Existing base template files (e.g., Program.cs, .csproj files) can be modified by overlays.
• Marker-Based Insertion: Overlays can define content to be inserted at specific markers within base files (e.g., // {{IDENTITY_SERVICES_START}} and // {{IDENTITY_SERVICES_END}}).
• XML/JSON Patching: For structured configuration files (e.g., .csproj, appsettings.json), XML or JSON patching techniques can be used to add/modify elements.

Token Replacement

• Placeholders (tokens) within the template files (e.g., {{ServiceName}}, {{RootNamespace}}) are replaced with values provided during template instantiation.
• Overlays can introduce new tokens or provide default values for existing tokens.

Documentation Merging

• Overlay-specific documentation files are added to the overall documentation structure.
• Sections from overlay markdown files can be merged into existing base markdown files using predefined markers or YAML front matter directives.
• mkdocs.yml navigation entries are composed from base and overlay definitions.

Metadata Merging (template.json Composition)

• As detailed in the Template Architecture Specification, template.json extend files are merged to produce the final template metadata.
• This includes overriding top-level fields, adding/overriding symbols (parameters), and composing post-actions and primary outputs.

Versioning & Compatibility

Robust versioning is crucial for managing template evolution and ensuring compatibility between base templates and overlays.

Base Template Versioning Scheme

• Semantic Versioning (SemVer): MAJOR.MINOR.PATCH (e.g., 1.2.3).
  • MAJOR: Incremented for breaking changes to the template's structure, core APIs, or fundamental extension points that require significant refactoring in dependent overlays.
  • MINOR: Incremented for new features, backward-compatible changes, or additions to extension points.
  • PATCH: Incremented for backward-compatible bug fixes.

Specialized Template Versioning Scheme

• Specialized templates also follow SemVer (MAJOR.MINOR.PATCH).
• They declare their compatibility with specific ranges of the base template version.

How Overlays Declare Compatibility Ranges

• Overlays include a manifest (e.g., overlay.json or within template.extend.json) that specifies their compatible base template versions.
• Compatibility Range Syntax: Standard npm-style range syntax is used (e.g., ^1.0.0 for 1.x.x, >=1.2.0 <2.0.0).

Compatibility Matrix Format

A compatibility matrix is maintained to visualize which versions of overlays are compatible with which base template versions.

graph TD
    subgraph BaseTemplateVersions[Base Template Versions]
        B1_0[Base v1.0.0]
        B1_1[Base v1.1.0]
        B2_0[Base v2.0.0]
    end

    subgraph OverlayVersions[Overlay Versions]
        O_ID_1_0[Identity v1.0.0]
        O_ID_1_1[Identity v1.1.0]
        O_ID_2_0[Identity v2.0.0]
        O_WORK_1_0[Worker v1.0.0]
        O_WORK_1_1[Worker v1.1.0]
    end

    B1_0 -->|Compatible with| O_ID_1_0
    B1_1 -->|Compatible with| O_ID_1_0
    B1_1 -->|Compatible with| O_ID_1_1
    B2_0 -->|Compatible with| O_ID_2_0

    B1_0 -->|Compatible with| O_WORK_1_0
    B1_1 -->|Compatible with| O_WORK_1_0
    B1_1 -->|Compatible with| O_WORK_1_1
    B2_0 -->|Not Compatible| O_WORK_1_0
    B2_0 -->|Not Compatible| O_WORK_1_1
    O_ID_2_0 -->|Requires| B2_0

Hold "Alt" / "Option" to enable pan & zoom

Breaking Change Handling

• Breaking changes in the base template necessitate a MAJOR version bump.
• Overlays dependent on the base template must update their compatibility declarations and adapt to the breaking changes, potentially releasing new MAJOR versions themselves.
• Clear communication of breaking changes is essential (release notes, migration guides).

Migration Strategies for Version Bumps

• Automated Migrations: For minor changes, the template generator can attempt automated code/config migrations.
• Manual Guidance: For major breaking changes, detailed migration guides are provided.

Compatibility Declaration Format

Overlays declare their compatibility in a dedicated manifest file or within their template.extend.json.

Overlay Manifest Structure (Example overlay.json)

{
  "overlayId": "microservice/overlays/identity-backend",
  "displayName": "Identity Backend Overlay",
  "version": "1.1.0",
  "description": "Adds Identity domain logic to the base microservice template.",
  "compatibility": {
    "baseTemplate": "^1.0.0",
    "dependencies": [
      {
        "overlayId": "microservice/overlays/persistence-sql",
        "version": ">=1.0.0 <2.0.0"
      }
    ]
  },
  "files": [
    "src/Identity.Api/",
    "src/Identity.Domain/",
    "src/Identity.Infrastructure/"
  ],
  "patches": [
    {
      "file": "Program.cs",
      "type": "marker-insertion",
      "marker": "IDENTITY_SERVICES"
    }
  ]
}

Base Template Version Requirements

• The baseTemplate field specifies the required version range of the base template.

Dependency on Other Overlays

• The dependencies array allows an overlay to declare its reliance on other overlays and their respective version ranges. This ensures that complex recipes are built with compatible components.

Version Bump Procedures

Clear procedures for versioning ensure a smooth development and deployment pipeline.

When to Bump Base Template Version

• MAJOR: When core interfaces, project structure, or fundamental extension points change in a non-backward-compatible way.
• MINOR: When new features or extension points are added without breaking existing functionality.
• PATCH: For bug fixes or minor internal improvements.

When to Bump Specialized Template Version

• MAJOR: When the specialized template introduces breaking changes to its own domain APIs or requires a new MAJOR version of the base template.
• MINOR: For new domain features, backward-compatible changes, or when updating to a new MINOR version of the base template.
• PATCH: For bug fixes or minor internal improvements.

How to Update Compatibility Declarations

• After a base template version bump, all dependent specialized templates must review their compatibility declarations and update them if necessary.
• Automated tooling can assist in identifying incompatible versions.

Testing Compatibility After Version Bumps

• Automated CI/CD: Comprehensive integration tests are run for all base-overlay combinations to verify compatibility.
• Regression Testing: Ensure existing functionalities are not broken.

Communication of Breaking Changes

• Release notes, migration guides, and documentation updates clearly articulate breaking changes and provide migration paths.

Recipe System

The recipe system defines how multiple templates and overlays are combined to generate a complete solution.

Recipe File Format (YAML/JSON)

Recipes are typically defined in YAML or JSON files, specifying the base template and a list of overlays to apply.

# Example: identity-worker-service.yaml
templateId: identity-worker-service
displayName: "Identity Backend with Worker"
description: "Generates an Identity Backend microservice with integrated background worker functionality."
baseTemplate: "microservice/base:^1.0.0"
overlays:
  - id: "microservice/overlays/identity-backend"
    version: "^1.0.0"
  - id: "microservice/overlays/worker"
    version: "^1.0.0"
parameters:
  ServiceName: "MyIdentityWorkerService"
  RootNamespace: "MyCompany.IdentityWorker"
  UseMassTransit: true
  UseNHibernate: true

Recipe Composition Rules

• Order Matters: Overlays are applied in the order specified in the recipe.
• Parameter Resolution: Parameters defined in the recipe override default values from template.json or extend files.
• Conflict Resolution: The last applied overlay (or recipe parameter) takes precedence in case of conflicts.

Recipe Validation

• Recipes are validated against the compatibility declarations of the specified base template and overlays.
• Ensures that all components are compatible before generation.

Recipe Execution Order

1. Load base template artifact.
2. Apply each overlay artifact sequentially, merging code, docs, and metadata.
3. Perform token replacement using recipe parameters.
4. Execute post-actions.
5. Output the final generated solution.

flowchart TD
    START[Start Recipe Execution]
    LOADBASE[Load Base Template Artifact]
    RESOLVE[Resolve Dependencies]
    APPLY1[Apply Identity Overlay]
    APPLY2[Apply Worker Overlay]
    RESOLVETOKENS[Resolve Tokens<br/>ServiceName, Namespace, etc.]
    VALIDATE[Validate Final Template]
    OUTPUT[Output Final Generated Solution]

    START --> LOADBASE
    LOADBASE --> RESOLVE
    RESOLVE --> APPLY1
    APPLY1 --> APPLY2
    APPLY2 --> RESOLVETOKENS
    RESOLVETOKENS --> VALIDATE
    VALIDATE --> OUTPUT

Hold "Alt" / "Option" to enable pan & zoom

Related Documents

• Template Architecture Specification - Core template architecture
• Template Layering and Reuse - Three-layer architecture guide
• Template Metadata Composition - Template.json composition details