Audit.NET in ConnectSoft Microservice Template

Purpose & Overview

Audit.NET is an extensible framework for auditing executing operations in .NET applications. In the ConnectSoft Microservice Template, Audit.NET provides comprehensive audit logging capabilities that track important operations, changes, and events throughout the application lifecycle, enabling compliance, debugging, and security analysis.

Why Audit.NET?

Audit.NET offers several key benefits:

• Comprehensive Auditing: Track all important operations with minimal code changes
• Multiple Data Providers: Support for SQL Server, file system, and other storage providers
• Rich Event Data: Capture detailed information including targets, environment, and duration
• Distributed Tracing: Integration with OpenTelemetry for distributed tracing
• Non-Intrusive: Minimal impact on application code using AuditScope
• Flexible Configuration: Enable/disable auditing, configure data providers, and customize event capture
• Compliance: Meet regulatory requirements for audit trails and change tracking
• Performance: Efficient audit logging with minimal overhead

Audit.NET Philosophy

Audit.NET provides a non-intrusive way to audit operations in .NET applications. It captures detailed information about operations, their targets, environment, and outcomes, enabling comprehensive audit trails for compliance, security, and debugging purposes.

Architecture Overview

Audit.NET Integration

Application Operations
    ↓ (AuditScope)
Audit.NET Framework
    ├── Event Capture
    ├── Event Enrichment
    └── Event Storage
    ↓
Data Providers
    ├── SQL Server
    └── File System

Project Structure

ConnectSoft.MicroserviceTemplate.ApplicationModel/
├── AuditNetExtensions.cs (Registration & Configuration)

ConnectSoft.MicroserviceTemplate.Options/
├── AuditNetOptions.cs (Options)
├── AuditNetSqlServerDataProviderOptions.cs (SQL Server Provider Options)
└── AuditDataProviderType.cs (Provider Type Enum)

ConnectSoft.MicroserviceTemplate.DomainModel.Impl/
└── DefaultMicroserviceAggregateRootsProcessor.cs (Usage Examples)

Service Registration

AddAuditNet Extension

Audit.NET is registered via AddAuditNet():

// MicroserviceRegistrationExtensions.cs
#if UseAuditNet
    services.AddAuditNet();
#endif

Registration Flow:

// AuditNetExtensions.cs
internal static IServiceCollection AddAuditNet(this IServiceCollection services)
{
    ArgumentNullException.ThrowIfNull(services);

    // Configure global Audit.NET settings
    Audit.Core.Configuration.IncludeTypeNamespaces = OptionsExtensions.AuditNetOptions.IncludeTypeNamespaces;

    var configurator = Audit.Core.Configuration.Setup()
        .AuditDisabled(!OptionsExtensions.AuditNetOptions.EnableAudit)
        .IncludeStackTrace(OptionsExtensions.AuditNetOptions.IncludeStackTrace)
        .IncludeActivityTrace(OptionsExtensions.AuditNetOptions.IncludeActivityTrace)
        .StartActivityTrace(OptionsExtensions.AuditNetOptions.StartActivityTrace);

    // Configure data provider
    if (OptionsExtensions.AuditNetOptions.AuditDataProvider == AuditDataProviderType.SqlServer)
    {
        // Configure SQL Server provider
    }
    else if (OptionsExtensions.AuditNetOptions.AuditDataProvider == AuditDataProviderType.FileSystem)
    {
        // Configure file system provider
    }

    return services;
}

Configuration

Options Structure

// AuditNetOptions.cs
public sealed class AuditNetOptions
{
    public const string AuditNetOptionsSectionName = "AuditNet";

    [Required]
    required public bool EnableAudit { get; set; }

    [Required]
    [EnumDataType(typeof(AuditDataProviderType))]
    required public AuditDataProviderType AuditDataProvider { get; set; }

    [Required]
    required public bool IncludeTypeNamespaces { get; set; } = false;

    [Required]
    required public bool IncludeStackTrace { get; set; } = false;

    [Required]
    required public bool IncludeActivityTrace { get; set; } = false;

    [Required]
    required public bool StartActivityTrace { get; set; } = false;

    [ValidateObjectMembers]
    public AuditNetSqlServerDataProviderOptions? SqlServerDataProvider { get; set; }
}

Configuration Example

{
  "AuditNet": {
    "EnableAudit": true,
    "AuditDataProvider": "SqlServer",
    "IncludeTypeNamespaces": false,
    "IncludeStackTrace": false,
    "IncludeActivityTrace": true,
    "StartActivityTrace": false,
    "SqlServerDataProvider": {
      "ConnectionString": "Server=(localdb)\\MSSQLLocalDB;Database=Audit;Integrated Security=True;MultipleActiveResultSets=true;Encrypt=false;TrustServerCertificate=true",
      "SchemaName": "audit",
      "TableName": "AuditEvents",
      "IdColumnName": "AuditEventId",
      "JsonColumnName": "AuditData",
      "LastUpdatedColumnName": "LastUpdated"
    }
  }
}

Configuration Options

Option	Type	Default	Description
EnableAudit	bool	true	Enable or disable Audit.NET
AuditDataProvider	AuditDataProviderType	Required	Data provider type (SqlServer, FileSystem)
IncludeTypeNamespaces	bool	false	Include type namespaces in audit events
IncludeStackTrace	bool	false	Include full stack trace in audit event environment
IncludeActivityTrace	bool	true	Include activity trace in audit events
StartActivityTrace	bool	false	Create and start new Distributed Tracing Activity for each audit scope

Data Providers

SQL Server Provider

Configuration:

if (OptionsExtensions.AuditNetOptions.AuditDataProvider == AuditDataProviderType.SqlServer)
{
    ArgumentNullException.ThrowIfNull(OptionsExtensions.AuditNetOptions.SqlServerDataProvider);

    string connectionString = OptionsExtensions.AuditNetOptions.SqlServerDataProvider.ConnectionString;
    var tableName = OptionsExtensions.AuditNetOptions.SqlServerDataProvider.TableName;
    var schemaName = OptionsExtensions.AuditNetOptions.SqlServerDataProvider.SchemaName;

    EnsureAuditTableExists(OptionsExtensions.AuditNetOptions.SqlServerDataProvider);

    configurator.UseSqlServer(config =>
    {
        config.ConnectionString(connectionString);
        config.Schema(schemaName);
        config.TableName(tableName);
        config.IdColumnName(OptionsExtensions.AuditNetOptions.SqlServerDataProvider.IdColumnName);
        config.JsonColumnName(OptionsExtensions.AuditNetOptions.SqlServerDataProvider.JsonColumnName);
        config.CustomColumn("TargetType", (@event) => @event.Target.Type);
        config.CustomColumn("EventType", (@event) => @event.EventType);
        config.CustomColumn("StartDate", (@event) => @event.StartDate);
        config.CustomColumn("EndDate", (@event) => @event.EndDate);
        config.CustomColumn("Duration", (@event) => @event.Duration);
        config.LastUpdatedColumnName(OptionsExtensions.AuditNetOptions.SqlServerDataProvider.LastUpdatedColumnName);
    });
}

Table Schema:

The SQL Server provider automatically creates the audit table:

CREATE TABLE [audit].[AuditEvents](
    [AuditEventId] UNIQUEIDENTIFIER NOT NULL PRIMARY KEY DEFAULT NEWID(), 
    [AuditData] NVARCHAR(MAX) NULL, 
    [InsertedDate] DATETIME NOT NULL DEFAULT(GETUTCDATE()),
    [LastUpdated] DATETIME NULL,
    [EventType] NVARCHAR(2000) NOT NULL,
    [TargetType] NVARCHAR(2000) NULL,
    [StartDate] DATETIME2(7) NOT NULL,
    [EndDate] DATETIME2(7) NULL,
    [Duration] INT NULL
)

Custom Columns: - TargetType: Type of the audited target object - EventType: Type of the audit event - StartDate: Event start time - EndDate: Event end time - Duration: Event duration in milliseconds

Benefits: - Structured Storage: SQL Server provides structured, queryable storage - Performance: Indexed columns for fast queries - Compliance: Relational database meets compliance requirements - Scalability: SQL Server handles large audit volumes

File System Provider

Configuration:

else if (OptionsExtensions.AuditNetOptions.AuditDataProvider == AuditDataProviderType.FileSystem)
{
    var currentWorkingDirectory = new DirectoryInfo(Directory.GetCurrentDirectory());
    DirectoryInfo logsHomeDirectory = currentWorkingDirectory.CreateSubdirectory("Logs");
    DirectoryInfo auditsDirectory = logsHomeDirectory.CreateSubdirectory("Audits");

    configurator.UseFileLogProvider(config =>
    {
        config.DirectoryBuilder(_ => Path.Combine(auditsDirectory.FullName, $@"{DateTime.Now:yyyy-MM-dd}"));
        config.FilenameBuilder(auditEvent => $"{auditEvent.Environment.UserName}_{DateTime.Now.Ticks}.json");
    });
}

File Structure:

Logs/
└── Audits/
    └── 2024-01-15/
        ├── user1_638412345678901234.json
        ├── user2_638412345678901567.json
        └── ...

Benefits: - Simple Setup: No database required - Development Friendly: Easy to inspect JSON files - Portable: Files can be archived or moved easily - No Dependencies: Works without SQL Server

Using Audit.NET

AuditScope

AuditScope is the primary way to audit operations:

using (AuditScope auditScope = await AuditScope.CreateAsync(
    eventType: "Create:MicroserviceAggregateRoot",
    target: () => newEntity,
    cancellationToken: token))
{
    // Perform operation
    this.repository.Insert(newEntity);

    // AuditScope automatically captures:
    // - Event type
    // - Target object (before and after)
    // - Environment (user, machine, etc.)
    // - Duration
    // - Exception (if any)
}

Creating Audit Events

Create Operation:

private async Task<IMicroserviceAggregateRoot> SaveNewEntity(
    CreateMicroserviceAggregateRootInput input, 
    CancellationToken token)
{
    MicroserviceAggregateRootEntity newEntity = new MicroserviceAggregateRootEntity()
    {
    };

#if UseAuditNet
    using (AuditScope auditScope = await AuditScope.CreateAsync(
        eventType: "Create:MicroserviceAggregateRoot",
        target: () => newEntity,
        cancellationToken: token)
        .ConfigureAwait(false))
    {
        this.unitOfWork.ExecuteTransactional(() =>
        {
            newEntity.ObjectId = input.ObjectId;
            this.repository.Insert(newEntity);
        });

        await this.unitOfWork.CommitAsync(token).ConfigureAwait(false);
    }
#else
    this.unitOfWork.ExecuteTransactional(() =>
    {
        newEntity.ObjectId = input.ObjectId;
        this.repository.Insert(newEntity);
    });

    await this.unitOfWork.CommitAsync(token).ConfigureAwait(false);
#endif

    return newEntity;
}

Delete Operation:

private async Task DeleteEntity(IMicroserviceAggregateRoot entityToDelete, CancellationToken token)
{
#if UseAuditNet
    using (AuditScope auditScope = await AuditScope.CreateAsync(
        eventType: "Delete:MicroserviceAggregateRoot",
        target: () => entityToDelete,
        cancellationToken: token)
        .ConfigureAwait(false))
    {
        this.unitOfWork.ExecuteTransactional(() =>
        {
            this.repository.Delete(entityToDelete);
        });
    }
#else
    this.unitOfWork.ExecuteTransactional(() =>
    {
        this.repository.Delete(entityToDelete);
    });
#endif
}

AuditScope Features

Event Type: Categorize audit events

AuditScope.CreateAsync(
    eventType: "Create:MicroserviceAggregateRoot", // Event category
    target: () => entity,
    cancellationToken: token)

Target: Object being audited

target: () => entityToDelete // Target object (captured before operation)

Custom Data: Add custom fields to audit events

using (var auditScope = await AuditScope.CreateAsync(
    eventType: "CustomEvent",
    target: () => targetObject,
    cancellationToken: token))
{
    auditScope.SetCustomField("CustomField", "CustomValue");
    auditScope.SetCustomField("UserId", userId);

    // Perform operation
}

Event Enrichment: Modify audit event before saving

using (var auditScope = await AuditScope.CreateAsync(
    eventType: "CustomEvent",
    target: () => targetObject,
    cancellationToken: token))
{
    auditScope.Event.Environment.CustomFields["RequestId"] = correlationId;

    // Perform operation
}

Audit Event Structure

Event Properties

AuditEvent contains:

• EventType: Type of the audit event (e.g., "Create:MicroserviceAggregateRoot")
• Target: Object being audited (before and after state)
• Environment: Execution environment (user, machine, call stack, etc.)
• StartDate: Event start time
• EndDate: Event end time
• Duration: Event duration in milliseconds
• Result: Operation result (success/failure)
• Exception: Exception details (if any)

Example Audit Event

{
  "EventType": "Create:MicroserviceAggregateRoot",
  "Target": {
    "Type": "MicroserviceAggregateRootEntity",
    "Old": null,
    "New": {
      "ObjectId": "123e4567-e89b-12d3-a456-426614174000",
      "SomeValue": "Test Value",
      "CreatedOn": "2024-01-15T10:30:00Z"
    }
  },
  "Environment": {
    "UserName": "DOMAIN\\user",
    "MachineName": "SERVER01",
    "CallingMethodName": "SaveNewEntity",
    "Exception": null,
    "ActivityTraceId": "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"
  },
  "StartDate": "2024-01-15T10:30:00.123Z",
  "EndDate": "2024-01-15T10:30:00.456Z",
  "Duration": 333,
  "Result": "Success"
}

Configuration Options

Enable/Disable Auditing

Enable/Disable:

{
  "AuditNet": {
    "EnableAudit": true  // Set to false to disable auditing
  }
}

Programmatic Control:

Audit.Core.Configuration.Setup()
    .AuditDisabled(!OptionsExtensions.AuditNetOptions.EnableAudit);

Include Type Namespaces

Configuration:

{
  "AuditNet": {
    "IncludeTypeNamespaces": false  // false = "MicroserviceAggregateRootEntity", true = "ConnectSoft.MicroserviceTemplate.EntityModel.MicroserviceAggregateRootEntity"
  }
}

Include Stack Trace

Configuration:

{
  "AuditNet": {
    "IncludeStackTrace": false  // Include full stack trace in environment (can be large)
  }
}

Use Cases: - Debugging complex issues - Security analysis - Performance investigation

Include Activity Trace

Configuration:

{
  "AuditNet": {
    "IncludeActivityTrace": true  // Include OpenTelemetry activity trace ID
  }
}

Benefits: - Distributed Tracing: Correlate audit events with distributed traces - Request Correlation: Link audit events to specific requests - End-to-End Tracking: Track operations across services

Start Activity Trace

Configuration:

{
  "AuditNet": {
    "StartActivityTrace": false  // Create new Distributed Tracing Activity for each audit scope
  }
}

When to Use: - Operations that should be tracked as separate activities - Long-running operations that need their own trace context - Operations that span multiple services

SQL Server Provider Configuration

Connection String

{
  "AuditNet": {
    "SqlServerDataProvider": {
      "ConnectionString": "Server=(localdb)\\MSSQLLocalDB;Database=Audit;Integrated Security=True;MultipleActiveResultSets=true;Encrypt=false;TrustServerCertificate=true"
    }
  }
}

Schema and Table Names

{
  "AuditNet": {
    "SqlServerDataProvider": {
      "SchemaName": "audit",
      "TableName": "AuditEvents"
    }
  }
}

Column Names

{
  "AuditNet": {
    "SqlServerDataProvider": {
      "IdColumnName": "AuditEventId",
      "JsonColumnName": "AuditData",
      "LastUpdatedColumnName": "LastUpdated"
    }
  }
}

Automatic Table Creation

The SQL Server provider automatically creates the audit table if it doesn't exist:

private static void EnsureAuditTableExists(AuditNetSqlServerDataProviderOptions options)
{
    SqlServerDatabaseHelper databaseHelper = new();
    databaseHelper.CreateIfNotExists(options.ConnectionString);
    databaseHelper.CreateSchema(options.ConnectionString, options.SchemaName);

    // Create table if not exists
    using (var connection = new SqlConnection(options.ConnectionString))
    {
        connection.Open();
        var cmdText = $@"
IF NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES 
               WHERE TABLE_SCHEMA = '{options.SchemaName}' 
               AND  TABLE_NAME = '{options.TableName}')
BEGIN
    CREATE TABLE [{options.SchemaName}].[{options.TableName}](...)
END";
        using (var command = new SqlCommand(cmdText, connection))
        {
            command.ExecuteNonQuery();
        }
    }
}

Best Practices

Do's

1. Use AuditScope for Important Operations

   // ✅ GOOD - Audit important operations
   using (var auditScope = await AuditScope.CreateAsync(
       eventType: "Create:MicroserviceAggregateRoot",
       target: () => entity,
       cancellationToken: token))
   {
       // Perform operation
   }
   

2. Use Descriptive Event Types

   // ✅ GOOD - Clear event type
   eventType: "Create:MicroserviceAggregateRoot"
   eventType: "Delete:MicroserviceAggregateRoot"
   eventType: "Update:MicroserviceAggregateRoot"
   

3. Include Relevant Context

   // ✅ GOOD - Add context
   auditScope.SetCustomField("UserId", userId);
   auditScope.SetCustomField("RequestId", correlationId);
   auditScope.SetCustomField("OperationReason", reason);
   

4. Use Conditional Compilation

   // ✅ GOOD - Only compile audit code when enabled
   #if UseAuditNet
   using (var auditScope = await AuditScope.CreateAsync(...))
   {
       // Operation
   }
   #else
   // Operation without audit
   #endif
   

5. Configure Activity Trace for Distributed Systems

   // ✅ GOOD - Enable activity trace for distributed tracing
   {
     "AuditNet": {
       "IncludeActivityTrace": true
     }
   }
   

Don'ts

1. Don't Audit High-Frequency Operations

   // ❌ BAD - Too frequent, will impact performance
   foreach (var item in thousandsOfItems)
   {
       using (var scope = await AuditScope.CreateAsync(...))
       {
           // Process item
       }
   }
   
   // ✅ GOOD - Audit the batch operation
   using (var scope = await AuditScope.CreateAsync(
       eventType: "BatchProcess",
       target: () => new { Count = items.Count },
       cancellationToken: token))
   {
       foreach (var item in items)
       {
           // Process item
       }
   }
   

2. Don't Include Sensitive Data

   // ❌ BAD - Sensitive data in audit
   auditScope.SetCustomField("Password", password);
   auditScope.SetCustomField("CreditCard", creditCard);
   
   // ✅ GOOD - Exclude sensitive data
   auditScope.SetCustomField("UserId", userId);
   auditScope.SetCustomField("OperationType", "Login");
   

3. Don't Use File System in Production

   // ❌ BAD - File system not suitable for production
   {
     "AuditNet": {
       "AuditDataProvider": "FileSystem"
     }
   }
   
   // ✅ GOOD - SQL Server for production
   {
     "AuditNet": {
       "AuditDataProvider": "SqlServer"
     }
   }
   

4. Don't Include Stack Traces in Production

   // ❌ BAD - Stack traces can be large
   {
     "AuditNet": {
       "IncludeStackTrace": true
     }
   }
   
   // ✅ GOOD - Disable in production, enable for debugging
   {
     "AuditNet": {
       "IncludeStackTrace": false
     }
   }
   

Querying Audit Events

SQL Server Queries

Query by Event Type:

SELECT 
    AuditEventId,
    EventType,
    TargetType,
    StartDate,
    EndDate,
    Duration,
    JSON_VALUE(AuditData, '$.Environment.UserName') AS UserName
FROM [audit].[AuditEvents]
WHERE EventType = 'Create:MicroserviceAggregateRoot'
ORDER BY StartDate DESC;

Query by Target Type:

SELECT 
    AuditEventId,
    EventType,
    StartDate,
    JSON_VALUE(AuditData, '$.Target.New.ObjectId') AS ObjectId
FROM [audit].[AuditEvents]
WHERE TargetType = 'MicroserviceAggregateRootEntity'
ORDER BY StartDate DESC;

Query by Date Range:

SELECT 
    AuditEventId,
    EventType,
    StartDate,
    EndDate,
    Duration
FROM [audit].[AuditEvents]
WHERE StartDate >= '2024-01-01'
  AND StartDate < '2024-02-01'
ORDER BY StartDate DESC;

Query by User:

SELECT 
    AuditEventId,
    EventType,
    StartDate,
    JSON_VALUE(AuditData, '$.Environment.UserName') AS UserName
FROM [audit].[AuditEvents]
WHERE JSON_VALUE(AuditData, '$.Environment.UserName') = 'DOMAIN\\user'
ORDER BY StartDate DESC;

Query Full Event Data:

SELECT 
    AuditEventId,
    EventType,
    AuditData,  -- Full JSON event data
    StartDate
FROM [audit].[AuditEvents]
WHERE AuditEventId = '123e4567-e89b-12d3-a456-426614174000';

Integration with Other Patterns

Audit.NET + Unit of Work

Transaction-Aware Auditing:

using (var auditScope = await AuditScope.CreateAsync(
    eventType: "Create:MicroserviceAggregateRoot",
    target: () => newEntity,
    cancellationToken: token))
{
    this.unitOfWork.ExecuteTransactional(() =>
    {
        this.repository.Insert(newEntity);
    });

    await this.unitOfWork.CommitAsync(token);

    // Audit event is saved after transaction commits
}

Audit.NET + OpenTelemetry

Distributed Tracing Integration:

{
  "AuditNet": {
    "IncludeActivityTrace": true,
    "StartActivityTrace": false
  }
}

Benefits: - Correlation: Audit events linked to distributed traces - End-to-End Tracking: Track operations across services - Request Correlation: Link audit events to specific requests

Audit.NET + Serilog

Serilog Integration:

<PackageReference Include="Audit.NET.Serilog" />

Configuration:

configurator.UseSerilog(config =>
{
    config.Message((auditEvent) => 
        $"Audit: {auditEvent.EventType} - {auditEvent.Target.Type}");
});

Troubleshooting

Issue: Audit Events Not Saved

Symptom: Operations execute but no audit events are saved.

Solutions: 1. Check EnableAudit is true in configuration 2. Verify data provider is configured correctly 3. Check connection string for SQL Server provider 4. Verify file system permissions for file system provider 5. Check for exceptions in audit event saving

Issue: Performance Impact

Symptom: Application performance degrades with auditing enabled.

Solutions: 1. Reduce audit scope (audit only important operations) 2. Use async audit operations 3. Disable IncludeStackTrace in production 4. Consider using file system provider for development only 5. Optimize SQL Server indexes on audit table

Issue: Missing Audit Data

Symptom: Audit events saved but missing expected data.

Solutions: 1. Verify target object is serializable 2. Check IncludeTypeNamespaces setting 3. Verify custom fields are set correctly 4. Check JSON column size limits in SQL Server

Issue: Table Not Created

Symptom: SQL Server table not created automatically.

Solutions: 1. Check connection string permissions 2. Verify schema name is valid 3. Check SQL Server logs for errors 4. Manually create table if needed

Summary

Audit.NET in the ConnectSoft Microservice Template provides:

• ✅ Comprehensive Auditing: Track important operations with minimal code
• ✅ Multiple Data Providers: SQL Server and file system support
• ✅ Rich Event Data: Detailed information about operations
• ✅ Distributed Tracing: Integration with OpenTelemetry
• ✅ Non-Intrusive: Minimal impact on application code
• ✅ Flexible Configuration: Enable/disable and customize as needed
• ✅ Compliance: Meet regulatory requirements for audit trails
• ✅ Performance: Efficient audit logging with minimal overhead

By following these patterns, teams can:

• Track Operations: Audit all important operations automatically
• Meet Compliance: Maintain audit trails for regulatory requirements
• Debug Issues: Use audit logs to understand what happened
• Security Analysis: Analyze audit events for security incidents
• Performance Monitoring: Track operation durations and performance
• Distributed Tracing: Correlate audit events with distributed traces

The Audit.NET integration ensures that important operations are audited automatically, providing comprehensive audit trails for compliance, security, and debugging while maintaining minimal impact on application performance.