Skip to main content

Infrastructure Packages

Overview of background jobs, resilience patterns, secrets management, Kubernetes integration, and service discovery for Muonroi applications.

Muonroi.BackgroundJobs.Abstractions​

NuGet: Muonroi.BackgroundJobs.Abstractions | Tier: OSS | Distribution: NuGet

Purpose​

Unified abstraction layer for background job scheduling. Provides provider-agnostic interface with automatic context capture and propagation, supporting both Hangfire and Quartz.NET as pluggable backends.

Key Types​

TypeKindPurpose
IBackgroundJobSchedulerInterfaceUnified scheduler: Enqueue<T>(), Schedule<T>(), AddOrUpdateRecurring<T>(), RemoveRecurring()
BackgroundJobHandlerStaticProvider registry and dispatcher. Self-registers providers via [ModuleInitializer]
JobTypeEnumSupported backends: Hangfire, Quartz
TenantAwareJobBaseAbstract ClassBase for jobs requiring system execution context restoration
IMuonroiJobExecutionContextInterfaceCaptures job context: JobId, JobType, ScheduledAt, plus tenant/user/correlation context
MuonroiJobExecutionContextRecordDefault implementation with sealed constructor
BackgroundJobConfigsClassConfiguration: JobType, ConnectionString

DI Registration​

// Program.cs
using Muonroi.BackgroundJobs.Abstractions;

// Automatic: AddBackgroundJobs dispatches to Hangfire or Quartz provider
services.AddBackgroundJobs(configuration);

// Configuration reads from "BackgroundJobConfigs" section
{
  "BackgroundJobConfigs": {
    "JobType": "Hangfire",  // or "Quartz"
    "ConnectionString": "Server=localhost;Database=hangfire"
  }
}

Usage Example​

public class ReportJob : TenantAwareJobBase
{
    private readonly IReportService _reportService;
    
    public ReportJob(
        ISystemExecutionContextAccessor contextAccessor,
        ITenantContextPolicy tenantPolicy,
        IReportService reportService) 
        : base(contextAccessor, tenantPolicy)
    {
        _reportService = reportService;
    }

    protected override async Task ExecuteAsync()
    {
        // Context (TenantId, UserId, CorrelationId) already restored
        await _reportService.GenerateMonthlyReportAsync();
    }
}

// Schedule via injected scheduler
public class ReportController(IBackgroundJobScheduler scheduler)
{
    public IActionResult ScheduleReport()
    {
        var context = new MuonroiJobExecutionContext(
            tenantId: "tenant-123",
            userId: "user-456",
            username: "john.doe",
            correlationId: Guid.NewGuid().ToString("N"),
            accessToken: null,
            apiKey: null,
            isAuthenticated: true,
            permissions: ["read:reports"],
            sourceType: "api",
            jobId: Guid.NewGuid().ToString("N"),
            jobType: nameof(ReportJob),
            scheduledAt: DateTimeOffset.UtcNow);

        // Enqueue immediately
        string jobId = scheduler.Enqueue<ReportJob>(job => job.RunAsync(context));

        // Or schedule for later
        string jobId2 = scheduler.Schedule<ReportJob>(
            job => job.RunAsync(context),
            DateTimeOffset.UtcNow.AddHours(2));

        // Or recurring (every day at 2 AM)
        scheduler.AddOrUpdateRecurring<ReportJob>(
            "daily-report",
            job => job.RunAsync(context),
            "0 0 2 * * ?");  // Cron: second minute hour day month dayofweek

        return Ok(new { jobId });
    }
}

Module Initializer Pattern (AOT-Safe)​

Provider packages self-register via [ModuleInitializer] at assembly load time—no reflection, fully AOT-compatible:

// In Muonroi.BackgroundJobs.Hangfire
[ModuleInitializer]
internal static void Register()
{
    BackgroundJobHandler.RegisterProvider(
        JobType.Hangfire,
        static (services, configuration) =>
            Hangfire.BackgroundJobHandler.AddBackgroundJobs(services, configuration));
}

Muonroi.BackgroundJobs.Hangfire​

NuGet: Muonroi.BackgroundJobs.Hangfire | Tier: OSS | Distribution: NuGet

Purpose​

Hangfire provider implementation. Suitable for smaller deployments, fire-and-forget jobs, and dashboard-based monitoring. SQL-backed job queue with built-in dashboard and recurring job support.

Key Types​

TypeKindPurpose
HangfireJobSchedulerImplementationAdapts Hangfire API to IBackgroundJobScheduler
HangfireProviderRegistrationModule InitializerSelf-registers with BackgroundJobHandler
JobContextActivatorFilterServer FilterRestores Muonroi execution context (TenantId, UserId, CorrelationId) before job runs

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// Automatic via AddBackgroundJobs
builder.Services.AddBackgroundJobs(builder.Configuration);

// OR explicit
builder.Services.AddHangfire((sp, config) =>
{
    config.UseSimpleAssemblyNameTypeSerializer();
    config.UseRecommendedSerializerSettings();
    config.UseFilter(new AutomaticRetryAttribute
    {
        Attempts = 3,
        DelaysInSeconds = [5, 10, 30]
    });
    config.UseFilter(sp.GetRequiredService<JobContextActivatorFilter>());
});
builder.Services.AddHangfireServer();
builder.Services.TryAddScoped<IBackgroundJobScheduler, HangfireJobScheduler>();

var app = builder.Build();
app.UseHangfireDashboard("/hangfire");

Configuration (appsettings.json)​

{
  "BackgroundJobConfigs": {
    "JobType": "Hangfire",
    "ConnectionString": "Server=localhost;Database=hangfire;User Id=sa;Password=Your_Password;"
  }
}

Usage Example​

public class UserNotificationJob : TenantAwareJobBase
{
    private readonly IEmailService _emailService;

    public UserNotificationJob(
        ISystemExecutionContextAccessor contextAccessor,
        ITenantContextPolicy tenantPolicy,
        IEmailService emailService)
        : base(contextAccessor, tenantPolicy)
    {
        _emailService = emailService;
    }

    protected override async Task ExecuteAsync()
    {
        var context = ExecutionContextAccessor.Get();
        await _emailService.SendWelcomeEmailAsync(context.UserId);
    }
}

// In service
public class UserService(IBackgroundJobScheduler scheduler)
{
    public async Task RegisterUserAsync(User user)
    {
        // ... user creation logic ...

        // Fire notification asynchronously
        var context = new MuonroiJobExecutionContext(
            tenantId: user.TenantId,
            userId: user.Id,
            username: user.Email,
            correlationId: Guid.NewGuid().ToString("N"),
            accessToken: null,
            apiKey: null,
            isAuthenticated: true,
            permissions: [],
            sourceType: "api",
            jobId: Guid.NewGuid().ToString("N"),
            jobType: nameof(UserNotificationJob),
            scheduledAt: DateTimeOffset.UtcNow);

        scheduler.Enqueue<UserNotificationJob>(job => job.RunAsync(context));
    }
}

Dashboard Access​

public class HangfireDashboardAuthFilter : IDashboardAuthorizationFilter
{
    public bool Authorize(DashboardContext context)
    {
        var user = context.GetHttpContext().User;
        return user.Identity?.IsAuthenticated == true 
            && user.IsInRole("Admin");
    }
}

// Register in Program.cs
app.UseHangfireDashboard("/hangfire", new DashboardOptions
{
    Authorization = [new HangfireDashboardAuthFilter()]
});

Access dashboard at: https://yourapp.com/hangfire

Trade-offs vs Quartz​

AspectHangfireQuartz
Expression JobsSupported nativelyNot supported (class-based only)
DashboardBuilt-in, web-basedRequires external monitoring app
Recurring JobsSimple, declarativeRequires trigger builder
Distributed ModeRequires SQL lock contentionTrue cluster-aware with Quartz Clustering Plugin
StorageSQL onlySQL, in-memory, MongoDB options
Ease of UseSimpler, less configMore complex, more flexible

Muonroi.BackgroundJobs.Quartz​

NuGet: Muonroi.BackgroundJobs.Quartz | Tier: OSS | Distribution: NuGet

Purpose​

Quartz.NET provider implementation. Enterprise-grade scheduler for distributed systems with advanced retry policies, clustering, and job persistence. Recommended for multi-instance deployments requiring high availability.

Key Types​

TypeKindPurpose
QuartzJobSchedulerImplementationAdapts Quartz API to IBackgroundJobScheduler (expression jobs throw NotSupported)
QuartzProviderRegistrationModule InitializerSelf-registers with BackgroundJobHandler
QuartzContextJobListenerJob ListenerRestores Muonroi execution context before/after job runs

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// Automatic via AddBackgroundJobs
builder.Services.AddBackgroundJobs(builder.Configuration);

// OR explicit with custom job registration
builder.Services.AddQuartz(q =>
{
    // Jobs are registered by application code or configuration
});
builder.Services.AddSingleton<QuartzContextJobListener>();
builder.Services.AddQuartzHostedService(o => o.WaitForJobsToComplete = true);
builder.Services.TryAddScoped<IBackgroundJobScheduler, QuartzJobScheduler>();

var app = builder.Build();

Configuration (appsettings.json)​

{
  "BackgroundJobConfigs": {
    "JobType": "Quartz",
    "ConnectionString": "Server=localhost;Database=quartz;User Id=sa;Password=Your_Password;"
  },
  "Quartz": {
    "Properties": {
      "quartz.scheduler.instanceName": "MyScheduler",
      "quartz.jobStore.type": "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz",
      "quartz.jobStore.driverDelegateType": "Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz",
      "quartz.jobStore.dataSource": "default",
      "quartz.jobStore.useProperties": false,
      "quartz.dataSource.default.connectionString": "Server=localhost;Database=quartz;User Id=sa;Password=Your_Password;",
      "quartz.dataSource.default.provider": "SqlServer",
      "quartz.threadPool.threadCount": 10,
      "quartz.threadPool.type": "Quartz.Simpl.SimpleThreadPool, Quartz"
    }
  }
}

Usage Example​

// Define a class-based job (Quartz requires this)
public class InventoryAuditJob : IJob
{
    private readonly IInventoryService _inventoryService;
    private readonly IMLog<InventoryAuditJob> _log;

    public InventoryAuditJob(IInventoryService inventoryService, IMLog<InventoryAuditJob> log)
    {
        _inventoryService = inventoryService;
        _log = log;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        var correlationId = context.Get<string>("CorrelationId") ?? Guid.NewGuid().ToString("N");
        using var scope = _log.BeginProperty("CorrelationId", correlationId);

        try
        {
            _log.Info("Starting inventory audit");
            await _inventoryService.AuditAsync();
            _log.Info("Inventory audit completed");
        }
        catch (Exception ex)
        {
            _log.Error($"Inventory audit failed: {ex.Message}", ex);
            throw;
        }
    }
}

// Register job and schedule via ISchedulerFactory (from Quartz DI)
public class JobSchedulingService(ISchedulerFactory schedulerFactory)
{
    public async Task ScheduleAuditAsync()
    {
        var scheduler = await schedulerFactory.GetScheduler();

        var job = JobBuilder.Create<InventoryAuditJob>()
            .WithIdentity("inventory-audit", "maintenance")
            .UsingJobData("CorrelationId", Guid.NewGuid().ToString("N"))
            .Build();

        var trigger = TriggerBuilder.Create()
            .WithIdentity("inventory-audit-trigger", "maintenance")
            .WithCronSchedule("0 0 1 * * ?")  // Daily at 1 AM
            .Build();

        await scheduler.ScheduleJob(job, trigger);
    }
}

Cluster Configuration​

For distributed deployments, enable Quartz clustering:

{
  "Quartz": {
    "Properties": {
      "quartz.jobStore.clustered": true,
      "quartz.scheduler.instanceId": "AUTO",
      "quartz.jobStore.clusterCheckinInterval": 7500
    }
  }
}

Cron Expression Reference​

Quartz uses 6-field cron: second minute hour dayOfMonth month dayOfWeek

"0 0 * * * ?"           // Every hour
"0 0 2 * * ?"           // Every day at 2 AM
"0 0 9 ? * MON"         // Every Monday at 9 AM
"0 */15 * * * ?"        // Every 15 minutes
"0 0 3 1 * ?"           // First day of month at 3 AM
"0 0 17 ? * MON-FRI"    // Every weekday at 5 PM
"*/30 * * * * ?"        // Every 30 seconds

Muonroi.Resilience​

NuGet: Muonroi.Resilience | Tier: OSS | Distribution: NuGet

Purpose​

Resilience patterns using Polly 8+ library. Provides standardized retry, circuit breaker, timeout, and bulkhead policies for handling transient failures and protecting against cascading failures.

Key Types​

TypeKindPurpose
MuonroiResilienceExtensionsStaticAddMuonroiResilience() extension registers standard pipeline
PolicyHandlerClassBuilds custom resilience pipelines with logging

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// Register standard Muonroi resilience pipeline
builder.Services.AddMuonroiResilience();

var app = builder.Build();

Standard Policy Details​

The standard "muonroi-standard" pipeline includes:

  1. Retry — Exponential backoff with jitter

    • Max attempts: 3
    • Initial delay: 1s → 2s → 4s
    • Handles: MTransientException, HttpRequestException

  2. Circuit Breaker — Protects downstream services

    • Failure ratio threshold: 50%
    • Sampling duration: 30s
    • Minimum throughput: 5 requests
    • Break duration: 30s

  3. Timeout — Prevents indefinite hangs

    • Timeout duration: 10s

Usage Example​

public class ExternalApiService(ResiliencePipelineProvider<HttpResponseMessage> pipelineProvider)
{
    private readonly HttpClient _httpClient = new();

    public async Task<string> FetchDataAsync(string endpoint)
    {
        var pipeline = pipelineProvider.GetPipeline<HttpResponseMessage>("muonroi-standard");
        
        var outcome = await pipeline.ExecuteAsync(
            async (ct) =>
            {
                var response = await _httpClient.GetAsync(endpoint, ct);
                return response;
            },
            CancellationToken.None);

        return await outcome.Result.Content.ReadAsStringAsync();
    }
}

Custom Policy Example​

public class PolicyHandler(IMLog<PolicyHandler> logger)
{
    public ResiliencePipeline<T> CreateDefaultPipeline<T>(string serviceName)
    {
        return new ResiliencePipelineBuilder<T>()
            .AddRetry(new RetryStrategyOptions<T>
            {
                ShouldHandle = new PredicateBuilder<T>().Handle<Exception>(),
                BackoffType = DelayBackoffType.Exponential,
                UseJitter = true,
                MaxRetryAttempts = 3,
                Delay = TimeSpan.FromSeconds(1),
                OnRetry = args =>
                {
                    logger.LogWarning("Retrying {ServiceName} due to {Exception}",
                        serviceName, args.Outcome.Exception?.Message);
                    return default;
                }
            })
            .AddCircuitBreaker(new CircuitBreakerStrategyOptions<T>
            {
                ShouldHandle = new PredicateBuilder<T>().Handle<Exception>(),
                FailureRatio = 0.5,
                SamplingDuration = TimeSpan.FromSeconds(30),
                MinimumThroughput = 5,
                BreakDuration = TimeSpan.FromSeconds(30)
            })
            .AddTimeout(TimeSpan.FromSeconds(10))
            .Build();
    }
}

Muonroi.Secrets​

NuGet: Muonroi.Secrets | Tier: OSS | Distribution: NuGet

Purpose​

Abstraction layer for secret management. Decouples applications from specific secret stores (configuration, Vault, Azure Key Vault) enabling environment-specific provider selection without code changes.

Key Types​

TypeKindPurpose
ISecretProviderInterfaceGetSecret(name) → string?
ConfigurationSecretProviderImplementationReads from .NET configuration (suitable for dev only)

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// Default: read from configuration/environment variables
builder.Services.AddSingleton<ISecretProvider, ConfigurationSecretProvider>();

// OR use a custom provider (see Secret Management Guide for Vault/AzureKeyVault)
// builder.Services.AddHashiCorpVaultSecretProvider(configuration.GetSection("VaultConfigs"));

var app = builder.Build();

Configuration (appsettings.json)​

{
  "Secrets": {
    "JwtSigningKey": "your-secret-key-here",
    "DatabasePassword": "secure-password",
    "ApiKey": "external-api-key"
  }
}

Usage Example​

public class AuthService(ISecretProvider secretProvider)
{
    public async Task<JwtSecurityToken> CreateTokenAsync(string userId)
    {
        var signingKey = secretProvider.GetSecret("Secrets:JwtSigningKey");
        
        if (string.IsNullOrEmpty(signingKey))
        {
            throw new InvalidOperationException("JWT signing key not configured");
        }

        var tokenHandler = new JwtSecurityTokenHandler();
        var key = Encoding.ASCII.GetBytes(signingKey);

        var tokenDescriptor = new SecurityTokenDescriptor
        {
            Subject = new ClaimsIdentity(new[] { new Claim("sub", userId) }),
            Expires = DateTime.UtcNow.AddHours(1),
            SigningCredentials = new SigningCredentials(new SymmetricSecurityKey(key), SecurityAlgorithms.HmacSha256Signature)
        };

        return tokenHandler.CreateJwtSecurityToken(tokenDescriptor);
    }
}

Advanced: Vault Provider​

For production, use HashiCorp Vault (requires Muonroi.Secrets.Vault extension package):

builder.Services.AddHashiCorpVaultSecretProvider(
    configuration.GetSection("VaultConfigs"));

Configuration:

{
  "VaultConfigs": {
    "Type": "HashiCorp",
    "Address": "https://vault.example.com:8200",
    "Token": "${VAULT_TOKEN}",
    "Namespace": "muonroi",
    "SecretsPath": "secret/data/production",
    "AuthMethod": "token",
    "TlsSkipVerify": false,
    "RequestTimeoutSeconds": 30,
    "CacheDurationMinutes": 5
  }
}

Store secrets:

vault kv put secret/production/jwt \
  signing-key="LS0tLS1CRUdJTi..."

vault kv put secret/production/db \
  connection-string="Server=db.internal;Database=muonroi;User=admin;Password=xxx"

See Secret Management Guide for detailed Vault, Azure KeyVault, and Kubernetes patterns.

Muonroi.Kubernetes​

NuGet: Muonroi.Kubernetes | Tier: OSS | Distribution: NuGet

Purpose​

Configuration and helpers for Kubernetes cluster integration. Enables feature detection, cluster type-aware behavior, and RBAC/service account discovery.

Key Types​

TypeKindPurpose
KubernetesConfigsClassCluster configuration: ClusterType, ClusterEndpoint
KubernetesClusterTypeEnumSupported: K8S (upstream), K3S (lightweight), Eks (AWS)

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

var kubeConfig = builder.Configuration.GetSection("KubernetesConfigs");
builder.Services.Configure<KubernetesConfigs>(kubeConfig);

var app = builder.Build();

Configuration (appsettings.json)​

{
  "KubernetesConfigs": {
    "ClusterType": "K8S",
    "ClusterEndpoint": "https://kubernetes.default.svc.cluster.local:443"
  }
}

Usage Example​

public class KubeHealthCheckService(IOptions<KubernetesConfigs> kubeOptions)
{
    public void CheckClusterType()
    {
        var config = kubeOptions.Value;
        
        switch (config.ClusterType)
        {
            case KubernetesClusterType.K8S:
                Console.WriteLine("Running on upstream Kubernetes");
                break;
            case KubernetesClusterType.K3S:
                Console.WriteLine("Running on K3s lightweight distribution");
                break;
            case KubernetesClusterType.Eks:
                Console.WriteLine("Running on AWS EKS");
                break;
        }
    }
}

Integration with external-secrets-operator​

For production secret injection from Vault/Azure KeyVault, use external-secrets-operator (ESO):

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
  --namespace external-secrets-system --create-namespace

Define SecretStore:

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-secret-store
  namespace: muonroi
spec:
  provider:
    vault:
      server: "https://vault.example.com:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "muonroi-app"

Define ExternalSecret mapping:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: muonroi-secrets
  namespace: muonroi
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-secret-store
    kind: SecretStore
  target:
    name: muonroi-app-secrets
    creationPolicy: Owner
  data:
    - secretKey: JwtSigningKey
      remoteRef:
        key: production/jwt
        property: signing-key
    - secretKey: DatabaseConnectionString
      remoteRef:
        key: production/db
        property: connection-string

Pod references secrets:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: muonroi-control-plane
  namespace: muonroi
spec:
  template:
    spec:
      containers:
      - name: control-plane
        env:
        - name: MUONROI_JWT_SIGNING_KEY
          valueFrom:
            secretKeyRef:
              name: muonroi-app-secrets
              key: JwtSigningKey

Muonroi.ServiceDiscovery.Consul​

NuGet: Muonroi.ServiceDiscovery.Consul | Tier: OSS | Distribution: NuGet

Purpose​

Service discovery and registration for distributed Muonroi deployments. Integrates with Consul for dynamic service registration, health checks, and DNS-based service routing.

Key Types​

TypeKindPurpose
ConsulConfigsClassConfiguration: enable, service name, address, port, metadata
ConsulHandlerStaticRegistration helpers: AddServiceDiscovery(), UseServiceDiscovery(), UseServiceDiscoveryAsync()

DI Registration​

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// Register Consul client if enabled
builder.Services.AddServiceDiscovery(builder.Configuration, builder.Environment);

var app = builder.Build();

// Register with Consul on startup
await app.UseServiceDiscoveryAsync(app.Environment);

await app.RunAsync();

Configuration (appsettings.json)​

{
  "ConsulConfigs": {
    "Enable": true,
    "UseDiscovery": true,
    "Id": "muonroi-control-plane-instance-1",
    "ServiceName": "muonroi-control-plane",
    "ConsulAddress": "http://consul.example.com:8500",
    "ServiceAddress": "control-plane.muonroi.svc.cluster.local",
    "ServicePort": 8080,
    "ServiceMetadata": {
      "version": "1.0.0",
      "environment": "production",
      "region": "us-east-1"
    }
  }
}

Usage Example​

public class ServiceDiscoveryStartup : IHostedService
{
    private readonly ConsulConfigs _consulConfigs;
    private readonly IConsulClient? _consulClient;
    private readonly IMLog<ServiceDiscoveryStartup> _log;

    public ServiceDiscoveryStartup(
        IOptions<ConsulConfigs> consulOptions,
        IConsulClient? consulClient,
        IMLog<ServiceDiscoveryStartup> log)
    {
        _consulConfigs = consulOptions.Value;
        _consulClient = consulClient;
        _log = log;
    }

    public async Task StartAsync(CancellationToken cancellationToken)
    {
        if (!_consulConfigs.Enable || _consulClient == null)
        {
            _log.Info("Service discovery is disabled");
            return;
        }

        // Service registration occurs in UseServiceDiscovery middleware
        _log.Info($"Service {_consulConfigs.ServiceName} registered with Consul");
        await Task.CompletedTask;
    }

    public async Task StopAsync(CancellationToken cancellationToken)
    {
        // Deregistration handled automatically by ConsulHandler on app shutdown
        await Task.CompletedTask;
    }
}

// Query services from Consul (in client application)
public class ServiceLocator(IConsulClient? consulClient)
{
    public async Task<string?> LocateServiceAsync(string serviceName)
    {
        if (consulClient == null)
        {
            return null;
        }

        var services = await consulClient.Health.Service(serviceName, null, true);
        var service = services.Response.FirstOrDefault();

        if (service != null)
        {
            return $"http://{service.Service.Address}:{service.Service.Port}";
        }

        return null;
    }
}

Health Check Registration​

Consul can perform periodic health checks. Implement a liveness endpoint:

// In your controller
[HttpGet("/health")]
public IActionResult Health()
{
    return Ok(new { status = "healthy", timestamp = DateTime.UtcNow });
}

Then configure in Consul or via service metadata:

{
  "ConsulConfigs": {
    "ServiceMetadata": {
      "health-check-url": "http://localhost:8080/health",
      "health-check-interval": "10s",
      "health-check-timeout": "5s"
    }
  }
}

Kubernetes Integration​

For K8s deployments, typically use native K8s DNS (service-name.namespace.svc.cluster.local) instead of Consul:

{
  "ConsulConfigs": {
    "Enable": false
  }
}

But Consul can still be used for service discovery across namespaces or outside K8s:

apiVersion: v1
kind: ConfigMap
metadata:
  name: muonroi-consul-config
  namespace: muonroi
data:
  ConsulConfigs.Enable: "true"
  ConsulConfigs.ServiceName: "muonroi-control-plane"
  ConsulConfigs.ConsulAddress: "http://consul.external.svc:8500"

Comparison Matrix​

FeatureBackgroundJobs.HangfireBackgroundJobs.QuartzResilienceSecretsKubernetesServiceDiscovery.Consul
PurposeJob schedulingJob schedulingRetry/circuit breakerSecret accessK8s configService discovery
ProviderSQL queueSQL/in-memory storePolly 8+Pluggable (config/Vault/Azure)Native K8sConsul
Expression Jobs✓ Yes✗ No (class-based)————
DashboardBuilt-inExternal required————
ClusteringSQL locksNative clustering——NativeNative
Single Instance✓ Good✓ Good——✓ OK—
Multi-InstanceOK (contention)✓ Best——✓ Best✓ Best

Best Practices​

Background Jobs​

  • Idempotency: Design jobs to be safely retryable—running twice should not cause duplicate side effects
  • Tenant Context: Always pass IMuonroiJobExecutionContext to capture tenant/user/correlation context
  • Correlation IDs: Carry correlation IDs through logs for end-to-end tracing
  • Error Handling: Log errors with context before re-throwing; let the scheduler handle retries
  • Testing: Use in-memory Quartz or mock Hangfire in unit tests; use real database in integration tests

Resilience​

  • Circuit Breaker: Protects downstream services from cascading failures—monitor break duration
  • Timeouts: Always use timeouts to prevent indefinite hangs on slow/failing services
  • Jitter: Enable jitter on retries to avoid thundering herd during service recovery
  • Monitoring: Track retry counts and circuit breaker state via metrics/logs

Secrets​

  • Never commit secrets: Store all sensitive config in secret provider, never in code/config files
  • Rotate regularly: License keys annually, API keys every 6–12 months, DB passwords every 90 days
  • Principle of least privilege: Grant secret access only to required applications
  • Audit access: Enable logging in Vault/Azure KeyVault to track who accessed what and when

Kubernetes​

  • Use native DNS: For in-cluster service discovery, prefer K8s Service DNS over Consul
  • External Secrets Operator: Use ESO to sync secrets from Vault/Azure KeyVault into K8s Secrets
  • RBAC: Restrict service account permissions to only required APIs and resources
  • Health Checks: Implement liveness and readiness probes in your controllers

Service Discovery​

  • Metadata: Include version, environment, region, and other attributes in service metadata
  • Health Checks: Implement periodic health checks so Consul can deregister unhealthy instances
  • Graceful Shutdown: Deregister service on shutdown to avoid routing traffic to dead instances
  • DNS Integration: Use Consul DNS (service-name.service.consul) for service lookup