Docs
Tenantry Pro

Telemetry

View as Markdown

Tenantry.Pro names the tenant a job or message runs as in its logs and traces, and gives each of its log messages an event id you can alert on.

Requests are Tenantry Core's: app.UseTenantry() names a request's tenant in its logs and traces, and tenant.TagRequestMetrics() (Tenantry Core 0.7.0 and later) adds it to ASP.NET Core's request metric, http.server.request.duration, for request rates, latency and errors per tenant. See Core's diagnostics.

Logs and traces

The Hangfire, MassTransit, Quartz.NET and Rebus integrations name the tenant a job or message runs as, with no setup:

  • While the job, consumer or handler runs, a log scope with one property, TenantId, is open. A logging provider that records scopes (Serilog's, or the console's and OpenTelemetry's with IncludeScopes) adds it to every entry written while it runs.
  • The job's or message's trace span is tagged tenant.id: a Hangfire job's span (Tenantry's job filter tags it inside other filters, such as OpenTelemetry's Hangfire instrumentation), a Quartz.NET job's, a Rebus handler's, and a MassTransit routing-slip activity's. For a MassTransit consumer it is the message's receive span: the span MassTransit starts for the consumer is a child of it, started after Tenantry's filter. Without tracing, there is no span to tag.

The value is the tenant's id, formatted with the invariant culture, as the job or message carries it. A job or message that runs without a tenant gets neither.

Log event ids

Each of Tenantry.Pro's log messages has an event id that does not change between versions, so you can alert on it. Tenantry Core's are 1xxx and 2xxx (see its diagnostics). Alert on these:

  • 30xx: licence errors (Tenantry.Pro's features stop working)
  • 3104, 3108: a provisioning or offboarding step failed
  • 4103, 4113: a migration run had failures (each is also logged as 4105 or 4106)
  • 4114: a migration run stopped early
  • 4116: migrate-tenants could not start
  • 4302: audit entries were lost
Event idNameLevelWhen
3001LicenceValidatedInformationThe licence key is valid, at startup.
3002LicenceKeyMissingErrorNo licence key is configured.
3003LicenceKeyMalformedErrorThe licence key is not three base64url segments.
3004LicenceKeyUnreadableErrorThe licence key's segments are not base64url-encoded JSON.
3005LicenceKeyNotEs256ErrorThe licence key is not signed with ES256.
3006LicenceKeyWithoutKeyIdErrorThe licence key names no signing key: a format from before 0.5.
3007LicenceKeyUnknownSigningKeyErrorThe licence key was signed by a key this version does not know.
3008LicenceKeyInvalidSignatureErrorThe licence key's signature is invalid.
3009LicenceKeyNotForTenantryProErrorThe licence key has the wrong issuer or audience.
3010LicenceKeyWithoutFormatVersionErrorThe licence key has no format version: a format from before 0.5.
3011LicenceKeyUnsupportedFormatErrorThe licence key is in a format this version does not read.
3101TenantProvisionedInformationA tenant was provisioned.
3102ProvisioningStepRunningInformationA provisioning step starts for a tenant.
3103ProvisioningStepSkippedDebugA provisioning step does not apply to a tenant.
3104ProvisioningStepFailedErrorA provisioning step failed for a tenant.
3105TenantDeprovisionedInformationA tenant was deprovisioned (offboarded).
3106DeprovisioningStepRunningInformationA deprovisioning step starts for a tenant.
3107DeprovisioningStepSkippedDebugA deprovisioning step does not apply to a tenant.
3108DeprovisioningStepFailedErrorA deprovisioning step failed for a tenant; the steps after it did not run.
3201TenantWorkFailedErrorA TenantBackgroundService's work failed for a tenant; the other tenants go on.
3202SweepFailedErrorA PeriodicTenantBackgroundService's sweep failed; the next one runs on schedule.
3203InactiveTenantSkippedDebugBackground work or a for-each-tenant schedule skips a tenant ValidateTenantActivity refuses.
3204SweepSkippedDebugA TenantBackgroundService's ShouldRunAsync returned false, so this instance skips the sweep.
3301ConnectionStringCacheBypassedWarningCacheConnectionStrings was called, but a connection string provider registered after UsePro replaced the cache.
3302ConnectionStringTenantUnreadableWarningThe tenant store could not be read for an invalidated tenant's connection string, so CacheConnectionStrings read it from the caller's descriptor and did not cache it. It does the same without reading the store for that tenant for the next 5 seconds, so it is logged at most once per tenant in that time.
3401JobWithoutTenantRunningWarningA job or message carries no tenant and runs without one (OnMissingTenant is Warn).
3402JobWithoutTenantSkippedWarningA job or message carries no tenant and is skipped (OnMissingTenant is Skip).
3403JobWithInvalidTenantRunningWarningA job or message carries an invalid tenant id and runs without a tenant (OnUnresolvedTenant is Warn).
3404JobWithInvalidTenantSkippedWarningA job or message carries an invalid tenant id and is skipped (OnUnresolvedTenant is Skip).
3405JobWithUnknownTenantRunningWarningA job or message carries a tenant the store does not have and runs without a tenant (OnUnresolvedTenant is Warn).
3406JobWithUnknownTenantSkippedWarningA job or message carries a tenant the store does not have and is skipped (OnUnresolvedTenant is Skip).
3407JobWithInactiveTenantRunningWarningA job or message carries a tenant that is not active and runs without a tenant (OnUnresolvedTenant is Warn).
3408JobWithInactiveTenantSkippedWarningA job or message carries a tenant that is not active and is skipped (OnUnresolvedTenant is Skip).
4001TenantDatabaseExistsDebugA tenant's database already exists, so provisioning does not create it.
4002TenantDatabaseCreatedElsewhereWarningCreating a tenant's database failed, but it exists: probably another provisioning run created it.
4003TenantDatabaseCreatedInformationA tenant's database was created.
4004TenantSchemaInPlaceInformationA tenant's schema exists, created or already there.
4005TenantDatabaseDroppedInformationA tenant's database was dropped (AddDatabaseDeprovisioning).
4006TenantSchemaDroppedInformationA tenant's schema was dropped (AddSchemaDeprovisioning).
4007TenantSharedDataDeletedInformationA tenant's rows were deleted from the shared database (AddSharedDataDeletion).
4008PoolNotClearedDebugProvisioning found a tenant's database, and the connection's provider has no ClearPool to release the pooled connection it checked with.
4101MigrationRunStartingInformationA migration run starts (MigrateAsync, MigrateAllAsync, MigrateTenantAsync, migrate-tenants).
4102MigrationRunCompletedInformationA migration run completed with no failures.
4103MigrationRunCompletedWithFailuresWarningA migration run completed, and some databases or schemas failed (each failure is logged as 4105 or 4106).
4104MigrationProgressReporterFailedWarningThe progress reporter passed to a migration run threw; the run goes on.
4105MigrationFailedErrorApplying a context's migrations failed for a tenant.
4106MigrationContextNotCreatedErrorA context could not be created for a tenant to apply its migrations.
4107MigrationsPendingInformationA tenant's database or schema has migrations to apply.
4108TenantMigratedInformationA tenant's database or schema was migrated.
4109MigrationStatusUnreadableWarningGetStatusAsync, GetTenantStatusAsync or the migration health check could not read a tenant's migrations.
4110MigrationStatusContextNotCreatedWarningGetStatusAsync, GetTenantStatusAsync or the migration health check could not create a context for a tenant.
4111StartupMigrationsStartingInformationMigrations at startup begin.
4112StartupMigrationsCompletedInformationMigrations at startup completed with no failures.
4113StartupMigrationsFailedWarningMigrations at startup failed for some databases or schemas.
4114MigrationRunStoppedWarningA migration run stopped before attempting every database or schema: MaxFailures (--max-failures) was reached, or migrate-tenants was asked to stop.
4115MigrationRunStoppingWarningmigrate-tenants got SIGTERM or Ctrl+C: the databases or schemas in progress finish, and no other starts.
4116MigrationRunNotStartedErrormigrate-tenants could not start: invalid arguments, no valid licence, a tenant the store does not have, or a tenant store that cannot be read.
4201TenantDatabaseUnreachableWarningThe database health check could not reach a tenant's database.
4202DatabaseHealthCheckContextNotCreatedWarningThe database health check could not create a context for a tenant.
4301AuditEntryInformationThe logging audit store writes an audit entry.
4302AuditEntriesNotWrittenErrorAudit entries for committed changes were not written: the store could not be resolved, or it threw.
4401SchemaPerTenantContextNotCheckedWarningSchema per tenant could not create a registered context as the host started, so whether it calls UseTenantry() is checked only when Pro creates it for a tenant. Not logged for a context from AddDbContextPerTenantDatabase that needs a tenant to find its database.
4402SharedTenantsRefusedContextWarningIn mixed mode, a context has entity types that are neither tenant-owned nor marked as shared, so a Shared tenant is refused it. Logged once per context type, in the category Tenantry.Pro.MixedMode.

See also

On this page