Troubleshooting

Troubleshooting and gotchas for the HVC services, grouped by service.

HVC Services (hvc-services)

• Missing config in development — the service loads extra config from {DEV_CONFIG_PATH}/hvc/services-config/appsettings.json. If DEV_CONFIG_PATH is not set (or points at the wrong folder), the service may start with missing/incomplete config. Verify the env var and that the config file exists.
• Emails stuck in the outbox — entries that fail send stay in emailOutboxEntries with retry/back-off. Check Mailgun credentials and the email/mailgun config, and look for errors in Seq.
• Recording filename convention is load-bearing — env__eventId__tenantId__... is parsed by both hvc-filePushApi and this service. Changing it requires updating both.
• One worker per environment — EmailSenderService runs per entry in environments. A missing environment entry means that environment never sends email.
• .ics attachments — calendar attachments are handled specially; malformed calendar data can fail a batch.

HVC Sink Api (hvc-sinkapi)

• "Processor not found" — POST /api/Cleanup/scan?environment=... returns this when there is no matching entry in processors[] for the given environment. Add the entry.
• Events not finalizing — verify the Pexip management API credentials in the processors[] entry and that the tenant DB connection string is correct.
• "New or Invalid tenant" — pexipDataEntries rows for an unknown tenant id are skipped. Check the tenant mapping (externalTenantId ↔ admin tenants.id).
• Already-processed conferences are skipped — matching is de-duplicated via eventInstances.pexipConferenceId; re-running is safe but won't duplicate.
• Stale rooms — cleanup clears rooms with no matching conference (assignedOn = null, eventId = 0). If rooms look stuck, run cleanup manually.
• Self-signed Pexip certificates — the service bypasses certificate validation; if the management node is unreachable, check TLS/network rather than certificates.

HVC Ws (hvc-ws)

• Seq is not wired up — the appsettings.json has a Serilog → Seq block (Application=HVC-WS) but Program.cs never calls UseSerilog, so no logs reach Seq.
• State is in-memory — restarting the service clears all presence rosters; clients reconnect and re-register on their own.
• Keep-alive — WebSocket keep-alive interval is 30 s; idle proxies may need tuning.

HVC File Push Api (hvc-filePushApi)

• Files not uploaded — the poll loop only picks files containing __ (the env__eventId__tenantId__... pattern). Files without it are ignored.
• No Seq logging — this service logs to the console, not Seq. Use container logs for debugging.
• concurrency is ignored — the upload loop is serial despite the config key.

HVC Web Logger (hvc-webLogger)

• Invalid templates — the body template must be a valid Serilog message template; mismatched properties count can throw.
• CORS — the service allows any origin by design (browser clients), so it should not be exposed publicly without consideration.

HVC Storage / HVC Shared (libraries)

• Unknown store type — HvcStorageFactory.getClientStore() throws for an unknown storeType; use file, azure/azure-storage, or s3/minio.
• Duplicated blob wrappers — hvc-admin-api and hvc-filePushApi carry their own SimpleAzureBlobFileSystem copies instead of referencing hvc-storage. Prefer the shared library for new code.

General

• Seq coverage — all services log to Seq except hvc-ws (not wired) and hvc-filePushApi (console).
• Development config — each service loads dev config from DEV_CONFIG_PATH (services-config, sink-config, filepush-config). Keep these folders in sync with the environment you are running against.
• Pexip dependency — hvc-api (VMR creation), hvc-sinkapi (finalization), and the recording pipeline all depend on Pexip/OvenMediaEngine being healthy.