Skip to Content
LYNKORIADocs

Step 10: Monitoring

Turn on the metrics worker, load the alert rules, route App Platform alerts and keep logs somewhere searchable.

Goal: problems reach a person before users report them.

Managed by OpenTofu: App Platform alerts (deployment failed, domain failed, CPU, restarts) with email destinations from ALERT_EMAILS; the metrics worker (pinned Prometheus) when MONITORING_REMOTE_WRITE_URL is set; VPC-only firewall rules for LiveKit metrics (6789) and node_exporter (9100). Not managed: the monitoring backend, node_exporter on the Droplets, log forwarding and the external uptime probe.

10.1 Metrics worker

  1. Get a Prometheus remote-write URL and write-only credentials from your provider.
  2. Set the variable MONITORING_REMOTE_WRITE_URL and the secrets MONITORING_REMOTE_WRITE_USERNAME / MONITORING_REMOTE_WRITE_PASSWORD on the environment (step 4).
  3. Run Deploy. OpenTofu renders infra/opentofu/platform/templates/prometheus.yml.tftpl into an encrypted variable and adds the metrics worker to the app. It scrapes api:9464 and the media Droplet's private IP, keeps six hours locally and remote-writes the rest.

Check: doctl apps logs YOUR_APP_ID metrics --type run, then query up{job="lynkoria-api"} and up{job="livekit"} in your backend; both must be 1.

10.2 Alert rules and receivers

Load infra/monitoring/alerts.yml into your backend's rule engine. Every alert links its section in runbooks. Tune MediaEgressHigh to your real bandwidth allowance, configure a receiver that pages a person, and fire a test alert.

10.3 Host metrics

To get CPU, memory and disk for the Droplets, install node_exporter on them (port 9100 is already open to the VPC). Because Droplets are replaced on configuration changes, add it to the cloud-init in media.tf / scanner.tf rather than installing it by hand. DigitalOcean's own Droplet monitoring is on (monitoring = true) and gives basic graphs and alert policies in the control panel.

10.4 Logs and uptime

The API logs JSON with request IDs to stdout; secrets, tokens and message bodies are redacted. App Platform keeps logs for a limited time: add a destination under the app's Settings → Log Forwarding (Datadog, Logtail, Papertrail, OpenSearch and others) for retention you control. Add an external uptime check for https://api.lynkoria.com/health/ready.

Traces are exported only when OTEL_EXPORTER_OTLP_ENDPOINT points at a collector; it is not set by the OpenTofu stacks.

Verify

Both up series are 1, a test alert reached a person, and an App Platform alert email arrives when you trigger one (for example by a failed deploy on staging).

On this page