tnl.dev :: docs
operate a tnl server.
Check health, pause new work, and drain relay processes.
check readiness and visitor traffic
Run these checks against your own standalone or split server. Replace control.example.com and the relay identifiers with your deployment's values.
Inside a configured project, tnl admin uses that project's control URL by default. Pass --server to target a different server; admin results show the server used.
curl --fail https://control.example.com/v1/health
curl --fail https://control.example.com/v1/ready
tnl admin server status --server https://control.example.com
tnl admin relays list --server https://control.example.com/v1/health means the public control HTTP server responds. /v1/ready returns 200 when the database and, if configured, Route 53 credentials are ready, or 503 otherwise; check it through the public control hostname to include public DNS and TLS. It does not check the whole visitor path. tnl admin server status reports role, leases, and publish runs. To check an intentionally anonymous public HTTP service end to end, run tnl publish 3000 --server https://control.example.com --anonymous, then request its printed public URL. For restricted access, omit --anonymous and use an authorized visitor; --ip adds an optional source filter.
Each tnld process also has a private metrics listener (the checked-in Compose files listen on 0.0.0.0:9090 inside the container and only expose port 9090). Its GET /health returns 204 for a live process, GET /ready returns 204 or 503 for that role's readiness, and GET /metrics serves Prometheus data. Scrape over a private network; do not treat metrics readiness as a visitor probe. Review control, ingress, and relay logs alongside lease and certificate state when a public request fails.
operate publisher replicas
Use tnl status --scope=server in the applied project to compare desired and observed member revisions, health, and lease expiry. A reconciled revision or an ECS RUNNING task does not prove a target works. Connect with a normal client through tnl connect NAME for private services; check a real public response for public HTTP.
Hosted and self-hosted control use the same applied-project and publisher-group contract. Operators run the infrastructure. Follow publisher groups for independent replica identities and nonoverlapping replacement. Preserve signing/TLS keys and certificate state on replica-private /identity; only /state SQLite and runtime caches are disposable. Do not share identities across replicas or mount SQLite WAL on NFS/EFS.
Both public HTTP and private services select ready pool members per connection. New connections can use another member during replacement, but failed live traffic closes and is never replayed or migrated. Stop a replica before rotating its identity; stopping its infrastructure alone does not revoke it.
inspect publisher egress evidence
tnl status --scope=server includes each named-group member's runtime observation.
tnl publisher list --team shop shows the latest observation for managed publisher
identities, including non-group installed and credential-scoped publishers. The
control API exposes PublisherGroupMemberStatus.runtime and
PublisherIdentity.runtime; neither is browser-safe project metadata.
The running process signs start, renew, and stop operations. Control records the
source IP it actually observes, with the process run ID, epoch, observation time,
last-seen time, lease expiry, and active state. A replacement advances the epoch
and fences stale renewals. Stop and lease expiry preserve last-seen evidence but
mark it offline. Missing runtime means it has not started; address_family: unknown
means control observed no public routable address, including on a private-network
connection. Provisioning and key verification never supply an operator's address
as runtime evidence, and no external IP lookup service is used.
Read these records only through current team/publisher-management authorization: team managers can inspect group status, and a member can list their own member-owned publisher. They are not visitor access authority. Treat even an active address as evidence for the control path, not a database allowlist: a target can see different NAT or IPv4/IPv6 egress. Automatic PlanetScale/Neon allowlisting is not implemented. See publisher observations.
read team service audit
tnl audit list --team shop --since 24h
tnl audit list --team shop --service orders --since 24h --output jsonTeam managers read one history for public and private services. It includes management facts such as applied configuration, group membership, grants, and revocation, with actor and revision context. Those management facts commit transactionally with their mutations in PostgreSQL.
Visitor connection observations are unsampled, asynchronous reports. Reporters use bounded queues and batches, retry submissions, and deduplicate at control. A crash, queue overflow, bounded shutdown, or extended outage can lose observations. The stored history is not a complete durable request log. Source, identity, device, service, run, and visitor connection identifiers can aid correlation; events contain no HTTP paths, SQL, commands, application payloads, or secrets.
--service accepts a private service name, public hostname, or immutable ID. --since accepts a duration or RFC3339 timestamp; --limit bounds the result, and --after resumes from its cursor. Keep the same filters when paging. Prefer immutable IDs across renames.
Private process metrics expose tnl_service_audit_events_total{result} with queued, delivered, retry, dropped, accepted, and duplicate results. These are reporter/ingestion stages, not distinct visitor totals. Watch drops and retries alongside control health. Local inspection and pseudonymous CLI telemetry are separate from team audit history.
pause new work
The three maintenance controls independently gate new public URL creation, new publish run creation, and certificate issuance. Blocking an operation does not stop existing visitor connections. Check state before and after a change:
tnl admin maintenance list --server https://control.example.com
tnl admin maintenance block public_url_creation --server https://control.example.com
tnl admin maintenance block publish_run_creation --server https://control.example.com
tnl admin maintenance block certificate_issuance --server https://control.example.com
tnl admin maintenance list --server https://control.example.comUse tnl admin maintenance allow <name> --server https://control.example.com to reopen each control when ready. Block only the operations your maintenance requires; blocking publish run creation or certificate issuance prevents a publisher from becoming routable.
drain a relay process
In a split deployment, list relays and copy the exact relay ID, process run ID, and lease revision of the process to remove. Drain it with a deadline, then check that publisher connections move to available relay processes before stopping the container:
tnl admin relays list --server https://control.example.com
RELAY_RUN_ID='replace-with-process-run-id-from-list'
RELAY_LEASE_REVISION='replace-with-lease-revision-from-list'
tnl admin relays drain relay-a-1 --relay-run-id "$RELAY_RUN_ID" --relay-lease-revision "$RELAY_LEASE_REVISION" --deadline 30s --server https://control.example.com
tnl admin relays list --server https://control.example.comThe matching run ID and revision keep a stale drain request from affecting a restarted relay. Draining rejects new work and allows existing connections to finish until the deadline; ensure another relay service and capacity remain. Standalone has two logical relay services in one process, so draining one does not give host-level redundancy.
back up and upgrade
Back up external PostgreSQL consistently and test a restore. Store the matching TNLD_STORAGE_KEY securely with the backup: it decrypts recoverable control secrets. Keep TNLD_STORAGE_KEY_PREVIOUS during a key rotation until re-encryption completes, and retain keys needed by older backups. Publisher TLS private keys live in each publisher's durable identity directory, not in that control backup. Protect .env, OIDC app secrets, cluster secrets, database credentials, and identity backups separately.
The fresh control, client, and hosted accounts baselines are not rolling upgrades of older schemas. Stop incompatible processes and explicitly provision new state; do not convert or restore historical migration journals into the new baseline. Hosted accounts initialization refuses an incompatible nonempty database and is idempotent on its exact baseline. It creates no first-user administrator.
Hosted company directories use a minute cron with healthy five-minute scans and validity capped at fifteen minutes from scan start. Failed/partial scans never offboard unseen users or extend the previous deadline. GitHub signed webhooks wake a full scan; Google and Entra are periodic reads. Monitor the owner screen's last complete scan and expiry, not just cron success. Company-proof expiry requires reauthentication; it does not change tnl roles or stop team-owned unattended publishers.
Before changing the digest-pinned image, back up the database and keys and read the release and compatibility notes. The updated public URL target constraints require a fresh PostgreSQL baseline with a new baseline identity. They do not migrate an older deployment in place or provide an import path. Keep fresh client state too; do not import older SQLite or publisher credentials.
Coordinate a stop/start of all server roles:
- Stop the old control, ingress, and relay processes, or the old standalone process.
- Provision a fresh PostgreSQL database and run the selected image's
tnld migratewithTNLD_DATABASE_DIRECT_URL. - Start all roles with that same image. Supply pooled
TNLD_DATABASE_URLonly to control or standalone; serving processes require the exact schema version and baseline identity. - Check
/v1/ready, leases, applied revisions, and real public and private traffic.
Compose's migrate dependency does not make mixed-version rolling upgrades safe or create backups. For process settings, see tnld reference.