Runbook¶

Operational procedures for the production EC2 environment.

For Tier-1 Carrier customers, additional gating applies — see carrier-noc-runbook.md (24/7 NOC paging, per-component RTO/RPO, change-window policy, per-customer sign-off checklist).

Quick reference¶

Resource	Value
EC2 instance ID	`i-045166a6a1933f507`
Region	`sa-east-1`
SSH user	`ubuntu`
Project path on host	`/home/ubuntu/TELECOM-TOWER-POWER`
Secrets path on host	`/home/ubuntu/TELECOM-TOWER-POWER/secrets/`
Caddy entry point	`:80` (ALB terminates TLS)
Railway API (custom domain)	`https://api.telecomtowerpower.com.br`
Railway edge target	`web-production-90b1f.up.railway.app` (pode ser rotacionado — re-confirmar no painel da Railway antes de mexer no DNS)

NÃO remova o registro TXT de verificação da Railway

O Route 53 contém um TXT _railway-verify.api.telecomtowerpower.com.br. A Railway usa esse registro para manter válido o certificado Let's Encrypt emitido para api.telecomtowerpower.com.br. Se qualquer automação (Terraform, external-dns, scripts de limpeza, etc.) apagar esse registro, a perna SECONDARY do failover vai falhar com erro de TLS na próxima vez que o ALB ficar unhealthy. Ele nunca deve ser editado sem antes confirmar o valor atual no painel da Railway.

Detectar drift antes de um incidente

Execute scripts/verify_failover.sh (somente leitura, seguro em cron/CI). Ele verifica se o CNAME SECONDARY ainda aponta para o edge correto da Railway, se o TXT _railway-verify está presente e se o edge ainda serve um certificado que cobre api.telecomtowerpower.com.br. Se a Railway rotacionar o edge, rode novamente scripts/setup_failover.sh com o novo valor: RAILWAY_DNS=<novo>.up.railway.app ./scripts/setup_failover.sh.

Common operations¶

Deploy latest main to EC2¶

Trigger the workflow:

gh workflow run deploy-ec2-docker.yml

The workflow opens an ephemeral SSH ingress on the EC2 security group, pushes a one-shot key via EC2 Instance Connect, then over SSH runs git pull && docker compose build && docker compose up -d on the host. Rolling replacement is performed per service — no manual downtime window needed.

Sync Stripe secrets to EC2¶

gh workflow run update-ec2-stripe-secrets.yml

Reads STRIPE_SECRET_KEY and STRIPE_WEBHOOK_SECRET from GitHub Actions secrets and writes them to /home/ubuntu/TELECOM-TOWER-POWER/secrets/. Containers consuming these secrets are restarted automatically.

Sync alerting secrets (Slack + SES) to EC2¶

gh workflow run update-ec2-alerting-secrets.yml

Syncs SLACK_WEBHOOK_URL, SES_SMTP_USERNAME, SES_SMTP_PASSWORD. After sync, Grafana is restarted and a health check is retried up to 30 × 2 s against localhost:3001/api/health.

Reload Caddy after a config change¶

gh workflow run deploy-caddy.yml

Copies the Caddyfile to EC2 and runs caddy reload. Verifies HTTP 200 on www.*, app.*, api.*, monitoring.*, and prometheus.*.

Troubleshooting¶

Grafana health check fails during alerting-secrets workflow¶

Ensure the port is 3001 on the EC2 host (not 3000 — that's the React frontend).
SSH in and check: docker compose logs grafana --tail=50.
Verify the secret files exist and are non-empty: ls -la /home/ubuntu/TELECOM-TOWER-POWER/secrets/.

502 on `api.telecomtowerpower.com.br`¶

First check Railway status — the Caddy config proxies 100% of API traffic there.
Check Caddy logs on EC2: docker compose logs caddy --tail=100.
Confirm ALB target is healthy: AWS console → Target Groups → ttp-caddy-tg.

5xx alert fires¶

The high-5xx-rate alert triggers when the API emits more than 10 errors/minute over a 1-minute window. Steps:

Check Grafana dashboard for the affected endpoint.
docker compose logs api --tail=200 on EC2, or check Railway logs if traffic is routed there.
If the spike is from a specific customer, check rate limits in key_store.json.

Monitoring subdomain returns 404/502¶

monitoring.* and prometheus.* bypass Caddy and hit ALB target groups directly.
Verify targets are healthy: ttp-grafana-tg (port 3001), ttp-prometheus-tg (port 9090).
If the EC2 instance was replaced, re-register it with scripts/manage_ec2_alb.sh register.

Security reminders¶

Secrets are never committed. If a secret leaks into git, rotate it immediately and force-push a history rewrite only as a last resort — prefer rotation.
All remote EC2 execution goes through SSM; no SSH keys are configured in CI.
Secret files on EC2 are mounted read-only at /run/secrets/<name> inside containers.

AI Playground — Ollama (self-hosted on Railway)¶

The AI Playground (/bedrock/* routes — chat, models, compare, batch-analyze, suggest-height) is backed by self-hosted Ollama (llama3.2:3b) running on Railway, selected via LLM_PROVIDER=ollama. Amazon Bedrock remains a wired-but-disabled legacy backend (its AWS credentials were revoked 2026-06-24; the /bedrock/* route names are kept for backward compatibility). It is treated as a production component, not a prototype.

Ownership¶

Concern	Owner	Source of truth
Code	`@danielnovais-tech`	`.github/CODEOWNERS`
Compliance / DPA	`@danielnovais-tech`	`DPA-LGPD.md` §6.1
Cost	`@danielnovais-tech`	Railway compute (the `ollama` service)

SLO¶

Metric	Target	Breach action
p95 latency (`/bedrock/chat`)	≤ 8 s (warm)	Page on-call if sustained > 15 min. First token after idle is slower; mitigated by `OLLAMA_KEEP_ALIVE=30m` on the `ollama` service
Error rate (5xx + 502)	≤ 1 % over 1 h	Investigate the `ollama` Railway service health (`/health/failover`)
Availability	≥ 99.0 % monthly	Post-mortem if breached
Cost cap	Railway compute for the `ollama` service (no per-token Bedrock egress)	Watch the Railway usage dashboard
Per-tenant quota	Enforced by `_enforce_metric_quota("ai", …)`	429 to caller

Metrics are exported via the existing Prometheus surface (LLM_EGRESS_TOTAL, plus standard FastAPI request histograms). Grafana dashboard panel: AI Playground / Bedrock.

Provider switch — `LLM_PROVIDER` / `BEDROCK_ENABLED`¶

Production runs on Railway, so the provider is controlled by service environment variables, not an EC2 .env. Ollama is already the primary provider (LLM_PROVIDER=ollama) and Bedrock is disabled (its AWS credentials were revoked 2026-06-24). Use these steps to force-disable the Bedrock surface (compliance request, cost spike) or to re-pin the provider. Expected wall-clock time: under five minutes.

Set the env vars (read at process start; setting a var triggers a redeploy):

railway variable set "LLM_PROVIDER=ollama" --service web --environment production
railway variable set "BEDROCK_ENABLED=0"  --service web --environment production
railway redeploy --service web --environment production --yes

Verify the live provider — do not use /health (it only returns {"status":"healthy"}):

curl -s https://api.telecomtowerpower.com.br/platform/maturity \
  | jq '.hosting_portability.dependencies.llm_inference'
# expect: { "active": "ollama", "aws_backed": false, ... }

/health/failover also reports failover readiness.

Open an incident ticket and tag @danielnovais-tech as code owner.
To re-enable Bedrock (requires valid AWS credentials on the web service): set BEDROCK_ENABLED=1 / LLM_PROVIDER=bedrock and redeploy.

The kill switch is unit-tested (tests/test_bedrock_service_hardening.py::test_kill_switch_disables_bedrock_provider) and wired into the import-time guard in llm_provider.py.

Patch cadence¶

boto3 / botocore / s3transfer updates are isolated in the aws-sdk Dependabot group (see .github/dependabot.yml) so they land in a single weekly PR.
CI gate: .github/workflows/test-llm-provider.yml runs the routing and hardening test suites on every push touching the AI Playground.

Entitlement probe (medium-term, gated on AWS grant)¶

Until the Amazon Bedrock model-access request for this account/region is approved, every invoke-model in sa-east-1 (bedrock-br policy) returns ValidationException: Operation not allowed. The short-term graceful fallback in bedrock_service.py (see Task 1) hides the 500s, but the "refused" path must be relaxed once real inference works.

Automated probe (no more manual aws bedrock-runtime invoke-model every few hours):

scripts/probe_bedrock_entitlement.py

Runs a 1-token minimal invoke (cost ≈ $0).
Exits 0 + prints STILL_REFUSED while the exact refusal string is present.
On first success (entitlement granted) → prints SUCCESS, fires optional Slack/PagerDuty via SLACK_WEBHOOK_URL / PAGERDUTY_INTEGRATION_URL, exits 0.
Cron example (every 2 h):

0 */2 * * * cd /srv/telecom-tower-power && \ python3 scripts/probe_bedrock_entitlement.py \ >> /var/log/bedrock-entitlement-probe.log 2>&1

When SUCCESS appears: 1. Confirm the Bedrock fix + telemetry + tests are in main. 2. (Optional) relax the if ValidationException and "Operation not allowed" branch in bedrock_service.py so real Bedrock becomes the dominant path again (keep the refused dict as a safety net for future transient errors). 3. ~~gh workflow run deploy-lambda.yml -f stage=prod~~ (workflow retired 2026-07 — AWS account closed; prod LLM is Ollama on Railway) 4. Update this runbook + close the internal entitlement ticket.

The script is intentionally dependency-light and can also run as a GitHub Action or k8s CronJob if preferred.

Decommission criteria¶

The Bedrock backend will be retired if any of the following hold for two consecutive months: <USD 5 / month spend, zero PRO+ tenants with AI quota usage, or compliance scope change disallowing US-region inference (see DPA-LGPD §8). In that case set BEDROCK_ENABLED=0 permanently, remove /bedrock/* endpoints, and follow the static-page replacement plan in this commit's PR description.

If a global retrain fails with "refused for global training", that is the gate working, not a bug. Do not "fix" it by widening a WHERE or by reverting the migration.

What changed and why the retrain stops¶

consent_for_training (#1920, migration a3f19c7e5d21) separates two purposes LGPD treats separately: processing a measurement to serve the tenant who sent it, versus training the global model that also serves that tenant's competitors. Only the first was ever agreed to.

The migration stamps every historical real measurement 0 (refused). It deliberately does not backfill source LIKE 'drivetest%' → consented: source is caller-controlled free text, so that backfill would launder exactly the rows the gate exists to stop — silently and irreversibly.

So on a store whose rows all pre-date the column, the consented corpus is empty, and the trainers refuse:

CalibrationInputError: observation store holds 7026 row(s) but 7026 are
refused for global training and none are consented (LGPD art. 9 §2, #1920).
This is a consent state, not an empty dataset ...

That refusal is the point. "No data" and "data you may not use" must not produce the same silent empty training run.

Declaring a historical campaign¶

Per campaign, with the basis recorded. Dry-run first — that is the default.

# 1. What would change?
python scripts/stamp_training_consent.py --source drivetest_tim_b3_20260804

# 2. Declare it. --basis is mandatory and goes to the audit log.
python scripts/stamp_training_consent.py \
  --source drivetest_tim_b3_20260804 \
  --basis "campanha própria, protocolo de campo interno, TKT-1234" \
  --apply

There is no --all and wildcards are refused: a bulk unlock is exactly what this must not be. Withdrawal is --withdraw, same shape.

Before stamping, satisfy yourself the campaign really is ours to consent for. A drivetest_* label is not evidence — any PRO+ tenant could have written it. Check submitted_by, the batch notes, and the campaign inventory (INVENTARIO_TRABALHO_CAMPO.md).

Going forward¶

New first-party ingest should declare consent at submission time rather than be stamped afterwards — send consent_for_training: true in the batch body. Rows arriving without it are stored, serve their tenant, and stay out of the model; the batch response reports granted_for_training / refused_for_training and the refusal is audited as observation.rejected.no_consent.

WISP data dividend — o ledger registra, o operador precifica¶

Se uma liquidação falhar com "liquidação de data dividend RECUSADA", isso é o desenho funcionando. Não conserte declarando um valor qualquer para destravar: a recusa existe porque ninguém assinou um.

O que existe e o que não existe¶

consent_for_wisp_training (#1921) admite uma medição no corpus do segmento WISP. A contrapartida por esse consentimento é o data dividend (#1923), e ele está pela metade de propósito:

Existe hoje	Não existe, e não deve ser inventado
`data_dividend_events` — quais observações foram remuneradas	Quanto cada uma vale
O eixo de consentimento que as tornou remuneráveis	A forma (crédito de fatura / desconto de tier / pagamento)
Recusa de liquidação com remediação no texto	Qualquer chamada a Stripe, cupom ou nota de crédito

As três formas candidatas não são intercambiáveis: crédito de fatura abate um valor devido, desconto de tier muda a assinatura daqui pra frente, e pagamento move dinheiro para um terceiro — desenhos de dados e consequências fiscais diferentes. Escolher "a mais fácil" não é uma decisão neutra, é a decisão comercial tomada por quem não devia.

Registrando os eventos remunerados¶

Idempotente por construção (UNIQUE(observation_id)): pode rodar de novo depois de um lote parcial ou de um carimbo novo sem creditar ninguém duas vezes. Dry-run é o default.

# 1. O que seria registrado — e se a liquidação já é possível
python scripts/record_data_dividend_events.py

# 2. Registrar. --basis é obrigatório e vai para o audit log.
python scripts/record_data_dividend_events.py \
  --basis "contrapartida do consentimento WISP, TKT-1234" --apply

Toda linha entra com amount_brl_cents, settlement_form e settled_at NULL — não zero. 0 e NULL são afirmações diferentes: a primeira diz "liquidado por nada", a segunda diz "não liquidado". Lido daqui a dois anos, só a segunda é honesta.

Um rótulo source não gera dividendo. Só a declaração explícita consent_for_wisp_training, de um titular identificável (submitted_by). Sementes sintéticas nunca geram: física não tem conta bancária.

Quando o operador decidir¶

Fixe as duas variáveis — e só as duas destravam qualquer coisa:

DATA_DIVIDEND_FORM=invoice_credit   # | tier_discount | payout
DATA_DIVIDEND_RATE_BRL_PER_EVENT=<a taxa que você decidiu>

Uma taxa <= 0 é recusada, não aceita como "ainda não decidido": "de graça" é uma posição comercial. Se a base da sua decisão não for taxa por evento (revenue share, tabela por faixa), o código precisa ser estendido — aproximar para taxa por evento seria inventar o compromisso com passos extras. O ledger não se compromete com base nenhuma: um evento por linha, qualquer base calculável depois.

Declarar as variáveis torna a política construível; não liquida nada sozinho. A implementação da liquidação é um PR posterior, escrito contra a decisão assinada, e ele terá de editar scripts/lint_data_dividend.py à vista para conseguir escrever nas colunas de montante. É esse o desenho: registrar é fácil e idempotente, liquidar é deliberado.

DSAR — LGPD art. 18 (Data-Subject Access Requests)¶

Hard legal deadline: 15 calendar days. LGPD art. 18 §5 requires a substantive response within 15 days of the request. The dsar_requests.due_by column is the SLA clock; the GET /admin/dsar/overdue endpoint surfaces any open row past it. Treat an overdue row as a Sev-2 incident.

Ownership¶

Role	Person
DPO (LGPD)	Daniel Azevedo Novais
Technical owner	Daniel Azevedo Novais
Code owners	See `.github/CODEOWNERS` for `dsar_service.py` and DSAR migrations

Two paths — check the self-service one FIRST¶

Since the tenant self-service endpoints shipped, most art. 18 requests never need to reach this runbook. Before opening a ticket, check which path applies:

Situation	Path	Why
Subject still holds their API key	Self-service — point them at `GET /tenant/me/data/export` / `POST /tenant/me/data/forget`	The key IS the identity proof; no ticket, no TOTP, no operator in the loop, and the 15-day clock is satisfied instantly
Subject lost the key, or requests by e-mail / legal representative / court order	Admin, below	Identity must be established out-of-band before an operator acts on someone else's data
Requester is an admin acting on a tenant's behalf	Admin, below	The self-service endpoints refuse admin keys with 403 precisely so this action keeps its justification + MFA trail

Self-service calls are what the subject runs themselves:

# Export (art. 18 II/V) — the caller's own bundle, nothing else.
curl -sS -H "X-Api-Key: $YOUR_API_KEY" \
  https://api.telecomtowerpower.com/tenant/me/data/export \
  > my-data.json

# Erase (art. 18 VI) — irreversible for the key; confirmation is literal.
curl -sS -X POST \
  -H "X-Api-Key: $YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"confirmation":"DELETE MY DATA"}' \
  https://api.telecomtowerpower.com/tenant/me/data/forget

After a successful forget, that API key is dead: every subsequent call returns 401 API key has been erased. That is the expected end state, not an incident — do NOT "fix" it by restoring the tier. If a subject reports a 500 there instead, that IS an incident: it means a key-resolution path is casting the stored tier straight to Tier(...) again (see _tier_or_none in telecom_tower_power_api.py and tests/test_tenant_dsar_self_service.py).

Both paths write to the same dsar_requests table and the same SLA clock. They differ in the audit taxonomy: tenant.dsar.export / tenant.dsar.forget.soft for self-service, admin.dsar.* for the mediated path. When compiling evidence for an auditor, do not merge the two — the distinction is the proof that the subject acted for themselves.

Receiving a request¶

(Everything below is the ADMIN path — use it only when the table above says so.)

Data subject e-mails the contact in DPA-LGPD.md §3 ("Contato do encarregado"). Open a support ticket and capture: the requester's e-mail, the right being exercised (acesso, correção, eliminação, portabilidade, anonimização), and the proof of identity.
Verify identity. Acceptable proof: confirmation reply from the e-mail on file in api_keys.email OR a signed declaration with a government ID number that matches the contract.
Record the ticket id; it goes into the justification field of every admin API call below.

Serving an EXPORT request (art. 18 II/V — acesso, portabilidade)¶

# 1. Step-up MFA: read your current TOTP code.
# 2. Resolve the tenant by api_key prefix (≥6 chars).
curl -sS -X POST \
  -H "X-Api-Key: $ADMIN_API_KEY" \
  -H "X-Admin-Totp: $TOTP_CODE" \
  -H "Content-Type: application/json" \
  -d '{"totp":"'"$TOTP_CODE"'","justification":"LGPD TKT-1234 acesso","confirm_subject_email":"user@example.com"}' \
  https://api.telecomtowerpower.com/admin/dsar/ttp_abcd1234/export \
  > dsar-export-<ticket>.json

The response contains a bundle field with:

identity — the api_keys row (the subject's own PII)
audit_log — last 1000 audit rows for the tenant
batch_jobs — every job the tenant submitted
observations — link_observations rows where submitted_by = api_key
billing — Stripe customer / subscription IDs (full payment history is exported separately from Stripe per DPA §6.1)

Deliver the JSON to the data subject through the same channel that originated the ticket. Mark the ticket resolved and reference the returned dsar_request_id.

Serving an ERASE request (art. 18 VI — eliminação)¶

curl -sS -X POST \
  -H "X-Api-Key: $ADMIN_API_KEY" \
  -H "X-Admin-Totp: $TOTP_CODE" \
  -H "Content-Type: application/json" \
  -d '{"totp":"'"$TOTP_CODE"'","justification":"LGPD TKT-1234 eliminação","confirm_subject_email":"user@example.com"}' \
  https://api.telecomtowerpower.com/admin/dsar/ttp_abcd1234/erase

This is a soft delete:

api_keys.email, owner, branding, oauth_*, stripe_* columns are replaced with a deterministic pseudonym; tier flips to "erased" so every auth gate refuses the key immediately.
audit_log rows are retained (DPA §10 — obrigação legal de guarda fiscal/contábil) but actor_email, ip, and user_agent are pseudonymised/cleared in the same transaction.
A hard-delete is scheduled for +30 days in dsar_requests.erase_scheduled_at. The 30-day window guarantees the 14-day backup rotation (DPA §10) cycles fully before the row physically disappears, so backups cannot resurrect erased PII.

Communicate to the data subject: the account is immediately inaccessible, billing has stopped, and physical row deletion will complete within 30 days.

Hard-delete worker¶

Physical purge after the 30-day grace window is automated by scripts/dsar_hard_delete.py, deployed as the SAM Lambda DsarHardDeleteFunction and triggered by an EventBridge schedule daily at 03:00 UTC (low-traffic for the Brazilian operating window). Failures are routed to the SQS telecom-dsar-hard-delete-dlq-<stage> queue (14-day retention) and CloudWatch alarms can be wired against the TelecomTowerPower/Dsar::HardDeleteRowsFailed EMF metric.

SLA promise to the data subject: physical deletion completes within 24 hours of the 30-day soft-delete mark (i.e. ≤ 31 days total). The worker is idempotent: re-runs against an already-purged tenant are a no-op.

For each row in dsar_requests where request_type='erase', erase_executed_at IS NULL, and erase_scheduled_at < now(), the worker — inside ONE transaction per tenant:

Deletes from every per-tenant child table (batch_jobs, auth_refresh_tokens, mfa_secrets, overage_events, pdf_usage_monthly, quota_alert_state, quota_usage_monthly, request_usage_monthly, webhook_deliveries, webhooks, stripe_webhook_events, link_observations).
DELETE FROM api_keys WHERE api_key = $1 (the root row).
UPDATE dsar_requests SET erase_executed_at = now(), status='completed'.
Writes an audit_log row with action='dsar.hard_delete' and the per-table delete counts.

Operational commands¶

Dry-run (count only — no DELETEs):

aws lambda invoke \
  --function-name $(aws cloudformation describe-stacks \
      --stack-name telecom-tower-power --region sa-east-1 \
      --query 'Stacks[0].Outputs[?OutputKey==`DsarHardDeleteFunctionArn`].OutputValue' \
      --output text | awk -F: '{print $NF}') \
  --payload '{"dry_run":true}' \
  --cli-binary-format raw-in-base64-out \
  --region sa-east-1 /tmp/dsar.json && cat /tmp/dsar.json

Manual force-run (skips the schedule):

aws lambda invoke --function-name <DsarHardDeleteFunctionArn> \
  --payload '{}' --region sa-east-1 /tmp/dsar.json

DLQ inspection:

aws sqs get-queue-attributes \
  --queue-url $(aws cloudformation describe-stacks \
      --stack-name telecom-tower-power --region sa-east-1 \
      --query 'Stacks[0].Outputs[?OutputKey==`DsarHardDeleteDLQUrl`].OutputValue' \
      --output text) \
  --attribute-names ApproximateNumberOfMessages --region sa-east-1

Local CLI run (for example, the on-prem deployment without Lambda):

DATABASE_URL=postgresql://... python scripts/dsar_hard_delete.py --dry-run
DATABASE_URL=postgresql://... python scripts/dsar_hard_delete.py

SLA monitoring¶

Synthetic monitor / on-call dashboard polls GET /admin/dsar/overdue every 24 h. Any non-zero count pages on-call.
Manual check: SELECT * FROM dsar_requests WHERE status IN ('received','processing') AND due_by < extract(epoch from now()) ORDER BY due_by ASC;

Audit trail¶

Every export and erase produces two audit rows: one in the target tenant's timeline (admin.dsar.export / admin.dsar.erase.soft) and one in the admin's own timeline (admin.dsar.export.served / admin.dsar.erase.served). The dsar_requests.id is included in both metadata blobs so the rows can be correlated.

Runbook¶

Quick reference¶

Common operations¶

Deploy latest main to EC2¶

Sync Stripe secrets to EC2¶

Sync alerting secrets (Slack + SES) to EC2¶

Reload Caddy after a config change¶

Troubleshooting¶

Grafana health check fails during alerting-secrets workflow¶

502 on api.telecomtowerpower.com.br¶

5xx alert fires¶

Monitoring subdomain returns 404/502¶

Security reminders¶

AI Playground — Ollama (self-hosted on Railway)¶

Ownership¶

SLO¶

Provider switch — LLM_PROVIDER / BEDROCK_ENABLED¶

Patch cadence¶

Entitlement probe (medium-term, gated on AWS grant)¶

Decommission criteria¶

Training consent — the global retrain refuses an unconsented corpus¶

What changed and why the retrain stops¶

Declaring a historical campaign¶

Going forward¶

WISP data dividend — o ledger registra, o operador precifica¶

O que existe e o que não existe¶

Registrando os eventos remunerados¶

Quando o operador decidir¶

DSAR — LGPD art. 18 (Data-Subject Access Requests)¶

Ownership¶

Two paths — check the self-service one FIRST¶

Receiving a request¶

Serving an EXPORT request (art. 18 II/V — acesso, portabilidade)¶

Serving an ERASE request (art. 18 VI — eliminação)¶

Hard-delete worker¶

Operational commands¶

SLA monitoring¶

Audit trail¶

502 on `api.telecomtowerpower.com.br`¶

Provider switch — `LLM_PROVIDER` / `BEDROCK_ENABLED`¶