Engineering docs

Uptime monitoring + error tracking

Last updated Oct 20, 2018 · 5 sections

Two free external monitors and one Sentry project per side. Nothing here needs code changes; the endpoints already exist.

What to watch

Monitor	URL	Expect	Interval	Why
Backend	`https://api.freebeeads.com/healthz`	`200` body `{"ok":true}`	1 min	Railway container up, app booted (config guard passed, DB migrated)
Web + backend	`https://freebeeads.com/api/health`	`200` body `{"ok":true,"web":true,"backend":true,…}`	1 min	Vercel serving and the web can reach the backend from the server side (the same path `/player` and `/w` use)

/api/health answers 503 {"ok":false,"backend":false,"detail":…} when the backend is down or NEXT_PUBLIC_API_URL / PLAYER_BACKEND_URL is wrong, so one monitor on it catches both halves; the direct /healthz monitor tells you which half when it fires. Keep both.

Optional third monitor: https://freebeeads.com/ (keyword check: Freebee) — the marketing page itself, which is static and only fails on a broken deploy.

Setting them up (UptimeRobot, free tier — 50 monitors, 5-min interval; or Better Stack, free — 10 monitors, 3-min)

Create an account with the team email (not a personal one).
Add monitor → HTTP(s) → paste the URL → interval as above → under Advanced, keyword match "ok":true so a 200 with "ok":false still alerts.
Alert contacts: email to the founder + WhatsApp/SMS (UptimeRobot) or the phone app (Better Stack).
Alert on: down for 2 consecutive checks (avoids one-off Railway cold starts); recover notice on.
Add a public status page later if advertisers ask for one — both tools generate one.

Alerts worth having beyond "down"

Response time > 3 s on /healthz for 5 min — Railway is starving (memory/CPU) before it dies.
SSL expiry < 14 days (both tools do this for free) — Vercel/Railway renew automatically, but a custom domain with a bad CNAME will not.
Sentry: new issue in production (see below) — email/Slack on first seen only, not every event.

Error tracking (Sentry) — on only when the DSN is set

Backend (backend/app/core/observability.py): set SENTRY_DSN on the Railway service. Environment tag comes from ENVIRONMENT, release from RAILWAY_GIT_COMMIT_SHA (Railway injects it). Nothing is sent by default except the exception and route; send_default_pii=False, and a before_send scrubber redacts anything shaped like an Indian mobile number or a 6-digit OTP from every string in the event (CLAUDE.md rule 3a). SENTRY_TRACES_SAMPLE_RATE stays 0 unless debugging latency.

Web (web/instrumentation-client.ts, web/instrumentation.ts): set NEXT_PUBLIC_SENTRY_DSN on Vercel (Production + Preview). No session replay, no user context; the same scrubber runs in web/lib/sentry-scrub.ts. Release = the Vercel commit SHA (NEXT_PUBLIC_BUILD_ID). Source maps are not uploaded (no SENTRY_AUTH_TOKEN) — stack traces are minified but grouped correctly; add the token later if traces get hard to read.

Suggested Sentry alert rules (one project per side, free tier is enough pre-launch):

Issue alert: a new issue is first seen in environment:production → email + Slack.
Issue alert: an issue is seen > 20 times in 1 hour → email (regression/loop).
Ignore environment:development and preview in rules; keep events for debugging.

Pre-launch checklist

Two monitors live, alert contacts verified (send a test alert).
SENTRY_DSN on Railway, NEXT_PUBLIC_SENTRY_DSN on Vercel, one deliberate test error each (/api/health with the backend URL mis-set is an easy way to trigger a web one).
Phone-number scrubbing verified once in the Sentry UI (search the event for [phone]).