# Testing — local and CI

Last updated Oct 27, 2026 · Page: https://freebeeads.com/docs/repo/testing · Markdown: https://freebeeads.com/docs/md/repo/testing.md · All docs: https://freebeeads.com/llms-full.txt

Three layers, all green before anything merges to `main` (`.github/workflows/ci.yml` runs them
on every push and pull request):

| Layer | Where | What it proves |
|---|---|---|
| Backend unit + API | `backend/tests` (pytest, 358 tests) | contract shapes, ledger rules, money path (`test_e2e_money_path.py`), Sentry scrubber |
| Web typecheck + build | `web` (`tsc --noEmit`, `next build`) | the app compiles and every route renders at build time |
| Playwright journeys | `web/e2e` (`@playwright/test`, 19 specs / 84 tests, about 2 minutes locally) | the product works end-to-end in a browser against a real backend |

## Backend

```bash
cd backend
python -m venv .venv && .venv/bin/pip install -r requirements.txt -r requirements-dev.txt
.venv/bin/python -m pytest -q -p no:warnings
```

Tests pin `DATABASE_URL` / `MEDIA_STORE_DIR` to a temp dir in `tests/conftest.py`; nothing touches
`var/dev.db`.

## Web

```bash
cd web
npm ci
npx tsc --noEmit -p .
npm run build
```

## Playwright journeys

The suite lives in `web/e2e` and codes to the seeded dev data (`backend/scripts/seed.py`: four
screens at Inorbit, `demo@dooh.local` / `ops@dooh.local`). Every journey creates its own advertiser,
creative and booking through the API first, so tests are independent and safe to retry; only the
seeded screens and the ops account are shared.

| Spec | Journey |
|---|---|
| `01-signup` | fresh sign-up → onboarding → Home shows the four metric bands under "Explore performance breakdowns" |
| `02-creative-upload` | upload a generated PNG → appears in the grid, "Waiting for review" |
| `03-booking` | book the first seeded screen for tomorrow → "Booked" card |
| `04-approval` | ops sees the Approvals badge → Review opens the slide-over → tick the three checks (Approve stays off until then) → Approve → advertiser sees Approved |
| `05-wall-pairing` | `/player` shows a code → `/ops/pair` claims it → player loops → `/v1/metrics/live` lists the screen |
| `06-wallet` | shopper requests a dev-mode OTP, verifies with consent → My coupons (masked number) |
| `07-smoke` (`@smoke`) | every page for public (incl. legal, forgot/reset, docs) / advertiser (incl. billing, messages, team) / ops (incl. devices, API keys) answers 200 with no console errors; `/healthz` and `/api/health` answer |
| `08-billing` | ops Mark paid on Bookings → advertiser Billing shows the invoice Paid → the printable invoice renders |
| `09-messages` | approval lands two messages → the bell counts them → Mark all read clears it without a reload |
| `10-team` | owner invites a viewer (dev echo link) → sign-up with the invite joins the team → the viewer's booking is refused with the view-only message |
| `11-devices` | a paired player's health shows on Devices → Reload → the player acks at its next check-in → the card shows the result |
| `12-proof` | Download proof of play → `/v1/proof/verify` accepts the file and rejects an edited copy |
| `13-password-reset` | forgot → dev echo reset link → new password works, old one does not, the link is single use |
| `14-jobs-hardening` | content security policy (`@smoke`: nonce policy on pages, engine pages load with no violation, game editor / kiosk / player too) → ops Today "Background jobs" Run now → owner changes a teammate's role and removes them after the confirm line |
| `14-languages` | the phone page switches to Telugu and Hindi (html lang, `?lang=`, remembered across a reload); `/w?lang=hi` opens in Hindi and English is one tap away |
| `14-mall` | ops invite a mall contact → the contact signs in and lands on `/mall` → Overview bands, Screens, Statement → creates a partner API key that reads its screens; an advertiser opening `/mall` is sent to `/dashboard` and `/v1/mall/*` answers 403 |

### Run locally

Start a backend on a throw-away DB with dev OTP on, seed it, start the web against it, run:

```bash
# terminal 1 — backend
cd backend
DATABASE_URL=sqlite:////tmp/e2e/dev.db MEDIA_STORE_DIR=/tmp/e2e/media \
  .venv/bin/python scripts/seed.py && \
DATABASE_URL=sqlite:////tmp/e2e/dev.db MEDIA_STORE_DIR=/tmp/e2e/media WALLET_OTP_DEV=1 \
  EMAIL_DEV_ECHO=1 WEB_URL=http://localhost:3000 .venv/bin/python -m uvicorn app.main:app --port 8001

# terminal 2 — web
cd web && NEXT_PUBLIC_API_URL=http://localhost:8001 npx next dev -p 3000

# terminal 3 — the suite (first time: npx playwright install chromium)
cd web
E2E_BASE_URL=http://localhost:3000 E2E_API_URL=http://localhost:8001 npx playwright test
npx playwright show-report          # HTML report; traces are kept for failures
```

Any ports work — the suite only reads `E2E_BASE_URL` (default `http://localhost:3000`) and
`E2E_API_URL` (default `http://localhost:8001`). `06-wallet` needs `WALLET_OTP_DEV=1` on the backend; `10-team` and `13-password-reset` need
`EMAIL_DEV_ECHO=1` (with the default `EMAIL_PROVIDER=log`). Bookings land on a random future day, so
the suite can run again and again on the same database without selling a seeded screen out.

### Against production — read-only smoke only

```bash
cd web
E2E_BASE_URL=https://freebeeads.com \
E2E_API_URL=https://api.freebeeads.com \
npx playwright test --grep @smoke
```

The `@smoke` subset never writes. The role pages need credentials for that environment
(`E2E_ADVERTISER_EMAIL` / `E2E_ADVERTISER_PASSWORD`, `E2E_OPS_EMAIL` / `E2E_OPS_PASSWORD`); when a
login fails the role block is **skipped**, never failed. Do **not** run the `@journey` specs against
production: they create real advertisers, bookings, devices and wallet users.

### What CI does (`.github/workflows/ci.yml`)

Three jobs in parallel, pip and npm caches on:

1. `backend` — Python 3.12, `pip install -r requirements.txt -r requirements-dev.txt`, pytest.
2. `web` — Node 20, `npm ci`, `tsc --noEmit`, `next build`.
3. `e2e` — starts the backend on `:8001` (temp SQLite under `/tmp/e2e`, `WALLET_OTP_DEV=1`, `EMAIL_DEV_ECHO=1`,
   `scripts/seed.py`), builds and starts the web on `:3000` with `NEXT_PUBLIC_API_URL`, installs
   Chromium (cached), runs the suite with one retry, and on failure prints the server logs and
   uploads `playwright-report/` + `test-results/` (traces) as the `playwright-report` artifact.

No GitHub secrets are needed for CI. Typical wall time: backend ~3 min, web ~3 min, e2e ~6–8 min.

## Product screenshots (`web/public/shots`)

`npm run shots` (in `web/`, with the same two dev servers up and `scripts/seed_templates.py` run
so the game ads exist) regenerates the landing-page screenshots at 1440×900 with Playwright and
converts them to WebP with `cwebp` when it is installed. See `web/scripts/shots.mjs`.