f1 intelligence
./docs / testing

Testing

Five suites, one per runtime boundary. Each app owns its own runner and config: there is no top-level test command.

SuiteLocationRunnerTouches a real DB?
Data enginedata-engine/tests/pytestNo: scripted fake DB
API unitapps/api/tests/unit/bun testNo
API integrationapps/api/tests/integration/bun testYes: local Postgres via docker-compose.test.yml (never Neon)
Web unitapps/web/tests/unit/VitestNo: MSW mocks the API
Web e2eapps/e2e/tests/smoke/PlaywrightNo: fixture API server

Test files are never co-located with source. They live under the app’s tests/ directory (or apps/e2e/tests/), mirroring the path of the file under test: e.g. tests/unit/common/mappers.test.ts covers src/common/mappers.ts.

Running everything

# data-engine
cd data-engine && pytest -v

# api  (integration suite needs the local test DB: docker-compose.test.yml)
cd apps/api && bun run test:db:up    # start Postgres + Neon HTTP proxy, apply migrations
cd apps/api && bun test              # unit + integration
cd apps/api && bun run test:unit     # unit only
cd apps/api && bun run test:integration

# web
cd apps/web && bun run test          # vitest run

# e2e  (spins up its own servers; E2E_WEB_PORT=4331 if your own `astro dev` holds :4321)
cd apps/e2e && bun run test

Data engine (data-engine/tests/)

pytest, config in pyproject.toml (pythonpath = ["."], testpaths = ["tests"]). Dev deps: requirements-dev.txt (= requirements.txt + pytest).

  • Pure functions are tested directly: math_utils.py (normalize_minmax, bayesian_win_rate, clamp), finishing_order.py, the model weight invariants (test_weights.py: the committed src/models/gp_v4.json scores exactly the computed GP features and its importances match docs/feature-weights.json), weighted-v4’s pieces (test_conditional_logit.py: the fit recovers known weights; test_gp_features_v4.py, over the in-memory history in tests/support/race_history.py; test_gp_model_v4.py: save/load, importances, training set; test_identity.py, against docs/identity-cases.json), the backtest scoring (test_backtest.py: metrics, calibration, GridWinRates), rank_by_probability, race_weekend_window, weekend_pipeline (test_weekend_pipeline.py: plan, the chains, due_statuses, repair_plan, prediction_steps), enrichment_coverage (test_enrichment_coverage.py: missing, compound normalisation from either source; the coverage query through the fake DB), and the FastF1 adapter (test_fastf1_session.py: stint, pit-stop and race-control rows over plain records, telemetry input reshaping).
  • The live ingest adapter runs against recorded responses: test_openf1_session.py feeds openf1_session (OpenF1 + Jolpica → ingest rows) the Bahrain 2026 responses in tests/fixtures/openf1/ and tests/fixtures/jolpica/, checked against the official results and what FastF1 ingested.
  • DB-touching functions are tested through a fake double: tests/support/fake_db.py provides FakeCursor / FakeConnection, shaped like psycopg2’s RealDictCursor. FakeConnection takes a list of result sets and hands one out per with conn.cursor() block, in call order. This covers compute_luck_score, same_person_rows, compute_team_circuit_perf, season_drivers, and ingest_runner.py’s two shared runners (run_ingest_job, run_qualifying_ingest_job): the latter via monkeypatched per-job callables and openf1_session readers, no real session data needed. The same double covers data_quality_audit._audit_race (per-status gate/threshold branching), data_quality_repair.run (issue grouping, repair plan per race, resolve bookkeeping), main.auto_detect_round, (with FastF1’s schedule / load_session mocked) sync_schedule and sync_season, compute_title_odds.run (incl. replaying an earlier round without seeing later results or predictions), auto_runner._retry_missing_race_events (only the missing coverage’s job reruns, a cleanly finished job is settled in the memo, budget-limited telemetry stays pending), and the CLI’s session-job dispatch (test_main_cli.py).
  • Anything that runs chain steps (auto_runner, data_quality_repair) is tested by swapping entries of the one job table, weekend_pipeline.JOBS (monkeypatch.setitem), never by patching job modules.
  • conftest.py sets a placeholder DATABASE_URL so importing a job module (which reads it at import time) works with or without a local .env. Tests never open a real connection.
  • upsert.py and the prediction jobs write via psycopg2.extras.execute_batch, which renders SQL through a C extension that needs a real connection even just to quote identifiers: so their tests (test_upsert.py, test_compute_sprint_features.py) monkeypatch execute_batch itself and assert on the params it receives, the same pattern test_ingest_runner.py already uses for its own execute_batch call, rather than extending FakeCursor to fake SQL rendering.
  • test_compute_sprint_features.py runs the sprint-v3 job against an in-memory RaceHistory with a sprint grid: the committed GP weights score sprint_features, rows are stored the GP’s way with the predicted winner in one transaction, sprints inside the committed fit are skipped and an unknown race fails.
  • test_compute_features.py covers the weighted-v4 GP job: each feature’s value and contribution land in the features JSONB, a plain softmax of raw_weighted_score is the model’s win probability, and run loads three seasons of history before the race and writes one row per driver.

API unit (apps/api/tests/unit/)

bun test, no DB, no network. Mirrors src/ by domain: the shared src/common/ layer (mappers, standings, cache, identity, feature-manifest) and each module’s pure *.helpers.ts (circuits/ track outline, track profile, circuit era/stats; drivers/ classification, qualifying gap; history/ helpers; predictions/ accuracy, prediction history, intel standings; race-events/openf1.mappers; quality). feature-manifest.test.ts checks the manifest’s labels against docs/feature-weights.json, the same file the Python and web drift tests read. Fixtures are plain objects typed as typeof <table>.$inferSelect; see mappers.test.ts for the pattern. identity.test.ts reads its cases from docs/identity-cases.json, the same file data-engine/tests/test_f1db.py checks normalize_person_name against, so the two languages can’t drift on the F1DB join key. The GP/sprint reads in common/session-reads.ts are DB-bound, so they are covered by integration tests instead (integration/session-kind/: the same assertions run once per kind).

Anything pure that a service delegates to (row→DTO mapping, aggregation, normalisation) belongs here. If a service method is hard to unit-test, that’s usually a sign the pure part should be extracted into common/ or a *.helpers.ts (this is what the max-lines budget nudges toward).

API integration (apps/api/tests/integration/)

bun test driving the real Hono app in-process via app.request() against a local Postgres (docker-compose.test.yml: postgres:16-alpine on :5433 behind local-neon-http-proxy on :4445). Neon Free’s 100 CU-hours/month are shared by every branch, and TDD runs this suite constantly, so the suite never touches Neon.

  • apps/api/.env.test (committed, docker-only credentials): bun test loads it over .env. It sets TEST_DATABASE_URL and NEON_FETCH_ENDPOINT, which src/config/localNeonProxy.ts uses to route the Neon HTTP driver to the proxy. bun run test:db:up / test:db:down start/stop it (tmpfs: data is thrown away).
  • Guards in tests/support/db/test-db.ts: refuses a *.neon.tech host (override TEST_DB_ALLOW_NEON=1), and refuses to truncate if TEST_DATABASE_URL equals DATABASE_URL (override TEST_DB_ALLOW_RESET=1).
  • tests/support/app/request.ts: apiRequest(path, init?) calls app.request() with a Bindings env pointing at the test DB.
  • tests/support/factories/: one insert helper per table, each taking Partial<$inferInsert> overrides and returning the inserted row. FK chain: season → team/circuit → driver → race → results/predictions.
  • Lifecycle: each test file calls truncateAll(db) in beforeAll and afterAll, seeds its own fixtures (typically under a sentinel year like 2097/2099), and asserts on the JSON response. truncateAll is a single TRUNCATE … RESTART IDENTITY CASCADE over every table.
  • Covered: races (incl. practice long runs, circuit outline + circuit telemetry), race-events, race-story, strategy, telemetry, drivers (incl. career, racecraft, teammate battle), teams (incl. pit stops), seasons, search, predictions (incl. title odds), sprint, history, the home hub’s on-this-day query, /api/health (never wakes the DB) and /api/health/db, the edgeCache middleware (common/edge-cache.ts, against tests/support/cache/fake-edge-cache.ts), the per-kind GP/sprint reads (session-kind/), and the route factories (route/route.test.ts: 400/404 answers and Cache-Control derived from the data’s race status or season, never from ?year=). Not yet: quality (dev-only).

Web unit (apps/web/tests/unit/)

Vitest, config in vitest.config.ts (include tests/unit/**/*.test.{ts,tsx}, @ alias, setup file).

  • Default environment is node: pure logic runs there: lib/f1/predictionMath.ts (all exports), lib/f1/predictionTerms.ts (served contributions as terms, edge over the field, What-If Lab), lib/f1/teamColors / lib/f1/teamLogos / features/history/f1dbCountries, features/compare/compareStats.ts, the compare stat rows (features/compare/{driver,team}CompareConfig.ts: Championship Points reads the API’s championshipPoints, never a client-side GP + sprint sum), features/history/historyView.ts, features/races/{raceView,calendar,loadRaceEvents} (race page and calendar view logic), components/data/{chartScale,labelLayout}, the hub/circuits/drivers/teams feature helpers, features/predictions/{buildPredictionPageData,predictionCopy} (API mocked with vi.mock, not MSW).
  • Page-data loaders are tested through MSW: features/drivers/loadDriverPageData, features/teams/loadTeamPageData, features/races/loadRacePageData (GP and sprint) and features/history/loadHistoryPageData. Each test stubs the API routes the page hits with apiOk/apiError (tests/support/msw/routes.ts), calls the loader with a URL and a pinned now, and asserts on the returned view model; recordApiRequests() captures which endpoints were requested, so “only the open tab is fetched” is asserted directly. Career-year resolution is covered for all three fallbacks (requested → current → latest season).
  • jsdom is opt-in per file via a // @vitest-environment jsdom docblock on line 1: used for hook/component tests (useCompareController, useGlobalSearch, GlobalSearch) with Testing Library.
  • tests/support/setup.ts: jest-dom matchers, a ResizeObserver stub (cmdk needs it), and the MSW server lifecycle (onUnhandledRequest: 'error', reset between tests).
  • tests/support/msw/: handlers.ts + fixtures.ts mock the src/lib/api.ts endpoints at http://localhost:8787. Client-side fetches in the components under test hit these. routes.ts adds per-test overrides (apiOk, apiError) and recordApiRequests(); setup.ts resets handlers and request listeners after each test.
  • Not covered: .astro component rendering.

Web e2e (apps/e2e/)

Playwright, its own package.json and playwright.config.ts (see docs/adr/0001-apps-layout-scoped-to-js-bun.md for why it isn’t nested under apps/web/).

  • webServer starts two processes: the fixture API server (fixtures/server.ts, Bun.serve on :4310, serves { data, error: null } + CORS headers for the routes under test) and astro dev for apps/web on :4321 with PUBLIC_API_URL pointed at the fixture server.
  • astro preview isn’t used (the @astrojs/cloudflare adapter doesn’t support it) so the SSR frontmatter runs under Vite’s dev server instead. Playwright can’t intercept the server-side fetches Astro makes in frontmatter, which is why a real fixture HTTP server is needed rather than request mocking.
  • The fixture server sends CORS headers because GlobalSearch / DriverCompareTool / TeamCompareTool fetch client-side: a genuine cross-origin browser request. Every other route is only ever hit server-side.
  • Every fixture in fixtures/data.ts is typed with apps/web/src/types, which re-exports the API’s common/types: bun run typecheck (tsc --noEmit, also a CI step) fails when a fixture drifts from the real response shape, e.g. a field the API added.
  • Smoke specs (tests/smoke/) cover the home hub, /prediction, /prediction/sprint/[id], /races/[id] (+ sprint, race story, telemetry), /circuits + /circuits/[key], /drivers + /drivers/[id], /teams + /teams/[id], both compare pages, /history (all four tabs, old-URL redirects), the sidebar, branding, and the global search palette: each asserts key content renders against the fixtures.
  • accessibility.spec.ts runs axe (@axe-core/playwright) on ten key pages and fails on any serious or critical violation, colour contrast included.
  • mobile.spec.ts loads the main pages at 375 px and fails if any of them scrolls sideways. It also checks that wide charts keep their labels legible in their own scroller and that the scorecard’s season column stays put.
  • reuseExistingServer is on outside CI, so a developer’s own astro dev on :4321 would be silently reused: against real data. Set E2E_WEB_PORT / E2E_API_PORT (e.g. 4329 / 4319) to run against fresh fixture-backed servers while your dev servers are up.

CI (.github/workflows/test.yml)

Four jobs, on every push to master and every PR:

JobSteps
data-enginepip install -r requirements-dev.txt → pytest -v
apibun install → test:db:up (docker compose + migrations) → bun run test → bun run typecheck → bun run lint
webbun install → bun run build → bun run test → bun run typecheck → bun run lint
e2ebun install (web + e2e) → bun run typecheck (fixtures against the API’s types) → playwright install chromium → bun run test

The api job runs the integration suite against the same docker-compose.test.yml stack: CI never spends Neon CU-hours, and no TEST_DATABASE_URL secret is needed any more.

Locally, a husky pre-commit hook runs lint-staged (lint-staged.config.mjs): staged files under apps/api/ or apps/web/ trigger that app’s bun run typecheck and bun run lint (including the max-lines budget). Tests are not part of the hook.

Adding a test

  • Pure helper (either app) → tests/unit/, mirror the source path. Prefer extracting the pure part over testing a DB-bound method.
  • New API endpoint / query → an integration file under tests/integration/<domain>/; add a factory if a table has none; truncateAll in beforeAll/afterAll.
  • New page logic (tabs, fallbacks, shaping) → put it in a page-data loader under features/<area>/ and test the loader with MSW (node environment), mirroring tests/unit/features/<area>/.
  • New interactive web island → a jsdom Vitest file; add MSW handlers for any endpoint it calls; consider an e2e smoke spec + a fixture route if it’s a whole page.
  • New ETL scoring function → pytest; pure if possible, otherwise script FakeConnection with one result set per query the function issues.