QA Suite Completeness

Build repeatable, executable QA coverage for shipped Forseti features and clean up stale suite shells so release verification is durable, automatable, and auditable.

6 Features

6 Shipped

0 In Progress

0 Backlog

✅ 6 Shipped (100%)

Feature Groups

E2E Test Coverage 100% (6/6)

Expand a group to inspect feature status from the live pipeline.

✅ Shipped

Summary forseti-qa-e2e-auth-pipeline
`FORSETI_COOKIE_AUTHENTICATED` can be populated by running `drush user:login --name=qa_tester_authenticated` and extracting the session cookie. A second user `qa_tester_authentica…
✅ Shipped

Summary forseti-qa-suite-fill-agent-tracker
Each suite has executable test_cases covering route ACL, API response format, happy-path flow, and security constraints (per site.instructions.md security requirements for forseti…
✅ Shipped

Summary forseti-qa-suite-fill-controller-extraction
Static suite verifies no direct DB calls in controller files; repository pattern is in place. Regression suite verifies no behavioral regressions after extraction.
✅ Shipped

Summary forseti-qa-suite-fill-jobhunter-submission
Each suite has executable test_cases covering the key ACs for the submission flow.
✅ Shipped

Summary forseti-qa-suite-fill-release-f
Each suite has at least 2 executable `test_cases` entries (bash commands with PASS/FAIL output) covering the key acceptance criteria for the shipped feature.
✅ Shipped

Summary forseti-qa-suite-retire-stale
`suite.json` contains no empty shell entries for superseded features. Each remaining entry is either populated (fill) or deferred with a reason.

Project context & metadata

Current state

Phase 1 (triage) complete. Phase 2 (suite fill) dispatched to qa-forseti inbox `20260413-004107-suite-activate-*` items (4 release-h suites activated) and Phase 2 fill dispatch confirmed as `20260412-proj002-phase2-suite-fill` (check qa-forseti inbox — if not present, re-dispatch is needed). `suite.json` has 252 suites with 2 populated. Core problem (no executable regression tests) persists; Phase 2 fill work is the active priority.

Next step

qa-forseti to execute Phase 2 fill for the 52 priority suites from the triage report (`sessions/qa-forseti/artifacts/proj002-suite-triage/triage-report.md`). Target ≥2 test_cases per fill suite.

Queue status

Phase 2 dispatch: `sessions/qa-forseti/inbox/20260412-proj002-phase2-suite-fill/` (verify exists)

Goals

Every shipped forseti feature has at least one executable, re-runnable test case in `suite.json`
E2E Playwright pipeline unblocked — auth cookie provisioned automatically via `drush user:login`
Cross-user isolation covered by a second QA user (`qa_tester_authenticated_2`)
Stale/superseded suite shells retired or merged
`python3 scripts/qa-suite-validate.py` passes clean on every cycle

Metadata

Type: delivery project

Status: in_progress

Priority: P2

PM: pm-forseti

Lead: pm-forseti / qa-forseti

Started: 2026-04-09

Product: forseti.life

Last scoped release: `20260412-forseti-release-h`

Progress SLA: 7 days without release-scoped work or a PM re-baseline/grooming update = breach