How to Set Up Framework and CI with TEA

How to Set Up Framework and CI with TEA

Use TEA’s framework skill to scaffold test directories, fixtures, configuration, and runner commands, and to configure CI when requested.

bmad-tea-framework owns framework and CI setup. Your prompt selects framework only, CI only, or both. TEA infers the scope from your request. When CI scope is unclear in an interactive session, it asks once: “Do you want CI too?” An unattended request with unclear scope runs framework setup only and states that CI was excluded. Explicit framework-only requests skip the CI phase. For both, TEA agrees the stack, framework, and test commands first, then can generate the scaffold and pipeline in parallel and validate them together. If a working framework already exists, TEA reuses it and configures CI for its actual commands.

Setup scope is separate from the operation: Create starts a new run, Resume picks up an interrupted run where it stopped, Validate reports checks without repairing outputs, and Edit revises the selected outputs and checks those changes.

Choose framework setup, CI setup, or both. Existing CI customizations and progress remain usable. Evaluation plans integrate scored checks into the pipeline.

Framework Setup

When to Use This

• No existing test framework in your project
• Current setup needs shared fixtures or runner configuration
• Starting a new project that needs testing infrastructure
• Phase 3 (Solutioning) after architecture is complete

Prerequisites

• Architecture completed (or at least tech stack decided)

Steps

1. Run Framework Setup

• Claude Code / Cursor / Windsurf: /bmad-tea-framework
• Codex: $bmad-tea-framework
• Inside a /bmad-tea chat: TF

For unattended runs, use tea-framework: tea-framework --agent codex --scope both.

Full invocation rules: Invoking a TEA Skill.

For framework only, add “Set up the test framework only.” For both phases, add “Set up the test framework and CI.” The same skill runs both phases using an agreed stack, framework, and test-command contract. TF starts Framework. Ask for CI setup to select its CI phase.

2. Answer TEA’s Questions

TEA will ask about:

• Your tech stack (React, Node, etc.)
• Preferred test framework:
  • Frontend/Fullstack: Playwright, Cypress
  • Backend (Node.js): Jest, Vitest, or Playwright (API testing via playwright-utils)
  • Backend (Python): pytest, or Playwright for Python
  • Backend (Java/Kotlin): JUnit, or Playwright for Java
  • Backend (Go): Go test
  • Backend (C#/.NET): dotnet test / xUnit, or Playwright for .NET
  • Backend (Ruby): RSpec
• Testing scope (E2E, integration, unit, API)
• CI/CD platform (GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Harness, etc.)

3. Review Generated Output

TEA generates:

• Test scaffold: Directory structure and config files (language-idiomatic)
• Sample specs: Example tests following best practices for your framework
• .env.example: Environment variable template
• Version file: .nvmrc (Node.js), .python-version (Python), global.json (.NET), etc.
• README updates: Testing documentation

What You Get

Frontend/Fullstack (Node.js):

tests/
├── e2e/
│   ├── example.spec.ts
│   └── fixtures/
├── integration/
├── unit/
├── playwright.config.ts  # or cypress.config.ts
└── README.md

Backend (Python example):

tests/
├── unit/
│   └── test_example.py
├── integration/
├── api/
├── conftest.py
└── README.md

Note: Playwright has official bindings for Python, Java, and .NET, so it is viable for API testing across those languages too.

Playwright Utils Integration (on by default)

This integration applies to JavaScript and TypeScript suites using the Playwright runner. TEA uses the selected framework’s conventions for other stacks.

tea_use_playwright_utils defaults to true, so unless you turned it off in bmad setup tea this skill asks to install @seontechnologies/playwright-utils and then scaffolds against it:

npm install -D @seontechnologies/playwright-utils

What gets created on the enabled branch:

• {test_dir}/support/merged-fixtures.ts: the single entry point every spec imports test from, composed with mergeTests
• {test_dir}/support/auth-fixture.ts: setAuthProvider and createAuthFixtures(). If the auth endpoint is unknown, getToken contains a TODO listed in the summary.
• global-setup.ts wiring for authStorageInit() and configureAuthSession(), with the token storage directory gitignored
• Sample tests written in the same style, since every later skill reads them as the reference

If you decline the install, TEA scaffolds plain Playwright fixtures.

Utilities available: api-request, network-recorder, auth-session, intercept-network-call, recurse, log, file-utils, burn-in, network-error-monitor

Set tea_use_playwright_utils = "false" under [modules.tea] for plain Playwright fixtures.

Write-Time Quality Checks

The framework phase’s Create operation installs the hook on Claude Code; Resume installs it when continuing through that setup step. TEA copies .claude/hooks/tea-enforce.cjs and registers pre-write, post-write, and stop hooks in the project’s .claude/settings.json, preserving existing settings. The hooks run in ordinary coding sessions in that project. Rules apply to the test and Pact config paths in .tea/enforce-config.json.

Seven rules block supported violations:

• Focused tests such as .only
• Tautological assertions such as expect(value).toBe(value)
• Hard waits such as waitForTimeout
• Test files above the configured line limit, which defaults to 1,000
• Maestro flows with no assertion or destination-state wait that can fail
• Pact config without fileParallelism: false
• Pact settings that defeat serialization, such as concurrent execution, multiple workers, or disabled isolation

Undocumented disabled tests produce a warning. Post-write and stop checks catch violations that bypassed the pre-write hook. If the hook itself errors, it fails open and allows the write.

The skill skips hook installation on platforms without this interception point and records that in its summary. Run test-review to audit test quality on those platforms.

Optional: MCP Enhancements

TEA can use Playwright MCP servers for enhanced capabilities:

• playwright: Browser automation
• playwright-test: Test runner with failure analysis

Configure in your IDE’s MCP settings.

Tips

• Run only once per repository: Framework setup is a one-time operation
• Run after architecture is complete: Framework aligns with tech stack
• Include CI in the same run: Ask for framework and CI setup together, or request CI scope from Framework later

Next Steps

System-level Test Design should precede framework setup so planned NFR evidence shapes the infrastructure.

After test framework setup:

1. Epic Test Design: Create the test plan for the next epic
2. CI Configuration (skip if you included CI): Set up automated test runs
3. Story Implementation: Tests are ready for development

CI Setup

Use Framework to configure test jobs, sharding, burn-in runs, quality gates, and result artifacts. Ask for CI setup to select this phase.

When to Use This

• Need to automate test execution in CI/CD
• Want selective testing (only run affected tests)
• Need parallel execution for faster feedback
• Want burn-in loops for flakiness detection
• Setting up new CI/CD pipeline
• Optimizing existing CI/CD workflow

Prerequisites

• A working test framework, or permission to create one during setup
• Tests written, or sample tests from framework setup
• CI/CD platform access (GitHub Actions, GitLab CI, etc.)

Steps

1. Run CI Setup

• Claude Code / Cursor / Windsurf: /bmad-tea-framework
• Codex: $bmad-tea-framework
• Inside a /bmad-tea chat: TF

Request CI scope explicitly:

Set up CI only for the existing test framework.

Full invocation rules: Invoking a TEA Skill.

If no framework exists, TEA offers to include framework setup. Declining stops the run without changes. On acceptance, it agrees the stack, framework, and test commands before generating both outputs; final validation checks them together. Choose Create, Resume, Validate, or Edit for the selected setup scope. Existing CI customizations and saved checkpoints remain usable.

Create installs the dependencies needed to run the existing test commands and starts their required local services during preflight. It uses your project’s scripts and configuration; common commands include npm test, pytest, mvn test, go test ./..., dotnet test, and bundle exec rspec. Validate records execution results and missing prerequisites in its report. Edit checks the pipeline changes you requested, such as syntax, injection safety, and evaluation-plan rendering.

2. Select CI/CD Platform

TEA will ask which platform you’re using.

Supported Platforms:

• GitHub Actions (most common)
• GitLab CI
• Jenkins: Generates Jenkinsfile with parallel stages, artifact archiving, and post-failure handling
• Azure DevOps: Generates azure-pipelines.yml with matrix strategy for sharding and Azure-specific caching
• Harness: Generates .harness/pipeline.yaml with Kubernetes-based execution and parallel steps
• Circle CI
• Other (TEA provides generic template)

Example:

GitHub Actions

3. Configure Test Strategy

TEA will ask about your test execution strategy.

Repository Structure

Question: “What’s your repository structure?”

Options:

• Single app: One application in root
• Monorepo: Multiple apps/packages
• Monorepo with affected detection: Only test changed packages

Example:

Monorepo with multiple apps
Need selective testing for changed packages only

Parallel Execution

Question: “Want to shard tests for parallel execution?”

Options:

• No sharding: Run tests sequentially
• Sharding: Split the suite across CI jobs. Runtime depends on the slowest shard and job setup.
• Shard by file: Each file runs in parallel

Example:

Yes, shard across 4 workers for faster execution

Burn-In Loops

Question: “Want burn-in loops for flakiness detection?”

Options:

• No burn-in: Run tests once
• PR burn-in: Run tests multiple times on PRs; catches flaky tests before they merge
• Nightly burn-in: Dedicated flakiness detection job

Example:

Yes, run tests 5 times on PRs to catch flaky tests early

4. Review Generated CI Configuration

TEA generates the pipeline file from the command contract agreed during preflight. The samples below assume the defaults the framework phase writes for a Node.js Playwright suite; replace each with your project’s recorded values:

• Test command: npm run test:e2e, the script the framework phase adds to package.json. With the config under tests/, as in the scaffold above, the script runs playwright test --config tests/playwright.config.ts. A bare npx playwright test from the repository root does not find that config.
• Dependencies: npm ci from a committed package-lock.json, then npx playwright install --with-deps chromium for browser suites.
• Application: the suite reads BASE_URL. Point it at a deployed environment through a CI variable, or start the app in the job before the test step (Playwright’s webServer config option can start it and wait for it).
• Secrets: none for the sample suite; add yours in step 5.
• Node: the .nvmrc the framework phase writes.

The generated file also runs your lint command before the tests and a report job after them; the samples leave both out. The samples run these jobs per trigger:

Trigger	Jobs
pull_request to main or develop	test (4 shards), burn-in
push to main or develop	test (4 shards)
schedule (nightly)	test (4 shards), burn-in

Burn-in follows the generated template’s default, pull requests and the nightly schedule. Answering “Nightly burn-in” in step 3 limits it to schedule; “No burn-in” drops the job.

GitHub Actions (.github/workflows/test.yml):

name: Test Suite

on:
  pull_request:
    branches: [main, develop]
  push:
    branches: [main, develop]
  schedule:
    - cron: '0 2 * * *' # Nightly at 2 AM UTC

jobs:
  test:
    name: Test (Shard ${{ matrix.shard }})
    runs-on: ubuntu-latest
    timeout-minutes: 30
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - name: Run tests
        run: npm run test:e2e -- --shard=${{ matrix.shard }}/4
      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-${{ matrix.shard }}
          path: |
            test-results/
            playwright-report/
          retention-days: 7

  burn-in:
    name: Burn-In (Flakiness Detection)
    needs: test
    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - name: Run burn-in loop
        run: |
          for i in 1 2 3 4 5; do
            echo "=== Burn-in iteration $i/5 ==="
            npm run test:e2e -- --retries=0 || exit 1
          done
      - name: Upload burn-in results
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: burn-in-failures
          path: test-results/

GitLab CI (.gitlab-ci.yml):

The GitLab sample runs the same jobs on merge request, branch push, and scheduled pipelines.

variables:
  NODE_VERSION: '22.20.0'

stages:
  - test
  - burn-in

test:
  stage: test
  image: node:$NODE_VERSION
  parallel: 4
  script:
    - npm ci
    - npx playwright install --with-deps chromium
    - npm run test:e2e -- --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: always
    paths:
      - test-results/
      - playwright-report/
    expire_in: 7 days
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"
    - if: $CI_PIPELINE_SOURCE == "push" && ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop")

burn-in:
  stage: burn-in
  image: node:$NODE_VERSION
  script:
    - npm ci
    - npx playwright install --with-deps chromium
    - |
      for i in 1 2 3 4 5; do
        echo "=== Burn-in iteration $i/5 ==="
        npm run test:e2e -- --retries=0 || exit 1
      done
  artifacts:
    when: on_failure
    paths:
      - test-results/
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"

Optional: Selective Tests on Pull Requests

Selective testing runs only the tests a change affects. It needs a test:changed script that you define and check first, for example with Playwright’s --only-changed:

{
  "scripts": {
    "test:changed": "playwright test --config tests/playwright.config.ts --only-changed=origin/main --pass-with-no-tests"
  }
}

The CI phase can also write a git diff based selection script when your project needs one. On a branch that edits one spec, run npm run test:changed and confirm it selects that spec before you wire it in. Then add a PR job beside the full test job as fast feedback, and keep the test job on pull requests. --only-changed follows imports from the test files, so a source change that no spec imports selects no test, and only the full suite covers it:

jobs:
  selective:
    name: Selective Tests
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # The diff needs the base branch history
      - uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run test:changed

A change that touches no test or tested source, such as a docs-only edit, selects no tests; --pass-with-no-tests lets the selective job pass then, and the full test job still runs.

Burn-In Testing

Option 1: Classic Burn-In (Playwright Built-In)

{
  "scripts": {
    "test:e2e": "playwright test --config tests/playwright.config.ts",
    "test:burn-in": "playwright test --config tests/playwright.config.ts --repeat-each=5 --retries=0"
  }
}

How it works:

• Runs every test 5 times
• Fails if any iteration fails
• Detects flakiness before merge

Use when: Small test suite, want to run everything multiple times

---

Option 2: Smart Burn-In (Playwright Utils)

If tea_use_playwright_utils: true:

scripts/burn-in-changed.ts:

import { runBurnIn } from '@seontechnologies/playwright-utils/burn-in';

await runBurnIn({
  configPath: 'playwright.burn-in.config.ts',
  baseBranch: 'main',
});

playwright.burn-in.config.ts:

import type { BurnInConfig } from '@seontechnologies/playwright-utils/burn-in';

const config: BurnInConfig = {
  skipBurnInPatterns: ['**/config/**', '**/*.md', '**/*types*'],
  burnInTestPercentage: 0.3,
  burnIn: { repeatEach: 5, retries: 0 },
};

export default config;

package.json:

{
  "scripts": {
    "test:burn-in": "tsx scripts/burn-in-changed.ts"
  }
}

How it works:

• Git diff analysis (only affected tests)
• Smart filtering (skip configs, docs, types)
• Volume control (run 30% of affected tests)
• Each test runs 5 times

Use when: Large test suite, want intelligent selection

---

Comparison:

Feature	Classic Burn-In	Smart Burn-In (PW-Utils)
Changed 1 file	Runs all 500 tests × 5 = 2500 runs	Runs 3 affected tests × 5 = 15 runs
Config change	Runs all tests	Skips (no tests affected)
Type change	Runs all tests	Skips (no runtime impact)
Setup	Zero config	Requires config file

Use classic burn-in for the full suite. Use smart selection when you have checked that its dependency analysis covers your project.

5. Configure Secrets

TEA provides a secrets checklist.

Required Secrets (add to CI/CD platform):

## GitHub Actions Secrets

Repository Settings → Secrets and variables → Actions

### Required

- None (tests run without external auth)

### Optional

- `TEST_USER_EMAIL`: Test user credentials
- `TEST_USER_PASSWORD`: Test user password
- `API_BASE_URL`: API endpoint for tests
- `DATABASE_URL`: Test database (if needed)

How to Add Secrets:

GitHub Actions:

1. Go to repo Settings → Secrets → Actions
2. Click “New repository secret”
3. Add name and value
4. Use in workflow: ${{ secrets.TEST_USER_EMAIL }}

GitLab CI:

1. Go to Project Settings → CI/CD → Variables
2. Add variable name and value
3. Use in workflow: $TEST_USER_EMAIL

6. Test the CI Pipeline

Push and Verify

Commit the workflow file:

git add .github/workflows/test.yml
git commit -m "ci: add automated test pipeline"
git push

Watch the CI run:

• GitHub Actions: Go to Actions tab
• GitLab CI: Go to CI/CD → Pipelines
• Circle CI: Go to Pipelines

Expected result: the push runs the four test shards, and each must pass. Burn-in runs on pull requests and the nightly schedule; the trigger table lists the jobs per trigger.

Test on Pull Request

Create test PR:

Change a spec, so a selective job has a test to select:

git checkout -b test-ci-setup
echo "// CI check" >> tests/e2e/example.spec.ts
git add tests/e2e/example.spec.ts
git commit -m "test: verify CI setup"
git push -u origin test-ci-setup

Open PR and verify:

• The four test shards run and pass, or the selective job does if you added it
• Burn-in runs after the shards pass, unless you chose nightly or no burn-in
• Close the PR without merging the change

What You Get

Automated Test Execution

• On every PR: Catch issues before merge
• On every push to main: Protect production
• Nightly: regression testing

Parallel Execution

• Split the suite across multiple CI jobs

Selective Testing

• Optional: run only the tests a PR affects, once a test:changed script is verified

Flakiness Detection

• Burn-in loops: Run tests multiple times
• Early detection: Catch flaky tests in PRs when you choose PR burn-in

Artifact Collection

• Test results: Saved for 7 days
• Screenshots: On test failures
• Videos: Full test recordings
• Traces: Playwright trace files for debugging

Tips

Start Simple, Add Complexity

1. Week 1: run the full suite on every PR in one job.
2. Week 2: shard across 4 jobs.
3. Week 3: add selective tests on PRs once test:changed is verified.
4. Week 4: add nightly burn-in, then PR burn-in.

Optimize for Feedback Speed

Goal: PR feedback in < 5 minutes

Strategies:

• Balance shards so one slow job does not hold up the suite
• Run affected tests on pull requests and the full suite on the agreed schedule
• Cache dependencies (actions/cache, cache: 'npm')
• Run smoke tests first, full suite after

Example fast workflow, using the @smoke tag from Use Test Tags:

name: Fast Test Feedback

on:
  pull_request:
  push:
    branches: [main, develop]

jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run test:e2e -- --grep @smoke

  full:
    needs: smoke
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version-file: '.nvmrc'
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run test:e2e

Use Test Tags

Tag tests for selective execution:

// Critical path tests (always run)
test('@critical should login', async ({ page }) => {});

// Smoke tests (run first)
test('@smoke should load homepage', async ({ page }) => {});

// Slow tests (run nightly only)
test('@slow should process large file', async ({ page }) => {});

// Skip in CI
test('@local-only should use local service', async ({ page }) => {});

In CI:

# PR: Run critical and smoke only
npm run test:e2e -- --grep "@critical|@smoke"

# Nightly: Run everything except local-only
npm run test:e2e -- --grep-invert "@local-only"

Monitor CI Performance

Track PR feedback time, full-suite time, flakiness rate, and CI cost against targets your team sets, for example PR feedback under 5 minutes and flakiness under 1%.

Handle Flaky Tests

When burn-in detects flakiness:

1. Quarantine flaky test:

   test.skip('flaky test: investigating', async ({ page }) => {
     // TODO: Fix flakiness
   });

2. Investigate with trace viewer:

   npx playwright show-trace test-results/trace.zip

3. Fix root cause:

   • Add network-first patterns
   • Remove hard waits
   • Fix race conditions

4. Verify fix:

   npm run test:e2e -- tests/e2e/flaky.spec.ts --repeat-each=20 --retries=0

Secure Secrets

Don’t commit secrets to code:

# ❌ Bad
- run: API_KEY=sk-1234... npm run test:e2e

# ✅ Good
- run: npm run test:e2e
  env:
    API_KEY: ${{ secrets.API_KEY }}

Use environment-specific secrets:

• STAGING_API_URL
• PROD_API_URL
• TEST_API_URL

Cache Aggressively

Speed up CI with caching:

# Cache npm downloads
- uses: actions/setup-node@v4
  with:
    cache: 'npm'

# Cache Playwright browsers
- name: Cache Playwright browsers
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ hashFiles('package-lock.json') }}

Common Issues

Tests Pass Locally, Fail in CI

Symptoms:

• Green locally, red in CI
• “Works on my machine”

Common Causes:

• Different Node version
• Different browser version
• Missing environment variables
• Timezone differences
• Race conditions (CI slower)

# Pin Node version
- uses: actions/setup-node@v4
  with:
    node-version-file: '.nvmrc'

# Pin the browser by pinning @playwright/test in package-lock.json. `playwright install`
# takes no version suffix: `chromium@1.40.0` fails with "Invalid installation targets".
- run: npx playwright install --with-deps chromium

# Set the test process timezone.
- run: npm run test:e2e
  env:
    TZ: 'America/New_York'

CI Takes Too Long

CI takes 30+ minutes, developers wait too long.

1. Shard tests: distribute the suite across CI jobs
2. Selective testing: Only run affected tests on PR
3. Smoke tests first: Run critical path (2 min), full suite after
4. Cache dependencies: npm ci with cache
5. Optimize tests: Remove slow tests, hard waits

Burn-In Always Fails

Find the failing tests in the burn-in report and check whether failures repeat consistently or intermittently:

1. Identify flaky tests (check which iteration fails)
2. Fix flaky tests using test-review
3. Re-run the classic burn-in script on specific files:

npm run test:burn-in -- tests/e2e/flaky.spec.ts

Out of CI Minutes

Using too many CI minutes, hitting plan limit.

1. Run full suite only on main branch
2. Use selective testing on PRs
3. Run expensive tests nightly only
4. Self-host runners (for GitHub Actions)

Related Guides

• How to Set Up Test Framework: Include the framework phase when needed
• How to Run Test Review: Audit CI tests
• Integrate Playwright Utils: Burn-in utility

Understanding the Concepts

• Test Quality Standards: Why determinism matters
• Network-First Patterns: Avoid CI flakiness

Reference

• Command: ci: Full command reference
• TEA Configuration: CI-related config options

Evaluation Plans

If the repository holds an evaluation written with bmad-tea-evaluate, Framework’s CI phase finds its ci/evaluation-ci-plan.json and renders it into the same pipeline file. A standalone run picks up existing plans after the quality gates step, and an edit-mode run detects them first and renders them into the pipeline it loaded. Evaluate writes the plan and Framework writes every pipeline file. The last stage of bmad-tea-evaluate invokes Framework in edit mode on the pipeline file once the plan is written, or in create mode when the repository has no pipeline file.

For each tier the plan places a check on, the pipeline gets one job, evaluation-pr for the pr tier:

• One step per tier. tea-evaluate ci --tier <tier> is the runtime’s one entry that runs a tier. It runs every check the plan places on the tier and writes the evidence bundle, and it takes no check selector, so the job runs it once as npm exec --prefix <evaluations folder> -- tea-evaluate ci --evaluation <evaluation folder> --tier <tier>, named for the ids of the checks it carries. A merge job runs the pr tier’s step first and its own second.
• The tooling stays in the evaluations folder. The job installs the private package.json Evaluate wrote there with npm ci --prefix (npm install --prefix when no lockfile is committed), so the repository’s own manifest stays untouched and a repository in any language works.
• The Node version respects the tooling’s floor. The evaluation job runs the project’s .nvmrc version only when it is at or above the Node floor TeA and eval-quality declare (22.20.0), and the current LTS otherwise, so a project pinned to an older Node is not blocked.
• Checks and tiers come from the plan. Framework preserves the complete check set and every check’s tier placement, without adding, dropping, or moving checks. The evaluation plan stays unchanged.
• A job the plan gates waits for its tier’s evaluation job. See gated publish and deploy jobs.
• Evidence is kept whatever the result. The job uploads <evaluation folder>/runs/ with if: always() under the artifact name <job id>-runs, so no two jobs upload one name, and each invocation writes runs/<invocationId>/ under that folder.
• Triggers follow the plan. The workflow gains the events the checks name, and each evaluation job gets an if: on the event that starts its tier when the workflow carries more events than the tier names. When an event is new to the workflow, or an event it already has gains a branch filter, a tag pattern or a cron, the jobs that existed before get an if: that keeps them on the events and filters they already ran on, and the summary names each guard. A job the plan gates inside the pipeline file is exempt, since its wait keeps it off the added runs. workflow_dispatch goes to every tier whose checks name manual-dispatch.
• Timeouts. The pr and merge jobs get 30 minutes and the scheduled and release jobs 120, since their live checks spend model calls.
• The plan is validated first. Framework runs tea-evaluate check and refuses a plan only for findings about ci/evaluation-ci-plan.json.
• Credentials stay yours. The plan carries none, so the summary lists what the live tiers need.

The jobs carry a # tea-evaluation-plan: marker with the plan’s path, so a later run, in create or edit mode, rewrites each of them under the id the current plan gives, renames one whose id differs and leaves every job without the marker as it was, apart from the event guards and gate waits described above. Edit mode never writes the create run’s checkpoint. See tea-evaluate CLI for the plan and its tiers.

Gated Publish and Deploy Jobs

A check’s optional gates names existing publish or deploy jobs by job id, so a blocking exit stops the shipment. A gated job in the pipeline file gets the evaluation job appended to its needs, and Framework changes only that job’s needs; its if: and every other key stay unchanged. A gated job in another workflow file gets a workflow_run trigger on the pipeline, an if: that requires a successful run of the pipeline file on the tier’s event, and the evaluated commit as the ref of its checkout steps, and that workflow’s other jobs get an event guard. Framework reports each edit in the summary and restores the wait when it rewrites the evaluation jobs. Across files, Framework limits changes to the gated job’s if: and checkout ref:, its workflow’s workflow_run trigger, and event guards on that workflow’s other jobs. All other job settings stay unchanged. A plan renders nothing and is reported when a name matches no job or a gate cannot hold. Inside the pipeline file a gate cannot hold when the gated job already ran in a run where its tier’s evaluation job is skipped (a deploy on every push beside an evaluation job guarded to tags), when the job already waits for another plan’s evaluation job through the cross-file form, when the tier’s evaluation job runs on a run the job did not run on before (a workflow_dispatch the plan adds to a tag-push release file; a deploy file that already starts on a dispatch gains no run), or when its if: calls a status function other than success() (always(), !cancelled(), failure()). A run the render adds, such as a merge tier’s branch filter or a new cron, is skipped through the wait and is no conflict. Across files a gate also cannot hold for a pull request tier, a pull_request_target tier or another event that runs code of a fork, for an evaluation job behind a ref or cron guard, for a gated workflow that already follows another workflow, for a job gated by another plan or by a second tier, and for a job with needs, a uses: job, a job with an if: of its own, or a job that reads github.ref, github.ref_name, github.sha, github.head_ref or github.event.*.