Goal or problem

The Agentic Workflows (gh-aw) tooling vendored into this repository is pinned to gh-aw compiler/runtime v0.72.1, while the latest published gh-aw release is v0.77.5 (https://github.com/github/gh-aw/releases/tag/v0.77.5). All six compiled smoke *.lock.yml workflows, the generated agentics-maintenance.yml maintenance workflow, and the hand-maintained detection-only.yml iteration workflow embed v0.72.1 action pins and a v0.72.1-era firewall/MCP image set. The repository has drifted five minor releases behind upstream, spanning two breaking changes that affect the Copilot smoke workflow.

This issue asks an implementer to upgrade the pinned gh-aw version to v0.77.5, add the copilot-requests: write permission required by the v0.77.2 breaking change, recompile the smoke source .md workflows to fresh *.lock.yml files, regenerate the *-container.lock.yml containerized siblings with scripts/create-threat-detection-sibling-workflows.py, realign the hand-maintained detection-only.yml, and verify the upgraded workflows compile, pass freshness checks, and run.

Background or context

gh-aw (GitHub Agentic Workflows) compiles a human-authored *.md workflow (frontmatter + prompt) into a self-contained *.lock.yml GitHub Actions workflow. The compiled lock file carries a gh-aw-metadata header (compiler version, schema version, frontmatter/body hashes) and a gh-aw-manifest header (pinned action SHAs and container image digests). This repository checks in both the source .md and the compiled .lock.yml.

This repo (gh-aw-threat-detection) is the threat-detection component for gh-aw. It builds a Go CLI (threat-detect) and a container image (ghcr.io/github/gh-aw-threat-detection). It runs three smoke workflows (one per engine: Copilot, Claude, Codex) that exercise the full agent + threat-detection path, plus three containerized siblings that swap the generated inline detection step for the detector binary extracted from the published container image. The siblings are deterministically generated from the compiled lock files by scripts/create-threat-detection-sibling-workflows.py, and the Smoke dispatcher workflow fans out to all six.

The current compiler version is recorded in the lock-file metadata header. For example, .github/workflows/smoke-copilot.lock.yml:1 reads:

# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"caaa156b493ba5433a950aa55d2d08775075997876ddbaf2e62eabe6ee6cef64","compiler_version":"v0.72.1","agent_id":"copilot"}

All six lock files carry compiler_version":"v0.72.1" and schema_version":"v3". The generated maintenance workflow header at .github/workflows/agentics-maintenance.yml:15 reads # This file was automatically generated by pkg/workflow/maintenance_workflow.go (v0.72.1). DO NOT EDIT.

Relevant upstream changes between v0.72.1 and v0.77.5:

• v0.77.2 — breaking (https://github.com/github/gh-aw/releases/tag/v0.77.2): features.copilot-requests was removed; Copilot inference via github.token is now gated by an explicit permissions: copilot-requests: write. The Copilot smoke workflow uses engine: id: copilot (.github/workflows/smoke-copilot.md:11-12) and a copilot-engine threat-detection job, so this permission must be added. Verified during research: gh aw fix reports "No fixes needed" and does not auto-add this permission for smoke-copilot.md; it must be added by hand to the source frontmatter. Adding copilot-requests: write to the permissions block and recompiling with v0.77.5 succeeds and propagates the permission into both the agent job and the detection job of the compiled lock file.
• v0.77.0 — breaking (https://github.com/github/gh-aw/releases/tag/v0.77.0): the yield keyword was removed. None of the smoke .md sources use yield (verified: no yield token in .github/workflows/smoke-*.md), so no source change is required and gh aw compile does not error.
• v0.75.0 (https://github.com/github/gh-aw/releases/tag/v0.75.0): gh aw upgrade now also rewrites source .md files and actions-lock.json. This repository does not track an actions-lock.json (verified); action pins are carried inside the gh-aw-manifest metadata line of each lock file (e.g. .github/workflows/smoke-copilot.lock.yml:2).

The v0.77.5 compiler changes the lock-file schema_version from v3 to v4 and adds a body_hash field to the gh-aw-metadata header (verified by recompiling with v0.77.5). It also updates the embedded firewall/MCP image set (see Target state).

Current state

• Compiler/runtime version pinned to v0.72.1 across:
  • All six smoke lock files: .github/workflows/smoke-{copilot,claude,codex}.lock.yml:1 and .github/workflows/smoke-{copilot,claude,codex}-container.lock.yml:1.
  • The maintenance workflow generator: .github/workflows/agentics-maintenance.yml:15, with github/gh-aw-actions/setup@v0.72.1 at lines 95, 133, 162, 207, 253, 297; github/gh-aw-actions/setup-cli@v0.72.1 at line 177; and version: v0.72.1 at line 179.
  • The hand-maintained detection iteration workflow .github/workflows/detection-only.yml: github/gh-aw-actions/setup@v0.72.1 at line 93.
• Firewall and MCP image versions embedded by the v0.72.1 compiler:
  • ghcr.io/github/gh-aw-firewall/{agent,api-proxy,squid}:0.25.41 (e.g. .github/workflows/smoke-copilot.lock.yml:2, .github/workflows/detection-only.yml:127).
  • install_awf_binary.sh v0.25.41 (.github/workflows/detection-only.yml:190).
  • ghcr.io/github/gh-aw-mcpg:v0.3.6 and ghcr.io/github/github-mcp-server:v1.0.3 (.github/workflows/smoke-copilot.lock.yml:2).
• Source .md frontmatter is engine-specific:
  • Copilot: engine: id: copilot (.github/workflows/smoke-copilot.md:11-12).
  • Claude: engine: { id: claude, max-turns: 20, bare: true } (.github/workflows/smoke-claude.md:11-14).
  • Codex: engine: codex (.github/workflows/smoke-codex.md).
  • All three declare permissions: contents/issues/pull-requests/actions: read (e.g. .github/workflows/smoke-copilot.md:5-9), network.allowed: [defaults, github], runtimes.go.version: "1.23", and safe-outputs.threat-detection.continue-on-error: false. None declare copilot-requests.
• The containerized siblings are generated by scripts/create-threat-detection-sibling-workflows.py from the three base lock files (ENGINES map at lines 21-26). The script writes both the *-container.lock.yml and copies the *-container.md sidecar (generated_outputs, lines ~190-205). Its default detector image is ghcr.io/github/gh-aw-threat-detection:latest (scripts/create-threat-detection-sibling-workflows.py:22).
• The Smoke dispatcher runs all six lock workflows (.github/workflows/smoke.yml:23-28).
• CI (.github/workflows/ci.yml) runs go vet, go test -race, go build, and make docker-smoke (lines 50-54). It does not run gh aw compile --check (lock staleness) nor scripts/create-threat-detection-sibling-workflows.py --check (sibling freshness), so version drift is not gated by CI.
• The release lifecycle registry lists only v0.0.2 as active (releases/threat-detection-lifecycle.json:6-19); the detector container is not the subject of this upgrade.
• Documentation inconsistency: README.md:252 states the sibling default image is ghcr.io/github/gh-aw-threat-detection:v0.0.2, but the script default is :latest (scripts/create-threat-detection-sibling-workflows.py:22). Pre-existing; correct as a low-risk side fix.
• The maintenance workflow header references make recompile (.github/workflows/agentics-maintenance.yml:24), but the Makefile has no recompile target (verified: .PHONY list at Makefile:1-4). Pre-existing; see Out of scope.

Target state

• The pinned gh-aw version is advanced from v0.72.1 to v0.77.5. All compiled and generated artifacts carry the v0.77.5 metadata, action pins, and image pins.
• smoke-copilot.md gains copilot-requests: write in its permissions block, satisfying the v0.77.2 migration. (Assumption: Claude and Codex smoke workflows do not require copilot-requests because they do not use Copilot inference; confirm by recompiling — the permission should only appear where required.)
• All six smoke *.lock.yml files and agentics-maintenance.yml are regenerated by the v0.77.5 compiler from their .md sources. After regeneration the lock metadata shows schema_version":"v4", compiler_version":"v0.77.5", and a body_hash field.
• The embedded image/binary set updates to what v0.77.5 emits (verified by recompiling): ghcr.io/github/gh-aw-firewall/{agent,api-proxy,squid}:0.25.58, install_awf_binary.sh v0.25.58, ghcr.io/github/gh-aw-mcpg:v0.3.22, ghcr.io/github/github-mcp-server:v1.1.0.
• The setup-action pin advances to github/gh-aw-actions/setup@v0.77.5 (and setup-cli@v0.77.5 / version: v0.77.5 in the maintenance workflow). Important: this only happens when compiling with the official v0.77.5 release binary. A source build that reports version dev instead emits uses: ./actions/setup (a local action path) and omits compiler_version — see Required changes and Unresolved questions.
• The three *-container.lock.yml siblings and their *-container.md sidecars are regenerated from the new base lock files via scripts/create-threat-detection-sibling-workflows.py, and --check passes.
• detection-only.yml is updated so its gh-aw-actions/setup pin (line 93), firewall image tags (line 127), and AWF binary version (line 190) match the regenerated smoke-copilot-container.lock.yml.
• The upgraded workflows compile cleanly (gh aw compile --check), siblings verify, and the diff has been reviewed and/or the Smoke workflow dispatched successfully.

Required changes

The version bump is performed by the gh-aw CLI; do not hand-edit *.lock.yml files (marked DO NOT EDIT, e.g. .github/workflows/smoke-copilot.lock.yml:16). Concretely:

1. Install the official gh-aw v0.77.5 release binary.

   • Use gh extension install github/gh-aw pinned to v0.77.5 (or download the v0.77.5 release binary). Do not rely on a go install ...@v0.77.5 source build: a source build stamps the compiler version as dev, which (verified during research) omits the compiler_version metadata field and emits uses: ./actions/setup local paths instead of github/gh-aw-actions/setup@v0.77.5, dropping the setup action from the manifest. Only the release binary produces release-correct lock files.
   • Confirm gh aw version reports v0.77.5.

2. Add the copilot-requests permission to the Copilot source workflow.

   • Edit .github/workflows/smoke-copilot.md and add copilot-requests: write to the permissions block (currently contents/issues/pull-requests/actions: read at lines 5-9). Do not add it to smoke-claude.md or smoke-codex.md unless recompilation shows it is required there.

3. Recompile the smoke workflows and maintenance workflow.

   • Run gh aw compile so that .github/workflows/smoke-{copilot,claude,codex}.lock.yml and .github/workflows/agentics-maintenance.yml are regenerated with the v0.77.5 metadata, action SHAs, and firewall/MCP image pins.
   • Confirm the metadata header now shows compiler_version":"v0.77.5" and schema_version":"v4" in all three base lock files, the maintenance generator comment (agentics-maintenance.yml:15) reads v0.77.5, and copilot-requests: write appears in the Copilot agent job and detection job.

4. Regenerate the containerized siblings.

   • Run scripts/create-threat-detection-sibling-workflows.py (no args) to rewrite the three *-container.lock.yml files and their *-container.md sidecars from the freshly compiled base lock files.
   • The script parses generated structure with anchored regexes (extract_workflow_description line 44; extract_awf_lines lines 49-60 matching printf '%s\\n' ... gh-aw-firewall and sudo -E awf; add_packages_read lines 158-170; and the Upload threat detection log end anchor / Execute <Engine> CLI start anchors in transform lines 178-196 via EXECUTION_STEPS lines 28-32). If the v0.77.5 compiler changed any of those generated step names, indentation, or AWF lines, the script will raise ValueError; update the corresponding extractor(s) to match the new output.
   • Re-run scripts/create-threat-detection-sibling-workflows.py --check and confirm it exits 0.

5. Update detection-only.yml to track the new containerized Copilot smoke workflow.

   • detection-only.yml is hand-maintained but mirrors the compiled/containerized detection job (README.md:227). Update its github/gh-aw-actions/setup@ pin (currently @v0.72.1 at line 93), firewall image tags (currently :0.25.41 at line 127), and AWF binary version (currently v0.25.41 at line 190) to match the regenerated smoke-copilot-container.lock.yml (expected: @v0.77.5, :0.25.58, v0.25.58). Diff the regenerated containerized Copilot smoke job against the detection-only.yml detection job and bring the copied body back into alignment.

6. Update documentation for accuracy.

   • Correct README.md:252 so the documented sibling default image matches the script default (:latest, per scripts/create-threat-detection-sibling-workflows.py:22), or update the script if v0.0.2 is intended — pick one and make them consistent.

7. No Go code changes are expected. The cmd/threat-detect, pkg/detector, pkg/engine, and pkg/artifacts packages are independent of the gh-aw compiler version. Do not modify them unless a regenerated workflow surfaces a contract change in how the detector binary is invoked (e.g. a changed AWF wrapper flag), in which case scope that fix narrowly.

Acceptance criteria

• gh aw version reports v0.77.5 (official release binary, not a dev source build).
• .github/workflows/smoke-{copilot,claude,codex}.lock.yml line 1 metadata shows compiler_version":"v0.77.5" and schema_version":"v4".
• .github/workflows/smoke-copilot.md permissions block contains copilot-requests: write, and the compiled smoke-copilot.lock.yml contains copilot-requests: write in both the agent and detection jobs.
• smoke-claude.md / smoke-codex.md are unchanged except for recompiled lock output, and do not carry copilot-requests unless the compiler requires it.
• .github/workflows/agentics-maintenance.yml regenerated; generator comment (line 15) and all gh-aw-actions/setup@ / setup-cli@ / version: pins read v0.77.5.
• .github/workflows/smoke-{copilot,claude,codex}-container.lock.yml and *-container.md regenerated via scripts/create-threat-detection-sibling-workflows.py, with no ./actions/setup local-path references (i.e. compiled by the release binary).
• scripts/create-threat-detection-sibling-workflows.py --check exits 0.
• gh aw compile --check reports no drift for all source .md workflows.
• .github/workflows/detection-only.yml setup@, firewall image tags, and AWF binary version match the regenerated smoke-copilot-container.lock.yml (@v0.77.5, :0.25.58, v0.25.58).
• README.md sibling default-image statement is consistent with scripts/create-threat-detection-sibling-workflows.py.
• Existing Go validation still passes: make lint (go vet ./...), make test (race suite), make build, make lifecycle-validate.
• CI container smoke (make docker-smoke IMAGE_TAG=ci) still succeeds.
• At least one of: the Smoke dispatcher (.github/workflows/smoke.yml) ran successfully against the branch, or the regenerated diff was reviewed and any inability to run is recorded.

Tests / validation steps

# From repo root
gh extension install github/gh-aw       # pin/confirm v0.77.5 release binary
gh aw version                           # must report v0.77.5

# add copilot-requests: write to .github/workflows/smoke-copilot.md permissions block
gh aw compile                           # regenerate base smoke locks + agentics-maintenance.yml
gh aw compile --check                   # must report no staleness

scripts/create-threat-detection-sibling-workflows.py        # regenerate siblings
scripts/create-threat-detection-sibling-workflows.py --check # must exit 0

make lint
make test
make build
make lifecycle-validate
make docker-smoke IMAGE_TAG=ci

# Optional runtime validation (requires configured secrets):
gh workflow run smoke.yml --ref <branch>

Design decisions and rejected alternatives

• Use gh aw compile rather than hand-editing lock files. Lock files are generated and marked DO NOT EDIT; manual edits would diverge from the manifest and break the frontmatter/body-hash staleness check. Regeneration is the only supported path.
• Require the official release binary, reject a go install source build. Verified during research: a source build reports version dev, omits compiler_version, and emits uses: ./actions/setup local paths instead of github/gh-aw-actions/setup@v0.77.5. Committing such output would produce broken, unpinned workflows.
• Add copilot-requests: write manually rather than relying on gh aw fix. Verified during research: gh aw fix reports "No fixes needed" for smoke-copilot.md and does not add the permission, so it must be added to the source frontmatter by hand.
• Regenerate siblings via the existing Python script rather than by hand. The siblings are deterministic transforms of the base lock files; the script and its --check mode are the contract (README.md:223,229-234). Updating the script's extractors (if upstream changed generated step shapes) is preferred over manual sibling edits.
• Rejected: bumping only the firewall/MCP image tags in place. Those tags are emitted by the compiler; changing them without recompiling would desynchronize them from the action pins and metadata and fail gh aw compile --check.

Unresolved questions

• Can the implementer's environment install the official gh-aw v0.77.5 release binary and dispatch the Smoke workflow? In the research sandbox, gh extension install github/gh-aw and direct release-asset downloads returned HTTP 403 because the ambient GITHUB_TOKEN was invalid, so only a go install-based source build was possible — and that build is not release-correct (see Design decisions). If the implementing environment has the same limitation, the regeneration must be performed by a maintainer (or in an environment with a valid token / network access to the gh-aw releases and the pinned action SHAs). Maintainer directions are provided in Required changes and Steps to take after merging.
• Will the v0.77.5 compiler change generated step names or AWF wrapper lines enough to break the sibling script's extractors? The base recompile succeeds; the sibling regeneration was not run end-to-end against release output during research. The script may need extractor updates (Required changes step 4).

Implications

• Recompiling changes pinned action SHAs and firewall/MCP image tags in the lock manifests and bumps the lock schema_version to v4; reviewers should expect large, mostly-mechanical diffs in the six lock files and the maintenance workflow.
• .gitattributes marks *.lock.yml and *-container.md as generated with merge=ours, so these diffs are tool-produced, not reviewed line-by-line for logic.
• Adding copilot-requests: write widens the Copilot smoke workflow's token scope; maintainers configuring repository permissions should be aware.

Out of scope or notes

• Bumping the detector container release (releases/threat-detection-lifecycle.json) or DefaultThreatDetectionVersion (README.md:301) is not part of this issue; only the gh-aw tooling version and generated workflows are upgraded.
• The missing make recompile target referenced by agentics-maintenance.yml:24 is a pre-existing gap; adding it is optional and may be tracked separately.
• replay-detection.yml implements its own detector replay logic and does not embed a gh-aw compiler version (verified); it is out of scope.

Steps to take after merging

• Maintainer directions (if the implementer could not use the official release binary): in an environment with a valid GITHUB_TOKEN and network access, run gh extension install github/gh-aw (v0.77.5), add copilot-requests: write to smoke-copilot.md, run gh aw compile, then scripts/create-threat-detection-sibling-workflows.py, and commit the regenerated lock files, sidecars, and the updated detection-only.yml. Verify with gh aw compile --check and scripts/create-threat-detection-sibling-workflows.py --check.
• If runtime validation was deferred, a maintainer should dispatch .github/workflows/smoke.yml against main and confirm all six smoke workflows (and their issue-creation safe outputs) succeed.
• Consider a follow-up issue to add gh aw compile --check and scripts/create-threat-detection-sibling-workflows.py --check to .github/workflows/ci.yml so future version drift is caught automatically.
• Consider a follow-up to add the recompile Makefile target referenced by the generated maintenance workflow header.