Skip to content

fix(workflow): prevent multiline GITHUB_OUTPUT in red-team benchmark#4176

Merged
lpcox merged 1 commit into
mainfrom
fix/red-team-benchmark-grep-output
Jun 2, 2026
Merged

fix(workflow): prevent multiline GITHUB_OUTPUT in red-team benchmark#4176
lpcox merged 1 commit into
mainfrom
fix/red-team-benchmark-grep-output

Conversation

@lpcox
Copy link
Copy Markdown
Collaborator

@lpcox lpcox commented Jun 2, 2026

Problem

The red-team benchmark workflow fails at step 15 ("Run AWF-protected benchmark") with:

##[error]Unable to process file command 'output' successfully.
##[error]Invalid format '0'

Root cause: grep -c "DENIED" outputs 0 to stdout even when it exits with code 1 (no matches). The || echo "0" fallback then appends a second 0, making AWF_BLOCKED a multiline value (0\n0). When written to $GITHUB_OUTPUT, the bare 0 on the second line has no key= prefix — invalid format.

Fix

Replace || echo "0" with || true. Since grep -c already outputs 0 on no-match, the variable captures the correct single-line value.

Failing run

https://github.com/github/gh-aw-firewall/actions/runs/26790836372/job/78976724535#step:15:1

grep -c outputs '0' to stdout even when it exits with code 1 (no
matches). The || echo "0" fallback then appends a second '0', making
AWF_BLOCKED a multiline value. When written to $GITHUB_OUTPUT, the
bare '0' on the second line has no key= prefix, causing:

  ##[error]Invalid format '0'

Fix: use || true instead — grep -c already outputs '0' on no-match,
so the variable captures the correct value without duplication.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 2, 2026 00:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a GitHub Actions workflow failure in the red-team benchmark by ensuring AWF_BLOCKED is written to $GITHUB_OUTPUT as a single-line key=value output, even when grep finds no matches.

Changes:

  • Remove the || echo "0" fallback after grep -c "DENIED" to prevent producing a multiline AWF_BLOCKED value.
  • Regenerate/update the locked workflow YAML to reflect the source workflow change.
Show a summary per file
File Description
.github/workflows/red-team-benchmark.md Updates the AWF_BLOCKED assignment to avoid multiline $GITHUB_OUTPUT values when grep -c exits non-zero on no matches.
.github/workflows/red-team-benchmark.lock.yml Propagates the same fix into the compiled/locked workflow output.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

Smoke Test: Claude Engine

  • ✅ GitHub API: 2 PR entries found
  • ✅ GitHub check: playwright_check PASS
  • ✅ File verify: smoke-test-claude-26791441773.txt exists

Result: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

🔥 Smoke Test: Copilot BYOK (Offline Mode)

Test Result
GitHub MCP (list PRs) ✅ PR #4172 returned correctly
GitHub.com connectivity ⚠️ Pre-step data unavailable (template vars unexpanded)
File write/read ⚠️ Pre-step data unavailable (template vars unexpanded)
BYOK inference (api-proxy → api.githubcopilot.com) ✅ Responding via BYOK offline mode

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: PARTIAL — BYOK inference ✅, pre-step data not injected into prompt.

PR by @lpcox.

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

Merged PRs: "fix(test): correct smoke-claude version assertion to match v0.76.1"; "fix: recompile all workflow lock files". Results: GitHub reads ✅; safeinputs-gh query ❌; Playwright ✅; file write ✅; bash verify ✅; discussion ✅; build ✅. Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

🤖 Smoke Test Results

Test Status
GitHub MCP connectivity
GitHub.com HTTP connectivity ✅ (HTTP 200/301)
File write/read ✅ (smoke-test-copilot-26791441801.txt verified)

Overall: PASS

PR: fix(workflow): prevent multiline GITHUB_OUTPUT in red-team benchmark — author @lpcox

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color passed ✅ PASS
Go env passed ✅ PASS
Go uuid passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #4176 · sonnet46 1.1M ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

Gemini Engine Smoke Test Results

  • GitHub MCP Testing: ✅
    • fix(test): correct smoke-claude version assertion to match v0.76.1
    • fix: recompile all workflow lock files
  • GitHub.com Connectivity: ✅
  • File Writing Testing: ✅
  • Bash Tool Testing: ✅

Overall status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

Smoke Test: GitHub Actions Services Connectivity

Check Result
Redis PING ❌ Connection timeout
PostgreSQL pg_isready ❌ No response
PostgreSQL SELECT 1 ❌ No response

host.docker.internal resolves to 172.17.0.1 (Docker bridge) but ports 6379 and 5432 are unreachable from inside the AWF agent container — iptables rules block non-HTTP/S ports to the host bridge.

Overall: FAIL

🔌 Service connectivity validated by Smoke Services

@lpcox lpcox merged commit b14271e into main Jun 2, 2026
59 of 62 checks passed
@lpcox lpcox deleted the fix/red-team-benchmark-grep-output branch June 2, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants