find_binary crashes with IntrinsicError when $PATH entry returns EPERM/EACCES (sandbox)

[acidghost]

Describe the bug

pants check (and any goal that needs find_binary via the nodejs subsystem) crashes with IntrinsicError: Operation not permitted (os error 1) when one or more $PATH entries return EPERM (or EACCES) on stat/symlink_metadata. The pattern is identical to #21990 / #22519: a system-binary lookup fans path_metadata_request over every $PATH entry; the Rust VFS only maps NotFound (and, post-#23327, NotADirectory) to Ok(None); any other io::Error propagates and aborts the goal — including PermissionDenied, which is what happens when running pants inside a process sandbox that blocks read on some $PATH directories.

Engine traceback:
  in `check` goal
IntrinsicError: Operation not permitted (os error 1)

fix, fmt, lint, hadolint, yamllint, and ruff pass; only goals that reach the nodejs shim path fan-out (e.g. check with pyright) crash.

Pants version
2.31.0 (current main / 2.31.x HEAD has the same unhandled branch — see code refs below).

OS
macOS (Darwin 25.5.0), running under a userspace process sandbox (nono) that denies read on a handful of $PATH directories.

Root cause

Pants' local find_binary uses an enumerate-all path_metadata_request fan-out across every $PATH entry. The Rust VFS only swallows ErrorKind::NotFound and ErrorKind::NotADirectory; any other io::Error — including PermissionDenied (EPERM/EACCES) — becomes an IntrinsicError that aborts the goal before the requested binary is ever resolved.

Code path (verified against main @ commit 5dd30b2f70):

• src/python/pants/backend/javascript/subsystems/nodejs.py:416-420 — get_nodejs_process_tools_shims fans find_binary over (sh, bash, mkdir, rm, touch, which, sed, dirname, uname, …) via concurrently().
• src/python/pants/core/util_rules/system_binaries.py:731-740 — local env takes _find_candidate_paths_via_path_metadata_lookups.
• src/python/pants/core/util_rules/system_binaries.py:606-614 — concurrently(path_metadata_request(...) for path in search_path). No error tolerance.
• src/rust/engine/src/nodes/path_metadata.rs:33-42 — throw(format!(\"{e}\")) on any VFS error.
• src/rust/fs/src/posixfs.rs (lines ~285-290 on main) — only ErrorKind::NotFound and ErrorKind::NotADirectory map to Ok(None); everything else (including PermissionDenied) falls through Err(err) => Err(err).

#23327 (merged, in 2.31.1rc0) added the ENOTDIR branch but did not touch PermissionDenied. #22519 (still open) is almost certainly the same underlying bug, manifesting as os error 13 (EACCES) on a Docker-binary lookup rather than os error 1 (EPERM) on a nodejs-shim lookup.

Reproduction

Minimal repro using chmod (no sandbox required):

mkdir -p /tmp/eperm-path
chmod 000 /tmp/eperm-path
PATH=\"/tmp/eperm-path:\$PATH\" pants --no-pantsd check ::

The goal fails with IntrinsicError: Permission denied (os error 13) even though /tmp/eperm-path contains nothing pants needs.

Original observation (sandboxed): $PATH contained three entries the sandbox denied read on (~/.local/share/nvim/mason/bin, ~/.scripts, ~/.volta/bin). Stripping them from $PATH before invoking pants makes pants --no-pantsd check libs/<...>:: succeed (pyright runs cleanly on 37 files).

Suggested fix

Treat PermissionDenied the same way NotFound and NotADirectory are already treated in PosixFS::path_metadata:

Err(err) if err.kind() == ErrorKind::NotFound => Ok(None),
Err(err) if err.kind() == ErrorKind::NotADirectory => Ok(None),
Err(err) if err.kind() == ErrorKind::PermissionDenied => Ok(None),
Err(err) => Err(err),

Rationale: an unreadable $PATH entry is, from the perspective of the binary search, equivalent to a non-existent one — the caller cannot list it, cannot stat its contents, and therefore cannot find a binary in it. Aborting the entire goal for one unreadable entry is strictly worse than skipping it. The find_binary flow already silently tolerates $PATH entries that don't exist or aren't directories; tolerating ones it can't read is the same logical category. Worth considering whether to also handle this at a higher level (e.g. tolerate per-entry errors in _find_candidate_paths_via_path_metadata_lookups rather than relying on the VFS to mask them), but the minimal Rust-side fix is sufficient to unblock sandboxed environments.

Workarounds

1. Strip the offending entries from \$PATH before invoking pants.
2. Pin [nodejs] search_path (and equivalent options for other binary lookups) in pants.toml to an explicit list of readable directories instead of relying on <PATH>.
3. Grant the sandbox read access to the relevant \$PATH directories.

Related

• Permission denied (os error 13) building Docker #22519 — open, very likely the same root cause (Docker binary lookup, EACCES / os error 13).
• Pants 2.24 - IntrinsicError: Not a directory (os error 20) #21990 / Fix: IntrinsicError when a non-directory PATH entry is used with system_binary #23327 — same code path, fixed the ENOTDIR shape; this issue is the unaddressed PermissionDenied shape.