Electron Stagewrightdocs

ADR-012: Production validation plugin

Status: Accepted. Current checks cover macOS bundle structure, metadata, update/crash machinery, code signing, notarization, and Gatekeeper; Windows Authenticode; and AppImage embedded signatures.

Context

Electron's sharpest production pain is distribution: an app that runs fine in development fails on a user's machine because it is unsigned, not notarized, or its bundle is malformed — and the failure is opaque. Stagewright drives running apps; nothing inspects the build artifact. The production validation plugin closes that: given a packaged .app, report — structured — whether it is production-ready.

The acceptance criteria carry one subtle requirement: distinguish missing evidence from failed evidence. "I checked and the signature is invalid" and "I could not check (no toolchain here)" are different answers; collapsing them produces false confidence or false alarms.

Decision

1. A three-valued evidence model

Every check returns status: 'pass' | 'fail' | 'unknown':

The tool returns { ok, app_path, passed, summary: { pass, fail, unknown }, checks }. The envelope ok is true whenever validation RAN; the app's verdict is passed (no fail). unknown checks do not flip passed, but summary discloses them so a green-with-skips result is never mistaken for full verification. Only a bad input (no app at appPath) is a tool error (production.APP_NOT_FOUND / production.NOT_A_BUNDLE) — a failed CHECK is data, not an error, matching the AC's "return structured failures".

2. Shell out to the toolchain, not into app code

The checks invoke the macOS toolchain (codesign --verify --deep --strict, spctl --assess, xcrun stapler validate, plutil -convert json) rather than evaluating app JavaScript. So the plugin needs no --allow-eval and no running session — it inspects a path on disk. Every spawn is timeout-bounded via a shared runCommand (execFile + timeout + capped output) that never rejects; a command-not-found or timeout becomes spawnError, which the checks map to unknown. There is deliberately no platform branch: on a non-macOS host the tools are simply absent (ENOENT → unknown), which also lets tests drive every branch through a fake runCommand on any OS.

3. macOS first; bundle structure stays dependency-free

macOS is the first-class target. The bundle-structure check is pure filesystem (Info.plist + Contents/MacOS/ executable present), so it runs anywhere and needs no plist parser. The notarization check uses xcrun stapler validate to confirm a ticket is stapled to the bundle (offline, so a non-zero exit is a real fail, never unknown); a pass reads the spctl source= line as best-effort evidence. The info-plist check shells out to plutil -convert json (which reads both XML and binary plists) and verifies CFBundleIdentifier (reverse-DNS), CFBundleShortVersionString, and a CFBundleExecutable that exists under Contents/MacOS/. The protocol-schemes check reads the same plist and validates every CFBundleURLTypes entry (RFC-3986 scheme shape, no duplicates across entries, no shadowing of well-known system schemes); declaring no schemes is an affirmative pass. The updater-feed check is pure filesystem: a packaged Contents/Resources/app-update.yml (electron-updater) must declare a provider with its per-provider required fields and https URLs. An ABSENT file is unknown, because the built-in autoUpdater can set its feed at runtime, which a static scan cannot see. When the app is under electron-builder's conventional mac / mac-<arch> unpacked output, the diagnostic instead explains that this staging layout is also produced by --dir, is not itself a distributable artifact, and directs validation to the app from the release DMG or ZIP. The crash-reporter check is pure filesystem: the crashpad handler must ship intact (and executable) under Electron Framework.framework/Versions/<v>/Helpers/; a missing framework is unknown (not an Electron-shaped bundle), while a present framework whose handler is missing or lost its execute bit is a fail — packaging silently disabled crash capture.

Alternatives considered

Consequences

Status update (library and standalone CLI, 2026-07-28)

Consumer dogfooding needed the production checks inside a CI evidence collector where no MCP session exists. The check engine is therefore exposed through a transport-neutral validateProductionApp() API and two equivalent command routes:

electron-stagewright production validate --app /path/to/My.app --json
electron-stagewright-production validate --app /path/to/My.app --json

The root command dynamically delegates to the separately installed production package. Core does not depend on a plugin, and the plugin continues to depend on core only for its MCP adapter, so the package graph remains acyclic. Both binaries emit the same versioned JSON report and stable CI exit codes: 0 for no failed checks, 1 for one or more failed checks, and 2 when validation could not start because usage or input was invalid. Unknown evidence remains explicit and does not become a failure.

The MCP tool, public API, and CLI all call the same orchestration function. Path checks, timeout validation, canonical check ordering, summary aggregation, and the pass/fail/unknown model now have one implementation.

Status update (cross-platform release artifacts, 2026-07-28)

Consumer release matrices now build macOS, Windows, and Linux artifacts from one commit. Validation therefore recognizes three artifact families and selects relevant checks when checks is omitted:

The report adds artifact_type while retaining app_path for compatibility. Explicit check subsets still use the canonical global order; a platform-specific check selected for an inapplicable artifact returns unknown, not a false failure. Unsupported regular files fail before validation with production.UNSUPPORTED_ARTIFACT.

Windows validation invokes the built-in Windows PowerShell Get-AuthenticodeSignature -LiteralPath. The path is UTF-8/base64 data decoded by a fixed script, never interpolated into executable PowerShell. Valid is a pass. NotSigned, HashMismatch, NotTrusted, and UnknownError are verified failures; unsupported/incompatible formats, missing PowerShell, command failures, and malformed output remain unknown.

AppImage's --appimage-signature flag only prints the embedded signature and would execute the untrusted artifact. The plugin deliberately does not use it. It invokes AppImageKit's external validate helper instead and requires both exit zero and a Good signature marker for a pass. A missing public key is unknown; an absent, bad, or malformed embedded signature is a failure. This keeps the original three-valued evidence rule and never turns tool absence into a defect.

References