· ENPT-BR
Grok Build for Alembic · Honest operation

Tests and the real Proof Gate

Separate contract tests from the host boundary that proves stdin, auth, streaming, and exact output.

Primary source
Pinned headless mode guide

Pinned upstream commit: b189869b7755d2b482969acf6c92da3ecfeffd36

The Simple layer stands alone; open Technical for exact contracts and edge cases.
You will be able to
  • Name what focused tests prove and what they cannot prove.
  • Read an exact sentinel as a real protocol boundary.
  • Keep Docker and production claims outside the host proof.
1

The big idea

Unit tests prove our translation rules without spending or depending on an installed CLI. A real smoke proves a different layer: the released `grok` binary accepts `/dev/stdin`, authenticates, streams JSONL, exits zero, and returns exactly the requested sentinel. Both are required; neither can stand in for the other.

Think of it like… Bench-test the adapter, then plug it into the real wall socket. A perfect bench reading cannot prove the building has power; a glowing lamp cannot prove every safety rule in the adapter. The analogy breaks because the live smoke also exercises a paid/remote model boundary.

Focused Vitest cases cover command construction, prompt isolation from argv, shell escaping, permission flag gating, effort enum mapping, mandatory built-in web-search disable, fail-closed sessions, text/end/error parsing, incomplete usage, and malformed input.

The host boundary was run on 2026-07-16 with Grok CLI 0.2.101: a prompt entered through `--prompt-file /dev/stdin`, the process exited 0, and ordered text deltas reconstructed exactly `GROK_DEV_STDIN_OK`. Prompt transport is proven on the current macOS host and expected in a future Linux image; native Windows, Docker readiness, and session resume are not yet proven.

Evidence: Pinned headless mode guide

Vitesttypecheckbuildgrok 0.2.101/dev/stdin Safe: report each gate's exact boundary and output.
Separate contract tests from the host boundary that proves stdin, auth, streaming, and exact output.
2

Compare the boundaries

Safe
Safe: report each gate's exact boundary and output.
Risky
Risky: call a typecheck or host smoke “production complete”.
SurfaceUpstreamAlembicVerdict
Unit contractDeterministic parser/command behaviorVitestPROVE
Host CLIBinary + stdin + auth + network + streamExact sentinelPROVE
Docker / rosterImage install + credentials + routingNot exercised hereNOT PROVEN

The host boundary was run on 2026-07-16 with Grok CLI 0.2.101: a prompt entered through `--prompt-file /dev/stdin`, the process exited 0, and ordered text deltas reconstructed exactly `GROK_DEV_STDIN_OK`. Prompt transport is proven on the current macOS host and expected in a future Linux image; native Windows, Docker readiness, and session resume are not yet proven.

3

Real code and commands

packages/factory/src/GrokAgentProvider.test.ts + live host
pnpm --filter @alembic/factory test
pnpm --filter @alembic/factory typecheck

printf '%s' 'Reply with exactly GROK_DEV_STDIN_OK.' | \
  grok --prompt-file /dev/stdin --model grok-4.5 \
  --reasoning-effort low --output-format streaming-json \
  --disable-web-search --always-approve --cwd /tmp

# observed text result: GROK_DEV_STDIN_OK
# observed exit: 0
How to reach it
sed -n '1,240p' packages/factory/src/GrokAgentProvider.test.ts
grok --version && grok --help | rg "prompt-file|streaming-json|reasoning-effort"

Evidence: RESOURCES.md

4

Visual atlas

inputproofVitest
Trace the boundary: Vitest.
surfaceseamownertypecheck
Locate the owner: typecheck.
claimwithout gateevidenceat boundaryvsbuild
Compare claim and evidence: build.
learnexecuteverifygrok 0.2.101
Read the sequence: grok 0.2.101.
complete?yes / no/dev/stdin
Find the decision gate: /dev/stdin.
inputproofauth
Trace the boundary: auth.
surfaceseamownerJSONL
Locate the owner: JSONL.
claimwithout gateevidenceat boundaryvstext
Compare claim and evidence: text.
learnexecuteverifyend
Read the sequence: end.
complete?yes / noexit 0
Find the decision gate: exit 0.
inputproofsentinel
Trace the boundary: sentinel.
surfaceseamownerFactory run
Locate the owner: Factory run.
claimwithout gateevidenceat boundaryvsDocker
Compare claim and evidence: Docker.
learnexecuteverifyreport
Read the sequence: report.
5

Practice and feedback

Predict before revealing

All unit tests pass, but `/dev/stdin` was never tried with the released CLI. Is the transport proven?

No. The contract is proven; the external boundary still needs the live sentinel.

Retrieval cards

Retrieval cardsContract testClick or press Enter to flip
Contract testProves deterministic adapter behavior.
Retrieval cardsSentinelClick or press Enter to flip
SentinelExact output chosen to make the smoke unambiguous.
Retrieval cardsReal boundaryClick or press Enter to flip
Real boundaryThe released binary and external service actually run.
Retrieval cardsProof scopeClick or press Enter to flip
Proof scopeThe precise claims one gate can support.
Unit tests prove our translation rules without spending or depending on an installed CLI. A real smoke proves a different layer: the released `grok` binary accepts `/dev/stdin`, authenticates, streams JSONL, exits zero, and returns exactly the requested sentinel. Both are required; neither can stand in for the other.
Focused Vitest cases cover command construction, prompt isolation from argv, shell escaping, permission flag gating, effort enum mapping, mandatory built-in web-search disable, fail-closed sessions, text/end/error parsing, incomplete usage, and malformed input.

Inspect each proof layer

Select a gate to see the strongest honest claim it supports.

Verdict

Four-slide summary

Contract test

Proves deterministic adapter behavior.

Sentinel

Exact output chosen to make the smoke unambiguous.

Real boundary

The released binary and external service actually run.

Proof scope

The precise claims one gate can support.

Check your model

Spaced review

What did `GROK_DEV_STDIN_OK` prove?

It proved the host CLI stdin/stream path only; image and session gates remain separate.
6

Sources and next move

Pinned headless mode guideb189869b7755 · headlessPinned session management guideb189869b7755 · sessionsRESOURCES.mdGrok Build for Alembic
Ask your teacher to challenge a verdict, replay an event, or review a future unit against the same proof discipline.