<!-- AUTO-GENERATED from SKILL.md.tmpl — do not edit directly --> <!-- Regenerate: bun run gen:skill-docs -->

Plan Status Footer

Skills that run plan reviews (/plan-*-review, /codex review) include the EXIT PLAN MODE GATE blocking checklist at the end of the skill, which verifies the plan file ends with ## GSTACK REVIEW REPORT before ExitPlanMode is called. Skills that don't run plan reviews (operational skills like /ship, /qa, /review) typically don't operate in plan mode and have no review report to verify; this footer is a no-op for them. Writing the plan file is the one edit allowed in plan mode.

/benchmark-models — Cross-Model Skill Benchmark

You are running the /benchmark-models workflow. Wraps the gstack-model-benchmark binary with an interactive flow that picks a prompt, confirms providers, previews auth, and runs the benchmark.

Different from /benchmark — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.

Step 0: Locate the binary

BIN="$HOME/.claude/skills/gstack/bin/gstack-model-benchmark"
[ -x "$BIN" ] || BIN=".claude/skills/gstack/bin/gstack-model-benchmark"
[ -x "$BIN" ] || { echo "ERROR: gstack-model-benchmark not found. Run ./setup in the gstack install dir." >&2; exit 1; }
echo "BIN: $BIN"

If not found, stop and tell the user to reinstall gstack.

Step 1: Choose a prompt

Use AskUserQuestion with the preamble format:

• Re-ground: current project + branch.
• Simplify: "A cross-model benchmark runs the same prompt through 2-3 AI models and shows you how they compare on speed, cost, and output quality. What prompt should we use?"
• RECOMMENDATION: A because benchmarking against a real skill exposes tool-use differences, not just raw generation.
• Options:
• A) Benchmark one of my gstack skills (we'll pick which skill next). Completeness: 10/10.
• B) Use an inline prompt — type it on the next turn. Completeness: 8/10.
• C) Point at a prompt file on disk — specify path on the next turn. Completeness: 8/10.

If A: list top-level gstack skills that have SKILL.md files (from find . -maxdepth 2 -name SKILL.md -not -path './.*'), ask the user to pick one via a second AskUserQuestion. Use the picked SKILL.md path as the prompt file.

If B: ask the user for the inline prompt. It never goes into a shell command: create a prompt file and write the prompt into it verbatim, then use the printed path as the prompt file.

_GT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)/.gstack/tmp"
mkdir -p "$_GT" && chmod 700 "$_GT" || { echo "Not sent: cannot create $_GT for the text file." >&2; exit 1; }
_EX=$(git rev-parse --git-path info/exclude 2>/dev/null) && mkdir -p "$(dirname "$_EX")" && { grep -qxF '/.gstack/tmp/' "$_EX" 2>/dev/null || echo '/.gstack/tmp/' >> "$_EX"; }
PROMPT_FILE=$(mktemp "${_GT:?}/benchmark-prompt.XXXXXX") || { echo "Not sent: mktemp failed in $_GT." >&2; exit 1; }; echo "PROMPT_FILE: $PROMPT_FILE (name: ${PROMPT_FILE##*/})"

Write the text into each printed file with your file-write tool (Claude Code's Write tool needs a Read of the empty file first), exactly as it should appear. The text never goes into a shell command, heredoc or quoted argument. If a write fails or is refused, do not send: print the cause, the file path and the command below for sending by hand.

If C: ask for the path. Verify it exists. Use as positional argument.

Step 2: Choose providers

"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini,particle,deepseek --dry-run

Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).

If every provider shows NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest claude login, codex login, gemini login / export GOOGLE_API_KEY, or export OPENROUTER_API_KEY for particle and deepseek (they run in Claude Code, so they also need the claude CLI).

particle and deepseek send the prompt and the working directory's files the model reads to OpenRouter and the model vendor. They load no Claude Code settings files (so a key in a settings.json env block is never sent) and run with the user's environment minus Anthropic credentials. Say so before including them.

If at least one is OK: AskUserQuestion:

• Simplify: "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
• RECOMMENDATION: A (all authed providers) because running as many as possible gives the richest comparison.
• Options:
• A) All authed providers. Completeness: 10/10.
• B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
• C) Pick two — specify on next turn. Completeness: 8/10.

Step 3: Decide on judge

[ -n "$ANTHROPIC_API_KEY" ] || grep -q 'ANTHROPIC' "$HOME/.claude/.credentials.json" 2>/dev/null && echo "JUDGE_AVAILABLE" || echo "JUDGE_UNAVAILABLE"

If judge is available, AskUserQuestion:

• Simplify: "The quality judge scores each model's output on a 0-10 scale using Anthropic's Claude as a tiebreaker. Adds about USD 0.05/run. Recommended if you care about output quality, not just latency and cost."
• RECOMMENDATION: A — the whole point is comparing quality, not just speed.
• Options:
• A) Enable judge (adds about USD 0.05). Completeness: 10/10.
• B) Skip judge — speed/cost/tokens only. Completeness: 7/10.

If judge is NOT available, skip this question and omit the --judge flag.

Step 4: Run the benchmark

Construct the command from Step 1, 2, 3 decisions:

PROMPT_PATH="<prompt-path>"
[ -s "$PROMPT_PATH" ] || { echo "ERROR: $PROMPT_PATH is missing or empty; the benchmark needs a prompt file." >&2; exit 1; }
"$BIN" --models <picked-models> [--judge] --output table -- "$PROMPT_PATH"

<prompt-path> is the prompt file from Step 1 (the SKILL.md path, the printed prompt file, or the user's path) and <picked-models> is the comma-separated list from Step 2. Use a path only if it has no ', ", backtick, $ or \; otherwise copy the file into a new Step 1B prompt file and use that.

Stream the output as it arrives. This is slow — each provider runs the prompt fully. Expect 30s-5min depending on prompt complexity and whether --judge is on.

Step 5: Interpret results

After the table prints, summarize for the user:

• Fastest — provider with lowest latency.
• Cheapest — provider with lowest cost.
• Highest quality (if --judge ran) — provider with highest score.
• Best overall — use judgment. If judge ran: quality-weighted. Otherwise: note the tradeoff the user needs to make.

If any provider hit an error (auth/timeout/rate_limit), call it out with the remediation path.

Step 6: Offer to save results

AskUserQuestion:

• Simplify: "Save this benchmark as JSON so you can compare future runs against it?"
• RECOMMENDATION: A — skill performance drifts as providers update their models; a saved baseline catches quality regressions.
• Options:
• A) Save to ~/.gstack/benchmarks/<date>-<skill-or-prompt-slug>.json. Completeness: 10/10.
• B) Just print, don't save. Completeness: 5/10 (loses trend data).

If A: re-run with --output json and tee to the dated file. Print the path so the user can diff future runs against it.

Important Rules

• Never run a real benchmark without Step 2's dry-run first. Users need to see auth status before spending API calls.
• Never hardcode model names. Always pass providers from user's Step 2 choice — the binary handles the rest.
• **Never auto-include --judge.** It adds real cost; user must opt in.
• If zero providers are authed, STOP. Don't attempt the benchmark — it produces no useful output.
• Cost is visible. Every run shows per-provider cost in the table. Users should see it before the next run.