Configured dogfood and transcript replay suites

Capability availability discovery has a separate bounded workflow; see the capability discovery guide. Its candidates are review inputs and do not change the suite’s governance or application-verification meaning.

praetorctl dogfood suite runs a finite, repeatable set of pinned public adoption cases and explicitly selected local transcript cases. The checked-in public suite contains Cobra and Flask plus six representative public repositories covering Rust, JavaScript, C++, C#, Java, and Haskell (ShellCheck). Every public entry is pinned to a live immutable GitHub commit; use it to rerun the same inputs after a Praetor change. Every invocation requires a new evidence directory; earlier results and original sources remain intact.

Public configuration

The checked-in .config/dogfood/public-suite.json selects eight pinned public repositories. Preview it without network or transcript access:

praetorctl dogfood suite --config .config/dogfood/public-suite.json \
  --source-root . --artifacts /existing/private/evidence/public-plan --stage plan

plan is the default. It validates declarations and writes plan.json and report.json; it does not check whether a source exists or whether its content matches a pin. Its status is always planned, never verified.

Execute actual adoption, verification and repeat-apply checks with a new output:

praetorctl dogfood suite --config .config/dogfood/public-suite.json \
  --source-root . --artifacts /existing/private/evidence/public-run --stage verify

The public loop retains fresh clones and checks actual lock digests, generated context, HISS debt and reconciliation stability. It never executes upstream application tests, build scripts, hooks or agent instructions.

The adoption command plan reports declared native commands and missing gates separately. A governance result of verified does not mean those commands ran; an unavailable native build contract remains unavailable and does not create a governance repair job.

Public cases use the resolved audit policy for both the original and post-adoption scans. Each attempt must retain the planned policy identity, and its saved baseline must match the independently scanned original debt. A checkout that already carries .standards-baseline.json is therefore re-recorded by the plan and the first attempt, with the loop's rationale stored when the count rises (recordPublicBaseline in internal/dogfood/public.go, TestRunPublicLoopReRecordsAnExistingBaseline); a plain adoption keeps an existing baseline (the baseline on a re-adoption). Reports include the effective digest, function-length limit and validated baseline hash. A stricter profile can therefore reject a touched function below the previous fixed scanner limit. Re-run older public results before treating them as evidence for a stricter policy.

Repair planning rejects old verified public reports that lack policy/baseline evidence, and checks retained policy digests and ratchet results for internal consistency. Failed historical reports remain available for triage. These metadata checks do not replace a fresh pinned run. Saved public baselines are bounded to 8 MiB; incomplete scans and malformed or inconsistent baseline entries cannot produce a verified result.

New HISS scan reports retain coverage: files actually read by supported scanners, files outside those extensions, and a bounded extension/count map. The latter includes documentation/configuration files and unsupported languages such as C# and TypeScript. It is not a list of violations. The map retains at most 32 extensions of at most 32 bytes; excess files remain counted separately. An empty extension key denotes an extensionless file. Ignored directories, symlinks and oversized supported inputs remain in skips; truncated limits the whole report, including observed coverage. A historical report without coverage has unknown coverage. Zero infractions does not establish that a repository's application language was analyzed. CLI/MCP audit summaries expose this scope; application verification still requires the native test tools.

Private workstation configuration

Use workstation inventory to precheck local repositories, worktrees and local-only identities through the CLI or MCP before selecting cases.

Store private configuration and evidence outside the public repository. A config must contain version, public_repositories and transcripts, with optional input_limits as described below. All case fields below are required; replace the example path and SHA with a reviewed source:

{
  "version": 1,
  "public_repositories": [],
  "transcripts": [
    {
      "id": "workstation-session",
      "source_path": "/absolute/path/to/session.jsonl",
      "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "format": "claude-code-jsonl-v1"
    }
  ]
}

source_path must be an explicit, clean absolute path. Supported formats are claude-code-jsonl-v1 and antigravity-jsonl-v1. For Antigravity, a sibling transcript_full.jsonl is preferred; pin that selected file's SHA256. No home directory discovery, uploads or global rule rewrites happen during a suite. See transcript ingestion for format and exclusion rules.

Verification ingests every page into a fresh private cache, then replays every page into that same cache. It requires consistent source identity, complete record accounting, at least one stored observation, no truncated records, and zero additional writes on replay. Metadata/thinking exclusions remain explicit; their whole-source counts in page reports must not be summed across pages. The result establishes observed-event persistence and replay, not verified facts or working semantic memory recall.

Configured input limits

Suite configs and discovery policies accept an optional input_limits object. Omitting it keeps the defaults below. If present, both groups and every field are required positive integers; unknown, duplicate, null, and case-variant keys fail before execution. Limits affect input admission, not HISS compliance thresholds.

"input_limits": {
  "snapshot": {
    "max_entries": 20000,
    "max_file_bytes": 33554432,
    "max_tree_bytes": 268435456
  },
  "verification": {
    "max_entries": 131072,
    "max_files": 128,
    "max_depth": 32,
    "max_file_bytes": 262144,
    "max_total_bytes": 2097152
  }
}

This example admits larger model assets and monorepo metadata. It is part of the pinned config or policy hash; CLI and MCP consume the same file. A suite records resolved limits and uses them for the original snapshot, every repeat application, and native command planning. Discovery records the limits and reuses them for its final tree readback.

Bound	Default	Maximum configurable value
Snapshot entries, including root	20,000	20,000
Snapshot bytes per file	16 MiB	256 MiB
Snapshot tree bytes	256 MiB	1 GiB
Native planning entries	65,536	200,000
Native metadata files	128	512
Native metadata depth	32	64
Native metadata bytes per file	64 KiB	1 MiB
Native metadata total bytes	2 MiB	16 MiB

The native planning defaults and ceilings are adoption's own (adopt.DefaultVerificationLimits and adopt.NormalizeVerificationLimits in internal/adopt/verification_limits.go), so a suite that omits input_limits plans with the same bounds as praetorctl adopt.

A per-file limit cannot exceed its corresponding total. Snapshot hashes stream file contents; symlinks record targets without reading linked content. Snapshots retain completion status, observed counters, and file/entry failure details. Over-budget or failed reads cannot produce a complete tree identity or a verified case. Input completeness does not qualify native application commands: these remain declared, unavailable, or preserved until independently exercised.

Evidence, bounds and failure behavior

The config is bounded to 64 KiB and 1–8 total cases. Each explicit public_repositories entry enrolls a public GitHub repository at an immutable commit; no separate repository allowlist or personal fork configuration is needed. Use https://github.com/owner/repo#<commit> with a lowercase 40- or 64-hex commit pin. The admitted spelling uses an ASCII alphanumeric/hyphen owner of 1–39 bytes with alphanumeric endpoints, and an ASCII alphanumeric/dot/underscore/hyphen repository name of 1–100 bytes. Credentials, alternate hosts, ports, queries, percent escapes, whitespace, extra path segments, trailing slashes, ./.. repository names and .git suffix aliases are rejected. Repository identities are compared without case, so spelling or pin changes cannot enroll duplicates.

The eight popular benchmark defaults remain available. Exploratory public-loop runs may omit pins only for those defaults; suites always require pins. Enrollment permits disposable clone and Praetor adoption checks only. It does not authorize upstream hooks, build scripts or application tests, and MCP public verification still requires the server's existing remote opt-in. Plan records declarations without cloning and remains unverified.

Duplicate IDs, duplicate transcript paths, unknown or duplicate JSON fields, invalid formats, null lists, and unpinned public URLs fail before evidence creation or execution. Config and evidence paths reject symlink ancestors; Unix config/source reads use nonblocking opens to reject FIFO replacement. Kernel filesystem I/O is not a hard real-time cancellation guarantee, especially on other platforms.

The suite has a 15-minute deadline. Each public case retains the lower public loop's five-minute deadline and two-application stability check. Each transcript pass uses pages of at most 10,000 records and at most ten pages, within the ingester's 64 MiB source and 100,000-record bounds.

New evidence directories use mode 0700 and metadata/cache files 0600, respecting stricter existing process permissions. A valid suite writes its declaration plan, one case-NN.json per completed/failed case, and a final report.json. Each case links retained public results or both transcript passes. Failures preserve actual page counters, including partial writes. Later cases still receive outcomes; deadline exhaustion marks them skipped_due_to_context. Any failed case or evidence-write error produces a nonzero exit and cannot claim aggregate success. An interrupted process may leave a plan or case reports without a final report; that is incomplete evidence. Run again into a new directory.

The report includes the exact config hash and the engine build in engine_build (engineBuild in internal/dogfood/suite.go; discovery and scheduled suite reports record the same map):

Key	Value
`version`	What `praetorctl version` prints for the binary that ran the suite: the injected release, the 12-character commit (suffixed `-dirty` for a modified tree), the module version of a `go install` build, or `unknown (<reason>)` (version each binary reports)
`revision`	The full commit from Go's VCS stamp, or `unavailable`
`modified`	`true` or `false`, present only with a VCS stamp
`go_version`	The Go toolchain, present when the binary carries build information

praetorctl and standards-mcp pass their own identity into the suite, so a report from a release binary names that release. A test binary has neither an injected release nor a VCS stamp; its report records unknown (<reason>) and stays loadable by praetorctl dogfood repairs. internal/dogfood/engine_build_test.go covers a release, an unstamped and a module-version build. The development MCP launcher additionally records the actual source and binary hashes.

Development MCP

standards_dogfood_suite exposes the same consumer. Both top-level paths and transcript paths inside the loaded config obey server confinement. Public verification requires the server's remote opt-in; a client argument cannot grant it.

python3 scripts/dev_mcp.py call --allow-remote-benchmarks --timeout 300 \
  standards_dogfood_suite \
  '{"config_path":".config/dogfood/public-suite.json","artifact_dir":".workingdir/evidence/new-suite-run","stage":"verify"}'

The artifact parent must already exist. A shorter MCP/client timeout can stop a suite before its 15-minute library ceiling. For private sources, select an explicit server root containing the config, sources and evidence, or use the CLI.

This stage provides one-shot execution; scheduled runs are described in dogfood scheduling. A suite run does not install timers, dispatch models, apply agent-generated fixes, publish upstream changes, or advance repository rollout stages automatically.