scripte/rea
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into codex/remove-static-proxies

Browse Source
This commit is contained in:
morluto authored and GitHub committed 2026-07-23 00:43:10 +08:00
2 parents aeb5cb3adb c02d33f381
commit 77cac68ded
82 files changed
+4621 -346
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+3 -1
View File
@@ -27,7 +27,7 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
- `src/config.ts`: Zod-validated parsing of environment configuration into `AppConfig`.
- `src/domain/`: pure, side-effect-free modules. `errors.ts` owns the tagged error algebra; `result.ts` owns `Result`/`ok`/`err`; `hopperValues.ts` owns shared function-dossier values plus Hopper boundary parsers; `symbolAnalysis.ts` parses Swift/ObjC names; `javascriptApplicationGraph.ts` validates and canonically commits the provider-neutral JavaScript Application Graph; `javascriptStaticAnalysis.ts` performs bounded AST-only JavaScript structure recovery.
- `src/contracts/`: caller-visible tool schemas and catalog metadata; `toolContracts.ts` owns the canonical inventory and `enhancedInputs.ts` owns enhanced input parsing.
- `src/process/`: provider-neutral process ownership and lifecycle primitives. It owns private runtime roots, absolute startup deadlines, correlated request waits, bounded output capture, and TERM-to-KILL cleanup without defining any provider wire protocol.
- `src/process/`: provider-neutral process ownership and lifecycle primitives. It owns private runtime roots, session-assigned run identity and token-verified lineage, absolute startup deadlines, correlated request waits, bounded output capture, and TERM-to-KILL cleanup without defining any provider wire protocol.
- `src/replay/`: Linux x64 controlled-JavaScript-replay adapter. It owns exact runtime closure inspection, Bubblewrap/seccomp/cgroup admission, the disposable worker, strict parent/worker protocol validation, and complete cleanup observation.
- `src/browser/`: loopback CDP discovery, bounded WebSocket transport, exact-origin target authorization, and passive browser observation normalization.
- `src/hopper/`: Hopper launch and Unix-socket protocol mechanics. `BridgeLauncher.ts` spawns the Hopper app with the in-process bridge, `HopperClient.ts` correlates request/response over the socket with timeouts and cancellation, `protocol.ts` frames bridge messages.
@@ -74,6 +74,8 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
- `npm run verify:ghidra:windows`: build and run the real Windows Ghidra verifier against `GHIDRA_INSTALL_DIR` and the source-owned x64 PE fixture.
- `npm run verify:browser`: build and run the real Chrome verifier against `REA_BROWSER_EXECUTABLE` or a platform-default Chrome-family executable.
- `npm run verify:managed`: build and run the source-owned managed PE/CLI conformance verifier for artifact triage, member inspection, managed/native boundary declarations, token drift comparison, malformed metadata, non-managed degradation, managed application-graph projection, manifest-verifier self-test, and optional BYO ILSpy oracle checks; set `REA_MANAGED_APP_MANIFEST_PATH` to verify an operator-local managed app manifest, including optional graph/trace assertions, and set `REA_ILSPY_CMD_PATH` to an absolute `ilspycmd` path to run the real ILSpy reconstruction oracle.
- `npm run evidence:generate`: regenerate the managed conformance manifest and Evidence v2 completion ledger from live verifier output.
- `npm run evidence:check`: rerun managed conformance and fail on generated manifest, completion ledger, or bundled skill drift.
- `npm run verify:replay`: build and run the real Linux Bubblewrap/seccomp/cgroup verifier against source-owned replay fixtures; set `REA_REPLAY_INPUT_PATH` to verify an operator-local manifest.
- `npm run verify:package`: pack and test the CLI, setup transaction, skill, and canonical target-free MCP catalog in an isolated environment.
- `npm run docs:api`: render uncommitted API documentation from JSDoc comments into `build/api-docs/` using TypeDoc.
README.md
+36 -2
View File
@@ -584,8 +584,15 @@ operations still work. An explicit unknown, unavailable, or unsupported
provider fails with candidate IDs, stable rejection codes, and actionable local
diagnostics. `binary_session`, `rea providers`, and `rea capabilities` expose
the authoritative `analysis_provider_candidates` and
`analysis_provider_binding` fields. Reopening the same target without a selector
keeps its binding; runtime failure never selects another provider silently.
`analysis_provider_binding` fields. Each open target also has an
`analysis_run.run_id` allocated before provider startup. When dynamic providers
start, `analysis_run.process_lineage` changes from `not_observed` to `snapshots`
and retains one provider-attributed, token-verified observation per started
provider. Each observation carries `observed_at` and remains `unavailable` or
`verified`; a verified empty descendant list is distinct from both. Snapshots
describe bounded observations, not current live state or historical absence.
Reopening same target without a selector keeps its binding; runtime failure
never selects another provider silently.
Ghidra can appear as an available, target-compatible candidate after doctor
validates its exact installation. Its capability list contains the 18 admitted
read-only inventory and function-analysis operations; selecting it still does
@@ -724,6 +731,33 @@ and browser prompts through a real local Codex CLI. Its JSON report measures
natural MCP use, first-tool routing, repeated calls, actual Codex token usage,
completion quality, and explicit treatment of authority and unknowns.
`npm run evidence:generate` regenerates the managed conformance manifest and
Evidence v2 completion ledger from live verifier output. `npm run
evidence:check` reruns the verifier and fails when artifacts, scenarios,
providers, schemas, claim counts, Evidence IDs, or the bundled skill have
drifted. Unsupported claims remain explicit and never count as passes.
Verifier JSON reports include an ephemeral `verifier_run` UUID allocated before
the verifier performs work and inherited by its child processes through
`REA_PROCESS_RUN_ID`. The final report includes the verifier and parent PIDs
plus `process_lineage` and its ISO `observed_at` timestamp: POSIX verifiers
report a token-verified, point-in-time launcher process group and live
descendants, while platforms without an owned
lineage primitive report `status: "unavailable"` and a reason. An empty verified
descendant list means no child was live during the final observation; it does
not claim the verifier launched no children earlier. Nested verifier entrypoints
in the same process reuse its UUID; each new verifier process replaces any
inherited parent token with a fresh run identity.
Generated completion commitments deliberately exclude this per-execution
identity, so check mode remains deterministic while live reports remain
attributable.
Owned Hopper shutdown logs retain the launcher PID, process-group ID, cleanup
status, and whether a verified group signal was required. They omit the run
token and unexpected exception text. The real-Hopper verifier also starts an
unrelated `Hopper`-named sentinel in a distinct process group and fails unless
that process survives every session close; the sentinel is removed only by the
verifier after the survival check.
## Project links
[npm](https://www.npmjs.com/package/rea-agents) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md) · [Hopper](https://www.hopperapp.com/) · [Ghidra](https://github.com/NationalSecurityAgency/ghidra)
docs/adr/0001-provider-selection-and-analysis-profiles.md
+11
View File
@@ -396,6 +396,17 @@ selection source, and complete profile commitment. Every candidate reports
selected. Candidate capability descriptors preserve supported, unsupported,
and degraded truth independently.
An open target also exposes `analysis_run`. Its run ID is allocated by the
session before client construction and propagated into selected Hopper or
Ghidra bridge handshake and process environment. Dynamic clients retain
provider-attributed lineage snapshots only after revalidating launcher
parent/command and every observed launcher or descendant run token. `not_observed`
means no dynamic provider has supplied a snapshot. Once one or more start,
`snapshots` preserves every provider observation; each remains explicitly
`unavailable` or `verified` and records its `observed_at` time. Retained
snapshots are not live inventories, and a point-in-time observation never proves
historical absence.
The existing flat `provider`, `providers`, and `capabilities` status fields stay
during the 1.x compatibility window. Their effective deep-operation entries
come only from the active binding; target-free ambiguity is unavailable rather
docs/mcp-contracts.md
+9
View File
@@ -27,6 +27,15 @@ concrete version, selection source, and complete analysis profile through
choices return typed selection details. A selected provider is never replaced
automatically after a runtime failure.
Every successful target transition allocates `binary_session.analysis_run.run_id`
before any provider startup. `process_lineage` is `not_observed` until a dynamic
provider starts, then becomes `snapshots` with every started provider's identity
and retained ownership observation. Each observation records `observed_at` and
is `unavailable` with a reason when ownership could not be revalidated, or
`verified` with launcher PID, parent PID, process group, and descendants observed
at that bounded check. These are historical snapshots, not live process
inventories, and do not claim that no short-lived descendant existed.
## Progress and cancellation
REA accepts ordinary `tools/call` progress tokens. Updates are monotonic,
docs/product-catalog.json
+10 -2
View File
@@ -404,6 +404,14 @@
"id": "evidence_bundle",
"version": 2
},
{
"id": "evidence_completion_ledger",
"version": 1
},
{
"id": "evidence_completion_manifest",
"version": 1
},
{
"id": "investigation_run",
"version": 1
@@ -569,10 +577,10 @@
"mcp_resource_templates": 9
},
"digests": {
"tools_sha256": "4dca2b113556f78ee65487810f9462d7523c8828a8f642b84b5e74829df84820",
"tools_sha256": "937d58e32d18f48970d26219d791838087ae71ac2893950db33bdf2d3bd51bc0",
"prompts_sha256": "f6da3ad38a1000c11364d1710f122ac4990551f04efd8c53ffb2e819b5cf5707",
"resources_sha256": "94bebbab1a1b3cf47644c38d3de6469de568caa7bd7b842f529ead8ad320a912",
"combined_sha256": "64387dcbe22c395e8b683344e5bfd2881e3e88c941b053723165f131c57ed46b"
"combined_sha256": "92837c2e99c99a2dd087e4fb01d210cce9b5948051c33db1a404348ef90d8bf2"
}
}
}
docs/verification/managed-conformance-ledger.json
+113
View File
@@ -0,0 +1,113 @@
{
"schema_version": 1,
"ledger_id": "ecl_a56a75b91375c9c9501d1545fee9977e74f57dbd1bf413a370f9cd1ed26987bd",
"records": [
{
"claim_id": "managed.application-graph-projection",
"status": "pass",
"evidence_ids": [
"ev_4d6821cdd212a77507dfbdf995fc8b086cd63b3f1d39dee4a2e12f62a6f0609d"
]
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"status": "pass",
"evidence_ids": [
"ev_cd9015f39a60251bee1b01baf5189de7427e5810f130aed39770f17ae69cee44"
]
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"status": "pass",
"evidence_ids": [
"ev_af02c0218a01138bd1f0f5ca052f95d348dfacb540040ab72d6ddc9889e3a7e4"
]
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"status": "unsupported",
"evidence_ids": [
"ev_5df12074aeb26d67bc954fd9230076bd0c3ce9969600e19c59ba02090a9988f2"
]
},
{
"claim_id": "managed.malformed-metadata",
"status": "pass",
"evidence_ids": [
"ev_5d6aadf199ae090e0ddebc10a51aab8a410fa778a6ae8454fedfb10d240cdac2"
]
},
{
"claim_id": "managed.managed-native-verification",
"status": "pass",
"evidence_ids": [
"ev_9f1862276fe8869b86105eb91154603922eeb95fec33a10c354b2d6c9c3fe077"
]
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"status": "pass",
"evidence_ids": [
"ev_c9aef74724cf555bcedce8a9dbf46327b2f92b95b03eb363f966885d429e6882"
]
},
{
"claim_id": "managed.mvid-and-token-drift",
"status": "pass",
"evidence_ids": [
"ev_55808bf3ee5ec9d87cee751462077a6bce88701c4fcf66822ba1d7d33706706b"
]
},
{
"claim_id": "managed.not-managed",
"status": "pass",
"evidence_ids": [
"ev_a8166506ca91a4595b825f355836bb4becf35852c8547c55408c09698f4953a6"
]
},
{
"claim_id": "managed.operator-local-manifest",
"status": "unsupported",
"evidence_ids": [
"ev_03d1a151446834fbee7c093e82121b714a0cc1e6167ad84f47a79ffb9e2602ce"
]
},
{
"claim_id": "managed.operator-manifest-self-test",
"status": "pass",
"evidence_ids": [
"ev_74d7f9c75a4476196664542d790e2532435f2a1c54349b481967e51e2ccc9453"
]
},
{
"claim_id": "managed.runtime-correlation-admission-plan",
"status": "pass",
"evidence_ids": [
"ev_5a80df0b92dc2280361e900ce590bc88ea52eba3dc8e216a0ce282f503e3cb05"
]
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"status": "pass",
"evidence_ids": [
"ev_3b61c79ec152c70693278a70ad2ca6528ce0a9c9ab389d3ec9e4520de79082a7"
]
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"status": "pass",
"evidence_ids": [
"ev_81f41b1b3809c2fe1964f7abbcbd2faeb14c723938639c2e0bb16cd58746aa1a"
]
}
],
"summary": {
"total": 14,
"pass": 12,
"fail": 0,
"unsupported": 2,
"truncated": 0,
"unknown": 0,
"complete": false
}
}
docs/verification/managed-conformance-manifest.json
+230
View File
@@ -0,0 +1,230 @@
{
"schema_version": 1,
"manifest_id": "ecm_87baf2161a85728dd5f8bf3c9d55a4dff6e96b26ccf229363a6cda8ee78c7d24",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
},
"environment": {
"platform": "linux",
"architecture": "x64",
"runtime": "node",
"runtime_version": "24"
},
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "f522ec8b7f29bfdb2c1d8ed9ffe9d550b38474bff6a5cd47f0ddb08882450c11"
}
],
"claims": [
{
"claim_id": "managed.application-graph-projection",
"scenario": {
"id": "managed-application-graph-projection",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"scenario": {
"id": "decompiler-reconstruction-import",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"scenario": {
"id": "dotnet-framework-x86-pinvoke",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"scenario": {
"id": "ilspy-reconstruction-oracle",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.malformed-metadata",
"scenario": {
"id": "malformed-metadata",
"version": 1
},
"artifact_sha256s": [
"827e10b6391d29edca48c191e2ddd79f3bc1165e46e4ba2f4edf41ab60f45e78"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.managed-native-verification",
"scenario": {
"id": "managed-native-verification",
"version": 1
},
"artifact_sha256s": [
"9999999999999999999999999999999999999999999999999999999999999999",
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"scenario": {
"id": "modern-dotnet-anycpu",
"version": 1
},
"artifact_sha256s": [
"92b588ae43d418117fa108c0004b5d0451f6112238d21bad8d75c0eb1da860f0"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.mvid-and-token-drift",
"scenario": {
"id": "mvid-and-token-drift",
"version": 1
},
"artifact_sha256s": [
"276395c671a656ab57d583158443c10de552dccfb37dc97cd013f673c0f86b43",
"ef009dab7424fdeba996f2524dbe86fd899c96394498cca0463fcfdb228c5572"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.not-managed",
"scenario": {
"id": "not-managed",
"version": 1
},
"artifact_sha256s": [
"de74dab8ab034054a811d4248efcda0bed505187a061a806e274ae87bc453754"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.operator-local-manifest",
"scenario": {
"id": "operator-local-manifest",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.operator-manifest-self-test",
"scenario": {
"id": "operator-manifest-self-test",
"version": 1
},
"artifact_sha256s": [
"cdc11ec7fd0f392e43b9b23971cb41ec49fd0a997fc33c24fcc553cd4386ae37"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.runtime-correlation-admission-plan",
"scenario": {
"id": "runtime-correlation-admission-plan",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"scenario": {
"id": "unicode-obfuscated-identifiers",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"scenario": {
"id": "x64-ready-to-run-native-body",
"version": 1
},
"artifact_sha256s": [
"e47402428322cdcb06f561036d7d108528cbc6e262d92a21f869218a20c80bfb"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
},
"result_schema_version": 1
}
]
}
knip.json
+2
View File
@@ -3,6 +3,8 @@
"entry": ["src/main.ts", "src/cli.ts"],
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],
"ignoreDependencies": [],
"ignoreBinaries": ["ps"],
"ignoreExportsUsedInFile": true,
"rules": {
"files": "error",
"dependencies": "error",
package.json
+4 -2
View File
@@ -110,10 +110,12 @@
"probe:windows-capabilities": "npm run build:cached && node scripts/windows-capabilities.mjs",
"verify:published": "node scripts/verify-published-package.mjs",
"config:print": "node scripts/print-mcp-config.mjs",
"evidence:generate": "npm run build:cached && node scripts/generate-completion-ledger.mjs",
"evidence:check": "npm run build:cached && node scripts/generate-completion-ledger.mjs --check",
"docs:api": "typedoc",
"docs:generate": "npm run metadata:generate && npm run build:cached && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && npm run docs:api && node scripts/generate-error-schema.mjs",
"docs:generate": "npm run metadata:generate && npm run build:cached && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && node scripts/generate-completion-ledger.mjs && npm run docs:api && node scripts/generate-error-schema.mjs",
"docs:check": "turbo run docs:check:built",
"docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/check-doc-facts.mjs && npm run docs:api && node scripts/generate-error-schema.mjs --check",
"docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && npm run docs:api && node scripts/generate-error-schema.mjs --check",
"docs:error-schema": "npm run build:cached && node scripts/generate-error-schema.mjs",
"prepack": "node scripts/prepack.mjs",
"prepublishOnly": "npm run deps:check && node scripts/generate-skill-metadata.mjs --check",
scripts/generate-completion-ledger.mjs
+97
View File
@@ -0,0 +1,97 @@
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { mkdir, readFile, readdir } from "node:fs/promises";
import { dirname, join, relative, sep } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { ensureGeneratedFile } from "./lib/generated-file.mjs";
const exec = promisify(execFile);
const arguments_ = new Set(process.argv.slice(2));
for (const argument of arguments_)
if (argument !== "--check")
throw new Error(`Unknown completion ledger option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const outputRoot = join(root, "docs/verification");
const check = arguments_.has("--check");
const verifierEnvironment = { ...process.env };
delete verifierEnvironment.REA_MANAGED_APP_MANIFEST_PATH;
delete verifierEnvironment.REA_ILSPY_CMD_PATH;
const { stdout } = await exec(
process.execPath,
[join(root, "scripts/verify-managed-conformance.mjs")],
{
cwd: root,
env: verifierEnvironment,
maxBuffer: 20 * 1024 * 1024,
},
);
const verifierOutput = JSON.parse(stdout);
const { createCompletionLedgerArtifacts } = await import(
`../dist/domain/completionLedgerGeneration.js?${String(Date.now())}`
);
const skillDigest = await directoryDigest(
join(root, "skills/reverse-engineer-anything"),
);
const generated = createCompletionLedgerArtifacts(
verifierOutput.completionReport,
[{ skill_id: "reverse-engineer-anything", sha256: skillDigest }],
);
if (!check) await mkdir(outputRoot, { recursive: true });
await Promise.all([
ensureGeneratedFile({
path: join(outputRoot, "managed-conformance-manifest.json"),
source: `${JSON.stringify(generated.manifest, null, 2)}\n`,
check,
generateCommand: "npm run evidence:generate",
}),
ensureGeneratedFile({
path: join(outputRoot, "managed-conformance-ledger.json"),
source: `${JSON.stringify(generated.ledger, null, 2)}\n`,
check,
generateCommand: "npm run evidence:generate",
}),
]);
process.stdout.write(
`${JSON.stringify({
manifest_id: generated.manifest.manifest_id,
ledger_id: generated.ledger.ledger_id,
claims: generated.ledger.summary.total,
complete: generated.ledger.summary.complete,
})}\n`,
);
async function directoryDigest(directory) {
const paths = await filePaths(directory);
const records = await Promise.all(
paths.map(async (path) => {
const name = relative(directory, path).split(sep).join("/");
const digest = createHash("sha256")
.update(await readFile(path))
.digest("hex");
return `${name}\0${digest}\n`;
}),
);
return createHash("sha256").update(records.join("")).digest("hex");
}
async function filePaths(directory) {
const entries = await readdir(directory, { withFileTypes: true });
const paths = [];
for (const entry of entries.sort((left, right) =>
left.name < right.name ? -1 : left.name > right.name ? 1 : 0,
)) {
const path = join(directory, entry.name);
if (entry.isDirectory()) paths.push(...(await filePaths(path)));
else if (entry.isFile()) paths.push(path);
else
throw new Error(
`Completion skill digest does not admit ${entry.name}: expected a regular file or directory`,
);
}
return paths;
}
scripts/lib/catalog-core.mjs
+2
View File
@@ -42,6 +42,8 @@ const SOURCE_PATHS = {
electronProvider: "dist/browser/CdpElectronProvider.js",
evidence: "dist/domain/evidence.js",
evidenceBundle: "dist/domain/evidenceBundle.js",
evidenceCompletion: "dist/domain/evidenceCompletionLedger.js",
completionGeneration: "dist/domain/completionLedgerGeneration.js",
processCapture: "dist/domain/processCapture.js",
analysisSnapshot: "dist/domain/analysisSnapshot.js",
artifactGraph: "dist/domain/artifactGraph.js",
scripts/lib/catalog-schemas.mjs
+10
View File
@@ -56,6 +56,16 @@ const durableSchemaDefinitions = (sources) => [
sources.evidenceBundle.evidenceBundleSchema,
["bundle_version"],
],
[
"evidence_completion_ledger",
sources.evidenceCompletion.evidenceCompletionLedgerSchema,
["schema_version"],
],
[
"evidence_completion_manifest",
sources.completionGeneration.completionManifestSchema,
["schema_version"],
],
[
"investigation_run",
sources.investigationWorkspace.investigationRunSchema,
scripts/lib/managed-completion-report.mjs
+167
View File
@@ -0,0 +1,167 @@
import { createEvidence } from "../../dist/domain/evidence.js";
const COMPLETION_PROVIDER = {
id: "rea-managed-conformance",
name: "REA managed conformance verifier",
version: "1",
};
const completionClaim = ({
claimId,
scenarioId,
targets = [],
status = "pass",
limitations = [],
}) => {
const artifactSha256s = [
...new Set(targets.map(({ sha256: digest }) => digest)),
].sort();
const evidence = createEvidence(targets[0], COMPLETION_PROVIDER, {
predicateType: "rea.verification/v1",
operation: "verify_managed_conformance",
parameters: {
scenario_id: scenarioId,
scenario_version: 1,
result_schema_version: 1,
artifact_sha256s: artifactSha256s,
},
result: { status },
rawResult: null,
limitations,
});
return {
claim_id: claimId,
scenario: { id: scenarioId, version: 1 },
artifact_sha256s: artifactSha256s,
provider: {
id: COMPLETION_PROVIDER.id,
version: COMPLETION_PROVIDER.version,
},
result_schema_version: 1,
status,
evidence_ids: [evidence.evidence_id],
};
};
const optionalTarget = (target) =>
target === undefined
? []
: [{ path: target.file_name, sha256: target.sha256, format: "pe" }];
const sourceOwnedClaims = ({
modern,
framework,
nativeFunctionTarget,
readyToRun,
obfuscated,
left,
right,
nativeOnly,
malformed,
manifestSelfTest,
}) => [
completionClaim({
claimId: "managed.modern-dotnet-anycpu",
scenarioId: "modern-dotnet-anycpu",
targets: [modern.target],
}),
completionClaim({
claimId: "managed.dotnet-framework-x86-pinvoke",
scenarioId: "dotnet-framework-x86-pinvoke",
targets: [framework.target],
}),
completionClaim({
claimId: "managed.x64-ready-to-run-native-body",
scenarioId: "x64-ready-to-run-native-body",
targets: [readyToRun.target],
}),
completionClaim({
claimId: "managed.managed-native-verification",
scenarioId: "managed-native-verification",
targets: [framework.target, nativeFunctionTarget],
}),
completionClaim({
claimId: "managed.application-graph-projection",
scenarioId: "managed-application-graph-projection",
targets: [framework.target],
}),
completionClaim({
claimId: "managed.unicode-obfuscated-identifiers",
scenarioId: "unicode-obfuscated-identifiers",
targets: [obfuscated.target],
}),
completionClaim({
claimId: "managed.decompiler-reconstruction-import",
scenarioId: "decompiler-reconstruction-import",
targets: [obfuscated.target],
}),
completionClaim({
claimId: "managed.runtime-correlation-admission-plan",
scenarioId: "runtime-correlation-admission-plan",
targets: [obfuscated.target],
}),
completionClaim({
claimId: "managed.mvid-and-token-drift",
scenarioId: "mvid-and-token-drift",
targets: [left.target, right.target],
}),
completionClaim({
claimId: "managed.not-managed",
scenarioId: "not-managed",
targets: [nativeOnly.target],
}),
completionClaim({
claimId: "managed.malformed-metadata",
scenarioId: "malformed-metadata",
targets: [malformed.target],
}),
completionClaim({
claimId: "managed.operator-manifest-self-test",
scenarioId: "operator-manifest-self-test",
targets: [
{
path: manifestSelfTest.target.file_name,
sha256: manifestSelfTest.target.sha256,
format: "pe",
},
],
}),
];
const optionalClaims = ({ operatorManifest, ilspyOracle }) => [
completionClaim({
claimId: "managed.operator-local-manifest",
scenarioId: "operator-local-manifest",
targets: optionalTarget(operatorManifest?.target),
status: operatorManifest === null ? "unsupported" : "pass",
limitations:
operatorManifest === null
? ["REA_MANAGED_APP_MANIFEST_PATH is not configured."]
: [],
}),
completionClaim({
claimId: "managed.ilspy-reconstruction-oracle",
scenarioId: "ilspy-reconstruction-oracle",
targets: optionalTarget(ilspyOracle?.fixture),
status: ilspyOracle === null ? "unsupported" : "pass",
limitations:
ilspyOracle === null ? ["REA_ILSPY_CMD_PATH is not configured."] : [],
}),
];
/** Project completed managed assertions into the standard verifier report. */
export const createManagedCompletionReport = (input, verifierRun) => ({
schema_version: 1,
verifier: {
id: COMPLETION_PROVIDER.id,
version: COMPLETION_PROVIDER.version,
},
verifier_run: verifierRun,
environment: {
platform: process.platform,
architecture: process.arch,
runtime: "node",
runtime_version: process.versions.node.split(".")[0],
},
claims: [...sourceOwnedClaims(input), ...optionalClaims(input)],
});
scripts/lib/unrelated-hopper-sentinel.mjs
+60
View File
@@ -0,0 +1,60 @@
import { spawn } from "node:child_process";
/** Start a source-owned decoy whose process title exercises broad Hopper kills. */
export const startUnrelatedHopperSentinel = async () => {
const environment = { ...process.env };
delete environment.REA_PROCESS_RUN_ID;
const child = spawn(
process.execPath,
["-e", "process.title = 'Hopper'; setInterval(() => undefined, 1000)"],
{
detached: true,
stdio: "ignore",
env: environment,
},
);
await new Promise((resolve, reject) => {
child.once("spawn", resolve);
child.once("error", reject);
});
await new Promise((resolve) => setTimeout(resolve, 25));
const assertAlive = () => {
const pid = child.pid;
if (
pid === undefined ||
child.exitCode !== null ||
child.signalCode !== null
) {
throw new Error(
"Unrelated Hopper sentinel was terminated by verification",
);
}
try {
process.kill(pid, 0);
} catch {
throw new Error(
"Unrelated Hopper sentinel was terminated by verification",
);
}
};
const close = async () => {
if (child.exitCode !== null || child.signalCode !== null) return;
const exited = new Promise((resolve) => child.once("exit", resolve));
child.kill("SIGKILL");
await exited;
};
const verifyAndClose = async () => {
try {
assertAlive();
return true;
} finally {
await close();
}
};
assertAlive();
return { assertAlive, close, verifyAndClose };
};
scripts/lib/verifier-run.d.mts
+43
View File
@@ -0,0 +1,43 @@
export interface VerifierRunStart {
readonly schema_version: 1;
readonly run_id: string;
readonly verifier_pid: number;
readonly parent_pid: number;
}
export interface VerifierRunDescendant {
readonly pid: number;
readonly parent_pid: number;
readonly process_group_id: number;
}
export type VerifierProcessLineage =
| {
readonly status: "verified";
readonly schema_version: 1;
readonly observed_at: string;
readonly launcher_pid: number;
readonly launcher_parent_pid: number;
readonly process_group_id: number;
readonly descendants: readonly VerifierRunDescendant[];
}
| {
readonly status: "unavailable";
readonly observed_at: string;
readonly launcher_pid: number;
readonly launcher_parent_pid: number;
readonly process_group_id: number | null;
readonly reason: string;
};
export interface VerifierRun extends VerifierRunStart {
readonly process_lineage: VerifierProcessLineage;
}
/** Allocate one process-local identity and propagate it to child processes. */
export function createVerifierRun(): VerifierRunStart;
/** Complete a verifier report with token-verified point-in-time lineage. */
export function completeVerifierRun(
run: VerifierRunStart,
): Promise<VerifierRun>;
scripts/lib/verifier-run.mjs
+206
View File
@@ -0,0 +1,206 @@
import { spawn } from "node:child_process";
import { randomUUID } from "node:crypto";
import { readFile, readdir } from "node:fs/promises";
let activeVerifierRun;
/** Allocate one verifier identity before work and propagate it to child processes. */
export const createVerifierRun = () => {
if (activeVerifierRun !== undefined) {
process.env.REA_PROCESS_RUN_ID = activeVerifierRun.run_id;
return activeVerifierRun;
}
activeVerifierRun = Object.freeze({
schema_version: 1,
run_id: randomUUID(),
verifier_pid: process.pid,
parent_pid: process.ppid,
});
process.env.REA_PROCESS_RUN_ID = activeVerifierRun.run_id;
return activeVerifierRun;
};
/**
* Complete a verifier report with a token-verified point-in-time lineage.
*
* An empty descendant list means none were live during this observation. It
* does not prove that the verifier never launched children earlier in the run.
*/
export const completeVerifierRun = async (run) => {
const observedAt = new Date().toISOString();
return {
...run,
process_lineage: await observeVerifierLineage(run, observedAt),
};
};
const observeVerifierLineage = async (run, observedAt) => {
if (process.platform === "win32")
return unavailableLineage(
run,
observedAt,
null,
"Windows verifier lineage ownership is unavailable without Job Objects",
);
try {
const { members, observerPid } = await processSnapshot();
const launcher = members.find(({ pid }) => pid === run.verifier_pid);
if (launcher === undefined)
return unavailableLineage(
run,
observedAt,
null,
"verifier launcher was absent from the process snapshot",
);
if (launcher.parentPid !== run.parent_pid)
return unavailableLineage(
run,
observedAt,
launcher.processGroupId,
"verifier launcher parent changed before lineage observation",
);
const descendants = descendantMembers(
members.filter(({ pid }) => pid !== observerPid),
run.verifier_pid,
);
for (const member of descendants) {
let memberRunId;
try {
memberRunId = await processRunId(member.pid);
} catch {
return unavailableLineage(
run,
observedAt,
launcher.processGroupId,
`descendant ${String(member.pid)} environment was unreadable`,
);
}
if (memberRunId !== run.run_id)
return unavailableLineage(
run,
observedAt,
launcher.processGroupId,
`descendant ${String(member.pid)} run token did not match`,
);
}
return {
status: "verified",
schema_version: 1,
observed_at: observedAt,
launcher_pid: run.verifier_pid,
launcher_parent_pid: run.parent_pid,
process_group_id: launcher.processGroupId,
descendants: descendants
.map(({ pid, parentPid, processGroupId }) => ({
pid,
parent_pid: parentPid,
process_group_id: processGroupId,
}))
.sort((left, right) => left.pid - right.pid),
};
} catch {
return unavailableLineage(
run,
observedAt,
null,
"operating-system process lineage could not be inspected",
);
}
};
const unavailableLineage = (run, observedAt, processGroupId, reason) => ({
status: "unavailable",
observed_at: observedAt,
launcher_pid: run.verifier_pid,
launcher_parent_pid: run.parent_pid,
process_group_id: processGroupId,
reason,
});
const processSnapshot = async () => {
if (process.platform === "linux") return linuxProcessSnapshot();
return posixProcessSnapshot();
};
const linuxProcessSnapshot = async () => {
const entries = await readdir("/proc", { withFileTypes: true });
const members = [];
for (const entry of entries) {
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
try {
const stat = await readFile(`/proc/${entry.name}/stat`, "utf8");
const match = /^\d+ \(.*\) (\S) (\d+) (\d+) /u.exec(stat);
if (match === null || match[1] === "Z") continue;
members.push({
pid: Number(entry.name),
parentPid: Number(match[2]),
processGroupId: Number(match[3]),
});
} catch {
// Processes can exit while /proc is being sampled.
}
}
return { members, observerPid: null };
};
const posixProcessSnapshot = async () => {
const child = spawn("ps", ["-axo", "pid=,ppid=,pgid=,stat="], {
stdio: ["ignore", "pipe", "pipe"],
});
const stdout = [];
const stderr = [];
child.stdout.on("data", (chunk) => stdout.push(chunk));
child.stderr.on("data", (chunk) => stderr.push(chunk));
const code = await new Promise((resolve, reject) => {
child.once("error", reject);
child.once("close", resolve);
});
if (code !== 0)
throw new Error(`ps failed: ${Buffer.concat(stderr).toString("utf8")}`);
const members = Buffer.concat(stdout)
.toString("utf8")
.split("\n")
.map((line) => /\s*(\d+)\s+(\d+)\s+(\d+)\s+(\S+)/u.exec(line))
.filter((match) => match !== null && !match[4]?.startsWith("Z"))
.map((match) => ({
pid: Number(match[1]),
parentPid: Number(match[2]),
processGroupId: Number(match[3]),
}));
return { members, observerPid: child.pid ?? null };
};
const descendantMembers = (members, launcherPid) => {
const descendantPids = new Set([launcherPid]);
let previousSize = -1;
while (previousSize !== descendantPids.size) {
previousSize = descendantPids.size;
for (const member of members)
if (descendantPids.has(member.parentPid)) descendantPids.add(member.pid);
}
descendantPids.delete(launcherPid);
return members.filter(({ pid }) => descendantPids.has(pid));
};
const processRunId = async (pid) => {
if (process.platform === "linux") {
const environment = await readFile(`/proc/${String(pid)}/environ`, "utf8");
return environment
.split("\0")
.find((entry) => entry.startsWith("REA_PROCESS_RUN_ID="))
?.slice("REA_PROCESS_RUN_ID=".length);
}
const child = spawn("ps", ["eww", "-p", String(pid)], {
stdio: ["ignore", "pipe", "ignore"],
});
const stdout = [];
child.stdout.on("data", (chunk) => stdout.push(chunk));
const code = await new Promise((resolve, reject) => {
child.once("error", reject);
child.once("close", resolve);
});
if (code !== 0) throw new Error("process environment unavailable");
return /(?:^|\s)REA_PROCESS_RUN_ID=([^\s]*)/u.exec(
Buffer.concat(stdout).toString("utf8"),
)?.[1];
};
scripts/verify-agent-experience.mjs
+3
View File
@@ -8,8 +8,10 @@ import { fileURLToPath } from "node:url";
import { createPackage } from "@electron/asar";
import { evaluateCodexEvents } from "../dist/evaluation/CodexAgentEval.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const verifierRun = createVerifierRun();
const evaluationRoot = await mkdtemp(join(tmpdir(), "rea-agent-eval-"));
const fixtureRoot = join(evaluationRoot, "targets");
const skillDestination = join(
@@ -152,6 +154,7 @@ try {
}
const summary = {
verifier_run: await completeVerifierRun(verifierRun),
schemaVersion: 1,
codex,
codexVersion,
scripts/verify-conformance-fixtures.mjs
+3 -1
View File
@@ -8,8 +8,10 @@ import {
sha256,
sourceDigest,
} from "./lib/conformance-fixtures.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const exec = promisify(execFile);
const verifierRun = createVerifierRun();
const manifestPath = await realpath(
resolve(process.argv[2] ?? "build/conformance/manifest.json"),
);
@@ -37,7 +39,7 @@ if (v1 === undefined || v2 === undefined)
if (v1.artifactSha256 === v2.artifactSha256)
throw new Error("Version-pair artifacts unexpectedly have equal hashes");
process.stdout.write(
`${JSON.stringify({ verified: manifest.fixtures.length, manifestPath, platform: manifest.platform })}\n`,
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), verified: manifest.fixtures.length, manifestPath, platform: manifest.platform })}\n`,
);
async function verifyFixture(fixture) {
scripts/verify-linux-setup-e2e.mjs
+4
View File
@@ -11,7 +11,10 @@ import { join } from "node:path";
import { tmpdir } from "node:os";
import { promisify } from "node:util";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
const command = process.env.REA_VERIFY_SERVER_COMMAND;
if (process.platform !== "linux" || process.getuid?.() !== 0)
throw new Error("Linux setup verification requires a root Linux runner");
@@ -113,6 +116,7 @@ if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(processes))
console.log(
JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
status: "verified",
setup: true,
cli: true,
scripts/verify-local-application-workflows.mjs
+4
View File
@@ -14,6 +14,9 @@ import { applicationVersionComparisonResultSchema } from "../dist/domain/javascr
import { applicationFeatureTraceResultSchema } from "../dist/domain/javascriptFeatureTraceSchemas.js";
import { javaScriptExportShapeComparisonResultSchema } from "../dist/domain/javascriptExportShapeComparisonSchemas.js";
import { createPermissionPolicy } from "../dist/domain/permissionPolicy.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const options = parseArgs({
options: {
@@ -94,6 +97,7 @@ const exportShapeResult =
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
comparison_id: comparisonResult.comparison_id,
evidence_id: comparison.value.evidence_id,
left: comparisonResult.left,
scripts/verify-local-javascript-runtime.mjs
+4
View File
@@ -7,6 +7,9 @@ import { analyzeJavaScriptApplication } from "../dist/application/JavaScriptAppl
import { PermissionAuthority } from "../dist/application/PermissionAuthority.js";
import { reconcileJavaScriptRuntimeEvidence } from "../dist/application/JavaScriptRuntimeReconciliationService.js";
import { createPermissionPolicy } from "../dist/domain/permissionPolicy.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const options = parseArgs({
options: {
@@ -71,6 +74,7 @@ const result = reconciled.value.normalized_result;
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
reconciliation_id: result.reconciliation_id,
evidence_id: reconciled.value.evidence_id,
graph_id: result.graph.graph_id,
scripts/verify-managed-conformance.mjs
+28 -6
View File
@@ -29,7 +29,10 @@ import {
memberLimits,
nativeBoundaryLimits,
} from "./lib/managed-conformance-config.mjs";
import { createManagedCompletionReport } from "./lib/managed-completion-report.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const workspace = await mkdtemp(join(tmpdir(), "rea-managed-conformance-"));
const {
fixture,
@@ -145,13 +148,14 @@ try {
locations: [{ kind: "artifact-path", path: framework.target.path }],
},
);
const nativeFunctionTarget = {
path: "/system/user32.dll",
sha256: "9".repeat(64),
format: "pe",
architecture: "x86",
};
const nativeFunctionEvidence = createEvidence(
{
path: "/system/user32.dll",
sha256: "9".repeat(64),
format: "pe",
architecture: "x86",
},
nativeFunctionTarget,
{
id: "ghidra",
name: "Ghidra",
@@ -437,6 +441,23 @@ try {
const manifestSelfTest = await runManagedAppManifestSelfTest();
const operatorManifest = await runOptionalManagedAppManifest();
const ilspyOracle = await runOptionalIlspyOracle();
const completionReport = createManagedCompletionReport(
{
modern,
framework,
nativeFunctionTarget,
readyToRun,
obfuscated,
left,
right,
nativeOnly,
malformed,
manifestSelfTest,
operatorManifest,
ilspyOracle,
},
await completeVerifierRun(verifierRun),
);
process.stdout.write(
`${JSON.stringify({
@@ -478,6 +499,7 @@ try {
env: "REA_ILSPY_CMD_PATH",
configured: false,
},
completionReport,
})}\n`,
);
} finally {
scripts/verify-package.mjs
+3 -1
View File
@@ -18,8 +18,10 @@ import { verifyPackagePlatform } from "./verify-package-platform.mjs";
import { verifyPackageSetup } from "./verify-package-setup.mjs";
import { verifyManaged } from "./verify-package-managed.mjs";
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const root = process.cwd();
const verifierRun = createVerifierRun();
const workspace = await mkdtemp(join(tmpdir(), "rea-package-"));
const evidenceRoot = join(workspace, "evidence");
const referenceRoot = join(workspace, "reference-source");
@@ -100,7 +102,7 @@ try {
investigationReplay,
});
process.stdout.write(
`${JSON.stringify({ cli: true, analysisCli: true, artifactCli: true, managedCli: true, managedReconstructionCli: true, managedNativeVerificationCli: true, managedRuntimePlanCli: true, managedApplicationGraphCli: true, evidenceCli: true, incurMcpCommand: PRODUCT_IDENTITY.mcpCommand, lifecycleScriptsRequired: false, doctor: "platform-appropriate", setup: supportedSetupHost ? "planned-then-idempotent" : "unsupported-host-rejected", setupPlanReadOnly: supportedSetupHost, existingHopperPreserved: supportedSetupHost, clients: supportedSetupHost ? 3 : 0, backupReadback: supportedSetupHost, failureRecovery: supportedSetupHost, configSymlinkLifecycle: supportedSetupHost, skill: supportedSetupHost, skillReferences: supportedSetupHost, mcpTools: TOOL_CONTRACTS.length, mcpPrompts: prompts.names.length, promptCompletion: true, promptCompletionLifecycle: true, evidenceMcp: true, targetFree: true, targetLifecycle: true, boundedRegexBridge: true })}\n`,
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), cli: true, analysisCli: true, artifactCli: true, managedCli: true, managedReconstructionCli: true, managedNativeVerificationCli: true, managedRuntimePlanCli: true, managedApplicationGraphCli: true, evidenceCli: true, incurMcpCommand: PRODUCT_IDENTITY.mcpCommand, lifecycleScriptsRequired: false, doctor: "platform-appropriate", setup: supportedSetupHost ? "planned-then-idempotent" : "unsupported-host-rejected", setupPlanReadOnly: supportedSetupHost, existingHopperPreserved: supportedSetupHost, clients: supportedSetupHost ? 3 : 0, backupReadback: supportedSetupHost, failureRecovery: supportedSetupHost, configSymlinkLifecycle: supportedSetupHost, skill: supportedSetupHost, skillReferences: supportedSetupHost, mcpTools: TOOL_CONTRACTS.length, mcpPrompts: prompts.names.length, promptCompletion: true, promptCompletionLifecycle: true, evidenceMcp: true, targetFree: true, targetLifecycle: true, boundedRegexBridge: true })}\n`,
);
} finally {
if (tarball) await rm(join(root, tarball), { force: true });
scripts/verify-published-package.mjs
+3 -1
View File
@@ -5,7 +5,9 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const version = process.argv[2];
if (!/^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/u.test(version ?? ""))
throw new Error("Usage: node scripts/verify-published-package.mjs <version>");
@@ -75,5 +77,5 @@ try {
}
process.stdout.write(
`${JSON.stringify({ package: "rea-agents", version, mcpTools: publishedToolCount, canonicalMcpTools: CATALOG_IDENTITY.counts.mcp_tools })}\n`,
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), package: "rea-agents", version, mcpTools: publishedToolCount, canonicalMcpTools: CATALOG_IDENTITY.counts.mcp_tools })}\n`,
);
scripts/verify-real-browser.mjs
+3
View File
@@ -21,6 +21,7 @@ import {
import { startBrowserVerifierSite } from "./fixtures/browser-verifier-site.mjs";
import { startPageCdpProxy } from "./fixtures/page-cdp-proxy.mjs";
import { sourceMapSummary } from "./lib/browser-verifier-assertions.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const SECRET_VALUES = [
"network-secret-value",
@@ -29,6 +30,7 @@ const SECRET_VALUES = [
"websocket-secret-value",
"ax-private-label-value",
];
const verifierRun = createVerifierRun();
const executable = await browserExecutable();
const profile = await mkdtemp(join(tmpdir(), "rea-real-browser-"));
@@ -209,6 +211,7 @@ try {
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
browser: observed.value.browser.product,
endpoint,
target,
scripts/verify-real-ghidra-windows.mjs
+6 -1
View File
@@ -25,6 +25,9 @@ import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProvider.js";
import { GHIDRA_SESSION_CAPABILITIES as SESSION_CAPABILITIES } from "../dist/ghidra/GhidraSessionValues.js";
import { parseBinaryTarget } from "../dist/domain/binaryTarget.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
if (process.platform !== "win32" || process.arch !== "x64")
throw new Error(
@@ -191,7 +194,9 @@ try {
}
if (report === undefined)
throw new Error("Windows Ghidra report was not produced");
process.stdout.write(`${JSON.stringify(report)}\n`);
process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
);
async function inventory(operation, parameters) {
const input = parseGhidraInventoryInput(operation, parameters);
scripts/verify-real-ghidra.mjs
+3
View File
@@ -33,8 +33,10 @@ import {
call,
verifyInventoryOperations,
} from "./verify-real-ghidra-inventory.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const exec = promisify(execFile);
const verifierRun = createVerifierRun();
const installDir = process.env.GHIDRA_INSTALL_DIR;
if (installDir === undefined || !isAbsolute(installDir))
throw new Error(
@@ -150,6 +152,7 @@ try {
customPath === undefined ? null : await verifyTarget(customPath, "custom");
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
ok: true,
provider: { id: "ghidra", version: SUPPORTED_GHIDRA_VERSION },
fixture_sources: [sourcePath, crossFormatSourcePath],
scripts/verify-real-hopper-linux.mjs
+3
View File
@@ -4,7 +4,10 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import { createVerifierRun } from "./lib/verifier-run.mjs";
const execFileAsync = promisify(execFile);
createVerifierRun();
if (process.platform !== "linux")
throw new Error("Linux real-Hopper verification must run on Linux");
scripts/verify-real-hopper.mjs
+8 -1
View File
@@ -27,7 +27,10 @@ import {
requireWorkflowEvidenceProvider,
} from "./lib/mcp-verifier-results.mjs";
import { openAndVerifyLargeFixture } from "./lib/real-hopper-pagination.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
const timeout = 180_000;
const parseServerArgs = (encoded) => {
const parsed = JSON.parse(encoded);
@@ -281,6 +284,8 @@ transport.stderr?.on("data", (chunk) => {
if (stderrBytes <= 16_384) stderrChunks.push(chunk.toString("utf8"));
});
const client = new Client({ name: "real-hopper-verifier", version: "1.0.0" });
const unrelatedHopper = await startUnrelatedHopperSentinel();
let unrelatedHopperSurvived = false;
let summary;
try {
@@ -456,6 +461,7 @@ try {
await transport.close();
} finally {
clearInterval(keepAlive);
unrelatedHopperSurvived = await unrelatedHopper.verifyAndClose();
}
}
@@ -482,9 +488,10 @@ if (
}
if (summary === undefined)
throw new Error("Real-Hopper verification did not produce a summary");
const completedVerifierRun = await completeVerifierRun(verifierRun);
await new Promise((resolve, reject) => {
process.stdout.write(
`${JSON.stringify({ ...summary, cleanShutdown: true }, null, 2)}\n`,
`${JSON.stringify({ verifier_run: completedVerifierRun, ...summary, unrelatedHopperSurvived, cleanShutdown: true }, null, 2)}\n`,
(cause) => {
if (cause) reject(cause);
else resolve();
scripts/verify-real-javascript-replay.mjs
+6 -1
View File
@@ -8,6 +8,9 @@ import { runControlledReplay } from "../dist/application/JavaScriptReplayService
import { createPermissionPolicy } from "../dist/domain/permissionPolicy.js";
import { LinuxJavaScriptReplayRunner } from "../dist/replay/LinuxJavaScriptReplayRunner.js";
import { SystemJavaScriptReplayHost } from "../dist/replay/SystemJavaScriptReplayHost.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const verifierRun = createVerifierRun();
const suppliedInputPath = process.env.REA_REPLAY_INPUT_PATH;
const suppliedInput =
@@ -147,7 +150,7 @@ const planned = await runControlledReplay(dependencies, input);
if (!planned.ok) {
if (process.env.REA_REPLAY_ALLOW_UNAVAILABLE === "true") {
process.stdout.write(
`${JSON.stringify({ available: false, error_tag: planned.error._tag })}\n`,
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), available: false, error_tag: planned.error._tag })}\n`,
);
process.exit(0);
}
@@ -179,6 +182,7 @@ if (
if (suppliedInput !== undefined) {
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
available: true,
supplied_manifest: true,
plan_digest: plan.plan_digest,
@@ -320,6 +324,7 @@ if (memory.termination !== "oom" || memory.cleanup.state !== "complete")
);
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
available: true,
plan_digest: plan.plan_digest,
evidence_id: evidence.evidence_id,
scripts/verify-windows-package.mjs
+3
View File
@@ -10,8 +10,10 @@ import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
const exec = promisify(execFile);
const verifierRun = createVerifierRun();
const root = process.cwd();
const workspace = await mkdtemp(join(tmpdir(), "rea-windows-package-"));
const prefix = join(workspace, "prefix");
@@ -98,6 +100,7 @@ try {
process.stdout.write(
`${JSON.stringify({
verifier_run: await completeVerifierRun(verifierRun),
ok: true,
platform: process.platform,
package: packageResult.filename,
skills/reverse-engineer-anything/SKILL.md
+1 -1
View File
@@ -4,7 +4,7 @@ description: Reverse engineer native, managed, Electron/JavaScript, packaged, an
metadata:
version: "23"
tool_count: 96
catalog_digest: "64387dcbe22c395e8b683344e5bfd2881e3e88c941b053723165f131c57ed46b"
catalog_digest: "92837c2e99c99a2dd087e4fb01d210cce9b5948051c33db1a404348ef90d8bf2"
---
# REA
src/application/AnalysisProvider.ts
+16
View File
@@ -13,6 +13,7 @@ import type { EvidenceSubjectTarget } from "../domain/evidence.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import type { ProgressReporter } from "./ProgressReporter.js";
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
import type { ProcessLineageObservation } from "../process/ProcessOwnership.js";
export interface ExecutionOptions {
readonly signal?: AbortSignal;
@@ -74,14 +75,28 @@ export interface AnalysisOperationPort {
): Promise<Result<AnalysisExecution, AnalysisError>>;
}
/** Retained provider-process observation attributed to its producing provider. */
export interface ProviderRuntimeLineageSnapshot {
readonly provider: ProviderIdentity;
readonly observation: ProcessLineageObservation;
}
/** Provider session bound to exactly one parsed artifact. */
export interface AnalysisClient extends AnalysisOperationPort {
/** Retained token-verified process snapshots for dynamic providers this client started. */
runtimeLineageSnapshots?(): readonly ProviderRuntimeLineageSnapshot[];
close(): Promise<void>;
}
/** Per-client identity allocated before any provider side effect starts. */
export interface AnalysisClientContext {
readonly runId: string;
}
export type AnalysisClientFactory = (
target: BinaryTarget,
profile?: AnalysisProfileCommitment,
context?: AnalysisClientContext,
) => AnalysisClient;
export interface ProviderIdentity {
@@ -144,6 +159,7 @@ export interface AnalysisProvider {
createClient(
target: BinaryTarget,
profile?: AnalysisProfileCommitment,
context?: AnalysisClientContext,
): AnalysisClient;
}
src/application/BinarySession.ts
+13 -2
View File
@@ -1,3 +1,5 @@
import { randomUUID } from "node:crypto";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import {
analysisProfilesEqual,
@@ -58,6 +60,7 @@ export class BinarySession
readonly profile: AnalysisProfileCommitment | null;
readonly compatibility: Readonly<Record<string, JsonValue>>;
readonly route: SessionProviderRoute;
readonly runId: string;
}
| undefined;
#transition: Promise<void> = Promise.resolve();
@@ -184,7 +187,8 @@ export class BinarySession
const previous = this.#active;
this.#active = undefined;
await previous?.client.close();
const client = route.createClient(target);
const runId = randomUUID();
const client = route.createClient(target, { runId });
const started = await client.execute("health", {}, options);
if (!started.ok) {
await client.close();
@@ -202,6 +206,7 @@ export class BinarySession
profile,
compatibility: structuredClone(compatibility),
route,
runId,
};
this.#clearRuntimeAvailability();
if (profile === null) this.clearSnapshot();
@@ -240,6 +245,9 @@ export class BinarySession
route: this.#currentRoute(),
router: this.#providerRouter,
runtimeAvailability: this.#runtimeAvailability,
runId: this.#active?.runId,
runtimeLineageSnapshots:
this.#active?.client.runtimeLineageSnapshots?.() ?? [],
});
}
@@ -396,11 +404,13 @@ export class BinarySession
readonly profile: AnalysisProfileCommitment | null;
readonly compatibility: Readonly<Record<string, JsonValue>>;
readonly route: SessionProviderRoute;
readonly runId: string;
}
| undefined,
): Promise<void> {
if (previous === undefined) return;
const client = previous.route.createClient(previous.target);
const runId = randomUUID();
const client = previous.route.createClient(previous.target, { runId });
const started = await client.execute("health", {});
if (started.ok)
this.#active = {
@@ -409,6 +419,7 @@ export class BinarySession
profile: previous.profile,
compatibility: previous.compatibility,
route: previous.route,
runId,
};
else await client.close();
}
src/application/BinarySessionStatus.ts
+56 -1
View File
@@ -1,6 +1,9 @@
import type { BinaryTarget } from "../domain/binaryTarget.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { CapabilityDescriptor } from "./AnalysisProvider.js";
import type {
CapabilityDescriptor,
ProviderRuntimeLineageSnapshot,
} from "./AnalysisProvider.js";
import type {
SessionProviderRoute,
SessionProviderRouter,
@@ -14,6 +17,8 @@ interface BinarySessionStatusInput {
string,
{ readonly available: boolean; readonly reason: string | null }
>;
readonly runId: string | undefined;
readonly runtimeLineageSnapshots: readonly ProviderRuntimeLineageSnapshot[];
}
/** Project internal provider routing state into the caller-visible session status. */
@@ -22,6 +27,8 @@ export const binarySessionStatus = ({
route,
router,
runtimeAvailability,
runId,
runtimeLineageSnapshots,
}: BinarySessionStatusInput): JsonValue => {
const configuredProvider = router.configuredIdentity();
const provider = providerSummary(configuredProvider);
@@ -59,6 +66,13 @@ export const binarySessionStatus = ({
capabilities,
analysis_provider_binding: analysisProviderBinding,
analysis_provider_candidates: analysisProviderCandidates,
analysis_run:
runId === undefined
? null
: {
run_id: runId,
process_lineage: processLineageStatus(runtimeLineageSnapshots),
},
};
return target === undefined
? { open: false, ...common }
@@ -83,6 +97,47 @@ const providerSummary = ({
readonly version: string | null;
}) => ({ id, name, version });
const processLineageStatus = (
snapshots: readonly ProviderRuntimeLineageSnapshot[],
) =>
snapshots.length === 0
? { status: "not_observed" }
: {
status: "snapshots",
snapshots: snapshots
.slice()
.sort((left, right) =>
left.provider.id.localeCompare(right.provider.id),
)
.map(({ provider, observation }) => ({
provider: providerSummary(provider),
observation:
observation.status === "unavailable"
? {
status: observation.status,
observed_at: observation.observedAt,
launcher_pid: observation.launcherPid,
process_group_id: observation.processGroupId,
reason: observation.reason,
}
: {
status: observation.status,
observed_at: observation.observedAt,
schema_version: observation.lineage.schemaVersion,
launcher_pid: observation.lineage.launcherPid,
launcher_parent_pid: observation.lineage.launcherParentPid,
process_group_id: observation.lineage.processGroupId,
descendants: observation.lineage.descendants.map(
({ pid, parentPid, processGroupId }) => ({
pid,
parent_pid: parentPid,
process_group_id: processGroupId,
}),
),
},
})),
};
const capabilityStatus = (
descriptor: Omit<CapabilityDescriptor, "available" | "reason"> & {
readonly available: boolean;
src/application/CompositeProvider.ts
+9
View File
@@ -4,6 +4,7 @@ import type { BinaryTarget } from "../domain/binaryTarget.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import type {
AnalysisClient,
AnalysisClientContext,
AnalysisProfileResolutionOptions,
AnalysisProvider,
CapabilityDescriptor,
@@ -88,6 +89,7 @@ export class CompositeProvider implements AnalysisProvider {
createClient(
target: BinaryTarget,
profile?: AnalysisProfileCommitment,
context?: AnalysisClientContext,
): AnalysisClient {
const clients = new Map<AnalysisProvider, AnalysisClient>();
const clientFor = (provider: AnalysisProvider): AnalysisClient => {
@@ -96,6 +98,7 @@ export class CompositeProvider implements AnalysisProvider {
const created = provider.createClient(
target,
profile?.provider.id === provider.identity().id ? profile : undefined,
context,
);
clients.set(provider, created);
return created;
@@ -126,6 +129,12 @@ export class CompositeProvider implements AnalysisProvider {
)
: clientFor(provider).execute(operation, parameters, options);
},
runtimeLineageSnapshots: () =>
[...clients.values()]
.flatMap((client) => client.runtimeLineageSnapshots?.() ?? [])
.sort((left, right) =>
left.provider.id.localeCompare(right.provider.id),
),
close: async () => {
await Promise.allSettled(
[...clients.values()].map(async (client) => client.close()),
src/application/LoopbackReplay.ts
+365 -154
View File
@@ -9,140 +9,216 @@ import type {
ProcessScenario,
ProtocolEvent,
} from "../domain/processCapture.js";
import {
type ReplayMachineDecision,
type ReplayTransitionRecord,
} from "../domain/replayMachineRuntime.js";
import {
closeReplayServer,
listenOnLoopback,
recordMachineEvent,
readReplayRequestBody,
ReplayRecorder,
requestHeaders,
waitForReplayDelay,
} from "./LoopbackReplayRecorder.js";
/** Owned loopback replay endpoints and their bounded protocol observations. */
export interface LoopbackReplay {
readonly httpUrl: string;
readonly websocketUrl: string;
readonly events: ProtocolEvent[];
readonly transitions: readonly ReplayTransitionRecord[];
readonly truncated: boolean;
close(): Promise<void>;
}
class ReplayRecorder {
readonly events: ProtocolEvent[] = [];
readonly started = Date.now();
truncated = false;
constructor(readonly scenario: ProcessScenario) {}
atMs(): number {
return (
Math.floor(
(Date.now() - this.started) /
this.scenario.normalization.time_bucket_ms,
) * this.scenario.normalization.time_bucket_ms
);
}
record(event: Omit<ProtocolEvent, "sequence">): void {
if (this.events.length < this.scenario.limits.protocol_events)
this.events.push({ sequence: this.events.length, ...event });
else this.truncated = true;
}
interface HttpReplayRequest {
readonly request: IncomingMessage;
readonly response: ServerResponse;
readonly method: string;
readonly path: string;
readonly body: string;
readonly rawAtMs: number;
readonly recordedAtMs: number;
}
const readRequestBody = async (
request: IncomingMessage,
limit: number,
): Promise<string | undefined> => {
const chunks: Buffer[] = [];
let bytes = 0;
for await (const chunk of request) {
const value = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
bytes += value.byteLength;
if (bytes > limit) return undefined;
chunks.push(value);
const recordHttpResponse = (
recorder: ReplayRecorder,
request: Pick<HttpReplayRequest, "method" | "path">,
data: string,
outcome: ProtocolEvent["outcome"],
): void =>
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "response",
method: request.method,
path: request.path,
data,
outcome,
});
const handleMachineHttp = async (
recorder: ReplayRecorder,
request: HttpReplayRequest,
): Promise<void> => {
const machine = recorder.machine;
if (machine === undefined) return;
const decision = machine.dispatch({
protocol: "http",
connection: "not_applicable",
at_ms: request.rawAtMs,
recorded_at_ms: request.recordedAtMs,
method: request.method,
path: request.path,
headers: requestHeaders(request.request),
body: request.body,
});
recordMachineEvent(
recorder,
{
at_ms: request.recordedAtMs,
protocol: "http",
direction: "request",
method: request.method,
path: request.path,
data: request.body,
},
decision,
);
if (decision.outcome !== "matched") {
request.response.statusCode = decision.outcome === "unmatched" ? 404 : 409;
request.response.end();
return;
}
for (const action of decision.actions) {
if (action.type === "delay") await waitForReplayDelay(action.duration_ms);
else if (action.type === "disconnect") {
request.response.destroy();
recordHttpResponse(recorder, request, "", "disconnected");
return;
} else if (action.type === "http_response") {
request.response.statusCode = action.status;
for (const [name, value] of Object.entries(action.headers))
request.response.setHeader(name, value);
request.response.end(action.body);
recordHttpResponse(
recorder,
request,
machine.redact(action.body),
"matched",
);
}
}
return Buffer.concat(chunks).toString("utf8");
};
const createHttpHandler = (
const handleStaticHttp = async (
recorder: ReplayRecorder,
): ((request: IncomingMessage, response: ServerResponse) => Promise<void>) => {
const calls = new Map<number, number>();
return async (request, response) => {
const method = request.method ?? "GET";
const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname;
const body = await readRequestBody(
request,
recorder.scenario.limits.protocol_body_bytes,
);
if (body === undefined) {
recorder.truncated = true;
response.statusCode = 413;
response.end();
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "request",
method,
path,
data: "<body-over-limit>",
outcome: "unmatched",
});
return;
}
const index = recorder.scenario.replay.http.findIndex((route) => {
if (route.method !== method || route.path !== path) return false;
if (route.request_body !== undefined && route.request_body !== body)
return false;
return Object.entries(route.request_headers).every(
([name, value]) => request.headers[name.toLowerCase()] === value,
);
});
const route = recorder.scenario.replay.http[index];
const used = index < 0 ? 0 : (calls.get(index) ?? 0);
const outcome =
route === undefined
? "unmatched"
: used >= route.max_calls
? "script_exhausted"
: "matched";
request: HttpReplayRequest,
calls: Map<number, number>,
): Promise<void> => {
const index = recorder.scenario.replay.http.findIndex((route) =>
route.method !== request.method || route.path !== request.path
? false
: (route.request_body === undefined ||
route.request_body === request.body) &&
Object.entries(route.request_headers).every(
([name, value]) =>
request.request.headers[name.toLowerCase()] === value,
),
);
const route = recorder.scenario.replay.http[index];
const used = index < 0 ? 0 : (calls.get(index) ?? 0);
const outcome =
route === undefined
? "unmatched"
: used >= route.max_calls
? "script_exhausted"
: "matched";
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "request",
method: request.method,
path: request.path,
data: request.body.length === 0 ? "" : "<redacted-request-body>",
outcome,
});
if (route === undefined || outcome === "script_exhausted") {
request.response.statusCode = route === undefined ? 404 : 409;
request.response.end();
return;
}
calls.set(index, used + 1);
if (route.delay_ms > 0) await waitForReplayDelay(route.delay_ms);
if (route.disconnect) {
request.response.destroy();
recordHttpResponse(recorder, request, "", "disconnected");
return;
}
request.response.statusCode = route.status;
for (const [name, value] of Object.entries(route.response_headers))
request.response.setHeader(name, value);
request.response.end(route.body);
recordHttpResponse(recorder, request, route.body, "matched");
};
const handleHttpRequest = async (
recorder: ReplayRecorder,
request: IncomingMessage,
response: ServerResponse,
calls: Map<number, number>,
): Promise<void> => {
const body = await readReplayRequestBody(
request,
recorder.scenario.limits.protocol_body_bytes,
);
const method = request.method ?? "GET";
const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname;
if (body === undefined) {
recorder.truncated = true;
response.statusCode = 413;
response.end();
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "request",
method,
path,
data: body.length === 0 ? "" : "<redacted-request-body>",
outcome,
data: "<body-over-limit>",
outcome: "unmatched",
});
if (route === undefined || outcome === "script_exhausted") {
response.statusCode = route === undefined ? 404 : 409;
response.end();
return;
}
calls.set(index, used + 1);
if (route.delay_ms > 0)
await new Promise((resolveDelay) =>
setTimeout(resolveDelay, route.delay_ms),
);
if (route.disconnect) {
response.destroy();
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "response",
method,
path,
data: "",
outcome: "disconnected",
});
return;
}
response.statusCode = route.status;
for (const [name, value] of Object.entries(route.response_headers))
response.setHeader(name, value);
response.end(route.body);
recorder.record({
at_ms: recorder.atMs(),
protocol: "http",
direction: "response",
method,
path,
data: route.body,
outcome: "matched",
return;
}
const replayRequest = {
request,
response,
method,
path,
body,
rawAtMs: recorder.rawAtMs(),
recordedAtMs: recorder.atMs(),
};
await (recorder.machine === undefined
? handleStaticHttp(recorder, replayRequest, calls)
: recorder.enqueueMachine(() =>
handleMachineHttp(recorder, replayRequest),
));
};
const createHttpHandler = (
recorder: ReplayRecorder,
): ((request: IncomingMessage, response: ServerResponse) => void) => {
const calls = new Map<number, number>();
return (request, response) => {
void handleHttpRequest(recorder, request, response, calls).catch(() => {
recorder.truncated = true;
if (response.headersSent) response.destroy();
else {
response.statusCode = 500;
response.end();
}
});
};
};
@@ -204,6 +280,146 @@ const sendWebSocketScript = async (
}
};
const runWebSocketActions = async (
client: WebSocket,
recorder: ReplayRecorder,
decision: Extract<ReplayMachineDecision, { readonly outcome: "matched" }>,
path: string,
): Promise<void> => {
for (const action of decision.actions) {
if (action.type === "delay") {
await waitForReplayDelay(action.duration_ms);
continue;
}
if (action.type === "disconnect") {
client.close();
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path,
data: "",
outcome: "disconnected",
});
return;
}
if (action.type === "websocket_send" && client.readyState === 1) {
client.send(action.data);
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "sent",
method: null,
path,
data: recorder.machine?.redact(action.data) ?? action.data,
outcome: "matched",
});
}
}
};
const handleStaticWebSocket = (
client: WebSocket,
recorder: ReplayRecorder,
connection: number,
path: string,
): void => {
client.on("message", (value) =>
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "received",
method: null,
path,
data: value.toString(),
outcome: "matched",
}),
);
void sendWebSocketScript(client, connection, recorder);
};
const handleMachineWebSocket = (options: {
readonly client: WebSocket;
readonly request: IncomingMessage;
readonly recorder: ReplayRecorder;
readonly connection: number;
readonly path: string;
}): void => {
const { client, request, recorder, connection, path } = options;
const machine = recorder.machine;
if (machine === undefined) return;
void recorder
.enqueueMachine(async () => {
const connectedAt = recorder.atMs();
const connectionDecision = machine.dispatch({
protocol: "websocket_connect",
connection: connection === 1 ? "initial" : "reconnect",
at_ms: recorder.rawAtMs(),
recorded_at_ms: connectedAt,
method: null,
path,
headers: requestHeaders(request),
body: "",
});
recordMachineEvent(
recorder,
{
at_ms: connectedAt,
protocol: "websocket",
direction: "received",
method: null,
path,
data: "",
},
connectionDecision,
);
if (connectionDecision.outcome === "matched")
await runWebSocketActions(client, recorder, connectionDecision, path);
else client.close();
})
.catch(() => {
recorder.truncated = true;
client.terminate();
});
client.on("message", (value) => {
const body = value.toString();
const rawAtMs = recorder.rawAtMs();
const recordedAtMs = recorder.atMs();
void recorder
.enqueueMachine(async () => {
const decision = machine.dispatch({
protocol: "websocket_message",
connection: connection === 1 ? "initial" : "reconnect",
at_ms: rawAtMs,
recorded_at_ms: recordedAtMs,
method: null,
path,
headers: {},
body,
});
recordMachineEvent(
recorder,
{
at_ms: recordedAtMs,
protocol: "websocket",
direction: "received",
method: null,
path,
data: body,
},
decision,
);
if (decision.outcome === "matched")
await runWebSocketActions(client, recorder, decision, path);
})
.catch(() => {
recorder.truncated = true;
client.terminate();
});
});
};
const createWebSocketReplay = (
server: Server,
recorder: ReplayRecorder,
@@ -215,7 +431,10 @@ const createWebSocketReplay = (
clientTracking: true,
});
server.on("upgrade", (request, socket, head) => {
if (new URL(request.url ?? "/", "http://127.0.0.1").pathname !== "/ws") {
if (
recorder.machine === undefined &&
new URL(request.url ?? "/", "http://127.0.0.1").pathname !== "/ws"
) {
socket.destroy();
return;
}
@@ -223,7 +442,7 @@ const createWebSocketReplay = (
websocket.emit("connection", client, request),
);
});
websocket.on("connection", (client: WebSocket) => {
websocket.on("connection", (client: WebSocket, request: IncomingMessage) => {
connections += 1;
if (connections > recorder.scenario.limits.connections) {
recorder.truncated = true;
@@ -231,43 +450,21 @@ const createWebSocketReplay = (
return;
}
client.on("error", () => undefined);
client.on("message", (value) =>
recorder.record({
at_ms: recorder.atMs(),
protocol: "websocket",
direction: "received",
method: null,
path: "/ws",
data: value.toString(),
outcome: "matched",
}),
);
void sendWebSocketScript(client, connections, recorder);
const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname;
if (recorder.machine === undefined)
handleStaticWebSocket(client, recorder, connections, path);
else
handleMachineWebSocket({
client,
request,
recorder,
connection: connections,
path,
});
});
return websocket;
};
const listen = async (server: Server): Promise<number> => {
await new Promise<void>((resolveListen, rejectListen) => {
server.once("error", rejectListen);
server.listen(0, "127.0.0.1", () => {
server.off("error", rejectListen);
resolveListen();
});
});
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("loopback replay did not acquire a TCP port");
return address.port;
};
const closeServer = (server: Server): Promise<void> =>
new Promise((resolveClose, rejectClose) => {
server.close((error) =>
error === undefined ? resolveClose() : rejectClose(error),
);
});
/** Start bounded HTTP and WebSocket replay endpoints on IPv4 loopback only. */
export const startLoopbackReplay = async (
scenario: ProcessScenario,
@@ -275,26 +472,40 @@ export const startLoopbackReplay = async (
const recorder = new ReplayRecorder(scenario);
const server = createServer(createHttpHandler(recorder));
const websocket = createWebSocketReplay(server, recorder);
let closePromise: Promise<void> | undefined;
let port: number;
try {
port = await listen(server);
port = await listenOnLoopback(server);
} catch (cause: unknown) {
await closeServer(server).catch(() => undefined);
await closeReplayServer(server).catch(() => undefined);
throw cause;
}
return {
httpUrl: `http://127.0.0.1:${String(port)}`,
websocketUrl: `ws://127.0.0.1:${String(port)}/ws`,
events: recorder.events,
get events() {
return recorder.events.map((event) => ({ ...event }));
},
get transitions() {
return recorder.transitions.map((transition) => ({
...transition,
sensitive_aliases: [...transition.sensitive_aliases],
}));
},
get truncated() {
return recorder.truncated;
},
async close() {
for (const client of websocket.clients) client.terminate();
await new Promise<void>((resolveClose) =>
websocket.close(() => resolveClose()),
);
await closeServer(server);
closePromise ??= (async () => {
recorder.stopMachineAdmission();
for (const client of websocket.clients) client.terminate();
await new Promise<void>((resolveClose) =>
websocket.close(() => resolveClose()),
);
await closeReplayServer(server);
await recorder.drainMachine();
})();
await closePromise;
},
};
};
src/application/LoopbackReplayRecorder.ts
+130
View File
@@ -0,0 +1,130 @@
import type { IncomingMessage, Server } from "node:http";
import type {
ProcessScenario,
ProtocolEvent,
} from "../domain/processCapture.js";
import {
ReplayMachineRuntime,
type ReplayMachineDecision,
type ReplayTransitionRecord,
} from "../domain/replayMachineRuntime.js";
/** Bounded protocol and replay-machine observations for one process run. */
export class ReplayRecorder {
readonly events: ProtocolEvent[] = [];
readonly started = Date.now();
readonly machine: ReplayMachineRuntime | undefined;
#machineQueue: Promise<void> = Promise.resolve();
#machineAdmissionOpen = true;
truncated = false;
constructor(readonly scenario: ProcessScenario) {
this.machine =
scenario.replay.machine === null
? undefined
: new ReplayMachineRuntime(scenario.replay.machine);
}
get transitions(): readonly ReplayTransitionRecord[] {
return this.machine?.timeline ?? [];
}
atMs(): number {
return (
Math.floor(
(Date.now() - this.started) /
this.scenario.normalization.time_bucket_ms,
) * this.scenario.normalization.time_bucket_ms
);
}
rawAtMs(): number {
return Date.now() - this.started;
}
enqueueMachine<T>(operation: () => Promise<T>): Promise<T> {
if (!this.#machineAdmissionOpen)
return Promise.reject(new Error("replay machine is closing"));
const result = this.#machineQueue.then(operation);
this.#machineQueue = result.then(
() => undefined,
() => undefined,
);
return result;
}
stopMachineAdmission(): void {
this.#machineAdmissionOpen = false;
}
async drainMachine(): Promise<void> {
await this.#machineQueue;
}
record(event: Omit<ProtocolEvent, "sequence">): void {
if (this.events.length < this.scenario.limits.protocol_events)
this.events.push({ sequence: this.events.length, ...event });
else this.truncated = true;
}
}
export const requestHeaders = (
request: IncomingMessage,
): Readonly<Record<string, string>> =>
Object.fromEntries(
Object.entries(request.headers).flatMap(([name, value]) =>
value === undefined
? []
: [[name, Array.isArray(value) ? value.join(", ") : value]],
),
);
export const readReplayRequestBody = async (
request: IncomingMessage,
limit: number,
): Promise<string | undefined> => {
const chunks: Buffer[] = [];
let bytes = 0;
for await (const chunk of request) {
const value = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
bytes += value.byteLength;
if (bytes > limit) return undefined;
chunks.push(value);
}
return Buffer.concat(chunks).toString("utf8");
};
export const recordMachineEvent = (
recorder: ReplayRecorder,
event: Omit<ProtocolEvent, "sequence" | "outcome">,
decision: ReplayMachineDecision,
): void =>
recorder.record({
...event,
data: recorder.machine?.redact(event.data) ?? event.data,
outcome: decision.outcome,
});
export const waitForReplayDelay = (durationMs: number): Promise<void> =>
new Promise((resolveDelay) => setTimeout(resolveDelay, durationMs));
export const listenOnLoopback = async (server: Server): Promise<number> => {
await new Promise<void>((resolveListen, rejectListen) => {
server.once("error", rejectListen);
server.listen(0, "127.0.0.1", () => {
server.off("error", rejectListen);
resolveListen();
});
});
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("loopback replay did not acquire a TCP port");
return address.port;
};
export const closeReplayServer = (server: Server): Promise<void> =>
new Promise((resolveClose, rejectClose) => {
server.close((error) =>
error === undefined ? resolveClose() : rejectClose(error),
);
});
src/application/ProcessCaptureLifecycle.ts
+1
View File
@@ -115,6 +115,7 @@ export const buildCaptureResult = (
options.replay.events,
options.scenario,
),
replay_transitions: options.replay.transitions,
files_before: options.before.files,
files_after: options.after.files,
filesystem_effects: classifyFilesystemEffects(
src/application/ProcessHarness.ts
+1
View File
@@ -382,6 +382,7 @@ const completeCapture = async (options: {
const after = await snapshotRoots(scenario, options.signal);
const renderedFrames = await runtime.renderer.frames();
const checkpoints = await runtime.checkpoints.finish(after);
await runtime.replay.close();
const manifest = await createRunManifest(
scenario,
runtime.startedAt,
src/application/SessionProviderRouter.ts
+15 -7
View File
@@ -15,6 +15,7 @@ import { jsonObjectSchema, type JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import type {
AnalysisClient,
AnalysisClientContext,
AnalysisClientFactory,
AnalysisProfileResolution,
AnalysisProfileResolutionOptions,
@@ -42,7 +43,10 @@ export interface SessionProviderRoute {
readonly compatibility: Readonly<Record<string, JsonValue>>;
readonly binding: AnalysisProviderBinding | null;
readonly selection: AnalysisProviderSelection | undefined;
createClient(target: BinaryTarget): AnalysisClient;
createClient(
target: BinaryTarget,
context: AnalysisClientContext,
): AnalysisClient;
}
interface LegacyProviderRuntime {
@@ -172,7 +176,8 @@ export class SessionProviderRouter {
compatibility: {},
binding: null,
selection: undefined,
createClient: (target) => createLegacyClient(runtime, target, undefined),
createClient: (target, context) =>
createLegacyClient(runtime, target, undefined, context),
};
}
@@ -225,11 +230,12 @@ export class SessionProviderRouter {
compatibility: normalized.value.compatibility,
binding: null,
selection: undefined,
createClient: (openedTarget) =>
createClient: (openedTarget, context) =>
createLegacyClient(
runtime,
openedTarget,
normalized.value.profile ?? undefined,
context,
),
});
}
@@ -280,8 +286,9 @@ const selectableRoute = (
compatibility: structuredClone(binding?.compatibility ?? {}),
binding,
selection,
createClient: (target) =>
provider?.createClient(target, binding?.profile) ?? emptyClient(identity),
createClient: (target, context) =>
provider?.createClient(target, binding?.profile, context) ??
emptyClient(identity),
};
};
@@ -289,10 +296,11 @@ const createLegacyClient = (
runtime: LegacyProviderRuntime,
target: BinaryTarget,
profile: AnalysisProfileCommitment | undefined,
context: AnalysisClientContext,
): AnalysisClient =>
typeof runtime.provider === "function"
? runtime.provider(target, profile)
: runtime.provider.createClient(target, profile);
? runtime.provider(target, profile, context)
: runtime.provider.createClient(target, profile, context);
const emptyClient = (identity: ProviderIdentity): AnalysisClient => ({
execute: (operation) =>
src/contracts/processCaptureExample.ts
+1
View File
@@ -58,6 +58,7 @@ export const EMPTY_PROCESS_CAPTURE_EXAMPLE = {
],
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
src/contracts/toolOutputSchemaPrimitives.ts
+47
View File
@@ -91,6 +91,52 @@ export const providerIdentity = z.object({
version: z.string().nullable(),
});
const processCoordinates = {
launcher_pid: z.number().int().min(1),
process_group_id: z.number().int().min(1),
};
const processLineageObservation = z.discriminatedUnion("status", [
z.object({
status: z.literal("unavailable"),
observed_at: z.iso.datetime(),
...processCoordinates,
reason: z.string().min(1),
}),
z.object({
status: z.literal("verified"),
observed_at: z.iso.datetime(),
schema_version: z.literal(1),
...processCoordinates,
launcher_parent_pid: z.number().int().min(0),
descendants: z.array(
z.object({
pid: z.number().int().min(1),
parent_pid: z.number().int().min(0),
process_group_id: z.number().int().min(1),
}),
),
}),
]);
const analysisRun = z
.object({
run_id: z.string().uuid(),
process_lineage: z.discriminatedUnion("status", [
z.object({ status: z.literal("not_observed") }),
z.object({
status: z.literal("snapshots"),
snapshots: z.array(
z.object({
provider: providerIdentity,
observation: processLineageObservation,
}),
),
}),
]),
})
.nullable();
const providerRejectionCode: z.ZodType<ProviderRejectionCode> = z.enum(
PROVIDER_REJECTION_CODES,
);
@@ -175,6 +221,7 @@ export const sessionProvider = z.object({
provider: providerIdentity,
providers: z.array(providerIdentity),
capabilities: z.array(providerCapability),
analysis_run: analysisRun,
analysis_provider_binding: z
.object({
provider: providerIdentity,
src/domain/completionLedgerGeneration.ts
+250
View File
@@ -0,0 +1,250 @@
import { createHash } from "node:crypto";
import canonicalize from "canonicalize";
import { z } from "zod";
import {
createEvidenceCompletionLedger,
evidenceCompletionRecordSchema,
type EvidenceCompletionLedger,
} from "./evidenceCompletionLedger.js";
const identifierSchema = z.string().regex(/^[A-Za-z][A-Za-z0-9._-]{0,99}$/u);
const digestSchema = z.string().regex(/^[a-f0-9]{64}$/u);
const identitySchema = z.strictObject({
id: identifierSchema,
version: z.string().min(1).max(100),
});
const environmentSchema = z.strictObject({
platform: identifierSchema,
architecture: identifierSchema,
runtime: identifierSchema,
runtime_version: z.string().min(1).max(100),
});
const scenarioSchema = z.strictObject({
id: identifierSchema,
version: z.number().int().positive(),
});
const skillDigestSchema = z.strictObject({
skill_id: identifierSchema,
sha256: digestSchema,
});
const verifierRunSchema = z
.strictObject({
schema_version: z.literal(1),
run_id: z.string().uuid(),
verifier_pid: z.number().int().positive(),
parent_pid: z.number().int().nonnegative(),
process_lineage: z.discriminatedUnion("status", [
z.strictObject({
status: z.literal("verified"),
schema_version: z.literal(1),
observed_at: z.iso.datetime(),
launcher_pid: z.number().int().positive(),
launcher_parent_pid: z.number().int().nonnegative(),
process_group_id: z.number().int().positive(),
descendants: z.array(
z.strictObject({
pid: z.number().int().positive(),
parent_pid: z.number().int().nonnegative(),
process_group_id: z.number().int().positive(),
}),
),
}),
z.strictObject({
status: z.literal("unavailable"),
observed_at: z.iso.datetime(),
launcher_pid: z.number().int().positive(),
launcher_parent_pid: z.number().int().nonnegative(),
process_group_id: z.number().int().positive().nullable(),
reason: z.string().min(1).max(500),
}),
]),
})
.superRefine((run, context) => {
if (run.process_lineage.launcher_pid !== run.verifier_pid)
context.addIssue({
code: "custom",
path: ["process_lineage", "launcher_pid"],
message: "Lineage launcher PID must identify the verifier process",
});
if (run.process_lineage.launcher_parent_pid !== run.parent_pid)
context.addIssue({
code: "custom",
path: ["process_lineage", "launcher_parent_pid"],
message: "Lineage parent PID must identify the verifier parent",
});
});
const reportClaimSchema = z.strictObject({
claim_id: identifierSchema,
scenario: scenarioSchema,
artifact_sha256s: z.array(digestSchema).max(100),
provider: identitySchema,
result_schema_version: z.number().int().positive(),
status: evidenceCompletionRecordSchema.shape.status,
evidence_ids: evidenceCompletionRecordSchema.shape.evidence_ids,
});
const completionVerifierReportObjectSchema = z.strictObject({
schema_version: z.literal(1),
verifier: identitySchema,
verifier_run: verifierRunSchema,
environment: environmentSchema,
claims: z.array(reportClaimSchema).min(1).max(10_000),
});
/** Strict output contract implemented by completion-aware verifiers. */
export const completionVerifierReportSchema =
completionVerifierReportObjectSchema.superRefine((report, context) => {
for (const [index, claim] of report.claims.entries())
if (claim.status === "pass" && claim.artifact_sha256s.length === 0)
context.addIssue({
code: "custom",
path: ["claims", index, "artifact_sha256s"],
message: "Passing claims must commit at least one artifact digest",
});
});
export type CompletionVerifierReport = z.infer<
typeof completionVerifierReportSchema
>;
const manifestClaimSchema = reportClaimSchema.omit({
status: true,
evidence_ids: true,
});
const completionManifestObjectSchema = z.strictObject({
schema_version: z.literal(1),
manifest_id: z.string().regex(/^ecm_[a-f0-9]{64}$/u),
verifier: identitySchema,
environment: environmentSchema,
skill_digests: z.array(skillDigestSchema).min(1).max(100),
claims: z.array(manifestClaimSchema).min(1).max(10_000),
});
type CompletionManifestValue = z.infer<typeof completionManifestObjectSchema>;
const compareIdentifiers = (left: string, right: string): number =>
left < right ? -1 : left > right ? 1 : 0;
const isSortedUnique = (values: readonly string[]): boolean =>
values.every(
(value, index) =>
index === 0 || compareIdentifiers(values[index - 1] ?? "", value) < 0,
);
const canonicalJson = (value: unknown): string => {
const serialized = canonicalize(value);
if (serialized === undefined)
throw new TypeError(
"RFC 8785 canonicalization rejected completion metadata",
);
return serialized;
};
const computeManifestId = (
manifest: Omit<CompletionManifestValue, "manifest_id">,
): string =>
`ecm_${createHash("sha256").update(canonicalJson(manifest)).digest("hex")}`;
const manifestIssues = (
manifest: CompletionManifestValue,
context: z.RefinementCtx,
): void => {
if (!isSortedUnique(manifest.claims.map(({ claim_id }) => claim_id)))
context.addIssue({
code: "custom",
path: ["claims"],
message: "Manifest claims must be sorted by unique claim ID",
});
if (!isSortedUnique(manifest.skill_digests.map(({ skill_id }) => skill_id)))
context.addIssue({
code: "custom",
path: ["skill_digests"],
message: "Skill digests must be sorted by unique skill ID",
});
for (const [index, claim] of manifest.claims.entries())
if (!isSortedUnique(claim.artifact_sha256s))
context.addIssue({
code: "custom",
path: ["claims", index, "artifact_sha256s"],
message: "Artifact digests must be sorted and unique",
});
const { manifest_id: manifestId, ...projection } = manifest;
if (computeManifestId(projection) !== manifestId)
context.addIssue({
code: "custom",
path: ["manifest_id"],
message: "Completion manifest identifier does not match its content",
});
};
/** Canonical immutable commitments for one verifier run. */
export const completionManifestSchema =
completionManifestObjectSchema.superRefine(manifestIssues);
export type CompletionManifest = z.infer<typeof completionManifestSchema>;
/** Generated completion metadata derived from a verifier report. */
export interface CompletionLedgerArtifacts {
readonly manifest: CompletionManifest;
readonly ledger: EvidenceCompletionLedger;
}
const canonicalClaims = (
claims: CompletionVerifierReport["claims"],
): CompletionVerifierReport["claims"] =>
claims
.map((claim) => ({
...claim,
artifact_sha256s: [...claim.artifact_sha256s].sort(compareIdentifiers),
evidence_ids: [...claim.evidence_ids].sort(compareIdentifiers),
}))
.sort((left, right) => compareIdentifiers(left.claim_id, right.claim_id));
/** Generate canonical manifest and ledger artifacts from live verifier output. */
export const createCompletionLedgerArtifacts = (
input: unknown,
skillDigests: readonly z.input<typeof skillDigestSchema>[],
): CompletionLedgerArtifacts => {
const report = completionVerifierReportSchema.parse(input);
const claims = canonicalClaims(report.claims);
if (!isSortedUnique(claims.map(({ claim_id }) => claim_id)))
throw new TypeError("Completion verifier claim IDs must be unique");
for (const claim of claims) {
if (!isSortedUnique(claim.artifact_sha256s))
throw new TypeError("Artifact digests must be unique per claim");
if (!isSortedUnique(claim.evidence_ids))
throw new TypeError("Evidence IDs must be unique per claim");
}
const skills = skillDigests
.map((skill) => skillDigestSchema.parse(skill))
.sort((left, right) => compareIdentifiers(left.skill_id, right.skill_id));
if (!isSortedUnique(skills.map(({ skill_id }) => skill_id)))
throw new TypeError("Completion skill IDs must be unique");
const projection = {
schema_version: 1 as const,
verifier: report.verifier,
environment: report.environment,
skill_digests: skills,
claims: claims.map(
({ status: _status, evidence_ids: _evidenceIds, ...claim }) => claim,
),
};
return {
manifest: completionManifestSchema.parse({
...projection,
manifest_id: computeManifestId(projection),
}),
ledger: createEvidenceCompletionLedger(
claims.map(({ claim_id, status, evidence_ids }) => ({
claim_id,
status,
evidence_ids,
})),
),
};
};
/** Parse generated completion commitments and reject manual tampering. */
export const parseCompletionManifest = (input: unknown): CompletionManifest =>
completionManifestSchema.parse(input);
src/domain/evidenceCompletionLedger.ts
+169
View File
@@ -0,0 +1,169 @@
import { createHash } from "node:crypto";
import canonicalize from "canonicalize";
import { z } from "zod";
const evidenceIdSchema = z.string().regex(/^ev_[a-f0-9]{64}$/u);
const claimIdSchema = z.string().regex(/^[A-Za-z][A-Za-z0-9._-]{0,99}$/u);
/** Terminal outcome recorded for one verifier claim. */
export const evidenceCompletionStatusSchema = z.enum([
"pass",
"fail",
"unsupported",
"truncated",
"unknown",
]);
/** Strict evidence-linked result for one verifier claim. */
export const evidenceCompletionRecordSchema = z.strictObject({
claim_id: claimIdSchema,
status: evidenceCompletionStatusSchema,
evidence_ids: z.array(evidenceIdSchema).min(1).max(100),
});
const completionSummarySchema = z.strictObject({
total: z.number().int().min(1).max(10_000),
pass: z.number().int().nonnegative(),
fail: z.number().int().nonnegative(),
unsupported: z.number().int().nonnegative(),
truncated: z.number().int().nonnegative(),
unknown: z.number().int().nonnegative(),
complete: z.boolean(),
});
const completionLedgerObjectSchema = z.strictObject({
schema_version: z.literal(1),
ledger_id: z.string().regex(/^ecl_[a-f0-9]{64}$/u),
records: z.array(evidenceCompletionRecordSchema).min(1).max(10_000),
summary: completionSummarySchema,
});
export type EvidenceCompletionStatus = z.infer<
typeof evidenceCompletionStatusSchema
>;
/** Readonly verifier result accepted by the canonical ledger builder. */
export interface EvidenceCompletionRecordInput {
readonly claim_id: string;
readonly status: EvidenceCompletionStatus;
readonly evidence_ids: readonly string[];
}
export type EvidenceCompletionRecord = z.infer<
typeof evidenceCompletionRecordSchema
>;
type EvidenceCompletionLedgerValue = z.infer<
typeof completionLedgerObjectSchema
>;
const compareIdentifiers = (left: string, right: string): number =>
left < right ? -1 : left > right ? 1 : 0;
const isSortedUnique = (values: readonly string[]): boolean =>
values.every(
(value, index) =>
index === 0 || compareIdentifiers(values[index - 1] ?? "", value) < 0,
);
const summarize = (
records: readonly EvidenceCompletionRecord[],
): EvidenceCompletionLedger["summary"] => {
const counts: Record<EvidenceCompletionStatus, number> = {
pass: 0,
fail: 0,
unsupported: 0,
truncated: 0,
unknown: 0,
};
for (const record of records) counts[record.status] += 1;
return {
total: records.length,
...counts,
complete: counts.pass === records.length,
};
};
const canonicalJson = (value: unknown): string => {
const serialized = canonicalize(value);
if (serialized === undefined)
throw new TypeError(
"RFC 8785 canonicalization rejected the completion ledger",
);
return serialized;
};
const computeLedgerId = (
records: readonly EvidenceCompletionRecord[],
): string =>
`ecl_${createHash("sha256")
.update(canonicalJson({ schema_version: 1, records }))
.digest("hex")}`;
const ledgerIssues = (
ledger: EvidenceCompletionLedgerValue,
context: z.RefinementCtx,
): void => {
if (!isSortedUnique(ledger.records.map((record) => record.claim_id)))
context.addIssue({
code: "custom",
path: ["records"],
message: "Completion records must be sorted by unique claim ID",
});
for (const [index, record] of ledger.records.entries())
if (!isSortedUnique(record.evidence_ids))
context.addIssue({
code: "custom",
path: ["records", index, "evidence_ids"],
message: "Evidence IDs must be sorted and unique",
});
if (
JSON.stringify(summarize(ledger.records)) !== JSON.stringify(ledger.summary)
)
context.addIssue({
code: "custom",
path: ["summary"],
message: "Completion summary does not match its records",
});
if (computeLedgerId(ledger.records) !== ledger.ledger_id)
context.addIssue({
code: "custom",
path: ["ledger_id"],
message: "Completion ledger identifier does not match its records",
});
};
/** Strict path-free Evidence v2 completion ledger. */
export const evidenceCompletionLedgerSchema =
completionLedgerObjectSchema.superRefine(ledgerIssues);
export type EvidenceCompletionLedger = z.infer<
typeof evidenceCompletionLedgerSchema
>;
/** Build a canonical path-independent ledger from verifier claim outcomes. */
export const createEvidenceCompletionLedger = (
input: readonly EvidenceCompletionRecordInput[],
): EvidenceCompletionLedger => {
const records = input
.map((record) => evidenceCompletionRecordSchema.parse(record))
.map((record) => ({
...record,
evidence_ids: [...record.evidence_ids].sort(compareIdentifiers),
}))
.sort((left, right) => compareIdentifiers(left.claim_id, right.claim_id));
if (!isSortedUnique(records.map((record) => record.claim_id)))
throw new TypeError("Completion claim IDs must be unique");
for (const record of records)
if (!isSortedUnique(record.evidence_ids))
throw new TypeError("Completion Evidence IDs must be unique per claim");
return evidenceCompletionLedgerSchema.parse({
schema_version: 1,
ledger_id: computeLedgerId(records),
records,
summary: summarize(records),
});
};
/** Parse a completion ledger and reject non-canonical or tampered content. */
export const parseEvidenceCompletionLedger = (
input: unknown,
): EvidenceCompletionLedger => evidenceCompletionLedgerSchema.parse(input);
src/domain/processCapture.ts
+27 -1
View File
@@ -5,6 +5,7 @@ import {
LEGACY_PROCESS_CAPTURE_MESSAGE,
normalizationSchema,
} from "./processScenario.js";
import type { ReplayTransitionRecord } from "./replayMachineRuntime.js";
export * from "./processScenario.js";
@@ -100,7 +101,13 @@ export interface ProtocolEvent {
| "matched"
| "unmatched"
| "script_exhausted"
| "disconnected";
| "disconnected"
| "invalid_state"
| "guard_failed"
| "transition_exhausted"
| "invalid_capture"
| "unexpected_reconnect"
| "limit_exhausted";
}
/**
@@ -148,6 +155,7 @@ export interface ProcessCapture {
readonly filesystem_checkpoints: readonly FilesystemCheckpoint[];
readonly shim_events: readonly ShimEvent[];
readonly protocol_events: readonly ProtocolEvent[];
readonly replay_transitions: readonly ReplayTransitionRecord[];
readonly files_before: readonly FileState[];
readonly files_after: readonly FileState[];
readonly filesystem_effects: readonly FileEffect[];
@@ -299,9 +307,27 @@ export const processCaptureSchema: z.ZodType<ProcessCapture> = z.object({
"unmatched",
"script_exhausted",
"disconnected",
"invalid_state",
"guard_failed",
"transition_exhausted",
"invalid_capture",
"unexpected_reconnect",
"limit_exhausted",
]),
}),
),
replay_transitions: z
.array(
z.object({
sequence: z.number().int().nonnegative(),
at_ms: z.number().int().nonnegative(),
transition_id: z.string().min(1).max(64),
state_before: z.string().min(1).max(64),
state_after: z.string().min(1).max(64),
sensitive_aliases: z.array(z.string().min(1).max(64)).max(32),
}),
)
.default([]),
files_before: z.array(fileStateSchema),
files_after: z.array(fileStateSchema),
filesystem_effects: z.array(fileEffectSchema),
src/domain/processCaptureValidation.ts
+94 -15
View File
@@ -1,4 +1,5 @@
import type { ProcessCapture } from "./processCapture.js";
import { replayMachineSchema, type ReplayMachine } from "./replayMachine.js";
import { digestProcessCommitment } from "./processScenario.js";
/** One pure semantic validation failure in an otherwise shaped v4 capture. */
@@ -54,6 +55,7 @@ const validateOrdering = (
["interaction_events", capture.interaction_events],
["shim_events", capture.shim_events],
["protocol_events", capture.protocol_events],
["replay_transitions", capture.replay_transitions],
] as const)
require(values.every(
({ sequence }, index) => sequence === index,
@@ -65,6 +67,7 @@ const validateOrdering = (
["filesystem_checkpoints", capture.filesystem_checkpoints],
["shim_events", capture.shim_events],
["protocol_events", capture.protocol_events],
["replay_transitions", capture.replay_transitions],
] as const)
require(orderedTimestamps(values), name, "timestamps must be ordered");
};
@@ -144,15 +147,34 @@ const validateShimEvents = (
}
};
const validateReplayEvents = (
const isRecord = (value: unknown): value is Readonly<Record<string, unknown>> =>
typeof value === "object" && value !== null;
const machinePlan = (
replayPlan: Readonly<Record<string, unknown>>,
): ReplayMachine | undefined => {
const parsed = replayMachineSchema.safeParse(replayPlan["machine"]);
return parsed.success ? parsed.data : undefined;
};
const matchesHttpPlan = (
value: unknown,
event: ProcessCapture["protocol_events"][number],
): boolean => {
if (!isRecord(value)) return false;
const route = isRecord(value["trigger"]) ? value["trigger"] : value;
return (
(route["protocol"] === undefined || route["protocol"] === "http") &&
route["method"] === event.method &&
route["path"] === event.path
);
};
const validateMatchedHttpEvents = (
capture: ProcessCapture,
plans: readonly unknown[],
require: RequireInvariant,
): void => {
const { replay_plan: replayPlan } = capture.manifest;
const routes =
"http" in replayPlan && Array.isArray(replayPlan.http)
? replayPlan.http
: [];
for (const event of capture.protocol_events) {
if (
event.protocol !== "http" ||
@@ -160,16 +182,73 @@ const validateReplayEvents = (
event.outcome !== "matched"
)
continue;
const declared = routes.some(
(route) =>
typeof route === "object" &&
route !== null &&
"method" in route &&
route.method === event.method &&
"path" in route &&
route.path === event.path,
require(plans.some((plan) =>
matchesHttpPlan(plan, event),
), "protocol_events", "matched HTTP event has no declared replay route");
}
};
const validateReplayEvents = (
capture: ProcessCapture,
require: RequireInvariant,
): void => {
const replayPlan = capture.manifest.replay_plan;
const routes = Array.isArray(replayPlan["http"]) ? replayPlan["http"] : [];
const machine = machinePlan(replayPlan);
require(replayPlan["machine"] === undefined ||
replayPlan["machine"] === null ||
machine !==
undefined, "manifest.replay_plan.machine", "replay machine does not satisfy its declared schema");
validateMatchedHttpEvents(
capture,
[...routes, ...(machine?.transitions ?? [])],
require,
);
const plans = new Map(
(machine?.transitions ?? []).map((transition) => [
transition.id,
transition,
]),
);
const transitionUses = new Map<string, number>();
const stateVisits = new Map<string, number>();
if (machine !== undefined) stateVisits.set(machine.initial_state, 1);
require(machine === undefined ||
capture.replay_transitions.length <=
machine.max_transitions, "replay_transitions", "replay transition journal exceeds the declared machine limit");
for (const [index, transition] of capture.replay_transitions.entries()) {
const plan = plans.get(transition.transition_id);
require(plan !==
undefined, "replay_transitions", "replay transition has no declared machine transition");
if (plan === undefined) continue;
require(transition.state_before === plan.from &&
transition.state_after ===
plan.to, "replay_transitions", "replay transition states do not match the declared transition");
require(JSON.stringify(transition.sensitive_aliases) ===
JSON.stringify(
plan.captures
.filter(({ sensitive }) => sensitive)
.map(({ variable }) => variable)
.sort(),
), "replay_transitions", "replay transition sensitive aliases do not match the declared captures");
const uses = (transitionUses.get(plan.id) ?? 0) + 1;
transitionUses.set(plan.id, uses);
require(uses <=
plan.max_uses, "replay_transitions", "replay transition journal exceeds the transition use limit");
const visits = (stateVisits.get(transition.state_after) ?? 0) + 1;
stateVisits.set(transition.state_after, visits);
const state = machine?.states.find(
({ name }) => name === transition.state_after,
);
require(declared, "protocol_events", "matched HTTP event has no declared replay route");
require(state === undefined ||
visits <=
state.max_visits, "replay_transitions", "replay transition journal exceeds the state visit limit");
if (index === 0)
require(transition.state_before ===
machine?.initial_state, "replay_transitions", "first replay transition does not start at the declared initial state");
if (index > 0)
require(capture.replay_transitions[index - 1]!.state_after ===
transition.state_before, "replay_transitions", "replay transition states are not contiguous");
}
};
src/domain/processComparison.ts
+8 -4
View File
@@ -130,7 +130,7 @@ export const processDimensionObservations = (
case "filesystem":
return filesystemObservations(capture);
case "protocol":
return capture.protocol_events;
return [...capture.protocol_events, ...capture.replay_transitions];
case "process":
return capture.process_samples;
case "shim":
@@ -316,7 +316,11 @@ const compareDimensions = (
filesystemObservations(left),
filesystemObservations(right),
),
protocol: classify("protocol", left.protocol_events, right.protocol_events),
protocol: classify(
"protocol",
[...left.protocol_events, ...left.replay_transitions],
[...right.protocol_events, ...right.replay_transitions],
),
process: classify("process", left.process_samples, right.process_samples),
shim: classify("shim", left.shim_events, right.shim_events),
};
@@ -365,8 +369,8 @@ export const compareProcessCaptures = (
),
firstCollectionDivergence(
"protocol",
left.protocol_events,
right.protocol_events,
[...left.protocol_events, ...left.replay_transitions],
[...right.protocol_events, ...right.replay_transitions],
),
firstCollectionDivergence(
"process",
src/domain/processScenario.ts
+15
View File
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { resolve } from "node:path";
import canonicalize from "canonicalize";
import { z } from "zod";
import { replayMachineSchema } from "./replayMachine.js";
const positiveBudget = z.number().int().positive();
const timedEventBase = { at_ms: z.number().int().nonnegative() };
@@ -241,6 +242,7 @@ export const processScenarioSchema = z
}),
replay: z
.object({
machine: replayMachineSchema.nullable().default(null),
http: z
.array(
z.object({
@@ -285,6 +287,7 @@ export const processScenarioSchema = z
.default([]),
})
.default({
machine: null,
http: [],
websocket_messages: [],
websocket_connections: [],
@@ -361,6 +364,18 @@ export const processScenarioSchema = z
message: "command shim names must be unique",
path: ["command_shims"],
});
if (
scenario.replay.machine !== null &&
(scenario.replay.http.length > 0 ||
scenario.replay.websocket_messages.length > 0 ||
scenario.replay.websocket_connections.length > 0)
)
context.addIssue({
code: "custom",
message:
"a replay machine cannot be combined with static HTTP or WebSocket scripts",
path: ["replay"],
});
});
/** Parsed instructions and resource bounds for one process experiment. */
src/domain/replayMachine.ts
+19 -2
View File
@@ -15,8 +15,17 @@ const sensitiveHeaderNames = new Set([
"proxy-authorization",
"set-cookie",
]);
const headerNameSchema = z
.string()
.min(1)
.max(256)
.regex(/^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/u);
const headerValueSchema = z
.string()
.max(8_192)
.regex(/^[\t\x20-\x7e\x80-\xff]*$/u);
const replayHeaderMatchersSchema = z
.record(z.string().min(1).max(256), z.string().max(8_192))
.record(headerNameSchema, headerValueSchema)
.superRefine((headers, context) => {
const names = Object.keys(headers).map((name) => name.toLowerCase());
if (new Set(names).size !== names.length)
@@ -86,7 +95,7 @@ const replayActionSchema = z.discriminatedUnion("type", [
z.strictObject({
type: z.literal("http_response"),
status: z.number().int().min(100).max(599),
headers: z.record(z.string(), z.string()).default({}),
headers: z.record(headerNameSchema, headerValueSchema).default({}),
body: z.string().max(1_000_000),
}),
z.strictObject({
@@ -220,6 +229,14 @@ const validateReplayVariables = (
const capturedVariables = new Map<string, boolean>();
for (const [transitionIndex, transition] of machine.transitions.entries()) {
for (const capture of transition.captures) {
if (capture.sensitive && capture.value.source === "action_json")
context.addIssue({
code: "custom",
message:
"sensitive replay captures cannot read literal action JSON persisted in the replay plan",
path: ["transitions", transitionIndex, "captures"],
input: machine,
});
const sensitivity = capturedVariables.get(capture.variable);
if (sensitivity !== undefined && sensitivity !== capture.sensitive)
context.addIssue({
src/domain/replayMachineRuntime.ts
+36 -9
View File
@@ -4,12 +4,23 @@ import type { ReplayMachine } from "./replayMachine.js";
type ReplayTransition = ReplayMachine["transitions"][number];
type ReplayValueSource = ReplayTransition["captures"][number]["value"];
interface TransitionRecordOptions {
readonly sequence: number;
readonly atMs: number;
readonly stateBefore: string;
readonly transition: ReplayTransition;
readonly captures: readonly {
readonly variable: string;
readonly sensitive: boolean;
}[];
}
/** One bounded inbound protocol event offered to a replay machine. */
export interface ReplayMachineEvent {
readonly protocol: "http" | "websocket_connect" | "websocket_message";
readonly connection: "not_applicable" | "initial" | "reconnect";
readonly at_ms: number;
readonly recorded_at_ms?: number;
readonly method: string | null;
readonly path: string;
readonly headers: Readonly<Record<string, string>>;
@@ -19,6 +30,7 @@ export interface ReplayMachineEvent {
/** Persistable transition journal entry containing aliases, never secrets. */
export interface ReplayTransitionRecord {
readonly sequence: number;
readonly at_ms: number;
readonly transition_id: string;
readonly state_before: string;
readonly state_after: string;
@@ -141,6 +153,24 @@ const replayValuesEqual = (left: unknown, right: unknown): boolean =>
isJsonCompatible(right) &&
(Object.is(left, right) || canonicalize(left) === canonicalize(right));
const createTransitionRecord = ({
sequence,
atMs,
stateBefore,
transition,
captures,
}: TransitionRecordOptions): ReplayTransitionRecord => ({
sequence,
at_ms: atMs,
transition_id: transition.id,
state_before: stateBefore,
state_after: transition.to,
sensitive_aliases: captures
.filter(({ sensitive }) => sensitive)
.map(({ variable }) => variable)
.sort(),
});
/** Stateful evaluator for one validated replay-machine instance. */
export class ReplayMachineRuntime {
readonly #variables = new Map<string, unknown>();
@@ -267,16 +297,13 @@ export class ReplayMachineRuntime {
this.#sensitiveValues.set(capture.variable, values);
}
}
const record = {
const record = createTransitionRecord({
sequence: this.#timeline.length,
transition_id: transition.id,
state_before: this.#state,
state_after: transition.to,
sensitive_aliases: captures
.filter(({ sensitive }) => sensitive)
.map(({ variable }) => variable)
.sort(),
} satisfies ReplayTransitionRecord;
atMs: normalizedEvent.recorded_at_ms ?? normalizedEvent.at_ms,
stateBefore: this.#state,
transition,
captures,
});
this.#uses.set(transition.id, used + 1);
this.#stateVisits.set(transition.to, nextVisits);
this.#state = transition.to;
src/ghidra/GhidraClient.ts
+13 -2
View File
@@ -9,6 +9,7 @@ import { silentLogger, type Logger } from "../logger.js";
import { PendingOperations } from "../process/PendingOperations.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import { ProviderRunLineage } from "../process/ProviderRunLineage.js";
import {
type ProviderProcessDiagnostic,
type ProviderProcessSnapshot,
@@ -84,6 +85,7 @@ export class GhidraClient {
#snapshotPath: string | undefined;
#token: string | undefined;
#runId: string | undefined;
readonly #lineage = new ProviderRunLineage();
#nextId = 1;
#closing = false;
#processSnapshot: ProviderProcessSnapshot | undefined;
@@ -217,6 +219,11 @@ export class GhidraClient {
return structuredClone(this.#diagnostics());
}
/** Latest token-verified launcher lineage for the active run. */
runtimeLineage() {
return this.#lineage.snapshot();
}
async #start(signal: AbortSignal): Promise<GhidraStartResult> {
if (signal.aborted)
return err(this.#failure("cancelled", "Ghidra startup was cancelled"));
@@ -274,7 +281,8 @@ export class GhidraClient {
}
if (deadline.signal.aborted) return this.#startupInterrupted(deadline);
this.#token = randomBytes(32).toString("hex");
this.#runId = randomUUID();
this.#lineage.reset();
this.#runId = this.#options.runId ?? randomUUID();
const launched = await this.#options.launcher
.launch(
{
@@ -312,7 +320,7 @@ export class GhidraClient {
await this.#cleanup();
return err(failure);
}
return completeGhidraStartupHandshake({
const completed = await completeGhidraStartupHandshake({
deadline,
request: (method, params, requestOptions) =>
this.#wire.request(method, params, requestOptions),
@@ -321,6 +329,8 @@ export class GhidraClient {
failure: this.#failure,
startupTimeoutMs: this.#options.startupTimeoutMs,
});
if (completed.ok) await this.#lineage.observe(this.#launch);
return completed;
}
async #connect(
@@ -497,6 +507,7 @@ export class GhidraClient {
transport: this.#options.transport,
providerVersion: this.#options.providerVersion,
profileDigest: this.#options.profileDigest,
...(this.#runId === undefined ? {} : { runId: this.#runId }),
...(this.#runtimeRoot === undefined
? {}
: { runtimeRoot: this.#runtimeRoot.path }),
src/ghidra/GhidraClientTypes.ts
+1
View File
@@ -33,6 +33,7 @@ export type GhidraDiagnostic =
/** Dependencies and exact profile commitment for one headless import. */
export interface GhidraClientOptions {
readonly launcher: GhidraLauncher;
readonly runId?: string;
readonly targetPath: string;
readonly targetSha256: string;
readonly transport?: GhidraTransportKind;
src/ghidra/GhidraDiagnostics.ts
+2
View File
@@ -12,6 +12,7 @@ export interface GhidraDiagnosticsOptions {
readonly transport: GhidraTransportKind;
readonly providerVersion: string;
readonly profileDigest: string;
readonly runId?: string;
readonly runtimeRoot?: string;
readonly launch?: GhidraLaunch;
readonly snapshot?: ProviderProcessSnapshot;
@@ -39,6 +40,7 @@ export const createGhidraDiagnostics = (
transport: options.transport,
provider_version: options.providerVersion,
profile_digest: options.profileDigest,
...(options.runId === undefined ? {} : { run_id: options.runId }),
...(options.runtimeRoot === undefined
? {}
: {
src/ghidra/GhidraLauncher.ts
+14 -3
View File
@@ -157,9 +157,7 @@ export class GhidraHeadlessLauncher implements GhidraLauncher {
}
const spawned = started;
return ok({
process: spawned.process,
ownsProcessLifetime: true,
cleanup: () => cleanupStartedProcess(spawned, platform),
...ownedGhidraProcess(spawned, platform),
projectRoot: paths.projectRoot,
ghidraLogPath: paths.ghidraLogPath,
scriptLogPath: paths.scriptLogPath,
@@ -176,6 +174,19 @@ export class GhidraHeadlessLauncher implements GhidraLauncher {
}
}
const ownedGhidraProcess = (
spawned: SpawnedOwnedProviderProcess,
platform: NodeJS.Platform,
): Pick<
GhidraLaunch,
"process" | "ownsProcessLifetime" | "ownership" | "cleanup"
> => ({
process: spawned.process,
ownsProcessLifetime: true,
ownership: spawned.ownership,
cleanup: () => cleanupStartedProcess(spawned, platform),
});
/** Exact executable and argv passed to shell-free process creation. */
export interface GhidraHeadlessCommand {
readonly command: string;
src/ghidra/GhidraProvider.ts
+11 -1
View File
@@ -3,6 +3,7 @@ import { fileURLToPath } from "node:url";
import {
createAnalysisExecution,
type AnalysisClient,
type AnalysisClientContext,
type AnalysisOperation,
type AnalysisProfileResolutionOptions,
type AnalysisProviderCandidate,
@@ -69,7 +70,8 @@ const SUPPORTED_FORMATS = new Set(["elf", "pe", "mach-o"]);
/** Production seam for exercising provider projection without a real process. */
export type GhidraProviderClientFactory = (
options: GhidraClientOptions,
) => Pick<GhidraClient, "start" | "callTool" | "close">;
) => Pick<GhidraClient, "start" | "callTool" | "close"> &
Partial<Pick<GhidraClient, "runtimeLineage">>;
/** Ghidra candidate backed by an isolated read-only headless import. */
export class GhidraProvider implements AnalysisProviderCandidate {
@@ -172,6 +174,7 @@ export class GhidraProvider implements AnalysisProviderCandidate {
createClient(
target: BinaryTarget,
profile?: AnalysisProfileCommitment,
context?: AnalysisClientContext,
): AnalysisClient {
const installation = this.#inspectInstallation();
const prerequisites = ghidraClientPrerequisites(
@@ -202,6 +205,7 @@ export class GhidraProvider implements AnalysisProviderCandidate {
: "unix-socket",
providerVersion: prerequisites.value.providerVersion,
profileDigest: committedProfile.digest,
...(context === undefined ? {} : { runId: context.runId }),
logger: this.logger.child({ layer: "ghidra-bridge" }),
});
return {
@@ -254,6 +258,12 @@ export class GhidraProvider implements AnalysisProviderCandidate {
}),
);
},
runtimeLineageSnapshots: () => {
const observation = client.runtimeLineage?.() ?? null;
return observation === null
? []
: [{ provider: committedProfile.provider, observation }];
},
close: () => client.close(),
};
}
src/hopper/BridgeLauncher.ts
+1
View File
@@ -183,6 +183,7 @@ export class HopperApplicationLauncher implements BridgeLauncher {
return ok({
process: child,
ownsProcessLifetime: true,
ownership: started.ownership,
shutdownByCleanup: ownsProcessLifetime,
cleanup: () => cleanupOwnedProcessGroup(started.ownership),
});
src/hopper/HopperClient.ts
+27 -14
View File
@@ -21,11 +21,17 @@ import { PendingOperations } from "../process/PendingOperations.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
import { ProviderRunLineage } from "../process/ProviderRunLineage.js";
import {
type ProviderProcessDiagnostic,
ProviderProcessSupervisor,
} from "../process/ProviderProcess.js";
import type { BridgeLaunch, BridgeLauncher } from "./BridgeLauncher.js";
import {
createOwnedHopperShutdownDiagnostic,
type HopperDiagnostic,
providerCleanupFailure,
} from "./HopperDiagnostics.js";
import {
isHopperCleanupRequired,
isHopperShutdownAcknowledgement,
@@ -45,6 +51,7 @@ const SESSION_ROOT = process.platform === "darwin" ? "/tmp" : tmpdir();
/** Dependencies, deadlines, and redacted diagnostics for one bridge client. */
export interface HopperClientOptions {
readonly launcher: BridgeLauncher;
readonly runId?: string;
readonly requestTimeoutMs?: number;
readonly startupTimeoutMs?: number;
readonly onDiagnostic?: (event: HopperDiagnostic) => void;
@@ -52,10 +59,6 @@ export interface HopperClientOptions {
}
/** Safe launcher telemetry; stderr content is intentionally never exposed. */
type HopperDiagnostic =
| { readonly type: "launcher-stderr"; readonly bytes: number }
| { readonly type: "launcher-exit"; readonly code: number | null };
/**
* Owns one authenticated NDJSON-over-Unix-socket bridge session.
*
@@ -79,6 +82,7 @@ export class HopperClient {
#process: ProviderProcessSupervisor | undefined;
#runtimeRoot: PrivateRuntimeRoot | undefined;
#token: string | undefined;
readonly #lineage = new ProviderRunLineage();
#nextId = 1;
#buffer = "";
#closing = false;
@@ -106,6 +110,11 @@ export class HopperClient {
};
}
/** Latest token-verified launcher lineage for the active run. */
runtimeLineage() {
return this.#lineage.snapshot();
}
/** Launch the bridge once and complete its authenticated health handshake. */
start(signal?: AbortSignal): Promise<Result<HopperServerInfo, HopperError>> {
if (this.#closePromise !== undefined) {
@@ -179,9 +188,10 @@ export class HopperClient {
}
const socketPath = join(this.#runtimeRoot.path, "bridge.sock");
this.#token = randomBytes(32).toString("hex");
this.#lineage.reset();
this.#launcherExitCode = undefined;
this.#launcherFailureDiagnostic = undefined;
const runId = randomUUID();
const runId = this.#options.runId ?? randomUUID();
const launched: Result<
BridgeLaunch,
HopperStartError | HopperCancelledError | HopperProcessError
@@ -229,7 +239,8 @@ export class HopperClient {
: health;
}
const parsed = parseHopperServerInfo(health.value, runId);
if (!parsed.ok) await this.#cleanup();
if (parsed.ok) await this.#lineage.observe(this.#launch);
else await this.#cleanup();
return parsed;
}
@@ -317,6 +328,16 @@ export class HopperClient {
const stopped = await this.#process.stop(
cleanupResult === undefined ? {} : { cleanupResult },
);
const diagnostic = createOwnedHopperShutdownDiagnostic(
this.#process.launch,
stopped,
cleanupResult,
);
this.#options.onDiagnostic?.(diagnostic);
this.#logger.info(
diagnostic,
"Owned Hopper launcher shutdown completed",
);
if (stopped.status === "incomplete")
this.#logger.warn(
{ reason: stopped.reason },
@@ -531,11 +552,3 @@ const startupInterruption = (
deadline.cancelled
? new HopperCancelledError()
: new HopperTimeoutError(deadline.timeoutMs);
const providerCleanupFailure = (cause: unknown): ProcessCleanupResult => ({
cleaned: false,
reason:
cause instanceof Error
? `owned provider cleanup failed: ${cause.message}`
: "owned provider cleanup failed",
});
src/hopper/HopperDiagnostics.ts
+38
View File
@@ -0,0 +1,38 @@
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
import type { ProviderProcessStopResult } from "../process/ProviderProcess.js";
import type { BridgeLaunch } from "./BridgeLauncher.js";
/** Sanitized Hopper launcher and owned-shutdown telemetry. */
export type HopperDiagnostic =
| { readonly type: "launcher-stderr"; readonly bytes: number }
| { readonly type: "launcher-exit"; readonly code: number | null }
| {
readonly type: "owned-shutdown";
readonly status: ProviderProcessStopResult["status"];
readonly launcher_pid: number | null;
readonly process_group_id: number | null;
readonly cleanup_signaled: boolean | null;
readonly reason: string | null;
};
/** Retain actionable ownership coordinates without exposing the run token. */
export const createOwnedHopperShutdownDiagnostic = (
launch: BridgeLaunch,
stopped: ProviderProcessStopResult,
cleanup: ProcessCleanupResult | undefined,
): Extract<HopperDiagnostic, { readonly type: "owned-shutdown" }> => ({
type: "owned-shutdown",
status: stopped.status,
launcher_pid: launch.ownership?.leaderPid ?? launch.process.pid ?? null,
process_group_id: launch.ownership?.processGroupId ?? null,
cleanup_signaled: cleanup?.cleaned === true ? cleanup.signaled : null,
reason: stopped.status === "incomplete" ? stopped.reason : null,
});
/** Convert an unexpected cleanup rejection without exposing a stack or token. */
export const providerCleanupFailure = (
_cause: unknown,
): ProcessCleanupResult => ({
cleaned: false,
reason: "owned provider cleanup failed",
});
src/hopper/HopperProvider.ts
+9
View File
@@ -4,6 +4,7 @@ import { accessSync, constants } from "node:fs";
import { createAnalysisExecution } from "../application/AnalysisProvider.js";
import type {
AnalysisClient,
AnalysisClientContext,
AnalysisProfileResolutionOptions,
AnalysisProviderCandidate,
CapabilityDescriptor,
@@ -186,6 +187,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
createClient(
target: BinaryTarget,
profile?: AnalysisProfileCommitment,
context?: AnalysisClientContext,
): AnalysisClient {
if (target.kind !== "executable" && target.kind !== "database")
return {
@@ -229,6 +231,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
}
: { launchMode: "native" as const }),
}),
...(context === undefined ? {} : { runId: context.runId }),
logger: this.logger.child({ layer: "bridge" }),
});
return {
@@ -246,6 +249,12 @@ export class HopperProvider implements AnalysisProviderCandidate {
}
: result;
},
runtimeLineageSnapshots: () => {
const observation = client.runtimeLineage();
return observation === null
? []
: [{ provider: executionProvider, observation }];
},
close: () => client.close(),
};
}
src/process/ProcessOwnership.ts
+329 -79
View File
@@ -15,8 +15,8 @@ export interface OwnedProcessGroup {
readonly expectedParentPid?: number;
}
/** One operating-system observation of a process-group member. */
interface ProcessGroupMember {
/** One entry from an operating-system process-table snapshot. */
export interface ProcessTableEntry {
readonly pid: number;
readonly parentPid: number;
readonly processGroupId: number;
@@ -24,17 +24,39 @@ interface ProcessGroupMember {
readonly command: string;
}
/** Narrow operating-system seam used to inspect and signal process groups. */
export interface ProcessOwnershipHost {
listMembers(processGroupId: number): Promise<
readonly {
readonly pid: number;
readonly parentPid: number;
/** Token-verified process lineage retained for one owned provider run. */
export interface OwnedProcessLineage {
readonly schemaVersion: 1;
readonly runId: string;
readonly launcherPid: number;
readonly launcherParentPid: number;
readonly processGroupId: number;
readonly descendants: readonly {
readonly pid: number;
readonly parentPid: number;
readonly processGroupId: number;
}[];
}
/** Result of observing owned lineage without signaling any process. */
export type ProcessLineageObservation =
| {
readonly status: "verified";
readonly observedAt: string;
readonly lineage: OwnedProcessLineage;
}
| {
readonly status: "unavailable";
readonly observedAt: string;
readonly runId: string;
readonly launcherPid: number;
readonly processGroupId: number;
readonly state: string;
readonly command: string;
}[]
>;
readonly reason: string;
};
/** Narrow operating-system seam used to inspect processes and signal groups. */
export interface ProcessOwnershipHost {
listProcesses(): Promise<readonly ProcessTableEntry[]>;
environment(pid: number): Promise<Readonly<Record<string, string>>>;
signalGroup(processGroupId: number, signal: NodeJS.Signals): void;
}
@@ -126,7 +148,7 @@ export const parseProcessEnvironment = (
);
const systemHost: ProcessOwnershipHost = {
async listMembers(processGroupId) {
async listProcesses() {
if (process.platform === "win32") return [];
const { stdout } = await execFileAsync("ps", [
"-axo",
@@ -142,8 +164,7 @@ const systemHost: ProcessOwnershipHost = {
processGroupId: Number(match[3]),
state: match[4] ?? "",
command: match[5] ?? "",
}))
.filter((member) => member.processGroupId === processGroupId);
}));
},
async environment(pid) {
if (process.platform === "linux") {
@@ -192,73 +213,186 @@ export const readProcessRunId = async (
): Promise<string | undefined> =>
(await host.environment(pid)).REA_PROCESS_RUN_ID;
/**
* Verify run-token ownership before signaling a POSIX process group.
*
* Process IDs and group IDs can be reused. REA therefore re-reads every live
* member's environment immediately before signaling and fails closed if any
* member cannot be inspected or lacks the per-capture run token.
*/
/** Token-validate every rooted POSIX process group before signaling any. */
export const cleanupOwnedProcessGroup = async (
ownership: OwnedProcessGroup,
host: ProcessOwnershipHost = systemHost,
): Promise<ProcessCleanupResult> => {
let members: readonly ProcessGroupMember[];
let processes: readonly ProcessTableEntry[];
try {
members = await host.listMembers(ownership.processGroupId);
processes = liveProcesses(await host.listProcesses());
} catch {
return { cleaned: false, reason: "process group could not be inspected" };
return { cleaned: false, reason: "process table could not be inspected" };
}
const liveMembers = liveProcessGroupMembers(members);
if (liveMembers.length === 0) return { cleaned: true, signaled: false };
const leader = liveMembers.find(
(member) => member.pid === ownership.leaderPid,
);
if (leader !== undefined) {
if (
ownership.expectedParentPid !== undefined &&
leader.parentPid !== ownership.expectedParentPid
)
return {
cleaned: false,
reason: "owned launcher parent identity did not match",
};
if (
ownership.expectedCommand !== undefined &&
!commandMatches(leader.command, ownership.expectedCommand)
)
return {
cleaned: false,
reason: "owned launcher command identity did not match",
};
}
const failures: ProcessOwnershipValidationFailure[] = [];
for (const member of liveMembers) {
const plan = await createOwnedCleanupPlan(ownership, processes, host);
if ("cleaned" in plan) return plan;
let signaled = false;
for (const processGroupId of plan.signalOrder) {
const revalidation = await revalidateOwnedProcessGroup(
processGroupId,
ownership,
host,
);
if ("cleaned" in revalidation) return revalidation;
if (revalidation.empty) continue;
try {
const environment = await host.environment(member.pid);
if (environment.REA_PROCESS_RUN_ID !== ownership.runId)
host.signalGroup(processGroupId, "SIGKILL");
signaled = true;
} catch (cause: unknown) {
const code =
cause instanceof Error && "code" in cause ? cause.code : undefined;
if (code !== "ESRCH")
return { cleaned: false, reason: "owned process group signal failed" };
}
}
return { cleaned: true, signaled };
};
interface OwnedCleanupPlan {
readonly signalOrder: readonly number[];
}
const createOwnedCleanupPlan = async (
ownership: OwnedProcessGroup,
processes: readonly ProcessTableEntry[],
host: ProcessOwnershipHost,
): Promise<OwnedCleanupPlan | ProcessCleanupResult> => {
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
const rootMembers = processes.filter(
({ processGroupId }) => processGroupId === ownership.processGroupId,
);
if (launcher === undefined && rootMembers.length === 0)
return { cleaned: true, signaled: false };
let descendants: readonly ProcessTableEntry[] = [];
if (launcher !== undefined) {
const identityFailure = launcherIdentityFailure(launcher, ownership);
if (identityFailure !== null)
return { cleaned: false, reason: identityFailure };
descendants = descendantsOf(launcher.pid, processes);
}
const descendantPids = new Set(descendants.map(({ pid }) => pid));
const processGroupIds = new Set<number>([ownership.processGroupId]);
for (const descendant of descendants)
processGroupIds.add(descendant.processGroupId);
for (const processGroupId of processGroupIds) {
if (processGroupId === ownership.processGroupId) continue;
const groupLeader = processes.find(
({ pid, processGroupId: observedGroupId }) =>
pid === processGroupId && observedGroupId === processGroupId,
);
if (groupLeader === undefined || !descendantPids.has(groupLeader.pid))
return {
cleaned: false,
reason:
"descendant process-group leader identity could not be verified",
};
}
const liveMembers = processes.filter(({ processGroupId }) =>
processGroupIds.has(processGroupId),
);
const failures = await processOwnershipFailures(
liveMembers,
ownership.runId,
host,
);
if (failures.length > 0) return cleanupValidationFailure(failures);
return {
signalOrder: [ownership.processGroupId].concat(
[...processGroupIds]
.filter((processGroupId) => processGroupId !== ownership.processGroupId)
.sort((left, right) => left - right),
),
};
};
const launcherIdentityFailure = (
launcher: ProcessTableEntry,
ownership: OwnedProcessGroup,
): string | null => {
if (launcher.processGroupId !== ownership.processGroupId)
return "owned launcher process-group identity did not match";
if (
ownership.expectedParentPid !== undefined &&
launcher.parentPid !== ownership.expectedParentPid
)
return "owned launcher parent identity did not match";
if (
ownership.expectedCommand !== undefined &&
!commandMatches(launcher.command, ownership.expectedCommand)
)
return "owned launcher command identity did not match";
return null;
};
const revalidateOwnedProcessGroup = async (
processGroupId: number,
ownership: OwnedProcessGroup,
host: ProcessOwnershipHost,
): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => {
let members: readonly ProcessTableEntry[];
try {
members = liveProcesses(await host.listProcesses()).filter(
({ processGroupId: observedGroupId }) =>
observedGroupId === processGroupId,
);
} catch {
return {
cleaned: false,
reason: "process ownership could not be revalidated",
};
}
const failures = await processOwnershipFailures(
members,
ownership.runId,
host,
);
if (failures.length > 0) return cleanupValidationFailure(failures);
if (processGroupId === ownership.processGroupId) {
const launcher = members.find(({ pid }) => pid === ownership.leaderPid);
if (launcher !== undefined) {
const identityFailure = launcherIdentityFailure(launcher, ownership);
if (identityFailure !== null)
return { cleaned: false, reason: identityFailure };
}
}
if (
processGroupId !== ownership.processGroupId &&
members.length > 0 &&
!members.some(({ pid }) => pid === processGroupId)
)
return {
cleaned: false,
reason:
"descendant process-group leader identity could not be revalidated",
};
return { empty: members.length === 0 };
};
const cleanupValidationFailure = (
failures: readonly ProcessOwnershipValidationFailure[],
): ProcessCleanupResult => ({
cleaned: false,
reason: failures.some(({ reason }) => reason === "run-token-mismatch")
? "process tree contains an unowned or PID-reused process"
: "process ownership could not be revalidated",
failures,
});
const processOwnershipFailures = async (
members: readonly ProcessTableEntry[],
runId: string,
host: ProcessOwnershipHost,
): Promise<readonly ProcessOwnershipValidationFailure[]> => {
const failures: ProcessOwnershipValidationFailure[] = [];
for (const member of members) {
try {
if ((await host.environment(member.pid)).REA_PROCESS_RUN_ID !== runId)
failures.push({ pid: member.pid, reason: "run-token-mismatch" });
} catch {
failures.push({ pid: member.pid, reason: "environment-unreadable" });
}
}
if (failures.length > 0)
return {
cleaned: false,
reason: failures.some(({ reason }) => reason === "run-token-mismatch")
? "process group contains an unowned or PID-reused process"
: "process ownership could not be revalidated",
failures,
};
try {
host.signalGroup(ownership.processGroupId, "SIGKILL");
} catch (cause: unknown) {
const code =
cause instanceof Error && "code" in cause ? cause.code : undefined;
if (code !== "ESRCH")
return { cleaned: false, reason: "owned process group signal failed" };
}
return { cleaned: true, signaled: true };
return failures;
};
/** Observe one group without signaling it, failing closed on identity doubt. */
@@ -266,16 +400,18 @@ export const observeOwnedProcessGroup = async (
ownership: OwnedProcessGroup,
host: ProcessOwnershipHost = systemHost,
): Promise<ProcessGroupObservation> => {
let members: readonly ProcessGroupMember[];
let members: readonly ProcessTableEntry[];
try {
members = await host.listMembers(ownership.processGroupId);
members = (await host.listProcesses()).filter(
({ processGroupId }) => processGroupId === ownership.processGroupId,
);
} catch {
return {
state: "unverifiable",
reason: "process group could not be inspected",
};
}
const liveMembers = liveProcessGroupMembers(members);
const liveMembers = liveProcesses(members);
if (liveMembers.length === 0) return { state: "empty" };
for (const member of liveMembers) {
try {
@@ -297,16 +433,130 @@ export const observeOwnedProcessGroup = async (
return { state: "alive" };
};
const liveProcessGroupMembers = (
members: readonly ProcessGroupMember[],
): readonly ProcessGroupMember[] =>
/**
* Record the live launcher and descendant lineage after run-token validation.
*
* The observation is intentionally point-in-time. A verified empty descendant
* list means no descendants were live during this observation, not that the
* run never created a short-lived child.
*/
export const observeOwnedProcessLineage = async (
ownership: OwnedProcessGroup,
host: ProcessOwnershipHost = systemHost,
): Promise<ProcessLineageObservation> => {
let processes: readonly ProcessTableEntry[];
try {
processes = liveProcesses(await host.listProcesses());
} catch {
return unavailableLineage(
ownership,
"process table could not be inspected",
);
}
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
if (launcher === undefined)
return unavailableLineage(ownership, "owned launcher is not live");
if (launcher.processGroupId !== ownership.processGroupId)
return unavailableLineage(
ownership,
"owned launcher process-group identity did not match",
);
if (
ownership.expectedParentPid !== undefined &&
launcher.parentPid !== ownership.expectedParentPid
)
return unavailableLineage(
ownership,
"owned launcher parent identity did not match",
);
if (
ownership.expectedCommand !== undefined &&
!commandMatches(launcher.command, ownership.expectedCommand)
)
return unavailableLineage(
ownership,
"owned launcher command identity did not match",
);
const descendants = descendantsOf(launcher.pid, processes);
for (const member of [launcher, ...descendants]) {
try {
if (
(await host.environment(member.pid)).REA_PROCESS_RUN_ID !==
ownership.runId
)
return unavailableLineage(
ownership,
"process lineage contains an unowned or PID-reused process",
);
} catch {
return unavailableLineage(
ownership,
"process ownership could not be revalidated",
);
}
}
return {
status: "verified",
observedAt: new Date().toISOString(),
lineage: {
schemaVersion: 1,
runId: ownership.runId,
launcherPid: launcher.pid,
launcherParentPid: launcher.parentPid,
processGroupId: launcher.processGroupId,
descendants: descendants
.sort((left, right) => left.pid - right.pid)
.map(({ pid, parentPid, processGroupId }) => ({
pid,
parentPid,
processGroupId,
})),
},
};
};
const descendantsOf = (
launcherPid: number,
processes: readonly ProcessTableEntry[],
): ProcessTableEntry[] => {
const childrenByParent = new Map<number, ProcessTableEntry[]>();
for (const process of processes) {
const siblings = childrenByParent.get(process.parentPid) ?? [];
siblings.push(process);
childrenByParent.set(process.parentPid, siblings);
}
const descendants: ProcessTableEntry[] = [];
const pending = [...(childrenByParent.get(launcherPid) ?? [])];
const visited = new Set<number>([launcherPid]);
for (const process of pending) {
if (visited.has(process.pid)) continue;
visited.add(process.pid);
descendants.push(process);
pending.push(...(childrenByParent.get(process.pid) ?? []));
}
return descendants;
};
const unavailableLineage = (
ownership: OwnedProcessGroup,
reason: string,
): Extract<ProcessLineageObservation, { readonly status: "unavailable" }> => ({
status: "unavailable",
observedAt: new Date().toISOString(),
runId: ownership.runId,
launcherPid: ownership.leaderPid,
processGroupId: ownership.processGroupId,
reason,
});
const liveProcesses = (
members: readonly ProcessTableEntry[],
): readonly ProcessTableEntry[] =>
members.filter(({ state }) => !state.startsWith("Z"));
const commandMatches = (actual: string, expected: string): boolean => {
const normalizedActual = actual.trim();
const normalizedExpected = expected.trim();
return (
normalizedActual === normalizedExpected ||
normalizedActual.startsWith(`${normalizedExpected} `)
actual.trim() === expected.trim() ||
actual.trim().startsWith(`${expected.trim()} `)
);
};
src/process/ProviderProcess.ts
+1
View File
@@ -33,6 +33,7 @@ export interface SpawnedOwnedProviderProcess {
export interface ProviderProcessLaunch {
readonly process: ChildProcess;
readonly ownsProcessLifetime: boolean;
readonly ownership?: OwnedProcessGroup;
readonly cleanup?: () => Promise<ProcessCleanupResult>;
}
src/process/ProviderRunLineage.ts
+26
View File
@@ -0,0 +1,26 @@
import type { ProcessLineageObservation } from "./ProcessOwnership.js";
import { observeOwnedProcessLineage } from "./ProcessOwnership.js";
import type { ProviderProcessLaunch } from "./ProviderProcess.js";
/** Retain the latest truthful process-lineage observation for one client. */
export class ProviderRunLineage {
#observation: ProcessLineageObservation | null = null;
/** Clear stale lineage before a new launch attempt. */
reset(): void {
this.#observation = null;
}
/** Observe a token-owned launcher without signaling it. */
async observe(launch: ProviderProcessLaunch | undefined): Promise<void> {
if (launch?.ownership === undefined) return;
this.#observation = await observeOwnedProcessLineage(launch.ownership);
}
/** Return a detached copy suitable for caller-visible status. */
snapshot(): ProcessLineageObservation | null {
return this.#observation === null
? null
: structuredClone(this.#observation);
}
}
src/server/registerEvidenceResources.ts
+5 -1
View File
@@ -451,7 +451,11 @@ const evidenceSection = (
if (section === "process")
return pick(result, ["process_samples", "exit", "settlement", "cleanup"]);
if (section === "protocol")
return pick(result, ["protocol_events", "shim_events"]);
return pick(result, [
"protocol_events",
"replay_transitions",
"shim_events",
]);
const value = result[section];
return value === undefined ? undefined : value;
};
tests/binarySession.test.ts
+82 -7
View File
@@ -32,6 +32,12 @@ describe("binary session", () => {
const [first] = await targets();
const session = new BinarySession(cacheProvider([]));
expect((await session.open(first)).ok).toBe(true);
expect(session.status()).toMatchObject({
analysis_run: {
run_id: expect.any(String),
process_lineage: { status: "not_observed" },
},
});
expect(session.listUnknowns()).toEqual([]);
const identity = session.providerIdentity();
Reflect.set(identity, "id", "forged");
@@ -90,13 +96,36 @@ describe("binary session", () => {
limitations: [],
},
],
createClient: () => ({
execute: (operation) => {
operations.push(operation);
return Promise.resolve(ok(operation));
},
close: () => Promise.resolve(),
}),
createClient: (_target, _profile, context) => {
if (context === undefined)
throw new Error("missing analysis run context");
return {
execute: (operation) => {
operations.push(operation);
return Promise.resolve(ok(operation));
},
runtimeLineageSnapshots: () => [
{
provider: provider.identity(),
observation: {
status: "verified",
observedAt: "2026-07-22T10:00:00.000Z",
lineage: {
schemaVersion: 1,
runId: context.runId,
launcherPid: 100,
launcherParentPid: 1,
processGroupId: 100,
descendants: [
{ pid: 101, parentPid: 100, processGroupId: 100 },
],
},
},
},
],
close: () => Promise.resolve(),
};
},
};
const session = new BinarySession(provider);
expect((await session.open(first)).ok).toBe(true);
@@ -131,11 +160,57 @@ describe("binary session", () => {
},
},
],
analysis_run: {
run_id: expect.stringMatching(
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/u,
),
process_lineage: {
status: "snapshots",
snapshots: [
{
provider: { id: "fixture", name: "Fixture", version: "1" },
observation: {
status: "verified",
observed_at: "2026-07-22T10:00:00.000Z",
schema_version: 1,
launcher_pid: 100,
launcher_parent_pid: 1,
process_group_id: 100,
descendants: [
{ pid: 101, parent_pid: 100, process_group_id: 100 },
],
},
},
],
},
},
});
expect(operations).toEqual(["health", "address_name"]);
await session.close();
});
it("allocates one fresh run identity per provider client lifetime", async () => {
const [first, second] = await targets();
const provider = cacheProvider([]);
const createClient = provider.createClient.bind(provider);
const runIds: string[] = [];
provider.createClient = (target, profile, context) => {
if (context === undefined)
throw new Error("missing analysis run context");
runIds.push(context.runId);
return createClient(target, profile, context);
};
const session = new BinarySession(provider);
expect((await session.open(first)).ok).toBe(true);
expect((await session.open(first)).ok).toBe(true);
expect((await session.open(second)).ok).toBe(true);
expect(runIds).toHaveLength(2);
expect(new Set(runIds).size).toBe(2);
await session.close();
});
it("replays exact immutable calls from a matching provider-neutral snapshot", async () => {
const [first, second] = await targets();
const initialCalls: string[] = [];
tests/completionLedgerGeneration.test.ts
+268
View File
@@ -0,0 +1,268 @@
import { describe, expect, it } from "vitest";
import {
completionVerifierReportSchema,
createCompletionLedgerArtifacts,
parseCompletionManifest,
type CompletionVerifierReport,
} from "../src/domain/completionLedgerGeneration.js";
import { parseEvidenceCompletionLedger } from "../src/domain/evidenceCompletionLedger.js";
const digest = (digit: string): string => digit.repeat(64);
const evidenceId = (digit: string): string => `ev_${digest(digit)}`;
const report = (): CompletionVerifierReport => ({
schema_version: 1,
verifier: { id: "managed-conformance", version: "1" },
verifier_run: {
schema_version: 1,
run_id: "11111111-1111-4111-8111-111111111111",
verifier_pid: 100,
parent_pid: 1,
process_lineage: {
status: "verified",
schema_version: 1,
observed_at: "2026-07-22T10:00:00.000Z",
launcher_pid: 100,
launcher_parent_pid: 1,
process_group_id: 100,
descendants: [{ pid: 101, parent_pid: 100, process_group_id: 100 }],
},
},
environment: {
platform: "linux",
architecture: "x64",
runtime: "node",
runtime_version: "24",
},
claims: [
{
claim_id: "scenario.inspect",
scenario: { id: "inspect-managed", version: 1 },
artifact_sha256s: [digest("1")],
provider: { id: "managed-static", version: "2" },
result_schema_version: 1,
status: "pass",
evidence_ids: [evidenceId("1")],
},
{
claim_id: "scenario.oracle",
scenario: { id: "external-oracle", version: 1 },
artifact_sha256s: [],
provider: { id: "managed-static", version: "2" },
result_schema_version: 1,
status: "unsupported",
evidence_ids: [evidenceId("2")],
},
],
});
const generate = (input: unknown = report(), skillSha256 = digest("a")) =>
createCompletionLedgerArtifacts(input, [
{ skill_id: "reverse-engineer-anything", sha256: skillSha256 },
]);
const firstClaim = (input: CompletionVerifierReport) => {
const claim = input.claims[0];
if (claim === undefined) throw new TypeError("Expected fixture claim");
return claim;
};
describe("completion ledger generation", () => {
it("separates verifier outcomes while preserving immutable commitments", () => {
const generated = generate();
expect(generated.ledger.summary).toEqual({
total: 2,
pass: 1,
fail: 0,
unsupported: 1,
truncated: 0,
unknown: 0,
complete: false,
});
expect(generated.manifest.claims[0]).toMatchObject({
claim_id: "scenario.inspect",
artifact_sha256s: [digest("1")],
scenario: { id: "inspect-managed", version: 1 },
provider: { id: "managed-static", version: "2" },
result_schema_version: 1,
});
expect(generated.manifest.claims[0]).not.toHaveProperty("status");
expect(generated.manifest.claims[0]).not.toHaveProperty("evidence_ids");
});
it("is deterministic regardless of verifier claim ordering", () => {
const input = report();
const reversed = { ...input, claims: [...input.claims].reverse() };
expect(generate(reversed)).toEqual(generate(input));
});
it("keeps ephemeral verifier run identity out of generated commitments", () => {
const first = report();
const second = {
...first,
verifier_run: {
...first.verifier_run,
run_id: "22222222-2222-4222-8222-222222222222",
verifier_pid: 200,
process_lineage: {
status: "unavailable" as const,
observed_at: "2026-07-22T11:00:00.000Z",
launcher_pid: 200,
launcher_parent_pid: 1,
process_group_id: null,
reason: "Windows verifier lineage ownership is unavailable",
},
},
};
expect(generate(second)).toEqual(generate(first));
});
it("requires a typed verifier-process identity", () => {
const input = report();
expect(
completionVerifierReportSchema.safeParse({
...input,
verifier_run: { ...input.verifier_run, run_id: "not-a-uuid" },
}).success,
).toBe(false);
});
it("rejects malformed verified descendant lineage", () => {
const input = report();
expect(
completionVerifierReportSchema.safeParse({
...input,
verifier_run: {
...input.verifier_run,
process_lineage: {
status: "verified",
schema_version: 1,
observed_at: "2026-07-22T10:00:00.000Z",
launcher_pid: 100,
launcher_parent_pid: 1,
process_group_id: 100,
descendants: [
{ pid: 101, parent_pid: 100, process_group_id: null },
],
},
},
}).success,
).toBe(false);
});
it.each([
[
"artifact",
(input: CompletionVerifierReport) => {
firstClaim(input).artifact_sha256s = [digest("3")];
},
],
[
"scenario",
(input: CompletionVerifierReport) => {
firstClaim(input).scenario.version = 2;
},
],
[
"provider",
(input: CompletionVerifierReport) => {
firstClaim(input).provider.version = "3";
},
],
[
"result schema",
(input: CompletionVerifierReport) => {
firstClaim(input).result_schema_version = 2;
},
],
[
"environment",
(input: CompletionVerifierReport) => {
input.environment.runtime_version = "25";
},
],
])("changes the manifest identity on %s drift", (_label, mutate) => {
const baseline = report();
const changed = structuredClone(baseline);
mutate(changed);
expect(generate(changed).manifest.manifest_id).not.toBe(
generate(baseline).manifest.manifest_id,
);
});
it("detects claim deletion and stale skill content", () => {
const baseline = generate();
const input = report();
input.claims.pop();
const deleted = generate(input);
const staleSkill = generate(report(), digest("b"));
expect(deleted.ledger.summary.total).toBe(1);
expect(deleted.manifest.manifest_id).not.toBe(
baseline.manifest.manifest_id,
);
expect(staleSkill.manifest.manifest_id).not.toBe(
baseline.manifest.manifest_id,
);
});
it("rejects duplicate claim, artifact, Evidence, and skill identities", () => {
const duplicateClaim = report();
duplicateClaim.claims.push(structuredClone(firstClaim(duplicateClaim)));
expect(() => generate(duplicateClaim)).toThrow("claim IDs");
const duplicateArtifact = report();
firstClaim(duplicateArtifact).artifact_sha256s = [digest("1"), digest("1")];
expect(() => generate(duplicateArtifact)).toThrow("Artifact digests");
const duplicateEvidence = report();
firstClaim(duplicateEvidence).evidence_ids = [
evidenceId("1"),
evidenceId("1"),
];
expect(() => generate(duplicateEvidence)).toThrow("Evidence IDs");
expect(() =>
createCompletionLedgerArtifacts(report(), [
{ skill_id: "same", sha256: digest("1") },
{ skill_id: "same", sha256: digest("2") },
]),
).toThrow("skill IDs");
});
it("rejects a passing claim without immutable artifact identity", () => {
const input = report();
firstClaim(input).artifact_sha256s = [];
expect(() => generate(input)).toThrow("artifact digest");
});
it("rejects report paths and tampered generated metadata", () => {
const input = report();
expect(() =>
completionVerifierReportSchema.parse({
...input,
local_path: "/private/operator/app.exe",
}),
).toThrow();
const generated = generate();
expect(() =>
parseCompletionManifest({
...generated.manifest,
manifest_id: `ecm_${digest("0")}`,
}),
).toThrow();
expect(() =>
parseEvidenceCompletionLedger({
...generated.ledger,
records: generated.ledger.records.slice(1),
}),
).toThrow();
});
});
tests/compositeProvider.test.ts
+95 -1
View File
@@ -1,4 +1,7 @@
import { describe, expect, it } from "vitest";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { BinarySession } from "../src/application/BinarySession.js";
import { CompositeProvider } from "../src/application/CompositeProvider.js";
@@ -19,6 +22,13 @@ const target: BinaryTarget = {
sha256: "0".repeat(64),
};
let directory: string | undefined;
afterEach(async () => {
if (directory !== undefined)
await rm(directory, { recursive: true, force: true });
directory = undefined;
});
describe("composite analysis provider", () => {
it("routes disjoint operations and does not start children during health", async () => {
const hopperCalls: string[] = [];
@@ -80,8 +90,92 @@ describe("composite analysis provider", () => {
]),
).toThrow(/Multiple providers declare operation address_name/u);
});
it("retains lineage snapshots from two lazily started dynamic providers", async () => {
directory = await mkdtemp(join(tmpdir(), "rea-composite-lineage-"));
const targetPath = join(directory, "fixture.hop");
await writeFile(targetPath, "fixture");
const first = dynamicProvider("first", "address_name", 100);
const second = dynamicProvider("second", "analyze_function", 200);
const session = new BinarySession(new CompositeProvider([second, first]));
expect((await session.open(targetPath)).ok).toBe(true);
expect(await session.execute("address_name", {})).toMatchObject({
ok: true,
});
expect(await session.execute("analyze_function", {})).toMatchObject({
ok: true,
});
expect(session.status()).toMatchObject({
analysis_run: {
process_lineage: {
status: "snapshots",
snapshots: [
{
provider: { id: "first" },
observation: {
status: "verified",
observed_at: "2026-07-22T10:00:00.000Z",
launcher_pid: 100,
},
},
{
provider: { id: "second" },
observation: {
status: "verified",
observed_at: "2026-07-22T10:00:00.000Z",
launcher_pid: 200,
},
},
],
},
},
});
await session.close();
});
});
const dynamicProvider = (
id: string,
operation: Exclude<AnalysisOperation, "health">,
launcherPid: number,
): AnalysisProvider => {
const identity: ProviderIdentity = { id, name: id, version: "1" };
return {
identity: () => identity,
capabilities: () => [capability(identity, operation)],
createClient: (_target, _profile, context) => {
if (context === undefined)
throw new Error("missing analysis run context");
return {
execute: (called) =>
Promise.resolve(
ok(createAnalysisExecution(`${id}:${called}`, identity)),
),
runtimeLineageSnapshots: () => [
{
provider: identity,
observation: {
status: "verified",
observedAt: "2026-07-22T10:00:00.000Z",
lineage: {
schemaVersion: 1,
runId: context.runId,
launcherPid,
launcherParentPid: 1,
processGroupId: launcherPid,
descendants: [],
},
},
},
],
close: () => Promise.resolve(),
};
},
};
};
const provider = (
id: string,
operation: Exclude<AnalysisOperation, "health">,
tests/evidenceCompletionLedger.test.ts
+124
View File
@@ -0,0 +1,124 @@
import { describe, expect, it } from "vitest";
import {
createEvidenceCompletionLedger,
parseEvidenceCompletionLedger,
type EvidenceCompletionStatus,
} from "../src/domain/evidenceCompletionLedger.js";
const evidenceId = (digit: string): string => `ev_${digit.repeat(64)}`;
const record = (
claimId: string,
status: EvidenceCompletionStatus,
evidenceIds: readonly string[] = [evidenceId("1")],
) => ({ claim_id: claimId, status, evidence_ids: evidenceIds });
describe("Evidence v2 completion ledger", () => {
it("marks only an all-pass ledger complete", () => {
const ledger = createEvidenceCompletionLedger([
record("scenario.launch", "pass"),
record("scenario.shutdown", "pass", [evidenceId("2")]),
]);
expect(ledger.summary).toEqual({
total: 2,
pass: 2,
fail: 0,
unsupported: 0,
truncated: 0,
unknown: 0,
complete: true,
});
});
it.each(["fail", "unsupported", "truncated", "unknown"] as const)(
"does not count %s as pass",
(status) => {
const ledger = createEvidenceCompletionLedger([
record("scenario.pass", "pass"),
record(`scenario.${status}`, status, [evidenceId("2")]),
]);
expect(ledger.summary).toMatchObject({ pass: 1, complete: false });
expect(ledger.summary[status]).toBe(1);
},
);
it("canonicalizes claim and Evidence ID ordering", () => {
const left = createEvidenceCompletionLedger([
record("scenario.z", "pass", [evidenceId("2"), evidenceId("1")]),
record("scenario.a", "unknown", [evidenceId("3")]),
]);
const right = createEvidenceCompletionLedger([
record("scenario.a", "unknown", [evidenceId("3")]),
record("scenario.z", "pass", [evidenceId("1"), evidenceId("2")]),
]);
expect(left).toEqual(right);
expect(left.ledger_id).toMatch(/^ecl_[a-f0-9]{64}$/u);
});
it("rejects missing, malformed, and duplicate Evidence IDs", () => {
expect(() =>
createEvidenceCompletionLedger([record("scenario.empty", "pass", [])]),
).toThrow();
expect(() =>
createEvidenceCompletionLedger([
record("scenario.bad", "pass", ["ev_not-a-digest"]),
]),
).toThrow();
expect(() =>
createEvidenceCompletionLedger([
record("scenario.duplicate", "pass", [
evidenceId("1"),
evidenceId("1"),
]),
]),
).toThrow("unique");
});
it("rejects duplicate claims and tampered derived fields", () => {
expect(() =>
createEvidenceCompletionLedger([
record("scenario.same", "pass"),
record("scenario.same", "unknown", [evidenceId("2")]),
]),
).toThrow("unique");
const ledger = createEvidenceCompletionLedger([
record("scenario.valid", "pass"),
]);
expect(() =>
parseEvidenceCompletionLedger({
...ledger,
summary: { ...ledger.summary, complete: false },
}),
).toThrow();
expect(() =>
parseEvidenceCompletionLedger({
...ledger,
ledger_id: `ecl_${"0".repeat(64)}`,
}),
).toThrow();
});
it("rejects path-bearing fields and emits path-independent output", () => {
const ledger = createEvidenceCompletionLedger([
record("scenario.path-free", "pass"),
]);
expect(() =>
parseEvidenceCompletionLedger({
...ledger,
records: [
{
...record("scenario.path-free", "pass"),
local_path: "/private/operator/input.bin",
},
],
}),
).toThrow();
expect(JSON.stringify(ledger)).not.toContain("/private/operator");
});
});
tests/evidenceResources.test.ts
+1
View File
@@ -29,6 +29,7 @@ describe("evidence MCP resources", () => {
result: {
frames: [{ sequence: 0, at_ms: 0, data: "ready" }],
protocol_events: [],
replay_transitions: [],
},
});
session.recordEvidence(capture);
tests/fixtures/fakeHopper.mjs
Vendored
+11 -10
View File
@@ -1,6 +1,13 @@
import { createServer } from "node:net";
const [socketPath, token, runId] = process.argv.slice(2);
const [socketPath, token, runId, shutdownMode = "acknowledge"] =
process.argv.slice(2);
const shutdownResult = () => ({
shutdown: true,
analysis_stopped: true,
document_closed: true,
...(shutdownMode === "cleanup-required" ? { cleanup_required: true } : {}),
});
const enhancedFixtureResult = (method) => {
switch (method) {
case "list_segments":
@@ -126,15 +133,9 @@ const server = createServer((socket) => {
true,
);
} else if (request.method === "shutdown") {
send({
id: request.id,
result: {
shutdown: true,
analysis_stopped: true,
document_closed: true,
},
});
setTimeout(() => server.close(), 2);
send({ id: request.id, result: shutdownResult() });
if (shutdownMode !== "cleanup-required")
setTimeout(() => server.close(), 2);
} else if (request.method === "hang") {
// Deliberately leave the request pending.
} else if (request.method === "exit") {
tests/fixtures/providerProcess.mjs
Vendored
+20
View File
@@ -1,3 +1,5 @@
import { spawn } from "node:child_process";
const [mode, value = "0"] = process.argv.slice(2);
const keepAlive = () => {
@@ -35,6 +37,24 @@ switch (mode) {
await write(process.stdout, "ready\n");
keepAlive();
break;
case "detached-child": {
const child = spawn(
process.execPath,
[
"-e",
"setInterval(() => undefined, 1000); setTimeout(() => process.exit(0), 10000)",
],
{ detached: true, stdio: "ignore" },
);
await new Promise((resolve, reject) => {
child.once("spawn", resolve);
child.once("error", reject);
});
await write(process.stdout, `ready:${child.pid}\n`);
child.unref();
keepAlive();
break;
}
default:
process.stderr.write(`unknown provider-process fixture mode: ${mode}\n`);
process.exitCode = 64;
tests/fixtures/providerProcess.ts
Vendored
+37 -1
View File
@@ -7,7 +7,7 @@ const fixturePath = fileURLToPath(
/** Spawn one deterministic process-lifecycle fixture with piped diagnostics. */
export const spawnProviderProcessFixture = (
mode: "burst" | "exit" | "graceful" | "stubborn",
mode: "burst" | "detached-child" | "exit" | "graceful" | "stubborn",
value?: number,
): ChildProcess =>
spawn(
@@ -52,6 +52,42 @@ export const waitForProviderProcessReady = (
child.once("exit", onExit);
});
/** Read the distinct process-group leader created by the detached-child mode. */
export const waitForDetachedProviderChild = (
child: ChildProcess,
timeoutMs = 2_000,
): Promise<number> =>
new Promise((resolve, reject) => {
const stdout = child.stdout;
if (stdout === null) {
reject(new Error("Provider process fixture stdout is unavailable"));
return;
}
let output = "";
const detach = (): void => {
clearTimeout(timer);
stdout.off("data", onData);
child.off("exit", onExit);
};
const onData = (chunk: Buffer | string): void => {
output += chunk.toString();
const match = /(?:^|\n)ready:(\d+)\n/u.exec(output);
if (match?.[1] === undefined) return;
detach();
resolve(Number(match[1]));
};
const onExit = (): void => {
detach();
reject(new Error("Provider process fixture exited before readiness"));
};
const timer = setTimeout(() => {
detach();
reject(new Error("Provider process fixture readiness timed out"));
}, timeoutMs);
stdout.on("data", onData);
child.once("exit", onExit);
});
/** Kill a fixture if necessary and wait for its process handle to settle. */
export const stopProviderProcessFixture = async (
child: ChildProcess,
tests/ghidraClient.test.ts
+6 -1
View File
@@ -95,6 +95,7 @@ const clientFor = (
readonly requestTimeoutMs?: number;
readonly onDiagnostic?: (event: GhidraDiagnostic) => void;
readonly transport?: GhidraTransportKind;
readonly runId?: string;
} = {},
): GhidraClient => {
const client = new GhidraClient({
@@ -106,6 +107,7 @@ const clientFor = (
profileDigest: PROFILE_DIGEST,
startupTimeoutMs: options.startupTimeoutMs ?? 1_000,
requestTimeoutMs: options.requestTimeoutMs ?? 100,
...(options.runId === undefined ? {} : { runId: options.runId }),
...(options.onDiagnostic === undefined
? {}
: { onDiagnostic: options.onDiagnostic }),
@@ -388,7 +390,9 @@ describe("GhidraClient", () => {
it("retains actionable runtime coordinates after successful cleanup", async () => {
const launcher = new FixtureLauncher();
const client = clientFor(launcher);
const client = clientFor(launcher, {
runId: "11111111-1111-4111-8111-111111111111",
});
await expect(client.start()).resolves.toMatchObject({ ok: true });
await client.close();
@@ -399,6 +403,7 @@ describe("GhidraClient", () => {
endpoint_path: launcher.endpointPaths[0],
project_root: join(launcher.runtimeRoots[0] ?? "", "project"),
process_id: expect.any(Number),
run_id: "11111111-1111-4111-8111-111111111111",
});
await expect(access(launcher.runtimeRoots[0] ?? "")).rejects.toMatchObject({
code: "ENOENT",
tests/ghidraProvider.test.ts
+10 -2
View File
@@ -210,11 +210,12 @@ describe("Ghidra provider", () => {
}),
);
const start = vi.fn().mockResolvedValue(ok(sessionInfo()));
const ghidra = provider(installationHost(), () => ({
const clientFactory = vi.fn(() => ({
start,
callTool,
close: () => Promise.resolve(),
}));
const ghidra = provider(installationHost(), clientFactory);
const resolved = await ghidra.resolveAnalysisProfile(
executableTarget("elf", "x86_64"),
);
@@ -238,8 +239,15 @@ describe("Ghidra provider", () => {
});
const result = await ghidra
.createClient(executableTarget("elf", "x86_64"), resolved.value.profile)
.createClient(executableTarget("elf", "x86_64"), resolved.value.profile, {
runId: "11111111-1111-4111-8111-111111111111",
})
.execute("list_procedures", {});
expect(clientFactory).toHaveBeenCalledWith(
expect.objectContaining({
runId: "11111111-1111-4111-8111-111111111111",
}),
);
expect(callTool).toHaveBeenCalledWith(
"list_procedures",
{ document: null, offset: 0, limit: 100 },
tests/hopperClient.test.ts
+114
View File
@@ -14,6 +14,9 @@ import type {
} from "../src/hopper/BridgeLauncher.js";
import { HopperClient } from "../src/hopper/HopperClient.js";
import { LINUX_PRIVATE_DISPLAY_DIAGNOSTIC_PREFIX } from "../src/hopper/LinuxPrivateDisplayDiagnostic.js";
import { providerCleanupFailure } from "../src/hopper/HopperDiagnostics.js";
import { cleanupOwnedProcessGroup } from "../src/process/ProcessOwnership.js";
import { spawnOwnedProviderProcess } from "../src/process/ProviderProcess.js";
const fixturePath = fileURLToPath(
new URL("./fixtures/fakeHopper.mjs", import.meta.url),
@@ -22,6 +25,7 @@ const fixturePath = fileURLToPath(
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
@@ -29,6 +33,7 @@ class FixtureLauncher implements BridgeLauncher {
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
@@ -68,6 +73,39 @@ class SilentLauncher implements BridgeLauncher {
}
}
class OwnedFixtureLauncher implements BridgeLauncher {
ownership:
| {
readonly runId: string;
readonly leaderPid: number;
readonly processGroupId: number;
}
| undefined;
async launch(session: BridgeSession) {
const started = await spawnOwnedProviderProcess({
command: process.execPath,
arguments: [
fixturePath,
session.socketPath,
session.token,
session.runId,
"cleanup-required",
],
runId: session.runId,
expectedCommand: null,
});
this.ownership = started.ownership;
return ok({
process: started.process,
ownsProcessLifetime: true,
ownership: started.ownership,
shutdownByCleanup: true,
cleanup: () => cleanupOwnedProcessGroup(started.ownership),
});
}
}
class CancelThenFixtureLauncher implements BridgeLauncher {
#launches = 0;
@@ -164,6 +202,30 @@ afterEach(async () => {
});
describe("HopperClient", () => {
it("sanitizes unexpected owned-cleanup exceptions", () => {
expect(
providerCleanupFailure(
new Error("secret-run-token 22222222-2222-4222-8222-222222222222"),
),
).toEqual({
cleaned: false,
reason: "owned provider cleanup failed",
});
});
it("uses the caller-assigned provider run identity", async () => {
const launcher = new FixtureLauncher();
const client = new HopperClient({
launcher,
runId: "11111111-1111-4111-8111-111111111111",
startupTimeoutMs: 1_000,
});
clients.push(client);
expect((await client.start()).ok).toBe(true);
expect(launcher.runIds).toEqual(["11111111-1111-4111-8111-111111111111"]);
});
it("keeps the native socket path below macOS sockaddr_un limits", async () => {
const launcher = new FixtureLauncher();
const client = new HopperClient({ launcher, startupTimeoutMs: 1_000 });
@@ -379,6 +441,49 @@ describe("HopperClient", () => {
await first;
});
it("kills only its owned Hopper group and emits sanitized shutdown coordinates", async () => {
const unrelated = spawn(
process.execPath,
["-e", "process.title = 'Hopper'; setInterval(() => undefined, 1000)"],
{ detached: true, stdio: "ignore" },
);
await new Promise<void>((resolve, reject) => {
unrelated.once("spawn", resolve);
unrelated.once("error", reject);
});
const launcher = new OwnedFixtureLauncher();
const diagnostics: unknown[] = [];
const runId = "22222222-2222-4222-8222-222222222222";
const client = new HopperClient({
launcher,
runId,
startupTimeoutMs: 1_000,
onDiagnostic: (event) => diagnostics.push(event),
});
clients.push(client);
try {
await expect(client.start()).resolves.toMatchObject({ ok: true });
await client.close();
const unrelatedPid = unrelated.pid;
if (unrelatedPid === undefined)
throw new Error("Unrelated Hopper fixture has no PID");
expect(() => process.kill(unrelatedPid, 0)).not.toThrow();
expect(diagnostics).toContainEqual({
type: "owned-shutdown",
status: "verified-cleanup",
launcher_pid: launcher.ownership?.leaderPid,
process_group_id: launcher.ownership?.processGroupId,
cleanup_signaled: true,
reason: null,
});
expect(JSON.stringify(diagnostics)).not.toContain(runId);
} finally {
await stopUnrelatedFixture(unrelated);
}
});
it("makes sequential double-close leave no runtime or process listeners", async () => {
const launcher = new FixtureLauncher();
const client = new HopperClient({ launcher, startupTimeoutMs: 1_000 });
@@ -444,3 +549,12 @@ describe("HopperClient", () => {
});
});
});
const stopUnrelatedFixture = async (child: ChildProcess): Promise<void> => {
if (child.exitCode !== null || child.signalCode !== null) return;
const exited = new Promise<void>((resolve) => {
child.once("exit", () => resolve());
});
child.kill("SIGKILL");
await exited;
};
tests/processCapture.test.ts
+463
View File
@@ -33,6 +33,7 @@ import {
parseProcessCapture,
parseProcessScenario,
processCaptureSchema,
validateProcessCapture,
type ProcessCapture,
type ProcessExecutionPolicy,
} from "../src/domain/processCapture.js";
@@ -98,6 +99,7 @@ const emptyCapture = (): ProcessCapture => {
],
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -383,6 +385,7 @@ describe("process capture domain", () => {
filesystem_checkpoints: emptyCapture().filesystem_checkpoints,
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -423,6 +426,77 @@ describe("process capture domain", () => {
).toThrow("executable_sha256");
});
it.each([
["machine limit", 1, 5, 10],
["transition use limit", 5, 1, 10],
["state visit limit", 5, 5, 1],
])(
"rejects replay journals that exceed the declared %s",
(_name, maxTransitions, maxUses, maxVisits) => {
const replayPlan = {
machine: {
initial_state: "active",
states: [
{ name: "active", max_visits: maxVisits },
{ name: "complete", terminal: true },
],
transitions: [
{
id: "again",
from: "active",
to: "active",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "again",
},
actions: [{ type: "websocket_send", data: "again" }],
max_uses: maxUses,
},
{
id: "finish",
from: "active",
to: "complete",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "finish",
},
actions: [{ type: "websocket_send", data: "done" }],
max_uses: 1,
},
],
max_transitions: maxTransitions,
},
};
const capture = emptyCapture();
const candidate: ProcessCapture = {
...capture,
manifest: {
...capture.manifest,
replay_plan: replayPlan,
replay_plan_sha256: digestProcessCommitment(replayPlan),
},
replay_transitions: [0, 1].map((sequence) => ({
sequence,
at_ms: sequence,
transition_id: "again",
state_before: "active",
state_after: "active",
sensitive_aliases: [],
})),
};
expect(validateProcessCapture(candidate)).toEqual(
expect.arrayContaining([
expect.objectContaining({
message: expect.stringContaining("limit"),
}),
]),
);
},
);
it("tells agents and users to recapture unsupported v3 evidence", () => {
const legacy = { ...emptyCapture(), schema_version: 3 };
expect(() => parseProcessCapture(legacy)).toThrow(
@@ -478,6 +552,7 @@ describe("process capture domain", () => {
filesystem_checkpoints: emptyCapture().filesystem_checkpoints,
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -537,6 +612,7 @@ describe("process capture domain", () => {
filesystem_checkpoints: emptyCapture().filesystem_checkpoints,
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -580,6 +656,7 @@ describe("process capture domain", () => {
filesystem_checkpoints: emptyCapture().filesystem_checkpoints,
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -652,6 +729,7 @@ describe("process capture domain", () => {
filesystem_checkpoints: emptyCapture().filesystem_checkpoints,
shim_events: [],
protocol_events: [],
replay_transitions: [],
files_before: [],
files_after: [],
filesystem_effects: [],
@@ -681,6 +759,63 @@ describe("process capture domain", () => {
expect(comparison.process).toBe("changed");
expect(comparison.status).toBe("changed");
});
it("compares replay state transitions as protocol evidence", () => {
const replayPlan = {
machine: {
initial_state: "anonymous",
states: [
{ name: "anonymous" },
{ name: "authenticated", terminal: true },
],
transitions: [
{
id: "login",
from: "anonymous",
to: "authenticated",
trigger: { protocol: "http", method: "POST", path: "/login" },
captures: [
{
variable: "token",
value: { source: "request_json", path: ["token"] },
sensitive: true,
},
],
actions: [{ type: "http_response", status: 204, body: "" }],
max_uses: 1,
},
],
max_transitions: 1,
},
};
const empty = emptyCapture();
const baseline = {
...empty,
manifest: {
...empty.manifest,
replay_plan: replayPlan,
replay_plan_sha256: digestProcessCommitment(replayPlan),
},
};
const transitioned = {
...baseline,
replay_transitions: [
{
sequence: 0,
at_ms: 0,
transition_id: "login",
state_before: "anonymous",
state_after: "authenticated",
sensitive_aliases: ["token"],
},
],
};
expect(compareProcessCaptures(baseline, transitioned)).toMatchObject({
status: "changed",
protocol: "added",
});
});
});
describe("process capture adapter", () => {
@@ -814,6 +949,334 @@ describe("process capture adapter", () => {
}
});
it("serializes WebSocket transition actions across rapid messages", async () => {
const scenario = parseProcessScenario({
approved: true,
executable: "/bin/sh",
working_directory: "/tmp",
replay: {
machine: {
initial_state: "connecting",
states: [
{ name: "connecting" },
{ name: "first" },
{ name: "second" },
{ name: "complete", terminal: true },
],
transitions: [
{
id: "connect",
from: "connecting",
to: "first",
trigger: { protocol: "websocket_connect", path: "/ws" },
actions: [{ type: "websocket_send", data: "ready" }],
max_uses: 1,
},
{
id: "first",
from: "first",
to: "second",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "first",
},
actions: [
{ type: "delay", duration_ms: 30 },
{ type: "websocket_send", data: "first-done" },
],
max_uses: 1,
},
{
id: "second",
from: "second",
to: "complete",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "second",
},
actions: [{ type: "websocket_send", data: "second-done" }],
max_uses: 1,
},
],
max_transitions: 3,
},
},
});
const replay = await startLoopbackReplay(scenario);
try {
const messages = await new Promise<string[]>((resolve, reject) => {
const received: string[] = [];
const socket = new WebSocket(replay.websocketUrl);
socket.on("message", (value) => {
received.push(value.toString());
if (received.length === 1) {
socket.send("first");
socket.send("second");
}
if (received.length === 3) {
socket.close();
resolve(received);
}
});
socket.once("error", reject);
});
expect(messages).toEqual(["ready", "first-done", "second-done"]);
} finally {
await replay.close();
}
});
it("drains queued machine work before exposing finalized snapshots", async () => {
const scenario = parseProcessScenario({
approved: true,
executable: "/bin/sh",
working_directory: "/tmp",
replay: {
machine: {
initial_state: "connecting",
states: [
{ name: "connecting" },
{ name: "first" },
{ name: "second" },
{ name: "complete", terminal: true },
],
transitions: [
{
id: "connect",
from: "connecting",
to: "first",
trigger: { protocol: "websocket_connect", path: "/ws" },
actions: [{ type: "websocket_send", data: "ready" }],
max_uses: 1,
},
{
id: "first",
from: "first",
to: "second",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "first",
},
actions: [
{ type: "delay", duration_ms: 30 },
{ type: "disconnect" },
],
max_uses: 1,
},
{
id: "second",
from: "second",
to: "complete",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "second",
},
actions: [{ type: "websocket_send", data: "done" }],
max_uses: 1,
},
],
max_transitions: 3,
},
},
});
const replay = await startLoopbackReplay(scenario);
await new Promise<void>((resolve, reject) => {
const socket = new WebSocket(replay.websocketUrl);
socket.once("message", () => {
socket.send("first");
socket.send("second");
setTimeout(resolve, 5);
});
socket.once("error", reject);
});
await replay.close();
const finalized = replay.transitions;
expect(finalized.map(({ transition_id }) => transition_id)).toEqual([
"connect",
"first",
"second",
]);
await new Promise((resolve) => setTimeout(resolve, 40));
expect(replay.transitions).toEqual(finalized);
});
it("enforces replay duration before normalizing recorded timestamps", async () => {
const scenario = parseProcessScenario({
approved: true,
executable: "/bin/sh",
working_directory: "/tmp",
normalization: { time_bucket_ms: 60_000 },
replay: {
machine: {
initial_state: "waiting",
states: [{ name: "waiting" }, { name: "complete", terminal: true }],
transitions: [
{
id: "late",
from: "waiting",
to: "complete",
trigger: { protocol: "http", method: "GET", path: "/late" },
actions: [{ type: "http_response", status: 200, body: "late" }],
max_uses: 1,
},
],
max_transitions: 1,
limits: { duration_ms: 5 },
},
},
});
const replay = await startLoopbackReplay(scenario);
try {
await new Promise((resolve) => setTimeout(resolve, 20));
const response = await fetch(`${replay.httpUrl}/late`);
expect(response.status).toBe(409);
expect(replay.events.at(-1)).toMatchObject({
at_ms: 0,
outcome: "limit_exhausted",
});
} finally {
await replay.close();
}
});
it("runs a login and reconnect replay machine inside process capture", async () => {
const root = await createTestTempDirectory("rea-replay-machine-test-");
const script = join(root, "client.mjs");
await writeFile(
script,
[
'const token = "machine-secret";',
'await fetch(`${process.env.REA_REPLAY_HTTP_URL}/login`, { method: "POST", body: JSON.stringify({ token }) });',
"const api = await fetch(`${process.env.REA_REPLAY_HTTP_URL}/api`, { headers: { authorization: token } });",
'if (await api.text() !== "connect") throw new Error("bad API response");',
"const connect = (acknowledge) => new Promise((resolve, reject) => {",
" const socket = new WebSocket(process.env.REA_REPLAY_WEBSOCKET_URL);",
" socket.addEventListener('error', reject);",
" socket.addEventListener('message', (event) => {",
" if (acknowledge) socket.send('ack');",
" else { console.log(`reply:${event.data}`); socket.close(); resolve(); }",
" });",
" if (acknowledge) socket.addEventListener('close', () => resolve());",
"});",
"await connect(true);",
"await connect(false);",
].join("\n"),
);
const scenario = parseProcessScenario({
approved: true,
executable: process.execPath,
arguments: [script],
working_directory: root,
replay: {
machine: {
initial_state: "login",
states: [
{ name: "login" },
{ name: "api" },
{ name: "socket" },
{ name: "ack" },
{ name: "reconnect" },
{ name: "complete", terminal: true },
],
transitions: [
{
id: "login",
from: "login",
to: "api",
trigger: { protocol: "http", method: "POST", path: "/login" },
captures: [
{
variable: "token",
value: { source: "request_json", path: ["token"] },
sensitive: true,
},
],
actions: [{ type: "http_response", status: 204, body: "" }],
max_uses: 1,
},
{
id: "api",
from: "api",
to: "socket",
trigger: { protocol: "http", method: "GET", path: "/api" },
guards: [
{
variable: "token",
value: { source: "request_header", name: "authorization" },
},
],
actions: [
{ type: "http_response", status: 200, body: "connect" },
],
max_uses: 1,
},
{
id: "initial_socket",
from: "socket",
to: "ack",
trigger: { protocol: "websocket_connect", path: "/ws" },
actions: [{ type: "websocket_send", data: "welcome" }],
max_uses: 1,
},
{
id: "ack",
from: "ack",
to: "reconnect",
trigger: {
protocol: "websocket_message",
path: "/ws",
body: "ack",
},
actions: [{ type: "disconnect" }],
max_uses: 1,
},
{
id: "reconnect",
from: "reconnect",
to: "complete",
trigger: { protocol: "websocket_connect", path: "/ws" },
actions: [{ type: "websocket_send", data: "done" }],
max_uses: 1,
},
],
max_transitions: 5,
},
},
});
try {
const capability = await probeProcessCaptureCapability();
if (!capability.available) return;
const capture = await captureProcessScenario(scenario, {
enabled: true,
executableRoots: [dirname(process.execPath)],
workingRoots: [root],
allowedEnvironment: [],
allowExternalNetwork: true,
});
expect(capture.ok).toBe(true);
if (!capture.ok) throw capture.error;
expect(
capture.value.replay_transitions.map(
({ transition_id }) => transition_id,
),
).toEqual(["login", "api", "initial_socket", "ack", "reconnect"]);
expect(capture.value.frames.map(({ data }) => data).join("")).toContain(
"reply:done",
);
expect(JSON.stringify(capture.value)).not.toContain("machine-secret");
expect(capture.value.manifest.replay_plan_sha256).toBe(
digestProcessCommitment(scenario.replay),
);
} finally {
await rm(root, { recursive: true, force: true });
}
}, 20_000);
it("captures PTY, filesystem, descendants, HTTP replay, and redacts environment", async () => {
const root = await createTestTempDirectory("rea-harness-test-");
const script = join(root, "fixture.mjs");
tests/processOwnership.test.ts
+300 -8
View File
@@ -2,6 +2,7 @@ import { describe, expect, it, vi } from "vitest";
import {
cleanupOwnedProcessGroup,
cleanupWindowsProcessTree,
observeOwnedProcessLineage,
observeOwnedProcessGroup,
parseProcessEnvironment,
selectCapturedProcessGroupIds,
@@ -24,7 +25,7 @@ const host = (
const signalGroup = vi.fn();
return {
adapter: {
listMembers: () =>
listProcesses: () =>
Promise.resolve(
Object.keys(environments).map((pid) => ({
pid: Number(pid),
@@ -72,6 +73,298 @@ describe("owned process-group cleanup", () => {
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
});
it("validates and signals rooted descendant groups once, root first", async () => {
const processes = [
{ pid: 100, parentPid: 1, processGroupId: 100 },
{ pid: 110, parentPid: 100, processGroupId: 100 },
{ pid: 101, parentPid: 100, processGroupId: 101 },
{ pid: 102, parentPid: 101, processGroupId: 102 },
{ pid: 103, parentPid: 102, processGroupId: 102 },
].map((process) => ({
...process,
state: "S",
command: "fixture",
}));
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve(processes),
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{ cleaned: true, signaled: true },
);
expect(signalGroup.mock.calls).toEqual([
[100, "SIGKILL"],
[101, "SIGKILL"],
[102, "SIGKILL"],
]);
});
it("leaves an unrelated Hopper process group outside the rooted tree untouched", async () => {
const processes = [
{ pid: 100, parentPid: 1, processGroupId: 100, command: "rea-hopper" },
{ pid: 101, parentPid: 100, processGroupId: 101, command: "owned-child" },
{
pid: 900,
parentPid: 1,
processGroupId: 900,
command:
"/Applications/Hopper Disassembler.app/Contents/MacOS/Hopper Disassembler",
},
].map((process) => ({ ...process, state: "S" }));
const environment = vi.fn((pid: number) =>
Promise.resolve({
REA_PROCESS_RUN_ID: pid === 900 ? "unrelated-run" : "run-token",
}),
);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () => Promise.resolve(processes),
environment,
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{ cleaned: true, signaled: true },
);
expect(signalGroup.mock.calls).toEqual([
[100, "SIGKILL"],
[101, "SIGKILL"],
]);
expect(environment.mock.calls.some(([pid]) => pid === 900)).toBe(false);
});
it("fails before signaling when any descendant-group member is unowned", async () => {
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () =>
Promise.resolve([
{
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{
pid: 101,
parentPid: 100,
processGroupId: 101,
state: "S",
command: "child-session",
},
{
pid: 104,
parentPid: 101,
processGroupId: 101,
state: "S",
command: "child-helper",
},
]),
environment: (pid) =>
Promise.resolve({
REA_PROCESS_RUN_ID: pid === 104 ? "other-run" : "run-token",
}),
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{
cleaned: false,
reason: "process tree contains an unowned or PID-reused process",
failures: [{ pid: 104, reason: "run-token-mismatch" }],
},
);
expect(signalGroup).not.toHaveBeenCalled();
});
it("fails before signaling when a descendant group leader is not rooted", async () => {
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listProcesses: () =>
Promise.resolve([
{
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{
pid: 105,
parentPid: 100,
processGroupId: 101,
state: "S",
command: "descendant",
},
{
pid: 101,
parentPid: 999,
processGroupId: 101,
state: "S",
command: "unrelated-group-leader",
},
]),
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
signalGroup,
};
await expect(cleanupOwnedProcessGroup(ownership, adapter)).resolves.toEqual(
{
cleaned: false,
reason:
"descendant process-group leader identity could not be verified",
},
);
expect(signalGroup).not.toHaveBeenCalled();
});
it("records sorted launcher and descendant lineage after token checks", async () => {
const { adapter } = host({
102: { REA_PROCESS_RUN_ID: "run-token" },
100: { REA_PROCESS_RUN_ID: "run-token" },
101: { REA_PROCESS_RUN_ID: "run-token" },
});
await expect(
observeOwnedProcessLineage(ownership, adapter),
).resolves.toEqual({
status: "verified",
observedAt: expect.any(String),
lineage: {
schemaVersion: 1,
runId: "run-token",
launcherPid: 100,
launcherParentPid: 1,
processGroupId: 100,
descendants: [
{ pid: 101, parentPid: 100, processGroupId: 100 },
{ pid: 102, parentPid: 100, processGroupId: 100 },
],
},
});
});
it("walks the PPID tree across process-group boundaries", async () => {
const environment = vi.fn((pid: number) =>
Promise.resolve({
REA_PROCESS_RUN_ID: pid === 103 ? "unrelated-run" : "run-token",
}),
);
const adapter: ProcessOwnershipHost = {
listProcesses: () =>
Promise.resolve([
{
pid: 103,
parentPid: 999,
processGroupId: 100,
state: "S",
command: "unrelated-same-group",
},
{
pid: 102,
parentPid: 101,
processGroupId: 102,
state: "S",
command: "grandchild-session",
},
{
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{
pid: 101,
parentPid: 100,
processGroupId: 101,
state: "S",
command: "child-group",
},
]),
environment,
signalGroup: vi.fn(),
};
await expect(
observeOwnedProcessLineage(ownership, adapter),
).resolves.toEqual({
status: "verified",
observedAt: expect.any(String),
lineage: {
schemaVersion: 1,
runId: "run-token",
launcherPid: 100,
launcherParentPid: 1,
processGroupId: 100,
descendants: [
{ pid: 101, parentPid: 100, processGroupId: 101 },
{ pid: 102, parentPid: 101, processGroupId: 102 },
],
},
});
expect(environment.mock.calls.map(([pid]) => pid)).toEqual([100, 101, 102]);
});
it("fails closed when a descendant in another process group lacks the token", async () => {
const adapter: ProcessOwnershipHost = {
listProcesses: () =>
Promise.resolve([
{
pid: 100,
parentPid: 1,
processGroupId: 100,
state: "S",
command: "fixture",
},
{
pid: 101,
parentPid: 100,
processGroupId: 101,
state: "S",
command: "child-session",
},
]),
environment: (pid) =>
Promise.resolve({
REA_PROCESS_RUN_ID: pid === 100 ? "run-token" : "other-run",
}),
signalGroup: vi.fn(),
};
await expect(
observeOwnedProcessLineage(ownership, adapter),
).resolves.toEqual({
status: "unavailable",
observedAt: expect.any(String),
runId: "run-token",
launcherPid: 100,
processGroupId: 100,
reason: "process lineage contains an unowned or PID-reused process",
});
});
it("does not publish lineage when any member fails ownership checks", async () => {
const { adapter } = host({
100: { REA_PROCESS_RUN_ID: "run-token" },
101: { REA_PROCESS_RUN_ID: "other-run" },
});
await expect(
observeOwnedProcessLineage(ownership, adapter),
).resolves.toEqual({
status: "unavailable",
observedAt: expect.any(String),
runId: "run-token",
launcherPid: 100,
processGroupId: 100,
reason: "process lineage contains an unowned or PID-reused process",
});
});
it("fails closed for stale metadata or an unrelated concurrent process", async () => {
const { adapter, signalGroup } = host({
100: { REA_PROCESS_RUN_ID: "run-token" },
@@ -79,7 +372,7 @@ describe("owned process-group cleanup", () => {
});
expect(await cleanupOwnedProcessGroup(ownership, adapter)).toEqual({
cleaned: false,
reason: "process group contains an unowned or PID-reused process",
reason: "process tree contains an unowned or PID-reused process",
failures: [{ pid: 101, reason: "run-token-mismatch" }],
});
expect(signalGroup).not.toHaveBeenCalled();
@@ -97,7 +390,7 @@ describe("owned process-group cleanup", () => {
});
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listMembers: () =>
listProcesses: () =>
Promise.resolve(
[100, 101, 102].map((pid) => ({
pid,
@@ -113,7 +406,7 @@ describe("owned process-group cleanup", () => {
expect(await cleanupOwnedProcessGroup(ownership, adapter)).toEqual({
cleaned: false,
reason: "process group contains an unowned or PID-reused process",
reason: "process tree contains an unowned or PID-reused process",
failures: [
{ pid: 100, reason: "environment-unreadable" },
{ pid: 101, reason: "run-token-mismatch" },
@@ -129,7 +422,7 @@ describe("owned process-group cleanup", () => {
);
const signalGroup = vi.fn();
const adapter: ProcessOwnershipHost = {
listMembers: () =>
listProcesses: () =>
Promise.resolve([
{
pid: 100,
@@ -154,15 +447,14 @@ describe("owned process-group cleanup", () => {
cleaned: true,
signaled: true,
});
expect(environment).toHaveBeenCalledTimes(1);
expect(environment).toHaveBeenCalledWith(100);
expect(environment.mock.calls).toEqual([[100], [100]]);
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
});
it("observes a zombie-only group as settled", async () => {
const environment = vi.fn(() => Promise.resolve({}));
const adapter: ProcessOwnershipHost = {
listMembers: () =>
listProcesses: () =>
Promise.resolve([
{
pid: 101,
tests/providerProcess.test.ts
+77
View File
@@ -8,6 +8,10 @@ import { afterEach, describe, expect, it, vi } from "vitest";
import { PendingOperations } from "../src/process/PendingOperations.js";
import { PrivateRuntimeRoot } from "../src/process/PrivateRuntimeRoot.js";
import { ProviderStartupDeadline } from "../src/process/ProviderDeadline.js";
import {
cleanupOwnedProcessGroup,
observeOwnedProcessLineage,
} from "../src/process/ProcessOwnership.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
@@ -16,6 +20,7 @@ import {
import {
spawnProviderProcessFixture,
stopProviderProcessFixture,
waitForDetachedProviderChild,
waitForProviderProcessReady,
} from "./fixtures/providerProcess.js";
@@ -23,6 +28,24 @@ const processFixturePath = fileURLToPath(
new URL("./fixtures/providerProcess.mjs", import.meta.url),
);
const waitForPidExit = async (
pid: number,
timeoutMs = 2_000,
): Promise<void> => {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
try {
process.kill(pid, 0);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ESRCH")
return;
throw cause;
}
await new Promise((resolve) => setTimeout(resolve, 10));
}
throw new Error(`Process ${String(pid)} did not exit within the test bound`);
};
afterEach(() => {
vi.useRealTimers();
});
@@ -254,6 +277,60 @@ describe("provider process lifecycle primitives", () => {
}
});
it.skipIf(process.platform === "win32")(
"cleans a real descendant that starts a distinct process group",
async () => {
const spawned = await spawnOwnedProviderProcess({
command: process.execPath,
arguments: [processFixturePath, "detached-child"],
runId: "provider-process-lineage-run",
});
let detachedChildPid: number | undefined;
try {
detachedChildPid = await waitForDetachedProviderChild(spawned.process);
const observation = await observeOwnedProcessLineage(spawned.ownership);
expect(observation).toEqual({
status: "verified",
observedAt: expect.stringMatching(
/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/u,
),
lineage: {
schemaVersion: 1,
runId: "provider-process-lineage-run",
launcherPid: spawned.process.pid,
launcherParentPid: process.pid,
processGroupId: spawned.process.pid,
descendants: [
{
pid: detachedChildPid,
parentPid: spawned.process.pid,
processGroupId: detachedChildPid,
},
],
},
});
await expect(
cleanupOwnedProcessGroup(spawned.ownership),
).resolves.toEqual({ cleaned: true, signaled: true });
await Promise.all([
waitForPidExit(spawned.ownership.leaderPid),
waitForPidExit(detachedChildPid),
]);
} finally {
await cleanupOwnedProcessGroup(spawned.ownership);
if (detachedChildPid !== undefined)
await cleanupOwnedProcessGroup({
runId: spawned.ownership.runId,
leaderPid: detachedChildPid,
processGroupId: detachedChildPid,
});
await waitForPidExit(spawned.ownership.leaderPid);
if (detachedChildPid !== undefined)
await waitForPidExit(detachedChildPid);
}
},
);
it("allows interpreter launchers to rely on parent and run-token identity", async () => {
const spawned = await spawnOwnedProviderProcess({
command: process.execPath,
tests/providerSelectionMcp.test.ts
+7
View File
@@ -62,6 +62,7 @@ describe("provider selection over MCP", () => {
);
expect(before.result).toMatchObject({
open: false,
analysis_run: null,
analysis_provider_binding: null,
analysis_provider_candidates: [
{
@@ -128,6 +129,12 @@ describe("provider selection over MCP", () => {
);
expect(status.result).toMatchObject({
open: true,
analysis_run: {
run_id: expect.stringMatching(
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/u,
),
process_lineage: { status: "not_observed" },
},
analysis_provider_binding: {
provider: { id: "beta", version: "1" },
selection_source: "request",
tests/replayMachine.test.ts
+24
View File
@@ -159,6 +159,30 @@ describe("replay machine", () => {
);
});
it("rejects sensitive captures sourced from persisted action JSON", () => {
const candidate = loginMachine();
const capture = candidate.transitions[0]?.captures?.[0];
if (capture === undefined) throw new TypeError("missing capture");
capture.value = { source: "action_json", path: ["token"] };
expect(() => replayMachineSchema.parse(candidate)).toThrow(
/literal action JSON/u,
);
});
it.each([
["invalid header name", { "bad header": "value" }],
["invalid header value", { "x-test": "value\nsmuggled" }],
])("rejects %s in HTTP responses", (_name, headers) => {
const candidate = loginMachine();
const action = candidate.transitions[0]?.actions[0];
if (action === undefined || action.type !== "http_response")
throw new TypeError("missing HTTP response");
action.headers = headers;
expect(() => replayMachineSchema.parse(candidate)).toThrow();
});
it("rejects unknown fields at nested machine boundaries", () => {
const candidate = loginMachine();
const action = candidate.transitions[0]?.actions[0];
tests/verifierRun.test.ts
+138
View File
@@ -0,0 +1,138 @@
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import {
completeVerifierRun,
createVerifierRun,
} from "../scripts/lib/verifier-run.mjs";
const execFileAsync = promisify(execFile);
describe.sequential("verifier run identity", () => {
it("allocates, propagates, and reuses one process-local identity", async () => {
const previousRunId = process.env.REA_PROCESS_RUN_ID;
process.env.REA_PROCESS_RUN_ID = "11111111-1111-4111-8111-111111111111";
try {
const first = createVerifierRun();
const nested = createVerifierRun();
expect(first).toEqual({
schema_version: 1,
run_id: expect.stringMatching(
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/u,
),
verifier_pid: process.pid,
parent_pid: process.ppid,
});
expect(nested.run_id).toBe(first.run_id);
expect(first.run_id).not.toBe("11111111-1111-4111-8111-111111111111");
expect(process.env.REA_PROCESS_RUN_ID).toBe(first.run_id);
const { stdout } = await execFileAsync(process.execPath, [
"-e",
"process.stdout.write(process.env.REA_PROCESS_RUN_ID ?? '')",
]);
expect(stdout).toBe(first.run_id);
const { stdout: nestedProcessRunId } = await execFileAsync(
process.execPath,
[
"--input-type=module",
"-e",
"import { createVerifierRun } from './scripts/lib/verifier-run.mjs'; process.stdout.write(createVerifierRun().run_id)",
],
);
expect(nestedProcessRunId).not.toBe(first.run_id);
} finally {
restoreRunId(previousRunId);
}
});
it.runIf(process.platform !== "win32")(
"reports token-verified live descendants",
async () => {
const previousRunId = process.env.REA_PROCESS_RUN_ID;
delete process.env.REA_PROCESS_RUN_ID;
const run = createVerifierRun();
const child = spawnReadyChild(process.env);
try {
await childReady(child);
const completed = await completeVerifierRun(run);
expect(completed.process_lineage).toMatchObject({
status: "verified",
schema_version: 1,
observed_at: expect.stringMatching(/Z$/u),
launcher_pid: process.pid,
launcher_parent_pid: process.ppid,
process_group_id: expect.any(Number),
descendants: expect.arrayContaining([
{
pid: child.pid,
parent_pid: process.pid,
process_group_id: expect.any(Number),
},
]),
});
} finally {
await stopChild(child);
restoreRunId(previousRunId);
}
},
);
it.runIf(process.platform !== "win32")(
"reports unavailable when a live descendant lacks the run token",
async () => {
const previousRunId = process.env.REA_PROCESS_RUN_ID;
delete process.env.REA_PROCESS_RUN_ID;
const run = createVerifierRun();
const child = spawnReadyChild({ ...process.env, REA_PROCESS_RUN_ID: "" });
try {
await childReady(child);
const completed = await completeVerifierRun(run);
expect(completed.process_lineage).toMatchObject({
status: "unavailable",
observed_at: expect.stringMatching(/Z$/u),
launcher_pid: process.pid,
launcher_parent_pid: process.ppid,
process_group_id: expect.any(Number),
reason: `descendant ${String(child.pid)} run token did not match`,
});
} finally {
await stopChild(child);
restoreRunId(previousRunId);
}
},
);
});
const spawnReadyChild = (env: NodeJS.ProcessEnv): ChildProcess =>
spawn(
process.execPath,
["-e", "process.stdout.write('ready'); setInterval(() => {}, 1000)"],
{ env, stdio: ["ignore", "pipe", "ignore"] },
);
const childReady = async (child: ChildProcess): Promise<void> => {
await new Promise<void>((resolve, reject) => {
child.once("error", reject);
child.stdout?.once("data", () => resolve());
child.once("exit", (code) =>
reject(new Error(`verifier child exited early with ${String(code)}`)),
);
});
};
const stopChild = async (child: ChildProcess): Promise<void> => {
if (child.exitCode !== null || child.signalCode !== null) return;
const exited = new Promise<void>((resolve) =>
child.once("exit", () => resolve()),
);
child.kill("SIGTERM");
await exited;
};
const restoreRunId = (runId: string | undefined): void => {
if (runId === undefined) delete process.env.REA_PROCESS_RUN_ID;
else process.env.REA_PROCESS_RUN_ID = runId;
};
typedoc.json
+7 -1
View File
@@ -16,7 +16,13 @@
"entryPointStrategy": "expand",
"out": "build/api-docs",
"tsconfig": "tsconfig.json",
"exclude": ["**/*.test.ts", "tests/**", "dist/**", "node_modules/**"],
"exclude": [
"**/*.test.ts",
"tests/**",
"dist/**",
"node_modules/**",
"src/application/LoopbackReplayRecorder.ts"
],
"excludeExternals": true,
"excludePrivate": true,
"excludeProtected": false,