Compare commits

...
Author SHA1 Message Date
morluto a74b11e99a chore: sync branch with latest main 2026-10-08 23:52:20 +08:00
morluto eb85e932ac docs(artifacts): clarify extraction materialization evidence 2026-10-08 23:48:06 +08:00
morluto 35a955ac15 fix(artifacts): correlate unpacked integrity with materialized companions 2026-10-08 23:46:05 +08:00
Ysopwnable a1ed4232de fix(mcp): retain complete application results across response limits
Retain complete authenticated application Evidence when the repeated MCP representation exceeds the default receive frame or Node string limit. Return a typed transport constraint, keep useful followup queries available, and stream full canonical bundles to atomic destinations.

Verify complete Notion and Obsidian workflows on Linux and native Windows, full CLI/Evidence parity, and real large Chrome screenshot recovery. Preserve complete success contracts and caller-selected evidence paths.

Refs #1065, #1053.
2026-10-08 23:46:02 +08:00
morluto 761a898926 chore: merge main and reconcile CI projections 2026-10-08 23:45:07 +08:00
morluto 3334db2bde fix(ci): restore generated projections in test shards 2026-10-08 23:31:52 +08:00
morluto c701376777 perf(build): avoid duplicate documentation generation 2026-10-08 23:29:43 +08:00
morluto 2af12a18a3 fix(process): preserve observations after successful cleanup 2026-10-08 23:29:43 +08:00
morluto 5e1afe454f fix(snapshot): retain MCP workflow replay bindings 2026-10-08 23:29:43 +08:00
morluto a9e00b83f3 docs: clarify evidence ownership and verification guidance 2026-10-08 23:29:43 +08:00
Ysopwnable 5f93b4b125 fix(ci): restore generated shard inputs and cleanup assertions
Restore ignored generated inputs before deterministic test shards consume the downloaded build. Preserve verified cleanup assertions and explicitly check residual unknowns for unrelated processes.
2026-10-08 23:20:52 +08:00
morluto 97359fe189 chore: synchronize integrity policy branch with latest main guidance 2026-10-08 23:16:56 +08:00
morluto 09a0c363d9 fix(artifacts): preserve integrity evidence across workflows and integrate main 2026-10-08 23:16:16 +08:00
morluto 46d195e8a4 docs: check surrounding cases when changing boundaries 2026-10-08 22:57:03 +08:00
morluto 9833c94656 docs: require surrounding boundary checks for review fixes 2026-10-08 22:56:00 +08:00
morluto 6e6ff7bd2e docs: move star milestone into history sections across readmes 2026-10-08 22:54:57 +08:00
Ysopwnable 9455e7c8bc docs: celebrate 20,000 GitHub stars 🎉 (#1079) 2026-10-08 22:44:10 +08:00
morluto e6134056cf fix(build): make catalogs and packaged skills generated outputs (#1086)
* fix(build): generate catalogs and packaged skills without snapshot commits

* fix(build): point skill drift recovery to uncached generation
2026-10-08 22:43:51 +08:00
morlutoandCursor Agent ac392da332 fix(artifacts): keep integrity policy and extraction claims aligned (#1083)
* fix(artifacts): keep integrity policy and extraction claims aligned

rea analyze routed a JavaScript application without the integrity policy,
so the remediation flag it recommends was rejected on that command. The
routed command now accepts --integrity-policy and forwards it.

Extraction copied every inventory contradiction into the result and said
those files were written. Only materialized occurrences stay in that list;
nested contradictions are named as unwritten.

* test(artifacts): type the nested extraction fixture as a directory path

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-08 22:41:35 +08:00
Ysopwnable b49beb42d7 fix(browser): normalize selected script manifest roots on Windows (#1078)
* fix(browser): normalize selected script manifest roots on Windows

* test(browser): keep manifest fixtures valid across Windows drives
2026-10-08 22:37:10 +08:00
Ysopwnable a4d77f0ff0 docs(website): add a CTF showcase and FAQ (#1077)
* docs(website): add a worked CTF reverse-engineering showcase

* docs(website): add a dedicated FAQ page
2026-10-08 22:30:10 +08:00
Kaoru0822 82a1a525d7 Merge pull request #1073 from morluto/fix/evidence-bundle-diagnostics
fix(evidence): name the failed constraint when a bundle is rejected
2026-10-08 07:27:21 -07:00
Kaoru0822 e8b92a40c0 Merge pull request #1074 from morluto/fix/evidence-file-missing
fix(evidence): report missing evidence files instead of permission advice
2026-10-08 07:22:31 -07:00
Ysopwnable d52f66d62a fix(metadata): stop schema changes churning generated skill evidence (#1075) 2026-10-08 22:21:30 +08:00
Kaoru0822 e2f03786f4 Merge pull request #1071 from morluto/fix/asset-catalog-absent
fix(apple): report apps without asset catalogs as empty inventories
2026-10-08 07:15:17 -07:00
Kaoru0822 5a55d5b1c6 Merge pull request #1070 from morluto/fix/keyed-archive-selection-errors
fix(artifacts): report keyed-archive selection mistakes as invalid input
2026-10-08 07:15:02 -07:00
Ysopwnable ef30343154 fix(setup): launch Windows npx registrations through cmd.exe (#1072) 2026-10-08 22:11:54 +08:00
Kaoru0822andClaude Opus 5.5 38bbb656da fix(evidence): require a complete record before the single-record hint
A malformed bundle that merely has an evidence_id field was described
as one Evidence record. Use the single-record message only when the
input matches the Evidence record schema; otherwise report the bundle
schema failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:09:28 +00:00
Kaoru0822andClaude Opus 5.5 89252f1f35 fix(evidence): report missing evidence files instead of permission advice
Reading a nonexistent Evidence bundle or snapshot, or exporting into a
directory that does not exist, failed as reason "io" with "Evidence
file could not be accessed. Check file permissions", and with no path.
In `rea compare` the caller could not tell which of the two files
failed.

Map ENOENT and ENOTDIR to a new "missing" reason with read- and
write-specific advice. Every EvidenceFileError now carries the resolved
caller path in its projected details. Permission and other I/O failures
keep the existing advice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:07:43 +00:00
muhammad-luayandN0zoM1z0 5547c1e697 fix(process): stop failing captures on tokens macOS cannot expose (#1057)
* fix(process): stop failing captures on tokens macOS cannot expose

On a busy macOS 26 host the residual process sweep fails every capture:
KERN_PROCARGS2 omits the environment of Apple platform binaries for any
caller and fails with EINVAL for processes owned by another account, so an
unrelated process created after the capture baseline escalates to a hard
cleanup failure with cleanup_incomplete.

Report environment withholding distinctly from the Darwin token reader by
reading the platform-binary flag through csops, carry the effective uid in
the process table, and re-read unresolved tokens with a bounded retry plus a
final liveness confirmation so transient exec and exit states settle.
Candidates whose token the OS provably withholds, foreign-uid candidates,
and candidates that exited during validation are now recorded as unverified
processes instead of failing an otherwise clean capture or being signaled.
Related candidates and every other unreadable candidate stay fail-closed.

* test(process): align ownership lanes with the uid-aware process table

The Darwin sweep reads the effective uid through ps, so the offline
verification lanes' syscall allowlists must accept the new argv in
verify-binary-layout and verify-evm-interface. The foreign-uid ownership
test requires POSIX process uids; skip it where the platform has none
(the curated Windows lane).

* fix(verify): recognize UID-bearing ownership process tables

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 22:01:05 +08:00
Kaoru0822andClaude Opus 5.5 cf96ceb211 fix(evidence): name the failed constraint when a bundle is rejected
evidence-import, evidence-export, compare and the MCP bundle tools gave
every rejected file the same message: "Evidence is invalid or has
changed. Recreate or re-import it". That included a single Evidence
record, which is what every command prints, so `rea compare a.json
b.json` on two command results looked like tampering.

Report a single record as such and point to export_evidence_bundle.
Locate the first schema issue, and keep thrown constraint messages such
as a semantic identifier mismatch. EvidenceIntegrityError now accepts
the userMessage option that other typed errors already use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:59:48 +00:00
Kaoru0822andClaude Opus 5.5 a079af297b fix(apple): keep skipped catalog entries from reading as absence
A symlink or directory named Assets.car is skipped by the
non-following directory reader. When no regular catalog remains, fail
with the skipped paths instead of reporting an empty inventory. When
catalogs were read, list the skipped entries in the limitations. The
absence limitation now says that symlinks are not followed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:52:18 +00:00
Kaoru0822andClaude Opus 5.5 79e5a3923d fix(apple): report apps without asset catalogs as empty inventories
inspect_asset_catalog failed with an "unavailable" artifact error for an
app bundle that has no compiled Assets.car. Its advice was "Check
artifact support and required tools", which does not apply: assetutil
was present, and the absence of a catalog is itself an observation.
decode_interface_builder already returns an empty graph for an app
without NIBs.

Return the empty page with an explicit limitation. Interface Builder
resource keys are still decoded and reported as unmatched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:45:43 +00:00
Kaoru0822andClaude Opus 5.5 25fb6d32a3 fix(artifacts): report keyed-archive selection mistakes as invalid input
inspect_keyed_archive reported caller selection failures as artifact
faults. A missing named root became "Artifact is invalid or has changed.
Get a fresh copy" and lost its reason, and a missing, non-canonical, or
directory archive path became "unsafe or conflicting internal path ...
correct the artifact".

Report these as invalid_request issues on `path` or `root`, listing the
available $top roots or the active plist basename. Decode failures keep
the decoder's reason, and a compiled NIBArchive points to
decode_interface_builder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:43:30 +00:00
40e18b2e37 fix(domain): preserve prototype-named JSON members through Evidence boundaries (#1049)
* fix(domain): preserve prototype-named JSON members through Evidence boundaries

Zod's z.json() record projection drops own __proto__ members as a
prototype-pollution guard, so provider replies carrying them lost evidence
before hashing/serialization and CLI/MCP output. The shared schema now
validates with z.json() as before, then rebuilds the value keeping
prototype-named own members as ordinary data properties (__proto__ via
Object.defineProperty, so the prototype is never mutated).

parseHarJson no longer rejects __proto__ members; the explicit unsupported
case from #992 is replaced with a preservation regression, and the
inspect_web_network_capture contract text is updated to match.

Fixes #1036

* fix(domain): bound JSON depth and oversized payloads at parsing boundaries

jsonValueSchema re-walked hostile input through z.json()'s recursive core, so a deeply nested payload from an MCP tool argument, inline evidence, provider result, or keyed archive threw RangeError: Maximum call stack size exceeded instead of a validation issue. The schema now runs an iterative depth guard first and keeps z.json() for full JSON semantics behind it, which also turns cyclic values into validation errors. The 1000-edge limit stays far below the range where the recursive core overflows its stack (measured between 2000 and 3000 on Node 24) while leaving room for legitimately deep target-derived trees such as compiled NIB view hierarchies, and the guard narrows with a typed record check instead of any or unchecked casts. The non-recursive JSON Schema advertisement stays intact on both the public and standard projection paths. Two payload sizes get explicit budgets tied to their producer constraints: web text artifacts reject beyond the 16 MiB source budget the web tooling already advertises, and native UI screenshot base64 beyond the 64 MiB helper output budget. Regenerated product catalog, skill catalog digest, and completion-ledger manifest follow the schema-boundary change.

* fix(validation): preserve depth diagnostics across process inputs

---------

Co-authored-by: DeryFerd <deryferdikao125@gmail.com>
Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 21:32:48 +08:00
Dery FerdikaandN0zoM1z0 bfbfc7330a fix(domain): reject excessive JSON depth with actionable diagnostics (#1029)
* fix(domain): bound JSON depth and oversized payloads at parsing boundaries

jsonValueSchema re-walked hostile input through z.json()'s recursive core, so a deeply nested payload from an MCP tool argument, inline evidence, provider result, or keyed archive threw RangeError: Maximum call stack size exceeded instead of a validation issue. The schema now runs an iterative depth guard first and keeps z.json() for full JSON semantics behind it, which also turns cyclic values into validation errors. The 1000-edge limit stays far below the range where the recursive core overflows its stack (measured between 2000 and 3000 on Node 24) while leaving room for legitimately deep target-derived trees such as compiled NIB view hierarchies, and the guard narrows with a typed record check instead of any or unchecked casts. The non-recursive JSON Schema advertisement stays intact on both the public and standard projection paths. Two payload sizes get explicit budgets tied to their producer constraints: web text artifacts reject beyond the 16 MiB source budget the web tooling already advertises, and native UI screenshot base64 beyond the 64 MiB helper output budget. Regenerated product catalog, skill catalog digest, and completion-ledger manifest follow the schema-boundary change.

* fix(validation): preserve depth diagnostics across process inputs

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 21:12:10 +08:00
ASVLCIIandN0zoM1z0 3dcb732da3 fix(cli): preserve blank JSON paths for input validation (#1068)
* fix(cli): preserve blank JSON paths for input validation

* test(cli): avoid unchecked path field assertions

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:57:54 +08:00
cevieirasandN0zoM1z0 6a7650e93c fix(mcp): flatten root union input schemas (#1062)
* fix(mcp): flatten root union input schemas

* fix(mcp): preserve minimum input requirements in object schemas

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:54:51 +08:00
Rudy Mizrahi Celekli 170600541b fix: select Node legacy package entry fields for import and require (#1035)
fix(javascript): use Node legacy package entry fields

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
2026-10-08 20:47:01 +08:00
Arian PashaeandN0zoM1z0 68b9fa489b fix(install): guard empty prefix_args expansion on macOS Bash 3.2 under set -u (#1061)
* fix(install): guard empty prefix_args expansion under Bash 3.2 set -u (#1060)

* test(install): cover native Bash macOS installation

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:39:39 +08:00
Ysopwnable 8a31508c89 Merge pull request #1064 from morluto/fix/release-checkpoint-guards
fix(release): validate frozen checkpoint versions and breaking notes
2026-10-08 18:31:45 +08:00
N0zoM1z0 f5009ed55f fix(release): recognize conventional titles in GitHub merge commits 2026-10-08 10:21:02 +00:00
N0zoM1z0 60825fb447 fix(release): validate frozen checkpoint versions and breaking notes
Audit actual first-parent ancestry instead of trusting the release bot
chronological cutoff, and block mismatched versions before tag creation.
Sync the published 6.0.0 baseline without dropping newer main changes.

Fixes #1063

Assisted-by: Sakuya (gensokyo-skills:sakuya-checkpointed-execution)
2026-10-08 10:13:05 +00:00
N0zoM1z0 5dfd9905ed chore(release): sync 6.0.0 checkpoint metadata 2026-10-08 09:49:53 +00:00
Kaoru0822andClaude Opus 5.5 5ccb9241be perf(mcp): advertise each tool JSON Schema once per target (#1059)
The MCP SDK reconverts every registered Zod schema on each tools/list.
With 139 tools that is ~45 ms of Zod JSON Schema work per listing, for a
catalog that cannot change after registration. Memoize the advertised
input and output projections per target on the registration clones.

Listings are byte-identical; repeated tools/list drops from ~62 ms to
~18 ms on the stdio server.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:31:08 +08:00
Kaoru0822 467f68412e Merge remote-tracking branch 'origin/main' into feat/artifact-integrity-policy 2026-10-07 14:50:36 +00:00
Kaoru0822andClaude Opus 5.5 22f4dc101a Merge origin/main into feat/artifact-integrity-policy
Keep main's skill metadata version (27) and regenerate the package, tool
catalog and product metadata for the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:18:28 +00:00
Kaoru0822andClaude Opus 5.5 be011ea3ba Merge origin/main into feat/artifact-integrity-policy
Keep both new MCP extraction tests, use main's relocated Apple domain
imports, and regenerate the catalog metadata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:41:30 +00:00
morluto 40a33d3d3b chore: merge main and regenerate combined catalog metadata 2026-10-07 18:44:31 +08:00
morluto 464115f2ec chore: merge main and retain artifact integrity evidence 2026-10-07 18:02:05 +08:00
morluto 883db73b3f chore: merge main and resolve PR #902 conflicts 2026-10-07 17:28:56 +08:00
Kaoru0822andClaude Opus 5.5 5aafaa00ec fix(artifacts): carry recorded integrity contradictions into application workflows
A recorded contradiction appeared only in the analyze_javascript_application
response. Version and source comparisons, feature traces, and export-shape
comparisons read the application graph, which still claimed complete
coverage, so they could report complete-within-inputs results over
untrusted bytes. The graph is now partial when contradictions were
recorded, and its limitations, inherited by the result and the semantic
graph, name every contradicted path.

The mismatch remediation names both spellings: integrity_policy in MCP
and --integrity-policy on the CLI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:50:41 +00:00
Kaoru0822 92b373e660 Merge remote-tracking branch 'origin/main' into feat/artifact-integrity-policy
# Conflicts:
#	docs/product-catalog.json
#	docs/verification/managed-conformance-manifest.json
#	skills/reverse-engineer-anything/SKILL.md
#	src/generatedMcpToolCatalog.ts
2026-10-07 08:37:04 +00:00
Kaoru0822andClaude Opus 5.5 69af884666 feat(artifacts): let extraction and JavaScript analysis record integrity mismatches
macOS code signing rewrites unpacked native modules after electron-builder
records their ASAR integrity, so the declared and observed hashes of
those .node files differ in most signed Electron apps. Claude, ChatGPT,
Notion, Obsidian, and Tabby all failed inspect_artifact,
extract_artifact, and analyze_javascript_application by default. Only
inspect_artifact offered integrity_policy, so extraction and JavaScript
analysis of these apps was impossible.

extract_artifact and analyze_javascript_application now accept the same
integrity_policy (default fail). With record-and-continue, extraction
writes the observed bytes, and JavaScript analysis reads contradicted
files as observed. Contradicted nested archives stay unexpanded. Both
results return the recorded integrity_contradictions and a limitation.
The mismatch remediation now names this option instead of only advising a
fresh copy, which cannot help for a signed installation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:28:44 +00:00
200 changed files with 8037 additions and 1951 deletions

No files matched your search

+5 -1
View File
@@ -28,7 +28,7 @@ intentionally not included. -->
- Provider, bridge, target-format, or platform compatibility:
- Evidence, artifact, provenance, or reconstruction contract:
- Process execution, authorization, cleanup, or containment impact:
- Generated metadata (`docs/product-catalog.json`), package, or installation impact (the regen-generated bot pushes tracked generated files; never hand-edit them):
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `skill-src/` and never commit build outputs):
## Evidence and regression coverage
@@ -59,6 +59,10 @@ relevant real-provider or platform workflow, or explain why it was unavailable.
limits, security/privacy/containment implications, and meaningful proof gaps. -->
- Breaking changes or migration steps:
<!-- For ! / BREAKING CHANGE: show a previously valid call/configuration,
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Compatible fixes/additions should use patch/minor semantics.
See docs/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
+21 -4
View File
@@ -102,8 +102,8 @@ jobs:
with:
name: rea-dist
path: dist
- name: Generate source catalog from downloaded build
run: node scripts/generate-mcp-tool-catalog.mjs
- name: Generate projections from downloaded build
run: npm run build:projections
- run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
@@ -317,8 +317,8 @@ jobs:
- run: npm run verify:keyed-archive
- run: npm run verify:asset-catalog
- run: npm run verify:macos-bundle
- name: Native compiled NIB hierarchy regressions
run: npm run test:focused -- tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
- name: Native installer and compiled NIB hierarchy regressions
run: npm run test:focused -- tests/boundary/filesystem/installScript.test.ts tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
- name: Real binary metadata and Interface Builder integration
run: npm run verify:apple-dispatch && npm run verify:interface-builder
- name: Native plist CLI selection regressions
@@ -327,6 +327,8 @@ jobs:
run: npm run test:focused -- tests/boundary/cli/nativeDemangleCli.test.ts tests/boundary/mcp/nativeDemangleMcp.test.ts
- name: Process filesystem snapshot descriptor regressions
run: npm run test:focused -- src/process/capture/FilesystemSnapshot.test.ts
- name: Native Darwin ownership and process cleanup regressions
run: npm run test:focused -- tests/boundary/process/darwinProcessRunTokenReader.test.ts tests/boundary/process/processCaptureLifecycle.test.ts tests/boundary/cli/processCli.test.ts tests/boundary/mcp/sessionMcp.test.ts
reconstruction-readiness:
name: Reconstruction readiness
@@ -441,6 +443,11 @@ jobs:
tests/boundary/filesystem/safeOutputTree.test.ts
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
tests/boundary/filesystem/setupUnavailableProvider.test.ts
tests/boundary/filesystem/windowsNpxRegistration.test.ts
src/application/ClientRegistrationIdentity.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
tests/boundary/cli/referenceSourceImportStatus.test.ts
src/cliReferenceSourceImportStatus.test.ts
tests/boundary/filesystem/asarInventoryFreshness.test.ts
@@ -481,3 +488,13 @@ jobs:
cache: npm
- run: npm ci
- run: npm run docs:check
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@v4
with:
name: generated-docs
path: |
docs/public/product-catalog.json
docs/verification/managed-conformance-*.json
skills/reverse-engineer-anything
if-no-files-found: error
-75
View File
@@ -1,75 +0,0 @@
name: Regenerate generated metadata
on:
pull_request:
branches: [main]
paths:
- "src/**"
- "skills/**/SKILL.md"
- "scripts/generate-*.mjs"
- "scripts/check-doc-facts.mjs"
- "scripts/lib/*.mjs"
- "scripts/verify/managed/**"
- "scripts/fixtures/managed/**"
- "package.json"
- "package-lock.json"
concurrency:
group: regen-${{ github.event.pull_request.number }}
cancel-in-progress: true
env:
HUSKY: "0"
permissions:
contents: write
jobs:
regen:
name: Regenerate and push tracked generated files
# Same-repo PRs only: the default token cannot push to forks.
# The bot-actor guard stops looped runs triggered by our own push.
if: >-
github.actor != 'github-actions[bot]' &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref }}
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: npm run docs:generate
- name: Commit regenerated files
shell: bash
env:
HEAD_REF: ${{ github.event.pull_request.head.ref }}
run: |
generated_paths=(
docs/product-catalog.json
docs/error-contract.schema.json
docs/verification/managed-conformance-manifest.json
docs/verification/managed-conformance-ledger.json
skills/reverse-engineer-anything/SKILL.md
src/generatedPackageMetadata.ts
)
if git diff --quiet -- "${generated_paths[@]}"; then
echo "Generated metadata is already canonical."
exit 0
fi
git diff --check
# Refuse to sweep unrelated working-tree changes into the bot commit.
unexpected="$(git diff --name-only | grep -vxF -f <(printf '%s\n' "${generated_paths[@]}") || true)"
if [[ -n "${unexpected}" ]]; then
echo "docs:generate left unexpected tracked changes:" >&2
printf '%s\n' "${unexpected}" >&2
exit 1
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -- "${generated_paths[@]}"
git commit -m "chore: regenerate generated metadata"
git push origin "HEAD:${HEAD_REF}"
+39 -36
View File
@@ -84,6 +84,32 @@ jobs:
exit 1
fi
- name: Check out release controller and Git history
uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
fetch-depth: 0
persist-credentials: false
- name: Set up Node.js for checkpoint validation
uses: actions/setup-node@v6
with:
node-version-file: .nvmrc
package-manager-cache: false
- name: Install checkpoint validation dependencies
run: npm ci
- name: Validate checkpoint version and ancestry before release creation
env:
RELEASE_PHASE: ${{ inputs.phase }}
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
run: >-
node scripts/verify-release-checkpoint.mjs
--phase "$RELEASE_PHASE" --release-branch "$RELEASE_BRANCH"
--source-sha "$SOURCE_SHA"
- uses: googleapis/release-please-action@v4
id: release
with:
@@ -116,6 +142,7 @@ jobs:
uses: actions/checkout@v6
with:
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
fetch-depth: 0
persist-credentials: false
- name: Set up Node.js for generated documentation
@@ -129,47 +156,23 @@ jobs:
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm ci
- name: Validate prepared release version and migration notes
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
env:
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
run: >-
node scripts/verify-release-checkpoint.mjs
--phase prepare --stage candidate --release-branch "$RELEASE_BRANCH"
--source-sha "$SOURCE_SHA"
- name: Regenerate release documentation
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm run docs:generate
- name: Commit canonical release catalog
- name: Validate generated release documentation
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
shell: bash
env:
# Use the same identity for the follow-up commit so it also triggers CI.
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
RELEASE_PR_JSON: ${{ steps.release.outputs.pr }}
run: |
release_pr_branch="$(node --input-type=module -e '
const value = JSON.parse(process.env.RELEASE_PR_JSON ?? "");
if (
typeof value !== "object" ||
value === null ||
typeof value.headBranchName !== "string" ||
value.headBranchName.length === 0
) {
process.exit(1);
}
process.stdout.write(value.headBranchName);
')"
changed_paths="$(git diff --name-only)"
if [[ -z "${changed_paths}" ]]; then
echo "Release documentation is already canonical."
exit 0
fi
if [[ "${changed_paths}" != "docs/product-catalog.json" ]]; then
echo "docs:generate changed unexpected tracked paths:" >&2
printf '%s\n' "${changed_paths}" >&2
exit 1
fi
git diff --check
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -- docs/product-catalog.json
git commit -m "chore: normalize release product catalog"
gh auth setup-git
git push origin "HEAD:${release_pr_branch}"
run: npm run docs:check && git diff --exit-code
publish:
name: Publish to npm
+8
View File
@@ -38,6 +38,7 @@ test-results.xml
.turbo/
.cache/
docs/.vitepress/cache/
docs/.vitepress/dist/
# OS
.DS_Store
@@ -51,3 +52,10 @@ Thumbs.db
# Build-generated MCP tool catalog (regenerated by `npm run build`; never commit)
src/generatedMcpToolCatalog.ts
# Derived catalogs, portable conformance projections, and packaged skill bundles
# (regenerated from this checkout by `npm run build:cached`; never commit)
/skills/
/docs/public/product-catalog.json
/docs/verification/managed-conformance-manifest.json
/docs/verification/managed-conformance-ledger.json
+9 -7
View File
@@ -21,10 +21,10 @@ REA is a layered ESM TypeScript application. Dependencies flow inward from pure
- `src/domain/` owns pure provider-neutral semantics; `src/contracts/` owns caller-visible schemas and the canonical tool inventory.
- `src/hopper/`, `src/ghidra/`, `src/ida/`, `src/browser/`, `src/inspector/`, `src/native/`, `src/artifacts/`, and `src/dotnet/` own provider-specific boundaries. Keep provider protocols out of domain and application code.
- `src/application/` composes shared CLI/MCP workflows; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. `bridge/` contains provider-side adapters.
- `src/application/` composes shared CLI/MCP workflows, including Evidence provenance, unknowns, and eligible snapshot bindings; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. Reuse its supervision and identity primitives before adding provider-local lifecycle code; a PID and executable pathname alone do not establish ownership after exit or reuse. `bridge/` contains provider-side adapters.
- `tests/` contains unit, composition, boundary, acceptance, and conformance tests. `scripts/verify-*` and capability directories under `scripts/verify/` contain real-toolchain checks.
- `docs/product-catalog.json` is generated. Update its source contracts and regenerate it; do not edit it directly.
- `docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored build outputs. Update source contracts and authored instructions in `skill-src/`, then run `npm run build:cached`; never commit derived catalog digests or portable conformance projections.
- `src/generatedMcpToolCatalog.ts` is build-generated and gitignored. Never commit it; resolve any trace of it in merges by deleting it and running `npm run build:cached`.
## Build, Test, and Development Commands
@@ -36,7 +36,7 @@ REA is a layered ESM TypeScript application. Dependencies flow inward from pure
- `npm run check:changed`: run cached static checks and source tests affected since the branch merge base (default `origin/main`).
- `npm run check:fast`: run cached typecheck and lint checks.
- `npm run check:pr`: opt into the complete local deterministic gate and generated-document checks for broad changes; CI owns full coverage. Routine iterations need focused tests and relevant checks, not the whole gate each time.
- `npm run docs:check`: check committed generated documents; `npm run docs:generate` regenerates them.
- `npm run docs:check`: build and validate generated documents for the current checkout; `npm run docs:generate` regenerates them. CI retains ignored outputs as artifacts and does not push snapshot commits to feature branches.
- For provider-dependent changes, see [docs/testing.md](docs/testing.md) and run the matching real-provider verification.
- Keep each verification lane's prerequisites limited to the claim it checks. Use host-native fixtures for host/provider acceptance; put optional cross-target formats and their external toolchains in a separate lane. Preflight required commands and report the missing dependency and lane clearly.
@@ -54,9 +54,11 @@ Use ESM TypeScript, two-space indentation, and the committed Oxfmt configuration
Treat a boundary as a contract between the producer's actual representation and the consumer's required meaning. When implementing or auditing a boundary, trace the value through parsing, normalization, authorization, serialization, and the CLI/MCP result. Establish affected callers from their code paths; similar tool names or workflows do not prove that they share a schema or failure mode.
Keep portable evidence and scenario validation distinct from host-native execution checks. Absolute filesystem paths, file URLs, and HTTP paths have different semantics; do not substitute one platform's syntax for the domain concept. Interpret provider metadata according to its documented or observed producer behavior. When a transformation loses information, preserve the reported value and an explicit unknown rather than guessing a canonical identity.
When changing boundary behavior, inspect adjacent input representations, failure paths, and affected callers, and correct the underlying assumption across those cases.
Preserve meaningful failure reasons through application and adapter layers. Malformed input is distinct from an unsupported target, unavailable provider, or host operating-system permission denial. Diagnostics should identify the failed constraint and the target or lifecycle request it applies to. Recovery advice must address that reason and point to an available workflow; generic catches must not erase actionable validation details.
Keep portable evidence and scenario validation distinct from host-native execution checks. Absolute filesystem paths, file URLs, and HTTP paths have different semantics; do not substitute one platform's syntax for the domain concept. Interpret provider metadata according to its documented or observed producer behavior. Keep values used for identity, provenance, matching, and path resolution separate from display formatting. When normalization loses information, preserve the source value and an explicit unknown; display placeholders must not feed back into lookup or selection.
Preserve meaningful failure reasons through application and adapter layers. Malformed input is distinct from an unsupported target, unavailable provider, or host operating-system permission denial. Diagnostics should identify the failed constraint and the target or lifecycle request it applies to. Recovery advice must address that reason and point to an available workflow; generic catches must not erase actionable validation details. Preserve collected observations on execution failure independently of whether cleanup succeeds.
Leave meaningful target, action, capture, and output choices to the agent. A selected operation already expresses intent; do not require approval booleans or repeated permission declarations. Trace each setting to its consumer: remove ignored options and single-value confirmations, derive built-in lifecycle behavior, and supply defaults for omitted optional metadata. Report actual effects and limitations where they help interpret results rather than asking callers to restate them.
@@ -80,7 +82,7 @@ Let agents compose experiments with ordinary commands, scripts, and local fixtur
- Use **observe/capture** only when runtime activity is required, and declare authority and lifecycle effects in the contract.
- Extend an existing tool when intent and result contract are unchanged. Add a tool for a distinct analyst outcome or materially different authority.
- Keep caller-facing names and results provider-neutral. Put engine-specific behavior in provider adapters and report each provider's exact coverage.
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Keep observed, derived, inferred, and unknown results distinct.
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Account for representation expansion and products of independently bounded dimensions before allocating or retaining output. Derive facet completeness from what was examined and exhausted, not an empty failure list. Keep observed, derived, inferred, and unknown results distinct.
- Include artifact identity, source locations, Evidence references, actionable errors, and relevant limitations when they affect conclusions. Return the evidence needed for the next analysis inline rather than requiring a resource or identifier lookup.
- Implement shared application workflows behind CLI and MCP adapters. Update canonical contracts and generated catalog together.
+19 -3
View File
@@ -35,7 +35,7 @@ links, and `npm run docs:preview` to preview that build at `/rea/`.
Site navigation lives in `docs/.vitepress/config.ts`. Keep links to guides
relative so they work on GitHub and the website; link to repository files
outside `docs/` using their full GitHub URLs. Generated reference documents
still use `npm run docs:generate`; `docs:build` only builds the website.
use `npm run docs:generate`; `docs:build` generates them before building the website.
Pull requests run `npm run docs:check`. `.github/workflows/pages.yml` is a
manual VitePress build and does not publish. The public site at
@@ -74,8 +74,24 @@ require the matching real-provider `verify:*` lane.
Formatting uses Oxfmt and the committed `.oxfmtrc.json`; generated sources use
the same configuration. Pre-commit formats and lints staged files; pre-push runs
`check:fast`.
`docs:check` checks committed generated metadata. `docs:generate` regenerates
those files, and the docs CI lane checks them.
`docs:check` builds and validates generated metadata for the current checkout.
The product catalog (`docs/public/product-catalog.json`), portable managed
conformance projections (`docs/verification/managed-conformance-*.json`), and
packaged skill (`skills/`) are ignored build outputs. Edit skill instructions
and references in `skill-src/`; the build adds catalog-dependent metadata to
the packaged copy without rewriting authored files. The generated manifest
commits to that exact packaged skill bundle. It is a portable projection of
the deterministic managed verifier, not a record of optional real-provider runs.
CI validates these outputs and retains them as artifacts instead of pushing
generated-only commits onto feature branches. Reviewed source metadata such as
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json` remains
tracked and checked for freshness. Do not commit ignored generated outputs.
The build-generated product catalog contains documented facts and their provider
identity, rather than full runtime schema hashes. The managed skill contains
instructions and inventory metadata; doctor compares its installed files with
the canonical bundle. Schema-only fixes should not change these outputs or the
skill commitment in the conformance manifest. Runtime schema identity remains
available through doctor and `binary_session`.
Real-provider execution remains uncached; deterministic builds use Turbo.
Local `npm test` runs every deterministic Vitest project without coverage or
+10 -6
View File
@@ -10,7 +10,7 @@
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![CI](https://img.shields.io/github/actions/workflow/status/morluto/rea/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/morluto/rea/actions/workflows/ci.yml)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/product-catalog.json)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![Node.js 22+](https://img.shields.io/badge/Node.js-22.19%2B-339933?style=flat-square&logo=nodedotjs&logoColor=white)](https://nodejs.org/)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -296,7 +296,7 @@ For exact options, prerequisites and result contracts:
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/product-catalog.json): generated inventory of tools, providers and CLI commands.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): build-generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -315,12 +315,12 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[Website](https://morluto.github.io/rea/) · [npm](https://www.npmjs.com/package/rea-agents) · [skills.sh](https://skills.sh/morluto/rea/reverse-engineer-anything) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md)
## License
[MIT](LICENSE)
## Star history
🎉 **20,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, testing builds, and contributing fixes.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
@@ -332,3 +332,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
## Disclaimer
REA provides tools for lawful reverse-engineering research, analysis, and reconstruction. You are responsible for obtaining any required authorization and complying with applicable laws. The project does not endorse illegal or unauthorized use.
## License
[MIT](LICENSE)
+14
View File
@@ -336,6 +336,20 @@ npx -y rea-agents@latest compare /absolute/path/to/left-evidence.json /absolute/
- [Hopper Disassembler](https://www.hopperapp.com/)
- [سياسة الأمان](SECURITY.md)
## تاريخ النجوم
🎉 **وصلنا إلى 20,000 نجمة على GitHub، شكرًا لكم!**
شكرًا لكل من يستخدم REA ويبلّغ عن الأخطاء ويختبر الإصدارات ويساهم في إصلاحها.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="تاريخ نجوم REA على GitHub" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## الترخيص
[MIT](LICENSE)
+14
View File
@@ -366,6 +366,20 @@ REA にホスト型の解析サービスはありません。現在のプロバ
開発環境、アーキテクチャ、テスト、リリース手順は [CONTRIBUTING.md](CONTRIBUTING.md) を参照してください。
## スター履歴
🎉 **GitHub スターが 20,000 を突破しました。ありがとうございます!**
REA を使い、問題を報告し、ビルドをテストし、修正を届けてくださる皆さんに感謝します。
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA の GitHub スター履歴" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## ライセンス
[MIT](LICENSE)
+14
View File
@@ -315,6 +315,20 @@ REA에는 호스팅 분석 서비스가 없습니다. 로컬 Unix 소켓을 통
개발 환경, 아키텍처, 테스트, 릴리스 지침은 [CONTRIBUTING.md](CONTRIBUTING.md)를 참고하세요.
## 스타 기록
🎉 **GitHub 스타 20,000개, 감사합니다!**
REA를 사용하고, 버그를 제보하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA GitHub 스타 기록" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## 라이선스
[MIT](LICENSE)
+14
View File
@@ -316,6 +316,20 @@ REA 不提供托管分析服务,而是通过本地 Unix 套接字把操作交
开发环境、架构、测试和发布说明请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。
## Star 历史
🎉 **GitHub Star 突破 20,000,感谢大家!**
感谢每一位使用 REA、反馈问题、测试构建和贡献修复的朋友。
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA GitHub Star 历史" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## 许可证
[MIT](LICENSE)
-2
View File
@@ -133,8 +133,6 @@ def project_record(state, secrets):
for key, item in value.items():
if not isinstance(key, str):
raise CaptureFailure("format", "Native record has a non-string dictionary key.", path)
if key == "__proto__":
raise CaptureFailure("unsupported", "Native object contains a __proto__ member that the current JSON schema boundary cannot preserve.", path)
if any(secret in key for secret in secrets):
# Validate the omitted subtree too, then discard all of its
# coordinates rather than inventing a replacement identity.
+31 -1
View File
@@ -112,6 +112,31 @@ enum RunTokenReadError: Error {
case ambiguousEnvironmentBoundary
}
@_silgen_name("csops")
private func csops(
_ pid: Int32,
_ ops: UInt32,
_ addr: UnsafeMutableRawPointer?,
_ size: Int
) -> Int32
private let csOpsStatus: UInt32 = 0
private let csPlatformBinary: UInt32 = 0x0400_0000
/**
* Whether the target image is an Apple platform binary. Current macOS omits
* the environment of platform binaries from KERN_PROCARGS2 for every caller,
* so this distinguishes a policy-withheld environment from a process whose
* environment is genuinely absent or transiently unreadable.
*/
func isPlatformBinary(pid: Int32) -> Bool {
var flags: UInt32 = 0
let result = withUnsafeMutablePointer(to: &flags) { pointer in
csops(pid, csOpsStatus, UnsafeMutableRawPointer(pointer), MemoryLayout<UInt32>.size)
}
return result == 0 && (flags & csPlatformBinary) != 0
}
private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
var mib = [CTL_KERN, KERN_PROCARGS2, Int32(pid)]
var size = 0
@@ -133,7 +158,12 @@ private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
} catch RunTokenReadError.invalidTokenEncoding {
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
} catch RunTokenReadError.environmentUnavailable {
return unavailable(pid: pid, reason: "environment_unavailable")
return unavailable(
pid: pid,
reason: isPlatformBinary(pid: pid)
? "platform_binary_environment_withheld"
: "environment_unavailable"
)
} catch RunTokenReadError.appleVectorUnavailable {
return unavailable(pid: pid, reason: "apple_vector_unavailable")
} catch RunTokenReadError.ambiguousEnvironmentBoundary {
+1 -1
View File
@@ -27,7 +27,7 @@ REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
REA never installs Python, packages or GDB, changes a user init file, launches
the selected object as a host process, or requests runtime library resolution. The Python process
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
[upstream provenance](../third_party/pwntools/README.md).
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/pwntools/README.md).
## Interpreting results
+1 -1
View File
@@ -24,7 +24,7 @@ their distinct unavailable-provider or capability classification.
The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
`REA_EVM_PRLIMIT_COMMAND` can select another absolute executable. REA installs
no system tools and changes no user configuration. [Upstream provenance and
license](../third_party/evmole/README.md) are packaged.
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
## Evidence semantics
+1 -1
View File
@@ -95,7 +95,7 @@ npx skills add morluto/rea --skill reverse-engineer-anything
This installs agent instructions and bundled references, not REA MCP
registration or analysis engines. Follow the skill's
[conditional connection guide](https://github.com/morluto/rea/blob/main/skills/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
[conditional connection guide](https://github.com/morluto/rea/blob/main/skill-src/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
Working tools can be used immediately. If tools are missing, inspect the current
client's registration with `doctor --client codex --json` (substitute its client
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
+12 -7
View File
@@ -100,17 +100,22 @@ a temporary output file larger than the running engine's string limit, verifies
its bytes and digest, and removes it. Use `-- jsonl` for the compact JSONL check.
Each check needs space for one output file plus a 1 GiB free-space reserve.
Other CLI formats, `--token-count`, and MCP transport serialization still
assemble whole strings. Field selection remains useful when the caller needs a
Other CLI formats and `--token-count` still assemble whole strings.
Field selection remains useful when the caller needs a
smaller view, for example `--format json --filter-output
evidence_id,normalized_result.statistics`. Streaming output does not bound the
memory needed to construct the analysis graph itself.
The separate large-response transport work is tracked in
[#1053](https://github.com/morluto/rea/issues/1053) and
MCP prepares the complete repeated response incrementally against the pinned
SDK's 10 MiB stdio receive-buffer budget. Oversized results return an actionable
transport constraint and the exact same-session Evidence reference. Use
`trace_application_feature` to inspect a selected module's relationships, or
`export_evidence_bundle` to write the complete canonical bundle without a
document-sized allocation. Same-session analysis reads reuse authenticated
immutable snapshots; foreign inline Evidence is still parsed and authenticated.
See [MCP tool results](mcp-contracts.md#tool-results) for larger client buffers
and `REA_MCP_MAX_RESPONSE_BYTES`. Follow-up results remain complete and can also
exceed the transport budget. Compact result views are tracked separately in
[#1050](https://github.com/morluto/rea/issues/1050).
Client framing limits also apply: the pinned Node MCP SDK's stdio transport
defaults to a 10 MiB buffer. Its caller-selected `maxBufferSize` must accommodate
the complete response, including text and structured Evidence projections.
## What is reconstructed
+1 -1
View File
@@ -25,7 +25,7 @@ part of the current tool set.
This guide describes the implementation and verification of
[ADR-0003](adr/0003-managed-code-evidence-and-provider-boundary.md). The canonical
tool inventory is [`product-catalog.json`](https://github.com/morluto/rea/blob/main/docs/product-catalog.json).
tool inventory is the [build-generated catalog](mcp-contracts.md#generated-catalog).
## Shipped scope
+42 -4
View File
@@ -1,5 +1,15 @@
# MCP runtime contracts
## Generated catalog
Run `npm run build:cached` in a source checkout to generate the machine-readable
catalog at `docs/public/product-catalog.json`. The [catalog](product-catalog.json)
is also included in documentation builds. PR CI retains it with the packaged
skill and portable conformance projections in the `generated-docs` artifact.
These outputs describe the exact source revision being built; they are not
checked-in snapshots. For a running server, `binary_session` remains the
authoritative source of catalog identity and tool availability.
## Identity and discovery
`binary_session` reports the active package, server, SDK, and negotiated
@@ -154,6 +164,26 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
retained session history or an explicit bundle for transfer.
REA prepares complete MCP results within the pinned stdio client's 10 MiB
receive-buffer budget, including both text and structured representations and
room for the JSON-RPC envelope. If a result cannot fit, REA returns
`resource_constraint` with `details.resource: "transport"` before constructing
a document-sized string. Analysis Evidence remains complete in the current
session. Its exact reference is reported in
`details.reported_limits.evidence_reference`; use it with a focused application
workflow, or call `export_evidence_bundle` with a destination path. Complete
bundle exports stream canonical JSON into an atomically published file. A broad
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
preserves the complete session without sending it through a single MCP frame.
Clients that explicitly configure a larger receive buffer can set the REA
server's `REA_MCP_MAX_RESPONSE_BYTES` environment variable to the same byte
count. This setting must be a safe decimal integer at least 10485760; REA
reserves 1024 bytes for the envelope. Raising it restores complete inline
delivery for responses that fit that buffer and Node's single-string limit.
It does not change the client's buffer, analysis coverage, or retained content.
Ordinary responses keep their existing complete result contract.
## Retained application Evidence inputs
`trace_application_feature`, `trace_javascript_semantics`,
@@ -246,11 +276,19 @@ and returns its result and Evidence inline.
## Integrity record-and-continue
Artifact integrity fails closed by default. A request can explicitly select
Artifact integrity fails closed by default. `inspect_artifact`,
`extract_artifact`, and `analyze_javascript_application` accept
`integrity_policy=record-and-continue` when the investigation needs verified
siblings to continue after a mismatch.
siblings to continue after a mismatch. A common expected mismatch is an
unpacked native module that macOS code signing rewrote after the ASAR header
recorded its integrity.
Contradictory bytes are quarantined from nested expansion and recorded with
declared and observed hashes, trust, provenance, path, and unpacked state.
Verified siblings continue. Comparisons classify the result as a contradiction
and reconstruction cannot treat it as unchanged.
Verified siblings continue. Extraction writes the observed bytes and returns the
contradictions with its result; JavaScript analysis reads contradicted files as
observed and lists them in `integrity_contradictions`. Comparisons classify the
result as a contradiction and reconstruction cannot treat it as unchanged.
Application comparisons and feature traces retain the source integrity warnings
and report incomplete coverage. Extraction retains nested contradiction records
without claiming that the nested members were materialized separately.
+6
View File
@@ -102,6 +102,12 @@ run the same owned-process cleanup path. Settlement reports whether the
sampled process group quiesced or whether cleanup was needed or unverifiable;
sampling cannot prove that every short-lived or detached descendant was seen.
When the host withholds an unrelated process’s ownership token, REA leaves that
process untouched and records its PID and reason in `cleanup.unverified_processes`
and process residual unknowns. Successful cleanup verifies the owned group;
it does not attribute those unrelated processes. Related or otherwise unexplained
unreadable processes still prevent successful cleanup.
## Compare two captures
Compare saved capture Evidence with:
-722
View File
@@ -1,722 +0,0 @@
{
"package": {
"name": "rea-agents",
"version": "6.0.0",
"sdk": {
"server": "2.3.1",
"client": "2.3.1",
"core": "2.3.1"
},
"skill_version": "33"
},
"tools": {
"total": 139,
"families": [
{
"id": "direct",
"surface": "official-proxy",
"count": 41,
"tools": [
"address_name",
"address_to_file_offset",
"annotate_native_function",
"comment",
"current_address",
"current_document",
"current_procedure",
"goto_address",
"inline_comment",
"inspect_native_data_type",
"inspect_native_instruction",
"inspect_native_load_image",
"list_bookmarks",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"next_address",
"prev_address",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"resolve_native_call_targets",
"search_procedures",
"search_strings",
"set_address_name",
"set_addresses_names",
"set_bookmark",
"set_comment",
"set_current_document",
"set_inline_comment",
"unset_bookmark",
"xrefs"
]
},
{
"id": "enhanced",
"surface": "enhanced",
"count": 14,
"tools": [
"analyze_function",
"analyze_swift_types",
"batch_decompile",
"binary_overview",
"find_xrefs_to_name",
"get_call_graph",
"get_objc_classes",
"get_objc_protocols",
"inspect_native_api",
"inspect_native_dispatch_metadata",
"trace_call_path",
"trace_feature",
"trace_native_ui_action",
"trace_native_values"
]
},
{
"id": "native",
"surface": "native-provider",
"count": 10,
"tools": [
"capture_native_ui_scenario",
"demangle_swift",
"inspect_binary_layout",
"inspect_macho",
"inspect_plist",
"inspect_recorded_crash",
"inspect_signature",
"list_architectures",
"observe_native_calls",
"observe_native_ui"
]
},
{
"id": "artifact",
"surface": "artifact-provider",
"count": 7,
"tools": [
"decode_interface_builder",
"extract_artifact",
"inspect_artifact",
"inspect_asset_catalog",
"inspect_evm_interface",
"inspect_keyed_archive",
"trace_dylib_resolution"
]
},
{
"id": "managed",
"surface": "managed",
"count": 7,
"tools": [
"compare_managed_members",
"import_managed_reconstruction",
"inspect_managed_artifact",
"inspect_managed_members",
"inspect_managed_native_boundaries",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
},
{
"id": "firmware",
"surface": "firmware-provider",
"count": 2,
"tools": ["extract_firmware", "inspect_firmware_regions"]
},
{
"id": "android",
"surface": "android-provider",
"count": 5,
"tools": [
"inspect_android_class",
"inspect_android_method",
"inspect_android_package",
"search_android_classes",
"trace_android_references"
]
},
{
"id": "browser",
"surface": "browser-provider",
"count": 12,
"tools": [
"analyze_web_bundle",
"capture_browser_scenario",
"capture_web_screenshot",
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_network_capture",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
{
"id": "electron",
"surface": "electron-provider",
"count": 5,
"tools": [
"analyze_javascript_application",
"capture_electron_scenario",
"inspect_electron_page",
"list_electron_targets",
"reconcile_javascript_runtime"
]
},
{
"id": "javascript-runtime",
"surface": "runtime-provider",
"count": 2,
"tools": [
"list_javascript_runtime_targets",
"observe_javascript_runtime"
]
},
{
"id": "application",
"surface": "application-workflow",
"count": 13,
"tools": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"export_web_scripts",
"project_android_application_graph",
"project_apple_application_graph",
"recover_javascript_sources",
"trace_application_feature",
"trace_javascript_semantics",
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
"id": "session",
"surface": "session",
"count": 21,
"tools": [
"binary_session",
"build_call_path",
"capture_process_scenario",
"close_binary",
"compare_artifacts",
"compare_bundles",
"compare_functions",
"compare_process_captures",
"correlate_static_and_runtime",
"export_evidence_bundle",
"find_changed_behavior",
"get_evidence_bundle",
"get_navigation_context",
"import_evidence_bundle",
"inspect_address_context",
"list_unknowns",
"open_binary",
"record_unknown",
"update_unknown",
"verify_reconstruction",
"verify_unknown_resolution"
]
}
]
},
"providers": [
{
"id": "binwalk",
"name": "Binwalk",
"version": "3.1.0",
"capabilities": ["inspect_firmware_regions"]
},
{
"id": "evmole-interface",
"name": "REA EVM interface adapter",
"version": "evmole@0.9.3",
"capabilities": ["inspect_evm_interface"]
},
{
"id": "ghidra",
"name": "Ghidra",
"version": null,
"capabilities": [
"address_name",
"address_to_file_offset",
"analyze_function",
"annotate_native_function",
"inspect_native_data_type",
"inspect_native_instruction",
"inspect_native_load_image",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"resolve_native_call_targets",
"search_procedures",
"search_strings",
"xrefs"
]
},
{
"id": "har-schema/lossless-json",
"name": "REA HAR capture adapter",
"version": "har-schema@2.0.0;jsonc-parser@3.3.1;lossless-json@4.3.1",
"capabilities": ["inspect_web_network_capture"]
},
{
"id": "hopper",
"name": "Hopper Disassembler",
"version": null,
"capabilities": [
"address_name",
"address_to_file_offset",
"analyze_function",
"comment",
"current_address",
"current_document",
"current_procedure",
"goto_address",
"inline_comment",
"list_bookmarks",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"next_address",
"prev_address",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"search_procedures",
"search_strings",
"set_address_name",
"set_addresses_names",
"set_bookmark",
"set_comment",
"set_current_document",
"set_inline_comment",
"unset_bookmark",
"xrefs"
]
},
{
"id": "ida",
"name": "IDA Pro MCP adapter",
"version": "1",
"capabilities": [
"analyze_function",
"list_procedures",
"list_strings",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_pseudo_code",
"read_function_instructions",
"search_procedures",
"search_strings",
"xrefs"
]
},
{
"id": "jadx",
"name": "JADX headless",
"version": "0.7.1",
"capabilities": [
"inspect_android_class",
"inspect_android_method",
"inspect_android_package",
"search_android_classes",
"trace_android_references"
]
},
{
"id": "mitmproxy-native-tnetstring",
"name": "REA offline native mitmproxy adapter",
"version": "12.2.3",
"capabilities": ["inspect_web_network_capture"]
},
{
"id": "native-macos",
"name": "macOS native inspection utilities",
"version": null,
"capabilities": [
"capture_native_ui_scenario",
"demangle_swift",
"inspect_macho",
"inspect_plist",
"inspect_signature",
"list_architectures",
"observe_native_calls",
"observe_native_ui"
]
},
{
"id": "pwntools-elf",
"name": "REA pwntools ELF adapter",
"version": "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
"capabilities": ["inspect_binary_layout", "inspect_recorded_crash"]
},
{
"id": "rea-android-application",
"name": "REA Android application workflows",
"version": "1",
"capabilities": ["project_android_application_graph"]
},
{
"id": "rea-apple-application",
"name": "REA Apple application workflows",
"version": "1",
"capabilities": ["project_apple_application_graph"]
},
{
"id": "rea-artifact-graph",
"name": "REA safe artifact graph provider",
"version": "1",
"capabilities": [
"decode_interface_builder",
"extract_artifact",
"inspect_artifact",
"inspect_asset_catalog",
"inspect_keyed_archive",
"trace_dylib_resolution"
]
},
{
"id": "rea-cdp-browser",
"name": "REA Chrome DevTools Protocol observation provider",
"version": "2",
"capabilities": [
"analyze_web_bundle",
"capture_web_screenshot",
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
{
"id": "rea-cdp-electron",
"name": "REA Electron file-page CDP observation provider",
"version": "1",
"capabilities": ["inspect_electron_page", "list_electron_targets"]
},
{
"id": "rea-dotnet-static",
"name": "REA managed static analysis provider",
"version": "1",
"capabilities": [
"inspect_managed_artifact",
"inspect_managed_members",
"inspect_managed_native_boundaries"
]
},
{
"id": "rea-dotnet-workflows",
"name": "REA managed-code workflows",
"version": "1",
"capabilities": [
"compare_managed_members",
"import_managed_reconstruction",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
},
{
"id": "rea-javascript-application",
"name": "REA JavaScript application analyzer",
"version": "1",
"capabilities": ["analyze_javascript_application"]
},
{
"id": "rea-javascript-application-workflows",
"name": "REA JavaScript application workflows",
"version": "1",
"capabilities": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"trace_application_feature",
"trace_javascript_semantics"
]
},
{
"id": "rea-javascript-runtime-reconciliation",
"name": "REA JavaScript runtime reconciliation",
"version": "1",
"capabilities": ["reconcile_javascript_runtime"]
},
{
"id": "rea-playwright-browser-scenario",
"name": "REA Playwright browser scenario capture provider",
"version": "1",
"capabilities": ["capture_browser_scenario"]
},
{
"id": "rea-playwright-electron-active",
"name": "REA Playwright active Electron observation provider",
"version": "1",
"capabilities": ["capture_electron_scenario"]
},
{
"id": "rea-v8-inspector",
"name": "REA passive Node/Electron V8 Inspector provider",
"version": "1",
"capabilities": [
"list_javascript_runtime_targets",
"observe_javascript_runtime"
]
},
{
"id": "rea-web-script-export",
"name": "REA captured script export",
"version": "1",
"capabilities": [
"export_web_scripts",
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
"id": "unblob",
"name": "Unblob",
"version": "26.6.4",
"capabilities": ["extract_firmware"]
},
{
"id": "wakaru",
"name": "Wakaru",
"version": "1.13.0",
"capabilities": ["recover_javascript_sources"]
}
],
"setup_clients": [
{
"id": "claude_code",
"display_name": "Claude Code",
"format": "json",
"configuration": "managed"
},
{
"id": "claude_desktop",
"display_name": "Claude Desktop",
"format": "json",
"configuration": "managed"
},
{
"id": "codex",
"display_name": "Codex",
"format": "toml",
"configuration": "managed"
},
{
"id": "cursor",
"display_name": "Cursor",
"format": "json",
"configuration": "managed"
},
{
"id": "gemini_cli",
"display_name": "Gemini CLI",
"format": "json",
"configuration": "managed"
},
{
"id": "windsurf",
"display_name": "Windsurf",
"format": "json",
"configuration": "managed"
},
{
"id": "devin",
"display_name": "Devin",
"format": "json",
"configuration": "managed"
},
{
"id": "opencode",
"display_name": "OpenCode",
"format": "opencode",
"configuration": "managed"
},
{
"id": "antigravity",
"display_name": "Antigravity",
"format": "json",
"configuration": "managed"
},
{
"id": "copilot_cli",
"display_name": "GitHub Copilot CLI",
"format": "copilot_cli",
"configuration": "managed"
},
{
"id": "commandcode",
"display_name": "Command Code",
"format": "commandcode",
"configuration": "managed"
},
{
"id": "vscode",
"display_name": "VS Code",
"format": "vscode",
"configuration": "managed"
}
],
"cli": {
"primary_count": 94,
"commands": [
"address-to-file-offset",
"analyze",
"analyze-javascript-application",
"analyze-web-bundle",
"annotate-native-function",
"build-reconstruction-obligation-ledger",
"capabilities",
"capture-browser-scenario",
"capture-electron-scenario",
"capture-native-ui-scenario",
"capture-process",
"capture-web-screenshot",
"compare",
"compare-application-versions",
"compare-javascript-export-shapes",
"compare-managed-members",
"compare-process-captures",
"compare-source-to-bundle",
"compare-web-captures",
"compare-web-screenshots",
"decode-interface-builder",
"decompile",
"demangle-swift",
"discover-webmcp-tools",
"doctor",
"evaluate-reconstruction-coverage",
"evidence-export",
"evidence-import",
"export-web-scripts",
"extract-artifact",
"extract-firmware",
"function",
"import-managed-reconstruction",
"import-reference-source",
"inspect",
"inspect-android-class",
"inspect-android-method",
"inspect-android-package",
"inspect-artifact",
"inspect-asset-catalog",
"inspect-binary-layout",
"inspect-electron-page",
"inspect-evm-interface",
"inspect-firmware-regions",
"inspect-keyed-archive",
"inspect-macho",
"inspect-managed-artifact",
"inspect-managed-members",
"inspect-managed-native-boundaries",
"inspect-native-api",
"inspect-native-data-type",
"inspect-native-dispatch-metadata",
"inspect-native-instruction",
"inspect-native-load-image",
"inspect-plist",
"inspect-recorded-crash",
"inspect-signature",
"inspect-web-event-listeners",
"inspect-web-network-capture",
"inspect-web-page",
"instructions",
"list-architectures",
"list-browser-targets",
"list-electron-targets",
"list-javascript-runtime-targets",
"observe-javascript-runtime",
"observe-native-calls",
"observe-native-ui",
"observe-web-execution",
"observe-web-session",
"project-android-application-graph",
"project-apple-application-graph",
"project-managed-application-graph",
"providers",
"read-bytes",
"reconcile-javascript-runtime",
"recover-javascript-sources",
"resolve-native-call-targets",
"search",
"search-android-classes",
"setup",
"trace",
"trace-android-references",
"trace-application-feature",
"trace-dylib-resolution",
"trace-javascript-semantics",
"trace-native-ui-action",
"trace-native-values",
"trace-web-module-imports",
"trace-web-source-location",
"uninstall",
"update",
"verify-managed-native-boundaries",
"xrefs"
],
"aliases": [
{
"name": "compare-bundles",
"target": "compare"
}
]
},
"runtime_catalog": {
"counts": {
"cli_commands": 94,
"mcp_tools": 139,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "9ee5392d18f04f25b5c97c1b69a3644a0d81e52c813167a6f140791bc692741d",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "4c5f67d076baf821b5cb6d6fb2cc0f6c274e96b627b85b892577760a6f71ab1a",
"providers_sha256": "954b499c76514890062d2b6c936d73cb3a9c45412d522a081125a325e81cdec5"
}
}
}
+52 -5
View File
@@ -11,6 +11,32 @@ breaking changes. Record the full source SHA and create `release/VERSION` at
that commit. The version is a maintainer choice; Release Please's proposed
version must agree before publication.
`release/VERSION` also sets the expected version for the workflow. Use an exact
SemVer such as `release/6.1.0` or `release/6.1.0-rc.1`, without build metadata.
The checkpoint validator rejects a candidate whose package, lockfile, manifest,
registry metadata or changelog disagrees with that version before creating a tag.
### Compatibility and version selection
REA uses the default SemVer strategy: compatible fixes increment patch;
compatible capabilities increment minor; incompatible public behavior increments
major. Adding tools, internal refactoring and correcting behavior outside the
supported contract do not by themselves require a major release.
The supported public surface includes documented CLI commands/options, MCP tool
names and input/result contracts, saved evidence formats and supported runtime
requirements. Before adding `!` or a `BREAKING CHANGE` footer, a PR must identify
a previously valid call or configuration that will fail or change meaning,
explain why compatibility cannot be preserved, and give its migration. Prefer
optional additions, compatibility adapters and a documented deprecation period;
group deliberate removals into a planned major release. Maintainers review this
impact before merging; the bot parses markers and cannot infer compatibility.
For example, adding an optional inspection tool is a minor change. Rejecting a
relative MCP path previously accepted by the public contract is breaking;
rejecting an input that the existing contract already prohibited is a fix.
Changing the version number alone does not restore compatibility.
For example, after selecting a reviewed commit for 5.0.0:
```bash
@@ -48,6 +74,24 @@ normalization, then approve that head's blocked runs from the PR page. Updates
to the bot branch can create new approval-required runs; approval of an older
head does not verify the normalized candidate. Wait for the final head's CI.
The validator reads the actual ancestry range from the published baseline tag
to the selected checkpoint, including visible first-parent Conventional Commits
and the PR titles in GitHub's default merge messages. The report preserves the
original merge subject alongside the extracted Conventional Commit title.
This matters after merging a side-branch release back into a newer main:
Release Please's chronological history cutoff can omit unreleased mainline
commits. Unreleased breaking markers require a major increment and references
in the new release's breaking-change migration section. Other omitted entries
are reported for review. Historical notes from an older release cannot satisfy
the new release's migration check. This audits declared markers and references;
it does not prove API compatibility or the quality of a migration explanation.
If preparation reports a mismatch or omitted breaking changes, inspect the
recorded ancestry report, correct the release PR's version artifacts and notes,
regenerate documentation, and review/test its final head. Do not repeatedly
prepare over manual corrections. Publication independently checks the merged
candidate again before Release Please can create a tag.
Review the candidate's version, notes, generated metadata, and package
contents. Wait for the candidate's CI and relevant real-provider checks.
Routine local iterations need focused checks; CI owns full deterministic
@@ -100,8 +144,8 @@ before cutting the next checkpoint; otherwise main retains the previous
release baseline and can propose an already-published version again.
After publication, open a PR from the release branch back to main. Preserve
main's later implementation changes and resolve generated-file conflicts by
regenerating from the combined contracts with the released package version.
main's later implementation changes and regenerate ignored build outputs from
the combined contracts with the released package version.
Review and test this synchronization PR, then use a merge commit so the release
tag remains in main's ancestry. Keep the released tag unchanged.
@@ -119,9 +163,12 @@ git merge-base --is-ancestor rea-agents-5.0.0 origin/main
Close any superseded rolling release PR. Future releases repeat the checkpoint
procedure from main; never resume automatic release-PR refreshes on main pushes.
The generated-metadata workflow stays limited to pull requests into main so it
does not push commits onto a frozen candidate. Preparation normalizes
`docs/product-catalog.json` on the bot pull request.
Preparation generates and validates the catalog, portable conformance
projections, and packaged skill from the candidate checkout without committing
them. The checkpoint validator binds tracked version authority to the source
SHA; generated-catalog and package checks validate the derived representation.
No generated-metadata workflow pushes follow-up commits onto feature or release
branches.
## Partial publication and retries
+11 -9
View File
@@ -425,7 +425,7 @@ behavior, and complete CI evidence before merging.
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks |
| `npm run docs:check` | Committed generated-document freshness |
| `npm run docs:check` | Generated-document validation from current source and build outputs |
For example:
@@ -460,19 +460,21 @@ broad changes or diagnosing CI, rather than after every edit. Package/install
changes additionally need package verification; provider changes need actual
provider evidence.
Local full-suite Vitest runs use up to two workers and schedule projects one
at a time. Process, acceptance and process-global projects serialize their
files to prevent competing lifecycle observations. CI retains its two-worker
budget.
Each Vitest project uses up to two workers, bounded by available host parallelism.
Default-group projects can run together. Only `process-boundary` runs in the
later sequence group and serializes its files because process-tree observations
share host sampling resources. Acceptance and process-global files retain isolated
forks but are not serialized; isolation does not imply serial scheduling. CI uses
the same per-project worker bound.
The pure domain/contracts and recording-port service projects share one worker
module context because their tests own no mutable runtime resources. MCP
boundary files also share the immutable server module graph while creating and
closing independent in-memory sessions. Adapter, composition, acceptance,
process-global, and other boundary projects retain per-file isolation.
`npm test`, `npm run docs:check`, and `npm run docs:generate` share
repository-local locks and fail fast when the same class of command is already
running. The `npm test` build is inside that lock. `check:pr` runs its test task
before starting generated-document validation.
Build and documentation writers use checkout-local locks for their respective
output files. `npm test` delegates its build to the build lock; the Vitest suite
does not hold a test-command lock. `check:pr` runs its test task before starting
generated-document validation.
Vitest and Node persistent compile caches are deliberately not enabled by
default. To evaluate repeated local runs, opt in for both cold and warm
+12 -1
View File
@@ -80,7 +80,18 @@ or model-authored loop when one call can answer the question.
## Implement and evaluate
- Put shared user workflows in the application layer; keep MCP translation in
the server adapter and CLI behavior aligned with the same workflow.
the server adapter and CLI behavior aligned with the same workflow. Share
Evidence provenance, unknown projection, and eligible snapshot binding through
application/session owners. Matching result payloads alone do not establish
equivalent retention or replay behavior.
- Derive each facet's coverage from examined records, unsupported representations,
and exhaustiveness. Keep issues structured until presentation; do not select a
facet by matching diagnostic prose or infer completeness from an empty failure
list. Preserve available observations independently of cleanup success.
- Measure the resource consumed before choosing a limit. Value alternatives,
retained records, trace-frame products, output bytes, and temporary-file growth
need different accounting. Enforce a justified budget before the costly step
and report its effect; an input-byte or graph-node cap does not bound every stage.
- Update the canonical contract inventory, output schemas, examples, generated
catalog, and relevant docs together.
- Verify valid, malformed, boundary, unavailable, cancellation, and partial
@@ -1,105 +0,0 @@
{
"ledger_id": "ecl_132e857a8a93f0b366cd72f9cfa1c338ffd4100460a7834ef69abff936c84472",
"records": [
{
"claim_id": "managed.application-graph-projection",
"status": "pass",
"evidence_ids": [
"ev_b061a5a559acb3748cb9784c637f6693f74faeb564ba2b11d6fc32619e757459"
]
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"status": "pass",
"evidence_ids": [
"ev_13f3401a3aaddc3f6d087ad160fa928374e142cdb9861254359b46865b8c992e"
]
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"status": "pass",
"evidence_ids": [
"ev_6f026216826ca06e080b77ff608f02634fa945fbd2e151b4cd261c1444f8bbb1"
]
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"status": "unsupported",
"evidence_ids": [
"ev_65a94c4de03f9c6ebdaff3f381cca377821f349d3ca6dfad47a31cc586fae939"
]
},
{
"claim_id": "managed.malformed-metadata",
"status": "pass",
"evidence_ids": [
"ev_deae4259ea4395623176de40d06c2abcbc2b3a77f0bfb068d76e568d801f45a8"
]
},
{
"claim_id": "managed.managed-native-verification",
"status": "pass",
"evidence_ids": [
"ev_0edcbbcaa1191a4e5324a437f739957b77f9ac45e7897724337f2d6d06b3f9d1"
]
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"status": "pass",
"evidence_ids": [
"ev_d1af50e8dc04cd589385769efd8758e82095425f6e9c1cc4bf2f530c6b434c93"
]
},
{
"claim_id": "managed.mvid-and-token-drift",
"status": "pass",
"evidence_ids": [
"ev_5493f40bc875535488d789a490361abcc1f76bfbdf52536d09ff127442a8e854"
]
},
{
"claim_id": "managed.not-managed",
"status": "pass",
"evidence_ids": [
"ev_03ce0cc0183ee0710f8120e837565129559ab3bc41927df326066370c2341699"
]
},
{
"claim_id": "managed.operator-local-manifest",
"status": "unsupported",
"evidence_ids": [
"ev_9d3e50da1d372c55f46649e69eae702cbe8f0ba6a81d1e46afc3e1a284c81757"
]
},
{
"claim_id": "managed.operator-manifest-self-test",
"status": "pass",
"evidence_ids": [
"ev_f974f4b9d039cda841894c6173cae86427ecef48f4e7a4d6c73edf7b23af2b2a"
]
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"status": "pass",
"evidence_ids": [
"ev_c4f38ce76d8359d92ee8e9442503e47b7913550442349a0df688bd274a3def67"
]
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"status": "pass",
"evidence_ids": [
"ev_3e05ef1d6a796d3d4cb3372fe8e549efec759c59bf09d8ff660059a2e448cbc0"
]
}
],
"summary": {
"total": 13,
"pass": 11,
"fail": 0,
"unsupported": 2,
"truncated": 0,
"unknown": 0,
"complete": false
}
}
@@ -1,201 +0,0 @@
{
"manifest_id": "ecm_8a67e7a3578e49c8d8d22107eab8f1258094dc40041d14deee93189fd0c514b2",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
},
"environment": {
"platform": "portable",
"architecture": "portable",
"runtime": "node",
"runtime_version": "portable"
},
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "ef398cf4b5ee7fb5e0bcf42ca787049ec145e1d911ee7951017fee1cb3980d74"
}
],
"claims": [
{
"claim_id": "managed.application-graph-projection",
"scenario": {
"id": "managed-application-graph-projection",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"scenario": {
"id": "decompiler-reconstruction-import",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"scenario": {
"id": "dotnet-framework-x86-pinvoke",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"scenario": {
"id": "ilspy-reconstruction-oracle",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.malformed-metadata",
"scenario": {
"id": "malformed-metadata",
"version": 1
},
"artifact_sha256s": [
"827e10b6391d29edca48c191e2ddd79f3bc1165e46e4ba2f4edf41ab60f45e78"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.managed-native-verification",
"scenario": {
"id": "managed-native-verification",
"version": 1
},
"artifact_sha256s": [
"9999999999999999999999999999999999999999999999999999999999999999",
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"scenario": {
"id": "modern-dotnet-anycpu",
"version": 1
},
"artifact_sha256s": [
"92b588ae43d418117fa108c0004b5d0451f6112238d21bad8d75c0eb1da860f0"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.mvid-and-token-drift",
"scenario": {
"id": "mvid-and-token-drift",
"version": 1
},
"artifact_sha256s": [
"276395c671a656ab57d583158443c10de552dccfb37dc97cd013f673c0f86b43",
"ef009dab7424fdeba996f2524dbe86fd899c96394498cca0463fcfdb228c5572"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.not-managed",
"scenario": {
"id": "not-managed",
"version": 1
},
"artifact_sha256s": [
"de74dab8ab034054a811d4248efcda0bed505187a061a806e274ae87bc453754"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.operator-local-manifest",
"scenario": {
"id": "operator-local-manifest",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.operator-manifest-self-test",
"scenario": {
"id": "operator-manifest-self-test",
"version": 1
},
"artifact_sha256s": [
"cdc11ec7fd0f392e43b9b23971cb41ec49fd0a997fc33c24fcc553cd4386ae37"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"scenario": {
"id": "unicode-obfuscated-identifiers",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"scenario": {
"id": "x64-ready-to-run-native-body",
"version": 1
},
"artifact_sha256s": [
"e47402428322cdcb06f561036d7d108528cbc6e262d92a21f869218a20c80bfb"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
}
]
}
+3 -3
View File
@@ -32,9 +32,9 @@ HAR uses unchanged `har-schema@2.0.0` draft-06 schemas, `ajv@8.20.0`,
`lossless-json@4.3.1` numeric representations, pinned in the npm lockfile.
Only the HAR 1.2 profile is supported. Unknown extensions remain reported
evidence. Duplicate object keys, including equal values and escaped spellings,
are rejected before materialization. A `__proto__` member returns an explicit
unsupported-boundary error in both formats: the current shared JSON schema
cannot preserve this member. Original capture files remain unchanged.
are rejected before materialization. Prototype-named members such as
`__proto__` are preserved as ordinary own JSON members without prototype
mutation. Original capture files remain unchanged.
The observed mitmproxy 12.2.3 `SaveHar` profile can emit `postData.text: null`
for a missing request body. REA preserves that reported null, omitting the
optional field only from its validation copy for this exact producer profile;
+1 -1
View File
@@ -107,7 +107,7 @@ if [[ "$dry_run" == true ]]; then
fi
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
npm install --global "${prefix_args[@]}" "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
npm install --global ${prefix_args[@]+"${prefix_args[@]}"} "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
+5 -2
View File
@@ -87,7 +87,8 @@
"mcp-catalog:check": "node scripts/generate-mcp-tool-catalog.mjs --check",
"prebuild": "npm run deps:check && npm run metadata:check",
"build": "node scripts/run-exclusive.mjs build npm run build:unlocked",
"build:unlocked": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/generate-mcp-tool-catalog.mjs",
"build:unlocked": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && npm run build:projections",
"build:projections": "node scripts/generate-mcp-tool-catalog.mjs && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && node scripts/generate-completion-ledger.mjs",
"precheck": "npm run deps:check",
"check": "turbo run typecheck lint format:check knip metadata:check",
"precheck:fast": "npm run deps:check",
@@ -179,11 +180,13 @@
"config:print": "node scripts/print-mcp-config.mjs",
"evidence:generate": "npm run build:cached && node scripts/generate-completion-ledger.mjs",
"evidence:check": "npm run build:cached && node scripts/generate-completion-ledger.mjs --check",
"predocs:dev": "npm run docs:generate",
"docs:dev": "vitepress dev docs",
"predocs:build": "npm run docs:generate",
"docs:build": "vitepress build docs",
"docs:preview": "vitepress preview docs",
"docs:generate": "node scripts/run-exclusive.mjs docs npm run docs:generate:unlocked",
"docs:generate:unlocked": "npm run metadata:generate && npm run build:cached && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && node scripts/generate-completion-ledger.mjs && node scripts/generate-error-schema.mjs",
"docs:generate:unlocked": "npm run metadata:generate && npm run build:cached && node scripts/generate-error-schema.mjs",
"docs:check": "node scripts/run-exclusive.mjs docs turbo run docs:check:built",
"docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-mcp-tool-catalog.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && node scripts/generate-error-schema.mjs --check",
"docs:error-schema": "npm run build:cached && node scripts/generate-error-schema.mjs",
-5
View File
@@ -34,11 +34,6 @@
"type": "json",
"path": "server.json",
"jsonpath": "$.packages[0].version"
},
{
"type": "json",
"path": "docs/product-catalog.json",
"jsonpath": "$.package.version"
}
]
}
+1 -1
View File
@@ -30,7 +30,7 @@ const { stdout } = await exec(
},
);
const verifierOutput = JSON.parse(stdout);
// Managed conformance claims are host-independent. Keep the checked-in
// Managed conformance claims are host-independent. Keep the generated
// commitment reproducible across macOS, Linux, and Windows rather than
// embedding the machine that happened to generate it.
const portableCompletionReport = {
+1 -1
View File
@@ -17,7 +17,7 @@ const lock = JSON.parse(
await readFile(join(root, "package-lock.json"), "utf8"),
);
const skill = await readFile(
join(root, "skills/reverse-engineer-anything/SKILL.md"),
join(root, "skill-src/reverse-engineer-anything/SKILL.md"),
"utf8",
);
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
+4 -1
View File
@@ -1,3 +1,4 @@
import { mkdir } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -14,8 +15,10 @@ for (const argument of arguments_)
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const catalog = await createProductCatalog(root);
const path = join(root, "docs/public/product-catalog.json");
if (!arguments_.has("--check")) await mkdir(dirname(path), { recursive: true });
await ensureGeneratedFile({
path: join(root, "docs/product-catalog.json"),
path,
source: await serializeProductCatalog(catalog),
check: arguments_.has("--check"),
generateCommand: "npm run docs:generate",
+62 -19
View File
@@ -1,4 +1,4 @@
import { readFile } from "node:fs/promises";
import { mkdir, readFile, readdir, rm } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
@@ -10,26 +10,69 @@ for (const argument of arguments_)
throw new Error(`Unknown skill metadata option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const path = join(root, "skills/reverse-engineer-anything/SKILL.md");
const current = await readFile(path, "utf8");
const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
const outputRoot = join(root, "skills/reverse-engineer-anything");
const check = arguments_.has("--check");
const current = (
await readFile(join(sourceRoot, "SKILL.md"), "utf8")
).replaceAll("\r\n", "\n");
const { CATALOG_IDENTITY } = await import(
`${pathToFileURL(join(root, "dist/catalogIdentity.js")).href}?${String(Date.now())}`
);
const withCount = current.replace(
/^\s{2}tool_count:\s*\d+\s*$/mu,
` tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}`,
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
throw new Error(
"Catalog metadata belongs in the generated skill, not skill-src",
);
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
if (!versionLine.test(current))
throw new Error("Missing authored skill version");
const source = current.replace(
versionLine,
`$&\n tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}`,
);
const digestLine = ` catalog_digest: "${CATALOG_IDENTITY.digests.combined_sha256}"`;
const source = /^\s{2}catalog_digest:/mu.test(withCount)
? withCount.replace(
/^\s{2}catalog_digest:\s*"[a-f0-9]{64}"\s*$/mu,
digestLine,
)
: withCount.replace(/^(\s{2}tool_count:\s*\d+\s*)$/mu, `$1\n${digestLine}`);
const paths = await filePaths(sourceRoot);
// Rebuild this owned output directory so removed references cannot survive a build.
if (!check) await rm(outputRoot, { recursive: true, force: true });
for (const relativePath of paths) {
const path = join(outputRoot, relativePath);
if (!check) await mkdir(dirname(path), { recursive: true });
await ensureGeneratedFile({
path,
source:
relativePath === "SKILL.md"
? source
: (await readFile(join(sourceRoot, relativePath), "utf8")).replaceAll(
"\r\n",
"\n",
),
check,
generateCommand: "npm run docs:generate",
});
}
if (
check &&
JSON.stringify(await filePaths(outputRoot)) !== JSON.stringify(paths)
)
throw new Error(
"Generated skill file inventory drifted; run npm run docs:generate",
);
await ensureGeneratedFile({
path,
source,
check: arguments_.has("--check"),
generateCommand: "npm run docs:generate",
});
async function filePaths(directory, prefix = "") {
const paths = [];
for (const entry of (await readdir(directory, { withFileTypes: true })).sort(
(left, right) =>
left.name < right.name ? -1 : left.name > right.name ? 1 : 0,
)) {
const relativePath = prefix === "" ? entry.name : `${prefix}/${entry.name}`;
if (entry.isDirectory())
paths.push(
...(await filePaths(join(directory, entry.name), relativePath)),
);
else if (entry.isFile()) paths.push(relativePath);
else
throw new Error(
`Skill bundle requires regular files or directories: ${relativePath}`,
);
}
return paths;
}
+86 -2
View File
@@ -1,5 +1,8 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import { fileURLToPath } from "node:url";
import { chromium } from "playwright-core";
@@ -62,13 +65,21 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
);
assert.equal(bytes.readUInt32BE(16), 2048);
assert.equal(bytes.readUInt32BE(20), 1536);
await verifyDefaultScreenshotDelivery({
entrypoint,
environment: env,
endpoint,
targetId: target.id,
screenshot,
});
const maxBufferSize = 64 * 1024 * 1024;
transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env,
env: { ...env, REA_MCP_MAX_RESPONSE_BYTES: String(maxBufferSize) },
stderr: "pipe",
// The SDK defaults to a 10 MiB receive buffer; inline PNG output is larger.
maxBufferSize: 64 * 1024 * 1024,
maxBufferSize,
});
client = new Client({ name: "browser-screenshot-real-e2e", version: "1" });
await client.connect(transport);
@@ -102,6 +113,9 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
mocked: false,
cli: true,
stdio_mcp: true,
default_buffer_recovery: true,
complete_export: true,
matched_large_buffer: true,
png_bytes: bytes.byteLength,
};
} finally {
@@ -117,3 +131,73 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
}
}
}
const verifyDefaultScreenshotDelivery = async ({
entrypoint,
environment,
endpoint,
targetId,
screenshot,
}) => {
const workspace = await mkdtemp(join(tmpdir(), "rea-screenshot-delivery-"));
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
stderr: "pipe",
});
const client = new Client({
name: "browser-screenshot-default-e2e",
version: "1",
});
try {
await client.connect(transport);
const constrained = await client.callTool(
{
name: "capture_web_screenshot",
arguments: { cdp_endpoint: endpoint, target_id: targetId },
},
{ timeout: 60000 },
);
assert.equal(constrained.isError, true);
assert.equal(
constrained.structuredContent?.error?.details?.resource,
"transport",
);
const reference =
constrained.structuredContent?.error?.details?.reported_limits
?.evidence_reference;
assert.equal(reference?.kind, "retained-evidence");
await client.ping();
const exported = await client.callTool({
name: "export_evidence_bundle",
arguments: { path: join(workspace, "screenshot-evidence.json") },
});
assert.notEqual(exported.isError, true, JSON.stringify(exported));
const bundle = JSON.parse(
await readFile(join(workspace, "screenshot-evidence.json"), "utf8"),
);
const record = bundle.records.find(
(record) => record.evidence_id === reference.evidence_id,
);
assert.ok(
record,
"Full screenshot Evidence must remain exportable after the delivery constraint",
);
const retained = webScreenshotSchema.parse(
parseEvidence(record).normalized_result,
);
assert.deepEqual(retained.artifact, screenshot.artifact);
await client.ping();
} finally {
try {
await client.close();
} finally {
try {
await transport.close();
} finally {
await rm(workspace, { recursive: true, force: true });
}
}
}
};
+13 -3
View File
@@ -77,7 +77,12 @@ export const documentationFactIssues = async (root, catalog) => {
requireText(issues, path, content, "MCP-tool_catalog");
if (path === "README.md") {
requireText(issues, path, content, "(docs/installation.md");
requireText(issues, path, content, "(docs/product-catalog.json)");
requireText(
issues,
path,
content,
"(docs/mcp-contracts.md#generated-catalog)",
);
continue;
}
for (const client of catalog.setup_clients)
@@ -99,11 +104,16 @@ export const documentationFactIssues = async (root, catalog) => {
requireText(issues, installationPath, installation, client.display_name);
const agents = await readFile(join(root, "AGENTS.md"), "utf8");
requireText(issues, "AGENTS.md", agents, "docs/product-catalog.json");
requireText(issues, "AGENTS.md", agents, "docs/public/product-catalog.json");
const templatePath = ".github/pull_request_template.md";
const template = await readFile(join(root, templatePath), "utf8");
requireText(issues, templatePath, template, "docs/product-catalog.json");
requireText(
issues,
templatePath,
template,
"docs/public/product-catalog.json",
);
return issues;
};
+4 -1
View File
@@ -10,7 +10,10 @@ export const withLargeResultMcp = async (
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
env: {
...environment,
REA_MCP_MAX_RESPONSE_BYTES: String(maxBufferSize),
},
stderr: "inherit",
maxBufferSize,
});
+3 -3
View File
@@ -58,16 +58,16 @@ export const createProductCatalog = async (root) => {
runtime_catalog: {
counts: sources.catalogIdentity.CATALOG_IDENTITY.counts,
digests: {
...sources.catalogIdentity.CATALOG_IDENTITY.digests,
// Full runtime schema digests belong to server identity, not this facts projection.
providers_sha256: providerCatalogDigest(providers),
},
},
};
};
/** Stable checked-in representation of the product catalog. */
/** Stable build-generated representation of the product catalog. */
export const serializeProductCatalog = (catalog) =>
formatGeneratedFile(
"docs/product-catalog.json",
"docs/public/product-catalog.json",
JSON.stringify(catalog, null, 2),
);
+333
View File
@@ -0,0 +1,333 @@
import { gt, major, valid } from "semver";
import { z } from "zod";
const shaSchema = z.string().regex(/^[a-f0-9]{40}$/u);
const versionSchema = z.object({ version: z.string() });
const manifestSchema = z.object({ ".": z.string() });
const sectionsSchema = z.object({
"changelog-sections": z.array(
z.object({ type: z.string(), hidden: z.boolean().optional() }),
),
});
const GITHUB_MERGE_SUBJECT = /^Merge pull request #(\d+) from \S+/u;
function conventionalTitle(subject, body) {
if (!GITHUB_MERGE_SUBJECT.test(subject)) return subject;
// GitHub's default merge message puts the PR title after its subject.
// Inspect that first nonempty line, not arbitrary examples in commit prose.
return (
body
.split(/\r?\n/u)
.slice(1)
.find((line) => line.trim() !== "")
?.trim() ?? subject
);
}
function releaseVersion(value, location) {
if (valid(value) !== value || value.includes("+")) {
throw new Error(
`${location} must contain an exact SemVer without build metadata: ${value}`,
);
}
return value;
}
async function readJson(git, sha, path, schema) {
const text = await git(["show", `${sha}:${path}`]);
let value;
try {
value = JSON.parse(text);
} catch (error) {
if (error instanceof SyntaxError) {
throw new Error(`${path}@${sha} contains invalid JSON: ${error.message}`);
}
throw error;
}
const parsed = schema.safeParse(value);
if (!parsed.success) {
throw new Error(`${path}@${sha} is malformed: ${parsed.error.message}`);
}
return parsed.data;
}
function readCommits(text) {
if (text.length === 0) return [];
const fields = text.split("\0");
if (fields.pop() !== "" || fields.length % 3 !== 0) {
throw new Error("Git returned malformed checkpoint history records.");
}
const commits = [];
for (let index = 0; index < fields.length; index += 3) {
const sha = shaSchema.parse(fields[index]);
const subject = z.string().parse(fields[index + 1]);
const body = z.string().parse(fields[index + 2]);
const title = conventionalTitle(subject, body);
const type = /^([a-z]+)(?:\([^)]*\))?!?: /u.exec(title)?.[1];
commits.push({
sha,
subject,
...(title !== subject ? { conventionalTitle: title } : {}),
type,
breaking:
/^[a-z]+(?:\([^)]*\))?!: /u.test(title) ||
/^BREAKING(?: CHANGE|-CHANGE):\s*\S/mu.test(body),
});
}
return commits;
}
function referencesCommit(text, commit) {
if (text.includes(commit.sha.slice(0, 7))) return true;
const pullRequest =
GITHUB_MERGE_SUBJECT.exec(commit.subject)?.[1] ??
/\(#(\d+)\)$/u.exec(commit.subject)?.[1];
return (
pullRequest !== undefined &&
new RegExp(`#${pullRequest}(?!\\d)`, "u").test(text)
);
}
function releaseNotes(text, expectedVersion) {
const headings = Array.from(text.matchAll(/^## \[([^\]]+)\].*$/gmu));
const first = headings[0];
if (first?.[1] !== expectedVersion || first.index === undefined) {
throw new Error(`CHANGELOG.md must start with release ${expectedVersion}.`);
}
return text.slice(first.index, headings[1]?.index ?? text.length);
}
function breakingSection(notes) {
const heading = /^### [^\n]*BREAKING CHANGES[^\n]*\n/imu.exec(notes);
if (heading === null) return "";
const body = notes.slice(heading.index + heading[0].length);
const end = body.search(/^### /mu);
return end < 0 ? body : body.slice(0, end);
}
async function requireAncestor(git, ancestor, descendant, description) {
try {
await git(["merge-base", "--is-ancestor", ancestor, descendant]);
} catch (error) {
const failure = z.object({ code: z.literal(1) }).safeParse(error);
if (!failure.success) throw error;
throw new Error(
`${description}: ${ancestor} is not an ancestor of ${descendant}. Sync the released tag into main before cutting a checkpoint, and prepare candidates from that frozen source.`,
);
}
}
async function resolveBaseline(git, tag) {
try {
return shaSchema.parse(
(await git(["rev-parse", "--verify", `${tag}^{commit}`])).trim(),
);
} catch (error) {
const failure = z.object({ code: z.literal(128) }).safeParse(error);
if (!failure.success) throw error;
throw new Error(
`Published baseline ${tag} is unavailable. Fetch release tags and finish the previous release sync before preparing this checkpoint. ${error instanceof Error ? error.message : String(error)}`,
);
}
}
async function validateCandidate(git, sha, version, commits) {
const versions = [
[
"package.json",
(await readJson(git, sha, "package.json", versionSchema)).version,
],
[
".release-please-manifest.json",
(
await readJson(
git,
sha,
".release-please-manifest.json",
manifestSchema,
)
)["."],
],
];
const lock = await readJson(
git,
sha,
"package-lock.json",
versionSchema.extend({
packages: z.object({ "": versionSchema }),
}),
);
versions.push(
["package-lock.json", lock.version],
["package-lock.json packages.root", lock.packages[""].version],
);
const server = await readJson(
git,
sha,
"server.json",
versionSchema.extend({
packages: z.array(
z.object({ identifier: z.string(), version: z.string() }),
),
}),
);
const npmPackage = server.packages.find(
(entry) => entry.identifier === "rea-agents",
);
if (npmPackage === undefined)
throw new Error("server.json is missing the rea-agents package.");
versions.push(
["server.json", server.version],
["server.json rea-agents package", npmPackage.version],
);
// Catalogs are generated and checked from this candidate by the release job;
// they are not version authority stored in the immutable source tree.
const generated = await git([
"show",
`${sha}:src/generatedPackageMetadata.ts`,
]);
const generatedVersions = Array.from(
generated.matchAll(/^\s+version: "([^"]+)"/gmu),
);
if (generatedVersions.length !== 1)
throw new Error(
"Generated package metadata must declare one literal version.",
);
versions.push(["src/generatedPackageMetadata.ts", generatedVersions[0]?.[1]]);
for (const [location, actual] of versions) {
if (actual !== version) {
throw new Error(
`${location}@${sha} declares ${actual}; release branch requires ${version}.`,
);
}
}
const notes = releaseNotes(
await git(["show", `${sha}:CHANGELOG.md`]),
version,
);
const breakingNotes = breakingSection(notes);
const missingBreaking = commits.filter(
(commit) => commit.breaking && !referencesCommit(breakingNotes, commit),
);
if (missingBreaking.length > 0) {
throw new Error(
`Release ${version} migration notes omit unreleased breaking changes: ${missingBreaking.map((commit) => `${commit.sha} ${commit.subject}`).join("; ")}`,
);
}
return commits.filter((commit) => !referencesCommit(notes, commit));
}
/** Validate a recorded checkpoint and optional candidate using exact Git ancestry. */
export async function inspectReleaseCheckpoint(git, options) {
const { phase, stage, releaseBranch, sourceSha, candidateSha } = z
.object({
phase: z.enum(["prepare", "publish"]),
stage: z.enum(["source", "candidate"]),
releaseBranch: z.string().startsWith("release/"),
sourceSha: shaSchema,
candidateSha: shaSchema.optional(),
})
.parse(options);
const expectedVersion = releaseVersion(
releaseBranch.slice("release/".length),
"release/VERSION",
);
if (phase === "publish" && stage === "candidate") {
throw new Error(
"Publish validates its merged source SHA; candidate stage applies only to prepare.",
);
}
const applicationSha =
phase === "publish"
? shaSchema.parse((await git(["rev-parse", `${sourceSha}^1`])).trim())
: sourceSha;
const manifest = await readJson(
git,
applicationSha,
".release-please-manifest.json",
manifestSchema,
);
const baselineVersion = releaseVersion(
manifest["."],
"published baseline manifest",
);
const sourcePackage = await readJson(
git,
applicationSha,
"package.json",
versionSchema,
);
if (sourcePackage.version !== baselineVersion) {
throw new Error(
`Source package ${sourcePackage.version} and release baseline ${baselineVersion} disagree.`,
);
}
if (!gt(expectedVersion, baselineVersion)) {
throw new Error(
`Expected release ${expectedVersion} must advance published baseline ${baselineVersion}.`,
);
}
const baselineTag = `refs/tags/rea-agents-${baselineVersion}`;
const baselineSha = await resolveBaseline(git, baselineTag);
await requireAncestor(
git,
baselineSha,
applicationSha,
"Published baseline is outside the checkpoint ancestry",
);
const configuration = await readJson(
git,
applicationSha,
"release-please-config.json",
sectionsSchema,
);
const visibleTypes = new Set(
configuration["changelog-sections"]
.filter((section) => !section.hidden)
.map((section) => section.type),
);
const commits = readCommits(
await git([
"log",
"--first-parent",
"-z",
"--format=%H%x00%s%x00%B",
`${baselineSha}..${applicationSha}`,
]),
).filter((commit) => commit.breaking || visibleTypes.has(commit.type));
const breaking = commits.filter((commit) => commit.breaking);
if (
breaking.length > 0 &&
major(baselineVersion) > 0 &&
major(expectedVersion) <= major(baselineVersion)
) {
throw new Error(
`Expected release ${expectedVersion} contains unreleased breaking markers since ${baselineVersion}: ${breaking.map((commit) => `${commit.sha} ${commit.subject}`).join("; ")}. Review compatibility or select a major version.`,
);
}
let snapshot = phase === "publish" ? sourceSha : candidateSha;
if (stage === "candidate" && snapshot === undefined) {
snapshot = shaSchema.parse((await git(["rev-parse", "HEAD"])).trim());
}
if (phase === "publish" || stage === "candidate") {
await requireAncestor(
git,
applicationSha,
snapshot,
"Candidate is outside the selected checkpoint ancestry",
);
}
const missingNotes =
phase === "publish" || stage === "candidate"
? await validateCandidate(git, snapshot, expectedVersion, commits)
: [];
return {
expectedVersion,
baselineVersion,
baselineSha,
sourceSha,
applicationSha,
commits,
missingNotes,
};
}
+1 -1
View File
@@ -1169,7 +1169,7 @@ try {
);
const ownershipInspection =
["/usr/bin/ps", "/bin/ps"].includes(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]');
assert.ok(
[process.execPath, python].includes(match[1]) || ownershipInspection,
`Unexpected attempted host execution: ${line}`,
+3 -1
View File
@@ -474,7 +474,9 @@ try {
);
const ownershipInspection =
["/usr/bin/ps", "/bin/ps"].includes(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
);
assert.ok(
[
process.execPath,
+21
View File
@@ -1,4 +1,5 @@
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { exec } from "./lib/verify-package-core.mjs";
import { verifyPackedBridge } from "./verify-packed-bridge.mjs";
@@ -28,6 +29,26 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error("package did not expose both rea command entry points");
if (packedManifest.mcpName !== "io.github.morluto/rea")
throw new Error("package did not retain its MCP Registry ownership marker");
if (packedFiles.some((path) => path.startsWith("package/skill-src/")))
throw new Error(
"package included authored skill sources instead of only the generated bundle",
);
const skill = (
await exec("tar", [
"-xOf",
join(root, tarball),
"package/skills/reverse-engineer-anything/SKILL.md",
])
).stdout;
if (
!skill.includes(
` tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}\n`,
) ||
skill.includes(" catalog_digest:")
)
throw new Error(
"packaged skill metadata did not match the catalog shipped in the package",
);
if (
packedFiles.some(
(path) => path.includes("__pycache__") || path.endsWith(".pyc"),
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env node
import { execFile } from "node:child_process";
import { parseArgs, promisify } from "node:util";
import { inspectReleaseCheckpoint } from "./lib/release-checkpoint.mjs";
const execFileAsync = promisify(execFile);
async function git(args) {
const result = await execFileAsync("git", args, {
encoding: "utf8",
maxBuffer: 16 * 1024 * 1024,
});
return result.stdout;
}
async function main() {
const { values } = parseArgs({
options: {
phase: { type: "string" },
stage: { type: "string", default: "source" },
"release-branch": { type: "string" },
"source-sha": { type: "string" },
},
});
const report = await inspectReleaseCheckpoint(git, {
phase: values.phase,
stage: values.stage,
releaseBranch: values["release-branch"],
sourceSha: values["source-sha"],
});
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
if (report.missingNotes.length > 0) {
process.stderr.write(
`Review ${report.missingNotes.length} additional unreleased mainline entries missing from these release notes; see the ancestry report above.\n`,
);
}
}
await main().catch((error) => {
process.stderr.write(
`Release checkpoint validation failed: ${error instanceof Error ? error.message : String(error)}\n`,
);
process.exitCode = 1;
});
+10 -8
View File
@@ -260,14 +260,16 @@ try {
);
const original = await readFile(path);
for (const mode of ["cli", "mcp"]) {
const error = await inspect(
mode,
{ capture_path: path, format },
"unsupported_provider",
);
assert.ok(
JSON.stringify(error).includes("JSON schema boundary cannot preserve"),
);
const value = await inspect(mode, { capture_path: path, format });
const reported =
format === "har"
? value.container.reported._extension
: value.records[0].reported.metadata;
assert.equal(Object.hasOwn(reported, "__proto__"), true);
assert.deepEqual(reported["__proto__"], { preserved: 7 });
assert.equal(Object.getPrototypeOf(reported), Object.prototype);
assert.equal(Reflect.get(Object.prototype, "preserved"), undefined);
if (format === "har") assert.equal(reported.constructor, "ordinary");
assert.deepEqual(await readFile(path), original);
cases++;
}
+4 -2
View File
@@ -79,7 +79,7 @@ export async function verifyCoreInspectionTrace({
(line) =>
/^execve\("\/(?:usr\/)?bin\/ps",/u.test(line) &&
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]',
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
) &&
line.endsWith(" = 0"),
);
@@ -128,7 +128,9 @@ export async function verifyCoreInspectionTrace({
const executable = await realpath(match[1]).catch(() => match[1]);
const ownership =
psPaths.has(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
);
const iconv =
iconvPaths.has(match[1]) && line.includes('["iconv", "-l"]');
const version =
@@ -3,8 +3,6 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "33"
tool_count: 139
catalog_digest: "4c5f67d076baf821b5cb6d6fb2cc0f6c274e96b627b85b892577760a6f71ab1a"
---
# REA
@@ -146,6 +144,13 @@ Start with the default result and use its inline Evidence and graph context.
Do not repeat an identical tool call. Make a focused follow-up only when the
returned result leaves a specific question unanswered.
If MCP reports `resource_constraint` with `details.resource: "transport"`, use
its reported same-session Evidence reference with a focused application tool,
or call `export_evidence_bundle` to write the complete session to a selected
path. The analysis remains complete in the ledger; avoid repeating it merely
to request the same oversized response. Complete CLI JSON output also streams
without a single MCP frame.
Every conclusion must distinguish observations, inferences, and unknowns. Cite
Evidence IDs, preserve limitations and incomplete coverage, and never imply
that static analysis observed execution. Runtime requests execute the declared
@@ -2,10 +2,17 @@
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. The complete result, graph, and Evidence context are returned
inline. Source-map contents are part of the static analysis. Start from its
inline when they fit the MCP frame. Source-map contents are part of the static analysis. Start from its
findings, coverage, unknowns, and graph context, then make a focused follow-up
only when a specific question remains unanswered.
An oversized response reports `resource_constraint` and
`details.resource: "transport"`. Reuse
`details.reported_limits.evidence_reference` with `trace_application_feature`
for the module or feature under investigation. `export_evidence_bundle` writes
the complete canonical session to a caller-selected file. Both workflows keep
the original Evidence and coverage; a broad follow-up can also exceed framing.
BrowserWindow preferences, preload and contextBridge surfaces, IPC
registrations, utility processes, and native binding requests are static syntax
observations. Only a unique exact literal IPC channel match supports an inferred
@@ -0,0 +1,52 @@
import { expect, it } from "vitest";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
isOwnedClientRegistrationCommand,
npxRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
import { setupRegistrationCommand } from "./SetupHost.js";
it("uses a Windows executable for npm-run setup and preserves POSIX launchers", () => {
expect(setupRegistrationCommand("win32", true)).toEqual([
"cmd.exe",
"/d",
"/c",
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
]);
for (const platform of ["linux", "darwin"] as const)
expect(setupRegistrationCommand(platform, true)).toEqual([
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
]);
});
it.each(
[
npxRegistrationCommand("win32"),
["cmd", "/c", "npx", "-y", "rea-agents@5.0.0", "mcp"],
["npx", "-y", "rea-agents@5.0.0", "mcp"],
].map((command) => ({ command })),
)("recognizes owned current and legacy launchers: $command", ({ command }) => {
expect(isOwnedClientRegistrationCommand(command)).toBe(true);
});
it.each(
[
["cmd.exe", "/d", "/c", "node", "custom.js", "mcp"],
["cmd.exe", "/k", "npx", "-y", "rea-agents@5.0.0", "mcp"],
["cmd.exe", "/c", "npx -y rea-agents@5.0.0 mcp"],
["cmd.exe", "/c", "npx", "-y", "another-package", "mcp"],
["cmd.exe", "/c", "npx", "-y", "rea-agents@5.0.0", "mcp", "&", "other"],
["cmd.exe", "/c", "npx", "-y", "rea-agents@5.0.0 & other", "mcp"],
].map((command) => ({ command })),
)(
"does not claim custom shell commands as REA-owned: $command",
({ command }) => {
expect(isOwnedClientRegistrationCommand(command)).toBe(false);
},
);
+28 -5
View File
@@ -3,6 +3,28 @@ import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
/** Resolve a pinned npx launcher that stdio clients can spawn on the host. */
export const npxRegistrationCommand = (
platform: NodeJS.Platform = process.platform,
): readonly string[] => {
const command = [
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
];
return platform === "win32" ? ["cmd.exe", "/d", "/c", ...command] : command;
};
const unwrapNpxCommand = (command: readonly string[]): readonly string[] => {
const executable = command[0]?.toLowerCase();
if (executable !== "cmd" && executable !== "cmd.exe") return command;
if (command[1]?.toLowerCase() === "/c") return command.slice(2);
if (command[1]?.toLowerCase() === "/d" && command[2]?.toLowerCase() === "/c")
return command.slice(3);
return [];
};
/** Check that one parsed command points to REA's MCP entry point. */
export const isOwnedClientRegistrationCommand = (
command: readonly string[],
@@ -27,15 +49,16 @@ export const isOwnedClientRegistrationCommand = (
return true;
}
const npxCommand = unwrapNpxCommand(command);
if (
command.length !== 4 ||
command[0] !== "npx" ||
command[1] !== "-y" ||
command[3] !== "mcp"
npxCommand.length !== 4 ||
npxCommand[0] !== "npx" ||
npxCommand[1] !== "-y" ||
npxCommand[3] !== "mcp"
)
return false;
const packageReference = command[2] ?? "";
const packageReference = npxCommand[2] ?? "";
if (
packageReference === PRODUCT_IDENTITY.packageName ||
packageReference === PRODUCT_IDENTITY.packageSpecifier ||
+14 -7
View File
@@ -9,7 +9,10 @@ import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import {
isOwnedClientRegistrationCommand,
npxRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
import { supportedClients } from "./SupportedClients.js";
import type { SetupClient } from "./SupportedClients.js";
@@ -108,7 +111,12 @@ export const readClientRegistrationStatuses = async (
client.name,
client.configPath,
command,
registrationAligned(registration, client, currentCommandPath)
registrationAligned(
registration,
client,
currentCommandPath,
options.platform ?? process.platform,
)
? "aligned"
: "stale",
),
@@ -132,6 +140,7 @@ const registrationAligned = (
registration: z.output<typeof registrationSchema>,
client: SetupClient,
currentCommandPath: string,
platform: NodeJS.Platform,
): boolean => {
const command = [registration.command, ...registration.args];
if (registration.disabled === true || registration.enabled === false)
@@ -160,12 +169,10 @@ const registrationAligned = (
resolve(command[1] ?? "") === currentCommandPath
)
return true;
const npxCommand = npxRegistrationCommand(platform);
if (
command.length === 4 &&
command[0] === "npx" &&
command[1] === "-y" &&
command[2] === PRODUCT_IDENTITY.registrationPackageSpecifier &&
command[3] === "mcp"
command.length === npxCommand.length &&
command.every((argument, index) => argument === npxCommand[index])
)
return true;
return (
+14 -40
View File
@@ -30,11 +30,7 @@ import {
snapshotEvidenceForQuery,
snapshotMatchesTarget,
} from "../domain/analysisSnapshot.js";
import {
analysisProfileSchema,
committedProviderSchema,
type AnalysisProfileCommitment,
} from "../domain/analysisProfile.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import { err, ok, type Result } from "../domain/result.js";
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import type {
@@ -45,6 +41,10 @@ import {
REA_WORKFLOW_PROVIDER,
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import {
createWorkflowEvidence,
workflowSnapshotRecord,
} from "./WorkflowEvidence.js";
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
@@ -268,35 +268,13 @@ const runAnalysis = async (
tool !== "trace_native_ui_action" &&
snapshotPath !== undefined &&
evidence !== undefined &&
"analysis_profile" in evidence &&
session.allowsSnapshotReplay(tool)
) {
const recorded = session.recordWorkflowSnapshot({
operation: tool,
parameters: arguments_,
execution: {
result: evidence.normalized_result,
rawResult: evidence.raw_result,
provider: committedProviderSchema.parse(evidence.provider),
analysisProfile: analysisProfileSchema.parse(
evidence.analysis_profile,
),
limitations: evidence.limitations,
locations: evidence.locations,
subject:
evidence.subject === null
? null
: {
path: evidence.subject.local_path,
sha256: evidence.subject.digest.sha256,
format: evidence.subject.format,
...(evidence.subject.architecture === null
? {}
: { architecture: evidence.subject.architecture }),
},
},
});
if (!recorded.ok) return cliError(recorded.error);
const workflowRecord = workflowSnapshotRecord(evidence, tool);
if (workflowRecord !== undefined) {
const recorded = session.recordWorkflowSnapshot(workflowRecord);
if (!recorded.ok) return cliError(recorded.error);
}
}
if (
tool !== "trace_native_ui_action" &&
@@ -356,15 +334,12 @@ const executeAnalysisTool = async (input: {
signal,
);
if (!result.ok) return { output: cliError(result.error) };
const evidence = createEvidence(input.openedTarget, REA_WORKFLOW_PROVIDER, {
const evidence = createWorkflowEvidence({
target: input.openedTarget,
operation: tool,
parameters: input.arguments,
result: result.value,
...(evidenceProfile === undefined
? {}
: { analysisProfile: evidenceProfile }),
confidence: "derived",
limitations: ["Derived by an REA composed workflow."],
upstreamProfile: evidenceProfile,
});
return { output: evidence, evidence };
}
@@ -493,8 +468,7 @@ const analysisProfileForEvidence = (
| DirectAnalysisTool,
): AnalysisProfileCommitment | undefined => {
if (!isWorkflowEvidenceTool(tool)) return session.analysisProfile(tool);
const upstream = session.analysisProfile();
return upstream === undefined ? undefined : workflowAnalysisProfile(upstream);
return session.analysisProfile();
};
const isWorkflowEvidenceTool = (
+2 -1
View File
@@ -26,7 +26,8 @@ export const createDoctorHostFixture = (
Promise.resolve({
version: PRODUCT_IDENTITY.skillVersion,
toolCount: CATALOG_IDENTITY.counts.mcp_tools,
catalogDigest: CATALOG_IDENTITY.digests.combined_sha256,
catalogDigest: null,
canonical: true,
}),
...overrides,
});
+1
View File
@@ -177,6 +177,7 @@ describe("doctor installation identity", () => {
version: "10",
toolCount: null,
catalogDigest: null,
canonical: false,
}),
}),
);
+7 -2
View File
@@ -14,6 +14,7 @@ import {
readLinuxDistribution,
type LinuxDistribution,
} from "./LinuxHopper.js";
import { canonicalSkillNeedsInstall } from "./SetupSkill.js";
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "../identity.js";
import {
@@ -95,11 +96,13 @@ export interface DoctorHost {
ilspyCmdVersion?(path: string): Promise<string | undefined>;
}
/** Parsed identity committed by an installed REA skill. */
/** Observed skill metadata and comparison with the packaged instruction bundle. */
interface InstalledSkillIdentity {
readonly version: string | null;
readonly toolCount: number | null;
readonly catalogDigest: string | null;
/** Whether all managed instruction and reference files match this package. */
readonly canonical: boolean;
}
/** Structured result returned by the read-only doctor workflow. */
@@ -139,6 +142,7 @@ interface DoctorIdentity {
readonly skill: {
readonly installed_version: string | null;
readonly installed_tool_count: number | null;
/** Legacy catalog digest observed in older bundles; current bundles omit it. */
readonly installed_catalog_digest: string | null;
readonly state: "aligned" | "stale" | "missing";
readonly remediation: string | null;
@@ -253,7 +257,7 @@ const skillIdentityAligned = (
): boolean =>
identity?.version === PRODUCT_IDENTITY.skillVersion &&
identity.toolCount === CATALOG_IDENTITY.counts.mcp_tools &&
identity.catalogDigest === CATALOG_IDENTITY.digests.combined_sha256;
identity.canonical;
const skillIdentityCheck = (
identity: InstalledSkillIdentity | undefined,
@@ -507,6 +511,7 @@ const installedSkillIdentity = async (
const content = await readInstalledSkill(home);
const countText = /^\s{2}tool_count:\s*(\d+)\s*$/mu.exec(content)?.[1];
return {
canonical: !(await canonicalSkillNeedsInstall(home)),
version: /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(content)?.[1] ?? null,
toolCount:
countText === undefined ? null : Number.parseInt(countText, 10),
+31 -11
View File
@@ -1,6 +1,7 @@
import {
describeEvidenceBundleFailure,
describeValidationFailure,
parseEvidenceBundle,
serializeEvidenceBundle,
type EvidenceBundle,
} from "../domain/evidenceBundle.js";
import {
@@ -9,7 +10,11 @@ import {
} from "../domain/evidenceErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { parseProcessCapture } from "../domain/process/processCapture.js";
import { readJsonFile, writeTextFile } from "./JsonFiles.js";
import {
bufferedJsonParts,
canonicalJsonParts,
} from "../domain/jsonSerialization.js";
import { readJsonFile, writeTextParts } from "./JsonFiles.js";
type EvidenceReadFailure = EvidenceFileError | EvidenceIntegrityError;
type EvidenceWriteFailure = EvidenceFileError | EvidenceIntegrityError;
@@ -20,20 +25,31 @@ export const readEvidenceBundle = async (
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> => {
const loaded = await readJsonFile(path);
if (!loaded.ok) return loaded;
let bundle: EvidenceBundle;
try {
const bundle = parseEvidenceBundle(loaded.value);
for (const record of bundle.records) {
if (record.predicate_type === "rea.process-capture")
parseProcessCapture(record.normalized_result);
}
return ok(bundle);
bundle = parseEvidenceBundle(loaded.value);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: describeEvidenceBundleFailure(loaded.value, cause),
}),
);
}
for (const record of bundle.records) {
if (record.predicate_type !== "rea.process-capture") continue;
try {
parseProcessCapture(record.normalized_result);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: `Process capture record ${record.evidence_id} has an invalid normalized_result (${describeValidationFailure(cause)}). Recreate or re-export the bundle, then try again.`,
}),
);
}
}
return ok(bundle);
};
/** Atomically write deterministic evidence JSON at the caller-supplied path. */
@@ -47,9 +63,9 @@ export const writeEvidenceBundle = async (
EvidenceWriteFailure
>
> => {
let encoded: string;
let checked: EvidenceBundle;
try {
encoded = serializeEvidenceBundle(bundle);
checked = parseEvidenceBundle(bundle);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
@@ -57,5 +73,9 @@ export const writeEvidenceBundle = async (
}),
);
}
return writeTextFile(encoded, path, overwrite);
return writeTextParts(
bufferedJsonParts(canonicalJsonParts(checked)),
path,
overwrite,
);
};
+82 -29
View File
@@ -1,8 +1,15 @@
import { link, lstat, mkdtemp, readFile, realpath, rm } from "node:fs/promises";
import {
link,
lstat,
mkdtemp,
open,
readFile,
realpath,
rename,
rm,
} from "node:fs/promises";
import { basename, dirname, resolve } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { EvidenceFileError } from "../domain/evidenceErrors.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -10,10 +17,14 @@ import { err, ok, type Result } from "../domain/result.js";
export const readJsonFile = async (
path: string,
): Promise<Result<unknown, EvidenceFileError>> => {
const requestedPath = resolve(path);
try {
const canonicalPath = await realpath(resolve(path));
const canonicalPath = await realpath(requestedPath);
const stats = await lstat(canonicalPath);
if (!stats.isFile()) return err(new EvidenceFileError("read", "not-file"));
if (!stats.isFile())
return err(
new EvidenceFileError("read", "not-file", { path: requestedPath }),
);
const encoded = await readFile(canonicalPath);
let decoded: unknown;
try {
@@ -23,11 +34,21 @@ export const readJsonFile = async (
),
);
} catch (cause: unknown) {
return err(new EvidenceFileError("read", "invalid-json", { cause }));
return err(
new EvidenceFileError("read", "invalid-json", {
cause,
path: requestedPath,
}),
);
}
return ok(decoded);
} catch (cause: unknown) {
return err(new EvidenceFileError("read", "io", { cause }));
return err(
new EvidenceFileError("read", missingOrIo(cause), {
cause,
path: requestedPath,
}),
);
}
};
@@ -38,10 +59,18 @@ export const writeTextFile = async (
overwrite: boolean,
): Promise<
Result<{ readonly path: string; readonly bytes: number }, EvidenceFileError>
> => writeTextParts([encoded], path, overwrite);
/** Publish complete streamed text atomically, keeping only one encoded part in memory. */
export const writeTextParts = async (
parts: Iterable<string>,
path: string,
overwrite: boolean,
): Promise<
Result<{ readonly path: string; readonly bytes: number }, EvidenceFileError>
> => {
const bytes = Buffer.byteLength(encoded, "utf8");
const requestedPath = resolve(path);
try {
const requestedPath = resolve(path);
const canonicalParent = await realpath(dirname(requestedPath));
const destination = resolve(canonicalParent, basename(requestedPath));
const existing = await lstat(destination).catch((cause: unknown) => {
@@ -49,41 +78,65 @@ export const writeTextFile = async (
throw cause;
});
if (existing !== undefined) {
if (!overwrite) return err(new EvidenceFileError("write", "exists"));
if (!overwrite)
return err(
new EvidenceFileError("write", "exists", { path: requestedPath }),
);
if (!existing.isFile() || existing.isSymbolicLink())
return err(new EvidenceFileError("write", "not-file"));
return err(
new EvidenceFileError("write", "not-file", { path: requestedPath }),
);
}
if (overwrite)
await writeFileAtomic(destination, encoded, {
encoding: "utf8",
mode: 0o600,
fsync: true,
});
else await publishNewFile(destination, encoded);
const bytes = await publishFile(destination, parts, overwrite);
return ok({ path: requestedPath, bytes });
} catch (cause: unknown) {
if (!overwrite && fileErrorCode(cause) === "EEXIST")
return err(new EvidenceFileError("write", "exists", { cause }));
return err(new EvidenceFileError("write", "io", { cause }));
return err(
new EvidenceFileError("write", "exists", {
cause,
path: requestedPath,
}),
);
return err(
new EvidenceFileError("write", missingOrIo(cause), {
cause,
path: requestedPath,
}),
);
}
};
const publishNewFile = async (
/** A missing path or parent is a selection error, not a permission failure. */
const missingOrIo = (cause: unknown): "missing" | "io" => {
const code = fileErrorCode(cause);
return code === "ENOENT" || code === "ENOTDIR" ? "missing" : "io";
};
const publishFile = async (
destination: string,
encoded: string,
): Promise<void> => {
parts: Iterable<string>,
overwrite: boolean,
): Promise<number> => {
const stagingDirectory = await mkdtemp(
resolve(dirname(destination), ".rea-write-"),
);
try {
const staged = resolve(stagingDirectory, "content");
await writeFileAtomic(staged, encoded, {
encoding: "utf8",
mode: 0o600,
fsync: true,
});
const file = await open(staged, "wx", 0o600);
let bytes = 0;
try {
for (const part of parts) {
await file.writeFile(part, { encoding: "utf8" });
bytes += Buffer.byteLength(part, "utf8");
}
await file.sync();
} finally {
await file.close();
}
if (overwrite) await rename(staged, destination);
// A hard link publishes complete bytes atomically and cannot replace an existing name.
await link(staged, destination);
else await link(staged, destination);
return bytes;
} finally {
await rm(stagingDirectory, { recursive: true, force: true });
}
+2 -6
View File
@@ -15,6 +15,7 @@ import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import type {
@@ -24,12 +25,7 @@ import type {
} from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js";
const defaultCommand = (): readonly string[] => [
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
];
const defaultCommand = (): readonly string[] => npxRegistrationCommand();
/** Back up, atomically update, and semantically read back one JSON MCP configuration. */
export const configureJsonClient = (
+2 -1
View File
@@ -3,6 +3,7 @@ import { homedir } from "node:os";
import { resolve } from "node:path";
import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import {
SUPPORTED_NODE_VERSION_PROSE,
supportsNodeVersion,
@@ -35,7 +36,7 @@ export const setupRegistrationCommand = (
useNpmRunner: boolean = process.env.npm_command === "exec",
): readonly string[] =>
useNpmRunner
? PRODUCT_IDENTITY.mcpCommand.split(" ")
? npxRegistrationCommand(platform)
: platform === "win32"
? [
process.execPath,
+64
View File
@@ -0,0 +1,64 @@
import {
REA_WORKFLOW_PROVIDER,
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import type { AnalysisOperation } from "./AnalysisProvider.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import type { WorkflowSnapshotRecordInput } from "./binary/BinarySessionRecords.js";
import {
analysisProfileSchema,
committedProviderSchema,
} from "../domain/analysisProfile.js";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import type { JsonValue } from "../domain/jsonValue.js";
/** Build the shared Evidence record for a successful composed binary workflow. */
export const createWorkflowEvidence = (input: {
readonly target: BinaryTarget | undefined;
readonly operation: AnalysisOperation;
readonly parameters: Readonly<Record<string, JsonValue>>;
readonly result: JsonValue;
readonly upstreamProfile: AnalysisProfileCommitment | undefined;
}): Evidence =>
createEvidence(input.target, REA_WORKFLOW_PROVIDER, {
operation: input.operation,
parameters: input.parameters,
result: input.result,
...(input.upstreamProfile === undefined
? {}
: { analysisProfile: workflowAnalysisProfile(input.upstreamProfile) }),
confidence: "derived",
limitations: ["Derived by an REA composed workflow."],
});
/** Convert workflow Evidence into its exact replay binding, when it has a profile. */
export const workflowSnapshotRecord = (
evidence: Evidence,
operation: AnalysisOperation,
): WorkflowSnapshotRecordInput | undefined => {
if (!("analysis_profile" in evidence)) return undefined;
return {
operation,
parameters: evidence.parameters,
execution: {
result: evidence.normalized_result,
rawResult: evidence.raw_result,
provider: committedProviderSchema.parse(evidence.provider),
analysisProfile: analysisProfileSchema.parse(evidence.analysis_profile),
limitations: evidence.limitations,
locations: evidence.locations,
subject:
evidence.subject === null
? null
: {
path: evidence.subject.local_path,
sha256: evidence.subject.digest.sha256,
format: evidence.subject.format,
...(evidence.subject.architecture === null
? {}
: { architecture: evidence.subject.architecture }),
},
},
};
};
@@ -7,6 +7,8 @@ import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js"
import type { AnalysisProviderSelector } from "../../contracts/providerSelection.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { WorkflowSnapshotRecordInput } from "./BinarySessionRecords.js";
import type {
AnalysisOperation,
@@ -43,6 +45,10 @@ export interface BinarySessionPort
analysisProfile(
operation?: AnalysisOperation,
): AnalysisProfileCommitment | undefined;
allowsSnapshotReplay(operation: AnalysisOperation): boolean;
recordWorkflowSnapshot(
input: WorkflowSnapshotRecordInput,
): Result<null, EvidenceIntegrityError>;
openCompatibility(): Readonly<Record<string, JsonValue>>;
onAvailabilityChanged?(listener: () => void | Promise<void>): () => void;
onAnalysisSnapshotChanged?(listener: () => void | Promise<void>): () => void;
+23 -9
View File
@@ -3,6 +3,7 @@ import type { AnalysisSnapshot } from "../../domain/analysisSnapshot.js";
import type { BinaryTarget } from "../../domain/binaryTarget.js";
import type { Evidence } from "../../domain/evidence.js";
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { evidenceBundleForTarget } from "../../domain/evidenceBundle.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { type AnalysisError } from "../../domain/analysisErrorBase.js";
@@ -26,6 +27,15 @@ export interface ActiveAnalysisBinding {
readonly profile: AnalysisProfileCommitment | null;
}
/** Validated composed-workflow payload bound to the active target and profile. */
export interface WorkflowSnapshotRecordInput {
readonly operation: AnalysisOperation;
readonly parameters: Readonly<Record<string, JsonValue>>;
readonly execution: Parameters<
AnalysisSnapshotCache["recordWorkflow"]
>[0]["execution"];
}
/** Binary snapshot owner and compatibility facade for composed investigation records. */
export abstract class BinarySessionRecords {
readonly #records: InvestigationRecords;
@@ -59,10 +69,20 @@ export abstract class BinarySessionRecords {
return this.#records.evidenceById(evidenceId);
}
/** Borrow immutable investigation Evidence without copying a complete graph. */
evidenceForAnalysis(evidenceId: string): Evidence | undefined {
return this.#records.evidenceForAnalysis(evidenceId);
}
exportEvidenceBundle(): EvidenceBundle {
return this.#records.exportEvidenceBundle();
}
/** Borrow a sealed bundle for complete serialization without cloning retained graphs. */
evidenceBundleForSerialization(): EvidenceBundle {
return this.#records.evidenceBundleForSerialization();
}
importEvidenceBundle(
bundle: unknown,
): Result<number, EvidenceIntegrityError> {
@@ -161,15 +181,9 @@ export abstract class BinarySessionRecords {
}
/** Retain one derived workflow result alongside its provider cache entries. */
recordWorkflowSnapshot(input: {
readonly operation: string;
readonly parameters: Readonly<
Record<string, import("../../domain/jsonValue.js").JsonValue>
>;
readonly execution: Parameters<
AnalysisSnapshotCache["recordWorkflow"]
>[0]["execution"];
}): Result<null, EvidenceIntegrityError> {
recordWorkflowSnapshot(
input: WorkflowSnapshotRecordInput,
): Result<null, EvidenceIntegrityError> {
const active = this.activeAnalysisBinding();
if (active === undefined || active.profile === null)
return err(
@@ -116,6 +116,36 @@ describe("evidence ledger recording", () => {
ledger.clear();
expect(ledger.export().records).toEqual([]);
});
it("protects borrowed snapshots while preserving detached mutable public reads", () => {
const evidence = createEvidence(TARGET, PROVIDER, {
operation: "health",
parameters: {},
result: { nested: { value: "original" } },
});
const ledger = new EvidenceLedger();
expect(ledger.record(evidence).ok).toBe(true);
const borrowed = ledger.forAnalysis(evidence.evidence_id);
if (borrowed === undefined) throw new Error("Missing recorded Evidence");
expect(Object.isFrozen(borrowed)).toBe(true);
expect(Object.isFrozen(borrowed.normalized_result)).toBe(true);
expect(Reflect.set(borrowed, "operation", "forged")).toBe(false);
expect(ledger.forAnalysis(evidence.evidence_id)).toBe(borrowed);
expect(ledger.record(borrowed)).toEqual({ ok: true, value: "duplicate" });
const detached = ledger.get(evidence.evidence_id);
if (detached === undefined) throw new Error("Missing detached Evidence");
expect(Reflect.set(detached, "operation", "forged")).toBe(true);
expect(ledger.record(detached).ok).toBe(false);
expect(ledger.forAnalysis(evidence.evidence_id)).toBe(borrowed);
const snapshot = ledger.forSerialization();
expect(snapshot).toEqual(ledger.export());
expect(snapshot.records[0]).toBe(borrowed);
expect(Object.isFrozen(snapshot.records)).toBe(true);
ledger.clear();
expect(ledger.forAnalysis(evidence.evidence_id)).toBeUndefined();
expect(ledger.forSerialization().records).toEqual([]);
expect(snapshot.records).toEqual([evidence]);
});
});
describe("evidence ledger metadata imports", () => {
@@ -2,13 +2,19 @@ import canonicalize from "canonicalize";
import {
createEvidenceBundle,
createImmutableEvidenceBundle,
describeEvidenceBundleFailure,
parseEvidenceBundle,
validateResidualUnknownAddition,
type EvidenceBundle,
} from "../../domain/evidenceBundle.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import { parseEvidence, type Evidence } from "../../domain/evidence.js";
import {
immutableEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import {
createResidualUnknown,
updateResidualUnknown,
@@ -41,7 +47,7 @@ export class EvidenceLedger {
record(input: Evidence): RecordResult {
let evidence: Evidence;
try {
evidence = parseEvidence(input);
evidence = immutableEvidence(input);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence record validation failed", {
@@ -51,6 +57,10 @@ export class EvidenceLedger {
}
const existing = this.#records.get(evidence.evidence_id);
if (existing !== undefined) {
// A borrowed record is already authenticated and deeply immutable. Its
// identity cannot change between reads, so comparing document-sized
// canonical strings again would add no integrity check.
if (existing === evidence) return ok("duplicate");
return recordsAgree(existing, evidence)
? ok("duplicate")
: err(new EvidenceIntegrityError("Conflicting evidence record"));
@@ -68,6 +78,7 @@ export class EvidenceLedger {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: describeEvidenceBundleFailure(input, cause),
}),
);
}
@@ -95,8 +106,9 @@ export class EvidenceLedger {
additions.set(evidence.evidence_id, evidence);
}
for (const [id, evidence] of replacements)
this.#records.set(id, evidence);
for (const [id, evidence] of additions) this.#records.set(id, evidence);
this.#records.set(id, immutableEvidence(evidence));
for (const [id, evidence] of additions)
this.#records.set(id, immutableEvidence(evidence));
return ok({
recordsAdded: additions.size,
unknownsAdded: 0,
@@ -163,6 +175,14 @@ export class EvidenceLedger {
);
}
/** Borrow a sealed canonical bundle for serialization without copying its records. */
forSerialization(): EvidenceBundle {
return createImmutableEvidenceBundle(
[...this.#records.values()],
[...this.#unknownRevisions.values()],
);
}
/** Check whether one immutable Evidence ID is present in this ledger. */
has(evidenceId: string): boolean {
return this.#records.has(evidenceId);
@@ -174,6 +194,11 @@ export class EvidenceLedger {
return evidence === undefined ? undefined : structuredClone(evidence);
}
/** Borrow authenticated immutable Evidence for read-only analysis without a full clone. */
forAnalysis(evidenceId: string): Evidence | undefined {
return this.#records.get(evidenceId);
}
/** Create one approved unknown and atomically append its mutation evidence. */
recordUnknown(
input: RecordUnknownInput,
@@ -378,7 +403,8 @@ export class EvidenceLedger {
} catch (cause: unknown) {
return err(new UnknownRegistryError("integrity", { cause }));
}
for (const [id, evidence] of additions) this.#records.set(id, evidence);
for (const [id, evidence] of additions)
this.#records.set(id, immutableEvidence(evidence));
if (unknown !== undefined && unknownKey !== undefined) {
this.#unknownRevisions.set(unknownKey, unknown);
this.#unknownHeads.set(unknown.unknown_id, unknown);
@@ -405,7 +431,8 @@ export class EvidenceLedger {
unknowns: ReadonlyMap<string, ResidualUnknown>,
): void {
this.#records.clear();
for (const [id, evidence] of records) this.#records.set(id, evidence);
for (const [id, evidence] of records)
this.#records.set(id, immutableEvidence(evidence));
this.#unknownRevisions.clear();
this.#unknownHeads.clear();
for (const [key, unknown] of unknowns) {
@@ -15,6 +15,10 @@ import type { Result } from "../../domain/result.js";
export interface EvidenceReader {
hasEvidence(evidenceId: string): boolean;
evidenceById(evidenceId: string): Evidence | undefined;
/** Optional immutable analysis read; older adapters can keep detached reads. */
evidenceForAnalysis?(evidenceId: string): Evidence | undefined;
/** Optional immutable bundle snapshot for complete serialization without graph copies. */
evidenceBundleForSerialization?(): EvidenceBundle;
exportEvidenceBundle(): EvidenceBundle;
}
@@ -51,11 +51,21 @@ export class InvestigationRecords
return this.#evidence.get(evidenceId);
}
/** Borrow an immutable authenticated record for a read-only application workflow. */
evidenceForAnalysis(evidenceId: string): Evidence | undefined {
return this.#evidence.forAnalysis(evidenceId);
}
/** Export a detached deterministic bundle of Evidence and Unknown revisions. */
exportEvidenceBundle(): EvidenceBundle {
return this.#evidence.export();
}
/** Borrow a sealed bundle snapshot for complete JSON serialization. */
evidenceBundleForSerialization(): EvidenceBundle {
return this.#evidence.forSerialization();
}
/** Atomically merge a bundle and report changes for the owner's post-commit observers. */
mergeEvidenceBundle(
bundle: unknown,
@@ -8,6 +8,18 @@ import type {
JavaScriptApplicationAnalysisResult,
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
import { JAVASCRIPT_APPLICATION_PROVIDER } from "../InvestigationProviders.js";
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
import { rememberOwnedApplicationGraphEvidence } from "./JavaScriptApplicationEvidenceGraph.js";
/** Transfer an already validated, application-owned result into immutable analysis Evidence. */
export const createOwnedJavaScriptApplicationEvidence = (
input: AnalyzeJavaScriptApplicationInput,
result: JavaScriptApplicationAnalysisResult,
): Evidence =>
rememberOwnedApplicationGraphEvidence(
createJavaScriptApplicationEvidence(input, freezeJsonSnapshot(result)),
result,
);
/** Create Evidence for one deterministic local JavaScript application graph. */
export const createJavaScriptApplicationEvidence = (
@@ -39,4 +51,5 @@ const parameters = (
input: AnalyzeJavaScriptApplicationInput,
): EvidenceObservation["parameters"] => ({
format: input.format,
integrity_policy: input.integrity_policy,
});
@@ -0,0 +1,47 @@
import { describe, expect, it } from "vitest";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { parseEvidence } from "../../domain/evidence.js";
import {
analyzeJavaScriptApplicationInputSchema,
javascriptApplicationAnalysisResultSchema,
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
import {
createJavaScriptApplicationEvidence,
createOwnedJavaScriptApplicationEvidence,
} from "./JavaScriptApplicationEvidence.js";
import {
parseApplicationGraphEvidence,
rememberOwnedApplicationGraphEvidence,
} from "./JavaScriptApplicationEvidenceGraph.js";
describe("owned application graph Evidence", () => {
it("reuses the exact authenticated graph and preserves ordinary wire identity", () => {
const result = javascriptApplicationAnalysisResultSchema.parse(
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
);
const input = analyzeJavaScriptApplicationInputSchema.parse({
input_path: result.input_path,
...JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.parameters,
});
const ordinary = createJavaScriptApplicationEvidence(input, result);
const owned = createOwnedJavaScriptApplicationEvidence(input, result);
expect(owned.evidence_id).toBe(ordinary.evidence_id);
expect(JSON.stringify(owned)).toBe(JSON.stringify(ordinary));
const parsed = parseApplicationGraphEvidence(owned);
expect(parsed.evidence).toBe(owned);
expect(parsed.graph).toBe(result.graph);
expect(parsed.semanticGraph).toBe(result.semantic_graph);
expect(Object.isFrozen(parsed.graph.nodes)).toBe(true);
expect(parseApplicationGraphEvidence(owned)).toBe(parsed);
const wire: unknown = JSON.parse(JSON.stringify(owned));
expect(parseApplicationGraphEvidence(wire)).toEqual(parsed);
expect(parseApplicationGraphEvidence(wire).graph).not.toBe(parsed.graph);
expect(() =>
rememberOwnedApplicationGraphEvidence(ordinary, result),
).toThrow("exact validated immutable result");
const tampered = { ...owned, normalized_result: {} };
expect(() => parseEvidence(Object.freeze(tampered))).toThrow();
expect(() => parseApplicationGraphEvidence(tampered)).toThrow();
});
});
@@ -3,7 +3,11 @@ import {
JAVASCRIPT_RUNTIME_RECONCILIATION_PROVIDER,
MANAGED_WORKFLOW_PROVIDER,
} from "../InvestigationProviders.js";
import { parseEvidence, type Evidence } from "../../domain/evidence.js";
import {
isImmutableEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import {
analyzeJavaScriptApplicationInputSchema,
javascriptApplicationAnalysisResultSchema,
@@ -26,11 +30,39 @@ export interface ApplicationGraphEvidenceSource {
readonly semanticGraph: JavaScriptSemanticGraph | null;
}
const ownedApplicationSources = new WeakMap<
Evidence,
ApplicationGraphEvidenceSource
>();
/** Retain the producer-validated meaning of its exact immutable result snapshot. */
export const rememberOwnedApplicationGraphEvidence = (
evidence: Evidence,
result: JavaScriptApplicationAnalysisResult,
): Evidence => {
if (
!isImmutableEvidence(evidence) ||
evidence.normalized_result !== result ||
evidence.operation !== "analyze_javascript_application" ||
!providerMatches(evidence, JAVASCRIPT_APPLICATION_PROVIDER)
)
throw new TypeError(
"Owned application Evidence must bind its exact validated immutable result",
);
ownedApplicationSources.set(
evidence,
Object.freeze(staticApplicationSourceForResult(evidence, result)),
);
return evidence;
};
/** Parse and authenticate one REA-produced JavaScript Application Graph Evidence. */
export const parseApplicationGraphEvidence = (
input: unknown,
): ApplicationGraphEvidenceSource => {
const evidence = parseEvidence(input);
const owned = ownedApplicationSources.get(evidence);
if (owned !== undefined) return owned;
if (
evidence.operation === "analyze_javascript_application" &&
evidence.predicate_type === "rea.javascript-application-analysis" &&
@@ -56,6 +88,16 @@ export const parseApplicationGraphEvidence = (
const staticApplicationSource = (
evidence: Evidence,
): ApplicationGraphEvidenceSource => {
const result = javascriptApplicationAnalysisResultSchema.parse(
evidence.normalized_result,
);
return staticApplicationSourceForResult(evidence, result);
};
const staticApplicationSourceForResult = (
evidence: Evidence,
result: JavaScriptApplicationAnalysisResult,
): ApplicationGraphEvidenceSource => {
if (
evidence.authority !== "shipped-artifact" ||
@@ -64,9 +106,6 @@ const staticApplicationSource = (
throw new TypeError(
"JavaScript application Evidence authority or confidence is invalid",
);
const result = javascriptApplicationAnalysisResultSchema.parse(
evidence.normalized_result,
);
if (evidence.predicate_type !== "rea.javascript-application-analysis")
throw new TypeError(
"JavaScript application Evidence predicate does not match its result shape",
@@ -1,12 +1,18 @@
import { writeFile } from "node:fs/promises";
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { createPackageWithOptions } from "@electron/asar";
import { describe, expect, it } from "vitest";
import { z } from "zod";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
import {
compareApplicationVersionsEvidence,
compareJavaScriptExportShapesEvidence,
traceApplicationFeatureEvidence,
} from "./JavaScriptApplicationWorkflowService.js";
describe("JavaScript application failure diagnostics", () => {
it("identifies the rejected result field in caller-visible diagnostics", async () => {
@@ -155,3 +161,107 @@ describe("JavaScript application failure diagnostics", () => {
},
);
});
const CONTRADICTION_LIMITATION =
"Observed bytes of 1 artifact file(s) contradict declared integrity and are untrusted: addon.node.";
describe("JavaScript application artifact integrity", () => {
it("analyzes a signed-after-packaging native module only when the mismatch is recorded", async () => {
const root = await createTestTempDirectory("rea-js-integrity-");
const source = join(root, "source");
await mkdir(source);
await writeFile(
join(source, "package.json"),
'{"name":"signed","main":"main.js"}',
);
await writeFile(
join(source, "main.js"),
"require('./addon.node'); export function value() { return {status: 'ready'}; }\n",
);
await writeFile(join(source, "addon.node"), "packed native bytes");
const archive = join(root, "app.asar");
await createPackageWithOptions(source, archive, { unpack: "*.node" });
// macOS code signing rewrites unpacked binaries after the archive header
// has recorded their integrity.
await writeFile(
join(`${archive}.unpacked`, "addon.node"),
"signed native bytes",
);
const strict = await analyzeJavaScriptApplication({ input_path: archive });
expect(strict.ok).toBe(false);
if (strict.ok) throw new Error("Expected an integrity failure");
expect(projectAnalysisError(strict.error)).toMatchObject({
code: "artifact_integrity_mismatch",
details: { logical_path: "addon.node", unpacked: true },
});
const recorded = await analyzeJavaScriptApplication({
input_path: archive,
integrity_policy: "record-and-continue",
});
expect(recorded.ok).toBe(true);
if (!recorded.ok) throw new Error("Expected recorded analysis");
expect(recorded.value.parameters).toEqual({
format: "auto",
integrity_policy: "record-and-continue",
});
expect(recorded.value.normalized_result).toMatchObject({
format: "asar",
integrity_contradictions: [
{
logical_path: "addon.node",
unpacked: true,
trust: "observed-untrusted",
},
],
graph: { coverage: { status: "partial" } },
limitations: expect.arrayContaining([CONTRADICTION_LIMITATION]),
semantic_graph: {
limitations: expect.arrayContaining([CONTRADICTION_LIMITATION]),
},
});
// Workflows built on the recorded graph inherit its partial coverage.
const clean = await analyzeJavaScriptApplication({
input_path: source,
format: "directory",
});
if (!clean.ok) throw new Error("Expected clean analysis");
const comparison = compareApplicationVersionsEvidence({
left: clean.value,
right: recorded.value,
});
expect(comparison.ok).toBe(true);
if (!comparison.ok) throw new Error("Expected comparison");
expect(comparison.value.normalized_result).toMatchObject({
coverage: { status: "partial", right_graph_status: "partial" },
limitations: expect.arrayContaining([CONTRADICTION_LIMITATION]),
});
const shapes = compareJavaScriptExportShapesEvidence({
left: clean.value,
right: recorded.value,
left_module_path: "main.js",
left_export_name: "value",
right_module_path: "main.js",
right_export_name: "value",
});
expect(shapes.ok).toBe(true);
if (!shapes.ok) throw shapes.error;
expect(shapes.value.normalized_result).toMatchObject({
coverage: { status: "partial" },
limitations: expect.arrayContaining([CONTRADICTION_LIMITATION]),
});
const trace = traceApplicationFeatureEvidence({
application: recorded.value,
seed: { kind: "string", value: "nonexistent-feature" },
direction: "outgoing",
});
expect(trace.ok).toBe(true);
if (!trace.ok) throw trace.error;
expect(trace.value.normalized_result).toMatchObject({
graph: null,
limitations: expect.arrayContaining([CONTRADICTION_LIMITATION]),
});
});
});
@@ -16,7 +16,7 @@ import { projectInputIssues } from "../../domain/inputIssueProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import { createJavaScriptApplicationEvidence } from "./JavaScriptApplicationEvidence.js";
import { createOwnedJavaScriptApplicationEvidence } from "./JavaScriptApplicationEvidence.js";
import { reconstructJavaScriptArtifact } from "./JavaScriptArtifactReconstruction.js";
import { JAVASCRIPT_APPLICATION_PROVIDER } from "../InvestigationProviders.js";
@@ -55,6 +55,7 @@ export const analyzeJavaScriptApplicationValidated = async (
{
input_path: input.input_path,
format: input.format,
integrity_policy: input.integrity_policy,
},
options.signal,
options.progress,
@@ -77,7 +78,7 @@ export const analyzeJavaScriptApplicationValidated = async (
total: 1,
message: "Creating and hashing application analysis Evidence",
});
const evidence = createJavaScriptApplicationEvidence(input, result);
const evidence = createOwnedJavaScriptApplicationEvidence(input, result);
await options.progress?.report({
phase: "analyze_javascript_application",
completed: 1,
@@ -92,6 +92,7 @@ const graphCoverage = (context: JavaScriptArtifactGraphContext) => {
javascript !== null && javascript.parse_status === "partial",
);
const unknownGap =
context.snapshot.integrity_contradictions.length > 0 ||
context.analysis.parse_failures > 0 ||
context.fileSet.invalid_utf8_files > 0 ||
sourceMapPolicyGap ||
@@ -103,6 +104,20 @@ const graphCoverage = (context: JavaScriptArtifactGraphContext) => {
return completeApplicationCoverage();
};
/**
* Disclose recorded integrity contradictions wherever the application graph
* is consumed; the graph is also partial, so comparisons and traces built on
* it do not claim completeness.
*/
export const integrityContradictionLimitations = (
snapshot: Pick<ArtifactInventorySnapshot, "integrity_contradictions">,
): string[] =>
snapshot.integrity_contradictions.length === 0
? []
: [
`Observed bytes of ${String(snapshot.integrity_contradictions.length)} artifact file(s) contradict declared integrity and are untrusted: ${snapshot.integrity_contradictions.map(({ logical_path: path }) => path).join(", ")}.`,
];
const graphLimitations = (
context: JavaScriptArtifactGraphContext,
coverage: "complete" | "partial" | "unknown" | "unavailable",
@@ -124,6 +139,7 @@ const graphLimitations = (
);
return [
...context.analysis.limitations,
...integrityContradictionLimitations(context.snapshot),
"CommonJS and ESM binding relationships were recovered from inert syntax and resolved only within the inventoried artifact container.",
"Webpack/Rspack factories were recovered from AST literals; REA did not invoke push handlers or bundle bootstrap code.",
"Static imports, entrypoints, workers, endpoints, and storage relationships do not prove runtime execution.",
@@ -362,6 +362,35 @@ describe("contextual JavaScript package path resolution", () => {
});
});
describe("generic package entry heuristics", () => {
it.each([
['{"main":"actual.cjs","module":"bundler.mjs"}', "actual.cjs"],
['{"module":"bundler.mjs"}', "bundler.mjs"],
])(
"preserves generic entry heuristics without a module kind for %s",
(metadata, entry) => {
const files = fileMap([
file("app/consumer.js", "root"),
file("app/node_modules/fixture/package.json", "root", metadata),
file("app/node_modules/fixture/actual.cjs", "root"),
file("app/node_modules/fixture/bundler.mjs", "root"),
file("app/node_modules/fixture/index.js", "root"),
]);
expect(
resolve({
declaredPath: "fixture",
sourcePath: "app/consumer.js",
context: "module-specifier",
files,
}),
).toMatchObject({
resolution_status: "resolved",
resolved_path: `app/node_modules/fixture/${entry}`,
});
},
);
});
describe("directory package entrypoint precedence", () => {
it.each([
["require", '{"main":"actual.cjs"}', "actual.cjs"],
@@ -461,10 +461,12 @@ const packageEntry = (
? { ...exported, source: "exports" }
: exported;
}
const preferred =
moduleKind === "import"
? [Reflect.get(value, "module"), Reflect.get(value, "main")]
: [Reflect.get(value, "main"), Reflect.get(value, "module")];
// Node imports and requires ignore the bundler-only module field. Keep
// that fallback for callers without an explicit module-loading kind.
const preferred = [
Reflect.get(value, "main"),
...(moduleKind === undefined ? [Reflect.get(value, "module")] : []),
];
const entry = preferred.find((candidate) => candidate !== undefined);
if (entry === undefined) return { status: "missing" };
const legacy = packagePathValue(entry);
@@ -5,9 +5,13 @@ import { createJavaScriptArtifactReader as createReader } from "../../artifacts/
import type { JavaScriptApplicationGraph } from "../../domain/javascript/javascriptApplicationGraph.js";
import type { JavaScriptSemanticGraph } from "../../domain/javascript/javascriptSemanticGraph.js";
import type { ElectronBoundarySummary } from "../../domain/javascript/javascriptApplicationAnalysis.js";
import type { IntegrityContradiction } from "../../domain/artifactGraph.js";
import { analyzeAndProjectJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
import { readJavaScriptArtifactFiles } from "../../artifacts/javascript/JavaScriptArtifactFiles.js";
import { buildJavaScriptArtifactGraph } from "./JavaScriptArtifactGraphBuilder.js";
import {
buildJavaScriptArtifactGraph,
integrityContradictionLimitations,
} from "./JavaScriptArtifactGraphBuilder.js";
import {
javascriptArtifactReconstructionInputSchema,
type JavaScriptArtifactReconstructionInput,
@@ -27,6 +31,7 @@ export interface JavaScriptArtifactReconstructionResult {
readonly graph: JavaScriptApplicationGraph;
readonly semantic_graph: JavaScriptSemanticGraph;
readonly electron_summary: ElectronBoundarySummary;
readonly integrity_contradictions: readonly IntegrityContradiction[];
readonly statistics: {
readonly relevant_files: number;
readonly nested_asar_containers: number;
@@ -55,7 +60,10 @@ export const reconstructJavaScriptArtifact = async (
abortIfNeeded(signal);
const path = await realpath(input.input_path);
const format = await resolveFormat(path, input);
const snapshot = await scanCanonicalArtifactInventory(path, { signal });
const snapshot = await scanCanonicalArtifactInventory(path, {
signal,
integrity: { mode: input.integrity_policy },
});
if (snapshot.manifest.root_format !== format)
throw new ArtifactReaderFailure(
"format",
@@ -101,6 +109,7 @@ export const reconstructJavaScriptArtifact = async (
snapshot.manifest.root_sha256,
graph,
analysis,
integrityContradictionLimitations(snapshot),
);
return {
input_path: path,
@@ -111,6 +120,7 @@ export const reconstructJavaScriptArtifact = async (
graph,
semantic_graph: semanticGraph,
electron_summary: summarizeElectronBoundaries(analysis),
integrity_contradictions: snapshot.integrity_contradictions,
statistics: {
relevant_files: files.files.length,
nested_asar_containers: files.containers.length,
@@ -1,9 +1,12 @@
import { z } from "zod";
import { artifactIntegrityPolicySchema } from "../../domain/artifactGraph.js";
/** Local ASAR/directory reconstruction request. */
export const javascriptArtifactReconstructionInputSchema = z.strictObject({
input_path: z.string().min(1),
format: z.enum(["auto", "asar", "directory"]).default("auto"),
integrity_policy: artifactIntegrityPolicySchema,
});
/** Parsed local reconstruction request. */
@@ -59,6 +59,8 @@ interface BuilderInput {
"graph_id" | "nodes"
>;
readonly analysis: JavaScriptArtifactAnalysis;
/** Source facts the semantic graph inherits, such as integrity contradictions. */
readonly sourceLimitations?: readonly string[];
}
interface FileContext extends SemanticFlowProjectionContext {
@@ -111,13 +113,19 @@ export const buildJavaScriptSemanticGraph = ({
rootArtifactSha256,
applicationGraph,
analysis,
sourceLimitations = [],
}: BuilderInput): JavaScriptSemanticGraph => {
const projection = createJavaScriptSemanticGraphProjection();
for (const analyzed of analysis.files) {
if (analyzed.semantic !== null)
projection.projectFile(analyzed.file, analyzed.semantic.ir);
}
return projection.finish(rootArtifactSha256, applicationGraph, analysis);
return projection.finish(
rootArtifactSha256,
applicationGraph,
analysis,
sourceLimitations,
);
};
/** File-local semantic projection that does not retain consumed source IR. */
@@ -130,6 +138,7 @@ export interface JavaScriptSemanticGraphProjection {
rootArtifactSha256: string,
applicationGraph: BuilderInput["applicationGraph"],
analysis: Pick<JavaScriptArtifactAnalysis, "truncated_scopes">,
sourceLimitations?: readonly string[],
) => JavaScriptSemanticGraph;
}
@@ -163,7 +172,12 @@ export const createJavaScriptSemanticGraphProjection =
};
return {
projectFile: projectSource,
finish: (rootArtifactSha256, applicationGraph, analysis) => {
finish: (
rootArtifactSha256,
applicationGraph,
analysis,
sourceLimitations = [],
) => {
bindSemanticGraphApplicationNodes(state, applicationGraph);
if (state.roots.size === 0) addFallbackRoot(rootArtifactSha256, state);
const unknowns = [...state.unknowns.values()];
@@ -206,6 +220,7 @@ export const createJavaScriptSemanticGraphProjection =
})),
},
limitations: [
...sourceLimitations,
"The semantic graph contains static syntax observations and conservative relationship candidates; it does not claim runtime execution.",
"Local data flow does not claim control-flow-sensitive reaching definitions or arbitrary dynamic property resolution.",
"Promise ownership covers explicit unshadowed Promise construction, static factories, aggregation, chaining, and await syntax only.",
+7
View File
@@ -1,4 +1,5 @@
import { readFile } from "node:fs/promises";
import { z } from "zod";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
@@ -113,6 +114,12 @@ const parseCaptureEvidence = (input: unknown) => {
try {
evidence = parseEvidence(input);
} catch (cause: unknown) {
if (cause instanceof z.ZodError)
throw new AnalysisInputError(
"compare_process_captures",
{ cause },
projectInputIssues(cause.issues, input),
);
throw invalidCaptureEvidence();
}
if (
+18 -5
View File
@@ -21,7 +21,10 @@ import {
type ArtifactAnalysisOperation,
} from "../contracts/artifactToolContracts.js";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
} from "../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../domain/artifactOperationError.js";
import { type AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -116,10 +119,15 @@ class ArtifactClient implements AnalysisClient {
parameters.path !== "." &&
parameters.path !== basename(this.target.path)
)
throw new ArtifactReaderFailure(
"path",
"For an active plist, path must select that archive (omit path or use its basename)",
);
throw new AnalysisInputError(operation, undefined, [
{
path: ["path"],
reason: "invalid_value",
message:
"For an active plist, path must select that archive (omit path or use its basename).",
expected: basename(this.target.path),
},
]);
const bundlePath = standalone
? dirname(this.target.path)
: this.target.sourcePath;
@@ -163,6 +171,9 @@ class ArtifactClient implements AnalysisClient {
inputPath: this.target.sourcePath ?? this.target.path,
inputFormat: this.target.format,
outputRoot: parsed.output_root,
integrity: resolveArtifactIntegrityPolicy({
mode: parsed.integrity_policy,
}),
},
options?.signal,
);
@@ -352,6 +363,8 @@ const translateFailure = (
cause.details,
cause.message,
);
// Caller-selection failures are already typed; keep their correction details.
if (cause instanceof AnalysisInputError) return cause;
return new ArtifactOperationError(operation, "io");
};
@@ -1,6 +1,14 @@
import { mkdir, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { collectInterfaceBuilderResourceKeys } from "./AppleAssetCatalogAnalysis.js";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
import {
analyzeAppleAssetCatalogs,
collectInterfaceBuilderResourceKeys,
} from "./AppleAssetCatalogAnalysis.js";
describe("Apple asset catalog application workflow", () => {
it("joins only explicit resource-key fields with archive provenance", () => {
@@ -40,4 +48,70 @@ describe("Apple asset catalog application workflow", () => {
},
]);
});
it("reports an app without compiled catalogs as an empty observed inventory", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-absent-");
await mkdir(join(bundle, "Contents", "Resources"), { recursive: true });
await writeFile(join(bundle, "Contents", "Resources", "icon.icns"), "");
const result = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => {
throw new Error("assetutil must not run without a catalog");
},
});
expect(result).toMatchObject({
catalogs: [],
total_records: 0,
records: [],
resource_key_matches: [],
next_offset: null,
truncated: false,
});
expect(result.limitations).toContain(
"No compiled Assets.car catalog was found among the app bundle's regular files (symlinks are not followed); any resource keys are reported as unmatched.",
);
});
it("does not report absence when the only catalog entry is a symlink", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-link-");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
await writeFile(join(bundle, "outside.car"), "catalog");
await symlink("../../outside.car", join(resources, "Assets.car"));
const rejected = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => Promise.resolve("[]"),
}).catch((cause: unknown) => cause);
expect(rejected).toBeInstanceOf(ArtifactReaderFailure);
expect(rejected).toMatchObject({
reason: "path",
message: expect.stringContaining(
"Contents/Resources/Assets.car (symlink)",
),
});
});
it("discloses skipped catalog entries beside inspected catalogs", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-mixed-");
const resources = join(bundle, "Contents", "Resources");
await mkdir(join(resources, "Nested"), { recursive: true });
await writeFile(join(resources, "Assets.car"), "catalog");
await symlink("../Assets.car", join(resources, "Nested", "Assets.car"));
const result = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => Promise.resolve("[]"),
});
expect(result.catalogs.map(({ path }) => path)).toEqual([
"Contents/Resources/Assets.car",
]);
expect(result.limitations).toContainEqual(
expect.stringContaining("Contents/Resources/Nested/Assets.car (symlink)"),
);
expect(result.limitations).not.toContainEqual(
expect.stringContaining("No compiled Assets.car catalog"),
);
});
});
@@ -48,10 +48,15 @@ export const analyzeAppleAssetCatalogs = async (input: {
sha256: string;
records: ReturnType<typeof parseAppleAssetCatalogRecords>;
}[] = [];
const uninspected: string[] = [];
let totalMetadataBytes = 0;
try {
for await (const entry of reader.entries(input.signal)) {
if (entry.kind !== "file" || !entry.path.endsWith("Assets.car")) continue;
if (!entry.path.endsWith("Assets.car")) continue;
if (entry.kind !== "file") {
uninspected.push(`${entry.path} (${entry.kind})`);
continue;
}
const inspected = await inspectAssetCatalogEntry({
reader,
entry,
@@ -69,10 +74,12 @@ export const analyzeAppleAssetCatalogs = async (input: {
} finally {
await reader.close();
}
if (catalogs.length === 0)
// Without any readable catalog, unmatched resource keys would overstate an
// absence that a skipped catalog entry may contradict.
if (catalogs.length === 0 && uninspected.length > 0)
throw new ArtifactReaderFailure(
"unavailable",
"No compiled Assets.car catalogs were found in the app bundle",
"path",
`Asset catalog entries are not regular files and were not inspected: ${uninspected.join(", ")}`,
);
const interfaceBuilder = await analyzeInterfaceBuilderBundle({
bundlePath: input.bundlePath,
@@ -93,6 +100,18 @@ export const analyzeAppleAssetCatalogs = async (input: {
...result,
limitations: [
...result.limitations,
// An app without a compiled catalog is an observed empty inventory, not
// a missing host tool.
...(catalogs.length === 0
? [
"No compiled Assets.car catalog was found among the app bundle's regular files (symlinks are not followed); any resource keys are reported as unmatched.",
]
: []),
...(uninspected.length === 0
? []
: [
`Asset catalog entries that are not regular files were not inspected, so resource keys reported as unmatched may resolve there: ${uninspected.join(", ")}`,
]),
"Exact resource-key joins use only recognized image or asset key fields on decoded Interface Builder resource objects; other archive resource references are not inferred.",
...(interfaceBuilder.graph.truncated
? [
+11 -1
View File
@@ -125,6 +125,16 @@ describe("inert keyed archive decoding", () => {
offset: 0,
limit: 1,
}),
).toThrow("does not exist");
).toThrow(
expect.objectContaining({
issues: [
expect.objectContaining({
path: ["root"],
message: expect.stringContaining("does not exist"),
expected: ["root", "other"],
}),
],
}),
);
});
});
+27 -10
View File
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { TextDecoder } from "node:util";
import { parseBinary } from "plist";
import { z } from "zod";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { projectPlistValue } from "../../domain/apple/plistValue.js";
import {
@@ -25,6 +26,10 @@ export const decodeKeyedArchiveBytes = (
) => {
if (bytes.length > MAX_BYTES)
throw new RangeError("Keyed archive exceeds 64 MiB");
if (bytes.subarray(0, 10).toString("ascii") === "NIBArchive")
throw new TypeError(
"Selected file is a compiled NIBArchive, not a Foundation plist archive; decode it with decode_interface_builder",
);
const binary = bytes.subarray(0, 8).toString("ascii") === "bplist00";
const parsed = binary
? { value: parseBinary(bytes), omittedPrototypeKeys: 0 }
@@ -83,18 +88,20 @@ export const inspectBundleKeyedArchive = async (input: {
.split("/")
.some((part) => part === ".." || part === "." || part.length === 0)
)
throw new ArtifactReaderFailure(
"path",
"Archive path must be a canonical relative bundle path",
throw archivePathError(
"invalid_format",
selected.path === "."
? "Select a keyed archive path relative to the active app bundle."
: `Archive path must be a canonical relative bundle path without empty, '.', or '..' segments: ${selected.path}`,
);
const reader = new DirectoryArtifactReader(input.bundlePath);
try {
for await (const entry of reader.entries(input.signal)) {
if (entry.path !== selected.path) continue;
if (entry.kind !== "file")
throw new ArtifactReaderFailure(
"path",
"Archive must be a regular file, not a symlink or directory",
throw archivePathError(
"invalid_value",
`Archive path selects a ${entry.kind}, not a regular file: ${selected.path}`,
);
if ((entry.declaredSize ?? 0) > MAX_BYTES)
throw new ArtifactReaderFailure(
@@ -132,18 +139,28 @@ export const inspectBundleKeyedArchive = async (input: {
...decodeKeyedArchiveBytes(bytes, selected),
});
} catch (cause) {
if (cause instanceof AnalysisInputError) throw cause;
throw new ArtifactReaderFailure(
cause instanceof RangeError ? "limit" : "format",
"Cannot decode selected Foundation keyed archive",
`Cannot decode selected Foundation keyed archive: ${cause instanceof Error ? cause.message : String(cause)}`,
{ cause },
);
}
}
throw new ArtifactReaderFailure(
"path",
`Archive not found in bundle: ${selected.path}`,
throw archivePathError(
"invalid_value",
`No regular file exists at ${selected.path} in the active app bundle.`,
);
} finally {
await reader.close();
}
};
/** A caller-selected archive path, not the artifact, failed its constraint. */
const archivePathError = (
reason: "invalid_format" | "invalid_value",
message: string,
) =>
new AnalysisInputError("inspect_keyed_archive", undefined, [
{ path: ["path"], reason, message },
]);
+58 -35
View File
@@ -21,15 +21,19 @@ import {
type ArtifactGraphManifest,
type ArtifactNode,
type ArtifactOccurrence,
type IntegrityContradiction,
} from "../../domain/artifactGraph.js";
import type { BinaryTarget } from "../../domain/binaryTarget.js";
import { scanArtifactInventory } from "../inventory/ArtifactInventory.js";
import type { ArtifactIntegrityPolicy } from "../inventory/types.js";
/** Local extraction input with the output root chosen by the adapter. */
export interface ArtifactExtractionInput {
readonly inputPath: string;
readonly inputFormat: BinaryTarget["format"];
readonly outputRoot: string;
/** Integrity mismatches fail unless the caller records and continues. */
readonly integrity?: ArtifactIntegrityPolicy;
}
/** Extract every regular inventory occurrence into an exclusively owned absent root. */
@@ -40,29 +44,25 @@ export const extractArtifact = async (
const sourcePath = await realpath(input.inputPath);
const snapshot = await scanArtifactInventory(sourcePath, {
signal,
integrity: input.integrity,
});
const selectedOccurrences = snapshot.occurrences.filter(
(occurrence) =>
(occurrence.entry_kind === "file" || occurrence.entry_kind === "slice") &&
occurrence.logical_path !== ".",
);
const selectedIds = new Set(
selectedOccurrences.map(({ occurrence_id: id }) => id),
const occurrences = new Map(
snapshot.occurrences.map((occurrence) => [
occurrence.occurrence_id,
occurrence,
]),
);
const occurrences = new Map<string, ArtifactOccurrence>();
const neededNodes = new Set<string>();
collectOccurrences(
snapshot.occurrences,
selectedIds,
occurrences,
neededNodes,
);
const nodes = new Map<string, ArtifactNode>();
collectNodes(snapshot.nodes, neededNodes, nodes);
const nodes = new Map(snapshot.nodes.map((node) => [node.artifact_id, node]));
const inventory: LoadedInventory = {
manifest: snapshot.manifest,
occurrences,
nodes,
integrityContradictions: snapshot.integrity_contradictions,
};
const selected = selectedOccurrences.map((occurrence) => {
if (
@@ -187,6 +187,39 @@ const createExtractionResult = (
selected: readonly SelectedOccurrence[],
extracted: readonly ExtractedOccurrence[],
): ArtifactExtractionResult => {
const materializedIds = new Set(
selected.map(({ occurrence }) => occurrence.occurrence_id),
);
const materializedPaths = new Map(
selected.map(({ occurrence }) => [occurrence.logical_path, occurrence]),
);
const wasMaterialized = (contradiction: IntegrityContradiction): boolean => {
if (materializedIds.has(contradiction.occurrence_id)) return true;
if (!contradiction.unpacked) return false;
// An unpacked member is read beside its enclosing ASAR, not from ASAR bytes.
const occurrence = inventory.occurrences.get(contradiction.occurrence_id);
let parentId = occurrence?.parent_occurrence_id;
while (parentId !== null && parentId !== undefined) {
const parent = inventory.occurrences.get(parentId);
if (parent === undefined) return false;
if (
parent.artifact_id !== null &&
inventory.nodes.get(parent.artifact_id)?.format === "asar"
) {
const companion = materializedPaths.get(
`${parent.logical_path}.unpacked${contradiction.logical_path.slice(parent.logical_path.length)}`,
);
return companion?.artifact_id === occurrence?.artifact_id;
}
parentId = parent.parent_occurrence_id;
}
return false;
};
const writtenContradictions =
inventory.integrityContradictions.filter(wasMaterialized);
const nestedContradictions = inventory.integrityContradictions.filter(
(contradiction) => !wasMaterialized(contradiction),
);
const extractionSemantic = {
source_manifest_id: inventory.manifest.manifest_id,
selected_occurrence_ids: selected
@@ -206,8 +239,19 @@ const createExtractionResult = (
containment_verified: true,
cleanup: { attempted: false, verified: true, residual_paths: [] },
provenance: [],
integrity_contradictions: inventory.integrityContradictions,
limitations: [
"All regular files in the active artifact were materialized; nested archive contents remain represented by their containing file.",
"All regular files in the active artifact were materialized. The artifacts list identifies written files; nested inventory observations may refer to additional logical paths.",
...(writtenContradictions.length === 0
? []
: [
`${String(writtenContradictions.length)} extracted file(s) contradict declared integrity; their observed bytes were written and are untrusted.`,
]),
...(nestedContradictions.length === 0
? []
: [
`${String(nestedContradictions.length)} nested integrity contradiction(s) were not written as their own files (${nestedContradictions.map(({ logical_path }) => logical_path).join(", ")}); their records describe inventory observations, not materialized files.`,
]),
],
});
};
@@ -216,30 +260,9 @@ interface LoadedInventory {
readonly manifest: ArtifactGraphManifest;
readonly occurrences: ReadonlyMap<string, ArtifactOccurrence>;
readonly nodes: ReadonlyMap<string, ArtifactNode>;
readonly integrityContradictions: readonly IntegrityContradiction[];
}
const collectOccurrences = (
items: readonly ArtifactOccurrence[],
selected: ReadonlySet<string>,
output: Map<string, ArtifactOccurrence>,
neededNodes: Set<string>,
): void => {
for (const item of items) {
if (!selected.has(item.occurrence_id)) continue;
output.set(item.occurrence_id, item);
if (item.artifact_id !== null) neededNodes.add(item.artifact_id);
}
};
const collectNodes = (
items: readonly ArtifactNode[],
selected: ReadonlySet<string>,
output: Map<string, ArtifactNode>,
): void => {
for (const item of items)
if (selected.has(item.artifact_id)) output.set(item.artifact_id, item);
};
const createReader = async (
path: string,
format: BinaryTarget["format"],
@@ -26,6 +26,7 @@ import {
type OwnedProcessGroup,
type ProcessCleanupResult,
type ProcessLineageObservation,
type OwnershipSweepUnverifiedProcess,
} from "../process/ProcessOwnership.js";
import {
observeOwnedProcessLineage,
@@ -238,6 +239,17 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
),
);
if (closeError !== undefined) return err(providerError(closeError));
if (outcome.ok && cleanup.unverified !== undefined)
outcome = ok({
...outcome.value,
limitations: [
...outcome.value.limitations,
...cleanup.unverified.map(
({ pid, diagnostic }) =>
`Ownership of unrelated process ${String(pid)} could not be verified; it was left untouched: ${diagnostic}`,
),
],
});
}
return outcome;
}
@@ -280,6 +292,7 @@ const cleanupElectronProcesses = async (
})),
);
let signaled = false;
const unverified: OwnershipSweepUnverifiedProcess[] = [];
for (const processGroupId of groupIds) {
const cleanupOwnership: OwnedProcessGroup =
processGroupId === ownership.processGroupId
@@ -292,14 +305,38 @@ const cleanupElectronProcesses = async (
const result = await cleanupOwnedProcessGroup(cleanupOwnership);
if (!result.cleaned) return result;
signaled ||= result.signaled;
unverified.push(...(result.unverified ?? []));
}
const remaining = await verifyNoTokenOwnedProcesses(
ownership.runId,
undefined,
ownership.captureBaseline,
{
leaderPid: ownership.leaderPid,
processGroupId: ownership.processGroupId,
...(ownership.sampledProcessGroupIds === undefined
? {}
: { sampledProcessGroupIds: ownership.sampledProcessGroupIds }),
},
);
if (!remaining.cleaned) return remaining;
return { cleaned: true, signaled };
unverified.push(...(remaining.unverified ?? []));
return {
cleaned: true,
signaled,
...(unverified.length === 0
? {}
: {
unverified: [
...new Map(
unverified.map((item) => [
`${String(item.pid)}:${item.diagnostic}`,
item,
]),
).values(),
],
}),
};
};
const createResult = (
@@ -1,5 +1,5 @@
import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, sep } from "node:path";
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
@@ -55,7 +55,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
"path",
`Selected source path changes during normalization: ${selected.content.relative_path}`,
);
const root = dirname(input.manifest_path);
const root = resolve(dirname(input.manifest_path));
const sourcePath = join(root, "files", portable);
targetPath = sourcePath;
await assertContained(root, sourcePath);
+18 -4
View File
@@ -161,14 +161,28 @@ it("rejects identical duplicate core fields before schema validation", () => {
expect(() => decodeHarCapture(text, [])).toThrow("duplicate");
});
it("reports an unsupported JSON schema boundary instead of dropping a producer property", () => {
it("preserves prototype-named producer members through the capture boundary", () => {
const text = JSON.stringify(historicalHar()).replace(
'"cache":{}',
'"cache":{},"_extension":{"__proto__":{"preserved":7},"constructor":"ordinary"}',
);
expect(() => decodeHarCapture(text, [])).toThrow(
"JSON schema boundary cannot preserve",
);
const result = decodeHarCapture(text, []);
const reported = result.records[0]?.reported;
expect(reported).toBeDefined();
if (
reported === undefined ||
reported === null ||
typeof reported !== "object" ||
Array.isArray(reported)
)
throw new Error("No HAR object record was retained");
const extension = reported["_extension"];
expect(extension).toEqual({
["__proto__"]: { preserved: 7 },
constructor: "ordinary",
});
expect(Object.getPrototypeOf(extension)).toBe(Object.prototype);
expect(Reflect.get(Object.prototype, "preserved")).toBeUndefined();
});
it("preserves Unicode, offsets, sizes, duplicate URLs and opaque extensions without inventing bytes", () => {
+3 -5
View File
@@ -58,11 +58,9 @@ export const parseHarJson = (text: string): unknown => {
});
},
onObjectProperty: (key) => {
if (key === "__proto__")
throw new CaptureFormatError(
"unsupported",
"HAR object contains a __proto__ member that the current JSON schema boundary cannot preserve.",
);
// Prototype-named members are kept as ordinary own properties: the
// members map keys them without prototype setters, and
// Object.fromEntries materializes them as own data properties.
const frame = frames.at(-1);
if (frame?.kind !== "object")
throw new TypeError(
+2 -1
View File
@@ -4,6 +4,7 @@ import { Cli } from "incur";
import { createLogger, parseLogLevel } from "./logger.js";
import { PRODUCT_IDENTITY } from "./identity.js";
import { npxRegistrationCommand } from "./application/ClientRegistrationIdentity.js";
import { registerSetupCommands } from "./cli/setupCommands.js";
import { registerCoreAnalysisCommands } from "./cli/coreAnalysisCommands.js";
import { registerUtilityCommands } from "./cli/utilityCommands.js";
@@ -47,7 +48,7 @@ export const createCli = (
version: PRODUCT_IDENTITY.packageVersion,
description: "Reverse engineer anything from your terminal or agent.",
mcp: {
command: PRODUCT_IDENTITY.mcpCommand,
command: npxRegistrationCommand().join(" "),
instructions:
"Ask what software, artifact, protocol, or behavior the user wants to understand, then choose the available investigation capabilities that can produce evidence.",
},
+11 -9
View File
@@ -1,4 +1,5 @@
import { z } from "incur";
import { artifactIntegrityPolicySchema } from "../domain/artifactGraph.js";
import { runProviderAnalysis } from "../composition/directAnalysis.js";
import { createArtifactExtractionDestination } from "../application/artifacts/ArtifactExtractionDestination.js";
@@ -193,14 +194,19 @@ const registerExtractionCommand = (cli: CliInstance, logger: Logger): void => {
args: z.object({
path: z.string().describe("Application or package path"),
}),
options: z.object({}),
run: ({ args }) =>
options: z.object({
integrityPolicy: artifactIntegrityPolicySchema.describe(
"Behavior when declared artifact integrity does not match",
),
}),
run: ({ args, options }) =>
logCliCommand(logger, "extract-artifact", () =>
runProviderAnalysis(
args.path,
"extract_artifact",
{
output_root: createArtifactExtractionDestination(),
integrity_policy: options.integrityPolicy,
},
logger,
),
@@ -216,14 +222,10 @@ const registerInspectionCommand = (cli: CliInstance, logger: Logger): void => {
path: z.string().describe("Application or package path"),
}),
options: z.object({
integrityPolicy: z
.enum(["fail", "record-and-continue"])
.default("fail")
.describe("Behavior when declared artifact integrity does not match"),
integrityPolicy: artifactIntegrityPolicySchema.describe(
"Behavior when declared artifact integrity does not match",
),
}),
alias: {
integrityPolicy: "integrity-policy",
},
run: ({ args, options }) =>
logCliCommand(logger, "inspect-artifact", () =>
runProviderAnalysis(
+11 -2
View File
@@ -1,4 +1,5 @@
import { z } from "incur";
import { artifactIntegrityPolicySchema } from "../domain/artifactGraph.js";
import { stat } from "node:fs/promises";
import { resolve } from "node:path";
@@ -51,7 +52,11 @@ const registerOverviewCommands = (
args: z.object({
path: z.string().describe("App, program, or analysis database path"),
}),
options: overviewOptions,
options: overviewOptions.extend({
integrityPolicy: artifactIntegrityPolicySchema.describe(
"Behavior when declared JavaScript artifact integrity does not match",
),
}),
run: ({ args, options, format }) =>
logCliCommand(logger, "analyze", () =>
runRoutedOverview(
@@ -123,6 +128,7 @@ const runRoutedOverview = async (
readonly snapshot?: string | undefined;
readonly "target-format"?: "dos-com" | undefined;
readonly provider?: string | undefined;
readonly integrityPolicy?: "fail" | "record-and-continue" | undefined;
},
logger: Logger,
output?: CliCommandOutput,
@@ -134,7 +140,10 @@ const runRoutedOverview = async (
(await isJavaScriptApplicationPath(path))
)
return runCliJavaScriptApplicationAnalysis(
{ input_path: resolve(path) },
{
input_path: resolve(path),
integrity_policy: options.integrityPolicy ?? "fail",
},
output,
);
return runDirectAnalysis(
+5 -1
View File
@@ -186,7 +186,11 @@ const registerJavaScriptApplicationCommand = (
run: ({ args, options, format }) =>
logCliCommand(logger, CLI_COMMANDS.analyzeJavaScriptApplication, () =>
runCliJavaScriptApplicationAnalysis(
{ input_path: args.path, format: options.artifactFormat },
{
input_path: args.path,
format: options.artifactFormat,
integrity_policy: options.integrityPolicy,
},
resultOutput === undefined
? undefined
: {
+5 -107
View File
@@ -1,5 +1,7 @@
import type { Writable } from "node:stream";
import { bufferedJsonParts, jsonParts } from "../domain/jsonSerialization.js";
type JsonOutputFormat = "json" | "jsonl";
/** Write ordinary JSON without constructing a document-sized string. */
@@ -8,7 +10,7 @@ export const writeJsonOutput = async (
destination: Writable,
format: JsonOutputFormat = "json",
): Promise<void> => {
for (const part of bufferedParts(documentParts(value, format)))
for (const part of bufferedJsonParts(documentParts(value, format)))
await writePart(destination, part);
};
@@ -39,115 +41,11 @@ function* documentParts(
): Generator<string> {
if (format === "jsonl" && Array.isArray(value)) {
for (const item of value) {
yield* encodeValue(item, false, 0, new Set());
yield* jsonParts(item);
yield "\n";
}
return;
}
yield* encodeValue(value, format === "json", 0, new Set());
yield* jsonParts(value, format === "json");
yield "\n";
}
function* bufferedParts(parts: Iterable<string>): Generator<string> {
let buffer = "";
for (const part of parts) {
if (buffer.length + part.length > 64 * 1024 && buffer !== "") {
yield buffer;
buffer = "";
}
buffer += part;
}
if (buffer !== "") yield buffer;
}
function* encodeString(value: string): Generator<string> {
if (value.length <= 8192) {
yield JSON.stringify(value);
return;
}
yield '"';
for (let start = 0; start < value.length;) {
let end = Math.min(start + 8192, value.length);
const last = value.charCodeAt(end - 1);
const next = value.charCodeAt(end);
// Keep a surrogate pair in the same JSON.stringify and UTF-8 write.
if (
end < value.length &&
last >= 0xd800 &&
last <= 0xdbff &&
next >= 0xdc00 &&
next <= 0xdfff
)
end -= 1;
yield JSON.stringify(value.slice(start, end)).slice(1, -1);
start = end;
}
yield '"';
}
function* encodeValue(
value: unknown,
pretty: boolean,
depth: number,
ancestors: Set<object>,
): Generator<string> {
if (typeof value === "string") {
yield* encodeString(value);
return;
}
if (value === null || typeof value === "boolean") {
yield value === null ? "null" : String(value);
return;
}
if (typeof value === "number" && Number.isFinite(value)) {
yield String(value);
return;
}
if (typeof value !== "object" || value === null)
throw new TypeError("CLI JSON output requires ordinary JSON values");
if (ancestors.has(value))
throw new TypeError("CLI JSON output contains a circular reference");
if (
!Array.isArray(value) &&
Object.prototype.toString.call(value) !== "[object Object]"
)
throw new TypeError("CLI JSON output requires ordinary JSON objects");
ancestors.add(value);
try {
const array = Array.isArray(value);
const keys = array
? undefined
: Object.keys(value).filter((key) => {
const item: unknown = Reflect.get(value, key);
// Incur's filter can select inherited methods. Match native JSON's
// omission of fields that have no JSON representation.
return (
item !== undefined &&
typeof item !== "function" &&
typeof item !== "symbol"
);
});
const count = array ? value.length : (keys?.length ?? 0);
yield array ? "[" : "{";
for (let index = 0; index < count; index += 1) {
if (index > 0) yield ",";
if (pretty) yield `\n${" ".repeat(depth + 1)}`;
let item: unknown;
if (array) {
item = value[index];
} else {
const key = keys?.[index];
if (key === undefined)
throw new TypeError("CLI JSON object changed during serialization");
yield* encodeString(key);
yield pretty ? ": " : ":";
item = Reflect.get(value, key);
}
yield* encodeValue(item, pretty, depth + 1, ancestors);
}
if (pretty && count > 0) yield `\n${" ".repeat(depth)}`;
yield array ? "]" : "}";
} finally {
ancestors.delete(value);
}
}
+9 -4
View File
@@ -37,9 +37,9 @@ export const parseCliJsonInput = async (
/**
* Resolve named string fields of CLI-supplied JSON against the operator
* working directory. Each entry is a key path from the document root; only
* fields holding strings are resolved, everything else passes through
* untouched. Shared MCP contracts reject relative paths, so CLI workflows
* resolve operator-relative values before validation.
* fields holding nonblank strings are resolved, everything else passes through
* untouched so shared schemas can reject blank paths. Shared MCP contracts reject
* relative paths, so CLI workflows resolve operator-relative values before validation.
*/
export const resolveCliJsonPaths = (
value: unknown,
@@ -61,7 +61,12 @@ const resolveJsonPath = (
([key, entry]: readonly [string, unknown]): readonly [string, unknown] => {
if (key !== head) return [key, entry];
if (tail.length === 0)
return [key, typeof entry === "string" ? resolve(entry) : entry];
return [
key,
typeof entry === "string" && entry.trim().length > 0
? resolve(entry)
: entry,
];
return [key, resolveJsonPath(entry, tail)];
},
);
+59
View File
@@ -0,0 +1,59 @@
import { resolve } from "node:path";
import { describe, expect, it } from "vitest";
import { resolveCliJsonPaths } from "./cliJsonInput.js";
import { electronActiveObservationInputSchema } from "./domain/javascript/electronActiveObservation.js";
const paths = [
["executable_path"],
["application_path"],
["application_root"],
] as const;
describe("CLI JSON path normalization", () => {
it.each([
["executable_path", ""],
["executable_path", " \t "],
["application_path", ""],
["application_path", " \t "],
["application_root", ""],
["application_root", " \t "],
])(
"keeps blank %s (%j) invalid at the shared input boundary",
(field, value) => {
const input = {
executable_path: "/synthetic/electron",
application_path: "/synthetic/app",
application_root: "/synthetic",
[field]: value,
};
const parsed = electronActiveObservationInputSchema.safeParse(
resolveCliJsonPaths(input, paths),
);
expect(parsed.success).toBe(false);
if (parsed.success) throw new Error("Expected blank path rejection");
expect(parsed.error.issues).toEqual(
expect.arrayContaining([expect.objectContaining({ path: [field] })]),
);
expect(input).toHaveProperty(field, value);
},
);
it("resolves selected relative paths without trimming names or changing unrelated fields", () => {
const input = {
browser: { executable_path: " browser ", label: "relative-label" },
application_path: "app",
};
expect(
resolveCliJsonPaths(input, [["browser", "executable_path"]]),
).toEqual({
browser: {
executable_path: resolve(" browser "),
label: "relative-label",
},
application_path: "app",
});
expect(input.browser.executable_path).toBe(" browser ");
});
});
+4
View File
@@ -1,4 +1,5 @@
import { z } from "incur";
import { artifactIntegrityPolicySchema } from "./domain/artifactGraph.js";
/** Accept an agent-selected browser observation duration without an artificial ceiling. */
export const observationDuration = (fallback: number, minimum = 0) =>
@@ -63,4 +64,7 @@ export const javascriptApplicationOptions = z.object({
.enum(["auto", "asar", "directory"])
.default("auto")
.describe("Application artifact format"),
integrityPolicy: artifactIntegrityPolicySchema.describe(
"Behavior when declared artifact integrity does not match",
),
});
Loaded 100 of 200 files, more files were not shown because too many files have changed in this diff. Show more