Compare commits

...
Author SHA1 Message Date
morluto 2e636ac58e test(native): isolate signature rejection fixtures 2026-10-08 23:55:06 +08:00
morluto 5eebf19e93 chore: sync branch with latest main 2026-10-08 23:52:20 +08:00
morluto 78a59d4e8f fix(native): bind signature results and preserve verification uncertainty 2026-10-08 23:47:39 +08:00
Ysopwnable a1ed4232de fix(mcp): retain complete application results across response limits
Retain complete authenticated application Evidence when the repeated MCP representation exceeds the default receive frame or Node string limit. Return a typed transport constraint, keep useful followup queries available, and stream full canonical bundles to atomic destinations.

Verify complete Notion and Obsidian workflows on Linux and native Windows, full CLI/Evidence parity, and real large Chrome screenshot recovery. Preserve complete success contracts and caller-selected evidence paths.

Refs #1065, #1053.
2026-10-08 23:46:02 +08:00
morluto 859ce82569 chore: merge main and reconcile CI projections 2026-10-08 23:45:07 +08:00
morluto ae5e778de2 fix(ci): restore generated projections in test shards 2026-10-08 23:32:34 +08:00
morluto c701376777 perf(build): avoid duplicate documentation generation 2026-10-08 23:29:43 +08:00
morluto 2af12a18a3 fix(process): preserve observations after successful cleanup 2026-10-08 23:29:43 +08:00
morluto 5e1afe454f fix(snapshot): retain MCP workflow replay bindings 2026-10-08 23:29:43 +08:00
morluto a9e00b83f3 docs: clarify evidence ownership and verification guidance 2026-10-08 23:29:43 +08:00
Ysopwnable 5f93b4b125 fix(ci): restore generated shard inputs and cleanup assertions
Restore ignored generated inputs before deterministic test shards consume the downloaded build. Preserve verified cleanup assertions and explicitly check residual unknowns for unrelated processes.
2026-10-08 23:20:52 +08:00
morluto 839f3e3d8e chore: synchronize signature posture branch with latest main guidance 2026-10-08 23:16:56 +08:00
morluto c1535d88ab fix(native): preserve signature uncertainty and validate ticket framing 2026-10-08 23:16:28 +08:00
morluto 46d195e8a4 docs: check surrounding cases when changing boundaries 2026-10-08 22:57:03 +08:00
morluto 9833c94656 docs: require surrounding boundary checks for review fixes 2026-10-08 22:56:00 +08:00
morluto 6e6ff7bd2e docs: move star milestone into history sections across readmes 2026-10-08 22:54:57 +08:00
Ysopwnable 9455e7c8bc docs: celebrate 20,000 GitHub stars 🎉 (#1079) 2026-10-08 22:44:10 +08:00
morluto e6134056cf fix(build): make catalogs and packaged skills generated outputs (#1086)
* fix(build): generate catalogs and packaged skills without snapshot commits

* fix(build): point skill drift recovery to uncached generation
2026-10-08 22:43:51 +08:00
morlutoandCursor Agent d46449e54b fix(native): keep signature failure evidence instead of collapsing it (#1084)
A regular CodeResources file replaced by a directory or FIFO before open
was reported absent. That hid the replacement; it is now unreadable with
reason changed.

A failed anchor apple probe kept only an exit code. The caller-visible
limitation now includes the probe's diagnostic.

Definitive codesign rejections are a phrase list, including unrecognized
bundle format, instead of one growing regular expression. Unlisted
nonzero diagnostics stay unknown.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-08 22:41:28 +08:00
Ysopwnable b49beb42d7 fix(browser): normalize selected script manifest roots on Windows (#1078)
* fix(browser): normalize selected script manifest roots on Windows

* test(browser): keep manifest fixtures valid across Windows drives
2026-10-08 22:37:10 +08:00
Ysopwnable a4d77f0ff0 docs(website): add a CTF showcase and FAQ (#1077)
* docs(website): add a worked CTF reverse-engineering showcase

* docs(website): add a dedicated FAQ page
2026-10-08 22:30:10 +08:00
Kaoru0822 82a1a525d7 Merge pull request #1073 from morluto/fix/evidence-bundle-diagnostics
fix(evidence): name the failed constraint when a bundle is rejected
2026-10-08 07:27:21 -07:00
Kaoru0822 e8b92a40c0 Merge pull request #1074 from morluto/fix/evidence-file-missing
fix(evidence): report missing evidence files instead of permission advice
2026-10-08 07:22:31 -07:00
Ysopwnable d52f66d62a fix(metadata): stop schema changes churning generated skill evidence (#1075) 2026-10-08 22:21:30 +08:00
Kaoru0822 e2f03786f4 Merge pull request #1071 from morluto/fix/asset-catalog-absent
fix(apple): report apps without asset catalogs as empty inventories
2026-10-08 07:15:17 -07:00
Kaoru0822 5a55d5b1c6 Merge pull request #1070 from morluto/fix/keyed-archive-selection-errors
fix(artifacts): report keyed-archive selection mistakes as invalid input
2026-10-08 07:15:02 -07:00
Ysopwnable ef30343154 fix(setup): launch Windows npx registrations through cmd.exe (#1072) 2026-10-08 22:11:54 +08:00
Kaoru0822andClaude Opus 5.5 38bbb656da fix(evidence): require a complete record before the single-record hint
A malformed bundle that merely has an evidence_id field was described
as one Evidence record. Use the single-record message only when the
input matches the Evidence record schema; otherwise report the bundle
schema failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:09:28 +00:00
Kaoru0822andClaude Opus 5.5 89252f1f35 fix(evidence): report missing evidence files instead of permission advice
Reading a nonexistent Evidence bundle or snapshot, or exporting into a
directory that does not exist, failed as reason "io" with "Evidence
file could not be accessed. Check file permissions", and with no path.
In `rea compare` the caller could not tell which of the two files
failed.

Map ENOENT and ENOTDIR to a new "missing" reason with read- and
write-specific advice. Every EvidenceFileError now carries the resolved
caller path in its projected details. Permission and other I/O failures
keep the existing advice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 14:07:43 +00:00
muhammad-luayandN0zoM1z0 5547c1e697 fix(process): stop failing captures on tokens macOS cannot expose (#1057)
* fix(process): stop failing captures on tokens macOS cannot expose

On a busy macOS 26 host the residual process sweep fails every capture:
KERN_PROCARGS2 omits the environment of Apple platform binaries for any
caller and fails with EINVAL for processes owned by another account, so an
unrelated process created after the capture baseline escalates to a hard
cleanup failure with cleanup_incomplete.

Report environment withholding distinctly from the Darwin token reader by
reading the platform-binary flag through csops, carry the effective uid in
the process table, and re-read unresolved tokens with a bounded retry plus a
final liveness confirmation so transient exec and exit states settle.
Candidates whose token the OS provably withholds, foreign-uid candidates,
and candidates that exited during validation are now recorded as unverified
processes instead of failing an otherwise clean capture or being signaled.
Related candidates and every other unreadable candidate stay fail-closed.

* test(process): align ownership lanes with the uid-aware process table

The Darwin sweep reads the effective uid through ps, so the offline
verification lanes' syscall allowlists must accept the new argv in
verify-binary-layout and verify-evm-interface. The foreign-uid ownership
test requires POSIX process uids; skip it where the platform has none
(the curated Windows lane).

* fix(verify): recognize UID-bearing ownership process tables

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 22:01:05 +08:00
Kaoru0822andClaude Opus 5.5 cf96ceb211 fix(evidence): name the failed constraint when a bundle is rejected
evidence-import, evidence-export, compare and the MCP bundle tools gave
every rejected file the same message: "Evidence is invalid or has
changed. Recreate or re-import it". That included a single Evidence
record, which is what every command prints, so `rea compare a.json
b.json` on two command results looked like tampering.

Report a single record as such and point to export_evidence_bundle.
Locate the first schema issue, and keep thrown constraint messages such
as a semantic identifier mismatch. EvidenceIntegrityError now accepts
the userMessage option that other typed errors already use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:59:48 +00:00
Kaoru0822andClaude Opus 5.5 a079af297b fix(apple): keep skipped catalog entries from reading as absence
A symlink or directory named Assets.car is skipped by the
non-following directory reader. When no regular catalog remains, fail
with the skipped paths instead of reporting an empty inventory. When
catalogs were read, list the skipped entries in the limitations. The
absence limitation now says that symlinks are not followed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:52:18 +00:00
Kaoru0822andClaude Opus 5.5 79e5a3923d fix(apple): report apps without asset catalogs as empty inventories
inspect_asset_catalog failed with an "unavailable" artifact error for an
app bundle that has no compiled Assets.car. Its advice was "Check
artifact support and required tools", which does not apply: assetutil
was present, and the absence of a catalog is itself an observation.
decode_interface_builder already returns an empty graph for an app
without NIBs.

Return the empty page with an explicit limitation. Interface Builder
resource keys are still decoded and reported as unmatched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:45:43 +00:00
Kaoru0822andClaude Opus 5.5 25fb6d32a3 fix(artifacts): report keyed-archive selection mistakes as invalid input
inspect_keyed_archive reported caller selection failures as artifact
faults. A missing named root became "Artifact is invalid or has changed.
Get a fresh copy" and lost its reason, and a missing, non-canonical, or
directory archive path became "unsafe or conflicting internal path ...
correct the artifact".

Report these as invalid_request issues on `path` or `root`, listing the
available $top roots or the active plist basename. Decode failures keep
the decoder's reason, and a compiled NIBArchive points to
decode_interface_builder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:43:30 +00:00
40e18b2e37 fix(domain): preserve prototype-named JSON members through Evidence boundaries (#1049)
* fix(domain): preserve prototype-named JSON members through Evidence boundaries

Zod's z.json() record projection drops own __proto__ members as a
prototype-pollution guard, so provider replies carrying them lost evidence
before hashing/serialization and CLI/MCP output. The shared schema now
validates with z.json() as before, then rebuilds the value keeping
prototype-named own members as ordinary data properties (__proto__ via
Object.defineProperty, so the prototype is never mutated).

parseHarJson no longer rejects __proto__ members; the explicit unsupported
case from #992 is replaced with a preservation regression, and the
inspect_web_network_capture contract text is updated to match.

Fixes #1036

* fix(domain): bound JSON depth and oversized payloads at parsing boundaries

jsonValueSchema re-walked hostile input through z.json()'s recursive core, so a deeply nested payload from an MCP tool argument, inline evidence, provider result, or keyed archive threw RangeError: Maximum call stack size exceeded instead of a validation issue. The schema now runs an iterative depth guard first and keeps z.json() for full JSON semantics behind it, which also turns cyclic values into validation errors. The 1000-edge limit stays far below the range where the recursive core overflows its stack (measured between 2000 and 3000 on Node 24) while leaving room for legitimately deep target-derived trees such as compiled NIB view hierarchies, and the guard narrows with a typed record check instead of any or unchecked casts. The non-recursive JSON Schema advertisement stays intact on both the public and standard projection paths. Two payload sizes get explicit budgets tied to their producer constraints: web text artifacts reject beyond the 16 MiB source budget the web tooling already advertises, and native UI screenshot base64 beyond the 64 MiB helper output budget. Regenerated product catalog, skill catalog digest, and completion-ledger manifest follow the schema-boundary change.

* fix(validation): preserve depth diagnostics across process inputs

---------

Co-authored-by: DeryFerd <deryferdikao125@gmail.com>
Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 21:32:48 +08:00
morluto 1325bf936f chore: merge main into feat/signature-posture 2026-10-08 21:29:24 +08:00
morluto 75460dae2f chore: merge main into feat/signature-posture 2026-10-08 21:20:04 +08:00
morluto d63b903535 fix(native): validate prepared and validated path records consistently 2026-10-08 21:17:12 +08:00
Dery FerdikaandN0zoM1z0 bfbfc7330a fix(domain): reject excessive JSON depth with actionable diagnostics (#1029)
* fix(domain): bound JSON depth and oversized payloads at parsing boundaries

jsonValueSchema re-walked hostile input through z.json()'s recursive core, so a deeply nested payload from an MCP tool argument, inline evidence, provider result, or keyed archive threw RangeError: Maximum call stack size exceeded instead of a validation issue. The schema now runs an iterative depth guard first and keeps z.json() for full JSON semantics behind it, which also turns cyclic values into validation errors. The 1000-edge limit stays far below the range where the recursive core overflows its stack (measured between 2000 and 3000 on Node 24) while leaving room for legitimately deep target-derived trees such as compiled NIB view hierarchies, and the guard narrows with a typed record check instead of any or unchecked casts. The non-recursive JSON Schema advertisement stays intact on both the public and standard projection paths. Two payload sizes get explicit budgets tied to their producer constraints: web text artifacts reject beyond the 16 MiB source budget the web tooling already advertises, and native UI screenshot base64 beyond the 64 MiB helper output budget. Regenerated product catalog, skill catalog digest, and completion-ledger manifest follow the schema-boundary change.

* fix(validation): preserve depth diagnostics across process inputs

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 21:12:10 +08:00
ASVLCIIandN0zoM1z0 3dcb732da3 fix(cli): preserve blank JSON paths for input validation (#1068)
* fix(cli): preserve blank JSON paths for input validation

* test(cli): avoid unchecked path field assertions

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:57:54 +08:00
cevieirasandN0zoM1z0 6a7650e93c fix(mcp): flatten root union input schemas (#1062)
* fix(mcp): flatten root union input schemas

* fix(mcp): preserve minimum input requirements in object schemas

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:54:51 +08:00
Rudy Mizrahi Celekli 170600541b fix: select Node legacy package entry fields for import and require (#1035)
fix(javascript): use Node legacy package entry fields

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
2026-10-08 20:47:01 +08:00
morluto 2d6f1cd23b fix(native): recognize forbidden extended attributes as invalid signing 2026-10-08 20:39:55 +08:00
Arian PashaeandN0zoM1z0 68b9fa489b fix(install): guard empty prefix_args expansion on macOS Bash 3.2 under set -u (#1061)
* fix(install): guard empty prefix_args expansion under Bash 3.2 set -u (#1060)

* test(install): cover native Bash macOS installation

---------

Co-authored-by: N0zoM1z0 <1705004398@qq.com>
2026-10-08 20:39:39 +08:00
morluto 1a5eec54e1 fix(native): preserve verification captures and separate path fragments 2026-10-08 20:35:31 +08:00
morluto fa68ca054c chore: merge main into feat/signature-posture 2026-10-08 19:59:45 +08:00
morluto bd2655e6c5 fix(native): preserve definitive failures alongside operational diagnostics 2026-10-08 19:52:21 +08:00
morluto 38e2a342d1 refactor(native): bound nested path checks and verify real ticket coverage 2026-10-08 19:35:56 +08:00
morluto 670b657d0e fix(native): keep path bytes out of signing reason classification 2026-10-08 18:46:13 +08:00
Ysopwnable 8a31508c89 Merge pull request #1064 from morluto/fix/release-checkpoint-guards
fix(release): validate frozen checkpoint versions and breaking notes
2026-10-08 18:31:45 +08:00
N0zoM1z0 f5009ed55f fix(release): recognize conventional titles in GitHub merge commits 2026-10-08 10:21:02 +00:00
morluto f1f2aa3d86 refactor(native): own signing policy and bind capture target versions 2026-10-08 18:14:58 +08:00
N0zoM1z0 60825fb447 fix(release): validate frozen checkpoint versions and breaking notes
Audit actual first-parent ancestry instead of trusting the release bot
chronological cutoff, and block mismatched versions before tag creation.
Sync the published 6.0.0 baseline without dropping newer main changes.

Fixes #1063

Assisted-by: Sakuya (gensokyo-skills:sakuya-checkpointed-execution)
2026-10-08 10:13:05 +00:00
N0zoM1z0 5dfd9905ed chore(release): sync 6.0.0 checkpoint metadata 2026-10-08 09:49:53 +00:00
morluto c202ee9479 refactor(native): isolate signature workflow and evidence classification 2026-10-08 17:43:48 +08:00
Kaoru0822andClaude Opus 5.5 5ccb9241be perf(mcp): advertise each tool JSON Schema once per target (#1059)
The MCP SDK reconverts every registered Zod schema on each tools/list.
With 139 tools that is ~45 ms of Zod JSON Schema work per listing, for a
catalog that cannot change after registration. Memoize the advertised
input and output projections per target on the registration clones.

Listings are byte-identical; repeated tools/list drops from ~62 ms to
~18 ms on the stdio server.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:31:08 +08:00
morluto d93f8209d4 fix(native): centralize slice knowledge and validate local ticket framing 2026-10-08 17:20:41 +08:00
morluto e45865f347 chore: merge main into feat/signature-posture 2026-10-08 16:59:55 +08:00
morluto 6a5cedfec4 fix(native): compare signed slice posture and reject ticket mutation 2026-10-08 16:59:08 +08:00
morluto d13143b5c5 docs: regenerate catalog and skill metadata 2026-10-08 16:43:33 +08:00
morluto cc706f347f fix(native): unknown verification on I/O failure, recheck ticket path, keep CR paths 2026-10-08 16:43:03 +08:00
morluto 96c2a93755 fix(native): probe every slice of universal binaries before aggregate signing state 2026-10-08 16:31:19 +08:00
morluto 1652c73902 fix(native): separate main-signature facets and I/O diagnostics from invalid 2026-10-08 16:12:06 +08:00
morluto e3c15c8bbc chore: merge main into feat/signature-posture 2026-10-08 15:49:33 +08:00
morluto cc78c18c47 fix(native): preserve whitespace in nested paths and detect ticket replacement 2026-10-08 15:45:34 +08:00
morluto 8f91add240 fix(native): harden signature posture for wrapped bundles, tickets, and invalid signatures 2026-10-08 15:24:29 +08:00
Kaoru0822andClaude Opus 5.5 33def5ae06 chore: merge main into feat/signature-posture
main now carries #1037 at skill version 30, so this branch moves to 31.
Regenerate the product catalog, conformance manifest, skill digest and
package metadata on top of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:28:32 +00:00
Kaoru0822andClaude Opus 5.5 48813a7de9 chore: merge main into feat/signature-posture
main now carries #964 at skill version 29, so this branch moves to 30.
Regenerate the product catalog, conformance manifest, skill digest and
package metadata on top of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:20:14 +00:00
Kaoru0822andClaude Opus 5.5 ea88d301f8 chore: merge main into feat/signature-posture
Resolve generated-metadata conflicts by regenerating them on top of
main. The skill references gain the inspect_signature paragraph, so the
skill version moves to 29 (main already uses 28). The MCP tool catalog
is now built rather than committed (#997), so it is removed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 04:43:20 +00:00
Kaoru0822andClaude Opus 5.5 a427ddf797 Merge origin/main into feat/signature-posture
Follow #973's native ownership: the code-signing posture schemas move to
src/domain/native/ beside nativeInspection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:48:16 +00:00
Kaoru0822 423a89e7e4 Merge remote-tracking branch 'origin/main' into feat/signature-posture
# Conflicts:
#	docs/product-catalog.json
#	docs/verification/managed-conformance-manifest.json
#	skills/reverse-engineer-anything/SKILL.md
#	src/generatedMcpToolCatalog.ts
2026-10-07 14:14:16 +00:00
Kaoru0822andClaude Opus 5.5 57e91187cf Merge origin/main into feat/signature-posture
Follow #952's layout: the code-signature lane moves to
scripts/verify/apple/code-signature.mjs and uses the relocated bundle
fixture builder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:37:22 +00:00
Kaoru0822andClaude Opus 5.5 949ab427fb fix(native): flag nested-code paths that codesign split on newlines
codesign prints one nested path per line, so a nested component whose name
contains a newline is split into fragments. inspect_signature now checks
that every --validated: path it reports exists, and names any that does not
in a limitation instead of trusting it as a complete path. The result stays
deterministic: codesign's progress lines arrive in no fixed order, so the raw
output is not echoed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:05:23 +00:00
Kaoru0822 4f46836178 Merge remote-tracking branch 'origin/main' into feat/signature-posture
# Conflicts:
#	docs/product-catalog.json
#	docs/verification/managed-conformance-manifest.json
#	skills/reverse-engineer-anything/SKILL.md
#	src/generatedMcpToolCatalog.ts
2026-10-07 13:01:47 +00:00
Kaoru0822andClaude Opus 5.5 ce8e969112 fix(native): require Apple origin for platform facets and keep cancellation
- Treat a CodeDirectory platform identifier as a platform binary only when
  the signature satisfies `anchor apple`, checked with
  `codesign --verify -R="anchor apple"`; otherwise the dependent facets are
  unknown. Developer ID code chains to Apple Root CA but fails this check.
- Report every security facet as unknown when architecture slices differ in
  signing state.
- Map an aborted signal that escapes a native helper, such as ticket
  hashing, to AnalysisCancelledError instead of an output error.
- Treat a target as an app bundle only when it was opened from a bundle
  directory, so a regular file named X.app is verified as a bare Mach-O.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 12:43:47 +00:00
Kaoru0822 73eb20a4b9 Merge remote-tracking branch 'origin/main' into feat/signature-posture
# Conflicts:
#	docs/product-catalog.json
#	docs/verification/managed-conformance-manifest.json
#	skills/reverse-engineer-anything/SKILL.md
#	src/generatedMcpToolCatalog.ts
2026-10-07 12:37:43 +00:00
Kaoru0822andClaude Opus 5.5 94243fa5a1 refactor(native): extract signature verification from inspect_signature
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:55:05 +00:00
Kaoru0822andClaude Opus 5.5 37824e8c46 fix(native): verify nested code and keep ticket read failures local
Address review findings in inspect_signature:
- Verify with `codesign --verify --deep --strict`. Shallow verification
  misses a file added inside a nested XPC service, or a modified dylib
  inside a framework. verify:code-signature now tampers inside the
  nested XPC service and requires `invalid`.
- Classify verification as `unsigned` only when the top-level target is
  unsigned. Unsigned nested code inside a signed bundle is `invalid`.
- Report a stapled ticket that exists but cannot be read as
  `unreadable`, with the error code in a new `reason` field, instead of
  failing the whole signature inspection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:54:32 +00:00
Kaoru0822andClaude Opus 5.5 25ca63f231 feat(native): report code-signing posture in inspect_signature
inspect_signature reduced the CodeDirectory to a hardened-runtime boolean
and never verified the signature. It now also reports:

- the parsed CodeDirectory: version, flags decoded from their numeric
  value (runtime, library-validation, restrict, kill, adhoc,
  linker-signed, unknown bits in hex), code and special slot counts,
  location, hash type, platform identifier, runtime version, and
  executable-segment flags;
- sealed resources (version, rules, files, or none);
- a local `codesign --verify --strict` result on the opened bundle or
  Mach-O, with diagnostics and the sorted nested code it validated;
- whether a notarization ticket is stapled at Contents/CodeResources,
  with its digest. The ticket and Apple's records are not checked
  because that needs the network;
- derived security facets with their supporting evidence: library
  validation, DYLD_* handling, debugger attach, executable-memory
  policy, and App Sandbox. Facets stay unknown when flags or aggregate
  entitlements are unavailable.

verify:code-signature re-signs copies of the clang-built fixture with
real codesign (ad hoc, hardened runtime, hardened runtime with relaxing
entitlements, library,restrict). It also checks a platform binary and a
tampered bundle, with CLI/MCP parity, and runs in the macOS CI job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:16:55 +00:00
205 changed files with 11234 additions and 2095 deletions

No files matched your search

+5 -1
View File
@@ -28,7 +28,7 @@ intentionally not included. -->
- Provider, bridge, target-format, or platform compatibility:
- Evidence, artifact, provenance, or reconstruction contract:
- Process execution, authorization, cleanup, or containment impact:
- Generated metadata (`docs/product-catalog.json`), package, or installation impact (the regen-generated bot pushes tracked generated files; never hand-edit them):
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `skill-src/` and never commit build outputs):
## Evidence and regression coverage
@@ -59,6 +59,10 @@ relevant real-provider or platform workflow, or explain why it was unavailable.
limits, security/privacy/containment implications, and meaningful proof gaps. -->
- Breaking changes or migration steps:
<!-- For ! / BREAKING CHANGE: show a previously valid call/configuration,
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Compatible fixes/additions should use patch/minor semantics.
See docs/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
+22 -4
View File
@@ -102,8 +102,8 @@ jobs:
with:
name: rea-dist
path: dist
- name: Generate source catalog from downloaded build
run: node scripts/generate-mcp-tool-catalog.mjs
- name: Generate projections from downloaded build
run: npm run build:projections
- run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
@@ -317,8 +317,9 @@ jobs:
- run: npm run verify:keyed-archive
- run: npm run verify:asset-catalog
- run: npm run verify:macos-bundle
- name: Native compiled NIB hierarchy regressions
run: npm run test:focused -- tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
- run: npm run verify:code-signature
- name: Native installer and compiled NIB hierarchy regressions
run: npm run test:focused -- tests/boundary/filesystem/installScript.test.ts tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
- name: Real binary metadata and Interface Builder integration
run: npm run verify:apple-dispatch && npm run verify:interface-builder
- name: Native plist CLI selection regressions
@@ -327,6 +328,8 @@ jobs:
run: npm run test:focused -- tests/boundary/cli/nativeDemangleCli.test.ts tests/boundary/mcp/nativeDemangleMcp.test.ts
- name: Process filesystem snapshot descriptor regressions
run: npm run test:focused -- src/process/capture/FilesystemSnapshot.test.ts
- name: Native Darwin ownership and process cleanup regressions
run: npm run test:focused -- tests/boundary/process/darwinProcessRunTokenReader.test.ts tests/boundary/process/processCaptureLifecycle.test.ts tests/boundary/cli/processCli.test.ts tests/boundary/mcp/sessionMcp.test.ts
reconstruction-readiness:
name: Reconstruction readiness
@@ -441,6 +444,11 @@ jobs:
tests/boundary/filesystem/safeOutputTree.test.ts
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
tests/boundary/filesystem/setupUnavailableProvider.test.ts
tests/boundary/filesystem/windowsNpxRegistration.test.ts
src/application/ClientRegistrationIdentity.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
tests/boundary/cli/referenceSourceImportStatus.test.ts
src/cliReferenceSourceImportStatus.test.ts
tests/boundary/filesystem/asarInventoryFreshness.test.ts
@@ -481,3 +489,13 @@ jobs:
cache: npm
- run: npm ci
- run: npm run docs:check
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@v4
with:
name: generated-docs
path: |
docs/public/product-catalog.json
docs/verification/managed-conformance-*.json
skills/reverse-engineer-anything
if-no-files-found: error
-75
View File
@@ -1,75 +0,0 @@
name: Regenerate generated metadata
on:
pull_request:
branches: [main]
paths:
- "src/**"
- "skills/**/SKILL.md"
- "scripts/generate-*.mjs"
- "scripts/check-doc-facts.mjs"
- "scripts/lib/*.mjs"
- "scripts/verify/managed/**"
- "scripts/fixtures/managed/**"
- "package.json"
- "package-lock.json"
concurrency:
group: regen-${{ github.event.pull_request.number }}
cancel-in-progress: true
env:
HUSKY: "0"
permissions:
contents: write
jobs:
regen:
name: Regenerate and push tracked generated files
# Same-repo PRs only: the default token cannot push to forks.
# The bot-actor guard stops looped runs triggered by our own push.
if: >-
github.actor != 'github-actions[bot]' &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.ref }}
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: npm run docs:generate
- name: Commit regenerated files
shell: bash
env:
HEAD_REF: ${{ github.event.pull_request.head.ref }}
run: |
generated_paths=(
docs/product-catalog.json
docs/error-contract.schema.json
docs/verification/managed-conformance-manifest.json
docs/verification/managed-conformance-ledger.json
skills/reverse-engineer-anything/SKILL.md
src/generatedPackageMetadata.ts
)
if git diff --quiet -- "${generated_paths[@]}"; then
echo "Generated metadata is already canonical."
exit 0
fi
git diff --check
# Refuse to sweep unrelated working-tree changes into the bot commit.
unexpected="$(git diff --name-only | grep -vxF -f <(printf '%s\n' "${generated_paths[@]}") || true)"
if [[ -n "${unexpected}" ]]; then
echo "docs:generate left unexpected tracked changes:" >&2
printf '%s\n' "${unexpected}" >&2
exit 1
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -- "${generated_paths[@]}"
git commit -m "chore: regenerate generated metadata"
git push origin "HEAD:${HEAD_REF}"
+39 -36
View File
@@ -84,6 +84,32 @@ jobs:
exit 1
fi
- name: Check out release controller and Git history
uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
fetch-depth: 0
persist-credentials: false
- name: Set up Node.js for checkpoint validation
uses: actions/setup-node@v6
with:
node-version-file: .nvmrc
package-manager-cache: false
- name: Install checkpoint validation dependencies
run: npm ci
- name: Validate checkpoint version and ancestry before release creation
env:
RELEASE_PHASE: ${{ inputs.phase }}
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
run: >-
node scripts/verify-release-checkpoint.mjs
--phase "$RELEASE_PHASE" --release-branch "$RELEASE_BRANCH"
--source-sha "$SOURCE_SHA"
- uses: googleapis/release-please-action@v4
id: release
with:
@@ -116,6 +142,7 @@ jobs:
uses: actions/checkout@v6
with:
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
fetch-depth: 0
persist-credentials: false
- name: Set up Node.js for generated documentation
@@ -129,47 +156,23 @@ jobs:
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm ci
- name: Validate prepared release version and migration notes
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
env:
RELEASE_BRANCH: ${{ inputs.release_branch }}
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
run: >-
node scripts/verify-release-checkpoint.mjs
--phase prepare --stage candidate --release-branch "$RELEASE_BRANCH"
--source-sha "$SOURCE_SHA"
- name: Regenerate release documentation
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm run docs:generate
- name: Commit canonical release catalog
- name: Validate generated release documentation
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
shell: bash
env:
# Use the same identity for the follow-up commit so it also triggers CI.
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
RELEASE_PR_JSON: ${{ steps.release.outputs.pr }}
run: |
release_pr_branch="$(node --input-type=module -e '
const value = JSON.parse(process.env.RELEASE_PR_JSON ?? "");
if (
typeof value !== "object" ||
value === null ||
typeof value.headBranchName !== "string" ||
value.headBranchName.length === 0
) {
process.exit(1);
}
process.stdout.write(value.headBranchName);
')"
changed_paths="$(git diff --name-only)"
if [[ -z "${changed_paths}" ]]; then
echo "Release documentation is already canonical."
exit 0
fi
if [[ "${changed_paths}" != "docs/product-catalog.json" ]]; then
echo "docs:generate changed unexpected tracked paths:" >&2
printf '%s\n' "${changed_paths}" >&2
exit 1
fi
git diff --check
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -- docs/product-catalog.json
git commit -m "chore: normalize release product catalog"
gh auth setup-git
git push origin "HEAD:${release_pr_branch}"
run: npm run docs:check && git diff --exit-code
publish:
name: Publish to npm
+8
View File
@@ -38,6 +38,7 @@ test-results.xml
.turbo/
.cache/
docs/.vitepress/cache/
docs/.vitepress/dist/
# OS
.DS_Store
@@ -51,3 +52,10 @@ Thumbs.db
# Build-generated MCP tool catalog (regenerated by `npm run build`; never commit)
src/generatedMcpToolCatalog.ts
# Derived catalogs, portable conformance projections, and packaged skill bundles
# (regenerated from this checkout by `npm run build:cached`; never commit)
/skills/
/docs/public/product-catalog.json
/docs/verification/managed-conformance-manifest.json
/docs/verification/managed-conformance-ledger.json
+9 -7
View File
@@ -21,10 +21,10 @@ REA is a layered ESM TypeScript application. Dependencies flow inward from pure
- `src/domain/` owns pure provider-neutral semantics; `src/contracts/` owns caller-visible schemas and the canonical tool inventory.
- `src/hopper/`, `src/ghidra/`, `src/ida/`, `src/browser/`, `src/inspector/`, `src/native/`, `src/artifacts/`, and `src/dotnet/` own provider-specific boundaries. Keep provider protocols out of domain and application code.
- `src/application/` composes shared CLI/MCP workflows; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. `bridge/` contains provider-side adapters.
- `src/application/` composes shared CLI/MCP workflows, including Evidence provenance, unknowns, and eligible snapshot bindings; `src/server/` translates MCP requests; `src/cli.ts` and `src/main.ts` are the CLI and MCP entry points.
- `src/process/` owns shared process lifecycle primitives, not provider wire protocols. Reuse its supervision and identity primitives before adding provider-local lifecycle code; a PID and executable pathname alone do not establish ownership after exit or reuse. `bridge/` contains provider-side adapters.
- `tests/` contains unit, composition, boundary, acceptance, and conformance tests. `scripts/verify-*` and capability directories under `scripts/verify/` contain real-toolchain checks.
- `docs/product-catalog.json` is generated. Update its source contracts and regenerate it; do not edit it directly.
- `docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored build outputs. Update source contracts and authored instructions in `skill-src/`, then run `npm run build:cached`; never commit derived catalog digests or portable conformance projections.
- `src/generatedMcpToolCatalog.ts` is build-generated and gitignored. Never commit it; resolve any trace of it in merges by deleting it and running `npm run build:cached`.
## Build, Test, and Development Commands
@@ -36,7 +36,7 @@ REA is a layered ESM TypeScript application. Dependencies flow inward from pure
- `npm run check:changed`: run cached static checks and source tests affected since the branch merge base (default `origin/main`).
- `npm run check:fast`: run cached typecheck and lint checks.
- `npm run check:pr`: opt into the complete local deterministic gate and generated-document checks for broad changes; CI owns full coverage. Routine iterations need focused tests and relevant checks, not the whole gate each time.
- `npm run docs:check`: check committed generated documents; `npm run docs:generate` regenerates them.
- `npm run docs:check`: build and validate generated documents for the current checkout; `npm run docs:generate` regenerates them. CI retains ignored outputs as artifacts and does not push snapshot commits to feature branches.
- For provider-dependent changes, see [docs/testing.md](docs/testing.md) and run the matching real-provider verification.
- Keep each verification lane's prerequisites limited to the claim it checks. Use host-native fixtures for host/provider acceptance; put optional cross-target formats and their external toolchains in a separate lane. Preflight required commands and report the missing dependency and lane clearly.
@@ -54,9 +54,11 @@ Use ESM TypeScript, two-space indentation, and the committed Oxfmt configuration
Treat a boundary as a contract between the producer's actual representation and the consumer's required meaning. When implementing or auditing a boundary, trace the value through parsing, normalization, authorization, serialization, and the CLI/MCP result. Establish affected callers from their code paths; similar tool names or workflows do not prove that they share a schema or failure mode.
Keep portable evidence and scenario validation distinct from host-native execution checks. Absolute filesystem paths, file URLs, and HTTP paths have different semantics; do not substitute one platform's syntax for the domain concept. Interpret provider metadata according to its documented or observed producer behavior. When a transformation loses information, preserve the reported value and an explicit unknown rather than guessing a canonical identity.
When changing boundary behavior, inspect adjacent input representations, failure paths, and affected callers, and correct the underlying assumption across those cases.
Preserve meaningful failure reasons through application and adapter layers. Malformed input is distinct from an unsupported target, unavailable provider, or host operating-system permission denial. Diagnostics should identify the failed constraint and the target or lifecycle request it applies to. Recovery advice must address that reason and point to an available workflow; generic catches must not erase actionable validation details.
Keep portable evidence and scenario validation distinct from host-native execution checks. Absolute filesystem paths, file URLs, and HTTP paths have different semantics; do not substitute one platform's syntax for the domain concept. Interpret provider metadata according to its documented or observed producer behavior. Keep values used for identity, provenance, matching, and path resolution separate from display formatting. When normalization loses information, preserve the source value and an explicit unknown; display placeholders must not feed back into lookup or selection.
Preserve meaningful failure reasons through application and adapter layers. Malformed input is distinct from an unsupported target, unavailable provider, or host operating-system permission denial. Diagnostics should identify the failed constraint and the target or lifecycle request it applies to. Recovery advice must address that reason and point to an available workflow; generic catches must not erase actionable validation details. Preserve collected observations on execution failure independently of whether cleanup succeeds.
Leave meaningful target, action, capture, and output choices to the agent. A selected operation already expresses intent; do not require approval booleans or repeated permission declarations. Trace each setting to its consumer: remove ignored options and single-value confirmations, derive built-in lifecycle behavior, and supply defaults for omitted optional metadata. Report actual effects and limitations where they help interpret results rather than asking callers to restate them.
@@ -80,7 +82,7 @@ Let agents compose experiments with ordinary commands, scripts, and local fixtur
- Use **observe/capture** only when runtime activity is required, and declare authority and lifecycle effects in the contract.
- Extend an existing tool when intent and result contract are unchanged. Add a tool for a distinct analyst outcome or materially different authority.
- Keep caller-facing names and results provider-neutral. Put engine-specific behavior in provider adapters and report each provider's exact coverage.
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Keep observed, derived, inferred, and unknown results distinct.
- Keep results complete by default. Add a limit only when it follows from a real format, protocol, authority, or resource-safety constraint; explain truncation and unsupported facets. Account for representation expansion and products of independently bounded dimensions before allocating or retaining output. Derive facet completeness from what was examined and exhausted, not an empty failure list. Keep observed, derived, inferred, and unknown results distinct.
- Include artifact identity, source locations, Evidence references, actionable errors, and relevant limitations when they affect conclusions. Return the evidence needed for the next analysis inline rather than requiring a resource or identifier lookup.
- Implement shared application workflows behind CLI and MCP adapters. Update canonical contracts and generated catalog together.
+19 -3
View File
@@ -35,7 +35,7 @@ links, and `npm run docs:preview` to preview that build at `/rea/`.
Site navigation lives in `docs/.vitepress/config.ts`. Keep links to guides
relative so they work on GitHub and the website; link to repository files
outside `docs/` using their full GitHub URLs. Generated reference documents
still use `npm run docs:generate`; `docs:build` only builds the website.
use `npm run docs:generate`; `docs:build` generates them before building the website.
Pull requests run `npm run docs:check`. `.github/workflows/pages.yml` is a
manual VitePress build and does not publish. The public site at
@@ -74,8 +74,24 @@ require the matching real-provider `verify:*` lane.
Formatting uses Oxfmt and the committed `.oxfmtrc.json`; generated sources use
the same configuration. Pre-commit formats and lints staged files; pre-push runs
`check:fast`.
`docs:check` checks committed generated metadata. `docs:generate` regenerates
those files, and the docs CI lane checks them.
`docs:check` builds and validates generated metadata for the current checkout.
The product catalog (`docs/public/product-catalog.json`), portable managed
conformance projections (`docs/verification/managed-conformance-*.json`), and
packaged skill (`skills/`) are ignored build outputs. Edit skill instructions
and references in `skill-src/`; the build adds catalog-dependent metadata to
the packaged copy without rewriting authored files. The generated manifest
commits to that exact packaged skill bundle. It is a portable projection of
the deterministic managed verifier, not a record of optional real-provider runs.
CI validates these outputs and retains them as artifacts instead of pushing
generated-only commits onto feature branches. Reviewed source metadata such as
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json` remains
tracked and checked for freshness. Do not commit ignored generated outputs.
The build-generated product catalog contains documented facts and their provider
identity, rather than full runtime schema hashes. The managed skill contains
instructions and inventory metadata; doctor compares its installed files with
the canonical bundle. Schema-only fixes should not change these outputs or the
skill commitment in the conformance manifest. Runtime schema identity remains
available through doctor and `binary_session`.
Real-provider execution remains uncached; deterministic builds use Turbo.
Local `npm test` runs every deterministic Vitest project without coverage or
+10 -6
View File
@@ -10,7 +10,7 @@
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![CI](https://img.shields.io/github/actions/workflow/status/morluto/rea/ci.yml?branch=main&style=flat-square&label=CI)](https://github.com/morluto/rea/actions/workflows/ci.yml)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/product-catalog.json)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![Node.js 22+](https://img.shields.io/badge/Node.js-22.19%2B-339933?style=flat-square&logo=nodedotjs&logoColor=white)](https://nodejs.org/)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -296,7 +296,7 @@ For exact options, prerequisites and result contracts:
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/product-catalog.json): generated inventory of tools, providers and CLI commands.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): build-generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -315,12 +315,12 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[Website](https://morluto.github.io/rea/) · [npm](https://www.npmjs.com/package/rea-agents) · [skills.sh](https://skills.sh/morluto/rea/reverse-engineer-anything) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md)
## License
[MIT](LICENSE)
## Star history
🎉 **20,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, testing builds, and contributing fixes.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
@@ -332,3 +332,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
## Disclaimer
REA provides tools for lawful reverse-engineering research, analysis, and reconstruction. You are responsible for obtaining any required authorization and complying with applicable laws. The project does not endorse illegal or unauthorized use.
## License
[MIT](LICENSE)
+14
View File
@@ -336,6 +336,20 @@ npx -y rea-agents@latest compare /absolute/path/to/left-evidence.json /absolute/
- [Hopper Disassembler](https://www.hopperapp.com/)
- [سياسة الأمان](SECURITY.md)
## تاريخ النجوم
🎉 **وصلنا إلى 20,000 نجمة على GitHub، شكرًا لكم!**
شكرًا لكل من يستخدم REA ويبلّغ عن الأخطاء ويختبر الإصدارات ويساهم في إصلاحها.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="تاريخ نجوم REA على GitHub" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## الترخيص
[MIT](LICENSE)
+14
View File
@@ -366,6 +366,20 @@ REA にホスト型の解析サービスはありません。現在のプロバ
開発環境、アーキテクチャ、テスト、リリース手順は [CONTRIBUTING.md](CONTRIBUTING.md) を参照してください。
## スター履歴
🎉 **GitHub スターが 20,000 を突破しました。ありがとうございます!**
REA を使い、問題を報告し、ビルドをテストし、修正を届けてくださる皆さんに感謝します。
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA の GitHub スター履歴" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## ライセンス
[MIT](LICENSE)
+14
View File
@@ -315,6 +315,20 @@ REA에는 호스팅 분석 서비스가 없습니다. 로컬 Unix 소켓을 통
개발 환경, 아키텍처, 테스트, 릴리스 지침은 [CONTRIBUTING.md](CONTRIBUTING.md)를 참고하세요.
## 스타 기록
🎉 **GitHub 스타 20,000개, 감사합니다!**
REA를 사용하고, 버그를 제보하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA GitHub 스타 기록" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## 라이선스
[MIT](LICENSE)
+14
View File
@@ -316,6 +316,20 @@ REA 不提供托管分析服务,而是通过本地 Unix 套接字把操作交
开发环境、架构、测试和发布说明请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。
## Star 历史
🎉 **GitHub Star 突破 20,000,感谢大家!**
感谢每一位使用 REA、反馈问题、测试构建和贡献修复的朋友。
<a href="https://www.star-history.com/?repos=morluto%2Frea&amp;type=date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date&amp;theme=dark&amp;legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
<img alt="REA GitHub Star 历史" src="https://api.star-history.com/chart?repos=morluto/rea&amp;type=date" />
</picture>
</a>
## 许可证
[MIT](LICENSE)
-2
View File
@@ -133,8 +133,6 @@ def project_record(state, secrets):
for key, item in value.items():
if not isinstance(key, str):
raise CaptureFailure("format", "Native record has a non-string dictionary key.", path)
if key == "__proto__":
raise CaptureFailure("unsupported", "Native object contains a __proto__ member that the current JSON schema boundary cannot preserve.", path)
if any(secret in key for secret in secrets):
# Validate the omitted subtree too, then discard all of its
# coordinates rather than inventing a replacement identity.
+31 -1
View File
@@ -112,6 +112,31 @@ enum RunTokenReadError: Error {
case ambiguousEnvironmentBoundary
}
@_silgen_name("csops")
private func csops(
_ pid: Int32,
_ ops: UInt32,
_ addr: UnsafeMutableRawPointer?,
_ size: Int
) -> Int32
private let csOpsStatus: UInt32 = 0
private let csPlatformBinary: UInt32 = 0x0400_0000
/**
* Whether the target image is an Apple platform binary. Current macOS omits
* the environment of platform binaries from KERN_PROCARGS2 for every caller,
* so this distinguishes a policy-withheld environment from a process whose
* environment is genuinely absent or transiently unreadable.
*/
func isPlatformBinary(pid: Int32) -> Bool {
var flags: UInt32 = 0
let result = withUnsafeMutablePointer(to: &flags) { pointer in
csops(pid, csOpsStatus, UnsafeMutableRawPointer(pointer), MemoryLayout<UInt32>.size)
}
return result == 0 && (flags & csPlatformBinary) != 0
}
private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
var mib = [CTL_KERN, KERN_PROCARGS2, Int32(pid)]
var size = 0
@@ -133,7 +158,12 @@ private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
} catch RunTokenReadError.invalidTokenEncoding {
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
} catch RunTokenReadError.environmentUnavailable {
return unavailable(pid: pid, reason: "environment_unavailable")
return unavailable(
pid: pid,
reason: isPlatformBinary(pid: pid)
? "platform_binary_environment_withheld"
: "environment_unavailable"
)
} catch RunTokenReadError.appleVectorUnavailable {
return unavailable(pid: pid, reason: "apple_vector_unavailable")
} catch RunTokenReadError.ambiguousEnvironmentBoundary {
+1 -1
View File
@@ -27,7 +27,7 @@ REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
REA never installs Python, packages or GDB, changes a user init file, launches
the selected object as a host process, or requests runtime library resolution. The Python process
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
[upstream provenance](../third_party/pwntools/README.md).
[upstream provenance](https://github.com/morluto/rea/blob/main/third_party/pwntools/README.md).
## Interpreting results
+1 -1
View File
@@ -24,7 +24,7 @@ their distinct unavailable-provider or capability classification.
The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
`REA_EVM_PRLIMIT_COMMAND` can select another absolute executable. REA installs
no system tools and changes no user configuration. [Upstream provenance and
license](../third_party/evmole/README.md) are packaged.
license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) are packaged.
## Evidence semantics
+1 -1
View File
@@ -95,7 +95,7 @@ npx skills add morluto/rea --skill reverse-engineer-anything
This installs agent instructions and bundled references, not REA MCP
registration or analysis engines. Follow the skill's
[conditional connection guide](https://github.com/morluto/rea/blob/main/skills/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
[conditional connection guide](https://github.com/morluto/rea/blob/main/skill-src/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
Working tools can be used immediately. If tools are missing, inspect the current
client's registration with `doctor --client codex --json` (substitute its client
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
+12 -7
View File
@@ -100,17 +100,22 @@ a temporary output file larger than the running engine's string limit, verifies
its bytes and digest, and removes it. Use `-- jsonl` for the compact JSONL check.
Each check needs space for one output file plus a 1 GiB free-space reserve.
Other CLI formats, `--token-count`, and MCP transport serialization still
assemble whole strings. Field selection remains useful when the caller needs a
Other CLI formats and `--token-count` still assemble whole strings.
Field selection remains useful when the caller needs a
smaller view, for example `--format json --filter-output
evidence_id,normalized_result.statistics`. Streaming output does not bound the
memory needed to construct the analysis graph itself.
The separate large-response transport work is tracked in
[#1053](https://github.com/morluto/rea/issues/1053) and
MCP prepares the complete repeated response incrementally against the pinned
SDK's 10 MiB stdio receive-buffer budget. Oversized results return an actionable
transport constraint and the exact same-session Evidence reference. Use
`trace_application_feature` to inspect a selected module's relationships, or
`export_evidence_bundle` to write the complete canonical bundle without a
document-sized allocation. Same-session analysis reads reuse authenticated
immutable snapshots; foreign inline Evidence is still parsed and authenticated.
See [MCP tool results](mcp-contracts.md#tool-results) for larger client buffers
and `REA_MCP_MAX_RESPONSE_BYTES`. Follow-up results remain complete and can also
exceed the transport budget. Compact result views are tracked separately in
[#1050](https://github.com/morluto/rea/issues/1050).
Client framing limits also apply: the pinned Node MCP SDK's stdio transport
defaults to a 10 MiB buffer. Its caller-selected `maxBufferSize` must accommodate
the complete response, including text and structured Evidence projections.
## What is reconstructed
+1 -1
View File
@@ -25,7 +25,7 @@ part of the current tool set.
This guide describes the implementation and verification of
[ADR-0003](adr/0003-managed-code-evidence-and-provider-boundary.md). The canonical
tool inventory is [`product-catalog.json`](https://github.com/morluto/rea/blob/main/docs/product-catalog.json).
tool inventory is the [build-generated catalog](mcp-contracts.md#generated-catalog).
## Shipped scope
+30
View File
@@ -1,5 +1,15 @@
# MCP runtime contracts
## Generated catalog
Run `npm run build:cached` in a source checkout to generate the machine-readable
catalog at `docs/public/product-catalog.json`. The [catalog](product-catalog.json)
is also included in documentation builds. PR CI retains it with the packaged
skill and portable conformance projections in the `generated-docs` artifact.
These outputs describe the exact source revision being built; they are not
checked-in snapshots. For a running server, `binary_session` remains the
authoritative source of catalog identity and tool availability.
## Identity and discovery
`binary_session` reports the active package, server, SDK, and negotiated
@@ -154,6 +164,26 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
retained session history or an explicit bundle for transfer.
REA prepares complete MCP results within the pinned stdio client's 10 MiB
receive-buffer budget, including both text and structured representations and
room for the JSON-RPC envelope. If a result cannot fit, REA returns
`resource_constraint` with `details.resource: "transport"` before constructing
a document-sized string. Analysis Evidence remains complete in the current
session. Its exact reference is reported in
`details.reported_limits.evidence_reference`; use it with a focused application
workflow, or call `export_evidence_bundle` with a destination path. Complete
bundle exports stream canonical JSON into an atomically published file. A broad
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
preserves the complete session without sending it through a single MCP frame.
Clients that explicitly configure a larger receive buffer can set the REA
server's `REA_MCP_MAX_RESPONSE_BYTES` environment variable to the same byte
count. This setting must be a safe decimal integer at least 10485760; REA
reserves 1024 bytes for the envelope. Raising it restores complete inline
delivery for responses that fit that buffer and Node's single-string limit.
It does not change the client's buffer, analysis coverage, or retained content.
Ordinary responses keep their existing complete result contract.
## Retained application Evidence inputs
`trace_application_feature`, `trace_javascript_semantics`,
+6
View File
@@ -102,6 +102,12 @@ run the same owned-process cleanup path. Settlement reports whether the
sampled process group quiesced or whether cleanup was needed or unverifiable;
sampling cannot prove that every short-lived or detached descendant was seen.
When the host withholds an unrelated process’s ownership token, REA leaves that
process untouched and records its PID and reason in `cleanup.unverified_processes`
and process residual unknowns. Successful cleanup verifies the owned group;
it does not attribute those unrelated processes. Related or otherwise unexplained
unreadable processes still prevent successful cleanup.
## Compare two captures
Compare saved capture Evidence with:
-722
View File
@@ -1,722 +0,0 @@
{
"package": {
"name": "rea-agents",
"version": "6.0.0",
"sdk": {
"server": "2.3.1",
"client": "2.3.1",
"core": "2.3.1"
},
"skill_version": "33"
},
"tools": {
"total": 139,
"families": [
{
"id": "direct",
"surface": "official-proxy",
"count": 41,
"tools": [
"address_name",
"address_to_file_offset",
"annotate_native_function",
"comment",
"current_address",
"current_document",
"current_procedure",
"goto_address",
"inline_comment",
"inspect_native_data_type",
"inspect_native_instruction",
"inspect_native_load_image",
"list_bookmarks",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"next_address",
"prev_address",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"resolve_native_call_targets",
"search_procedures",
"search_strings",
"set_address_name",
"set_addresses_names",
"set_bookmark",
"set_comment",
"set_current_document",
"set_inline_comment",
"unset_bookmark",
"xrefs"
]
},
{
"id": "enhanced",
"surface": "enhanced",
"count": 14,
"tools": [
"analyze_function",
"analyze_swift_types",
"batch_decompile",
"binary_overview",
"find_xrefs_to_name",
"get_call_graph",
"get_objc_classes",
"get_objc_protocols",
"inspect_native_api",
"inspect_native_dispatch_metadata",
"trace_call_path",
"trace_feature",
"trace_native_ui_action",
"trace_native_values"
]
},
{
"id": "native",
"surface": "native-provider",
"count": 10,
"tools": [
"capture_native_ui_scenario",
"demangle_swift",
"inspect_binary_layout",
"inspect_macho",
"inspect_plist",
"inspect_recorded_crash",
"inspect_signature",
"list_architectures",
"observe_native_calls",
"observe_native_ui"
]
},
{
"id": "artifact",
"surface": "artifact-provider",
"count": 7,
"tools": [
"decode_interface_builder",
"extract_artifact",
"inspect_artifact",
"inspect_asset_catalog",
"inspect_evm_interface",
"inspect_keyed_archive",
"trace_dylib_resolution"
]
},
{
"id": "managed",
"surface": "managed",
"count": 7,
"tools": [
"compare_managed_members",
"import_managed_reconstruction",
"inspect_managed_artifact",
"inspect_managed_members",
"inspect_managed_native_boundaries",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
},
{
"id": "firmware",
"surface": "firmware-provider",
"count": 2,
"tools": ["extract_firmware", "inspect_firmware_regions"]
},
{
"id": "android",
"surface": "android-provider",
"count": 5,
"tools": [
"inspect_android_class",
"inspect_android_method",
"inspect_android_package",
"search_android_classes",
"trace_android_references"
]
},
{
"id": "browser",
"surface": "browser-provider",
"count": 12,
"tools": [
"analyze_web_bundle",
"capture_browser_scenario",
"capture_web_screenshot",
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_network_capture",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
{
"id": "electron",
"surface": "electron-provider",
"count": 5,
"tools": [
"analyze_javascript_application",
"capture_electron_scenario",
"inspect_electron_page",
"list_electron_targets",
"reconcile_javascript_runtime"
]
},
{
"id": "javascript-runtime",
"surface": "runtime-provider",
"count": 2,
"tools": [
"list_javascript_runtime_targets",
"observe_javascript_runtime"
]
},
{
"id": "application",
"surface": "application-workflow",
"count": 13,
"tools": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"export_web_scripts",
"project_android_application_graph",
"project_apple_application_graph",
"recover_javascript_sources",
"trace_application_feature",
"trace_javascript_semantics",
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
"id": "session",
"surface": "session",
"count": 21,
"tools": [
"binary_session",
"build_call_path",
"capture_process_scenario",
"close_binary",
"compare_artifacts",
"compare_bundles",
"compare_functions",
"compare_process_captures",
"correlate_static_and_runtime",
"export_evidence_bundle",
"find_changed_behavior",
"get_evidence_bundle",
"get_navigation_context",
"import_evidence_bundle",
"inspect_address_context",
"list_unknowns",
"open_binary",
"record_unknown",
"update_unknown",
"verify_reconstruction",
"verify_unknown_resolution"
]
}
]
},
"providers": [
{
"id": "binwalk",
"name": "Binwalk",
"version": "3.1.0",
"capabilities": ["inspect_firmware_regions"]
},
{
"id": "evmole-interface",
"name": "REA EVM interface adapter",
"version": "evmole@0.9.3",
"capabilities": ["inspect_evm_interface"]
},
{
"id": "ghidra",
"name": "Ghidra",
"version": null,
"capabilities": [
"address_name",
"address_to_file_offset",
"analyze_function",
"annotate_native_function",
"inspect_native_data_type",
"inspect_native_instruction",
"inspect_native_load_image",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"resolve_native_call_targets",
"search_procedures",
"search_strings",
"xrefs"
]
},
{
"id": "har-schema/lossless-json",
"name": "REA HAR capture adapter",
"version": "har-schema@2.0.0;jsonc-parser@3.3.1;lossless-json@4.3.1",
"capabilities": ["inspect_web_network_capture"]
},
{
"id": "hopper",
"name": "Hopper Disassembler",
"version": null,
"capabilities": [
"address_name",
"address_to_file_offset",
"analyze_function",
"comment",
"current_address",
"current_document",
"current_procedure",
"goto_address",
"inline_comment",
"list_bookmarks",
"list_documents",
"list_names",
"list_procedures",
"list_segments",
"list_strings",
"next_address",
"prev_address",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_bytes",
"read_function_instructions",
"resolve_containing_procedure",
"search_procedures",
"search_strings",
"set_address_name",
"set_addresses_names",
"set_bookmark",
"set_comment",
"set_current_document",
"set_inline_comment",
"unset_bookmark",
"xrefs"
]
},
{
"id": "ida",
"name": "IDA Pro MCP adapter",
"version": "1",
"capabilities": [
"analyze_function",
"list_procedures",
"list_strings",
"procedure_address",
"procedure_assembly",
"procedure_callees",
"procedure_callers",
"procedure_pseudo_code",
"read_function_instructions",
"search_procedures",
"search_strings",
"xrefs"
]
},
{
"id": "jadx",
"name": "JADX headless",
"version": "0.7.1",
"capabilities": [
"inspect_android_class",
"inspect_android_method",
"inspect_android_package",
"search_android_classes",
"trace_android_references"
]
},
{
"id": "mitmproxy-native-tnetstring",
"name": "REA offline native mitmproxy adapter",
"version": "12.2.3",
"capabilities": ["inspect_web_network_capture"]
},
{
"id": "native-macos",
"name": "macOS native inspection utilities",
"version": null,
"capabilities": [
"capture_native_ui_scenario",
"demangle_swift",
"inspect_macho",
"inspect_plist",
"inspect_signature",
"list_architectures",
"observe_native_calls",
"observe_native_ui"
]
},
{
"id": "pwntools-elf",
"name": "REA pwntools ELF adapter",
"version": "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
"capabilities": ["inspect_binary_layout", "inspect_recorded_crash"]
},
{
"id": "rea-android-application",
"name": "REA Android application workflows",
"version": "1",
"capabilities": ["project_android_application_graph"]
},
{
"id": "rea-apple-application",
"name": "REA Apple application workflows",
"version": "1",
"capabilities": ["project_apple_application_graph"]
},
{
"id": "rea-artifact-graph",
"name": "REA safe artifact graph provider",
"version": "1",
"capabilities": [
"decode_interface_builder",
"extract_artifact",
"inspect_artifact",
"inspect_asset_catalog",
"inspect_keyed_archive",
"trace_dylib_resolution"
]
},
{
"id": "rea-cdp-browser",
"name": "REA Chrome DevTools Protocol observation provider",
"version": "2",
"capabilities": [
"analyze_web_bundle",
"capture_web_screenshot",
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
{
"id": "rea-cdp-electron",
"name": "REA Electron file-page CDP observation provider",
"version": "1",
"capabilities": ["inspect_electron_page", "list_electron_targets"]
},
{
"id": "rea-dotnet-static",
"name": "REA managed static analysis provider",
"version": "1",
"capabilities": [
"inspect_managed_artifact",
"inspect_managed_members",
"inspect_managed_native_boundaries"
]
},
{
"id": "rea-dotnet-workflows",
"name": "REA managed-code workflows",
"version": "1",
"capabilities": [
"compare_managed_members",
"import_managed_reconstruction",
"project_managed_application_graph",
"verify_managed_native_boundaries"
]
},
{
"id": "rea-javascript-application",
"name": "REA JavaScript application analyzer",
"version": "1",
"capabilities": ["analyze_javascript_application"]
},
{
"id": "rea-javascript-application-workflows",
"name": "REA JavaScript application workflows",
"version": "1",
"capabilities": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
"compare_javascript_export_shapes",
"compare_source_to_bundle",
"evaluate_reconstruction_coverage",
"trace_application_feature",
"trace_javascript_semantics"
]
},
{
"id": "rea-javascript-runtime-reconciliation",
"name": "REA JavaScript runtime reconciliation",
"version": "1",
"capabilities": ["reconcile_javascript_runtime"]
},
{
"id": "rea-playwright-browser-scenario",
"name": "REA Playwright browser scenario capture provider",
"version": "1",
"capabilities": ["capture_browser_scenario"]
},
{
"id": "rea-playwright-electron-active",
"name": "REA Playwright active Electron observation provider",
"version": "1",
"capabilities": ["capture_electron_scenario"]
},
{
"id": "rea-v8-inspector",
"name": "REA passive Node/Electron V8 Inspector provider",
"version": "1",
"capabilities": [
"list_javascript_runtime_targets",
"observe_javascript_runtime"
]
},
{
"id": "rea-web-script-export",
"name": "REA captured script export",
"version": "1",
"capabilities": [
"export_web_scripts",
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
"id": "unblob",
"name": "Unblob",
"version": "26.6.4",
"capabilities": ["extract_firmware"]
},
{
"id": "wakaru",
"name": "Wakaru",
"version": "1.13.0",
"capabilities": ["recover_javascript_sources"]
}
],
"setup_clients": [
{
"id": "claude_code",
"display_name": "Claude Code",
"format": "json",
"configuration": "managed"
},
{
"id": "claude_desktop",
"display_name": "Claude Desktop",
"format": "json",
"configuration": "managed"
},
{
"id": "codex",
"display_name": "Codex",
"format": "toml",
"configuration": "managed"
},
{
"id": "cursor",
"display_name": "Cursor",
"format": "json",
"configuration": "managed"
},
{
"id": "gemini_cli",
"display_name": "Gemini CLI",
"format": "json",
"configuration": "managed"
},
{
"id": "windsurf",
"display_name": "Windsurf",
"format": "json",
"configuration": "managed"
},
{
"id": "devin",
"display_name": "Devin",
"format": "json",
"configuration": "managed"
},
{
"id": "opencode",
"display_name": "OpenCode",
"format": "opencode",
"configuration": "managed"
},
{
"id": "antigravity",
"display_name": "Antigravity",
"format": "json",
"configuration": "managed"
},
{
"id": "copilot_cli",
"display_name": "GitHub Copilot CLI",
"format": "copilot_cli",
"configuration": "managed"
},
{
"id": "commandcode",
"display_name": "Command Code",
"format": "commandcode",
"configuration": "managed"
},
{
"id": "vscode",
"display_name": "VS Code",
"format": "vscode",
"configuration": "managed"
}
],
"cli": {
"primary_count": 94,
"commands": [
"address-to-file-offset",
"analyze",
"analyze-javascript-application",
"analyze-web-bundle",
"annotate-native-function",
"build-reconstruction-obligation-ledger",
"capabilities",
"capture-browser-scenario",
"capture-electron-scenario",
"capture-native-ui-scenario",
"capture-process",
"capture-web-screenshot",
"compare",
"compare-application-versions",
"compare-javascript-export-shapes",
"compare-managed-members",
"compare-process-captures",
"compare-source-to-bundle",
"compare-web-captures",
"compare-web-screenshots",
"decode-interface-builder",
"decompile",
"demangle-swift",
"discover-webmcp-tools",
"doctor",
"evaluate-reconstruction-coverage",
"evidence-export",
"evidence-import",
"export-web-scripts",
"extract-artifact",
"extract-firmware",
"function",
"import-managed-reconstruction",
"import-reference-source",
"inspect",
"inspect-android-class",
"inspect-android-method",
"inspect-android-package",
"inspect-artifact",
"inspect-asset-catalog",
"inspect-binary-layout",
"inspect-electron-page",
"inspect-evm-interface",
"inspect-firmware-regions",
"inspect-keyed-archive",
"inspect-macho",
"inspect-managed-artifact",
"inspect-managed-members",
"inspect-managed-native-boundaries",
"inspect-native-api",
"inspect-native-data-type",
"inspect-native-dispatch-metadata",
"inspect-native-instruction",
"inspect-native-load-image",
"inspect-plist",
"inspect-recorded-crash",
"inspect-signature",
"inspect-web-event-listeners",
"inspect-web-network-capture",
"inspect-web-page",
"instructions",
"list-architectures",
"list-browser-targets",
"list-electron-targets",
"list-javascript-runtime-targets",
"observe-javascript-runtime",
"observe-native-calls",
"observe-native-ui",
"observe-web-execution",
"observe-web-session",
"project-android-application-graph",
"project-apple-application-graph",
"project-managed-application-graph",
"providers",
"read-bytes",
"reconcile-javascript-runtime",
"recover-javascript-sources",
"resolve-native-call-targets",
"search",
"search-android-classes",
"setup",
"trace",
"trace-android-references",
"trace-application-feature",
"trace-dylib-resolution",
"trace-javascript-semantics",
"trace-native-ui-action",
"trace-native-values",
"trace-web-module-imports",
"trace-web-source-location",
"uninstall",
"update",
"verify-managed-native-boundaries",
"xrefs"
],
"aliases": [
{
"name": "compare-bundles",
"target": "compare"
}
]
},
"runtime_catalog": {
"counts": {
"cli_commands": 94,
"mcp_tools": 139,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "9ee5392d18f04f25b5c97c1b69a3644a0d81e52c813167a6f140791bc692741d",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "4c5f67d076baf821b5cb6d6fb2cc0f6c274e96b627b85b892577760a6f71ab1a",
"providers_sha256": "954b499c76514890062d2b6c936d73cb3a9c45412d522a081125a325e81cdec5"
}
}
}
+52 -5
View File
@@ -11,6 +11,32 @@ breaking changes. Record the full source SHA and create `release/VERSION` at
that commit. The version is a maintainer choice; Release Please's proposed
version must agree before publication.
`release/VERSION` also sets the expected version for the workflow. Use an exact
SemVer such as `release/6.1.0` or `release/6.1.0-rc.1`, without build metadata.
The checkpoint validator rejects a candidate whose package, lockfile, manifest,
registry metadata or changelog disagrees with that version before creating a tag.
### Compatibility and version selection
REA uses the default SemVer strategy: compatible fixes increment patch;
compatible capabilities increment minor; incompatible public behavior increments
major. Adding tools, internal refactoring and correcting behavior outside the
supported contract do not by themselves require a major release.
The supported public surface includes documented CLI commands/options, MCP tool
names and input/result contracts, saved evidence formats and supported runtime
requirements. Before adding `!` or a `BREAKING CHANGE` footer, a PR must identify
a previously valid call or configuration that will fail or change meaning,
explain why compatibility cannot be preserved, and give its migration. Prefer
optional additions, compatibility adapters and a documented deprecation period;
group deliberate removals into a planned major release. Maintainers review this
impact before merging; the bot parses markers and cannot infer compatibility.
For example, adding an optional inspection tool is a minor change. Rejecting a
relative MCP path previously accepted by the public contract is breaking;
rejecting an input that the existing contract already prohibited is a fix.
Changing the version number alone does not restore compatibility.
For example, after selecting a reviewed commit for 5.0.0:
```bash
@@ -48,6 +74,24 @@ normalization, then approve that head's blocked runs from the PR page. Updates
to the bot branch can create new approval-required runs; approval of an older
head does not verify the normalized candidate. Wait for the final head's CI.
The validator reads the actual ancestry range from the published baseline tag
to the selected checkpoint, including visible first-parent Conventional Commits
and the PR titles in GitHub's default merge messages. The report preserves the
original merge subject alongside the extracted Conventional Commit title.
This matters after merging a side-branch release back into a newer main:
Release Please's chronological history cutoff can omit unreleased mainline
commits. Unreleased breaking markers require a major increment and references
in the new release's breaking-change migration section. Other omitted entries
are reported for review. Historical notes from an older release cannot satisfy
the new release's migration check. This audits declared markers and references;
it does not prove API compatibility or the quality of a migration explanation.
If preparation reports a mismatch or omitted breaking changes, inspect the
recorded ancestry report, correct the release PR's version artifacts and notes,
regenerate documentation, and review/test its final head. Do not repeatedly
prepare over manual corrections. Publication independently checks the merged
candidate again before Release Please can create a tag.
Review the candidate's version, notes, generated metadata, and package
contents. Wait for the candidate's CI and relevant real-provider checks.
Routine local iterations need focused checks; CI owns full deterministic
@@ -100,8 +144,8 @@ before cutting the next checkpoint; otherwise main retains the previous
release baseline and can propose an already-published version again.
After publication, open a PR from the release branch back to main. Preserve
main's later implementation changes and resolve generated-file conflicts by
regenerating from the combined contracts with the released package version.
main's later implementation changes and regenerate ignored build outputs from
the combined contracts with the released package version.
Review and test this synchronization PR, then use a merge commit so the release
tag remains in main's ancestry. Keep the released tag unchanged.
@@ -119,9 +163,12 @@ git merge-base --is-ancestor rea-agents-5.0.0 origin/main
Close any superseded rolling release PR. Future releases repeat the checkpoint
procedure from main; never resume automatic release-PR refreshes on main pushes.
The generated-metadata workflow stays limited to pull requests into main so it
does not push commits onto a frozen candidate. Preparation normalizes
`docs/product-catalog.json` on the bot pull request.
Preparation generates and validates the catalog, portable conformance
projections, and packaged skill from the candidate checkout without committing
them. The checkpoint validator binds tracked version authority to the source
SHA; generated-catalog and package checks validate the derived representation.
No generated-metadata workflow pushes follow-up commits onto feature or release
branches.
## Partial publication and retries
+25 -9
View File
@@ -98,6 +98,20 @@ an XML graph golden. `verify:asset-catalog` compiles source-owned colors with
`actool`, invokes real `assetutil`, then checks CLI/MCP results, exact catalog
digest, every raw metadata field, pagination and malformed input rejection.
Neither artifact workflow requires Hopper or launches it. Both run in macOS CI.
`verify:code-signature` needs only macOS with Command Line Tools. It re-signs
copies of a clang-built fixture with real `codesign`: ad hoc, hardened
runtime, hardened runtime with relaxing entitlements, and `library,restrict`
flags. It checks that `inspect-signature` decodes those flags and derives the
expected security facets. It also checks a platform binary, which must report a
platform identifier and pass the `anchor apple` origin check, and bundles with an added unsealed file, at the top level
and inside a nested XPC service, which must fail deep strict verification. CLI and stdio MCP results must match.
The default lane explicitly reports real stapled-ticket coverage as unverified:
its source-built ad hoc fixtures cannot receive genuine Apple notarization tickets.
To verify ticket producer output through CLI and MCP, supply an existing stapled app:
`npm run verify:code-signature -- --stapled-app "/Applications/Firefox.app"`.
This optional subcase preflights `xcrun stapler`, validates the selected app with
Apple's tool, and checks that both REA surfaces return the same ticket digest and
`present` framing observation. It does not install an app or submit anything for notarization.
`verify:macos-bundle` needs only macOS with Command Line Tools. It compiles a
source-owned app with `clang`: a versioned framework, XPC services, an app
extension, a login item, a privileged helper, launchd plists, and a helper tool,
@@ -425,7 +439,7 @@ behavior, and complete CI evidence before merging.
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks |
| `npm run docs:check` | Committed generated-document freshness |
| `npm run docs:check` | Generated-document validation from current source and build outputs |
For example:
@@ -460,19 +474,21 @@ broad changes or diagnosing CI, rather than after every edit. Package/install
changes additionally need package verification; provider changes need actual
provider evidence.
Local full-suite Vitest runs use up to two workers and schedule projects one
at a time. Process, acceptance and process-global projects serialize their
files to prevent competing lifecycle observations. CI retains its two-worker
budget.
Each Vitest project uses up to two workers, bounded by available host parallelism.
Default-group projects can run together. Only `process-boundary` runs in the
later sequence group and serializes its files because process-tree observations
share host sampling resources. Acceptance and process-global files retain isolated
forks but are not serialized; isolation does not imply serial scheduling. CI uses
the same per-project worker bound.
The pure domain/contracts and recording-port service projects share one worker
module context because their tests own no mutable runtime resources. MCP
boundary files also share the immutable server module graph while creating and
closing independent in-memory sessions. Adapter, composition, acceptance,
process-global, and other boundary projects retain per-file isolation.
`npm test`, `npm run docs:check`, and `npm run docs:generate` share
repository-local locks and fail fast when the same class of command is already
running. The `npm test` build is inside that lock. `check:pr` runs its test task
before starting generated-document validation.
Build and documentation writers use checkout-local locks for their respective
output files. `npm test` delegates its build to the build lock; the Vitest suite
does not hold a test-command lock. `check:pr` runs its test task before starting
generated-document validation.
Vitest and Node persistent compile caches are deliberately not enabled by
default. To evaluate repeated local runs, opt in for both cold and warm
+12 -1
View File
@@ -80,7 +80,18 @@ or model-authored loop when one call can answer the question.
## Implement and evaluate
- Put shared user workflows in the application layer; keep MCP translation in
the server adapter and CLI behavior aligned with the same workflow.
the server adapter and CLI behavior aligned with the same workflow. Share
Evidence provenance, unknown projection, and eligible snapshot binding through
application/session owners. Matching result payloads alone do not establish
equivalent retention or replay behavior.
- Derive each facet's coverage from examined records, unsupported representations,
and exhaustiveness. Keep issues structured until presentation; do not select a
facet by matching diagnostic prose or infer completeness from an empty failure
list. Preserve available observations independently of cleanup success.
- Measure the resource consumed before choosing a limit. Value alternatives,
retained records, trace-frame products, output bytes, and temporary-file growth
need different accounting. Enforce a justified budget before the costly step
and report its effect; an input-byte or graph-node cap does not bound every stage.
- Update the canonical contract inventory, output schemas, examples, generated
catalog, and relevant docs together.
- Verify valid, malformed, boundary, unavailable, cancellation, and partial
@@ -1,105 +0,0 @@
{
"ledger_id": "ecl_132e857a8a93f0b366cd72f9cfa1c338ffd4100460a7834ef69abff936c84472",
"records": [
{
"claim_id": "managed.application-graph-projection",
"status": "pass",
"evidence_ids": [
"ev_b061a5a559acb3748cb9784c637f6693f74faeb564ba2b11d6fc32619e757459"
]
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"status": "pass",
"evidence_ids": [
"ev_13f3401a3aaddc3f6d087ad160fa928374e142cdb9861254359b46865b8c992e"
]
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"status": "pass",
"evidence_ids": [
"ev_6f026216826ca06e080b77ff608f02634fa945fbd2e151b4cd261c1444f8bbb1"
]
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"status": "unsupported",
"evidence_ids": [
"ev_65a94c4de03f9c6ebdaff3f381cca377821f349d3ca6dfad47a31cc586fae939"
]
},
{
"claim_id": "managed.malformed-metadata",
"status": "pass",
"evidence_ids": [
"ev_deae4259ea4395623176de40d06c2abcbc2b3a77f0bfb068d76e568d801f45a8"
]
},
{
"claim_id": "managed.managed-native-verification",
"status": "pass",
"evidence_ids": [
"ev_0edcbbcaa1191a4e5324a437f739957b77f9ac45e7897724337f2d6d06b3f9d1"
]
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"status": "pass",
"evidence_ids": [
"ev_d1af50e8dc04cd589385769efd8758e82095425f6e9c1cc4bf2f530c6b434c93"
]
},
{
"claim_id": "managed.mvid-and-token-drift",
"status": "pass",
"evidence_ids": [
"ev_5493f40bc875535488d789a490361abcc1f76bfbdf52536d09ff127442a8e854"
]
},
{
"claim_id": "managed.not-managed",
"status": "pass",
"evidence_ids": [
"ev_03ce0cc0183ee0710f8120e837565129559ab3bc41927df326066370c2341699"
]
},
{
"claim_id": "managed.operator-local-manifest",
"status": "unsupported",
"evidence_ids": [
"ev_9d3e50da1d372c55f46649e69eae702cbe8f0ba6a81d1e46afc3e1a284c81757"
]
},
{
"claim_id": "managed.operator-manifest-self-test",
"status": "pass",
"evidence_ids": [
"ev_f974f4b9d039cda841894c6173cae86427ecef48f4e7a4d6c73edf7b23af2b2a"
]
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"status": "pass",
"evidence_ids": [
"ev_c4f38ce76d8359d92ee8e9442503e47b7913550442349a0df688bd274a3def67"
]
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"status": "pass",
"evidence_ids": [
"ev_3e05ef1d6a796d3d4cb3372fe8e549efec759c59bf09d8ff660059a2e448cbc0"
]
}
],
"summary": {
"total": 13,
"pass": 11,
"fail": 0,
"unsupported": 2,
"truncated": 0,
"unknown": 0,
"complete": false
}
}
@@ -1,201 +0,0 @@
{
"manifest_id": "ecm_8a67e7a3578e49c8d8d22107eab8f1258094dc40041d14deee93189fd0c514b2",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
},
"environment": {
"platform": "portable",
"architecture": "portable",
"runtime": "node",
"runtime_version": "portable"
},
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "ef398cf4b5ee7fb5e0bcf42ca787049ec145e1d911ee7951017fee1cb3980d74"
}
],
"claims": [
{
"claim_id": "managed.application-graph-projection",
"scenario": {
"id": "managed-application-graph-projection",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.decompiler-reconstruction-import",
"scenario": {
"id": "decompiler-reconstruction-import",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.dotnet-framework-x86-pinvoke",
"scenario": {
"id": "dotnet-framework-x86-pinvoke",
"version": 1
},
"artifact_sha256s": [
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.ilspy-reconstruction-oracle",
"scenario": {
"id": "ilspy-reconstruction-oracle",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.malformed-metadata",
"scenario": {
"id": "malformed-metadata",
"version": 1
},
"artifact_sha256s": [
"827e10b6391d29edca48c191e2ddd79f3bc1165e46e4ba2f4edf41ab60f45e78"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.managed-native-verification",
"scenario": {
"id": "managed-native-verification",
"version": 1
},
"artifact_sha256s": [
"9999999999999999999999999999999999999999999999999999999999999999",
"f17fe6286cb07c6385e786e0aa877564d44123197f02617dfec70c537a33e4ea"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.modern-dotnet-anycpu",
"scenario": {
"id": "modern-dotnet-anycpu",
"version": 1
},
"artifact_sha256s": [
"92b588ae43d418117fa108c0004b5d0451f6112238d21bad8d75c0eb1da860f0"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.mvid-and-token-drift",
"scenario": {
"id": "mvid-and-token-drift",
"version": 1
},
"artifact_sha256s": [
"276395c671a656ab57d583158443c10de552dccfb37dc97cd013f673c0f86b43",
"ef009dab7424fdeba996f2524dbe86fd899c96394498cca0463fcfdb228c5572"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.not-managed",
"scenario": {
"id": "not-managed",
"version": 1
},
"artifact_sha256s": [
"de74dab8ab034054a811d4248efcda0bed505187a061a806e274ae87bc453754"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.operator-local-manifest",
"scenario": {
"id": "operator-local-manifest",
"version": 1
},
"artifact_sha256s": [],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.operator-manifest-self-test",
"scenario": {
"id": "operator-manifest-self-test",
"version": 1
},
"artifact_sha256s": [
"cdc11ec7fd0f392e43b9b23971cb41ec49fd0a997fc33c24fcc553cd4386ae37"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.unicode-obfuscated-identifiers",
"scenario": {
"id": "unicode-obfuscated-identifiers",
"version": 1
},
"artifact_sha256s": [
"cead4cc99733e4e56a5fd0cec8a0813bbc24e249aa72a10eca4c47ee3b0e129d"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
},
{
"claim_id": "managed.x64-ready-to-run-native-body",
"scenario": {
"id": "x64-ready-to-run-native-body",
"version": 1
},
"artifact_sha256s": [
"e47402428322cdcb06f561036d7d108528cbc6e262d92a21f869218a20c80bfb"
],
"provider": {
"id": "rea-managed-conformance",
"version": "1"
}
}
]
}
+3 -3
View File
@@ -32,9 +32,9 @@ HAR uses unchanged `har-schema@2.0.0` draft-06 schemas, `ajv@8.20.0`,
`lossless-json@4.3.1` numeric representations, pinned in the npm lockfile.
Only the HAR 1.2 profile is supported. Unknown extensions remain reported
evidence. Duplicate object keys, including equal values and escaped spellings,
are rejected before materialization. A `__proto__` member returns an explicit
unsupported-boundary error in both formats: the current shared JSON schema
cannot preserve this member. Original capture files remain unchanged.
are rejected before materialization. Prototype-named members such as
`__proto__` are preserved as ordinary own JSON members without prototype
mutation. Original capture files remain unchanged.
The observed mitmproxy 12.2.3 `SaveHar` profile can emit `postData.text: null`
for a missing request body. REA preserves that reported null, omitting the
optional field only from its validation copy for this exact producer profile;
+1 -1
View File
@@ -107,7 +107,7 @@ if [[ "$dry_run" == true ]]; then
fi
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
npm install --global "${prefix_args[@]}" "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
npm install --global ${prefix_args[@]+"${prefix_args[@]}"} "$PACKAGE@$version" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
+6 -2
View File
@@ -87,7 +87,8 @@
"mcp-catalog:check": "node scripts/generate-mcp-tool-catalog.mjs --check",
"prebuild": "npm run deps:check && npm run metadata:check",
"build": "node scripts/run-exclusive.mjs build npm run build:unlocked",
"build:unlocked": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/generate-mcp-tool-catalog.mjs",
"build:unlocked": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && npm run build:projections",
"build:projections": "node scripts/generate-mcp-tool-catalog.mjs && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && node scripts/generate-completion-ledger.mjs",
"precheck": "npm run deps:check",
"check": "turbo run typecheck lint format:check knip metadata:check",
"precheck:fast": "npm run deps:check",
@@ -141,6 +142,7 @@
"verify:apple-dispatch": "npm run build:cached && node scripts/verify/apple/dispatch.mjs",
"verify:native-ui": "npm run build:cached && node scripts/verify-native-ui.mjs",
"verify:keyed-archive": "npm run build:cached && node scripts/verify/apple/keyed-archive.mjs",
"verify:code-signature": "npm run build:cached && node scripts/verify/apple/code-signature.mjs",
"verify:macos-bundle": "npm run build:cached && node scripts/verify/apple/macos-bundle.mjs",
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
"verify:ida": "npm run build:cached && node scripts/verify-real-ida.mjs",
@@ -179,11 +181,13 @@
"config:print": "node scripts/print-mcp-config.mjs",
"evidence:generate": "npm run build:cached && node scripts/generate-completion-ledger.mjs",
"evidence:check": "npm run build:cached && node scripts/generate-completion-ledger.mjs --check",
"predocs:dev": "npm run docs:generate",
"docs:dev": "vitepress dev docs",
"predocs:build": "npm run docs:generate",
"docs:build": "vitepress build docs",
"docs:preview": "vitepress preview docs",
"docs:generate": "node scripts/run-exclusive.mjs docs npm run docs:generate:unlocked",
"docs:generate:unlocked": "npm run metadata:generate && npm run build:cached && node scripts/generate-skill-metadata.mjs && node scripts/generate-product-catalog.mjs && node scripts/generate-completion-ledger.mjs && node scripts/generate-error-schema.mjs",
"docs:generate:unlocked": "npm run metadata:generate && npm run build:cached && node scripts/generate-error-schema.mjs",
"docs:check": "node scripts/run-exclusive.mjs docs turbo run docs:check:built",
"docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-mcp-tool-catalog.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && node scripts/generate-error-schema.mjs --check",
"docs:error-schema": "npm run build:cached && node scripts/generate-error-schema.mjs",
-5
View File
@@ -34,11 +34,6 @@
"type": "json",
"path": "server.json",
"jsonpath": "$.packages[0].version"
},
{
"type": "json",
"path": "docs/product-catalog.json",
"jsonpath": "$.package.version"
}
]
}
+1 -1
View File
@@ -30,7 +30,7 @@ const { stdout } = await exec(
},
);
const verifierOutput = JSON.parse(stdout);
// Managed conformance claims are host-independent. Keep the checked-in
// Managed conformance claims are host-independent. Keep the generated
// commitment reproducible across macOS, Linux, and Windows rather than
// embedding the machine that happened to generate it.
const portableCompletionReport = {
+1 -1
View File
@@ -17,7 +17,7 @@ const lock = JSON.parse(
await readFile(join(root, "package-lock.json"), "utf8"),
);
const skill = await readFile(
join(root, "skills/reverse-engineer-anything/SKILL.md"),
join(root, "skill-src/reverse-engineer-anything/SKILL.md"),
"utf8",
);
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
+4 -1
View File
@@ -1,3 +1,4 @@
import { mkdir } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -14,8 +15,10 @@ for (const argument of arguments_)
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const catalog = await createProductCatalog(root);
const path = join(root, "docs/public/product-catalog.json");
if (!arguments_.has("--check")) await mkdir(dirname(path), { recursive: true });
await ensureGeneratedFile({
path: join(root, "docs/product-catalog.json"),
path,
source: await serializeProductCatalog(catalog),
check: arguments_.has("--check"),
generateCommand: "npm run docs:generate",
+62 -19
View File
@@ -1,4 +1,4 @@
import { readFile } from "node:fs/promises";
import { mkdir, readFile, readdir, rm } from "node:fs/promises";
import { dirname, join } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
@@ -10,26 +10,69 @@ for (const argument of arguments_)
throw new Error(`Unknown skill metadata option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const path = join(root, "skills/reverse-engineer-anything/SKILL.md");
const current = await readFile(path, "utf8");
const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
const outputRoot = join(root, "skills/reverse-engineer-anything");
const check = arguments_.has("--check");
const current = (
await readFile(join(sourceRoot, "SKILL.md"), "utf8")
).replaceAll("\r\n", "\n");
const { CATALOG_IDENTITY } = await import(
`${pathToFileURL(join(root, "dist/catalogIdentity.js")).href}?${String(Date.now())}`
);
const withCount = current.replace(
/^\s{2}tool_count:\s*\d+\s*$/mu,
` tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}`,
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
throw new Error(
"Catalog metadata belongs in the generated skill, not skill-src",
);
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
if (!versionLine.test(current))
throw new Error("Missing authored skill version");
const source = current.replace(
versionLine,
`$&\n tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}`,
);
const digestLine = ` catalog_digest: "${CATALOG_IDENTITY.digests.combined_sha256}"`;
const source = /^\s{2}catalog_digest:/mu.test(withCount)
? withCount.replace(
/^\s{2}catalog_digest:\s*"[a-f0-9]{64}"\s*$/mu,
digestLine,
)
: withCount.replace(/^(\s{2}tool_count:\s*\d+\s*)$/mu, `$1\n${digestLine}`);
const paths = await filePaths(sourceRoot);
// Rebuild this owned output directory so removed references cannot survive a build.
if (!check) await rm(outputRoot, { recursive: true, force: true });
for (const relativePath of paths) {
const path = join(outputRoot, relativePath);
if (!check) await mkdir(dirname(path), { recursive: true });
await ensureGeneratedFile({
path,
source:
relativePath === "SKILL.md"
? source
: (await readFile(join(sourceRoot, relativePath), "utf8")).replaceAll(
"\r\n",
"\n",
),
check,
generateCommand: "npm run docs:generate",
});
}
if (
check &&
JSON.stringify(await filePaths(outputRoot)) !== JSON.stringify(paths)
)
throw new Error(
"Generated skill file inventory drifted; run npm run docs:generate",
);
await ensureGeneratedFile({
path,
source,
check: arguments_.has("--check"),
generateCommand: "npm run docs:generate",
});
async function filePaths(directory, prefix = "") {
const paths = [];
for (const entry of (await readdir(directory, { withFileTypes: true })).sort(
(left, right) =>
left.name < right.name ? -1 : left.name > right.name ? 1 : 0,
)) {
const relativePath = prefix === "" ? entry.name : `${prefix}/${entry.name}`;
if (entry.isDirectory())
paths.push(
...(await filePaths(join(directory, entry.name), relativePath)),
);
else if (entry.isFile()) paths.push(relativePath);
else
throw new Error(
`Skill bundle requires regular files or directories: ${relativePath}`,
);
}
return paths;
}
+86 -2
View File
@@ -1,5 +1,8 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import { fileURLToPath } from "node:url";
import { chromium } from "playwright-core";
@@ -62,13 +65,21 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
);
assert.equal(bytes.readUInt32BE(16), 2048);
assert.equal(bytes.readUInt32BE(20), 1536);
await verifyDefaultScreenshotDelivery({
entrypoint,
environment: env,
endpoint,
targetId: target.id,
screenshot,
});
const maxBufferSize = 64 * 1024 * 1024;
transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env,
env: { ...env, REA_MCP_MAX_RESPONSE_BYTES: String(maxBufferSize) },
stderr: "pipe",
// The SDK defaults to a 10 MiB receive buffer; inline PNG output is larger.
maxBufferSize: 64 * 1024 * 1024,
maxBufferSize,
});
client = new Client({ name: "browser-screenshot-real-e2e", version: "1" });
await client.connect(transport);
@@ -102,6 +113,9 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
mocked: false,
cli: true,
stdio_mcp: true,
default_buffer_recovery: true,
complete_export: true,
matched_large_buffer: true,
png_bytes: bytes.byteLength,
};
} finally {
@@ -117,3 +131,73 @@ export async function verifyLargeScreenshotE2e(endpoint, origin) {
}
}
}
const verifyDefaultScreenshotDelivery = async ({
entrypoint,
environment,
endpoint,
targetId,
screenshot,
}) => {
const workspace = await mkdtemp(join(tmpdir(), "rea-screenshot-delivery-"));
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
stderr: "pipe",
});
const client = new Client({
name: "browser-screenshot-default-e2e",
version: "1",
});
try {
await client.connect(transport);
const constrained = await client.callTool(
{
name: "capture_web_screenshot",
arguments: { cdp_endpoint: endpoint, target_id: targetId },
},
{ timeout: 60000 },
);
assert.equal(constrained.isError, true);
assert.equal(
constrained.structuredContent?.error?.details?.resource,
"transport",
);
const reference =
constrained.structuredContent?.error?.details?.reported_limits
?.evidence_reference;
assert.equal(reference?.kind, "retained-evidence");
await client.ping();
const exported = await client.callTool({
name: "export_evidence_bundle",
arguments: { path: join(workspace, "screenshot-evidence.json") },
});
assert.notEqual(exported.isError, true, JSON.stringify(exported));
const bundle = JSON.parse(
await readFile(join(workspace, "screenshot-evidence.json"), "utf8"),
);
const record = bundle.records.find(
(record) => record.evidence_id === reference.evidence_id,
);
assert.ok(
record,
"Full screenshot Evidence must remain exportable after the delivery constraint",
);
const retained = webScreenshotSchema.parse(
parseEvidence(record).normalized_result,
);
assert.deepEqual(retained.artifact, screenshot.artifact);
await client.ping();
} finally {
try {
await client.close();
} finally {
try {
await transport.close();
} finally {
await rm(workspace, { recursive: true, force: true });
}
}
}
};
+13 -3
View File
@@ -77,7 +77,12 @@ export const documentationFactIssues = async (root, catalog) => {
requireText(issues, path, content, "MCP-tool_catalog");
if (path === "README.md") {
requireText(issues, path, content, "(docs/installation.md");
requireText(issues, path, content, "(docs/product-catalog.json)");
requireText(
issues,
path,
content,
"(docs/mcp-contracts.md#generated-catalog)",
);
continue;
}
for (const client of catalog.setup_clients)
@@ -99,11 +104,16 @@ export const documentationFactIssues = async (root, catalog) => {
requireText(issues, installationPath, installation, client.display_name);
const agents = await readFile(join(root, "AGENTS.md"), "utf8");
requireText(issues, "AGENTS.md", agents, "docs/product-catalog.json");
requireText(issues, "AGENTS.md", agents, "docs/public/product-catalog.json");
const templatePath = ".github/pull_request_template.md";
const template = await readFile(join(root, templatePath), "utf8");
requireText(issues, templatePath, template, "docs/product-catalog.json");
requireText(
issues,
templatePath,
template,
"docs/public/product-catalog.json",
);
return issues;
};
+4 -1
View File
@@ -10,7 +10,10 @@ export const withLargeResultMcp = async (
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
env: {
...environment,
REA_MCP_MAX_RESPONSE_BYTES: String(maxBufferSize),
},
stderr: "inherit",
maxBufferSize,
});
+3 -3
View File
@@ -58,16 +58,16 @@ export const createProductCatalog = async (root) => {
runtime_catalog: {
counts: sources.catalogIdentity.CATALOG_IDENTITY.counts,
digests: {
...sources.catalogIdentity.CATALOG_IDENTITY.digests,
// Full runtime schema digests belong to server identity, not this facts projection.
providers_sha256: providerCatalogDigest(providers),
},
},
};
};
/** Stable checked-in representation of the product catalog. */
/** Stable build-generated representation of the product catalog. */
export const serializeProductCatalog = (catalog) =>
formatGeneratedFile(
"docs/product-catalog.json",
"docs/public/product-catalog.json",
JSON.stringify(catalog, null, 2),
);
+333
View File
@@ -0,0 +1,333 @@
import { gt, major, valid } from "semver";
import { z } from "zod";
const shaSchema = z.string().regex(/^[a-f0-9]{40}$/u);
const versionSchema = z.object({ version: z.string() });
const manifestSchema = z.object({ ".": z.string() });
const sectionsSchema = z.object({
"changelog-sections": z.array(
z.object({ type: z.string(), hidden: z.boolean().optional() }),
),
});
const GITHUB_MERGE_SUBJECT = /^Merge pull request #(\d+) from \S+/u;
function conventionalTitle(subject, body) {
if (!GITHUB_MERGE_SUBJECT.test(subject)) return subject;
// GitHub's default merge message puts the PR title after its subject.
// Inspect that first nonempty line, not arbitrary examples in commit prose.
return (
body
.split(/\r?\n/u)
.slice(1)
.find((line) => line.trim() !== "")
?.trim() ?? subject
);
}
function releaseVersion(value, location) {
if (valid(value) !== value || value.includes("+")) {
throw new Error(
`${location} must contain an exact SemVer without build metadata: ${value}`,
);
}
return value;
}
async function readJson(git, sha, path, schema) {
const text = await git(["show", `${sha}:${path}`]);
let value;
try {
value = JSON.parse(text);
} catch (error) {
if (error instanceof SyntaxError) {
throw new Error(`${path}@${sha} contains invalid JSON: ${error.message}`);
}
throw error;
}
const parsed = schema.safeParse(value);
if (!parsed.success) {
throw new Error(`${path}@${sha} is malformed: ${parsed.error.message}`);
}
return parsed.data;
}
function readCommits(text) {
if (text.length === 0) return [];
const fields = text.split("\0");
if (fields.pop() !== "" || fields.length % 3 !== 0) {
throw new Error("Git returned malformed checkpoint history records.");
}
const commits = [];
for (let index = 0; index < fields.length; index += 3) {
const sha = shaSchema.parse(fields[index]);
const subject = z.string().parse(fields[index + 1]);
const body = z.string().parse(fields[index + 2]);
const title = conventionalTitle(subject, body);
const type = /^([a-z]+)(?:\([^)]*\))?!?: /u.exec(title)?.[1];
commits.push({
sha,
subject,
...(title !== subject ? { conventionalTitle: title } : {}),
type,
breaking:
/^[a-z]+(?:\([^)]*\))?!: /u.test(title) ||
/^BREAKING(?: CHANGE|-CHANGE):\s*\S/mu.test(body),
});
}
return commits;
}
function referencesCommit(text, commit) {
if (text.includes(commit.sha.slice(0, 7))) return true;
const pullRequest =
GITHUB_MERGE_SUBJECT.exec(commit.subject)?.[1] ??
/\(#(\d+)\)$/u.exec(commit.subject)?.[1];
return (
pullRequest !== undefined &&
new RegExp(`#${pullRequest}(?!\\d)`, "u").test(text)
);
}
function releaseNotes(text, expectedVersion) {
const headings = Array.from(text.matchAll(/^## \[([^\]]+)\].*$/gmu));
const first = headings[0];
if (first?.[1] !== expectedVersion || first.index === undefined) {
throw new Error(`CHANGELOG.md must start with release ${expectedVersion}.`);
}
return text.slice(first.index, headings[1]?.index ?? text.length);
}
function breakingSection(notes) {
const heading = /^### [^\n]*BREAKING CHANGES[^\n]*\n/imu.exec(notes);
if (heading === null) return "";
const body = notes.slice(heading.index + heading[0].length);
const end = body.search(/^### /mu);
return end < 0 ? body : body.slice(0, end);
}
async function requireAncestor(git, ancestor, descendant, description) {
try {
await git(["merge-base", "--is-ancestor", ancestor, descendant]);
} catch (error) {
const failure = z.object({ code: z.literal(1) }).safeParse(error);
if (!failure.success) throw error;
throw new Error(
`${description}: ${ancestor} is not an ancestor of ${descendant}. Sync the released tag into main before cutting a checkpoint, and prepare candidates from that frozen source.`,
);
}
}
async function resolveBaseline(git, tag) {
try {
return shaSchema.parse(
(await git(["rev-parse", "--verify", `${tag}^{commit}`])).trim(),
);
} catch (error) {
const failure = z.object({ code: z.literal(128) }).safeParse(error);
if (!failure.success) throw error;
throw new Error(
`Published baseline ${tag} is unavailable. Fetch release tags and finish the previous release sync before preparing this checkpoint. ${error instanceof Error ? error.message : String(error)}`,
);
}
}
async function validateCandidate(git, sha, version, commits) {
const versions = [
[
"package.json",
(await readJson(git, sha, "package.json", versionSchema)).version,
],
[
".release-please-manifest.json",
(
await readJson(
git,
sha,
".release-please-manifest.json",
manifestSchema,
)
)["."],
],
];
const lock = await readJson(
git,
sha,
"package-lock.json",
versionSchema.extend({
packages: z.object({ "": versionSchema }),
}),
);
versions.push(
["package-lock.json", lock.version],
["package-lock.json packages.root", lock.packages[""].version],
);
const server = await readJson(
git,
sha,
"server.json",
versionSchema.extend({
packages: z.array(
z.object({ identifier: z.string(), version: z.string() }),
),
}),
);
const npmPackage = server.packages.find(
(entry) => entry.identifier === "rea-agents",
);
if (npmPackage === undefined)
throw new Error("server.json is missing the rea-agents package.");
versions.push(
["server.json", server.version],
["server.json rea-agents package", npmPackage.version],
);
// Catalogs are generated and checked from this candidate by the release job;
// they are not version authority stored in the immutable source tree.
const generated = await git([
"show",
`${sha}:src/generatedPackageMetadata.ts`,
]);
const generatedVersions = Array.from(
generated.matchAll(/^\s+version: "([^"]+)"/gmu),
);
if (generatedVersions.length !== 1)
throw new Error(
"Generated package metadata must declare one literal version.",
);
versions.push(["src/generatedPackageMetadata.ts", generatedVersions[0]?.[1]]);
for (const [location, actual] of versions) {
if (actual !== version) {
throw new Error(
`${location}@${sha} declares ${actual}; release branch requires ${version}.`,
);
}
}
const notes = releaseNotes(
await git(["show", `${sha}:CHANGELOG.md`]),
version,
);
const breakingNotes = breakingSection(notes);
const missingBreaking = commits.filter(
(commit) => commit.breaking && !referencesCommit(breakingNotes, commit),
);
if (missingBreaking.length > 0) {
throw new Error(
`Release ${version} migration notes omit unreleased breaking changes: ${missingBreaking.map((commit) => `${commit.sha} ${commit.subject}`).join("; ")}`,
);
}
return commits.filter((commit) => !referencesCommit(notes, commit));
}
/** Validate a recorded checkpoint and optional candidate using exact Git ancestry. */
export async function inspectReleaseCheckpoint(git, options) {
const { phase, stage, releaseBranch, sourceSha, candidateSha } = z
.object({
phase: z.enum(["prepare", "publish"]),
stage: z.enum(["source", "candidate"]),
releaseBranch: z.string().startsWith("release/"),
sourceSha: shaSchema,
candidateSha: shaSchema.optional(),
})
.parse(options);
const expectedVersion = releaseVersion(
releaseBranch.slice("release/".length),
"release/VERSION",
);
if (phase === "publish" && stage === "candidate") {
throw new Error(
"Publish validates its merged source SHA; candidate stage applies only to prepare.",
);
}
const applicationSha =
phase === "publish"
? shaSchema.parse((await git(["rev-parse", `${sourceSha}^1`])).trim())
: sourceSha;
const manifest = await readJson(
git,
applicationSha,
".release-please-manifest.json",
manifestSchema,
);
const baselineVersion = releaseVersion(
manifest["."],
"published baseline manifest",
);
const sourcePackage = await readJson(
git,
applicationSha,
"package.json",
versionSchema,
);
if (sourcePackage.version !== baselineVersion) {
throw new Error(
`Source package ${sourcePackage.version} and release baseline ${baselineVersion} disagree.`,
);
}
if (!gt(expectedVersion, baselineVersion)) {
throw new Error(
`Expected release ${expectedVersion} must advance published baseline ${baselineVersion}.`,
);
}
const baselineTag = `refs/tags/rea-agents-${baselineVersion}`;
const baselineSha = await resolveBaseline(git, baselineTag);
await requireAncestor(
git,
baselineSha,
applicationSha,
"Published baseline is outside the checkpoint ancestry",
);
const configuration = await readJson(
git,
applicationSha,
"release-please-config.json",
sectionsSchema,
);
const visibleTypes = new Set(
configuration["changelog-sections"]
.filter((section) => !section.hidden)
.map((section) => section.type),
);
const commits = readCommits(
await git([
"log",
"--first-parent",
"-z",
"--format=%H%x00%s%x00%B",
`${baselineSha}..${applicationSha}`,
]),
).filter((commit) => commit.breaking || visibleTypes.has(commit.type));
const breaking = commits.filter((commit) => commit.breaking);
if (
breaking.length > 0 &&
major(baselineVersion) > 0 &&
major(expectedVersion) <= major(baselineVersion)
) {
throw new Error(
`Expected release ${expectedVersion} contains unreleased breaking markers since ${baselineVersion}: ${breaking.map((commit) => `${commit.sha} ${commit.subject}`).join("; ")}. Review compatibility or select a major version.`,
);
}
let snapshot = phase === "publish" ? sourceSha : candidateSha;
if (stage === "candidate" && snapshot === undefined) {
snapshot = shaSchema.parse((await git(["rev-parse", "HEAD"])).trim());
}
if (phase === "publish" || stage === "candidate") {
await requireAncestor(
git,
applicationSha,
snapshot,
"Candidate is outside the selected checkpoint ancestry",
);
}
const missingNotes =
phase === "publish" || stage === "candidate"
? await validateCandidate(git, snapshot, expectedVersion, commits)
: [];
return {
expectedVersion,
baselineVersion,
baselineSha,
sourceSha,
applicationSha,
commits,
missingNotes,
};
}
+1 -1
View File
@@ -1169,7 +1169,7 @@ try {
);
const ownershipInspection =
["/usr/bin/ps", "/bin/ps"].includes(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]');
assert.ok(
[process.execPath, python].includes(match[1]) || ownershipInspection,
`Unexpected attempted host execution: ${line}`,
+3 -1
View File
@@ -474,7 +474,9 @@ try {
);
const ownershipInspection =
["/usr/bin/ps", "/bin/ps"].includes(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
);
assert.ok(
[
process.execPath,
+21
View File
@@ -1,4 +1,5 @@
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { exec } from "./lib/verify-package-core.mjs";
import { verifyPackedBridge } from "./verify-packed-bridge.mjs";
@@ -28,6 +29,26 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error("package did not expose both rea command entry points");
if (packedManifest.mcpName !== "io.github.morluto/rea")
throw new Error("package did not retain its MCP Registry ownership marker");
if (packedFiles.some((path) => path.startsWith("package/skill-src/")))
throw new Error(
"package included authored skill sources instead of only the generated bundle",
);
const skill = (
await exec("tar", [
"-xOf",
join(root, tarball),
"package/skills/reverse-engineer-anything/SKILL.md",
])
).stdout;
if (
!skill.includes(
` tool_count: ${String(CATALOG_IDENTITY.counts.mcp_tools)}\n`,
) ||
skill.includes(" catalog_digest:")
)
throw new Error(
"packaged skill metadata did not match the catalog shipped in the package",
);
if (
packedFiles.some(
(path) => path.includes("__pycache__") || path.endsWith(".pyc"),
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env node
import { execFile } from "node:child_process";
import { parseArgs, promisify } from "node:util";
import { inspectReleaseCheckpoint } from "./lib/release-checkpoint.mjs";
const execFileAsync = promisify(execFile);
async function git(args) {
const result = await execFileAsync("git", args, {
encoding: "utf8",
maxBuffer: 16 * 1024 * 1024,
});
return result.stdout;
}
async function main() {
const { values } = parseArgs({
options: {
phase: { type: "string" },
stage: { type: "string", default: "source" },
"release-branch": { type: "string" },
"source-sha": { type: "string" },
},
});
const report = await inspectReleaseCheckpoint(git, {
phase: values.phase,
stage: values.stage,
releaseBranch: values["release-branch"],
sourceSha: values["source-sha"],
});
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
if (report.missingNotes.length > 0) {
process.stderr.write(
`Review ${report.missingNotes.length} additional unreleased mainline entries missing from these release notes; see the ancestry report above.\n`,
);
}
}
await main().catch((error) => {
process.stderr.write(
`Release checkpoint validation failed: ${error instanceof Error ? error.message : String(error)}\n`,
);
process.exitCode = 1;
});
+10 -8
View File
@@ -260,14 +260,16 @@ try {
);
const original = await readFile(path);
for (const mode of ["cli", "mcp"]) {
const error = await inspect(
mode,
{ capture_path: path, format },
"unsupported_provider",
);
assert.ok(
JSON.stringify(error).includes("JSON schema boundary cannot preserve"),
);
const value = await inspect(mode, { capture_path: path, format });
const reported =
format === "har"
? value.container.reported._extension
: value.records[0].reported.metadata;
assert.equal(Object.hasOwn(reported, "__proto__"), true);
assert.deepEqual(reported["__proto__"], { preserved: 7 });
assert.equal(Object.getPrototypeOf(reported), Object.prototype);
assert.equal(Reflect.get(Object.prototype, "preserved"), undefined);
if (format === "har") assert.equal(reported.constructor, "ordinary");
assert.deepEqual(await readFile(path), original);
cases++;
}
+291
View File
@@ -0,0 +1,291 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import {
copyFile,
lstat,
mkdir,
mkdtemp,
realpath,
rm,
writeFile,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import {
artifactCli,
artifactMcpResult,
withArtifactMcp,
} from "../../lib/artifact-e2e.mjs";
import {
buildMacosBundleFixture,
preflightMacosBundleFixture,
} from "../../fixtures/apple/macos-bundle.mjs";
const exec = promisify(execFile);
const args = process.argv.slice(2);
if (args.length !== 0 && (args.length !== 2 || args[0] !== "--stapled-app"))
throw new Error("Usage: verify:code-signature [--stapled-app PATH]");
const stapledApp = args[1];
if (stapledApp !== undefined) {
try {
await exec("/usr/bin/xcrun", ["--find", "stapler"]);
} catch (cause) {
throw new Error("Real stapled-ticket verification requires xcrun stapler", {
cause,
});
}
}
await preflightMacosBundleFixture();
const root = await mkdtemp(join(tmpdir(), "rea-code-signature-"));
const facets = (signature) =>
Object.fromEntries(
signature.security_facets.map(({ facet, state }) => [facet, state]),
);
/** Re-sign a copy of one fixture executable ad hoc with the given options. */
const signedCopy = async (source, name, options) => {
const path = join(root, name);
await copyFile(source, path);
await exec("/usr/bin/xcrun", [
"codesign",
"--force",
"--sign",
"-",
...options,
path,
]);
return path;
};
const ENTITLEMENTS = `<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>com.apple.security.cs.disable-library-validation</key><true/>
<key>com.apple.security.cs.allow-dyld-environment-variables</key><true/>
<key>com.apple.security.get-task-allow</key><true/>
<key>com.apple.security.cs.allow-jit</key><true/>
</dict></plist>
`;
try {
const { app } = await buildMacosBundleFixture(root);
let realTicket = {
status: "unverified",
reason:
"No known stapled app supplied; use --stapled-app PATH to exercise producer output.",
};
if (stapledApp !== undefined) {
const appPath = await realpath(stapledApp);
const oracle = await exec("/usr/bin/xcrun", [
"stapler",
"validate",
appPath,
]);
const observed = await artifactCli("inspect-signature", appPath);
assert.equal(
observed.stapled_ticket.status,
"present",
JSON.stringify(observed.stapled_ticket),
);
assert.equal(observed.stapled_ticket.reason, null);
assert.match(observed.stapled_ticket.sha256, /^[a-f0-9]{64}$/u);
await withArtifactMcp(appPath, async (client) => {
const viaMcp = await artifactMcpResult(client, "inspect_signature");
assert.deepEqual(viaMcp.stapled_ticket, observed.stapled_ticket);
});
realTicket = {
status: "verified",
selected_app: stapledApp,
ticket_sha256: observed.stapled_ticket.sha256,
size: observed.stapled_ticket.size,
oracle: "xcrun stapler validate",
stdout: oracle.stdout,
stderr: oracle.stderr,
};
}
const plain = await artifactCli("inspect-signature", app);
assert.deepEqual(plain.code_directory.flags.names, ["adhoc"]);
assert.equal(plain.verification.status, "valid");
// The opened bundle is reported by its canonical path.
assert.equal(plain.verification.path, await realpath(app));
assert.equal(plain.stapled_ticket.status, "absent");
assert.equal(plain.sealed_resources.status, "sealed");
assert.deepEqual(facets(plain), {
"library-validation": "not-enforced",
"dyld-environment-variables": "honored",
"debugger-attach": "allowed",
"executable-memory": "unrestricted",
"app-sandbox": "not-sandboxed",
});
// Deep verification reports the fixture's real nested code, all on disk.
assert.ok(plain.verification.validated_nested_code.length > 0);
const checkNestedProgress = async (signature) => {
for (const path of signature.verification.validated_nested_code) {
try {
await lstat(path);
} catch {
assert.ok(
signature.limitations.some((limitation) =>
limitation.includes(JSON.stringify(path)),
),
`Unconfirmed progress pathname must retain its limitation: ${path}`,
);
}
}
};
await checkNestedProgress(plain);
await withArtifactMcp(app, async (client) => {
const viaMcp = await artifactMcpResult(client, "inspect_signature");
assert.equal(viaMcp.verification.status, plain.verification.status);
assert.equal(viaMcp.verification.path, plain.verification.path);
await checkNestedProgress(viaMcp);
// Separate live codesign runs can interleave progress records differently.
// Compare the artifact's signature posture; check each capture independently.
const semantic = ({ provenance, verification, limitations, ...posture }) =>
posture;
assert.deepEqual(semantic(viaMcp), semantic(plain));
});
const tool = join(app, "Contents/Helpers/rea-tool");
const hardened = await artifactCli(
"inspect-signature",
await signedCopy(tool, "hardened", ["--options", "runtime"]),
);
assert.deepEqual(hardened.code_directory.flags.names, ["adhoc", "runtime"]);
assert.equal(hardened.stapled_ticket.status, "not-applicable");
assert.deepEqual(facets(hardened), {
"library-validation": "enforced",
"dyld-environment-variables": "ignored",
"debugger-attach": "blocked",
"executable-memory": "restricted",
"app-sandbox": "not-sandboxed",
});
const entitlements = join(root, "relaxed.entitlements");
await writeFile(entitlements, ENTITLEMENTS);
const relaxed = await artifactCli(
"inspect-signature",
await signedCopy(tool, "relaxed", [
"--options",
"runtime",
"--entitlements",
entitlements,
]),
);
assert.deepEqual(facets(relaxed), {
"library-validation": "disabled",
"dyld-environment-variables": "honored",
"debugger-attach": "allowed",
"executable-memory": "jit-allowed",
"app-sandbox": "not-sandboxed",
});
const flagged = await artifactCli(
"inspect-signature",
await signedCopy(tool, "flagged", ["--options", "library,restrict"]),
);
assert.deepEqual(flagged.code_directory.flags.names, [
"adhoc",
"restrict",
"library-validation",
]);
assert.equal(facets(flagged)["library-validation"], "enforced");
assert.equal(facets(flagged)["dyld-environment-variables"], "ignored");
const platform = await artifactCli("inspect-signature", "/usr/bin/true");
assert.notEqual(platform.code_directory.platform_identifier, null);
assert.equal(platform.verification.status, "valid");
assert.equal(facets(platform)["library-validation"], "enforced");
// The platform rules rest on a real `anchor apple` requirement check.
assert.ok(
platform.provenance.some(({ command }) =>
command.includes("-R=anchor apple"),
),
);
// Negative fixtures contain only the code needed for each rejection. A large
// nested-code graph can interleave codesign progress, making that capture
// inconclusive independently of the invalidity these checks exercise.
const minimalApp = async (name, includeNested) => {
const bundle = join(root, `${name}.app`);
const contents = join(bundle, "Contents");
await mkdir(join(contents, "MacOS"), { recursive: true });
await copyFile(tool, join(contents, "MacOS", "Fixture"));
await writeFile(
join(contents, "Info.plist"),
`<?xml version="1.0" encoding="UTF-8"?>
<plist version="1.0"><dict><key>CFBundleExecutable</key><string>Fixture</string>
<key>CFBundleIdentifier</key><string>com.example.rea.signature</string>
<key>CFBundlePackageType</key><string>APPL</string></dict></plist>`,
);
if (includeNested) {
await mkdir(join(contents, "XPCServices"));
await exec("/usr/bin/ditto", [
join(app, "Contents/XPCServices/Svc.xpc"),
join(contents, "XPCServices/Svc.xpc"),
]);
}
await exec("/usr/bin/xcrun", [
"codesign",
"--force",
"--deep",
"--sign",
"-",
bundle,
]);
assert.equal(
(await artifactCli("inspect-signature", bundle)).verification.status,
"valid",
);
return bundle;
};
const nested = await minimalApp("Nested", true);
await writeFile(
join(nested, "Contents/XPCServices/Svc.xpc/Contents/added.txt"),
"unsealed",
);
const nestedTamper = await artifactCli("inspect-signature", nested);
assert.equal(nestedTamper.verification.status, "invalid");
assert.ok(
nestedTamper.verification.diagnostics.some((line) =>
line.includes("Svc.xpc"),
),
JSON.stringify(nestedTamper.verification),
);
const mainTamper = await minimalApp("Tampered", false);
await mkdir(join(mainTamper, "Contents/Resources"));
await writeFile(join(mainTamper, "Contents/Resources/added.txt"), "unsealed");
const tampered = await artifactCli("inspect-signature", mainTamper);
assert.equal(tampered.verification.status, "invalid");
assert.ok(
tampered.verification.diagnostics.some((line) => line.includes("added")),
JSON.stringify(tampered.verification),
);
process.stdout.write(
`${JSON.stringify({
ok: true,
mocked: false,
cli: true,
stdio_mcp: true,
variants: [
"adhoc",
"runtime",
"runtime+entitlements",
"library,restrict",
],
platform_identifier: platform.code_directory.platform_identifier,
tampered_verification: tampered.verification.status,
nested_tamper_verification: nestedTamper.verification.status,
real_stapled_ticket: realTicket,
})}\n`,
);
} finally {
await rm(root, { recursive: true, force: true });
}
+4 -2
View File
@@ -79,7 +79,7 @@ export async function verifyCoreInspectionTrace({
(line) =>
/^execve\("\/(?:usr\/)?bin\/ps",/u.test(line) &&
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]',
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
) &&
line.endsWith(" = 0"),
);
@@ -128,7 +128,9 @@ export async function verifyCoreInspectionTrace({
const executable = await realpath(match[1]).catch(() => match[1]);
const ownership =
psPaths.has(match[1]) &&
line.includes('["ps", "-axo", "pid=,ppid=,pgid=,stat=,command="]');
line.includes(
'["ps", "-axo", "pid=,ppid=,pgid=,uid=,stat=,command="]',
);
const iconv =
iconvPaths.has(match[1]) && line.includes('["iconv", "-l"]');
const version =
@@ -3,8 +3,6 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "33"
tool_count: 139
catalog_digest: "4c5f67d076baf821b5cb6d6fb2cc0f6c274e96b627b85b892577760a6f71ab1a"
---
# REA
@@ -146,6 +144,13 @@ Start with the default result and use its inline Evidence and graph context.
Do not repeat an identical tool call. Make a focused follow-up only when the
returned result leaves a specific question unanswered.
If MCP reports `resource_constraint` with `details.resource: "transport"`, use
its reported same-session Evidence reference with a focused application tool,
or call `export_evidence_bundle` to write the complete session to a selected
path. The analysis remains complete in the ledger; avoid repeating it merely
to request the same oversized response. Complete CLI JSON output also streams
without a single MCP frame.
Every conclusion must distinguish observations, inferences, and unknowns. Cite
Evidence IDs, preserve limitations and incomplete coverage, and never imply
that static analysis observed execution. Runtime requests execute the declared
@@ -2,10 +2,17 @@
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. The complete result, graph, and Evidence context are returned
inline. Source-map contents are part of the static analysis. Start from its
inline when they fit the MCP frame. Source-map contents are part of the static analysis. Start from its
findings, coverage, unknowns, and graph context, then make a focused follow-up
only when a specific question remains unanswered.
An oversized response reports `resource_constraint` and
`details.resource: "transport"`. Reuse
`details.reported_limits.evidence_reference` with `trace_application_feature`
for the module or feature under investigation. `export_evidence_bundle` writes
the complete canonical session to a caller-selected file. Both workflows keep
the original Evidence and coverage; a broad follow-up can also exceed framing.
BrowserWindow preferences, preload and contextBridge surfaces, IPC
registrations, utility processes, and native binding requests are static syntax
observations. Only a unique exact literal IPC channel match supports an inferred
@@ -91,6 +91,12 @@ ASAR, or DMG inputs when the artifact graph and findings help answer the
question. It returns the complete artifact graph inline. Cite graph manifest
IDs when using them.
`inspect_signature` decodes CodeDirectory flags, runs local strict
verification, reports a stapled notarization ticket, and derives security
facets: library validation, `DYLD_*` handling, debugger attach, executable
memory, and sandbox. Cite each facet's `evidence`; system policy such as SIP and
AMFI is not evaluated.
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
`project_apple_application_graph`. It reports application roots and nested
bundles by path convention: app extensions, XPC services, frameworks, login
@@ -0,0 +1,52 @@
import { expect, it } from "vitest";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
isOwnedClientRegistrationCommand,
npxRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
import { setupRegistrationCommand } from "./SetupHost.js";
it("uses a Windows executable for npm-run setup and preserves POSIX launchers", () => {
expect(setupRegistrationCommand("win32", true)).toEqual([
"cmd.exe",
"/d",
"/c",
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
]);
for (const platform of ["linux", "darwin"] as const)
expect(setupRegistrationCommand(platform, true)).toEqual([
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
]);
});
it.each(
[
npxRegistrationCommand("win32"),
["cmd", "/c", "npx", "-y", "rea-agents@5.0.0", "mcp"],
["npx", "-y", "rea-agents@5.0.0", "mcp"],
].map((command) => ({ command })),
)("recognizes owned current and legacy launchers: $command", ({ command }) => {
expect(isOwnedClientRegistrationCommand(command)).toBe(true);
});
it.each(
[
["cmd.exe", "/d", "/c", "node", "custom.js", "mcp"],
["cmd.exe", "/k", "npx", "-y", "rea-agents@5.0.0", "mcp"],
["cmd.exe", "/c", "npx -y rea-agents@5.0.0 mcp"],
["cmd.exe", "/c", "npx", "-y", "another-package", "mcp"],
["cmd.exe", "/c", "npx", "-y", "rea-agents@5.0.0", "mcp", "&", "other"],
["cmd.exe", "/c", "npx", "-y", "rea-agents@5.0.0 & other", "mcp"],
].map((command) => ({ command })),
)(
"does not claim custom shell commands as REA-owned: $command",
({ command }) => {
expect(isOwnedClientRegistrationCommand(command)).toBe(false);
},
);
+28 -5
View File
@@ -3,6 +3,28 @@ import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
/** Resolve a pinned npx launcher that stdio clients can spawn on the host. */
export const npxRegistrationCommand = (
platform: NodeJS.Platform = process.platform,
): readonly string[] => {
const command = [
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
];
return platform === "win32" ? ["cmd.exe", "/d", "/c", ...command] : command;
};
const unwrapNpxCommand = (command: readonly string[]): readonly string[] => {
const executable = command[0]?.toLowerCase();
if (executable !== "cmd" && executable !== "cmd.exe") return command;
if (command[1]?.toLowerCase() === "/c") return command.slice(2);
if (command[1]?.toLowerCase() === "/d" && command[2]?.toLowerCase() === "/c")
return command.slice(3);
return [];
};
/** Check that one parsed command points to REA's MCP entry point. */
export const isOwnedClientRegistrationCommand = (
command: readonly string[],
@@ -27,15 +49,16 @@ export const isOwnedClientRegistrationCommand = (
return true;
}
const npxCommand = unwrapNpxCommand(command);
if (
command.length !== 4 ||
command[0] !== "npx" ||
command[1] !== "-y" ||
command[3] !== "mcp"
npxCommand.length !== 4 ||
npxCommand[0] !== "npx" ||
npxCommand[1] !== "-y" ||
npxCommand[3] !== "mcp"
)
return false;
const packageReference = command[2] ?? "";
const packageReference = npxCommand[2] ?? "";
if (
packageReference === PRODUCT_IDENTITY.packageName ||
packageReference === PRODUCT_IDENTITY.packageSpecifier ||
+14 -7
View File
@@ -9,7 +9,10 @@ import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import {
isOwnedClientRegistrationCommand,
npxRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
import { supportedClients } from "./SupportedClients.js";
import type { SetupClient } from "./SupportedClients.js";
@@ -108,7 +111,12 @@ export const readClientRegistrationStatuses = async (
client.name,
client.configPath,
command,
registrationAligned(registration, client, currentCommandPath)
registrationAligned(
registration,
client,
currentCommandPath,
options.platform ?? process.platform,
)
? "aligned"
: "stale",
),
@@ -132,6 +140,7 @@ const registrationAligned = (
registration: z.output<typeof registrationSchema>,
client: SetupClient,
currentCommandPath: string,
platform: NodeJS.Platform,
): boolean => {
const command = [registration.command, ...registration.args];
if (registration.disabled === true || registration.enabled === false)
@@ -160,12 +169,10 @@ const registrationAligned = (
resolve(command[1] ?? "") === currentCommandPath
)
return true;
const npxCommand = npxRegistrationCommand(platform);
if (
command.length === 4 &&
command[0] === "npx" &&
command[1] === "-y" &&
command[2] === PRODUCT_IDENTITY.registrationPackageSpecifier &&
command[3] === "mcp"
command.length === npxCommand.length &&
command.every((argument, index) => argument === npxCommand[index])
)
return true;
return (
+14 -40
View File
@@ -30,11 +30,7 @@ import {
snapshotEvidenceForQuery,
snapshotMatchesTarget,
} from "../domain/analysisSnapshot.js";
import {
analysisProfileSchema,
committedProviderSchema,
type AnalysisProfileCommitment,
} from "../domain/analysisProfile.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import { err, ok, type Result } from "../domain/result.js";
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import type {
@@ -45,6 +41,10 @@ import {
REA_WORKFLOW_PROVIDER,
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import {
createWorkflowEvidence,
workflowSnapshotRecord,
} from "./WorkflowEvidence.js";
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
@@ -268,35 +268,13 @@ const runAnalysis = async (
tool !== "trace_native_ui_action" &&
snapshotPath !== undefined &&
evidence !== undefined &&
"analysis_profile" in evidence &&
session.allowsSnapshotReplay(tool)
) {
const recorded = session.recordWorkflowSnapshot({
operation: tool,
parameters: arguments_,
execution: {
result: evidence.normalized_result,
rawResult: evidence.raw_result,
provider: committedProviderSchema.parse(evidence.provider),
analysisProfile: analysisProfileSchema.parse(
evidence.analysis_profile,
),
limitations: evidence.limitations,
locations: evidence.locations,
subject:
evidence.subject === null
? null
: {
path: evidence.subject.local_path,
sha256: evidence.subject.digest.sha256,
format: evidence.subject.format,
...(evidence.subject.architecture === null
? {}
: { architecture: evidence.subject.architecture }),
},
},
});
if (!recorded.ok) return cliError(recorded.error);
const workflowRecord = workflowSnapshotRecord(evidence, tool);
if (workflowRecord !== undefined) {
const recorded = session.recordWorkflowSnapshot(workflowRecord);
if (!recorded.ok) return cliError(recorded.error);
}
}
if (
tool !== "trace_native_ui_action" &&
@@ -356,15 +334,12 @@ const executeAnalysisTool = async (input: {
signal,
);
if (!result.ok) return { output: cliError(result.error) };
const evidence = createEvidence(input.openedTarget, REA_WORKFLOW_PROVIDER, {
const evidence = createWorkflowEvidence({
target: input.openedTarget,
operation: tool,
parameters: input.arguments,
result: result.value,
...(evidenceProfile === undefined
? {}
: { analysisProfile: evidenceProfile }),
confidence: "derived",
limitations: ["Derived by an REA composed workflow."],
upstreamProfile: evidenceProfile,
});
return { output: evidence, evidence };
}
@@ -493,8 +468,7 @@ const analysisProfileForEvidence = (
| DirectAnalysisTool,
): AnalysisProfileCommitment | undefined => {
if (!isWorkflowEvidenceTool(tool)) return session.analysisProfile(tool);
const upstream = session.analysisProfile();
return upstream === undefined ? undefined : workflowAnalysisProfile(upstream);
return session.analysisProfile();
};
const isWorkflowEvidenceTool = (
+2 -1
View File
@@ -26,7 +26,8 @@ export const createDoctorHostFixture = (
Promise.resolve({
version: PRODUCT_IDENTITY.skillVersion,
toolCount: CATALOG_IDENTITY.counts.mcp_tools,
catalogDigest: CATALOG_IDENTITY.digests.combined_sha256,
catalogDigest: null,
canonical: true,
}),
...overrides,
});
+1
View File
@@ -177,6 +177,7 @@ describe("doctor installation identity", () => {
version: "10",
toolCount: null,
catalogDigest: null,
canonical: false,
}),
}),
);
+7 -2
View File
@@ -14,6 +14,7 @@ import {
readLinuxDistribution,
type LinuxDistribution,
} from "./LinuxHopper.js";
import { canonicalSkillNeedsInstall } from "./SetupSkill.js";
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY, SDK_IDENTITY } from "../identity.js";
import {
@@ -95,11 +96,13 @@ export interface DoctorHost {
ilspyCmdVersion?(path: string): Promise<string | undefined>;
}
/** Parsed identity committed by an installed REA skill. */
/** Observed skill metadata and comparison with the packaged instruction bundle. */
interface InstalledSkillIdentity {
readonly version: string | null;
readonly toolCount: number | null;
readonly catalogDigest: string | null;
/** Whether all managed instruction and reference files match this package. */
readonly canonical: boolean;
}
/** Structured result returned by the read-only doctor workflow. */
@@ -139,6 +142,7 @@ interface DoctorIdentity {
readonly skill: {
readonly installed_version: string | null;
readonly installed_tool_count: number | null;
/** Legacy catalog digest observed in older bundles; current bundles omit it. */
readonly installed_catalog_digest: string | null;
readonly state: "aligned" | "stale" | "missing";
readonly remediation: string | null;
@@ -253,7 +257,7 @@ const skillIdentityAligned = (
): boolean =>
identity?.version === PRODUCT_IDENTITY.skillVersion &&
identity.toolCount === CATALOG_IDENTITY.counts.mcp_tools &&
identity.catalogDigest === CATALOG_IDENTITY.digests.combined_sha256;
identity.canonical;
const skillIdentityCheck = (
identity: InstalledSkillIdentity | undefined,
@@ -507,6 +511,7 @@ const installedSkillIdentity = async (
const content = await readInstalledSkill(home);
const countText = /^\s{2}tool_count:\s*(\d+)\s*$/mu.exec(content)?.[1];
return {
canonical: !(await canonicalSkillNeedsInstall(home)),
version: /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(content)?.[1] ?? null,
toolCount:
countText === undefined ? null : Number.parseInt(countText, 10),
+31 -11
View File
@@ -1,6 +1,7 @@
import {
describeEvidenceBundleFailure,
describeValidationFailure,
parseEvidenceBundle,
serializeEvidenceBundle,
type EvidenceBundle,
} from "../domain/evidenceBundle.js";
import {
@@ -9,7 +10,11 @@ import {
} from "../domain/evidenceErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { parseProcessCapture } from "../domain/process/processCapture.js";
import { readJsonFile, writeTextFile } from "./JsonFiles.js";
import {
bufferedJsonParts,
canonicalJsonParts,
} from "../domain/jsonSerialization.js";
import { readJsonFile, writeTextParts } from "./JsonFiles.js";
type EvidenceReadFailure = EvidenceFileError | EvidenceIntegrityError;
type EvidenceWriteFailure = EvidenceFileError | EvidenceIntegrityError;
@@ -20,20 +25,31 @@ export const readEvidenceBundle = async (
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> => {
const loaded = await readJsonFile(path);
if (!loaded.ok) return loaded;
let bundle: EvidenceBundle;
try {
const bundle = parseEvidenceBundle(loaded.value);
for (const record of bundle.records) {
if (record.predicate_type === "rea.process-capture")
parseProcessCapture(record.normalized_result);
}
return ok(bundle);
bundle = parseEvidenceBundle(loaded.value);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: describeEvidenceBundleFailure(loaded.value, cause),
}),
);
}
for (const record of bundle.records) {
if (record.predicate_type !== "rea.process-capture") continue;
try {
parseProcessCapture(record.normalized_result);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: `Process capture record ${record.evidence_id} has an invalid normalized_result (${describeValidationFailure(cause)}). Recreate or re-export the bundle, then try again.`,
}),
);
}
}
return ok(bundle);
};
/** Atomically write deterministic evidence JSON at the caller-supplied path. */
@@ -47,9 +63,9 @@ export const writeEvidenceBundle = async (
EvidenceWriteFailure
>
> => {
let encoded: string;
let checked: EvidenceBundle;
try {
encoded = serializeEvidenceBundle(bundle);
checked = parseEvidenceBundle(bundle);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
@@ -57,5 +73,9 @@ export const writeEvidenceBundle = async (
}),
);
}
return writeTextFile(encoded, path, overwrite);
return writeTextParts(
bufferedJsonParts(canonicalJsonParts(checked)),
path,
overwrite,
);
};
+82 -29
View File
@@ -1,8 +1,15 @@
import { link, lstat, mkdtemp, readFile, realpath, rm } from "node:fs/promises";
import {
link,
lstat,
mkdtemp,
open,
readFile,
realpath,
rename,
rm,
} from "node:fs/promises";
import { basename, dirname, resolve } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { EvidenceFileError } from "../domain/evidenceErrors.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -10,10 +17,14 @@ import { err, ok, type Result } from "../domain/result.js";
export const readJsonFile = async (
path: string,
): Promise<Result<unknown, EvidenceFileError>> => {
const requestedPath = resolve(path);
try {
const canonicalPath = await realpath(resolve(path));
const canonicalPath = await realpath(requestedPath);
const stats = await lstat(canonicalPath);
if (!stats.isFile()) return err(new EvidenceFileError("read", "not-file"));
if (!stats.isFile())
return err(
new EvidenceFileError("read", "not-file", { path: requestedPath }),
);
const encoded = await readFile(canonicalPath);
let decoded: unknown;
try {
@@ -23,11 +34,21 @@ export const readJsonFile = async (
),
);
} catch (cause: unknown) {
return err(new EvidenceFileError("read", "invalid-json", { cause }));
return err(
new EvidenceFileError("read", "invalid-json", {
cause,
path: requestedPath,
}),
);
}
return ok(decoded);
} catch (cause: unknown) {
return err(new EvidenceFileError("read", "io", { cause }));
return err(
new EvidenceFileError("read", missingOrIo(cause), {
cause,
path: requestedPath,
}),
);
}
};
@@ -38,10 +59,18 @@ export const writeTextFile = async (
overwrite: boolean,
): Promise<
Result<{ readonly path: string; readonly bytes: number }, EvidenceFileError>
> => writeTextParts([encoded], path, overwrite);
/** Publish complete streamed text atomically, keeping only one encoded part in memory. */
export const writeTextParts = async (
parts: Iterable<string>,
path: string,
overwrite: boolean,
): Promise<
Result<{ readonly path: string; readonly bytes: number }, EvidenceFileError>
> => {
const bytes = Buffer.byteLength(encoded, "utf8");
const requestedPath = resolve(path);
try {
const requestedPath = resolve(path);
const canonicalParent = await realpath(dirname(requestedPath));
const destination = resolve(canonicalParent, basename(requestedPath));
const existing = await lstat(destination).catch((cause: unknown) => {
@@ -49,41 +78,65 @@ export const writeTextFile = async (
throw cause;
});
if (existing !== undefined) {
if (!overwrite) return err(new EvidenceFileError("write", "exists"));
if (!overwrite)
return err(
new EvidenceFileError("write", "exists", { path: requestedPath }),
);
if (!existing.isFile() || existing.isSymbolicLink())
return err(new EvidenceFileError("write", "not-file"));
return err(
new EvidenceFileError("write", "not-file", { path: requestedPath }),
);
}
if (overwrite)
await writeFileAtomic(destination, encoded, {
encoding: "utf8",
mode: 0o600,
fsync: true,
});
else await publishNewFile(destination, encoded);
const bytes = await publishFile(destination, parts, overwrite);
return ok({ path: requestedPath, bytes });
} catch (cause: unknown) {
if (!overwrite && fileErrorCode(cause) === "EEXIST")
return err(new EvidenceFileError("write", "exists", { cause }));
return err(new EvidenceFileError("write", "io", { cause }));
return err(
new EvidenceFileError("write", "exists", {
cause,
path: requestedPath,
}),
);
return err(
new EvidenceFileError("write", missingOrIo(cause), {
cause,
path: requestedPath,
}),
);
}
};
const publishNewFile = async (
/** A missing path or parent is a selection error, not a permission failure. */
const missingOrIo = (cause: unknown): "missing" | "io" => {
const code = fileErrorCode(cause);
return code === "ENOENT" || code === "ENOTDIR" ? "missing" : "io";
};
const publishFile = async (
destination: string,
encoded: string,
): Promise<void> => {
parts: Iterable<string>,
overwrite: boolean,
): Promise<number> => {
const stagingDirectory = await mkdtemp(
resolve(dirname(destination), ".rea-write-"),
);
try {
const staged = resolve(stagingDirectory, "content");
await writeFileAtomic(staged, encoded, {
encoding: "utf8",
mode: 0o600,
fsync: true,
});
const file = await open(staged, "wx", 0o600);
let bytes = 0;
try {
for (const part of parts) {
await file.writeFile(part, { encoding: "utf8" });
bytes += Buffer.byteLength(part, "utf8");
}
await file.sync();
} finally {
await file.close();
}
if (overwrite) await rename(staged, destination);
// A hard link publishes complete bytes atomically and cannot replace an existing name.
await link(staged, destination);
else await link(staged, destination);
return bytes;
} finally {
await rm(stagingDirectory, { recursive: true, force: true });
}
+2 -6
View File
@@ -15,6 +15,7 @@ import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import type {
@@ -24,12 +25,7 @@ import type {
} from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js";
const defaultCommand = (): readonly string[] => [
"npx",
"-y",
PRODUCT_IDENTITY.registrationPackageSpecifier,
"mcp",
];
const defaultCommand = (): readonly string[] => npxRegistrationCommand();
/** Back up, atomically update, and semantically read back one JSON MCP configuration. */
export const configureJsonClient = (
+2 -1
View File
@@ -3,6 +3,7 @@ import { homedir } from "node:os";
import { resolve } from "node:path";
import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import {
SUPPORTED_NODE_VERSION_PROSE,
supportsNodeVersion,
@@ -35,7 +36,7 @@ export const setupRegistrationCommand = (
useNpmRunner: boolean = process.env.npm_command === "exec",
): readonly string[] =>
useNpmRunner
? PRODUCT_IDENTITY.mcpCommand.split(" ")
? npxRegistrationCommand(platform)
: platform === "win32"
? [
process.execPath,
+64
View File
@@ -0,0 +1,64 @@
import {
REA_WORKFLOW_PROVIDER,
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import type { AnalysisOperation } from "./AnalysisProvider.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import type { WorkflowSnapshotRecordInput } from "./binary/BinarySessionRecords.js";
import {
analysisProfileSchema,
committedProviderSchema,
} from "../domain/analysisProfile.js";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import type { JsonValue } from "../domain/jsonValue.js";
/** Build the shared Evidence record for a successful composed binary workflow. */
export const createWorkflowEvidence = (input: {
readonly target: BinaryTarget | undefined;
readonly operation: AnalysisOperation;
readonly parameters: Readonly<Record<string, JsonValue>>;
readonly result: JsonValue;
readonly upstreamProfile: AnalysisProfileCommitment | undefined;
}): Evidence =>
createEvidence(input.target, REA_WORKFLOW_PROVIDER, {
operation: input.operation,
parameters: input.parameters,
result: input.result,
...(input.upstreamProfile === undefined
? {}
: { analysisProfile: workflowAnalysisProfile(input.upstreamProfile) }),
confidence: "derived",
limitations: ["Derived by an REA composed workflow."],
});
/** Convert workflow Evidence into its exact replay binding, when it has a profile. */
export const workflowSnapshotRecord = (
evidence: Evidence,
operation: AnalysisOperation,
): WorkflowSnapshotRecordInput | undefined => {
if (!("analysis_profile" in evidence)) return undefined;
return {
operation,
parameters: evidence.parameters,
execution: {
result: evidence.normalized_result,
rawResult: evidence.raw_result,
provider: committedProviderSchema.parse(evidence.provider),
analysisProfile: analysisProfileSchema.parse(evidence.analysis_profile),
limitations: evidence.limitations,
locations: evidence.locations,
subject:
evidence.subject === null
? null
: {
path: evidence.subject.local_path,
sha256: evidence.subject.digest.sha256,
format: evidence.subject.format,
...(evidence.subject.architecture === null
? {}
: { architecture: evidence.subject.architecture }),
},
},
};
};
@@ -7,6 +7,8 @@ import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js"
import type { AnalysisProviderSelector } from "../../contracts/providerSelection.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { WorkflowSnapshotRecordInput } from "./BinarySessionRecords.js";
import type {
AnalysisOperation,
@@ -43,6 +45,10 @@ export interface BinarySessionPort
analysisProfile(
operation?: AnalysisOperation,
): AnalysisProfileCommitment | undefined;
allowsSnapshotReplay(operation: AnalysisOperation): boolean;
recordWorkflowSnapshot(
input: WorkflowSnapshotRecordInput,
): Result<null, EvidenceIntegrityError>;
openCompatibility(): Readonly<Record<string, JsonValue>>;
onAvailabilityChanged?(listener: () => void | Promise<void>): () => void;
onAnalysisSnapshotChanged?(listener: () => void | Promise<void>): () => void;
+23 -9
View File
@@ -3,6 +3,7 @@ import type { AnalysisSnapshot } from "../../domain/analysisSnapshot.js";
import type { BinaryTarget } from "../../domain/binaryTarget.js";
import type { Evidence } from "../../domain/evidence.js";
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { evidenceBundleForTarget } from "../../domain/evidenceBundle.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { type AnalysisError } from "../../domain/analysisErrorBase.js";
@@ -26,6 +27,15 @@ export interface ActiveAnalysisBinding {
readonly profile: AnalysisProfileCommitment | null;
}
/** Validated composed-workflow payload bound to the active target and profile. */
export interface WorkflowSnapshotRecordInput {
readonly operation: AnalysisOperation;
readonly parameters: Readonly<Record<string, JsonValue>>;
readonly execution: Parameters<
AnalysisSnapshotCache["recordWorkflow"]
>[0]["execution"];
}
/** Binary snapshot owner and compatibility facade for composed investigation records. */
export abstract class BinarySessionRecords {
readonly #records: InvestigationRecords;
@@ -59,10 +69,20 @@ export abstract class BinarySessionRecords {
return this.#records.evidenceById(evidenceId);
}
/** Borrow immutable investigation Evidence without copying a complete graph. */
evidenceForAnalysis(evidenceId: string): Evidence | undefined {
return this.#records.evidenceForAnalysis(evidenceId);
}
exportEvidenceBundle(): EvidenceBundle {
return this.#records.exportEvidenceBundle();
}
/** Borrow a sealed bundle for complete serialization without cloning retained graphs. */
evidenceBundleForSerialization(): EvidenceBundle {
return this.#records.evidenceBundleForSerialization();
}
importEvidenceBundle(
bundle: unknown,
): Result<number, EvidenceIntegrityError> {
@@ -161,15 +181,9 @@ export abstract class BinarySessionRecords {
}
/** Retain one derived workflow result alongside its provider cache entries. */
recordWorkflowSnapshot(input: {
readonly operation: string;
readonly parameters: Readonly<
Record<string, import("../../domain/jsonValue.js").JsonValue>
>;
readonly execution: Parameters<
AnalysisSnapshotCache["recordWorkflow"]
>[0]["execution"];
}): Result<null, EvidenceIntegrityError> {
recordWorkflowSnapshot(
input: WorkflowSnapshotRecordInput,
): Result<null, EvidenceIntegrityError> {
const active = this.activeAnalysisBinding();
if (active === undefined || active.profile === null)
return err(
@@ -116,6 +116,36 @@ describe("evidence ledger recording", () => {
ledger.clear();
expect(ledger.export().records).toEqual([]);
});
it("protects borrowed snapshots while preserving detached mutable public reads", () => {
const evidence = createEvidence(TARGET, PROVIDER, {
operation: "health",
parameters: {},
result: { nested: { value: "original" } },
});
const ledger = new EvidenceLedger();
expect(ledger.record(evidence).ok).toBe(true);
const borrowed = ledger.forAnalysis(evidence.evidence_id);
if (borrowed === undefined) throw new Error("Missing recorded Evidence");
expect(Object.isFrozen(borrowed)).toBe(true);
expect(Object.isFrozen(borrowed.normalized_result)).toBe(true);
expect(Reflect.set(borrowed, "operation", "forged")).toBe(false);
expect(ledger.forAnalysis(evidence.evidence_id)).toBe(borrowed);
expect(ledger.record(borrowed)).toEqual({ ok: true, value: "duplicate" });
const detached = ledger.get(evidence.evidence_id);
if (detached === undefined) throw new Error("Missing detached Evidence");
expect(Reflect.set(detached, "operation", "forged")).toBe(true);
expect(ledger.record(detached).ok).toBe(false);
expect(ledger.forAnalysis(evidence.evidence_id)).toBe(borrowed);
const snapshot = ledger.forSerialization();
expect(snapshot).toEqual(ledger.export());
expect(snapshot.records[0]).toBe(borrowed);
expect(Object.isFrozen(snapshot.records)).toBe(true);
ledger.clear();
expect(ledger.forAnalysis(evidence.evidence_id)).toBeUndefined();
expect(ledger.forSerialization().records).toEqual([]);
expect(snapshot.records).toEqual([evidence]);
});
});
describe("evidence ledger metadata imports", () => {
@@ -2,13 +2,19 @@ import canonicalize from "canonicalize";
import {
createEvidenceBundle,
createImmutableEvidenceBundle,
describeEvidenceBundleFailure,
parseEvidenceBundle,
validateResidualUnknownAddition,
type EvidenceBundle,
} from "../../domain/evidenceBundle.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import { parseEvidence, type Evidence } from "../../domain/evidence.js";
import {
immutableEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import {
createResidualUnknown,
updateResidualUnknown,
@@ -41,7 +47,7 @@ export class EvidenceLedger {
record(input: Evidence): RecordResult {
let evidence: Evidence;
try {
evidence = parseEvidence(input);
evidence = immutableEvidence(input);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Evidence record validation failed", {
@@ -51,6 +57,10 @@ export class EvidenceLedger {
}
const existing = this.#records.get(evidence.evidence_id);
if (existing !== undefined) {
// A borrowed record is already authenticated and deeply immutable. Its
// identity cannot change between reads, so comparing document-sized
// canonical strings again would add no integrity check.
if (existing === evidence) return ok("duplicate");
return recordsAgree(existing, evidence)
? ok("duplicate")
: err(new EvidenceIntegrityError("Conflicting evidence record"));
@@ -68,6 +78,7 @@ export class EvidenceLedger {
return err(
new EvidenceIntegrityError("Evidence bundle validation failed", {
cause,
userMessage: describeEvidenceBundleFailure(input, cause),
}),
);
}
@@ -95,8 +106,9 @@ export class EvidenceLedger {
additions.set(evidence.evidence_id, evidence);
}
for (const [id, evidence] of replacements)
this.#records.set(id, evidence);
for (const [id, evidence] of additions) this.#records.set(id, evidence);
this.#records.set(id, immutableEvidence(evidence));
for (const [id, evidence] of additions)
this.#records.set(id, immutableEvidence(evidence));
return ok({
recordsAdded: additions.size,
unknownsAdded: 0,
@@ -163,6 +175,14 @@ export class EvidenceLedger {
);
}
/** Borrow a sealed canonical bundle for serialization without copying its records. */
forSerialization(): EvidenceBundle {
return createImmutableEvidenceBundle(
[...this.#records.values()],
[...this.#unknownRevisions.values()],
);
}
/** Check whether one immutable Evidence ID is present in this ledger. */
has(evidenceId: string): boolean {
return this.#records.has(evidenceId);
@@ -174,6 +194,11 @@ export class EvidenceLedger {
return evidence === undefined ? undefined : structuredClone(evidence);
}
/** Borrow authenticated immutable Evidence for read-only analysis without a full clone. */
forAnalysis(evidenceId: string): Evidence | undefined {
return this.#records.get(evidenceId);
}
/** Create one approved unknown and atomically append its mutation evidence. */
recordUnknown(
input: RecordUnknownInput,
@@ -378,7 +403,8 @@ export class EvidenceLedger {
} catch (cause: unknown) {
return err(new UnknownRegistryError("integrity", { cause }));
}
for (const [id, evidence] of additions) this.#records.set(id, evidence);
for (const [id, evidence] of additions)
this.#records.set(id, immutableEvidence(evidence));
if (unknown !== undefined && unknownKey !== undefined) {
this.#unknownRevisions.set(unknownKey, unknown);
this.#unknownHeads.set(unknown.unknown_id, unknown);
@@ -405,7 +431,8 @@ export class EvidenceLedger {
unknowns: ReadonlyMap<string, ResidualUnknown>,
): void {
this.#records.clear();
for (const [id, evidence] of records) this.#records.set(id, evidence);
for (const [id, evidence] of records)
this.#records.set(id, immutableEvidence(evidence));
this.#unknownRevisions.clear();
this.#unknownHeads.clear();
for (const [key, unknown] of unknowns) {
@@ -15,6 +15,10 @@ import type { Result } from "../../domain/result.js";
export interface EvidenceReader {
hasEvidence(evidenceId: string): boolean;
evidenceById(evidenceId: string): Evidence | undefined;
/** Optional immutable analysis read; older adapters can keep detached reads. */
evidenceForAnalysis?(evidenceId: string): Evidence | undefined;
/** Optional immutable bundle snapshot for complete serialization without graph copies. */
evidenceBundleForSerialization?(): EvidenceBundle;
exportEvidenceBundle(): EvidenceBundle;
}
@@ -51,11 +51,21 @@ export class InvestigationRecords
return this.#evidence.get(evidenceId);
}
/** Borrow an immutable authenticated record for a read-only application workflow. */
evidenceForAnalysis(evidenceId: string): Evidence | undefined {
return this.#evidence.forAnalysis(evidenceId);
}
/** Export a detached deterministic bundle of Evidence and Unknown revisions. */
exportEvidenceBundle(): EvidenceBundle {
return this.#evidence.export();
}
/** Borrow a sealed bundle snapshot for complete JSON serialization. */
evidenceBundleForSerialization(): EvidenceBundle {
return this.#evidence.forSerialization();
}
/** Atomically merge a bundle and report changes for the owner's post-commit observers. */
mergeEvidenceBundle(
bundle: unknown,
@@ -8,6 +8,18 @@ import type {
JavaScriptApplicationAnalysisResult,
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
import { JAVASCRIPT_APPLICATION_PROVIDER } from "../InvestigationProviders.js";
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
import { rememberOwnedApplicationGraphEvidence } from "./JavaScriptApplicationEvidenceGraph.js";
/** Transfer an already validated, application-owned result into immutable analysis Evidence. */
export const createOwnedJavaScriptApplicationEvidence = (
input: AnalyzeJavaScriptApplicationInput,
result: JavaScriptApplicationAnalysisResult,
): Evidence =>
rememberOwnedApplicationGraphEvidence(
createJavaScriptApplicationEvidence(input, freezeJsonSnapshot(result)),
result,
);
/** Create Evidence for one deterministic local JavaScript application graph. */
export const createJavaScriptApplicationEvidence = (
@@ -0,0 +1,47 @@
import { describe, expect, it } from "vitest";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { parseEvidence } from "../../domain/evidence.js";
import {
analyzeJavaScriptApplicationInputSchema,
javascriptApplicationAnalysisResultSchema,
} from "../../domain/javascript/javascriptApplicationAnalysis.js";
import {
createJavaScriptApplicationEvidence,
createOwnedJavaScriptApplicationEvidence,
} from "./JavaScriptApplicationEvidence.js";
import {
parseApplicationGraphEvidence,
rememberOwnedApplicationGraphEvidence,
} from "./JavaScriptApplicationEvidenceGraph.js";
describe("owned application graph Evidence", () => {
it("reuses the exact authenticated graph and preserves ordinary wire identity", () => {
const result = javascriptApplicationAnalysisResultSchema.parse(
JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.normalized_result,
);
const input = analyzeJavaScriptApplicationInputSchema.parse({
input_path: result.input_path,
...JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.parameters,
});
const ordinary = createJavaScriptApplicationEvidence(input, result);
const owned = createOwnedJavaScriptApplicationEvidence(input, result);
expect(owned.evidence_id).toBe(ordinary.evidence_id);
expect(JSON.stringify(owned)).toBe(JSON.stringify(ordinary));
const parsed = parseApplicationGraphEvidence(owned);
expect(parsed.evidence).toBe(owned);
expect(parsed.graph).toBe(result.graph);
expect(parsed.semanticGraph).toBe(result.semantic_graph);
expect(Object.isFrozen(parsed.graph.nodes)).toBe(true);
expect(parseApplicationGraphEvidence(owned)).toBe(parsed);
const wire: unknown = JSON.parse(JSON.stringify(owned));
expect(parseApplicationGraphEvidence(wire)).toEqual(parsed);
expect(parseApplicationGraphEvidence(wire).graph).not.toBe(parsed.graph);
expect(() =>
rememberOwnedApplicationGraphEvidence(ordinary, result),
).toThrow("exact validated immutable result");
const tampered = { ...owned, normalized_result: {} };
expect(() => parseEvidence(Object.freeze(tampered))).toThrow();
expect(() => parseApplicationGraphEvidence(tampered)).toThrow();
});
});
@@ -3,7 +3,11 @@ import {
JAVASCRIPT_RUNTIME_RECONCILIATION_PROVIDER,
MANAGED_WORKFLOW_PROVIDER,
} from "../InvestigationProviders.js";
import { parseEvidence, type Evidence } from "../../domain/evidence.js";
import {
isImmutableEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import {
analyzeJavaScriptApplicationInputSchema,
javascriptApplicationAnalysisResultSchema,
@@ -26,11 +30,39 @@ export interface ApplicationGraphEvidenceSource {
readonly semanticGraph: JavaScriptSemanticGraph | null;
}
const ownedApplicationSources = new WeakMap<
Evidence,
ApplicationGraphEvidenceSource
>();
/** Retain the producer-validated meaning of its exact immutable result snapshot. */
export const rememberOwnedApplicationGraphEvidence = (
evidence: Evidence,
result: JavaScriptApplicationAnalysisResult,
): Evidence => {
if (
!isImmutableEvidence(evidence) ||
evidence.normalized_result !== result ||
evidence.operation !== "analyze_javascript_application" ||
!providerMatches(evidence, JAVASCRIPT_APPLICATION_PROVIDER)
)
throw new TypeError(
"Owned application Evidence must bind its exact validated immutable result",
);
ownedApplicationSources.set(
evidence,
Object.freeze(staticApplicationSourceForResult(evidence, result)),
);
return evidence;
};
/** Parse and authenticate one REA-produced JavaScript Application Graph Evidence. */
export const parseApplicationGraphEvidence = (
input: unknown,
): ApplicationGraphEvidenceSource => {
const evidence = parseEvidence(input);
const owned = ownedApplicationSources.get(evidence);
if (owned !== undefined) return owned;
if (
evidence.operation === "analyze_javascript_application" &&
evidence.predicate_type === "rea.javascript-application-analysis" &&
@@ -56,6 +88,16 @@ export const parseApplicationGraphEvidence = (
const staticApplicationSource = (
evidence: Evidence,
): ApplicationGraphEvidenceSource => {
const result = javascriptApplicationAnalysisResultSchema.parse(
evidence.normalized_result,
);
return staticApplicationSourceForResult(evidence, result);
};
const staticApplicationSourceForResult = (
evidence: Evidence,
result: JavaScriptApplicationAnalysisResult,
): ApplicationGraphEvidenceSource => {
if (
evidence.authority !== "shipped-artifact" ||
@@ -64,9 +106,6 @@ const staticApplicationSource = (
throw new TypeError(
"JavaScript application Evidence authority or confidence is invalid",
);
const result = javascriptApplicationAnalysisResultSchema.parse(
evidence.normalized_result,
);
if (evidence.predicate_type !== "rea.javascript-application-analysis")
throw new TypeError(
"JavaScript application Evidence predicate does not match its result shape",
@@ -16,7 +16,7 @@ import { projectInputIssues } from "../../domain/inputIssueProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import { createJavaScriptApplicationEvidence } from "./JavaScriptApplicationEvidence.js";
import { createOwnedJavaScriptApplicationEvidence } from "./JavaScriptApplicationEvidence.js";
import { reconstructJavaScriptArtifact } from "./JavaScriptArtifactReconstruction.js";
import { JAVASCRIPT_APPLICATION_PROVIDER } from "../InvestigationProviders.js";
@@ -77,7 +77,7 @@ export const analyzeJavaScriptApplicationValidated = async (
total: 1,
message: "Creating and hashing application analysis Evidence",
});
const evidence = createJavaScriptApplicationEvidence(input, result);
const evidence = createOwnedJavaScriptApplicationEvidence(input, result);
await options.progress?.report({
phase: "analyze_javascript_application",
completed: 1,
@@ -362,6 +362,35 @@ describe("contextual JavaScript package path resolution", () => {
});
});
describe("generic package entry heuristics", () => {
it.each([
['{"main":"actual.cjs","module":"bundler.mjs"}', "actual.cjs"],
['{"module":"bundler.mjs"}', "bundler.mjs"],
])(
"preserves generic entry heuristics without a module kind for %s",
(metadata, entry) => {
const files = fileMap([
file("app/consumer.js", "root"),
file("app/node_modules/fixture/package.json", "root", metadata),
file("app/node_modules/fixture/actual.cjs", "root"),
file("app/node_modules/fixture/bundler.mjs", "root"),
file("app/node_modules/fixture/index.js", "root"),
]);
expect(
resolve({
declaredPath: "fixture",
sourcePath: "app/consumer.js",
context: "module-specifier",
files,
}),
).toMatchObject({
resolution_status: "resolved",
resolved_path: `app/node_modules/fixture/${entry}`,
});
},
);
});
describe("directory package entrypoint precedence", () => {
it.each([
["require", '{"main":"actual.cjs"}', "actual.cjs"],
@@ -461,10 +461,12 @@ const packageEntry = (
? { ...exported, source: "exports" }
: exported;
}
const preferred =
moduleKind === "import"
? [Reflect.get(value, "module"), Reflect.get(value, "main")]
: [Reflect.get(value, "main"), Reflect.get(value, "module")];
// Node imports and requires ignore the bundler-only module field. Keep
// that fallback for callers without an explicit module-loading kind.
const preferred = [
Reflect.get(value, "main"),
...(moduleKind === undefined ? [Reflect.get(value, "module")] : []),
];
const entry = preferred.find((candidate) => candidate !== undefined);
if (entry === undefined) return { status: "missing" };
const legacy = packagePathValue(entry);
+7
View File
@@ -1,4 +1,5 @@
import { readFile } from "node:fs/promises";
import { z } from "zod";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
@@ -113,6 +114,12 @@ const parseCaptureEvidence = (input: unknown) => {
try {
evidence = parseEvidence(input);
} catch (cause: unknown) {
if (cause instanceof z.ZodError)
throw new AnalysisInputError(
"compare_process_captures",
{ cause },
projectInputIssues(cause.issues, input),
);
throw invalidCaptureEvidence();
}
if (
+15 -5
View File
@@ -21,7 +21,10 @@ import {
type ArtifactAnalysisOperation,
} from "../contracts/artifactToolContracts.js";
import type { BinaryTarget } from "../domain/binaryTarget.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
} from "../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../domain/artifactOperationError.js";
import { type AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -116,10 +119,15 @@ class ArtifactClient implements AnalysisClient {
parameters.path !== "." &&
parameters.path !== basename(this.target.path)
)
throw new ArtifactReaderFailure(
"path",
"For an active plist, path must select that archive (omit path or use its basename)",
);
throw new AnalysisInputError(operation, undefined, [
{
path: ["path"],
reason: "invalid_value",
message:
"For an active plist, path must select that archive (omit path or use its basename).",
expected: basename(this.target.path),
},
]);
const bundlePath = standalone
? dirname(this.target.path)
: this.target.sourcePath;
@@ -352,6 +360,8 @@ const translateFailure = (
cause.details,
cause.message,
);
// Caller-selection failures are already typed; keep their correction details.
if (cause instanceof AnalysisInputError) return cause;
return new ArtifactOperationError(operation, "io");
};
@@ -1,6 +1,14 @@
import { mkdir, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { collectInterfaceBuilderResourceKeys } from "./AppleAssetCatalogAnalysis.js";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
import {
analyzeAppleAssetCatalogs,
collectInterfaceBuilderResourceKeys,
} from "./AppleAssetCatalogAnalysis.js";
describe("Apple asset catalog application workflow", () => {
it("joins only explicit resource-key fields with archive provenance", () => {
@@ -40,4 +48,70 @@ describe("Apple asset catalog application workflow", () => {
},
]);
});
it("reports an app without compiled catalogs as an empty observed inventory", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-absent-");
await mkdir(join(bundle, "Contents", "Resources"), { recursive: true });
await writeFile(join(bundle, "Contents", "Resources", "icon.icns"), "");
const result = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => {
throw new Error("assetutil must not run without a catalog");
},
});
expect(result).toMatchObject({
catalogs: [],
total_records: 0,
records: [],
resource_key_matches: [],
next_offset: null,
truncated: false,
});
expect(result.limitations).toContain(
"No compiled Assets.car catalog was found among the app bundle's regular files (symlinks are not followed); any resource keys are reported as unmatched.",
);
});
it("does not report absence when the only catalog entry is a symlink", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-link-");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
await writeFile(join(bundle, "outside.car"), "catalog");
await symlink("../../outside.car", join(resources, "Assets.car"));
const rejected = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => Promise.resolve("[]"),
}).catch((cause: unknown) => cause);
expect(rejected).toBeInstanceOf(ArtifactReaderFailure);
expect(rejected).toMatchObject({
reason: "path",
message: expect.stringContaining(
"Contents/Resources/Assets.car (symlink)",
),
});
});
it("discloses skipped catalog entries beside inspected catalogs", async () => {
const bundle = await createTestTempDirectory("rea-asset-catalog-mixed-");
const resources = join(bundle, "Contents", "Resources");
await mkdir(join(resources, "Nested"), { recursive: true });
await writeFile(join(resources, "Assets.car"), "catalog");
await symlink("../Assets.car", join(resources, "Nested", "Assets.car"));
const result = await analyzeAppleAssetCatalogs({
bundlePath: bundle,
targetSha256: "a".repeat(64),
runAssetUtil: () => Promise.resolve("[]"),
});
expect(result.catalogs.map(({ path }) => path)).toEqual([
"Contents/Resources/Assets.car",
]);
expect(result.limitations).toContainEqual(
expect.stringContaining("Contents/Resources/Nested/Assets.car (symlink)"),
);
expect(result.limitations).not.toContainEqual(
expect.stringContaining("No compiled Assets.car catalog"),
);
});
});
@@ -48,10 +48,15 @@ export const analyzeAppleAssetCatalogs = async (input: {
sha256: string;
records: ReturnType<typeof parseAppleAssetCatalogRecords>;
}[] = [];
const uninspected: string[] = [];
let totalMetadataBytes = 0;
try {
for await (const entry of reader.entries(input.signal)) {
if (entry.kind !== "file" || !entry.path.endsWith("Assets.car")) continue;
if (!entry.path.endsWith("Assets.car")) continue;
if (entry.kind !== "file") {
uninspected.push(`${entry.path} (${entry.kind})`);
continue;
}
const inspected = await inspectAssetCatalogEntry({
reader,
entry,
@@ -69,10 +74,12 @@ export const analyzeAppleAssetCatalogs = async (input: {
} finally {
await reader.close();
}
if (catalogs.length === 0)
// Without any readable catalog, unmatched resource keys would overstate an
// absence that a skipped catalog entry may contradict.
if (catalogs.length === 0 && uninspected.length > 0)
throw new ArtifactReaderFailure(
"unavailable",
"No compiled Assets.car catalogs were found in the app bundle",
"path",
`Asset catalog entries are not regular files and were not inspected: ${uninspected.join(", ")}`,
);
const interfaceBuilder = await analyzeInterfaceBuilderBundle({
bundlePath: input.bundlePath,
@@ -93,6 +100,18 @@ export const analyzeAppleAssetCatalogs = async (input: {
...result,
limitations: [
...result.limitations,
// An app without a compiled catalog is an observed empty inventory, not
// a missing host tool.
...(catalogs.length === 0
? [
"No compiled Assets.car catalog was found among the app bundle's regular files (symlinks are not followed); any resource keys are reported as unmatched.",
]
: []),
...(uninspected.length === 0
? []
: [
`Asset catalog entries that are not regular files were not inspected, so resource keys reported as unmatched may resolve there: ${uninspected.join(", ")}`,
]),
"Exact resource-key joins use only recognized image or asset key fields on decoded Interface Builder resource objects; other archive resource references are not inferred.",
...(interfaceBuilder.graph.truncated
? [
+11 -1
View File
@@ -125,6 +125,16 @@ describe("inert keyed archive decoding", () => {
offset: 0,
limit: 1,
}),
).toThrow("does not exist");
).toThrow(
expect.objectContaining({
issues: [
expect.objectContaining({
path: ["root"],
message: expect.stringContaining("does not exist"),
expected: ["root", "other"],
}),
],
}),
);
});
});
+27 -10
View File
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { TextDecoder } from "node:util";
import { parseBinary } from "plist";
import { z } from "zod";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { projectPlistValue } from "../../domain/apple/plistValue.js";
import {
@@ -25,6 +26,10 @@ export const decodeKeyedArchiveBytes = (
) => {
if (bytes.length > MAX_BYTES)
throw new RangeError("Keyed archive exceeds 64 MiB");
if (bytes.subarray(0, 10).toString("ascii") === "NIBArchive")
throw new TypeError(
"Selected file is a compiled NIBArchive, not a Foundation plist archive; decode it with decode_interface_builder",
);
const binary = bytes.subarray(0, 8).toString("ascii") === "bplist00";
const parsed = binary
? { value: parseBinary(bytes), omittedPrototypeKeys: 0 }
@@ -83,18 +88,20 @@ export const inspectBundleKeyedArchive = async (input: {
.split("/")
.some((part) => part === ".." || part === "." || part.length === 0)
)
throw new ArtifactReaderFailure(
"path",
"Archive path must be a canonical relative bundle path",
throw archivePathError(
"invalid_format",
selected.path === "."
? "Select a keyed archive path relative to the active app bundle."
: `Archive path must be a canonical relative bundle path without empty, '.', or '..' segments: ${selected.path}`,
);
const reader = new DirectoryArtifactReader(input.bundlePath);
try {
for await (const entry of reader.entries(input.signal)) {
if (entry.path !== selected.path) continue;
if (entry.kind !== "file")
throw new ArtifactReaderFailure(
"path",
"Archive must be a regular file, not a symlink or directory",
throw archivePathError(
"invalid_value",
`Archive path selects a ${entry.kind}, not a regular file: ${selected.path}`,
);
if ((entry.declaredSize ?? 0) > MAX_BYTES)
throw new ArtifactReaderFailure(
@@ -132,18 +139,28 @@ export const inspectBundleKeyedArchive = async (input: {
...decodeKeyedArchiveBytes(bytes, selected),
});
} catch (cause) {
if (cause instanceof AnalysisInputError) throw cause;
throw new ArtifactReaderFailure(
cause instanceof RangeError ? "limit" : "format",
"Cannot decode selected Foundation keyed archive",
`Cannot decode selected Foundation keyed archive: ${cause instanceof Error ? cause.message : String(cause)}`,
{ cause },
);
}
}
throw new ArtifactReaderFailure(
"path",
`Archive not found in bundle: ${selected.path}`,
throw archivePathError(
"invalid_value",
`No regular file exists at ${selected.path} in the active app bundle.`,
);
} finally {
await reader.close();
}
};
/** A caller-selected archive path, not the artifact, failed its constraint. */
const archivePathError = (
reason: "invalid_format" | "invalid_value",
message: string,
) =>
new AnalysisInputError("inspect_keyed_archive", undefined, [
{ path: ["path"], reason, message },
]);
@@ -26,6 +26,7 @@ import {
type OwnedProcessGroup,
type ProcessCleanupResult,
type ProcessLineageObservation,
type OwnershipSweepUnverifiedProcess,
} from "../process/ProcessOwnership.js";
import {
observeOwnedProcessLineage,
@@ -238,6 +239,17 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
),
);
if (closeError !== undefined) return err(providerError(closeError));
if (outcome.ok && cleanup.unverified !== undefined)
outcome = ok({
...outcome.value,
limitations: [
...outcome.value.limitations,
...cleanup.unverified.map(
({ pid, diagnostic }) =>
`Ownership of unrelated process ${String(pid)} could not be verified; it was left untouched: ${diagnostic}`,
),
],
});
}
return outcome;
}
@@ -280,6 +292,7 @@ const cleanupElectronProcesses = async (
})),
);
let signaled = false;
const unverified: OwnershipSweepUnverifiedProcess[] = [];
for (const processGroupId of groupIds) {
const cleanupOwnership: OwnedProcessGroup =
processGroupId === ownership.processGroupId
@@ -292,14 +305,38 @@ const cleanupElectronProcesses = async (
const result = await cleanupOwnedProcessGroup(cleanupOwnership);
if (!result.cleaned) return result;
signaled ||= result.signaled;
unverified.push(...(result.unverified ?? []));
}
const remaining = await verifyNoTokenOwnedProcesses(
ownership.runId,
undefined,
ownership.captureBaseline,
{
leaderPid: ownership.leaderPid,
processGroupId: ownership.processGroupId,
...(ownership.sampledProcessGroupIds === undefined
? {}
: { sampledProcessGroupIds: ownership.sampledProcessGroupIds }),
},
);
if (!remaining.cleaned) return remaining;
return { cleaned: true, signaled };
unverified.push(...(remaining.unverified ?? []));
return {
cleaned: true,
signaled,
...(unverified.length === 0
? {}
: {
unverified: [
...new Map(
unverified.map((item) => [
`${String(item.pid)}:${item.diagnostic}`,
item,
]),
).values(),
],
}),
};
};
const createResult = (
@@ -1,5 +1,5 @@
import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, sep } from "node:path";
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
@@ -55,7 +55,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
"path",
`Selected source path changes during normalization: ${selected.content.relative_path}`,
);
const root = dirname(input.manifest_path);
const root = resolve(dirname(input.manifest_path));
const sourcePath = join(root, "files", portable);
targetPath = sourcePath;
await assertContained(root, sourcePath);
+18 -4
View File
@@ -161,14 +161,28 @@ it("rejects identical duplicate core fields before schema validation", () => {
expect(() => decodeHarCapture(text, [])).toThrow("duplicate");
});
it("reports an unsupported JSON schema boundary instead of dropping a producer property", () => {
it("preserves prototype-named producer members through the capture boundary", () => {
const text = JSON.stringify(historicalHar()).replace(
'"cache":{}',
'"cache":{},"_extension":{"__proto__":{"preserved":7},"constructor":"ordinary"}',
);
expect(() => decodeHarCapture(text, [])).toThrow(
"JSON schema boundary cannot preserve",
);
const result = decodeHarCapture(text, []);
const reported = result.records[0]?.reported;
expect(reported).toBeDefined();
if (
reported === undefined ||
reported === null ||
typeof reported !== "object" ||
Array.isArray(reported)
)
throw new Error("No HAR object record was retained");
const extension = reported["_extension"];
expect(extension).toEqual({
["__proto__"]: { preserved: 7 },
constructor: "ordinary",
});
expect(Object.getPrototypeOf(extension)).toBe(Object.prototype);
expect(Reflect.get(Object.prototype, "preserved")).toBeUndefined();
});
it("preserves Unicode, offsets, sizes, duplicate URLs and opaque extensions without inventing bytes", () => {
+3 -5
View File
@@ -58,11 +58,9 @@ export const parseHarJson = (text: string): unknown => {
});
},
onObjectProperty: (key) => {
if (key === "__proto__")
throw new CaptureFormatError(
"unsupported",
"HAR object contains a __proto__ member that the current JSON schema boundary cannot preserve.",
);
// Prototype-named members are kept as ordinary own properties: the
// members map keys them without prototype setters, and
// Object.fromEntries materializes them as own data properties.
const frame = frames.at(-1);
if (frame?.kind !== "object")
throw new TypeError(
+2 -1
View File
@@ -4,6 +4,7 @@ import { Cli } from "incur";
import { createLogger, parseLogLevel } from "./logger.js";
import { PRODUCT_IDENTITY } from "./identity.js";
import { npxRegistrationCommand } from "./application/ClientRegistrationIdentity.js";
import { registerSetupCommands } from "./cli/setupCommands.js";
import { registerCoreAnalysisCommands } from "./cli/coreAnalysisCommands.js";
import { registerUtilityCommands } from "./cli/utilityCommands.js";
@@ -47,7 +48,7 @@ export const createCli = (
version: PRODUCT_IDENTITY.packageVersion,
description: "Reverse engineer anything from your terminal or agent.",
mcp: {
command: PRODUCT_IDENTITY.mcpCommand,
command: npxRegistrationCommand().join(" "),
instructions:
"Ask what software, artifact, protocol, or behavior the user wants to understand, then choose the available investigation capabilities that can produce evidence.",
},
+5 -107
View File
@@ -1,5 +1,7 @@
import type { Writable } from "node:stream";
import { bufferedJsonParts, jsonParts } from "../domain/jsonSerialization.js";
type JsonOutputFormat = "json" | "jsonl";
/** Write ordinary JSON without constructing a document-sized string. */
@@ -8,7 +10,7 @@ export const writeJsonOutput = async (
destination: Writable,
format: JsonOutputFormat = "json",
): Promise<void> => {
for (const part of bufferedParts(documentParts(value, format)))
for (const part of bufferedJsonParts(documentParts(value, format)))
await writePart(destination, part);
};
@@ -39,115 +41,11 @@ function* documentParts(
): Generator<string> {
if (format === "jsonl" && Array.isArray(value)) {
for (const item of value) {
yield* encodeValue(item, false, 0, new Set());
yield* jsonParts(item);
yield "\n";
}
return;
}
yield* encodeValue(value, format === "json", 0, new Set());
yield* jsonParts(value, format === "json");
yield "\n";
}
function* bufferedParts(parts: Iterable<string>): Generator<string> {
let buffer = "";
for (const part of parts) {
if (buffer.length + part.length > 64 * 1024 && buffer !== "") {
yield buffer;
buffer = "";
}
buffer += part;
}
if (buffer !== "") yield buffer;
}
function* encodeString(value: string): Generator<string> {
if (value.length <= 8192) {
yield JSON.stringify(value);
return;
}
yield '"';
for (let start = 0; start < value.length;) {
let end = Math.min(start + 8192, value.length);
const last = value.charCodeAt(end - 1);
const next = value.charCodeAt(end);
// Keep a surrogate pair in the same JSON.stringify and UTF-8 write.
if (
end < value.length &&
last >= 0xd800 &&
last <= 0xdbff &&
next >= 0xdc00 &&
next <= 0xdfff
)
end -= 1;
yield JSON.stringify(value.slice(start, end)).slice(1, -1);
start = end;
}
yield '"';
}
function* encodeValue(
value: unknown,
pretty: boolean,
depth: number,
ancestors: Set<object>,
): Generator<string> {
if (typeof value === "string") {
yield* encodeString(value);
return;
}
if (value === null || typeof value === "boolean") {
yield value === null ? "null" : String(value);
return;
}
if (typeof value === "number" && Number.isFinite(value)) {
yield String(value);
return;
}
if (typeof value !== "object" || value === null)
throw new TypeError("CLI JSON output requires ordinary JSON values");
if (ancestors.has(value))
throw new TypeError("CLI JSON output contains a circular reference");
if (
!Array.isArray(value) &&
Object.prototype.toString.call(value) !== "[object Object]"
)
throw new TypeError("CLI JSON output requires ordinary JSON objects");
ancestors.add(value);
try {
const array = Array.isArray(value);
const keys = array
? undefined
: Object.keys(value).filter((key) => {
const item: unknown = Reflect.get(value, key);
// Incur's filter can select inherited methods. Match native JSON's
// omission of fields that have no JSON representation.
return (
item !== undefined &&
typeof item !== "function" &&
typeof item !== "symbol"
);
});
const count = array ? value.length : (keys?.length ?? 0);
yield array ? "[" : "{";
for (let index = 0; index < count; index += 1) {
if (index > 0) yield ",";
if (pretty) yield `\n${" ".repeat(depth + 1)}`;
let item: unknown;
if (array) {
item = value[index];
} else {
const key = keys?.[index];
if (key === undefined)
throw new TypeError("CLI JSON object changed during serialization");
yield* encodeString(key);
yield pretty ? ": " : ":";
item = Reflect.get(value, key);
}
yield* encodeValue(item, pretty, depth + 1, ancestors);
}
if (pretty && count > 0) yield `\n${" ".repeat(depth)}`;
yield array ? "]" : "}";
} finally {
ancestors.delete(value);
}
}
+9 -4
View File
@@ -37,9 +37,9 @@ export const parseCliJsonInput = async (
/**
* Resolve named string fields of CLI-supplied JSON against the operator
* working directory. Each entry is a key path from the document root; only
* fields holding strings are resolved, everything else passes through
* untouched. Shared MCP contracts reject relative paths, so CLI workflows
* resolve operator-relative values before validation.
* fields holding nonblank strings are resolved, everything else passes through
* untouched so shared schemas can reject blank paths. Shared MCP contracts reject
* relative paths, so CLI workflows resolve operator-relative values before validation.
*/
export const resolveCliJsonPaths = (
value: unknown,
@@ -61,7 +61,12 @@ const resolveJsonPath = (
([key, entry]: readonly [string, unknown]): readonly [string, unknown] => {
if (key !== head) return [key, entry];
if (tail.length === 0)
return [key, typeof entry === "string" ? resolve(entry) : entry];
return [
key,
typeof entry === "string" && entry.trim().length > 0
? resolve(entry)
: entry,
];
return [key, resolveJsonPath(entry, tail)];
},
);
+59
View File
@@ -0,0 +1,59 @@
import { resolve } from "node:path";
import { describe, expect, it } from "vitest";
import { resolveCliJsonPaths } from "./cliJsonInput.js";
import { electronActiveObservationInputSchema } from "./domain/javascript/electronActiveObservation.js";
const paths = [
["executable_path"],
["application_path"],
["application_root"],
] as const;
describe("CLI JSON path normalization", () => {
it.each([
["executable_path", ""],
["executable_path", " \t "],
["application_path", ""],
["application_path", " \t "],
["application_root", ""],
["application_root", " \t "],
])(
"keeps blank %s (%j) invalid at the shared input boundary",
(field, value) => {
const input = {
executable_path: "/synthetic/electron",
application_path: "/synthetic/app",
application_root: "/synthetic",
[field]: value,
};
const parsed = electronActiveObservationInputSchema.safeParse(
resolveCliJsonPaths(input, paths),
);
expect(parsed.success).toBe(false);
if (parsed.success) throw new Error("Expected blank path rejection");
expect(parsed.error.issues).toEqual(
expect.arrayContaining([expect.objectContaining({ path: [field] })]),
);
expect(input).toHaveProperty(field, value);
},
);
it("resolves selected relative paths without trimming names or changing unrelated fields", () => {
const input = {
browser: { executable_path: " browser ", label: "relative-label" },
application_path: "app",
};
expect(
resolveCliJsonPaths(input, [["browser", "executable_path"]]),
).toEqual({
browser: {
executable_path: resolve(" browser "),
label: "relative-label",
},
application_path: "app",
});
expect(input.browser.executable_path).toBe(" browser ");
});
});
+2
View File
@@ -4,9 +4,11 @@ import { isAbsolute } from "node:path";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { analysisProviderSelectorSchema } from "../contracts/providerSelection.js";
import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";
const environmentSchema = z.object({
REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),
REA_MCP_MAX_RESPONSE_BYTES: mcpResponseBudgetSchema.optional(),
REA_IDA_MCP_CONFIG: z
.string()
.min(1)
+31
View File
@@ -0,0 +1,31 @@
import { z } from "zod";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
// This override supports clients with larger frames. Keep the configuration
// parser independent of the SDK's eager server imports; conformance verifies
// this floor against the pinned SDK's actual default receive-buffer size.
const MINIMUM_RESPONSE_BYTES = 10 * 1024 * 1024;
/** Optional stdio response budget, including room for the protocol envelope. */
export const mcpResponseBudgetSchema = z
.string()
.regex(/^[1-9]\d*$/u)
.transform(Number)
.pipe(z.number().int().safe().min(MINIMUM_RESPONSE_BYTES));
/** Parse only the MCP-specific budget without loading unrelated provider settings. */
export const parseMcpResponseBudget = (
value: string | undefined,
): Result<number | undefined, ConfigurationError> => {
if (value === undefined) return ok(undefined);
const parsed = mcpResponseBudgetSchema.safeParse(value);
return parsed.success
? ok(parsed.data)
: err(
new ConfigurationError(
`REA_MCP_MAX_RESPONSE_BYTES must be a decimal safe integer at least ${String(MINIMUM_RESPONSE_BYTES)} bytes, the default stdio receive-buffer size.`,
),
);
};
+55
View File
@@ -0,0 +1,55 @@
import { describe, expect, it } from "vitest";
import { TOOL_CONTRACTS, toolContract } from "./toolContracts.js";
import {
toolInputSchemaWithMetadata,
toolOutputSchemaWithMetadata,
} from "./toolSchemaMetadata.js";
const target = "draft-2020-12";
describe("advertised tool JSON Schema", () => {
it("projects each canonical contract once per target", () => {
for (const contract of TOOL_CONTRACTS) {
const input = toolInputSchemaWithMetadata(contract)["~standard"];
const output = toolOutputSchemaWithMetadata(contract)["~standard"];
const advertisedInput = input.jsonSchema.input({ target });
const advertisedOutput = output.jsonSchema.output({ target });
expect(input.jsonSchema.input({ target }), contract.name).toBe(
advertisedInput,
);
expect(output.jsonSchema.output({ target }), contract.name).toBe(
advertisedOutput,
);
expect(advertisedInput.examples, contract.name).toEqual(
contract.examples.map(({ input: example }) => example),
);
expect(advertisedOutput, contract.name).toEqual(
contract.outputSchema["~standard"].jsonSchema.output({ target }),
);
}
});
it("keeps other targets and library options distinct", () => {
const contract = toolContract("open_binary");
const input = toolInputSchemaWithMetadata(contract)["~standard"];
const latest = input.jsonSchema.input({ target });
const draft07 = input.jsonSchema.input({ target: "draft-07" });
const libraryOptions = { unrepresentable: "any" };
const configured = input.jsonSchema.input({ target, libraryOptions });
expect(latest.$schema).toBe("https://json-schema.org/draft/2020-12/schema");
expect(draft07.$schema).toBe("http://json-schema.org/draft-07/schema#");
expect(input.jsonSchema.input({ target: "draft-07" })).toBe(draft07);
expect(configured).not.toBe(latest);
expect(input.jsonSchema.input({ target, libraryOptions })).not.toBe(
configured,
);
expect(
toolInputSchemaWithMetadata(contract).safeParse(
contract.examples[0]?.input,
).success,
).toBe(true);
});
});
+100 -13
View File
@@ -47,37 +47,124 @@ const PROPERTY_DESCRIPTIONS: Readonly<Record<string, string>> = {
unknown_id: "Exact residual-unknown identifier.",
};
type JsonSchemaProjection = z.ZodType["~standard"]["jsonSchema"]["input"];
/** Attach caller guidance to a canonical schema for the SDK wire projection. */
export const toolInputSchemaWithMetadata = <Contract extends ToolContract>(
contract: Contract,
): Contract["inputSchema"] => {
const schema = contract.inputSchema.meta(
z.globalRegistry.get(contract.inputSchema) ?? {},
): Contract["inputSchema"] =>
withAdvertisedJsonSchema(
contract.inputSchema,
"input",
(project) => (options) =>
flattenRootUnion({
...describeProperties(project(options)),
examples: contract.examples.map(({ input }) => input),
}),
);
const standard = schema["~standard"];
const inputJsonSchema = standard.jsonSchema?.input;
if (inputJsonSchema === undefined)
throw new TypeError(
"Tool input schema does not expose Standard JSON Schema",
);
/** Advertise a canonical output schema without reconverting it per listing. */
export const toolOutputSchemaWithMetadata = <Contract extends ToolContract>(
contract: Contract,
): Contract["outputSchema"] =>
withAdvertisedJsonSchema(contract.outputSchema, "output");
const withAdvertisedJsonSchema = <Schema extends z.ZodType>(
canonical: Schema,
io: "input" | "output",
advertise: (project: JsonSchemaProjection) => JsonSchemaProjection = (
project,
) => project,
): Schema => {
// Zod's input projection drops root metadata when a descendant transforms.
// Preserve the parser and let the SDK own conversion of everything else.
const schema = canonical.meta(z.globalRegistry.get(canonical) ?? {});
const standard = schema["~standard"];
const project = standard.jsonSchema?.[io];
if (project === undefined)
throw new TypeError(
`Tool ${io} schema does not expose Standard JSON Schema`,
);
Object.defineProperty(schema, "~standard", {
value: {
...standard,
jsonSchema: {
...standard.jsonSchema,
input: (options: Parameters<typeof inputJsonSchema>[0]) => ({
...describeProperties(inputJsonSchema(options)),
examples: contract.examples.map(({ input }) => input),
}),
[io]: memoizeByTarget(advertise(project)),
},
},
});
return schema;
};
// The SDK reconverts every registered schema on each `tools/list`, although a
// registered contract cannot change. Like the SDK's own memoized output
// projection, the advertised value is shared and treated as read-only.
const memoizeByTarget = (
project: JsonSchemaProjection,
): JsonSchemaProjection => {
const byTarget = new Map<string, Record<string, unknown>>();
return (options) => {
if (options.libraryOptions !== undefined) return project(options);
const cached = byTarget.get(options.target);
if (cached !== undefined) return cached;
const projected = project(options);
byTarget.set(options.target, projected);
return projected;
};
};
/** Keep MCP tool roots object-shaped for clients that reject root unions. */
const flattenRootUnion = (
value: Record<string, unknown>,
): Record<string, unknown> => {
if (!Array.isArray(value.anyOf)) return value;
const branches = value.anyOf.filter(isObject);
if (branches.length === 0 || branches.length !== value.anyOf.length)
return value;
const properties: Record<string, unknown> = {};
for (const branch of branches) {
if (!isObject(branch.properties)) return value;
for (const [name, schema] of Object.entries(branch.properties)) {
const previous = properties[name];
if (previous === undefined) properties[name] = schema;
else if (JSON.stringify(previous) !== JSON.stringify(schema))
properties[name] = {
anyOf: [previous, schema],
description: fallbackPropertyDescription(name),
};
}
}
// The canonical Zod parser still enforces branch validation at invocation.
const { anyOf: _anyOf, required: _required, ...root } = value;
const requiredByBranch = branches.map((branch) =>
Array.isArray(branch.required)
? branch.required.filter(
(item): item is string => typeof item === "string",
)
: [],
);
const required = requiredByBranch.reduce((common, current) =>
common.filter((name) => current.includes(name)),
);
const minProperties = Math.max(
typeof root.minProperties === "number" ? root.minProperties : 0,
Math.min(...requiredByBranch.map((names) => names.length)),
);
return {
...root,
type: "object",
properties,
...(minProperties > 0 ? { minProperties } : {}),
...(required.length > 0 ? { required } : {}),
...(branches.every((branch) => branch.additionalProperties === false)
? { additionalProperties: false }
: {}),
};
};
const describeProperties = (
value: Readonly<Record<string, unknown>>,
): Record<string, unknown> =>
@@ -12,7 +12,7 @@ export const WEB_NETWORK_CAPTURE_TOOL_CONTRACTS = [
name: "inspect_web_network_capture",
...toolContractMetadata("inspect_web_network_capture"),
description:
"Inspect retained local HAR 1.2 or native mitmproxy capture records without contacting recorded URLs. Returns original ordinals, pointers/byte ranges, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate HAR members are invalid; prototype-named __proto__ members are unsupported by the current shared JSON boundary. Uses pinned upstream parsers, owned temporary files and bounded offline processes. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.",
"Inspect retained local HAR 1.2 or native mitmproxy capture records without contacting recorded URLs. Returns original ordinals, pointers/byte ranges, preserved producer fields, precise numeric representations and exact exposed binary bytes inline. Excludes known transport credentials and explicitly marked values. Decodes every record before optional ordinal selection; malformed or oversized captures return no partial success. Duplicate HAR members are invalid; prototype-named members such as __proto__ are preserved as ordinary own JSON members without prototype mutation. Uses pinned upstream parsers, owned temporary files and bounded offline processes. Native mitmproxy requires caller-supplied mitmdump 12.2.3 on Linux. Browser transaction IDs, runtime attribution and original bytes of HAR Unicode text remain unknown.",
kind: "browser-provider",
inputSchema: inspectWebNetworkCaptureInputSchema,
outputSchema: evidenceResultOf(webNetworkCaptureSchema),
+2 -2
View File
@@ -17,13 +17,13 @@ export class AnalysisUnsupportedTargetError extends AnalysisError {
}
}
/** Provider reported a resource failure, distinct from malformed input or unsupported coverage. */
/** An analysis or transport resource failure, distinct from malformed input or unsupported coverage. */
export class AnalysisResourceConstraintError extends AnalysisError {
readonly _tag = "AnalysisResourceConstraintError";
constructor(
readonly operation: string,
readonly resource: "memory" | "cpu" | "file-size",
readonly resource: "memory" | "cpu" | "file-size" | "transport",
readonly reason: string,
readonly reportedLimits: Readonly<Record<string, JsonValue>> | null,
options?: AnalysisErrorOptions,
+16 -8
View File
@@ -37,11 +37,13 @@ export const analysisErrorRemediationAction = (
if (error instanceof AnalysisUnsupportedTargetError)
return "Select a target supported by this operation or choose an operation supporting the reported target format.";
if (error instanceof AnalysisResourceConstraintError)
return error.resource === "cpu"
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
: error.resource === "file-size"
? "Review the reported worker file-size limits and write failure. Retry with a sufficient file-size allowance for the evidence reply; REA retains tighter inherited limits."
: "Review the reported worker memory limits and available host memory. Retry with sufficient memory or a smaller artifact; REA retains tighter inherited limits.";
return error.resource === "transport"
? "Use the reported Evidence reference with a focused analysis tool when retained in this session, or export the session through export_evidence_bundle to a caller-selected path. Complete CLI JSON output is also available. The connection remains usable."
: error.resource === "cpu"
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
: error.resource === "file-size"
? "Review the reported worker file-size limits and write failure. Retry with a sufficient file-size allowance for the evidence reply; REA retains tighter inherited limits."
: "Review the reported worker memory limits and available host memory. Retry with sufficient memory or a smaller artifact; REA retains tighter inherited limits.";
if (error instanceof HopperTimeoutError)
return error.providerState === "busy"
? "Check binary_session.analysis_activity, wait for the active Hopper request to finish, then retry."
@@ -162,8 +164,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => {
: `Evidence ${error.evidenceId} does not match the requested reference (${error.reason}). Check the expected and actual identity in the diagnostic details.`;
if (error instanceof EvidenceIntegrityError)
return "Evidence is invalid or has changed. Recreate or re-import it, then try again.";
if (error instanceof EvidenceFileError)
return evidenceFileMessage(error.reason);
if (error instanceof EvidenceFileError) return evidenceFileMessage(error);
if (error instanceof UnknownRegistryError && error.reason === "not-found")
return "The requested residual unknown does not exist in this session. Check the unknown_id and try again.";
if (error instanceof UnknownRegistryError)
@@ -258,7 +259,14 @@ const artifactMessage = (reason: ArtifactOperationError["reason"]): string => {
return "Artifact could not be read or written. Check file access and try again.";
};
const evidenceFileMessage = (reason: EvidenceFileError["reason"]): string => {
const evidenceFileMessage = ({
operation,
reason,
}: EvidenceFileError): string => {
if (reason === "missing")
return operation === "read"
? "Evidence file does not exist at the selected path. Check the path and try again."
: "Evidence output directory does not exist. Choose an existing directory and try again.";
if (reason === "not-file")
return "Evidence path does not point to a regular file. Choose a file and try again.";
if (reason === "exists")
+6 -3
View File
@@ -62,6 +62,7 @@ it.each([
["cpu", { cpu_seconds: 1 }],
["file-size", null],
["file-size", { file_size_bytes: 1024 }],
["transport", { result_budget_bytes: 10485760 }],
] as const)(
"projects reported %s constraints with observed or unknown limits: %j",
(resource, limits) => {
@@ -89,9 +90,11 @@ it.each([
expect(projected.remediation.action).toContain(
resource === "cpu"
? "CPU"
: resource === "file-size"
? "file-size"
: "memory",
: resource === "transport"
? "Evidence reference"
: resource === "file-size"
? "file-size"
: "memory",
);
expect(projected.remediation.action).not.toContain("doctor");
expect(analysisErrorProjectionSchema.safeParse(projected).success).toBe(
+26 -11
View File
@@ -259,7 +259,11 @@ const artifactStateErrorDetails = (
};
if (error instanceof UnknownRegistryError) return { reason: error.reason };
if (error instanceof EvidenceFileError)
return { operation: error.operation, reason: error.reason };
return {
operation: error.operation,
reason: error.reason,
...(error.path === undefined ? {} : { path: error.path }),
};
return undefined;
};
@@ -389,16 +393,27 @@ const lifecycleErrorDetails = (
),
}),
};
if (
error._tag === "ProcessCaptureError" &&
error.userCategory === "cancelled"
)
return { operation: "process_capture", cleanup: "complete" };
if (
error._tag === "ProcessCaptureError" &&
error.executionFailure !== undefined
)
return { execution_failure: error.executionFailure };
if (error._tag === "ProcessCaptureError") {
const details = {
...(error.userCategory === "cancelled"
? { operation: "process_capture", cleanup: "complete" }
: {}),
...(error.cleanupReport === undefined
? {}
: { cleanup_report: jsonValueSchema.parse(error.cleanupReport) }),
...(error.executionFailure === undefined
? {}
: { execution_failure: error.executionFailure }),
...(error.partialObservation === undefined
? {}
: {
partial_observation: jsonValueSchema.parse(
error.partialObservation,
),
}),
};
return Object.keys(details).length === 0 ? undefined : details;
}
if (error instanceof BinaryTargetError)
return {
path: error.path,
Loaded 100 of 205 files, more files were not shown because too many files have changed in this diff. Show more