Compare commits

..
Author SHA1 Message Date
morluto 655b3eff6f docs(lifecycle): define completion invariants and regression scenarios
Document distinct ownership/readiness/exit facts, late cancellation and atomic publication boundaries, partial child cleanup, and diagnostic retention. Link representative regressions and route contributor guidance through AGENTS.md.

Refs: #4227
2026-10-16 04:30:05 +08:00
morluto e78455338f fix(cli): preserve analysis results when provider cleanup fails
Reuse the existing one-shot finalizer across thirteen command families to retain completed Evidence and primary diagnostics on close failure. Cover success, denied cleanup, and combined primary/cleanup failure through the public CLI.

Refs: #3935
2026-10-16 04:25:42 +08:00
morluto b6d5479ff0 fix(native): honor cancellation after trace finalization
Check cancellation after runtime-root cleanup and retain collected trace observations as partial cancellation evidence.

Refs: #4009
2026-10-16 04:25:25 +08:00
morluto 5f6342db46 fix(native): preserve failed LLDB launch ownership and exit proof
Publish failed-launch process identity and observed exit separately from object validity. Retire only the exact owned process and retain uncertainty when exit cannot be established.

Refs: #4232
2026-10-16 04:25:18 +08:00
morluto 02d92e9830 fix(process): honor cancellation through capture cleanup
Check the request signal after cleanup and before final publication, retaining completed observations under typed cancellation.

Refs: #4005
2026-10-16 04:25:13 +08:00
morluto c4c9a23bef fix(providers): retain composite lifecycle facts across retries
Preserve observed child health with incomplete peer coverage, retry only unresolved clients, and retain all child cleanup recovery actions.

Refs: #4233
2026-10-16 04:24:57 +08:00
morluto 8fd495bf13 fix(hopper): recover late startup and retained cleanup
Authenticate initial and cleanup handshakes through the same identity owner. Expose startup and cleanup activity, retain health after target retirement, and expire abandoned unauthenticated listeners without timing out authenticated work.

Refs: #4227
2026-10-16 04:24:52 +08:00
morluto 760233c0b3 fix(cleanup): preserve owner recovery instructions
Keep cleanup remediation through combined errors and direct or managed one-shot analysis while retaining primary diagnostics.

Refs: #4227
2026-10-16 04:24:45 +08:00
morluto bde139c5d9 test(windows): hold sharing conflict through first close attempt
Keep the delete-sharing-denied reader held until the next event-loop turn and assert close is still pending before releasing it. This distinguishes an internal retry from a holder that releases before cleanup starts.
2026-10-16 03:38:28 +08:00
8b96354497 fix(packaging): distinguish tar line endings from filename bytes
Parse one listing separator so CRLF output matches packaged bridge paths while literal CR bytes in LF listings remain part of filenames. Retain generated-bytecode rejection.

Refs #4154

Co-authored-by: abdooo2235 <abdalrhmantheiv15@gmail.com>
Co-authored-by: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-16 03:38:24 +08:00
morlutoandjackie_cqz c83de52762 fix(artifacts): classify ZIP offsets beyond EOF as malformed
Reject negative bounded reads before allocation while retaining short-read and exact-EOF behavior. Exercise the corrupted central-directory offset through the public CLI.

Refs #4006, #3953

Co-authored-by: jackie_cqz <88996311+jackie-cqz@users.noreply.github.com>
2026-10-16 03:38:20 +08:00
morlutoandVinícius DS Andrade 0ccc7572c7 fix(ida): share string-page failure interpretation
Preserve nullable producer errors in the shared page contract and reject explicit failures before returning matches. Retain both successful and failed replies as partial evidence.

Refs #4151, #3990

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-16 03:38:03 +08:00
98f22358e6 fix(sqlite): preserve snapshot admission failure semantics
Refuse oversized database and WAL inputs before acquisition with exact observed bytes and export guidance. Keep initial path failures separate from file-set changes and retain cleanup and partial observations on late growth.

Refs #4208, #4188, #4141, #4127

Co-authored-by: hima <1ep23ic023@eastpoint.ac.in>
Co-authored-by: Royce <roycelam@umich.edu>
2026-10-16 03:37:54 +08:00
morlutoandSwatantra Yadav 663d07f02c fix(server): retain session admission through investigation publication
Admit all four derived investigation workflows until Evidence recording settles, so close drains earlier work before clearing the ledger. Honor cancellation after awaited terminal progress before publishing a result.

Refs #4002, #4174

Co-authored-by: Swatantra Yadav <maverickswatantra@gmail.com>
2026-10-16 03:13:20 +08:00
morlutoandHarsH a2e30168a0 fix(mcp): relax Boolean literals without narrowing valid inputs
Normalize only advertised schema positions while keeping canonical validation. Preserve branch guidance and relax nonmonotone constraints as a unit. Verify full/compact catalogs and canonical examples after SDK conversion.

Refs #4025, #4087

Co-authored-by: HarsH <harshasalkar7@gmail.com>
2026-10-16 03:13:14 +08:00
morlutoandDeryFerd 0679a7972b fix(windows): retry owned runtime cleanup on sharing conflicts
Bound retries of native close conflicts at the retained runtime owner. Preserve exact failures and later retry authority after exhaustion. Attempt all verifier cleanup while retaining primary failures; extend held-file Windows verification.

Refs #4136, #4185

Co-authored-by: DeryFerd <deryferdikao125@gmail.com>
2026-10-16 03:13:10 +08:00
morlutoandArjun Prabhulal 8d8257928b fix(cli): validate integer tokens before coercion
Validate decimal tokens at the CLI owner before safe-number parsing. Apply the same rule to positional indexes/generations and repeated record selectors, retaining leading-zero decimals and canonical MCP inputs.

Refs #4182

Co-authored-by: Arjun Prabhulal <arjun.prabhulal@gmail.com>
2026-10-16 03:13:06 +08:00
morluto 08a897b2cd chore(ci): remove the redundant exact-test runner entry
The workflow action-pinning gate now discovers this runner through its explicit command, so Knip no longer needs a duplicate manual entry. Preserve its normal unused-code checks.
2026-10-16 02:48:45 +08:00
morluto 2ff086796a fix(browser): disclose connected platform-emulation teardown limits
Preserve the CDP detach limitation through successful and retained partial observations at their shared projector. Do not infer or replay unavailable caller user-agent/client-hint state.

This documents the reported Chromium platform reset; it does not prevent it. Source composition tests ran, without a new real-browser reproduction. Issue #4089 remains open.
2026-10-16 02:47:04 +08:00
morlutoandGabriel Dias 045c84db76 fix(pe): retain empty resources at virtual section endpoints
Map a zero-byte payload at a section endpoint only when no section contains that RVA, preserving adjacent-section ownership and raw backing checks. Extend the real CLI and stdio MCP workflow across PE32/PE32+ empty and nonempty resources.

The portable parser and public adapter regressions ran locally; Windows API loader behavior and GNU-produced images were not independently rerun.

Refs #3970

Co-authored-by: Gabriel Dias <gabrieldiasnevess@outlook.com>
2026-10-16 02:47:04 +08:00
morlutoandAdkid-Zephyr 6b1c36e336 fix(artifacts): normalize ZIP current-directory prefixes at the reader
Preserve original member spellings and central-directory identities while omitting actual root directory markers. Derive entry kind once so Unix symlink facts and size metadata agree; retain traversal and non-directory root rejection.

Verify public inspect/extract workflows for prefixed and root-only archives, distinct duplicate occurrences, and malformed/symlink controls.

Refs #4124

Co-authored-by: Adkid-Zephyr <laufs0310@163.com>
2026-10-16 02:47:04 +08:00
3406dc4d85 ci: pin Cutter actions and guard workflow-only changes (#4199)
Pin the remaining Cutter upload/download actions and enforce immutable GitHub/Docker references at actual action call sites. Run the same exact guard for workflow-only changes without preparing runtime artifacts.

Co-authored-by: Govindu Sasanka <31107592+GovinduSasanka@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 18:38:19 +00:00
morlutoandakushonkamen 5e30969cc5 fix(mcp): advertise self-contained tool inputs
Inline reused input schemas at their projection owner and remove the superseded reference-budget fallback. Validate actual SDK input and output schemas while preserving canonical parsing, output references, and projection cache options.

Claude Desktop and affected Windows connector behavior remain unverified; the SDK catalog has no schema-local input references.

Co-authored-by: akushonkamen <akushonkamen@163.com>
2026-10-16 02:23:25 +08:00
morlutoandgongfpp 5fb6edc34b fix(target): confine app program directories to their canonical bundle
Validate the canonical programs directory before resolving its executable, preserving containment failures even when the external child is missing. Retain in-bundle directory and bundle aliases.

Co-authored-by: gongfpp <22683417+gongfpp@users.noreply.github.com>
2026-10-16 02:23:25 +08:00
morlutoandArjun Prabhulal fb51960d7d fix(native): preserve Mach-O operand boundaries and integer limits
Consume signed and unsigned LEB operands through their terminators while validating 64-bit payloads and bounded extension. Preserve independent decoded metadata when a malformed stream fails.

Co-authored-by: Arjun Prabhulal <arjun.prabhulal@gmail.com>
2026-10-16 02:23:25 +08:00
claudetrail2026-droidandKaoru0822 1b828dbb10 ci(ghidra): compile the packaged bridge source cohort (#3142)
Compile the complete packaged Java source cohort against the verified SDK in selected CI. Share release and source ownership with runtime/package consumers, fail closed on drift, and remove compiler scratch on failure.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 18:11:19 +00:00
YsopwnableandKaoru0822 c7f6b8bad1 fix(javascript): bind native handoffs to canonical byte identity
Require the native node owner's content digest instead of borrowing an
observation artifact digest, which can identify the containing ASAR. Keep
unavailable nodes and their path/provenance/partial facts in the trace,
without a false actionable target or native Evidence link. Remove the old
fallback and allocate the query index only for eligible native targets.

Verify real public CLI/stdio MCP workflows, known/missing companion inputs
and shipped Joplin controls. Preserve reproducible scripts, source pins and
compact correctness/resource/token records without archiving applications.

Fixes #3926. Part of #4027.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 17:08:49 +00:00
ShuaixinHuang 633e4ff434 fix(javascript): retain static paths through TypeScript wrappers (#3998)
Normalize erased TypeScript wrappers through existing expression/string owners for static paths, call targets, collection/configuration receivers and global aliases. Preserve binding/mutation guards and explicit unknown targets, with CLI and application-boundary regressions.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
2026-10-16 01:04:24 +08:00
ShuaixinHuang d67d895c26 fix(jeb): isolate invalid timeout from shared startup (#3964)
Keep malformed JEB-only timeouts isolated from shared CLI/MCP startup while strict registration and JEB configuration checks remain authoritative. Cover built CLI discovery and the production stdio MCP refusal and peer requests.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
2026-10-16 01:03:39 +08:00
af6845cddb test(browser): preserve resource uncertainty through MCP (#3416)
Preserve resource uncertainty through production MCP registration and the real SDK. Test surviving resource fields/exclusions, forward/reverse/self comparisons, and complete-empty controls. Production behavior is unchanged.

Baron approved the unchanged head 07ea877a388ee2cd0586f04b1fb75bd104b484be; current required CI passes.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-16 01:01:18 +08:00
7b86d5caf6 fix(process): preserve unrelated trace comparison uncertainty (#4178)
Only apply a different trace verdict to its attributed dimensions. Preserve
unrelated unknown coverage and independently observed additions/removals in
the shared CLI/MCP comparison owner.

Fixes #3931

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-16 00:56:55 +08:00
Kaoru0822andClaude Opus 5.5 62bf09809e fix(kuna): answer a repeated single-function request without re-running Kuna (#4169)
* fix(kuna): answer a repeated single-function request without re-running Kuna

A session dropped each single-function run once it settled, so a repeat
request for the same procedure re-ran `kuna decompile --addr`, which
analyzes the whole image again. After a failed export every pseudocode
request uses that path for the rest of the session.

Keep each completed text by entry address, within a 16 MiB least recently
used budget, until the export's reader is adopted or the session closes.
Failed and cancelled runs are not kept. The export's text still supersedes
a kept one.

Refs #1957

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(kuna): pin that a kept text's hit refreshes its recency

With two 9 MiB texts only one fit the budget, so a repeated lookup could
not change which entry was dropped. Use three 6 MiB texts: re-reading main
leaves parse least recently used, so keeping broken must drop parse.

Refs #1957

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:36:05 -07:00
Kaoru0822andClaude Opus 5.5 8686b32209 fix(browser): omit erased type-only import-equals from source-map dependencies (#4165)
Source-map original-source parsing reported `import type X = require("m")`
as a require dependency, although TypeScript erases the declaration. The
fetcher re-derived import-equals targets instead of using the existing
value-only rule. That rule now lives with the other type-only module
declaration predicates in javascriptModuleDeclaration, and the fetcher,
static analysis, and semantic analysis all use it.

Fixes #3932

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:34:18 -07:00
Kaoru0822andClaude Opus 5.5 0cfced12ea fix(reference): resolve extensionless CommonJS requires to JSON files (#4164)
The reference-source resolver probed only JavaScript and TypeScript
extensions, so require("./data") stayed unresolved when the tree held
data.json, which Node's CommonJS loader selects. Resolution now uses
extension slots: a requires edge also tries X.json after X.js and
before any directory index, and X/index.json after X/index.js,
including a package main. ESM and TypeScript imports still need an
explicit .json extension.

Fixes #3915

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:34:00 -07:00
Kaoru0822andClaude Opus 5.5 d816470bcb fix(apple): count Interface Builder hierarchy records beyond the object limit (#4162)
The ibtool dictionary parser cut the hierarchy array at a fixed 20,000
entries without counting what it discarded, so an identified record past
that position vanished while the document still reported complete
hierarchy coverage. The graph walker already owns the hierarchy limit;
the parser now passes the decoded array through.

The walker applies max_objects per identified record and keeps walking
past the limit to count omitted records, instead of stopping on a
pending structural entry. Coverage now reports the omitted count, and a
hierarchy that exactly reaches the limit is no longer marked partial.

Fixes #3950

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:33:02 -07:00
Guilherme Dias FusariandKaoru0822 89faa863ed test(applications): import capture fixtures with file URLs (#3050)
Convert the selected filesystem path at the fixture execution boundary. Exercise spaces and hash characters without changing real execution, CLI/MCP comparisons or the test timeout. Verified the full Linux and native Windows files with real ESM-loader negative controls.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 16:18:04 +00:00
Guilherme Dias FusariandKaoru0822 5b16f24310 test(applications): pass Evidence requests to the CLI through files (#3049)
Use existing test-owned request files for the remaining full Evidence CLI inputs. Preserve complete inputs, semantic assertions and inline transport coverage. Verified the full Linux file and native Windows baseline failures plus corrected public CLI cases.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:51:03 +00:00
YsopwnableandKaoru0822 9f8659e053 fix(javascript): enforce semantic byte bounds for literals
Fixes #3933. Part of #4027.

Normalize exact literals and configuration defaults through the shared primitive owner. Preserve tagged limits through member/destructuring projection and record encountered bounds in per-analysis state independently of later selected values. Remove the retained-value rescan and duplicate default reader. Preserve source Evidence, bounded/null values, aggregate wire capacity and complete endpoint lookup identities for query/trace/comparison.

Validated 66 focused and 932 semantic tests on the restored identical runtime; final public selection 7/7 adds an augmented large-endpoint acceptance (67 distinct focused cases across invocations). Full static/docs and fresh real Obsidian controls passed. Compact measurements and rejected attempts remain recorded.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 23:43:05 +08:00
KenmegeandKaoru0822 40f73f62bc test(rizin): own fake provider cleanup in shared fixtures (#3735)
Retain native fixture ownership through observed bounded cleanup and preserve files and handles when cleanup cannot be confirmed. Verified all seven files on Linux and native macOS, including cleanup failure and exit-wait controls.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:24:55 +00:00
Ysopwnable 6ae59f3d19 fix(verifiers): launch npm through Node on Windows (#4153)
* fix(verifiers): launch npm through Node on Windows

* test(verifiers): cover Windows npm fallback
2026-10-15 23:12:29 +08:00
Kaoru0822andClaude Opus 5.5 9d6651e5b5 fix(artifacts): never treat a directory named *.map as a source map (#4131)
Artifact inventory decided source-map edges from the `.map` path suffix,
so an archive directory such as `main.js.map/` received a maps-source
edge whose parent was the sibling `main.js`, although the same
occurrence was classified as a directory container. Edge construction
now uses the occurrence's own artifact_kind, and the duplicate suffix
rule in relationFor is removed; such directories are contained by their
parent, and real source-map files keep maps-source.

Fixes #3909

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:41:04 -07:00
Kaoru0822andClaude Opus 5.5 a90c9eac6a fix(apple): keep Interface Builder objects whose archived class is unresolved (#4118)
* fix(apple): keep Interface Builder objects whose archived class is unresolved

Interface Builder keyed-archive projection skipped every object-table
entry without a resolved class name, so an archived object whose $class
reference dangled vanished from the objects, its count and its coverage,
which still reported complete; as a hierarchy root it lost its label.
Entries that carry their own $class are archived objects: they are now
kept with class_name null and the generic kind, their fields and labels
preserved. Scalars and class descriptors remain excluded.

Fixes #3958

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): keep the dangling class reference of an unknown IB object

Review follow-up. An archived object kept with an unresolved class lost
its original `$class` reference, because class resolution replaces it
with a null class name. The object's attributes now retain that raw
reference when the class cannot be resolved. A graph-level test covers
the reported hierarchy-root case: the unresolved root keeps its label,
unknown kind, containment edge, object count and coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:40:10 -07:00
Kaoru0822andClaude Opus 5.5 cfd3c35a50 fix(kuna): keep the real lane's wide-string marker independent of layout (#4116)
On macOS arm64 the lane's UTF-8 marker ends right before the UTF-16 array,
and its last byte plus NUL decode as the UTF-16LE unit 'r'. Kuna therefore
reports "rREA wide marker", and the lane, which requires the exact text,
failed with "Required string \"REA wide marker\" was not recovered". A
leading NUL unit now ends any run the preceding bytes start, so the marker
is recovered exactly on any layout.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:33 -07:00
Kaoru0822andClaude Opus 5.5 3776f218e5 fix(apple): bound Interface Builder XML plists by shared value depth (#4114)
* fix(apple): bound Interface Builder XML plists by shared value depth

The Interface Builder preflight rejected XML archives whose raw element
depth exceeded 128, but the shared plist parser bounds zero-based value
depth at 128; the plist envelope and dictionary keys add elements, not
value depth. Archives whose deepest value sat at depth 127 or 128 were
therefore omitted as over budget although keyed-archive inspection
decodes them. The preflight's duplicate depth rule is removed; the
shared parser's depth limit is mapped to the existing budget omission,
so deeper archives stay explicitly omitted.

Fixes #3949

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): return the decode reservation of a depth-rejected archive

Review follow-up. The removed element-depth preflight rejected an archive
before its plist representation was reserved; the shared depth limit now
rejects it during decoding, after the reservation. A large over-depth
archive therefore kept its share of the aggregate budget and could
starve later valid archives. A document omitted by the budget after
reserving now releases that reservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:07 -07:00
Kaoru0822andClaude Opus 5.5 f7250fe35f fix(kuna): say why Kuna cannot answer instead of a generic message (#4113)
Kuna capability gaps such as "no code at this address" or "no function node
at this address" were built without a user message, so MCP and the CLI
showed only "This analysis is unavailable for the current target. Choose
another analysis or target." and kept the reason in details. One Kuna
helper now presents the reason as the message for every such gap, and the
unsupported-operation reason names call relationships and xrefs, which Kuna
now answers.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:37:20 -07:00
Ysopwnable cd64e24ff2 fix(verifiers): align WABT operation timeout (#4117)
* fix(verifiers): align WABT MCP request timeout

* fix(verifiers): bound packaged WABT workflow
2026-10-15 22:29:15 +08:00
Abdul Samad SiddiquiandKaoru0822 a40ee741c2 test(ghidra): align MIPS verifier ABI and snapshot expectations
Fixes #3415. Retain independently inspected GNU attribute facts and compare exact snapshot wire fields. Preserve strict metadata comparisons, specific incomplete-ABI refusals and source immutability checks.

Original implementation by @AbdulSamad94.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 22:00:16 +08:00
211 changed files with 10537 additions and 1748 deletions

No files matched your search

+13
View File
@@ -60,6 +60,7 @@ jobs:
web_runtime: ${{ steps.plan.outputs.web_runtime }}
web_source_map: ${{ steps.plan.outputs.web_source_map }}
nativeaot: ${{ steps.plan.outputs.nativeaot }}
ghidra_bridge: ${{ steps.plan.outputs.ghidra_bridge }}
website: ${{ steps.plan.outputs.website }}
matrix: ${{ steps.plan.outputs.matrix }}
test_selection: ${{ steps.plan.outputs.test_selection }}
@@ -139,6 +140,11 @@ jobs:
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
# Code tiers run this source-only policy test in their existing shards.
# Workflow-only changes need the guard, without preparing runtime artifacts.
- name: Check action pins for workflow-only changes
if: needs.changes.outputs.workflows == 'true' && needs.changes.outputs.code != 'true'
run: node scripts/run-exact-tests.mjs run tests/boundary/filesystem/workflowActionPinning.test.ts
- name: Record static task source and execution identity
continue-on-error: true
if: needs.changes.outputs.code == 'true'
@@ -741,6 +747,7 @@ jobs:
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/boundary/process/packageNpmInvocation.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
@@ -898,6 +905,11 @@ jobs:
if: needs.changes.outputs.nativeaot == 'true'
uses: ./.github/workflows/nativeaot-fixtures.yml
ghidra_bridge:
needs: [changes, static]
if: needs.changes.outputs.ghidra_bridge == 'true'
uses: ./.github/workflows/ghidra-bridge-sources.yml
website:
needs: [changes, static]
if: needs.changes.outputs.website == 'true'
@@ -933,6 +945,7 @@ jobs:
web_runtime,
web_source_map,
nativeaot,
ghidra_bridge,
website,
]
runs-on: ubuntu-24.04
@@ -0,0 +1,48 @@
name: Ghidra bridge source compilation
on:
workflow_dispatch:
workflow_call:
permissions:
contents: read
env:
HUSKY: "0"
jobs:
ghidra-bridge-sources:
name: Compile packaged Ghidra bridge sources
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
GHIDRA_RELEASE_URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_12.1.4_build/ghidra_12.1.4_PUBLIC_20260921.zip
GHIDRA_RELEASE_SHA256: ddac49f903da9d5bac833e5cc79395098b9c33cfd3279be5f31bd00387d2d4db
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# Only the jars and application.properties are extracted: the check
# compiles against them and never launches Ghidra or imports a target.
- name: Acquire checksum-verified Ghidra jars in isolated runner storage
shell: bash
run: |
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
mkdir -p "${ghidra_root}"
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
unzip -q "${ghidra_root}/ghidra.zip" '*/Ghidra/*.jar' '*/Ghidra/application.properties' -d "${ghidra_root}"
rm "${ghidra_root}/ghidra.zip"
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
test -n "${properties}"
echo "GHIDRA_INSTALL_DIR=$(dirname "$(dirname "${properties}")")" >> "${GITHUB_ENV}"
# JAVA_HOME_<major>_X64 comes from the runner image, not the env context.
- name: Compile bridge sources with the verified runner JDK
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
+3 -3
View File
@@ -22,7 +22,7 @@ jobs:
sudo apt-get update
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
- run: npm run build:windows-native
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: cutter-windows-native
path: native/windows/build/
@@ -53,7 +53,7 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- if: matrix.platform == 'Windows'
uses: actions/download-artifact@v4
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: cutter-windows-native
path: native/windows/build/
@@ -67,7 +67,7 @@ jobs:
env:
EXPECTED_PLATFORM: ${{ matrix.platform }}
run: npm run verify:cutter
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: always()
with:
name: cutter-upstream-${{ matrix.platform }}
+2
View File
@@ -20,6 +20,8 @@ Give each interpretation and resource a clear owner. Consumers must use that own
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
+9 -1
View File
@@ -1057,6 +1057,10 @@ def _serve_connection(connection):
):
raise PermissionError("Invalid bridge capability")
authenticated = True
# The connection grace period only bounds unauthenticated idle
# clients. Native analysis and a connected owner's idle time
# remain unbounded after capability validation.
connection.settimeout(None)
_write_message(file, {"id": request_id, "event": {
"type": "progress",
"phase": "hopper_bridge",
@@ -1148,13 +1152,17 @@ def _run():
server.bind(REA_SOCKET)
os.chmod(REA_SOCKET, 0o600)
server.listen(1)
# A bridge can start after its caller's startup/cleanup deadline. Bound
# the first accept as well as reconnects so an orphan cannot pin Hopper's
# dedicated Python thread and prevent later documents from starting REA.
server.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
try:
while True:
try:
connection, _ = server.accept()
except socket.timeout:
break
connection.settimeout(None)
connection.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
if _serve_connection(connection):
break
# Match REA's cleanup handshake budget without pinning Hopper's
+48 -4
View File
@@ -555,6 +555,38 @@ def _failed_launch_description(process):
return None
def _retire_failed_launch(process, config, listener, stdout, stderr):
"""Reconcile an SBProcess returned alongside a failed Launch error."""
if not process.IsValid():
stdout.close()
stderr.close()
return False
pid = process.GetProcessID()
if pid > 0:
# Even an already-exited process needs its PID published so the host
# can distinguish a completed launch failure from an unknown target.
with open(config["pid_path"], "w") as handle:
handle.write(str(pid))
if process.GetState() != lldb.eStateExited:
stdout.mark_launched()
stderr.mark_launched()
if pid > 0:
# Hold a surviving target until the host has captured its start
# identity. If that proof is unavailable, Kill still addresses
# this exact SBProcess created by the current launch request.
_wait_for_identity_ack(config)
process.Kill()
_wait_for_exit(listener, process, 5)
exit_observed = process.GetState() == lldb.eStateExited
expected_exit = exit_observed
stdout.close(expected_exit=expected_exit)
stderr.close(expected_exit=expected_exit)
return exit_observed
def trace(debugger, config):
global _TRACE_CONTEXT
debugger.SetAsync(True)
@@ -630,8 +662,9 @@ def trace(debugger, config):
)
process = target.Launch(_launch_info(target, config, listener), error)
if not error.Success() or not process.IsValid():
stdout.close()
stderr.close()
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
return {
"status": "launch-error",
"error": error.GetCString(),
@@ -640,15 +673,26 @@ def trace(debugger, config):
if process.IsValid()
else {}
),
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
launch_failure = _failed_launch_description(process)
if launch_failure is not None:
stdout.close(expected_exit=True)
stderr.close(expected_exit=True)
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
return {
"status": "launch-error",
"error": launch_failure,
"target_process_created": True,
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
stdout.mark_launched()
stderr.mark_launched()
+132
View File
@@ -15,6 +15,8 @@ fake_lldb = sys.modules.setdefault("lldb", types.ModuleType("lldb"))
fake_lldb.LLDB_INVALID_ADDRESS = -1
fake_lldb.eLaunchFlagDebug = 2
fake_lldb.eLaunchFlagStopAtEntry = 4
fake_lldb.eStateExited = 10
fake_lldb.SBEvent = lambda: object()
spec = importlib.util.spec_from_file_location(
"rea_lldb_tracer", os.path.join(os.path.dirname(__file__), "rea_lldb_tracer.py")
)
@@ -23,6 +25,136 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return 6
def Kill(self):
return None
class Sink:
close_exit = None
def mark_launched(self):
pass
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
stdout, stderr = Sink(), Sink()
with patch.object(tracer, "_wait_for_identity_ack", return_value="unavailable"), patch.object(
tracer, "_wait_for_exit", return_value=None
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": os.path.join(directory, "pid")}, object(), stdout, stderr
)
self.assertFalse(exited)
self.assertFalse(stdout.close_exit)
self.assertFalse(stderr.close_exit)
def test_failed_launch_publishes_pid_and_retires_live_process(self):
class Process:
state = 6
killed = False
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return self.state
def Kill(self):
self.killed = True
self.state = tracer.lldb.eStateExited
class Sink:
launched = False
close_exit = None
def mark_launched(self):
self.launched = True
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
process = Process()
stdout, stderr = Sink(), Sink()
def acknowledge(config):
with open(config["pid_path"], "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
return "readable"
with patch.object(tracer, "_wait_for_identity_ack", acknowledge):
exited = tracer._retire_failed_launch(
process,
{"pid_path": pid_path},
object(),
stdout,
stderr,
)
self.assertTrue(exited)
self.assertTrue(process.killed)
self.assertTrue(stdout.launched)
self.assertTrue(stderr.launched)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_failed_launch_records_when_valid_process_is_already_exited(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return tracer.lldb.eStateExited
def Kill(self):
raise AssertionError("an exited process must not be killed")
class Sink:
close_exit = None
def mark_launched(self):
raise AssertionError("an exited process has no output to drain")
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
stdout, stderr = Sink(), Sink()
with patch.object(
tracer,
"_wait_for_identity_ack",
side_effect=AssertionError("exit is already observed"),
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": pid_path}, object(), stdout, stderr
)
with open(pid_path, "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
self.assertTrue(exited)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_pre_target_integrity_failure_reports_no_process_created(self):
original_debugger = getattr(tracer.lldb, "SBDebugger", None)
+22 -6
View File
@@ -635,6 +635,11 @@ bridge readiness. For large targets, set `REA_HOPPER_STARTUP_TIMEOUT_MS` to a
decimal integer from 1 to 2,147,483,647 milliseconds in the environment that
starts REA. CLI and MCP use the same setting; invalid values fail configuration
validation. An MCP client's own request timeout must also allow enough time.
`open_binary` validates and selects a target without waiting for native provider
readiness. During Hopper startup, `binary_session.provider_operation_health`
reports `busy` at stage `launch`; `analysis_activity` describes bridge requests
and can remain idle until the socket is ready. Cleanup work reports `busy` at
stage `connection`.
If macOS denies Apple Events, allow the application that starts REA under
**System Settings → Privacy & Security → Automation**, then retry. REA releases
@@ -644,9 +649,12 @@ Arguments containing quotes, backslashes, or line breaks remain unchanged but
cannot establish that a failed handoff opened no document. Other failed handoffs
retain ownership until document closure is confirmed. If a timed-out bridge
becomes ready later, `close_binary` connects to that run, authenticates its
identity, and requests document shutdown. After a connection drops, the bridge
allows 30 seconds for an authenticated cleanup reconnect, then releases Hopper's
Python execution thread. Expiry does not establish document closure, so REA
identity, and requests document shutdown. Once its listener is ready, the bridge
allows 30 seconds for the first connection and capability validation. After a
connection drops, it allows the same grace period for an authenticated cleanup
reconnect, then releases Hopper's Python execution thread. Initial analysis is
outside this listener grace period, and authenticated native work is not bounded
by it. Expiry does not establish document closure, so REA
retains unconfirmed ownership. Retry close while the bridge remains available.
If a Linux launcher exists but cannot start, inspect its shared libraries:
@@ -736,9 +744,17 @@ Once document and process closure are confirmed, the lease can be released even
if temporary-file cleanup fails. Another client can then launch, but the owning
client must finish its retained cleanup before starting again.
An unconfirmed external document can be retried while its authenticated bridge
remains connected. If that bridge has disconnected, another close cannot confirm
the document: inspect and close the reported document in Hopper before ending
the owning REA connection.
remains available. The same authentication and process-identity capture apply
when the first successful handshake occurs during cleanup after a startup
timeout. REA can confirm closure if that exact authenticated process exits,
including while a shutdown request is in flight. A helper exit alone is
insufficient, and an application that never authenticated has no verified
process identity. If no matching bridge becomes available, inspect and close
the reported document in Hopper before ending the owning REA connection.
Export the Evidence bundle before `close_binary` clears session records.
`binary_session.cleanup_pending` remains true while provider cleanup is retained,
and provider health remains visible even with `open: false`; that flag alone
does not establish successful cleanup.
### Hopper in CI
+11
View File
@@ -95,6 +95,14 @@ matches exactly. Otherwise the result recommends provider-neutral follow-up
tools and reports `requires-provider-analysis`; it never starts or switches a
provider implicitly.
A handoff requires the add-on's exact content identity. Unavailable unpacked
companions remain visible in the trace graph with their canonical path,
availability and container-backed Evidence. They yield no actionable handoff
until their bytes are available and the application is reanalyzed; the archive
digest does not identify the missing add-on. An empty handoff list does not prove
that the trace contains no native components. Source partial coverage remains
visible independently of handoff eligibility.
## Semantic relation tracing
`trace_javascript_semantics` queries the authenticated companion graph without
@@ -168,6 +176,9 @@ valid inputs.
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
lookup identities are preserved independently of the primitive-value lattice's
byte budget. An oversized normalized literal can become unknown while the same
string remains an exact endpoint for tracing and comparison. Endpoint
identity keys use `value-sha256:<digest>` over the canonical JSON string when
that is shorter, or when the original value starts with the reserved digest
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
+12 -1
View File
@@ -437,7 +437,18 @@ retained candidate strings and temporary canonical-key serialization. This is
an in-memory semantic allocation bound independent of CLI or MCP transport
budgets. Source literal bytes remain in the parsed source and artifact evidence;
when a normalized semantic value exceeds the bound, its value is unknown and
coverage is partial. Expression
coverage is partial. Direct string literals, cooked templates without
substitutions, and derived strings all use this normalization boundary. A
single exact primitive is validated without allocating deduplication or sort
state or serializing a canonical key. Configuration defaults use the same
normalizer; limited defaults retain `value_resource_limit` and an anchored
resource-limit unknown. Known null, false, zero, and empty-string defaults stay
exact. This primitive-value bound does not truncate lookup identities such as
source paths, field names or request endpoints. Full endpoint properties remain
available for endpoint queries, tracing and comparison. Each analysis records encountered limit classifications independently of
later projections: a selected healthy sibling can remain exact while whole-file
coverage reports the encountered bound. Projection preserves limiting values
from the shared property-slot owner. Expression
evaluation and provenance walks stop after 256 nested levels. Union products
and string lengths are checked before allocating combinations or concatenated
strings. When a boundary is reached, the value stays unknown, semantic coverage
+1
View File
@@ -48,6 +48,7 @@ The first text or assembly request in a session looks for this target's export i
- REA starts `kuna decompile-project` in the background on a private copy of the target.
- Meanwhile, pseudocode comes from one `kuna decompile --addr` run. Name and address lookups, and listings, use the inventory the export takes with `kuna functions --json` before it decompiles. The session runs `kuna functions` itself, under `REA_KUNA_TIMEOUT_MS`, only when no export is building (an inventory-only request does not start one) or the export ends without an inventory.
- Requests for the same function share one run; it stops when its last waiting request is cancelled. Each run analyzes the whole image again in its own process, so one REA process runs at most as many at once as the host has cores beyond `REA_KUNA_JOBS` (at least one). Further requests wait in order, and a waiting request is answered from the export if it finishes first. A run whose stop REA cannot prove keeps its slot until `close_binary` proves it stopped.
- The session keeps each completed single-function text, up to 16 MiB in all with the least recently used dropped first, and answers repeats from it until the export can. After a failed export that lasts for the session.
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
+25
View File
@@ -402,6 +402,13 @@ is `unavailable` with a reason when ownership could not be revalidated, or
at that bounded check. These are historical snapshots, not live process
inventories, and do not claim that no short-lived descendant existed.
`binary_session.cleanup_pending` reports whether the session retains a provider
client whose cleanup has not succeeded. It can be true with `open: false`;
target retirement does not establish resource release. Health, activity, and
lineage remain available from that retained client where the provider supplies
them. Follow the cleanup error's reported resources and recovery action before
reopening; a missing health observation does not establish an idle provider.
## Progress and cancellation
REA accepts ordinary `tools/call` progress tokens. The numeric `progress` counts
@@ -428,6 +435,10 @@ Native call tracing and process capture retain available observations in
`details.partial_observation` on failure, including when cleanup succeeds.
The observation reports its partial coverage; cleanup details describe host
state separately from the execution failure.
For these tools, cancellation observed during result finalization or owned
cleanup also returns typed cancellation with the available observations.
A normal target exit does not establish request completion. Incomplete cleanup
still takes precedence and retains its resource diagnostics.
A process capture that fails during a finalization interval keeps every signal
attempt and its delivery result in `details.partial_observation`, with
`elapsed_ms` and `exit_code` null when no exit was observed.
@@ -694,6 +705,20 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
Boolean literal constraints in input schemas are relaxed only in this
advertisement. For example, a boolean `const` is presented as `type: boolean`;
an affected `not` is omitted, conditional predicates and their branches are
omitted together, pure boolean-literal `oneOf` branches collapse to a boolean,
and other affected `oneOf` branches are presented as `anyOf` to retain branch
guidance without making overlapping widened branches reject inputs.
This can make the advertised schema accept inputs the canonical contract
rejects. The MCP handler still validates every call with the canonical
contract. The Gemini CLI verifier checks the outgoing
`parametersJsonSchema` and canonical contract examples against that declaration
using a loopback model fixture; it does not establish live Google API or
OpenCode client behavior. Set `REA_VERIFY_GEMINI_SCHEMA_PROFILE=compact` to
verify the compact advertisement through the same client boundary.
### Compact input schema profile
Some function-calling providers reject a request while any single tool's
+5
View File
@@ -451,6 +451,11 @@ The run ends when the process exits, `max_events` entries are recorded,
Bounded termination reports partial coverage. REA checks process termination
using a launch-time OS start identity; if ownership or cleanup cannot be
verified, the result or error preserves that uncertainty. The result reports:
Launch errors keep the fact that LLDB created a process separate from whether
it remains alive. An exited `SBProcess` is reported as exited only after LLDB
confirms that state; unresolved ownership remains retained for cleanup retry.
Cancellation observed while output and Evidence are finalized returns partial
cancellation Evidence, even if the target has already exited.
- the outcome, exit status and captured stdout/stderr (1 MiB kept per stream
while excess bytes are drained and counted without growing capture files).
+3 -1
View File
@@ -425,7 +425,9 @@ results for unaffected dimensions and mark affected dimensions unknown; an
incomplete capture cannot locate the first divergence across all dimensions.
Trace assertions use coverage for the sources they select, so an assertion
about complete raw terminal output need not fail because rendered snapshots
were omitted. Captures missing `truncation_details` are rejected on import;
were omitted. A trace difference applies to the dimensions identified by its
diagnostic; unrelated dimensions retain their own comparison, including unknown
coverage. Captures missing `truncation_details` are rejected on import;
preserve them as historical files and recapture for current comparisons.
`limits.file_bytes` is already the cumulative whole-file hashing budget,
independent of `limits.output_bytes`, which bounds retained terminal content.
+8
View File
@@ -105,6 +105,14 @@ overall native heap and total resident memory are not capped. Row limits do not
override these byte and resource limits. Resource-limit failures do not return
an incomplete schema as successful complete evidence.
If the combined database and WAL exceed the snapshot budget, the refusal is a
`resource_constraint` and reports both the configured byte limit and the exact
observed combined size as a decimal string. The remediation directs callers to
create a smaller consistent offline snapshot containing only relevant data
through the application's supported export workflow. A generic backup or copy
does not guarantee a smaller result; REA does not truncate or modify the
selected source.
MCP also applies its configured response budget to the complete Evidence
envelope. If delivery exceeds that budget, it reports a transport constraint
with guidance for exporting retained Evidence, requesting a matching larger
+51 -1
View File
@@ -199,6 +199,44 @@ Measure slow cases before pruning capacity coverage. Retain inputs beyond the
formerly failing size/depth and distinguish cold runs, warm builds, coverage,
and toolchain conditions when comparing timings.
## Lifecycle regressions
For changes to resource acquisition, readiness, cancellation, or cleanup, select
the affected owners and applicable scenarios below. Include temporary snapshots
and one-shot commands. Use public CLI/MCP workflows where practical and a
controlled production seam for interleavings that cannot be reproduced reliably.
| Scenario | Assertion and control |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
Synchronize cancellation with the finalization boundary being tested; a fixed
delay after dispatch does not establish that phase. Use markers, barriers, or
owned cleanup seams while preserving real cleanup where feasible. Check server
settlement and retained resources even if the MCP SDK rejects the client promise
before the handler finishes. A cancellation during active execution does not
cover cancellation during final cleanup.
For atomic writes, exercise cancellation on both sides of the documented commit
point and preserve any publication receipt. A late interruption does not prove
that no output was published.
Representative regressions include
[process cleanup-time SIGINT](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/process/processCaptureCancellationAdapters.test.ts),
[late authenticated cleanup](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/hopper/hopperClient.handshakeRecovery.test.ts),
[LLDB launch and finalization](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/native/lldbCallTracerFailures.test.ts),
[composed child ownership](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/src/application/binary/CompositeProvider.test.ts),
and [public CLI cleanup denial](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/cli/wasmCleanupCli.test.ts).
These controlled probes do not replace the matching real-provider lane. Record
the base failure, passing control, exact host/provider, verified cleanup, and
unavailable coverage; a missing engine is not a pass.
## Behavioral depths
| Depth | Location | What it proves |
@@ -989,7 +1027,19 @@ the Apple or Windows curated lanes invoke explicitly, which also select that
lane. Edits to verifier entrypoints run only by manual, licensed or self-hosted
workflows run static checks and the Linux suite. Website changes run website
checks. Provider workflow changes validate the workflow and run that provider's
verifier. Manual-provider/Android workflow changes run workflow validation only.
verifier. Changes under `bridge/ghidra/` also select the `ghidra_bridge` lane,
which compiles the packaged bridge sources against the pinned Ghidra 12.1.4
jars with `npm run verify:ghidra:bridge-sources`. It never launches Ghidra or
imports a target, so a missing import fails the PR instead of every Ghidra
session. The launcher, compiler and package checks share named required source
roles. The manifest must match the complete Java cohort, so deleting a helper
or adding an unregistered dependency cannot silently change the shipped set.
The preflight reuses the provider's Java Properties parser with the pinned Node
toolchain's native type stripping, without an npm install or runtime build.
The shared release policy owns the verified version and JDK major. CI checks
the archive pin before downloading and verifies the extracted SDK version;
changes to these source owners also select the compile lane.
Manual-provider/Android workflow changes run workflow validation only.
The four-version SQLite compatibility matrix has its own `sqlite` scope for
SQLite implementation, contracts, CLI/MCP registration, fixtures, and shared
runtime boundaries. SQLite and server-lifecycle test edits and the SQLite npm
+39
View File
@@ -68,6 +68,45 @@ satisfy. A plan without execution does not establish runtime behavior.
constraints. Report pagination, truncation, cancellation, and partial failure.
An assumed agent budget is insufficient justification for a limit.
## Own the lifecycle
For tools that acquire resources, identify the owner and the observations that
establish each transition. Include temporary files and one-shot commands, as
well as persistent engine sessions. Use the existing provider contracts;
providers need not expose identical state machines.
- Keep target admission, provider readiness, active work, observed process exit,
and resource release separate. A valid process handle, successful launcher,
or selected target establishes only its own fact. Publish available identity
and exit observations even when launch fails; retain unknown ownership.
- Authenticate and capture identity through the same boundary on initial and
recovery handshakes. Bound abandoned unauthenticated listeners where the
protocol permits it; that deadline must not expire authenticated idle sessions
or long-running native work. Listener expiry does not prove target closure.
- For cancellable observations, carry cancellation through result consumption
and awaited cleanup. Before returning success, check the signal after the
last awaited finalization step.
Retain collected observations under typed cancellation. Preserve an existing
primary failure and let incomplete cleanup retain its own classification.
Client cancellation alone does not prove native work stopped.
For atomic publication, document the cancellation cutoff and preserve any
committed output's publication receipt.
- Release only owned resources. Attaching owns a connection or session unless
the provider contract explicitly establishes target ownership. A helper exit,
missing reply, or missing PID does not establish target exit.
- Retain successful child cleanup and retry only unresolved children, including
when close options change. Preserve observed child failure or activity when
another child's health is unavailable; idle plus missing coverage stays
unknown.
- Await one-shot cleanup without discarding the completed result or primary
diagnostic. Preserve partial observations, remaining resources, and the cleanup
owner's recovery instructions. CLI adapters should reuse
[`finishAnalysisCommand`](https://github.com/morluto/rea/blob/main/src/cli/finishAnalysisCommand.ts) where its contract
applies rather than duplicate close/finally handling.
Exercise the applicable [lifecycle regression scenarios](testing.md#lifecycle-regressions)
through the producing owner and affected CLI/MCP consumers.
## Implement and evaluate
Put provider protocols/parsing in adapters and shared CLI/MCP workflows in the
@@ -0,0 +1,547 @@
{
"base_revision": "e9a71efa31d6c3675b67938b8a79fa4ddf01604c",
"scope": "Semantic per-value string-byte normalization; real static shipped-application control, no hosted-model performance claim",
"runtime": {
"platform": "Linux x64",
"node": "24.18.0",
"npm": "11.16.0",
"tiktoken": "0.12.0"
},
"baseline_source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "df3fc30b5ba91901a85fe56f200355930da32d054bb026a747e3414af233b6b5",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "c12b29f0faf4b6511777cba7ed8ac007c6d6170d657f01e43e194feacb3e0601",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"final_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/acceptance/applications/javascriptEndpointPayload.test.ts": "b3cfcde1c5b5e0698f84afd51f4b53e42e56101267707c20c0d038320f5cfbd2",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"selected_source_paths": ["package.json", "main.js"],
"where": {
"path": "main.js",
"kind": "javascript-module"
},
"identical_parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
},
"unchanged_query_shapes": {
"legacy": {
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
"references": {
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
},
"unchanged_token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"legacy": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 2836,
"structured_replies": 71761,
"selected_catalog_task_total": 85302,
"full_catalog_task_total": 272847
},
"references": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204093
}
}
},
"cohorts": {
"baseline": {
"preparation_ms": 19089.119622,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 383.8260569999984
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.778886999996757
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 13.15173700000014
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 240.44363500000327
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 346.04796499999793
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 13.727306999997381
}
],
"resources": {
"elapsed_seconds": 33.394,
"cpu_seconds": 33.016802,
"peak_cgroup_bytes": 1979863040,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"final": {
"preparation_ms": 19061.292544,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 344.6316079999997
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 14.438351000000694
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.38763100000142
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 298.41579600000114
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 211.45682099999976
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.275370999999723
}
],
"resources": {
"elapsed_seconds": 32.949,
"cpu_seconds": 32.559187,
"peak_cgroup_bytes": 1880096768,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
},
"conditions": "First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children. Before/after used separate fresh MCP sessions. CPU/memory includes analysis, query arms, export, CLI parity and tokenization; cgroup memory includes file cache, not summed RSS. Counters are at payload exit while observer lives, not final scope counters. One before/after control establishes no speedup or equivalence bound. Catalog BPE projection excludes annotations/output schemas.",
"resource_policy": {
"memory_max_bytes": 8589934592,
"early_stop_bytes": 6442450944,
"cpu_quota_percent": 100,
"swap": false
},
"observer_sources_sha256": {
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
},
"regressions": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed across five files on the restored 38d263a runtime. Final corrected public selection passed 7/7 in two files: the six literal/default/member cases plus the augmented original large-endpoint acceptance. 67 distinct focused cases across invocations; not a single 67-case invocation.",
"final_semantic_suite": "932 passed in 36 files on 38d263a. Final production src tree (including these domain tests) verified byte-identical to that revision; no repeated semantic run claimed.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_contract_correction": "Restored exact endpoint lookup identities rather than shrinking the existing large-endpoint fixture. Added the identical oversized string as an exported primitive in that acceptance workflow: literal normalization must report its bound while full endpoint lookup/feature trace/semantic trace/version comparison continue to work. Production src tree restored byte-identical to 38d263a; 932 semantic tests from that state remain applicable. Final corrected public tests/gates recorded after execution.",
"final_corrected_gates": "Full static gate and docs gate (5 consistency tests) passed after endpoint identity restoration and strengthened unchanged-large-input acceptance. Production src remains byte-identical to 38d263a; final public 7/7 and fresh real control passed."
},
"prior_attempts": [
{
"stage": "Initial literal normalization, incomplete for member/default paths identified in review",
"revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"real_control_cohort": {
"preparation_ms": 18876.764036,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 346.7870870000006
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 18.34765800000241
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.56626999999935
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 227.17703500000061
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 251.5148330000011
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.888227999999799
}
],
"resources": {
"elapsed_seconds": 31.347,
"cpu_seconds": 31.05517,
"peak_cgroup_bytes": 1589915648,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"baseline_public": "Quoted and cooked-template cases retained 200000-unit normalized literals and failed; addition and small exact controls passed. Aggregate-capacity test passed.",
"baseline_domain_corrected": "2 over-bound failures, 16 passed. Initial combined-run domain assertions had incorrectly required an empty optional resourceLimits array; corrected before using the domain reproduction as evidence.",
"fixed_focused": "55 passed across five files.",
"semantic_suite": "923 passed across 36 files.",
"capacity_after_fixture_cleanup": "1 passed; all 16 bounded values, aggregate size >= original, typed identities and retained tracing preserved.",
"interim_static_failure": "Typecheck rejected resume() on the new test SDK stderr Stream declaration. Switched to its declared data event, matching existing typed tests; analyzer source unchanged.",
"public_boundary_after_stream_type_fix": "4 passed across actual CLI and stdio MCP; analyzer source unchanged."
}
},
{
"stage": "Projection/default correction; final review subsequently found unnormalized request endpoints",
"revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19599.643274,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 396.9265450000021
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.658120000000054
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.24905100000251
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 237.03392900000108
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 357.78818299999693
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 14.37622100000226
}
],
"resources": {
"elapsed_seconds": 33.969,
"cpu_seconds": 33.499223,
"peak_cgroup_bytes": 1893015552,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed in five files.",
"final_semantic_suite": "932 passed in 36 files.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt."
}
},
{
"stage": "Rejected endpoint normalization: violated existing full-endpoint lookup/trace contract; reverted after CI acceptance failure",
"revision": "580887209c4fceda432eac4083fac82a0dd78f15",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "a648f97e631985edcb5259ff4f1fdf0332dca18b8c10b57e050e36a785c1c524",
"src/domain/javascript/javascriptSemanticAnalysis.requestBudget.test.ts": "8e84ffa70d834f5589d04118f0aaf33b0383b15ea97545ff69c66a74473aca73",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "eed961e0b1d8a320e76bbba933408e950b4a4a446110401e8cf470f1e74d8315",
"src/domain/javascript/javascriptSemanticIr.ts": "3b2ede73dbcfc74c4014e4d1affcbf00ecccc29a422ed495a7a3f2f517056c9b",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "61319ea954de7f256841f43ca9355a1706ce8e93d30f2a8ba851d5e717bd19e7",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19939.160272,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 348.3974309999976
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 13.97052899999835
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.060379000002285
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 212.01217800000086
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 223.73954800000138
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.450989999997546
}
],
"resources": {
"elapsed_seconds": 33.062,
"cpu_seconds": 32.71624,
"peak_cgroup_bytes": 1897406464,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "81 passed in six files; the same CLI/MCP file then passed 9/9 after adding a linked-response case. That follow-up replaces its earlier 8-case coverage; 82 distinct cases across the final six-file selection, not one 82-case invocation.",
"final_semantic_suite": "945 passed in 37 files, overlapping focused domain coverage.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt.",
"endpoint_typecheck_attempt": "Full static gate found the inferred optional dataField in the public test computed key. Replaced the in-property check with an explicit bare === true discriminant; scenario inputs and runtime implementation unchanged. Corrected static/public checks recorded separately.",
"final_gates": "Full static gate passed after the explicit test discriminant; final public file passed 9/9 again, then docs gate passed including 5 consistency tests. Runtime implementation unchanged after the 945-test semantic suite and real control."
},
"ci_failure": {
"run_id": 38758262643,
"job_id": 116503076738,
"test": "tests/acceptance/applications/javascriptEndpointPayload.test.ts",
"result": "1 failed/1573 passed/19 skipped; request endpoint no longer matched its complete source value. Other local tests and real Obsidian control passed but did not establish this compatibility contract.",
"aggregate_result": "Merged source/built report: 1 failed/15625 passed/138 skipped. Portable tests and CI required failed as aggregates of the same endpoint assertion; no separate additional failed test."
}
}
],
"review_baseline_revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"review_baseline_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "ee541e634264363822bf4ce05c42c72272029c7e8da72bbeed643ad1341d0756",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a1cf791a61ee4742c351c006b69b64eba6bef3068f7a8450ed8f9366bfe4ff88",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "5ee7af18dd847726d8fe1828ea25f386ff9ef317d996a3f7969253bf73973cfa",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "4c62462016246bc7198294a74a60e87598fc009c53f1601abc2d4662d41b5bd1",
"src/domain/javascript/javascriptSemanticIr.ts": "14955cb4b7912ed6a4f26f804a2480f841dcb6084dbddd15856375fa62cdcfa1",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticState.ts": "d94c5a88bb105284fab192406a8c88860d59f80c93394ed3ee7264969df254a6",
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"implementation": "Per-analysis state records encountered resource classifications at expression/default normalization; final coverage consumes those facts. Slot projection preserves tagged unknown values. Removed retained-value recursive coverage scan and duplicate raw default reader. Defaults retain optional valueResourceLimit and graph value_resource_limit plus an anchored resource-limit unknown; known null/false/0/empty remain exact. Endpoint identities intentionally remain exact lookup values, outside the normalized primitive lattice; rejected endpoint normalization and its CI failure are preserved separately.",
"restored_production_revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"final_corrected_resource_receipts": {
"public_7": {
"elapsed_seconds": 43.621,
"cpu_seconds": 43.097453,
"peak_cgroup_bytes": 1520324608,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
},
"static_and_docs": {
"elapsed_seconds": 111.403,
"cpu_seconds": 110.830839,
"peak_cgroup_bytes": 4268244992,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
}
@@ -0,0 +1,86 @@
# Semantic literal byte-budget verification
Tracks #3933 under #4027. All normalized semantic literals and configuration
defaults use the primitive owner. Its singleton path validates strings/nonfinite
numbers/negative zero without a Set, sorting or canonical-key serialization.
Each source analysis owns a bounded set of encountered limit classifications;
coverage consumes these observations after evaluation. This replaces a recursive
scan that only saw retained binding/return values. No persistent cache is added.
## Correctness and coverage
Public CLI and actual stdio MCP cases cover quoted strings, cooked templates,
equivalent concatenation, member receivers, a configuration-only expression
statement and a small exact control. Direct/default oversized values become
`resource-limit` unknowns with partial application coverage and
`javascript_semantic_primitive_bytes = 1048576`. Defaults carry
`value_resource_limit` inline and an unknown anchored to their config node.
Known null/false/zero/empty defaults remain exact. Original source/digests and
CLI/MCP identity parity are preserved.
Member and destructuring projections use the shared slot owner's tagged value.
An encountered limit stays visible even when a subsequent projection selects a
healthy sibling: the selected value remains exactly `7`, while file coverage is
partial. This avoids inferring completeness from the final retained values alone.
The duplicate raw default reader and superseded retained-value scan are removed.
Boundary controls admit 174,762 UTF-16 units (1,048,574 estimated JSON bytes)
and reject 174,763 (1,048,580 bytes). Actual arithmetic corrected the earlier
published 1,048,578 total; the unit-count boundary assertions were already correct.
The estimate counts six bytes per unit plus two quotes. It does not establish
actual ASCII JSON size above 1 MiB or an observed OOM.
The initial main baseline reproduced both public literal failures, with addition
and small controls passing. A mistaken optional-limit-array control was corrected
before the separate domain reproduction (2 failed/16 passed). Review head
`40b9d3c5` then reproduced the additional projection/default gaps (8 failed,
25 passed). The first review correction passed 65/66 focused tests; a doubling
chain now truthfully retains an additional encountered expression-depth limit,
so its byte-growth assertion was updated to allow other recorded limits.
Prior validation on the restored runtime: **66 focused tests** in five files and **932 semantic tests**
in 36 files, with overlapping domain coverage. The large-wire fixture retains
16 individually admitted values and at least the former aggregate UTF-16 payload,
plus complete CLI/MCP delivery, typed identities, empty strings and retained
tracing. Inputs regenerate from the tests.
## Endpoint identity compatibility
An attempted endpoint normalization at `5808872` passed its new local cases and real Obsidian control, but failed the existing CI endpoint-payload acceptance workflow (1 failed/1,573 passed/19 skipped in that shard). It violated the established full endpoint lookup/trace contract. This extension was reverted; endpoint values are lookup identities, outside the normalized primitive lattice. The original large endpoint is preserved. Corrected public validation passed **7/7** across two files; this adds one acceptance workflow to the previous 66-case focused selection (**67 distinct cases**, not one 67-case invocation). Its acceptance workflow now also normalizes that identical string as an exported primitive, requiring the typed value bound while endpoint lookup, both trace workflows and version comparison remain usable. Prior measurements and failed attempts remain in the JSON record. Production source is restored byte-identical to `38d263a`; the 932-test semantic run belongs to that state. Final full static and documentation gates (five consistency tests) passed after the compatibility correction.
## Real application control and costs
Official Obsidian 1.14.4 ASAR, selected `package.json` and `main.js`. Follow the
acquisition/digest and existing verifier command in
[module-reference verification](javascript-module-references-2026-10-15.md).
Use three alternating-order query pairs with CLI parity on each revision.
Application code is not executed.
The final correction retains the exact baseline full parent/view Evidence IDs,
23-row fact digest, source coverage and unknowns. Static `cl100k_base` task costs
remain 16,548 with selected name/description/inputSchema contracts, or 204,093
with the full catalog projection, for the reference workflow. Serial legacy
reads also keep their original costs. The query projection improvement belongs
to #4097; these are controls for semantic normalization and coverage.
| Complete real verifier | Before | Final correction |
| ---------------------- | ------------: | ---------------: |
| Elapsed seconds | 33.394 | 32.949 |
| CPU seconds | 33.016802 | 32.559187 |
| Peak cgroup bytes | 1,979,863,040 | 1,880,096,768 |
Includes initial analysis, both query arms, export, CLI parity and tokenizer.
One CPU, 8 GiB hard cap, 6 GiB early stop, no swap; cleanup verified and no OOM.
Memory includes charged file cache, not summed RSS. Counters are at payload exit
while the observer remains alive. Fresh MCP sessions do not establish OS-cold
cache conditions. Shared-host scheduling and filesystem/build warmness were
uncontrolled. This one before/after control establishes neither a speedup nor
an equivalence bound. Static BPE excludes annotations/output schemas, client
rendering/replay, reasoning and final answers; no hosted agent was run. Linux
static ASAR analysis was exercised, not live Electron or other platforms.
[Compact records, samples, source hashes and prior attempts](javascript-literal-byte-budget-2026-10-15.json)
preserve the initial incomplete correction separately from the final measurements.
No downloadable application or complete bundle is archived. Reproduce boundary
outcomes through the listed public tests and reuse the existing real verifier
and preserved 8 GiB observer linked above. Observer hashes and result-affecting
touched source hashes are pinned in the records.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,179 @@
# Native handoff byte-identity verification
Tracks #3926 under #4027. Baseline: `9f8659e0536e4e50a438f5b9d5661f3c9624f6ac`.
Final integration base: `62bf09809ef7bb224028916424f8f07652a20f4f`.
The analyst already knows an ASAR-relative native path and supplies prior native
Evidence. The question is which Evidence can be linked to that component's exact
bytes. Discovery and native-provider execution are outside this task.
## Observed behavior and correction
The native-node producer distinguishes a content digest from a canonical path
whose bytes are unavailable. Its unavailable observation legitimately cites the
containing archive. The handoff consumer previously treated that supporting
artifact digest as the native component's digest.
Public CLI/stdio MCP regression: the missing-companion case failed and the
available-byte control passed. Real Joplin reproduced the same result: missing
keytar bytes produced an `evidence-linked` handoff with the ASAR digest and the
simulated container observation. CLI/MCP agreed on that incorrect answer.
The consumer now uses only the node owner's content-digest identity. The fallback
observation scan is removed. Unavailable native nodes retain their canonical
path, availability, container-backed provenance and partial coverage in the
inline graph; they yield no false actionable handoff or native Evidence link.
An empty handoff list describes eligibility, not absence of native components.
The per-query node index is allocated only after an eligible target exists; no
persistent cache, schema, tool or target authority is added.
Final real checks passed both cases with exact CLI/MCP trace parity. The available
control preserved the complete parent/trace Evidence IDs, native facts, links,
coverage and all token projections. The missing control preserved its parent,
native node, source provenance and coverage, while removing the wrong handoff
and container Evidence link.
## Real input and scope
[Official Joplin v3.7.21 Linux AppImage](https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage),
published 2026-09-25, 201,462,481 bytes, SHA-256
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`.
Read-only 7z 26.02 inspection found 6,324 ASAR files, 46 unpacked entries and
three `.node` components. Extract only the ASAR and those three binaries:
```sh
input_root="$(mktemp -d -t rea-native-handoff-XXXXXX)"
curl --fail --location --output "$input_root/Joplin-3.7.21.AppImage" \
https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage
sha256sum "$input_root/Joplin-3.7.21.AppImage"
7z x -mmt=1 "-o$input_root/extracted" "$input_root/Joplin-3.7.21.AppImage" \
resources/app.asar \
resources/app.asar.unpacked/node_modules/keytar/build/Release/keytar.node \
resources/app.asar.unpacked/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node \
resources/app.asar.unpacked/node_modules/onnxruntime-node/bin/napi-v6/linux/x64/onnxruntime_binding.node
python3 - "$input_root" <<'PY'
import os, sys
from pathlib import Path
root = Path(sys.argv[1])
source = root / 'extracted/resources'
for mode in ('available', 'missing'):
case = root / 'cases' / mode
case.mkdir(parents=True)
os.link(source / 'app.asar', case / 'app.asar')
if mode == 'available':
for native in (source / 'app.asar.unpacked').rglob('*.node'):
target = case / native.relative_to(source)
target.parent.mkdir(parents=True, exist_ok=True)
os.link(native, target)
PY
```
Verify the asset digest above before extraction. The ASAR is 233,575,014 bytes,
SHA-256 `767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
Keytar is 76,960 bytes, SHA-256
`0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a`.
All extracted component hashes are in the JSON record. 7z reported one
trailing-data warning for the AppImage's embedded SquashFS and exited 0.
Both cases select only `package.json` for static source analysis while retaining
the inventory. The available case stages all three native companions; the other
43 unpacked entries are not staged. The missing case stages none. Whole-application
coverage is therefore partial in both cases, independently of the selected
native component's known or unknown byte identity. Neither the application nor
a native analysis provider is executed. Native Evidence is explicitly simulated
to test exact subject linkage. Live Electron, real Ghidra/Hopper/IDA behavior and
Windows/macOS verification are **NOT_ENOUGH_INFO**.
## Reproduce the public workflow
Use pinned Node 24.18.0/npm 11.16.0, `npm ci`, then `npm run build:cached`.
The verifier returns compact records and deletes its private exported parent and
CLI input after owned cleanup. No downloadable application or large Evidence is
archived.
```sh
for mode in available missing; do
node scripts/verify/javascript/native-handoffs.mjs \
--artifact "$input_root/cases/$mode/app.asar" \
--addon-path node_modules/keytar/build/Release/keytar.node \
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a \
--source-path package.json --rounds 3 --cli-parity --require-safe
done
```
Add `--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0` for the
reported BPE projections. The npm alias `verify:javascript:native-handoffs`
builds first; recorded costs use the direct Node invocation above after building.
For a pre-fix observation, run the saved verifier on the baseline build without
`--require-safe`; `safe_identity: false` is an incorrect answer, not a passing
analyst task. Paths affect serialized requests and some Evidence identities.
The exact resource runs additionally used the preserved
[8 GiB observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
and adjacent `resource_payload.py`; both hashes are pinned in the JSON.
## Completed workflow costs and limits
| Case | Revision | Wall s | CPU s | Peak cgroup bytes | Identity oracle |
| ---------------------- | -------- | ------: | ---------: | ----------------: | -------------------------- |
| Native bytes available | Before | 60.369 | 59.806670 | 2,662,629,376 | pass |
| Native bytes available | After | 258.885 | 106.221003 | 2,678,456,320 | pass, exact baseline facts |
| Native bytes missing | Before | 57.930 | 57.479109 | 2,719,457,280 | wrong identity/linkage |
| Native bytes missing | After | 59.738 | 59.254776 | 2,571,235,328 | pass, unknown retained |
Whole-verifier resources include input hashing, analysis, three retained trace
queries, export, CLI parity and tokenization. One CPU, 8 GiB hard cap, 6 GiB early
stop, no swap; cleanup verified and no OOM. Memory includes charged file cache,
not summed RSS. Counters are at payload exit while the observer remains alive.
The available control's final wall and CPU times are substantially higher than
baseline. Earlier post-fix controls on the original base took 61.657/74.956 s wall
and 61.135962/60.352428 s CPU for available/missing cases; their complete receipts
remain separately recorded. The final missing case took 59.738 s wall. This
variability is unresolved and cannot be attributed to the native consumer change.
These runs establish neither speedup nor performance equivalence. Input hashing warms the ASAR
cache, build/OS caches and shared-host scheduling are uncontrolled. Fresh MCP
sessions do not establish OS-cold conditions. First post-analysis and repeated
warm-query samples are preserved separately; query CPU/memory are unmeasured.
One defined task is one analysis summary plus one retained trace, with the path
and prior Evidence already supplied. Static `cl100k_base`/tiktoken 0.12.0 counts
include both requests, replies and name/description/inputSchema contracts:
| Available-byte task, unchanged before/after | BPE tokens |
| ----------------------------------------------- | ---------: |
| Selected catalog + structured replies | 12,340 |
| Selected catalog + complete tool replies | 18,906 |
| Full catalog projection + structured replies | 202,292 |
| Full catalog projection + complete tool replies | 208,858 |
These are alternative reply/catalog projections. The selected catalog alone is
5,570 BPE; the full projection is 195,522. Output schemas, annotations, host
rendering/replay, reasoning, final answers and model billing are unmeasured.
Repeated queries/export/CLI/tokenization are validation overhead outside this
one-task token count, but inside resource receipts. Missing-case structured-task
counts change 11,372 → 11,103 by removing unsupported claims; the incorrect
baseline earns no efficiency credit.
## Regression and verification record
The initial four-file focused selection passed 24 tests. The two public cases
were rerun after adding named summary-result parsing and correcting an optional
architecture factory input. Omitting that field instead of passing null produced
byte-identical normalized simulated Evidence in all four real-case inputs.
The initial CLI-selector harness error and first typecheck failure are retained
as harness/validation attempts, not product reproductions.
After syncing the reviewed upstream changes, the updated application workflow
CLI acceptance file passed 34 tests, bringing the focused selection to 58 distinct
cases across five files. Both Joplin controls were then repeated on the final
integration base. Canonical byte identity is also used by the related version
matching/content comparison consumers; no equivalent observation-digest fallback
was found there.
Final `npm run check && npm run docs:check` passed on the integration base,
including five documentation consistency tests: 128.732 s wall, 115.518920 s CPU,
4,301,684,736 B peak cgroup memory, clean owned exit and no OOM.
Final public and static/docs outcomes are recorded in the
[compact measurements and source pins](javascript-native-handoff-identity-2026-10-15.json).
-1
View File
@@ -10,7 +10,6 @@
"scripts/verify/fridaBindingsCompatibility.ts",
"scripts/verify/termux/browser.mjs",
"scripts/verify-real-ghidra-noreturn.mjs",
"scripts/run-exact-tests.mjs",
"tests/processOwnershipRunner.ts"
],
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],
+11 -4
View File
@@ -111,9 +111,14 @@ automatically. Alternatively, build it using the Linux/MinGW lane above.
- Cleanup walks locked directory objects, removes files by handle, and unlinks
reparse entries without traversing their targets. Failed removal retains the
root and ancestor leases for retry; a partially removed runtime accepts only
cleanup, and its registry entry remains until removal succeeds. The owned root
handle includes attribute-write access so cleanup can clear a read-only marker,
matching the child cleanup handles. Normal close, cancellation,
cleanup, and its registry entry remains until removal succeeds. The runtime
owner retries only native close failures with Win32 sharing/lock codes 32/33,
for at most ten attempts with a 100 ms delay between attempts. Other native
errors are returned immediately; exhausting the retry budget preserves the
runtime owner for a later explicit close. These cleanup retries do not use an
analysis cancellation signal and schedule up to 900 ms of backoff, in
addition to native cleanup time. The owned root handle includes attribute-write access so cleanup can clear a
read-only marker, matching the child cleanup handles. Normal close, cancellation,
startup failure, and timeout release provider resources. Abrupt owner death
kills the owned job. The Android provider additionally uses a detached helper
that owns its private runtime directory and cleans it when the owner pipe
@@ -133,7 +138,9 @@ text do not establish a safe retry. Analysis errors propagate the producer's
retry disposition through JavaScript, source-map and WABT workflows; transient
failures report `category: unavailable` and retain the original reason.
Incomplete cleanup blocks an operation retry until its owned resources settle.
REA adds no automatic retry loop or new analysis cache for this behavior.
The close-stage retry is separate from allocation and analysis retries: it
repeats only the existing owned cleanup operation, and does not retry provider
work or create another runtime.
Ordinary drive-absolute paths accept backslashes, forward slashes, or mixed
separators. File identity retains the original requested spelling. Separator
+2
View File
@@ -214,6 +214,7 @@
"verify:javascript:digests": "npm run build:cached && node scripts/verify/javascript/digests.mjs",
"verify:javascript:output": "npm run build:cached && node scripts/verify/javascript/output.mjs",
"verify:javascript:module-references": "npm run build:cached && node scripts/verify/javascript/module-references.mjs",
"verify:javascript:native-handoffs": "npm run build:cached && node scripts/verify/javascript/native-handoffs.mjs",
"verify:application-workflows": "npm run build:cached && node scripts/verify/javascript/application-workflows.mjs",
"verify:managed": "npm run build:cached && node scripts/verify/managed/conformance.mjs",
"verify:firmware": "npm run build:cached && node scripts/verify/firmware/analysis.mjs",
@@ -272,6 +273,7 @@
"verify:windows-native": "node scripts/verify-windows-native.mjs",
"verify:ghidra:windows:package": "node scripts/verify-packaged-ghidra-windows.mjs",
"build:ghidra:nativeaot": "node scripts/build-ghidra-nativeaot.mjs",
"verify:ghidra:bridge-sources": "node scripts/verify-ghidra-bridge-sources.mjs",
"build:fixtures:nativeaot": "node scripts/generate-nativeaot-fixtures.mjs",
"verify:ghidra:nativeaot": "npm run build:cached && node scripts/verify-real-ghidra-nativeaot.mjs",
"verify:browser:scripts": "npm run build:cached && node scripts/verify-browser-scripts.mjs",
+1 -1
View File
@@ -157,7 +157,7 @@ function propertyValue(content, name) {
return null;
}
/** Same 12.1 line as GhidraInstallationPolicy, including a distro suffix. */
/** Release line supported by the separately pinned NativeAOT integration. */
function ghidraReleaseLineAccepted(raw) {
const match = /^(\d+)\.(\d+)(?:\.(\d+))?([-+].*)?$/u.exec(raw ?? "");
return match !== null && Number(match[1]) === 12 && Number(match[2]) === 1;
+10
View File
@@ -104,6 +104,7 @@ export const SCOPES = {
"tests/boundary/filesystem/npmRelease.test.ts",
"tests/boundary/process/releaseArtifactCanary.test.ts",
"src/windows/**",
"src/ghidra/GhidraBridgeSources.ts",
"native/windows/**",
"src/process/capture/FilesystemSnapshot.ts",
"src/process/capture/FilesystemSnapshot.test.ts",
@@ -138,6 +139,7 @@ export const SCOPES = {
"src/domain/javascript/javascriptRuntimeStaticCandidates.ts",
"tests/boundary/filesystem/webScriptManifestPaths.test.ts",
"tests/boundary/process/ProcessEvidence.host.test.ts",
"tests/boundary/process/packageNpmInvocation.test.ts",
"tests/boundary/process/windowsOutputCapture.test.ts",
"tests/acceptance/webModuleCli.test.ts",
"tests/acceptance/webSourceLocationCli.test.ts",
@@ -435,6 +437,14 @@ export const SCOPES = {
"tests/fixtures/nativeaot/**",
".github/workflows/nativeaot-fixtures.yml",
],
ghidra_bridge: [
"bridge/ghidra/**",
"src/ghidra/GhidraBridgeSources.ts",
"src/ghidra/GhidraJavaProperties.ts",
"src/ghidra/GhidraReleasePolicy.ts",
"scripts/verify-ghidra-bridge-sources.mjs",
".github/workflows/ghidra-bridge-sources.yml",
],
website: [
"website/**",
"scripts/*website*.py",
+1 -1
View File
@@ -17,7 +17,7 @@ export const SOURCE_TEST_PROJECTS = [
group: "source",
include: [
"src/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,wasm,windows}/**/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
],
},
{
+20
View File
@@ -1,5 +1,6 @@
import { execFile } from "node:child_process";
import { lstat } from "node:fs/promises";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { Ajv2020 } from "ajv/dist/2020.js";
@@ -17,6 +18,25 @@ const loadMcpVerifierResults = () => import("./mcp-verifier-results.mjs");
/** Promisified child_process execution helper. */
export const exec = promisify(execFile);
/** Run npm through its JavaScript entry point so Windows command scripts are not required. */
export const execNpm = (arguments_, options = {}) => {
const environment = options.env ?? process.env;
const selectedEntry = environment.npm_execpath;
if (
selectedEntry !== undefined &&
(typeof selectedEntry !== "string" || selectedEntry.length === 0)
)
throw new Error("npm_execpath must name the selected npm CLI entry point");
const entry =
selectedEntry ??
(process.platform === "win32"
? join(dirname(process.execPath), "node_modules/npm/bin/npm-cli.js")
: undefined);
return entry === undefined
? exec("npm", arguments_, options)
: exec(process.execPath, [entry, ...arguments_], options);
};
/** Run a command and return its stdout. */
export const run = async (command, args, env) =>
(await exec(command, args, { env })).stdout;
+4
View File
@@ -0,0 +1,4 @@
// A single inspection can include native admission plus several WABT commands.
// The aggregate verifier covers seven complete fixtures and their failure paths.
export const WABT_OPERATION_TIMEOUT_MS = 4 * 60_000;
export const WABT_WORKFLOW_TIMEOUT_MS = 15 * 60_000;
+62
View File
@@ -5,6 +5,11 @@ import { tmpdir } from "node:os";
import { join, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import Ajv2020 from "ajv/dist/2020.js";
import {
schemaArrays,
schemaChildren,
schemaMaps,
} from "../dist/contracts/inputSchemaPresentation.js";
import {
TOOL_CONTRACTS,
toolContract,
@@ -17,6 +22,8 @@ const repo = fileURLToPath(new URL("..", import.meta.url));
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
const command = process.env.REA_VERIFY_GEMINI_COMMAND ?? "gemini";
const mode = process.argv[2] ?? "call";
const schemaProfile = process.env.REA_VERIFY_GEMINI_SCHEMA_PROFILE ?? "full";
assert(["full", "compact"].includes(schemaProfile));
// npm installs of Gemini CLI provide a JavaScript entry point; on Windows only a gemini.cmd shim reaches
// PATH, which Node does not spawn without a shell, so a selected .js entry runs under this Node. It is
// resolved once, because the preflight runs from the caller's directory and the client from the workspace.
@@ -60,6 +67,7 @@ const environment = {
GEMINI_CLI_NO_RELAUNCH: "1",
GEMINI_CLI_TRUST_WORKSPACE: "true",
REA_PROCESS_RUN_ID: verifier.run_id,
REA_MCP_INPUT_SCHEMA_PROFILE: schemaProfile,
};
let clientVersion;
try {
@@ -161,8 +169,43 @@ let skillRequested = false;
let finalSeen = false;
let catalogSize = 0;
let schemasChecked = 0;
let examplesChecked = 0;
let artifactDigest;
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
const booleanLiteralPath = (schema, path = "$", seen = new Set()) => {
if (!schema || typeof schema !== "object" || seen.has(schema))
return undefined;
seen.add(schema);
for (const [key, child] of Object.entries(schema)) {
const childPath = `${path}.${key}`;
if (key === "const" && typeof child === "boolean") return childPath;
if (
key === "enum" &&
Array.isArray(child) &&
child.some((item) => typeof item === "boolean")
)
return childPath;
const schemaChildrenAtKey =
schemaMaps.has(key) &&
child &&
typeof child === "object" &&
!Array.isArray(child)
? Object.entries(child).map(([name, item]) => [
`${childPath}.${name}`,
item,
])
: schemaArrays.has(key) && Array.isArray(child)
? child.map((item, index) => [`${childPath}[${index}]`, item])
: schemaChildren.has(key)
? [[childPath, child]]
: [];
for (const [nestedPath, nested] of schemaChildrenAtKey) {
const found = booleanLiteralPath(nested, nestedPath, seen);
if (found !== undefined) return found;
}
}
return undefined;
};
const extractEvidence = (value) => {
if (typeof value === "string") {
if (
@@ -226,6 +269,22 @@ const fixture = await createGeminiModelFixture({
schemaValidator.validateSchema(tool.parametersJsonSchema),
`${tool.name}: invalid advertised JSON Schema: ${JSON.stringify(schemaValidator.errors)}`,
);
const booleanLiteral = booleanLiteralPath(tool.parametersJsonSchema);
assert.equal(
booleanLiteral,
undefined,
`${tool.name}: Gemini SDK declaration retains a boolean const/enum at ${booleanLiteral}`,
);
const name = tool.name.slice(prefix.length);
const contract = toolContract(name);
const validate = schemaValidator.compile(tool.parametersJsonSchema);
for (const example of contract.examples) {
assert(
validate(example.input),
`${tool.name}: Gemini SDK declaration rejects canonical example ${example.title}: ${JSON.stringify(schemaValidator.errors)}`,
);
examplesChecked++;
}
schemasChecked++;
}
}
@@ -356,6 +415,9 @@ try {
client: "Gemini CLI",
clientVersion,
mode,
schemaProfile,
schemasChecked,
examplesChecked,
passed: failure === undefined,
artifacts: lab,
runtimeRoot,
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env node
// Compile the packaged Ghidra bridge sources against an existing Ghidra
// installation's jars. Ghidra's script provider compiles the whole bridge
// directory when a session starts, so one unresolved symbol disables every
// operation and surfaces only after import and auto-analysis. This check
// reports the same javac diagnostics without launching Ghidra or selecting,
// importing or executing a target.
import { spawnSync } from "node:child_process";
import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, relative, resolve } from "node:path";
import { fileURLToPath } from "node:url";
// The pinned Node toolchain strips types from these dependency-free owners;
// this preflight therefore uses runtime facts without needing npm ci or a build.
import { GHIDRA_BRIDGE_SOURCES } from "../src/ghidra/GhidraBridgeSources.ts";
import { parseGhidraJavaProperties } from "../src/ghidra/GhidraJavaProperties.ts";
import {
SUPPORTED_GHIDRA_VERSION,
SUPPORTED_GHIDRA_JAVA_MAJOR,
ghidraReleaseAccepted,
ghidraReleaseLine,
} from "../src/ghidra/GhidraReleasePolicy.ts";
const args = process.argv.slice(2);
if (
args.length > 1 ||
(args.length === 1 &&
!["--check-sdk-pin", "--verified-sdk"].includes(args[0]))
)
fail(
"Usage: verify-ghidra-bridge-sources.mjs [--check-sdk-pin|--verified-sdk]",
);
if (args[0] === "--check-sdk-pin") {
let url;
try {
url = new URL(process.env.GHIDRA_RELEASE_URL);
} catch {
fail("GHIDRA_RELEASE_URL must name the checksum-pinned official release");
}
const prefix = `/NationalSecurityAgency/ghidra/releases/download/Ghidra_${SUPPORTED_GHIDRA_VERSION}_build/ghidra_${SUPPORTED_GHIDRA_VERSION}_PUBLIC_`;
if (
url.origin !== "https://github.com" ||
url.username ||
url.password ||
url.search ||
url.hash ||
!url.pathname.startsWith(prefix) ||
!url.pathname.endsWith(".zip") ||
!/^[a-f0-9]{64}$/iu.test(process.env.GHIDRA_RELEASE_SHA256 ?? "")
)
fail(
`SDK URL and SHA256 must match verified Ghidra ${SUPPORTED_GHIDRA_VERSION}`,
);
console.log(`GHIDRA_JAVA_MAJOR=${SUPPORTED_GHIDRA_JAVA_MAJOR}`);
process.exit(0);
}
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const bridge = join(root, "bridge/ghidra");
const install = process.env.GHIDRA_INSTALL_DIR;
if (!install)
fail(
`Set GHIDRA_INSTALL_DIR to an existing Ghidra ${ghidraReleaseLine()} installation (verified with ${SUPPORTED_GHIDRA_VERSION}). REA does not install Ghidra.`,
);
const properties = await readFile(
join(install, "Ghidra/application.properties"),
"latin1",
).catch((cause) =>
fail(
`GHIDRA_INSTALL_DIR has no readable Ghidra/application.properties: ${cause.message}`,
),
);
const parsedProperties = parseGhidraJavaProperties(properties);
if (!parsedProperties.ok) fail(parsedProperties.error);
const version = parsedProperties.value.get("application.version");
if (!ghidraReleaseAccepted(version))
fail(
`Bridge sources require Ghidra ${ghidraReleaseLine()} (verified with ${SUPPORTED_GHIDRA_VERSION}). Observed application.version=${version === undefined ? "missing" : JSON.stringify(version)}.`,
);
if (args[0] === "--verified-sdk" && version !== SUPPORTED_GHIDRA_VERSION)
fail(
`CI SDK must be verified Ghidra ${SUPPORTED_GHIDRA_VERSION}; observed ${JSON.stringify(version)}`,
);
const sourceNames = (await readdir(bridge, { withFileTypes: true }))
.filter((entry) => entry.isFile() && entry.name.endsWith(".java"))
.map((entry) => entry.name);
for (const name of Object.values(GHIDRA_BRIDGE_SOURCES))
if (!sourceNames.includes(name))
fail(`Required Ghidra bridge source is missing: bridge/ghidra/${name}`);
const requiredNames = new Set(Object.values(GHIDRA_BRIDGE_SOURCES));
const unregistered = sourceNames.filter((name) => !requiredNames.has(name));
if (unregistered.length)
fail(
`Ghidra bridge sources absent from the manifest: ${unregistered.join(", ")}`,
);
const sources = sourceNames
.map((name) => relative(root, join(bridge, name)))
.sort();
const javac = process.env.JAVA_HOME
? join(process.env.JAVA_HOME, "bin", "javac")
: "javac";
const banner = run(javac, ["-version"]);
if (banner.error || banner.status !== 0)
fail(`javac is unavailable: ${banner.error?.message ?? output(banner)}`);
const major = Number(/^javac (\d+)/mu.exec(output(banner))?.[1]);
if (!Number.isInteger(major) || major < SUPPORTED_GHIDRA_JAVA_MAJOR)
fail(
`Ghidra ${ghidraReleaseLine()} compiles bridge scripts with Java ${SUPPORTED_GHIDRA_JAVA_MAJOR} or newer. Observed: ${output(banner) || "no version"}. REA does not install or upgrade Java.`,
);
const jars = (
await readdir(join(install, "Ghidra"), {
recursive: true,
withFileTypes: true,
})
)
.filter((entry) => entry.isFile() && entry.name.endsWith(".jar"))
.map((entry) => join(entry.parentPath, entry.name))
.sort();
if (jars.length === 0) fail(`No jars found under ${join(install, "Ghidra")}.`);
const scratch = await mkdtemp(join(tmpdir(), "rea-ghidra-bridge-"));
try {
// An argument file keeps the jar class path below Windows' command length.
const argumentFile = join(scratch, "javac.args");
await writeFile(
argumentFile,
[
"-proc:none",
"-encoding",
"UTF-8",
"--release",
String(SUPPORTED_GHIDRA_JAVA_MAJOR),
"-cp",
jars.join(process.platform === "win32" ? ";" : ":"),
"-d",
join(scratch, "classes"),
...sources,
]
.map((s) => `"${s.replaceAll("\\", "\\\\").replaceAll('"', '\\"')}"`)
.join("\n"),
);
// Sources are repository-relative, so diagnostics name bridge/ghidra/<file>:<line>.
const result = run(javac, ["-J-Xmx1g", `@${argumentFile}`]);
if (result.error || result.status !== 0) {
// Set the status instead of exiting so the scratch directory is removed.
process.stderr.write(
`${output(result) || result.error?.message || "javac failed"}\nGhidra ${version} could not compile the packaged bridge sources; the provider would fail at session start.\n`,
);
process.exitCode = 1;
} else {
console.log(
JSON.stringify({
ghidra_version: version,
javac: output(banner),
class_path_jars: jars.length,
sources,
}),
);
}
} finally {
await rm(scratch, { recursive: true, force: true });
}
function run(command, args) {
return spawnSync(command, args, {
cwd: root,
encoding: "utf8",
timeout: 300_000,
maxBuffer: 16 * 1024 * 1024,
});
}
function output(result) {
return `${result.stdout ?? ""}${result.stderr ?? ""}`.trim();
}
function fail(message) {
process.stderr.write(`${message}\n`);
process.exit(1);
}
+4 -6
View File
@@ -2,7 +2,7 @@ import { rm } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { exec, json, run } from "./lib/verify-package-core.mjs";
import { exec, execNpm, json, run } from "./lib/verify-package-core.mjs";
/** Install the packed CLI globally with and without optional dependencies and verify PTY degradation. */
export async function verifyPackageInstall({
@@ -11,8 +11,7 @@ export async function verifyPackageInstall({
workspace,
environment,
}) {
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -53,8 +52,7 @@ export async function verifyPackageInstall({
`packaged PTY backend failed without lifecycle scripts: ${JSON.stringify(processCaptureCapability)}`,
);
const noOptionalPrefix = join(workspace, "prefix-no-optional");
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -108,7 +106,7 @@ export async function verifyPackageInstall({
const verifyInstalledPackageCanRepack = async (prefix, environment) => {
const installedPackageRoot = join(prefix, "lib/node_modules/rea-agents");
const repackedTarball = (
await exec("npm", ["pack", "--silent"], {
await execNpm(["pack", "--silent"], {
cwd: installedPackageRoot,
env: environment,
})
+8 -9
View File
@@ -2,7 +2,7 @@ import { mkdir } from "node:fs/promises";
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { exec } from "./lib/verify-package-core.mjs";
import { exec, execNpm } from "./lib/verify-package-core.mjs";
import { verifyPackedBridge } from "./verify-packed-bridge.mjs";
/** Create a tarball and assert production packaging constraints. */
@@ -12,16 +12,15 @@ export async function verifyPackagePack({ root, workspace }) {
const tarball = join(
packDirectory,
(
await exec(
"npm",
["pack", "--silent", "--pack-destination", packDirectory],
{
cwd: root,
},
)
await execNpm(["pack", "--silent", "--pack-destination", packDirectory], {
cwd: root,
})
).stdout.trim(),
);
const packedFiles = (await exec("tar", ["-tf", tarball])).stdout.split("\n");
const listing = (await exec("tar", ["-tf", tarball])).stdout;
// A bare LF identifies LF output; preceding CRs then belong to filenames.
const lineEnding = /(^|[^\r])\n/u.test(listing) ? "\n" : "\r\n";
const packedFiles = listing.split(lineEnding);
const packedManifest = JSON.parse(
(await exec("tar", ["-xOf", tarball, "package/package.json"])).stdout,
);
+3 -9
View File
@@ -28,6 +28,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { execNpm } from "./lib/verify-package-core.mjs";
if (process.platform !== "win32" || process.arch !== "x64")
throw new Error(
@@ -82,15 +83,10 @@ let packageRoot = resolve(packageRootArgument ?? ".");
// package roots support an already-installed artifact without a second install.
if (packageRootArgument === undefined) {
try {
const npmEntry =
process.env.npm_execpath ??
join(dirname(process.execPath), "node_modules/npm/bin/npm-cli.js");
const packed = JSON.parse(
(
await exec(
process.execPath,
await execNpm(
[
npmEntry,
"pack",
"--json",
"--ignore-scripts",
@@ -107,10 +103,8 @@ if (packageRootArgument === undefined) {
"The package lane must produce one artifact.",
);
const prefix = join(workspace, "installed");
await exec(
process.execPath,
await execNpm(
[
npmEntry,
"install",
"--prefix",
prefix,
+14 -5
View File
@@ -2,12 +2,15 @@
import { join } from "node:path";
import { mkdir } from "node:fs/promises";
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
import { exec } from "./lib/verify-package-core.mjs";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
import { exec, execNpm } from "./lib/verify-package-core.mjs";
import { WABT_WORKFLOW_TIMEOUT_MS } from "./lib/wabt-verifier-timeouts.mjs";
import { verifyPackagePack } from "./verify-package-pack.mjs";
const root = await PrivateRuntimeRoot.create({
prefix: "rea-wabt-package-verifier-",
});
let primaryError;
try {
const { tarball } = await verifyPackagePack({
root: process.cwd(),
@@ -23,8 +26,7 @@ try {
XDG_CONFIG_HOME: join(home, ".config"),
XDG_CACHE_HOME: join(home, ".cache"),
};
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -49,9 +51,16 @@ try {
const verified = await exec(
process.execPath,
["scripts/verify-wasm-artifact.mjs", entrypoint, "--require-tools"],
{ env: environment, timeout: 180_000, maxBuffer: 4 * 1024 * 1024 },
{
env: environment,
timeout: WABT_WORKFLOW_TIMEOUT_MS,
maxBuffer: 4 * 1024 * 1024,
},
);
process.stdout.write(verified.stdout);
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
await root.close();
await closeVerifierResources([() => root.close()], primaryError);
}
+4 -5
View File
@@ -1,21 +1,20 @@
import { execFile } from "node:child_process";
import { join } from "node:path";
import { promisify } from "node:util";
import { GHIDRA_BRIDGE_SOURCES } from "../src/ghidra/GhidraBridgeSources.ts";
const exec = promisify(execFile);
const packedHopperBridge = "package/bridge/hopper_bridge.py";
const packedCutterBridge = "package/bridge/cutter/rea_bridge.py";
const packedGhidraBridge = "package/bridge/ghidra/ReaGhidraBridge.java";
const packedGhidraActivator =
"package/bridge/ghidra/ReaGhidraBundleActivator.java";
/** Check shipped bridge files; real-provider lanes verify runtime behavior. */
export async function verifyPackedBridge({ workspace, tarball, packedFiles }) {
for (const path of [
packedHopperBridge,
packedCutterBridge,
packedGhidraBridge,
packedGhidraActivator,
...Object.values(GHIDRA_BRIDGE_SOURCES).map(
(name) => `package/bridge/ghidra/${name}`,
),
]) {
if (!packedFiles.includes(path)) throw new Error(`package omitted ${path}`);
}
+9 -3
View File
@@ -12,6 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { Ajv2020 } from "ajv/dist/2020.js";
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
if (process.platform !== "linux")
@@ -106,6 +107,7 @@ const expectedAddressSpace = Math.min(
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
let primaryError;
try {
await writeFile(path, source);
await writeFile(bluetooth, bluetoothSource);
@@ -463,10 +465,14 @@ try {
"network with spaces.pcap",
"paged.pcap",
]);
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
await client.close();
await transport.close();
await root.close();
await closeVerifierResources(
[() => client.close(), () => transport.close(), () => root.close()],
primaryError,
);
}
console.log(
JSON.stringify(
+1 -1
View File
@@ -3,7 +3,7 @@ import { hasTypedSwitchEvidence } from "./verify-real-ghidra-switch-assertions.m
import { access } from "node:fs/promises";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GHIDRA_SESSION_CAPABILITIES } from "../dist/ghidra/GhidraSessionValues.js";
export function assertLocalProcedureInfo(info, expectedAddress) {
+49 -84
View File
@@ -12,6 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import {
inspectMipsReadelf,
expectedMipsElfProfile,
inspectLegacyMipsReadelf,
assertMipsLoadedImage,
assertMipsProbe,
@@ -120,6 +121,17 @@ async function verifySoftFloat(call, symbols, byteOrder) {
);
return { storage: boundaries };
}
function assertIncompleteMipsInspection(error) {
assert.equal(error.code, "capability_unavailable");
assert.equal(error.details.selection_reason, "target_unsupported");
const rejection = error.details.rejections.find(
(candidate) => candidate.provider_id === "ghidra",
);
assert.equal(rejection?.code, "architecture_unsupported");
assert.match(rejection.reason, /ABI declaration inspection is incomplete/u);
assert.match(rejection.reason, /section header table/u);
}
const reports = [];
let primaryFailure;
try {
@@ -475,28 +487,9 @@ try {
JSON.parse(await readFile(snapshotPath, "utf8")),
);
assert.equal(snapshot.target.architecture, "mips");
const abi = independent.mips.abiFlags;
assert.deepEqual(
snapshot.binding.analysis_profile.parameters.mips_elf,
{
elf_class: independent.mips.elfClass,
byte_order: independent.mips.byteOrder,
type: independent.mips.type,
flags: independent.mips.flags,
abi_flags: {
version: abi.version,
isa_level: abi.isaLevel,
isa_revision: abi.isaRevision,
gpr_size: abi.gprSize,
cpr1_size: abi.cpr1Size,
cpr2_size: abi.cpr2Size,
fp_abi: abi.fpAbi,
isa_extension: abi.isaExtension,
ases: abi.ases,
flags1: abi.flags1,
flags2: abi.flags2,
},
},
expectedMipsElfProfile(independent.mips),
"Snapshot profile did not retain the independently inspected ELF/ABI declaration",
);
assert.equal(
@@ -617,7 +610,8 @@ try {
// PSP keeps its independent Allegrex/A-B-A checks; carved PSP is not verified.
let carved;
if (!psp) {
// Carved firmware keeps all program headers/load bytes but loses sections.
// A program-header ABI record survives carving, but GNU declarations in
// uncaptured section headers remain unknown: both adapters must refuse.
const read16 = (offset) =>
byteOrder === "little"
? sourceBytes.readUInt16LE(offset)
@@ -655,79 +649,50 @@ try {
carvedResolved.value.mips.limitations.join(" "),
/section header table/u,
);
const carvedTarget = await call("open_binary", {
path: carvedPath,
provider_id: "ghidra",
});
assert.equal(carvedTarget.sha256, carvedSha256);
const carvedImage = await call("inspect_native_load_image");
assertMipsLoadedImage(
carvedImage.observations,
byteOrder,
carvedSha256,
);
const carvedProcedures = await call("list_procedures");
assert.ok(
carvedProcedures.some(
(p) =>
BigInt(p.address) === BigInt(independent.symbols.rea_mips_entry),
const carvedError = requireMcpToolError(
await client.callTool(
{
name: "open_binary",
arguments: { path: carvedPath, provider_id: "ghidra" },
},
{ timeout: 60000 },
),
);
const carvedDossier = functionDossierSchema.parse(
await call("analyze_function", {
procedure: independent.symbols.rea_mips_entry,
}),
);
assert.ok(carvedDossier.pseudocode.trim().length > 0);
const carvedSnapshotPath = join(
workspace,
`${byteOrder}-${floatAbi}-carved.snapshot.json`,
);
await call("close_binary", { snapshot_path: carvedSnapshotPath });
const carvedSnapshot = parseAnalysisSnapshot(
JSON.parse(await readFile(carvedSnapshotPath, "utf8")),
);
assert.equal(carvedSnapshot.target.sha256, carvedSha256);
assertIncompleteMipsInspection(carvedError);
let carvedCliError;
try {
await execute(
process.execPath,
[
entrypoint,
"function",
carvedPath,
independent.symbols.rea_mips_entry,
"--provider",
"ghidra",
"--json",
],
{ env, timeout: 60000, maxBuffer: 1024 * 1024 },
);
} catch (cause) {
carvedCliError = cause;
}
assert.ok(
carvedSnapshot.evidence_bundle.records.some((record) =>
parseEvidence(record).limitations.some((item) =>
item.includes("section header table"),
),
),
);
const carvedCli = await execute(
process.execPath,
[
entrypoint,
"function",
carvedPath,
independent.symbols.rea_mips_entry,
"--provider",
"ghidra",
"--json",
],
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
);
const carvedEvidence = parseEvidence(JSON.parse(carvedCli.stdout));
assert.equal(carvedEvidence.subject?.digest.sha256, carvedSha256);
assert.ok(
carvedEvidence.limitations.some((item) =>
item.includes("section header table"),
),
);
assert.equal(
functionDossierSchema.parse(carvedEvidence.normalized_result)
.procedure.address,
carvedDossier.procedure.address,
carvedCliError &&
Number.isInteger(carvedCliError.code) &&
carvedCliError.code !== 0,
"Expected an incomplete-ABI refusal, not a timeout or success",
);
assertIncompleteMipsInspection(JSON.parse(carvedCliError.stdout));
assert.deepEqual(await readFile(carvedPath), carvedBytes);
carved = {
sha256: carvedSha256,
source_bytes: carvedBytes.length,
section_header_offset: sectionOffset,
effective_language: carvedImage.observations.language_id,
status: "expected-incomplete-abi-refusal",
limitations: carvedResolved.value.mips.limitations,
procedures: carvedProcedures.length,
cli_exit: carvedCliError.code,
mcp_error: carvedError,
};
}
reports.push({
+1 -1
View File
@@ -11,7 +11,7 @@ import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { parseConfig } from "../dist/config/parseConfig.js";
import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GhidraProvider } from "../dist/ghidra/GhidraProvider.js";
import { silentLogger } from "../dist/logger.js";
import { cleanupOwnedProcessGroup } from "../dist/process/ProcessOwnership.js";
+1 -1
View File
@@ -19,7 +19,7 @@ import {
parseGhidraInventoryResult,
} from "../dist/ghidra/GhidraInventoryValues.js";
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderCapabilities.js";
import {
+1 -1
View File
@@ -11,7 +11,7 @@ import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { resolveGhidraAnalysisProfile } from "../dist/ghidra/GhidraAnalysisProfile.js";
import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderCapabilities.js";
+4 -1
View File
@@ -83,8 +83,11 @@ const FIXTURE_SOURCE = String.raw`
#include <stdio.h>
#include <stdlib.h>
static const char rea_kuna_utf8[] __attribute__((used)) = "_φ( °-°)/ REA unicode marker";
/* The leading NUL unit ends any UTF-16 run that bytes before this array would
start: the UTF-8 marker's last byte and NUL decode as the unit 'r', so on
layouts where it precedes this array Kuna reports "rREA wide marker". */
static const unsigned short rea_kuna_wide[] __attribute__((used)) = {
'R','E','A',' ','w','i','d','e',' ','m','a','r','k','e','r',0
0,'R','E','A',' ','w','i','d','e',' ','m','a','r','k','e','r',0
};
#include <string.h>
__attribute__((noinline)) unsigned rea_checksum(const unsigned char *p, size_t n) {
+2 -3
View File
@@ -1,7 +1,7 @@
import { cp, mkdir, readFile, readdir } from "node:fs/promises";
import { dirname, join, relative } from "node:path";
import { exec, pathExists } from "./lib/verify-package-core.mjs";
import { exec, execNpm, pathExists } from "./lib/verify-package-core.mjs";
/** Exercise Skills' default root discovery without ignored build outputs. */
export async function verifyRepositorySkill({ root, workspace }) {
@@ -34,8 +34,7 @@ export async function verifyRepositorySkill({ root, workspace }) {
await mkdir(consumer, { recursive: true });
await mkdir(home, { recursive: true });
const skillsVersion = "1.7.2";
await exec(
"npm",
await execNpm(
[
"exec",
"--yes",
+9 -3
View File
@@ -12,6 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { Ajv2020 } from "ajv/dist/2020.js";
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
if (process.platform !== "linux")
@@ -83,6 +84,7 @@ const expectedAddressSpace = Math.min(
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
let primaryError;
try {
await writeFile(path, source);
await client.connect(transport);
@@ -383,10 +385,14 @@ try {
"paged.vcd",
"recording with spaces.vcd",
]);
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
await client.close();
await transport.close();
await root.close();
await closeVerifierResources(
[() => client.close(), () => transport.close(), () => root.close()],
primaryError,
);
}
console.log(
JSON.stringify(
+8 -5
View File
@@ -24,6 +24,7 @@ import {
requireMcpToolError,
} from "./lib/mcp-verifier-results.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
import { WABT_OPERATION_TIMEOUT_MS } from "./lib/wabt-verifier-timeouts.mjs";
const directory = process.env.REA_WABT_BIN_DIRECTORY;
if (!isAbsolute(directory ?? "")) {
@@ -76,6 +77,8 @@ try {
throw cause;
}
const client = new Client({ name: "wabt-artifact-verifier", version: "1" });
const callTool = (request) =>
client.callTool(request, { timeout: WABT_OPERATION_TIMEOUT_MS });
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
@@ -128,7 +131,7 @@ try {
TMP: childTempRoot,
TEMP: childTempRoot,
},
timeout: 120_000,
timeout: WABT_OPERATION_TIMEOUT_MS,
maxBuffer: 64 * 1024 * 1024,
},
)
@@ -238,7 +241,7 @@ try {
const cliValue = await inspectCli(input);
assert.equal(cliValue.category, undefined, JSON.stringify(cliValue));
const cli = parseEvidence(cliValue);
const reply = await client.callTool({
const reply = await callTool({
name: "inspect_wasm_artifact",
arguments: input,
});
@@ -332,7 +335,7 @@ try {
},
);
}
const bundleReply = await client.callTool({
const bundleReply = await callTool({
name: "get_evidence_bundle",
arguments: {},
});
@@ -362,7 +365,7 @@ try {
JSON.stringify(cliError),
);
const error = requireMcpToolError(
await client.callTool({
await callTool({
name: "inspect_wasm_artifact",
arguments: invalidInput,
}),
@@ -370,7 +373,7 @@ try {
assert.equal(error.category, "invalid_input");
const partial = error.details.partial_observation;
assert.equal(partial.kind, "retained-evidence");
const failedBundleReply = await client.callTool({
const failedBundleReply = await callTool({
name: "get_evidence_bundle",
arguments: {},
});
+118 -29
View File
@@ -23,6 +23,7 @@ import { pathToFileURL } from "node:url";
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
import { verifyWindowsJobMemoryLimit } from "./lib/windows-job-memory-limit.mjs";
import { verifyVolumeRootAppendLease } from "./lib/windows-volume-root-append-lease.mjs";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
if (process.platform !== "win32")
throw new Error(
@@ -66,6 +67,49 @@ const exists = (path) =>
() => true,
() => false,
);
const startDeleteSharingDeniedReader = async (path) => {
const child = spawn(
"powershell.exe",
[
"-NoProfile",
"-NonInteractive",
"-Command",
"$ErrorActionPreference = 'Stop'; " +
"$stream = [IO.File]::Open($env:REA_CLEANUP_HELD_PATH, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::ReadWrite); " +
"try { [Console]::WriteLine('ready'); [Console]::In.ReadLine() | Out-Null } " +
"finally { $stream.Dispose() }",
],
{
env: { ...process.env, REA_CLEANUP_HELD_PATH: path },
stdio: ["pipe", "pipe", "pipe"],
},
);
const closed = once(child, "close");
void closed.catch(() => undefined);
try {
const [ready] = await once(child.stdout, "data", {
signal: AbortSignal.timeout(15_000),
});
assert.equal(String(ready).trim(), "ready");
} catch (cause) {
try {
child.kill();
await closed;
} catch (cleanupCause) {
throw new AggregateError(
[cause, cleanupCause],
"Reader startup and child cleanup failed",
{ cause },
);
}
throw cause;
}
return { child, closed };
};
const stopDeleteSharingDeniedReader = async ({ child, closed }) => {
child.kill();
await closed;
};
const live = (pid) => {
try {
process.kill(pid, 0);
@@ -516,32 +560,14 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
"cleanup-retry-",
);
const heldPath = join(retryRuntime.observation.path, "snapshot.bin");
await retryRuntime.snapshot(source, "snapshot.bin");
const holder = spawn(
"powershell.exe",
[
"-NoProfile",
"-NonInteractive",
"-Command",
"$ErrorActionPreference = 'Stop'; " +
"$stream = [IO.File]::Open($env:REA_CLEANUP_HELD_PATH, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::ReadWrite); " +
"try { [Console]::WriteLine('ready'); [Console]::In.ReadLine() | Out-Null } " +
"finally { $stream.Dispose() }",
],
{
env: { ...process.env, REA_CLEANUP_HELD_PATH: heldPath },
stdio: ["pipe", "pipe", "pipe"],
},
);
const holderClosed = once(holder, "close");
void holderClosed.catch(() => undefined);
let holder;
let primaryError;
try {
const [ready] = await once(holder.stdout, "data", {
signal: AbortSignal.timeout(15_000),
});
assert.equal(String(ready).trim(), "ready");
await retryRuntime.snapshot(source, "snapshot.bin");
holder = await startDeleteSharingDeniedReader(heldPath);
// The reader denies delete sharing. Cleanup must retain ownership when
// its handle-relative open receives ERROR_SHARING_VIOLATION (32).
// its handle-relative open receives ERROR_SHARING_VIOLATION (32) through
// the bounded automatic retries; a later explicit close must succeed.
await assert.rejects(retryRuntime.close(), { win32Code: 32 });
assert.equal(
windowsPrivateRuntime(retryRuntime.observation.path),
@@ -552,8 +578,8 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
() => retryRuntime.writeFile("after-close.txt", "forbidden"),
/cleanup is pending/u,
);
holder.stdin.end("\n");
const [exitCode] = await holderClosed;
holder.child.stdin.end("\n");
const [exitCode] = await holder.closed;
assert.equal(exitCode, 0);
await retryRuntime.close();
await retryRuntime.close();
@@ -563,10 +589,73 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
/No native private runtime/u,
);
report.controls.cleanupFailureRetainsOwnershipForRetry = true;
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
holder.kill();
await holderClosed.catch(() => undefined);
await retryRuntime.close();
await closeVerifierResources(
[
...(holder === undefined
? []
: [() => stopDeleteSharingDeniedReader(holder)]),
() => retryRuntime.close(),
],
primaryError,
);
}
const transientRuntime = WindowsPrivateRuntime.create(
workspace,
"cleanup-transient-",
);
const transientHeldPath = join(
transientRuntime.observation.path,
"snapshot.bin",
);
let transientHolder;
let transientPrimaryError;
try {
await transientRuntime.snapshot(source, "snapshot.bin");
transientHolder = await startDeleteSharingDeniedReader(transientHeldPath);
// Keep the reader held through the first close attempt. Without retries,
// close rejects before the next event-loop turn; with retries, it remains
// pending until the caller releases this sharing conflict.
let closeSettled = false;
const closing = transientRuntime.close();
void closing.then(
() => {
closeSettled = true;
},
() => {
closeSettled = true;
},
);
await new Promise((resolve) => setImmediate(resolve));
assert.equal(closeSettled, false);
transientHolder.child.stdin.end("\n");
const [, [exitCode]] = await Promise.all([
closing,
transientHolder.closed,
]);
assert.equal(exitCode, 0);
assert.equal(await exists(transientRuntime.observation.path), false);
assert.throws(
() => windowsPrivateRuntime(transientRuntime.observation.path),
/No native private runtime/u,
);
report.controls.cleanupTransientConflictRetries = true;
} catch (cause) {
transientPrimaryError = cause;
throw cause;
} finally {
await closeVerifierResources(
[
...(transientHolder === undefined
? []
: [() => stopDeleteSharingDeniedReader(transientHolder)]),
() => transientRuntime.close(),
],
transientPrimaryError,
);
}
report.controls = {
...report.controls,
+14 -21
View File
@@ -14,11 +14,15 @@ import { join } from "node:path";
import { parse as parseJsonc } from "jsonc-parser";
import { supportedClients } from "../dist/application/SupportedClients.js";
import { promisify } from "node:util";
import { GHIDRA_BRIDGE_SOURCES } from "../src/ghidra/GhidraBridgeSources.ts";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { verifyCompleteToolCatalog } from "./lib/verify-package-core.mjs";
import {
execNpm,
verifyCompleteToolCatalog,
} from "./lib/verify-package-core.mjs";
import { verifyPackageUpdate } from "./verify-package-update.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
@@ -32,9 +36,9 @@ let report;
try {
const packed = JSON.parse(
(
await npm(
await execNpm(
["pack", "--json", "--silent", "--pack-destination", workspace],
root,
{ cwd: root, windowsHide: true },
)
).stdout,
);
@@ -49,7 +53,9 @@ try {
);
const packagedPaths = new Set(packageResult.files.map(({ path }) => path));
for (const required of [
"bridge/ghidra/ReaGhidraBridge.java",
...Object.values(GHIDRA_BRIDGE_SOURCES).map(
(name) => `bridge/ghidra/${name}`,
),
"native/windows/build/manifest.json",
"native/windows/build/rea-windows-x64.node",
"dist/android/JadxWorkspaceRoot.js",
@@ -63,9 +69,9 @@ try {
const tarball = join(workspace, packageResult.filename);
await access(tarball);
await npm(
await execNpm(
["install", "--no-package-lock", "--no-save", "--prefix", prefix, tarball],
workspace,
{ cwd: workspace, windowsHide: true },
);
const entry = join(
prefix,
@@ -209,9 +215,9 @@ try {
throw new Error(`Packaged uninstall did not remove ${id}`);
if (process.platform !== "win32")
await npm(
await execNpm(
["install", "--global", "--ignore-scripts", "--prefix", prefix, tarball],
workspace,
{ cwd: workspace, windowsHide: true },
);
const update = await verifyPackageUpdate({
prefix,
@@ -241,16 +247,3 @@ try {
process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
);
function npm(arguments_, cwd) {
const npmExecPath = process.env.npm_execpath;
return npmExecPath === undefined
? exec(process.platform === "win32" ? "npm.cmd" : "npm", arguments_, {
cwd,
windowsHide: true,
})
: exec(process.execPath, [npmExecPath, ...arguments_], {
cwd,
windowsHide: true,
});
}
@@ -0,0 +1,348 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { createHash, randomUUID } from "node:crypto";
import { createReadStream } from "node:fs";
import { readFile, writeFile } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { performance } from "node:perf_hooks";
import { fileURLToPath } from "node:url";
import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { createEvidence } from "../../../dist/domain/evidence.js";
import { PrivateRuntimeRoot } from "../../../dist/process/PrivateRuntimeRoot.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../../../dist/process/OwnedCommand.js";
import { closeVerifierResources } from "../../lib/verifier-cleanup.mjs";
const { values } = parseArgs({
options: {
artifact: { type: "string" },
"addon-path": { type: "string" },
"native-sha256": { type: "string" },
"source-path": { type: "string", multiple: true },
rounds: { type: "string", default: "3" },
"cli-parity": { type: "boolean", default: false },
"tokenizer-python": { type: "string" },
"require-safe": { type: "boolean", default: false },
},
strict: true,
});
assert.ok(
values.artifact && isAbsolute(values.artifact),
"--artifact must be absolute",
);
assert.ok(
values["addon-path"] &&
!isAbsolute(values["addon-path"]) &&
values["addon-path"]
.split("/")
.every((part) => part !== "" && part !== "." && part !== ".."),
"--addon-path must be a normalized archive-relative path",
);
assert.match(
values["native-sha256"] ?? "",
/^[a-f0-9]{64}$/,
"--native-sha256 pins prior native bytes; it does not prove availability",
);
const rounds = Number(values.rounds);
assert.ok(Number.isSafeInteger(rounds) && rounds > 0);
const environment = {
PATH: process.env.PATH ?? "/usr/bin:/bin",
REA_LOG_LEVEL: "silent",
REA_ANALYSIS_PROVIDER: "auto",
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-deep-provider/hopper",
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [fileURLToPath(new URL("../../rea.mjs", import.meta.url)), "mcp"],
env: environment,
stderr: "pipe",
});
const client = new Client({
name: "javascript-native-handoff-verifier",
version: "1",
});
const serialize = (value) => JSON.stringify(value);
const digest = (value) =>
createHash("sha256").update(serialize(value)).digest("hex");
const fileDigest = async (path) => {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
};
const simulatedNativeEvidence = (path, sha256) =>
createEvidence(
{ path, sha256, format: "raw-binary" },
{ id: "ghidra", name: "Ghidra", version: "simulated" },
{
operation: "binary_overview",
parameters: {},
result: {
summary:
"Simulated subject-linkage input; no native provider executed.",
},
limitations: ["No native provider behavior is established."],
},
);
let temporary;
let primaryError;
let retainTemporary = false;
let report;
const ownedCommand = async (command, arguments_, timeoutMs = 60_000) => {
try {
return await runOwnedCommand(
{
command,
arguments: arguments_,
cwd: temporary.path,
env: environment,
runId: `rea-native-handoff-${randomUUID()}`,
expectedCommand: command === process.execPath ? process.execPath : null,
sweepTokenOwnedProcesses: true,
},
{ timeoutMs, diagnosticBytes: 16 * 1024 * 1024 },
);
} catch (cause) {
retainTemporary ||=
cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined;
throw cause;
}
};
try {
const archiveSha256 = await fileDigest(values.artifact);
const companion = join(`${values.artifact}.unpacked`, values["addon-path"]);
let companionSha256 = null;
try {
companionSha256 = await fileDigest(companion);
} catch (cause) {
if (cause?.code !== "ENOENT") throw cause;
}
if (companionSha256 !== null)
assert.equal(companionSha256, values["native-sha256"]);
const native = simulatedNativeEvidence(companion, values["native-sha256"]);
const container = simulatedNativeEvidence(values.artifact, archiveSha256);
await client.connect(transport);
transport.stderr?.on("data", () => undefined);
const tools = (await client.listTools()).tools;
const catalog = tools.map(({ name, description, inputSchema }) => ({
name,
description,
inputSchema,
}));
const selected = catalog.filter(({ name }) =>
["analyze_javascript_application", "trace_application_feature"].includes(
name,
),
);
assert.equal(selected.length, 2);
const call = async (name, arguments_) => {
const response = await client.callTool(
{ name, arguments: arguments_ },
{ timeout: 360_000 },
);
assert.notEqual(response.isError, true, serialize(response));
assert.ok(response.structuredContent?.evidence_id);
return {
request: { name, arguments: arguments_ },
response: response.structuredContent,
wire_response: response,
};
};
const prepStart = performance.now();
const preparation = await call("analyze_javascript_application", {
input_path: values.artifact,
detail: "summary",
...(values["source-path"] === undefined
? {}
: { source_paths: values["source-path"] }),
});
const preparationMs = performance.now() - prepStart;
assert.equal(preparation.response.subject.digest.sha256, archiveSha256);
const sourceId = preparation.response.normalized_result.parent_evidence_id;
assert.match(sourceId, /^ev_[a-f0-9]{64}$/);
const query = {
application: { kind: "retained-evidence", evidence_id: sourceId },
native_observations: [container, native],
seed: {
kind: "string",
value: values["addon-path"],
match: "exact",
case_sensitive: true,
},
direction: "outgoing",
};
const samples = [];
let first;
for (let round = 0; round < rounds; round++) {
const start = performance.now();
const result = await call("trace_application_feature", query);
const elapsedMs = performance.now() - start;
if (first === undefined) first = result;
else assert.equal(result.response.evidence_id, first.response.evidence_id);
samples.push({
round,
elapsed_ms: elapsedMs,
structured_reply_bytes: Buffer.byteLength(serialize(result.response)),
full_tool_reply_bytes: Buffer.byteLength(serialize(result.wire_response)),
});
}
const trace = first.response.normalized_result;
const addons = trace.graph.nodes.filter(
({ kind }) => kind === "native-addon",
);
assert.equal(
addons.length,
1,
"The selected native add-on must remain visible",
);
const addon = addons[0];
const unavailable = addon.observations.some(
({ properties }) => properties.availability === "unavailable",
);
const expectedLinks = [
sourceId,
...(companionSha256 === null ? [] : [native.evidence_id]),
].sort();
const safeIdentity =
serialize(trace.evidence_links) === serialize(expectedLinks) &&
(companionSha256 === null
? unavailable &&
addon.identity.strategy !== "content-digest" &&
trace.coverage.status === "partial" &&
trace.native_handoffs.length === 0
: addon.identity.strategy === "content-digest" &&
addon.identity.sha256 === companionSha256 &&
trace.native_handoffs.length === 1 &&
trace.native_handoffs[0].artifact_sha256 === companionSha256 &&
trace.native_handoffs[0].status === "evidence-linked" &&
serialize(trace.native_handoffs[0].evidence_ids) ===
serialize([native.evidence_id]));
let cliParity = null;
if (values["cli-parity"]) {
temporary = await PrivateRuntimeRoot.create({
prefix: "rea-native-handoff-verifier-",
});
const bundlePath = join(temporary.path, "bundle.json");
const exported = await client.callTool({
name: "export_evidence_bundle",
arguments: { path: bundlePath },
});
assert.notEqual(exported.isError, true, serialize(exported));
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
const parent = bundle.records.find(
({ evidence_id }) => evidence_id === sourceId,
);
assert.ok(parent, "Export omitted the retained parent");
const request = join(temporary.path, "trace.json");
// The CLI's portable file contract takes complete Evidence; MCP resolves a
// same-session reference into the same shared application workflow.
await writeFile(request, serialize({ ...query, application: parent }));
const result = await ownedCommand(
process.execPath,
[
fileURLToPath(new URL("../../rea.mjs", import.meta.url)),
"trace-application-feature",
request,
"--json",
],
120_000,
);
const evidence = JSON.parse(result.stdout.text);
assert.deepEqual(evidence.normalized_result, trace);
assert.equal(evidence.evidence_id, first.response.evidence_id);
cliParity = {
exact_cli_mcp_trace_parity: true,
evidence_id: evidence.evidence_id,
};
}
let tokenCosts = null;
if (values["tokenizer-python"]) {
temporary ??= await PrivateRuntimeRoot.create({
prefix: "rea-native-handoff-verifier-",
});
const input = join(temporary.path, "token-input.json");
await writeFile(
input,
serialize({ catalog, selected, preparation, query: first }),
);
const result = await ownedCommand(values["tokenizer-python"], [
"-c",
`import json,sys,tiktoken
with open(sys.argv[1],encoding='utf-8') as f:d=json.load(f)
enc=tiktoken.get_encoding('cl100k_base')
def n(v):return len(enc.encode(json.dumps(v,ensure_ascii=False,separators=(',',':')),disallowed_special=()))
requests=n(d['preparation']['request'])+n(d['query']['request'])
structured=n(d['preparation']['response'])+n(d['query']['response'])
full_reply=n(d['preparation']['wire_response'])+n(d['query']['wire_response'])
s=n(d['selected']);a=n(d['catalog'])
print(json.dumps(dict(tokenizer='cl100k_base',tiktoken_version=tiktoken.__version__,selected_catalog=s,full_catalog=a,requests=requests,structured_replies=structured,full_tool_replies=full_reply,selected_catalog_structured_task=s+requests+structured,full_catalog_structured_task=a+requests+structured,selected_catalog_full_reply_task=s+requests+full_reply,full_catalog_full_reply_task=a+requests+full_reply)))`,
input,
]);
tokenCosts = JSON.parse(result.stdout.text);
}
report = {
scope:
"Real shipped-ASAR native identity through public CLI/stdio MCP; simulated native Evidence only, no provider/application executed",
artifact: { path: values.artifact, sha256: archiveSha256 },
addon_path: values["addon-path"],
companion_sha256_observed: companionSha256,
prior_native_sha256_supplied: values["native-sha256"],
source_paths: values["source-path"] ?? null,
parent_evidence_id: sourceId,
trace_evidence_id: first.response.evidence_id,
preparation_ms: preparationMs,
samples,
safe_identity: safeIdentity,
node: {
node_id: addon.node_id,
identity: addon.identity,
unavailable_observed: unavailable,
basis: addon.observations.map(({ evidence }) => ({
authority: evidence.authority,
state: evidence.state,
confidence: evidence.confidence,
artifact: evidence.artifact,
location: evidence.location,
})),
},
native_handoffs: trace.native_handoffs,
evidence_links: trace.evidence_links,
source_coverage: trace.coverage,
graph_coverage: trace.graph.coverage,
facts_sha256: digest({
node: addon,
native_handoffs: trace.native_handoffs,
evidence_links: trace.evidence_links,
coverage: trace.coverage,
}),
cli_parity: cliParity,
token_costs: tokenCosts,
conditions:
"Fresh MCP session, one selected analysis followed by repeated identical retained traces. Filesystem/build caches uncontrolled. Task token counts include one analysis summary and one trace with name/description/inputSchema catalog projections; structured and complete-tool-reply projections are alternatives, not billing. Repeated rounds, export, CLI parity and tokenizer are validation overhead outside that one-task token count but inside total resource receipts. A wrong-identity baseline is not a successful equivalent task or efficiency credit.",
};
if (values["require-safe"])
assert.ok(safeIdentity, "Native identity/linkage oracle failed");
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
await closeVerifierResources(
[
() => client.close(),
() => transport.close(),
async () => {
if (retainTemporary)
throw new Error(
`Command cleanup incomplete; retaining owned inputs at ${temporary?.path}.`,
);
await temporary?.close();
},
],
primaryError,
);
}
console.log(serialize(report));
@@ -32,3 +32,46 @@ describe("app bundle Info.plist magic", () => {
}
});
});
describe("app bundle program directory containment", () => {
it("rejects an escaping programs directory before resolving its child", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-app-programs-"));
try {
const bundle = join(root, "Fixture.app");
const programs = join(bundle, "Contents", "MacOS");
const executable = join(programs, "Missing");
const canonicalBundle = join(root, "Canonical.app");
const outsidePrograms = join(root, "outside");
const plist =
"<plist><dict><key>CFBundleExecutable</key><string>Missing</string></dict></plist>";
await mkdir(join(bundle, "Contents"), { recursive: true });
await writeFile(join(bundle, "Contents", "Info.plist"), plist);
let childResolved = false;
const result = await resolveAppBundleExecutable(bundle, {
readFile: async () => Buffer.from(plist),
realpath: async (path) => {
if (path === bundle) return canonicalBundle;
if (path === programs) return outsidePrograms;
if (path === executable) {
childResolved = true;
throw Object.assign(new Error("missing executable"), {
code: "ENOENT",
});
}
return path;
},
});
expect(result).toMatchObject({
ok: false,
error: {
reason: "app program directory leaves Contents/MacOS",
},
});
expect(childResolved).toBe(false);
} finally {
await rm(root, { recursive: true, force: true });
}
});
});
+17 -4
View File
@@ -219,11 +219,24 @@ const resolveLayoutExecutable = async (
);
const programs = join(bundle, ...layout.programs);
const executable = join(programs, name);
let canonicalBundle: string;
try {
const [canonicalPrograms, canonicalExecutable] = await Promise.all([
fileSystem.realpath(programs),
fileSystem.realpath(executable),
]);
canonicalBundle = await fileSystem.realpath(bundle);
} catch (cause: unknown) {
return err(
new BinaryTargetError(bundle, errorReason(cause, bundle), { cause }),
);
}
try {
const canonicalPrograms = await fileSystem.realpath(programs);
if (!isPathWithinRoot(canonicalBundle, canonicalPrograms))
return err(
new BinaryTargetError(
bundle,
`app program directory leaves ${layout.programsLabel}`,
),
);
const canonicalExecutable = await fileSystem.realpath(executable);
if (!isPathWithinRoot(canonicalPrograms, canonicalExecutable))
return err(
new BinaryTargetError(
+2
View File
@@ -46,6 +46,7 @@ export const directAnalysisCleanupFailure = (
...primary.data,
code: "cleanup_incomplete",
retryable: false,
remediation: cleanup.remediation,
details: {
...primary.data.details,
cleanup: "incomplete",
@@ -86,6 +87,7 @@ export const managedAnalysisCleanupFailure = (
"close_binary",
{
cause: error,
remediationAction: cleanup.remediation.action,
cleanup: { reason: cleanup.message, resources: error.cleanupResources },
diagnostics: {
cleanup_error: cleanup,
+27 -42
View File
@@ -95,46 +95,6 @@ const firstAnalysisCancellation = <Value>(
* Direct callers use `execute`; adapters that own validation use
* `executeValidated` so an MCP input is parsed exactly once.
*/
const NO_PSEUDOCODE =
"The provider reports no pseudocode for this procedure; the batch limitations for procedure_pseudo_code say why.";
/** One batch item from its pseudocode observation, keeping null distinct from unreadable output. */
const batchPseudocodeItem = (
address: string,
procedure: BatchProcedureIdentity,
observed: Result<AnalysisExecution, AnalysisError>,
): BatchDecompileItem => {
const failed = (error: AnalysisError): BatchDecompileItem => ({
address,
procedure,
status: "error",
error: projectAnalysisError(error),
});
if (!observed.ok) return failed(observed.error);
const pseudocode = observed.value.result;
// The contract's null is the provider's own answer that this procedure has
// no pseudocode (an import slot, a failed body); the provider's reason is
// in the batch limitations, so it is not reported as unreadable output.
if (pseudocode === null)
return failed(
new AnalysisCapabilityUnavailableError(
observed.value.provider.id,
"procedure_pseudo_code",
NO_PSEUDOCODE,
{ userMessage: NO_PSEUDOCODE },
),
);
if (typeof pseudocode !== "string" || pseudocode.length === 0)
return failed(
new AnalysisOutputError(
"procedure_pseudo_code",
"provider returned empty pseudocode",
),
);
return { address, procedure, status: "ok", pseudocode };
};
export class EnhancedTools {
constructor(private readonly analysis: AnalysisOperationPort) {}
@@ -415,7 +375,7 @@ export class EnhancedTools {
): Promise<Result<BatchDecompileItem, AnalysisCancelledError>[]> {
return Promise.all(
ready.map(async ({ address, procedure, providerProcedure }) => {
const result = await this.#observe(
const result = await this.#call(
"procedure_pseudo_code",
{ procedure: providerProcedure },
signal,
@@ -423,7 +383,32 @@ export class EnhancedTools {
);
if (!result.ok && isAnalysisCancellation(result.error))
return err(result.error);
return ok(batchPseudocodeItem(address, procedure, result));
const item: BatchDecompileItem = !result.ok
? {
address,
procedure,
status: "error",
error: projectAnalysisError(result.error),
}
: typeof result.value !== "string" || result.value.length === 0
? {
address,
procedure,
status: "error",
error: projectAnalysisError(
new AnalysisOutputError(
"procedure_pseudo_code",
"provider returned empty pseudocode",
),
),
}
: {
address,
procedure,
status: "ok",
pseudocode: result.value,
};
return ok(item);
}),
);
}
@@ -137,7 +137,23 @@ const readPackageMains = (
return mains;
};
const RESOLUTION_SUFFIXES = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
/**
* Extension slots in probe order. Within a slot, files precede directory
* indexes. The CommonJS loader tries `X.json` after `X.js` and before any
* directory index; ESM and TypeScript require an explicit `.json`.
*/
const RESOLUTION_SUFFIXES: readonly (readonly string[])[] = [
[""],
[".ts"],
[".js"],
[".mjs"],
[".cjs"],
[".tsx"],
[".jsx"],
];
const REQUIRE_RESOLUTION_SUFFIXES = RESOLUTION_SUFFIXES.map((slot) =>
slot.includes(".js") ? [...slot, ".json"] : slot,
);
/** Inventory facts an internal specifier resolves against. */
interface SpecifierContext {
@@ -153,6 +169,7 @@ interface SpecifierContext {
const selectPackageMain = (
index: string,
context: SpecifierContext,
slots: readonly (readonly string[])[],
): { to: string; resolution: "internal" | "unresolved" } => {
const directory = posix.dirname(index);
const manifest = context.packageMains.get(directory);
@@ -170,11 +187,12 @@ const selectPackageMain = (
// Node would load a declared entry outside the imported root.
if (logicalPathEscapesRoot(target))
return { to: target, resolution: "unresolved" };
const suffixes = slots.flat();
const selected = context.lookupPath.resolveCandidates([
...RESOLUTION_SUFFIXES.map((suffix) => `${target}${suffix}`),
...RESOLUTION_SUFFIXES.filter((suffix) => suffix !== "").map((suffix) =>
posix.join(target, `index${suffix}`),
),
...suffixes.map((suffix) => `${target}${suffix}`),
...suffixes
.filter((suffix) => suffix !== "")
.map((suffix) => posix.join(target, `index${suffix}`)),
]);
return { to: selected ?? index, resolution: "internal" };
};
@@ -183,6 +201,7 @@ const resolveInternalSpecifier = (
fromPath: string,
specifier: string,
context: SpecifierContext,
slots: readonly (readonly string[])[],
): { to: string; resolution: "internal" | "unresolved" } => {
if (posix.isAbsolute(specifier))
return { to: specifier, resolution: "unresolved" };
@@ -196,9 +215,13 @@ const resolveInternalSpecifier = (
specifier.endsWith("/") || [".", ".."].includes(posix.basename(specifier));
// A directory resolves only to an index with an extension; neither Node nor
// TypeScript selects an extensionless `index` file.
const candidates = RESOLUTION_SUFFIXES.flatMap((suffix) => [
...(directoryReference ? [] : [`${normalized}${suffix}`]),
...(suffix === "" ? [] : [posix.join(normalized, `index${suffix}`)]),
const candidates = slots.flatMap((slot) => [
...(directoryReference
? []
: slot.map((suffix) => `${normalized}${suffix}`)),
...slot
.filter((suffix) => suffix !== "")
.map((suffix) => posix.join(normalized, `index${suffix}`)),
]);
// Retain present runtime files first; substitute only a missing counterpart.
// Counterparts cover explicit filenames, not directory/index resolution.
@@ -214,7 +237,7 @@ const resolveInternalSpecifier = (
posix.basename(match).startsWith("index.") &&
posix.dirname(match).normalize("NFC") === normalized.normalize("NFC");
return resolvedThroughIndex
? selectPackageMain(match, context)
? selectPackageMain(match, context, slots)
: { to: match, resolution: "internal" };
};
@@ -250,6 +273,9 @@ const appendFile = (
relationship.from_path,
relationship.to,
context,
relationship.kind === "requires"
? REQUIRE_RESOLUTION_SUFFIXES
: RESOLUTION_SUFFIXES,
);
output.relationships.push({
...relationship,
+5 -6
View File
@@ -596,18 +596,17 @@ export class BinarySession
/** Describe the active binary session. */
status(): JsonValue {
const client = this.#active?.client ?? this.#pendingCleanup?.client;
return binarySessionStatus({
target: this.#active?.target,
route: this.#currentRoute(),
router: this.#providerRouter,
runtimeUnavailability: this.#runtimeUnavailability,
runId: this.#active?.runId,
runtimeLineageSnapshots:
this.#active?.client.runtimeLineageSnapshots?.() ?? [],
requestActivitySnapshots:
this.#active?.client.requestActivitySnapshots?.() ?? [],
providerOperationHealth:
this.#active?.client.operationHealthSnapshot?.() ?? null,
runtimeLineageSnapshots: client?.runtimeLineageSnapshots?.() ?? [],
requestActivitySnapshots: client?.requestActivitySnapshots?.() ?? [],
providerOperationHealth: client?.operationHealthSnapshot?.() ?? null,
cleanupPending: this.#pendingCleanup !== undefined,
});
}
@@ -30,6 +30,7 @@ interface BinarySessionStatusInput {
readonly runtimeLineageSnapshots: readonly ProviderRuntimeLineageSnapshot[];
readonly requestActivitySnapshots: readonly ProviderRequestActivitySnapshot[];
readonly providerOperationHealth?: ProviderOperationHealth | null;
readonly cleanupPending?: boolean;
}
/** Project internal provider routing state into the caller-visible session status. */
@@ -42,6 +43,7 @@ export const binarySessionStatus = ({
runtimeLineageSnapshots,
requestActivitySnapshots,
providerOperationHealth,
cleanupPending = false,
}: BinarySessionStatusInput): JsonValue => {
const routeCapabilities = [...route.capabilities.values()]
.filter(({ operation }) => operation !== "inventory_artifact")
@@ -84,6 +86,7 @@ export const binarySessionStatus = ({
),
}));
const common = {
cleanup_pending: cleanupPending,
capabilities,
analysis_provider_binding: analysisProviderBinding,
analysis_provider_candidates: analysisProviderCandidates,
@@ -9,6 +9,7 @@ import {
type ProviderIdentity,
} from "../AnalysisProvider.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { err, ok } from "../../domain/result.js";
import { CompositeProvider } from "./CompositeProvider.js";
@@ -44,10 +45,15 @@ const failingProvider = (
resource: string,
) => {
const providerIdentity = identity(id);
const cleanupError = new ProviderCleanupError(id, [resource], {
reason: `${id} cleanup is unresolved`,
child_marker: `${id}-detail`,
});
const cleanupError = new ProviderCleanupError(
id,
[resource],
{
reason: `${id} cleanup is unresolved`,
child_marker: `${id}-detail`,
},
{ remediationAction: `Resolve ${resource} before retrying ${id}.` },
);
const close = vi.fn(async () => err(cleanupError));
const provider: AnalysisProvider = {
identity: () => providerIdentity,
@@ -98,6 +104,13 @@ it("retains every failed child resource across composite cleanup retries", async
},
});
expect(closed.error).toBeInstanceOf(ProviderCleanupError);
const projected = projectAnalysisError(closed.error);
expect(projected.remediation.action).toContain(
"Resolve left-db before retrying provider-left.",
);
expect(projected.remediation.action).toContain(
"Resolve right-db before retrying provider-right.",
);
expect(closed.error.diagnostics).toMatchObject({
failures: [
{
@@ -127,3 +140,97 @@ it("preserves one child's typed cleanup error unchanged", async () => {
if (closed.ok) throw new Error("Expected child cleanup failure");
expect(closed.error).toBe(child.cleanupError);
});
it("does not close a child again after its cleanup succeeded", async () => {
const firstIdentity = identity("provider-first");
const secondIdentity = identity("provider-second");
let secondCloseAttempts = 0;
const firstClose = vi.fn(async () => ok(null));
const secondClose = vi.fn(async () => {
secondCloseAttempts += 1;
return secondCloseAttempts === 1
? err(new ProviderCleanupError("provider-second", ["second-db"], {}))
: ok(null);
});
const makeProvider = (
providerIdentity: ProviderIdentity,
operation: Exclude<AnalysisOperation, "health">,
close: AnalysisClient["close"],
): AnalysisProvider => ({
identity: () => providerIdentity,
capabilities: () => [operationCapability(providerIdentity, operation)],
createClient: () => ({
execute: async () => ok(createAnalysisExecution(null, providerIdentity)),
close,
}),
});
const client = new CompositeProvider([
makeProvider(firstIdentity, "address_name", firstClose),
makeProvider(secondIdentity, "procedure_info", secondClose),
]).createClient(executableTarget);
await client.execute("address_name", {});
await client.execute("procedure_info", {});
expect((await client.close({ retainDocument: true })).ok).toBe(false);
expect((await client.close()).ok).toBe(true);
expect(firstClose).toHaveBeenCalledTimes(1);
expect(secondClose).toHaveBeenCalledTimes(2);
expect((await client.close({ retainDocument: true })).ok).toBe(true);
expect(firstClose).toHaveBeenCalledTimes(1);
expect(secondClose).toHaveBeenCalledTimes(2);
});
it.each(["busy", "unknown", "idle", undefined] as const)(
"keeps partial health truthful with child state %s and an unobserved peer",
async (state) => {
const observedIdentity = identity("provider-observed");
const unobservedIdentity = identity("provider-unobserved");
const observed: AnalysisProvider = {
identity: () => observedIdentity,
capabilities: () => [
operationCapability(observedIdentity, "address_name"),
],
createClient: () => ({
execute: async () =>
ok(createAnalysisExecution(null, observedIdentity)),
operationHealthSnapshot: () =>
state === undefined
? undefined
: {
state,
stage: state === "idle" ? null : "analysis",
retryAction:
state === "busy"
? "wait"
: state === "idle"
? null
: "unknown",
exitCode: null,
requests: [],
},
close: async () => ok(null),
}),
};
const unobserved: AnalysisProvider = {
identity: () => unobservedIdentity,
capabilities: () => [
operationCapability(unobservedIdentity, "procedure_info"),
],
createClient: () => ({
execute: async () =>
ok(createAnalysisExecution(null, unobservedIdentity)),
close: async () => ok(null),
}),
};
const client = new CompositeProvider([observed, unobserved]).createClient(
executableTarget,
);
await client.execute("address_name", {});
await client.execute("procedure_info", {});
if (state === "idle" || state === undefined)
expect(client.operationHealthSnapshot?.()).toBeUndefined();
else expect(client.operationHealthSnapshot?.()).toMatchObject({ state });
expect((await client.close()).ok).toBe(true);
},
);
+35 -16
View File
@@ -1,6 +1,7 @@
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { err, ok } from "../../domain/result.js";
import type { BinaryTarget } from "../../domain/binaryTargetTypes.js";
import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js";
@@ -27,18 +28,20 @@ const compositeProviderIdentity = (
version: null,
});
const closeCompositeClients =
(
identity: ProviderIdentity,
clients: ReadonlyMap<AnalysisProvider, AnalysisClient>,
): AnalysisClient["close"] =>
async (options) => {
const closeCompositeClients = (
identity: ProviderIdentity,
clients: ReadonlyMap<AnalysisProvider, AnalysisClient>,
): AnalysisClient["close"] => {
const completed = new Set<AnalysisClient>();
return async (options) => {
const outcomes = await Promise.all(
[...clients.entries()].map(([provider, client]) =>
closeAnalysisClient(client, provider.identity().id, options).then(
(result) => ({ providerId: provider.identity().id, result }),
),
),
[...clients.entries()].map(async ([provider, client]) => {
const providerId = provider.identity().id;
if (completed.has(client)) return { providerId, result: ok(null) };
const result = await closeAnalysisClient(client, providerId, options);
if (result.ok) completed.add(client);
return { providerId, result };
}),
);
const failures = outcomes.flatMap(({ providerId, result }) =>
result.ok ? [] : [{ providerId, error: result.error }],
@@ -68,6 +71,13 @@ const closeCompositeClients =
})),
},
{
remediationAction: [
...new Set(
failures.map(
({ error }) => projectAnalysisError(error).remediation.action,
),
),
].join("\n"),
cause: new AggregateError(
failures.map(({ error }) => error),
"Composed provider cleanup failed.",
@@ -76,6 +86,7 @@ const closeCompositeClients =
),
);
};
};
const executeCompositeHealth = async (
identity: ProviderIdentity,
@@ -213,13 +224,21 @@ export class CompositeProvider implements AnalysisProvider {
const snapshots = [...clients.values()].map((client) =>
client.operationHealthSnapshot?.(),
);
const failed = snapshots.find(
(snapshot) =>
snapshot !== undefined &&
(snapshot.state === "exited" ||
snapshot.state === "unreachable" ||
snapshot.state === "not_started"),
);
if (failed !== undefined) return failed;
const nonIdle = snapshots.find(
(snapshot) => snapshot !== undefined && snapshot.state !== "idle",
);
if (nonIdle !== undefined) return nonIdle;
if (snapshots.some((snapshot) => snapshot === undefined))
return undefined;
return (
snapshots.find(
(snapshot) => snapshot !== undefined && snapshot.state !== "idle",
) ?? snapshots[0]
);
return snapshots[0];
},
close,
};
@@ -12,6 +12,7 @@ import {
} from "./JavaScriptSemanticGraphConstruction.js";
import { unknownSemanticEvidence } from "./JavaScriptSemanticGraphEvidence.js";
import type { SemanticFlowProjectionContext } from "./JavaScriptSemanticGraphFlowProjection.js";
import { semanticResourceLimitReason } from "../../domain/javascript/javascriptSemanticResourceLimits.js";
/** Project environment, argv, file, default, and precedence candidates. */
export const projectSemanticConfiguration = (
@@ -56,7 +57,17 @@ export const projectSemanticConfiguration = (
...(operation.kind === "default" ? (["overrides"] as const) : []),
],
location: operation.location,
detail: `Static ${operation.kind} configuration identity is partial.`,
...(operation.valueResourceLimit === undefined
? {
detail:
operation.kind === "default"
? "Static default configuration value is partial."
: `Static ${operation.kind} configuration identity is partial.`,
}
: {
reason: "resource-limit",
detail: semanticResourceLimitReason(operation.valueResourceLimit),
}),
candidateNodeIds: [],
});
}
@@ -80,6 +91,9 @@ const addConfigurationNode = (
key: operation.key,
source_kind: operation.kind,
value: operation.value,
...(operation.valueResourceLimit === undefined
? {}
: { value_resource_limit: operation.valueResourceLimit }),
},
});
@@ -279,6 +293,7 @@ const bindingOrOwner = (
: (context.callableNodes.get(callableId) ?? context.moduleNode));
interface DataUnknownInput {
readonly reason?: "ambiguous-target" | "resource-limit";
readonly context: SemanticFlowProjectionContext;
readonly node: JavaScriptSemanticGraphNode;
readonly family: "boundary" | "configuration" | "request";
@@ -317,7 +332,7 @@ const addDataUnknown = (input: DataUnknownInput): void => {
node_id: node.node_id,
family,
relation_kinds: relationKinds,
reason: "ambiguous-target",
reason: input.reason ?? "ambiguous-target",
detail,
candidate_node_ids: candidateNodeIds,
evidence: unknownSemanticEvidence(context.file, location),
+34 -7
View File
@@ -127,6 +127,13 @@ class NodeFileReader extends Reader<string> {
const handle = this.#handle;
if (handle === undefined) throw new Error("ZIP reader is not initialized");
const readBytes = Math.min(length, this.size - index);
// A central-directory offset can point past EOF. Type that malformed ZIP
// read before a negative size reaches Buffer.alloc and becomes an I/O error.
if (readBytes < 0)
throw new ArtifactReaderFailure(
"format",
`ZIP archive is malformed: read at ${index} extends beyond the ${this.size}-byte archive`,
);
if (
this.maximumReadBytes !== undefined &&
readBytes > this.maximumReadBytes
@@ -191,6 +198,7 @@ export class ZipArtifactReader implements ArtifactReader {
// ZIP member identity is its directory occurrence, not its filename.
// Logical path uniqueness belongs to the artifact inventory boundary.
const metadata = projectZipEntry(entry, index++);
if (metadata === undefined) continue;
// zip.js enforces each member's declared output size, so this sum bounds streamed work.
if (
metadata.kind === "file" &&
@@ -258,19 +266,38 @@ export class ZipArtifactReader implements ArtifactReader {
}
}
const projectZipEntry = (entry: Entry, index: number): ArtifactEntry => {
const symlink = isSymlink(entry);
const projectZipEntry = (
entry: Entry,
index: number,
): ArtifactEntry | undefined => {
const kind = isSymlink(entry)
? "symlink"
: entry.directory
? "directory"
: "file";
const sourcePath = zipEntryPath(entry);
const path = sourcePath.replace(/^(?:\.\/)+/u, "");
// Root directory markers carry no inventory occurrence. A file with this
// spelling is still passed to the shared path validator and rejected there.
if (kind === "directory" && (path === "" || path === ".")) return undefined;
return {
path: zipEntryPath(entry),
kind: entry.directory ? "directory" : symlink ? "symlink" : "file",
declaredSize: entry.directory ? null : entry.uncompressedSize,
compressedSize: entry.directory ? null : entry.compressedSize,
path,
kind,
declaredSize: kind === "directory" ? null : entry.uncompressedSize,
compressedSize: kind === "directory" ? null : entry.compressedSize,
executable: entry.executable,
encrypted: entry.encrypted,
byteOffset: null,
declaredSha256: null,
unpacked: false,
limitations: symlink ? ["Archive symlink target was not read."] : [],
limitations: [
...(kind === "symlink" ? ["Archive symlink target was not read."] : []),
...(path === sourcePath
? []
: [
`ZIP member name ${JSON.stringify(sourcePath)} has a leading current-directory prefix omitted from its logical path.`,
]),
],
adapterKey: `zip-entry:${index}`,
};
};
@@ -84,6 +84,8 @@ export const analyzeInterfaceBuilderBundle = async (input: {
recordOmittedArchive(entry.path, "max_documents");
unreportedOmittedArchives += selection.unreportedOmitted;
for (const entry of selection.selected) {
// Plist representation reserved for this document before decoding.
let plistReservation = 0;
try {
const remainingInputBytes =
MAX_AGGREGATE_INPUT_BYTES - aggregateInputBytes;
@@ -132,6 +134,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
decodedReservation.estimatedBytes,
"plist representation exceeds the aggregate Interface Builder decode budget",
);
plistReservation = decodedReservation.estimatedBytes;
binaryPrototypeKeyObserved =
decodedReservation.binaryPrototypeKeyObserved;
}
@@ -165,6 +168,9 @@ export const analyzeInterfaceBuilderBundle = async (input: {
} catch (cause: unknown) {
if (input.signal?.aborted === true) throw cause;
if (cause instanceof InterfaceBuilderDecodeBudgetExceeded) {
// A document rejected while decoding, e.g. beyond the shared depth
// limit, retains no representation; later documents keep its share.
decodeBudget.release(plistReservation);
omitted += 1;
recordOmittedArchive(entry.path, cause.reason);
continue;
@@ -700,6 +706,24 @@ const readEntry = async (
return Buffer.concat(chunks, length);
};
/**
* The shared parser bounds plist value nesting before its recursive decode and
* reports the limit as a RangeError; that is an omitted archive, not a
* malformed one.
*/
const parseXmlWithinDecodeBudget = (text: string) => {
try {
return parseXmlPropertyList(text);
} catch (cause: unknown) {
if (cause instanceof RangeError)
throw new InterfaceBuilderDecodeBudgetExceeded(
"aggregate_decode_budget_exhausted",
cause.message,
);
throw cause;
}
};
/** Project decoded plist data and dates while retaining omitted-key coverage. */
const decodePlist = (
bytes: Buffer,
@@ -716,7 +740,7 @@ const decodePlist = (
value: parseBinary(bytes),
omittedPrototypeKeys: binaryPrototypeKeyObserved ? null : 0,
}
: parseXmlPropertyList(xmlText ?? decodeXmlPlistText(bytes));
: parseXmlWithinDecodeBudget(xmlText ?? decodeXmlPlistText(bytes));
const projected = projectPlistValue(value);
return {
value: projected.value,
@@ -49,6 +49,14 @@ export class InterfaceBuilderDecodeBudget {
this.#remainingBytes -= bytes;
}
/** Return a reservation whose representation was rejected and not retained. */
release(bytes: number): void {
this.#remainingBytes = Math.min(
this.#initialBytes,
this.#remainingBytes + bytes,
);
}
/** Reserve an upper bound for one JSON-serialized string occurrence. */
reserveJsonStringOccurrence(value: string): void {
// JSON escaping uses at most six ASCII bytes per UTF-16 code unit, plus
@@ -117,7 +125,8 @@ const estimateXmlPlistExpansion = (
const xml = xmlText ?? decodeXmlPlistText(bytes);
const decodedSourceBytes = xml.length * 2;
let elementCount = 0;
let xmlDepth = 0;
// Value nesting is bounded by the shared plist parser, which counts plist
// value depth; raw element depth would also count the envelope and keys.
for (const token of xmlPreflightTokens(xml)) {
if (token.doctype) {
// Only a declaration outside opaque text can introduce an internal
@@ -130,15 +139,6 @@ const estimateXmlPlistExpansion = (
continue;
}
elementCount += 1;
if (token.closing) xmlDepth -= 1;
else if (!token.selfClosing) {
xmlDepth += 1;
if (xmlDepth > 128)
throw new InterfaceBuilderDecodeBudgetExceeded(
"aggregate_decode_budget_exhausted",
"XML plist nesting exceeds the aggregate Interface Builder decode budget",
);
}
if (decodedSourceBytes * 2 + elementCount * 128 > maxBytes)
return maxBytes + 1;
}
@@ -179,13 +179,7 @@ function* xmlPreflightTokens(xml: string) {
}
if (end === xml.length) return;
cursor = end + 1;
if (doctype || element)
yield {
doctype,
internalSubset,
closing: xml[start + 1] === "/",
selfClosing: xml[end - 1] === "/",
};
if (doctype || element) yield { doctype, internalSubset };
}
}
@@ -121,6 +121,43 @@ describe("source-map artifact relationships", () => {
);
},
);
it("keeps a directory named like a source map contained by its parent", () => {
const archive = createOccurrence(entry("app", "directory"), "app", null);
archive.artifact_id = contentId("1");
const script = createOccurrence(
entry("app/main.js", "file"),
"app/main.js",
archive.occurrence_id,
);
script.artifact_id = contentId("2");
const directory = createOccurrence(
entry("app/main.js.map", "directory"),
"app/main.js.map",
archive.occurrence_id,
);
directory.artifact_id = contentId("3");
const { edges, limitations } = createArtifactEdges(contentId("4"), [
archive,
script,
directory,
]);
expect(directory).toMatchObject({
artifact_kind: "container",
artifact_format: "directory",
});
expect(
edges.find(
({ occurrence_id }) => occurrence_id === directory.occurrence_id,
),
).toMatchObject({
relation: "contains",
parent_artifact_id: contentId("1"),
});
expect(limitations).toEqual([]);
});
});
const fat = (wide = false): Buffer =>
@@ -295,7 +295,9 @@ export const createArtifactEdges = (
const edges: ArtifactEdge[] = [];
const limitations: string[] = [];
for (const occurrence of occurrences) {
const isSourceMap = occurrence.logical_path.toLowerCase().endsWith(".map");
// Use the occurrence's own classification: a directory named `x.map` is
// a container, not a source map of its sibling.
const isSourceMap = occurrence.artifact_kind === "source-map";
const sourcePath = isSourceMap
? occurrence.logical_path.slice(0, -".map".length)
: undefined;
@@ -337,8 +339,10 @@ export const createArtifactEdges = (
relation:
occurrence.entry_kind === "slice"
? ("slice-of" as const)
: isSourceMap && mappedSource === undefined
? ("contains" as const)
: isSourceMap
? mappedSource === undefined
? ("contains" as const)
: ("maps-source" as const)
: relationFor(occurrence.logical_path),
occurrence_id: occurrence.occurrence_id,
};
@@ -587,11 +591,8 @@ const classifyFatArtifact = (
return identity.state === "mach-o" ? "mach-o-universal" : "unknown";
};
const relationFor = (path: string): ArtifactEdge["relation"] => {
const lower = path.toLowerCase();
if (lower.endsWith(".map")) return "maps-source";
if (lower.includes(".framework/")) return "embeds";
return "contains";
};
/** Relation of a non-source-map occurrence to its parent. */
const relationFor = (path: string): ArtifactEdge["relation"] =>
path.toLowerCase().includes(".framework/") ? "embeds" : "contains";
const depth = (path: string): number => path.split("/").length;
@@ -57,6 +57,7 @@ const browserScenarioLimitations = (
? [
"CDP attachment cannot recover pre-attach events or guarantee launch-time context options.",
`Attached locale ${scenario.environment.locale} applies to Intl formatting only; navigator.language(s) and Accept-Language remain caller-controlled. Use launch mode for matching page and request languages.`,
"Detaching a CDP page session may clear caller-owned navigator.platform emulation. REA cannot snapshot the caller's complete user-agent/client-hint state before attachment; use launch mode to avoid mutating that external browser.",
]
: []),
];
@@ -78,6 +78,10 @@ const scenario = (
},
});
const connectedPlatformEmulationLimitation = expect.stringContaining(
"Detaching a CDP page session may clear caller-owned navigator.platform emulation",
);
type SnapshotKind = BrowserScenario["capture"]["after_each_step"][number];
interface FakeSessionFailures {
@@ -355,7 +359,9 @@ describe("PlaywrightBrowserScenarioProvider", () => {
expect(session.closeCalls).toBe(1);
expect(result.value.completeness.equality_eligible).toBe(false);
});
});
describe("connected browser scenario captures", () => {
it("disconnects external CDP sessions and marks pre-attach events missing", async () => {
const session = new FakeSession("connect");
const provider = new PlaywrightBrowserScenarioProvider(() =>
@@ -374,8 +380,37 @@ describe("PlaywrightBrowserScenarioProvider", () => {
equality_eligible: false,
missing_sections: ["events"],
});
expect(result.value.limitations).toEqual(
expect.arrayContaining([connectedPlatformEmulationLimitation]),
);
expect(session.closeCalls).toBe(1);
});
it("retains the attached platform-emulation limitation in partial observations", async () => {
const session = new FakeSession("connect", {
capture: { call: 1, error: new Error("fixture capture failure") },
});
const provider = new PlaywrightBrowserScenarioProvider(() =>
Promise.resolve(session),
);
const result = await provider.captureScenario(
scenario({ mode: "connect" }),
);
expect(result.ok).toBe(false);
if (result.ok) return;
expect(projectAnalysisError(result.error)).toMatchObject({
details: {
partial_observation: {
capture: {
limitations: expect.arrayContaining([
connectedPlatformEmulationLimitation,
]),
},
},
},
});
});
});
describe("PlaywrightBrowserScenarioProvider timing", () => {
+7 -11
View File
@@ -14,6 +14,7 @@ import { parseJavaScriptSource } from "../domain/javascript/javascriptSourcePars
import {
isTypeOnlyExportDeclaration,
isTypeOnlyImportDeclaration,
semanticImportEqualsSpecifier,
} from "../domain/javascript/javascriptModuleDeclaration.js";
import { stringValue } from "../domain/javascript/javascriptStaticAnalysisHelpers.js";
import type {
@@ -1100,16 +1101,11 @@ const originalDependency = (
: { kind: "require", specifier, requireCallee: node.callee };
}
// `import x = require("m")` is only legal at module top level and always
// refers to the host loader, so it needs no unbound-`require` check. The
// `moduleReference` is an entity name for a local alias instead, which
// declares no dependency.
if (
t.isTSImportEqualsDeclaration(node) &&
t.isTSExternalModuleReference(node.moduleReference)
)
return {
kind: "require",
specifier: node.moduleReference.expression.value,
};
// refers to the host loader, so it needs no unbound-`require` check. An
// entity-name alias or an erased `import type` declares no dependency.
if (t.isTSImportEqualsDeclaration(node)) {
const specifier = semanticImportEqualsSpecifier(node);
return specifier === null ? null : { kind: "require", specifier };
}
return null;
};
+9 -13
View File
@@ -1,16 +1,15 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import { AdbDeviceAnalysisService } from "../application/adb/AdbDeviceAnalysisService.js";
import type { AdbDeviceAnalysisPort } from "../application/adb/AdbDeviceAnalysisPort.js";
import { createAdbDeviceAnalysisProvider } from "../composition/adb.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AdbOperation } from "../domain/adb/adbDeviceAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit ADB requests through one-shot CLI commands. */
export const registerAdbCommands = (
@@ -24,17 +23,14 @@ export const registerAdbCommands = (
const service = new AdbDeviceAnalysisService(provider);
const execute = (name: string, operation: AdbOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError)
return projectAnalysisError(cause);
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
{ close: () => provider.close() },
"adb",
operation,
),
),
);
const serial = z
+9 -13
View File
@@ -1,14 +1,13 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { AndroidAnalysisService } from "../application/android/AndroidAnalysisService.js";
import { createAndroidAnalysisProvider } from "../composition/android.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit APK requests through one-shot CLI commands. */
export const registerAndroidCommands = (
@@ -20,17 +19,14 @@ export const registerAndroidCommands = (
const service = new AndroidAnalysisService(provider);
const execute = (name: string, operation: AndroidOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError)
return projectAnalysisError(cause);
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
{ close: () => provider.close() },
"jadx",
operation,
),
),
);
const path = z.string().describe("Local standalone APK path");
const className = z.string().describe("Exact fully qualified class name");
+9 -13
View File
@@ -1,11 +1,9 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { ApktoolResourceAnalysisService } from "../application/apktool/ApktoolResourceAnalysisService.js";
import type { ApktoolResourceAnalysisPort } from "../application/apktool/ApktoolResourceAnalysisPort.js";
import { createApktoolResourceAnalysisProvider } from "../composition/apktool.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
apktoolInputSchemas,
type ApktoolOperation,
@@ -13,6 +11,7 @@ import {
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit Apktool requests through one-shot CLI commands. */
export const registerApktoolCommands = (
@@ -26,17 +25,14 @@ export const registerApktoolCommands = (
const service = new ApktoolResourceAnalysisService(provider);
const execute = (name: string, operation: ApktoolOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError)
return projectAnalysisError(cause);
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
{ close: () => provider.close() },
"apktool",
operation,
),
),
);
cli.command(CLI_COMMANDS.inspectApktoolClient, {
+4 -5
View File
@@ -8,6 +8,7 @@ import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { withCommandCancellation } from "./commandCancellation.js";
import type { CliInstance } from "./types.js";
import { cliIntegerArgument } from "./options.js";
/** Register commands that control an already-running Cutter instance. */
export const registerCutterCommands = (
@@ -33,11 +34,9 @@ export const registerCutterCommands = (
.string()
.uuid()
.describe("Session ID copied from list-cutter-sessions."),
expectedGeneration: z.coerce
.number()
.int()
.nonnegative()
.describe("Session generation returned by list-cutter-sessions."),
expectedGeneration: cliIntegerArgument(0).describe(
"Session generation returned by list-cutter-sessions.",
),
command: z
.string()
.min(1)
+13 -18
View File
@@ -1,11 +1,11 @@
import { z } from "incur";
import { evmInterfaceModule } from "../composition/evm.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Resolve operator paths and preserve explicit encoding through the shared workflow. */
export const registerEvmCommands = (
@@ -23,27 +23,22 @@ export const registerEvmCommands = (
.describe("Selected carrier representation"),
}),
run: ({ args }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
evmInterfaceModule.cliCommand,
async () => {
const result = await service.inspect(
withCommandCancellation((signal) =>
logCliCommand(logger, evmInterfaceModule.cliCommand, () =>
finishAnalysisCommand(
() =>
service.inspect(
{
path: absoluteCallerPath(args.path),
encoding: args.encoding,
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
),
service,
"evmole-interface",
"inspect_evm_interface",
),
),
),
});
};
+9 -9
View File
@@ -5,12 +5,12 @@ import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { firmwareInputSchemas } from "../domain/firmware/firmwareAnalysis.js";
import { localPathStringSchema } from "../domain/localPath.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { FirmwareOperation } from "../domain/firmware/firmwareAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** One-shot firmware CLI commands backed by the same workflows as MCP. */
export const registerFirmwareCommands = (
@@ -29,14 +29,14 @@ export const registerFirmwareCommands = (
input: unknown,
) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
try {
const result = await service.execute(operation, input, { signal });
return result.ok ? result.value : projectAnalysisError(result.error);
} finally {
await service.close();
}
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
service,
operation === "inspect_firmware_regions" ? "binwalk" : "unblob",
operation,
),
),
);
const path = localPathStringSchema.describe(
"Local firmware path; never executed",
+14 -19
View File
@@ -1,15 +1,14 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { FlutterBuildAnalysisService } from "../application/flutter/FlutterBuildAnalysisService.js";
import type { FlutterBuildAnalysisPort } from "../application/flutter/FlutterBuildAnalysisPort.js";
import { createFlutterBuildAnalysisProvider } from "../composition/flutter.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { FlutterOperation } from "../domain/flutter/flutterBuildAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** One flutter command: CLI shape and the request it produces. */
interface FlutterCommandSpec {
@@ -74,23 +73,19 @@ export const registerFlutterCommands = (
: { args: spec.argumentSchema, options: spec.optionSchema }),
run: ({ args, options }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, spec.command, async () => {
const settled = await new FlutterBuildAnalysisService(
provider,
).execute(spec.operation, spec.toInput(args, options ?? {}), {
signal,
});
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError)
return projectAnalysisError(cause);
throw cause;
}
return settled.ok
? settled.value
: projectAnalysisError(settled.error);
}),
logCliCommand(logger, spec.command, () =>
finishAnalysisCommand(
() =>
new FlutterBuildAnalysisService(provider).execute(
spec.operation,
spec.toInput(args, options ?? {}),
{ signal },
),
{ close: () => provider.close() },
"flutter",
spec.operation,
),
),
),
});
}
+13 -18
View File
@@ -3,10 +3,10 @@ import { JavaScriptRecoveryService } from "../application/javascript/JavaScriptR
import { createJavaScriptRecoveryProvider } from "../composition/javascriptRecovery.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Share source recovery with MCP while keeping CLI spelling at this boundary. */
export const registerJavaScriptRecoveryCommands = (
@@ -47,13 +47,11 @@ export const registerJavaScriptRecoveryCommands = (
),
}),
run: ({ args, options }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.recoverJavaScriptSources,
async () => {
const result = await service.recover(
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.recoverJavaScriptSources, () =>
finishAnalysisCommand(
() =>
service.recover(
{
path: args.path,
output_directory: args.outputDirectory,
@@ -68,15 +66,12 @@ export const registerJavaScriptRecoveryCommands = (
: { rewrite_level: options.rewriteLevel }),
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
),
service,
"wakaru",
"recover_javascript_sources",
),
),
),
});
};
+9 -13
View File
@@ -1,15 +1,14 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { JebAnalysisService } from "../application/jeb/JebAnalysisService.js";
import type { JebAnalysisPort } from "../application/jeb/JebAnalysisPort.js";
import { createJebAnalysisProvider } from "../composition/jeb.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { JebOperation } from "../domain/jeb/jebAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit JEB requests through one-shot CLI commands. */
export const registerJebCommands = (
@@ -21,17 +20,14 @@ export const registerJebCommands = (
const service = new JebAnalysisService(provider);
const execute = (name: string, operation: JebOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError)
return projectAnalysisError(cause);
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
{ close: () => provider.close() },
"jeb",
operation,
),
),
);
cli.command(CLI_COMMANDS.inspectJebClient, {
+9
View File
@@ -1,3 +1,5 @@
import { z } from "incur";
import {
executableFormatHintSchema,
type ExecutableFormatHint,
@@ -11,6 +13,13 @@ import { analysisProviderSelectorSchema } from "../contracts/providerSelection.j
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
import type { Logger } from "pino";
/** Parse decimal integer tokens before Number coercion can erase their syntax. */
export const cliIntegerArgument = (min: number) =>
z
.string()
.regex(/^[0-9]+$/u, "Expected a decimal integer")
.pipe(z.coerce.number<string>().safe().min(min));
export const providerSelectionOption = analysisProviderSelectorSchema
.optional()
.describe(
+9 -24
View File
@@ -1,16 +1,14 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { PcapAnalysisService } from "../application/pcap/PcapAnalysisService.js";
import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js";
import { createPcapAnalysisProvider } from "../composition/pcap.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { PcapOperation } from "../domain/pcap/pcapAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit pcap requests through one-shot CLI commands. */
export const registerPcapCommands = (
@@ -22,27 +20,14 @@ export const registerPcapCommands = (
const service = new PcapAnalysisService(provider);
const execute = (name: string, operation: PcapOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError) {
if (result.ok) cause.retainPartialObservation(result.value);
return projectAnalysisError(
result.ok
? cause
: analysisErrorWithCleanupFailure(
result.error,
cause,
operation,
),
);
}
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
logCliCommand(logger, name, () =>
finishAnalysisCommand(
() => service.execute(operation, input, { signal }),
{ close: () => provider.close() },
"pcap",
operation,
),
),
);
cli.command(CLI_COMMANDS.inspectPcapClient, {
+27 -58
View File
@@ -2,13 +2,11 @@ import { z } from "incur";
import { createSqliteDatabaseService } from "../composition/sqlite.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Resolve an operator-selected SQLite snapshot through the shared inspection workflow. */
export const registerSqliteCommands = (
@@ -51,61 +49,32 @@ export const registerSqliteCommands = (
}),
run: ({ args, options }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectSqliteDatabase, async () => {
const inspection = await service
.inspect(
{
path: absoluteCallerPath(args.path),
...(options.table === undefined
? {}
: { table: options.table }),
...(options.rowLimit === undefined
? {}
: { row_limit: options.rowLimit }),
...(options.column === undefined
? {}
: { columns: options.column }),
...(options.schemaScope === undefined
? {}
: { schema_scope: options.schemaScope }),
},
{ signal },
)
.then(
(result) => ({ result }),
(cause: unknown) => ({ cause }),
);
try {
await service.close();
} catch (cause: unknown) {
const primary =
"cause" in inspection
? inspection.cause
: inspection.result.ok
? undefined
: inspection.result.error;
if (
primary instanceof AnalysisError &&
cause instanceof AnalysisError
)
throw analysisErrorWithCleanupFailure(
primary,
cause,
"inspect_sqlite_database",
);
if ("cause" in inspection)
throw new AggregateError(
[inspection.cause, cause],
"SQLite inspection and provider cleanup both failed",
{ cause: inspection.cause },
);
throw cause;
}
if ("cause" in inspection) throw inspection.cause;
return inspection.result.ok
? inspection.result.value
: projectAnalysisError(inspection.result.error);
}),
logCliCommand(logger, CLI_COMMANDS.inspectSqliteDatabase, () =>
finishAnalysisCommand(
() =>
service.inspect(
{
path: absoluteCallerPath(args.path),
...(options.table === undefined
? {}
: { table: options.table }),
...(options.rowLimit === undefined
? {}
: { row_limit: options.rowLimit }),
...(options.column === undefined
? {}
: { columns: options.column }),
...(options.schemaScope === undefined
? {}
: { schema_scope: options.schemaScope }),
},
{ signal },
),
service,
"sqlite",
"inspect_sqlite_database",
),
),
),
});
};
+13 -17
View File
@@ -9,6 +9,7 @@ import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Resolve all caller-selected artifact paths through the shared workflow. */
export const registerWasmCommands = (
@@ -56,13 +57,11 @@ export const registerWasmCommands = (
.describe("Explicit local WASM candidates"),
}),
run: ({ args, options }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
wasmArtifactModule.cliCommand,
async () => {
const result = await service.inspect(
withCommandCancellation((signal) =>
logCliCommand(logger, wasmArtifactModule.cliCommand, () =>
finishAnalysisCommand(
() =>
service.inspect(
{
path: absoluteCallerPath(args.path),
glue_paths: options.glue_paths?.map((path) =>
@@ -73,15 +72,12 @@ export const registerWasmCommands = (
),
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
),
service,
"wabt-artifact",
"inspect_wasm_artifact",
),
),
),
});
};
+17 -23
View File
@@ -2,11 +2,12 @@ import { z } from "incur";
import { createWebModuleTraceService } from "../composition/webModules.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { webModuleTraceInputSchema } from "../domain/webModuleTrace.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { cliIntegerArgument } from "./options.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
const importMapInputSchema =
webModuleTraceInputSchema.shape.import_map.unwrap();
@@ -23,11 +24,9 @@ export const registerWebModuleCommands = (
"Trace one captured script's native imports to URLs and captured source candidates",
args: z.object({
manifestPath: webModuleTraceInputSchema.shape.manifest_path,
scriptIndex: z.coerce
.number()
.int()
.min(0)
.describe("Zero-based manifest script index"),
scriptIndex: cliIntegerArgument(0).describe(
"Zero-based manifest script index",
),
}),
options: z.object({
importerUrl: webModuleTraceInputSchema.shape.importer_url,
@@ -41,13 +40,11 @@ export const registerWebModuleCommands = (
.describe("HTTP(S) import-map base URL; requires --import-map-path"),
}),
run: ({ args, options }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.traceWebModuleImports,
async () => {
const result = await service.trace(
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.traceWebModuleImports, () =>
finishAnalysisCommand(
() =>
service.trace(
{
manifest_path: args.manifestPath,
script_index: args.scriptIndex,
@@ -65,15 +62,12 @@ export const registerWebModuleCommands = (
}),
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
),
service,
"web-module-trace",
"trace_web_module_imports",
),
),
),
});
};
+2 -1
View File
@@ -5,6 +5,7 @@ import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { cliIntegerArgument } from "./options.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
@@ -26,7 +27,7 @@ export const registerWebNetworkCaptureCommands = (
}),
options: z.object({
record: z
.array(z.coerce.number().int().nonnegative())
.array(cliIntegerArgument(0))
.optional()
.describe("Zero-based producer ordinals; repeat to select several"),
sensitiveValue: z
+21 -29
View File
@@ -2,10 +2,11 @@ import { z } from "incur";
import { createWebSourceLocationService } from "../composition/webSourceLocations.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { cliIntegerArgument } from "./options.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose the same explicit selected map/point contract through the CLI. */
export const registerWebSourceLocationCommands = (
@@ -21,30 +22,24 @@ export const registerWebSourceLocationCommands = (
manifestPath: z
.string()
.describe("Absolute export_web_scripts manifest path"),
scriptIndex: z.coerce
.number()
.int()
.min(0)
.describe("Zero-based manifest script index"),
scriptIndex: cliIntegerArgument(0).describe(
"Zero-based manifest script index",
),
sourceMapPath: z.string().describe("Absolute local source-map path"),
sourceMapUrl: z
.string()
.describe("Absolute source-map URL context; no fetch"),
line: z.coerce.number().int().min(1).describe("One-based generated line"),
column: z.coerce
.number()
.int()
.min(0)
.describe("Zero-based UTF-16 generated column"),
line: cliIntegerArgument(1).describe("One-based generated line"),
column: cliIntegerArgument(0).describe(
"Zero-based UTF-16 generated column",
),
}),
run: ({ args }) =>
withCommandCancellation(async (signal) => {
try {
return await logCliCommand(
logger,
CLI_COMMANDS.traceWebSourceLocation,
async () => {
const result = await service.trace(
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.traceWebSourceLocation, () =>
finishAnalysisCommand(
() =>
service.trace(
{
manifest_path: args.manifestPath,
script_index: args.scriptIndex,
@@ -55,15 +50,12 @@ export const registerWebSourceLocationCommands = (
generated_position: { line: args.line, column: args.column },
},
{ signal },
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
);
} finally {
await service.close();
}
}),
),
service,
"web-source-location",
"trace_web_source_location",
),
),
),
});
};
+6 -1
View File
@@ -43,7 +43,12 @@ export const defaultHopperLauncherPath = (
export const parseConfig = (
environment: Readonly<Record<string, string | undefined>>,
): Result<AppConfig, ConfigurationError> => {
const parsed = parseEnvironment(environment);
// JEB validates its own timeout during optional provider composition. It is
// not an AppConfig setting and must not prevent unrelated CLI/MCP startup.
// Registration validation still checks the complete environment directly.
const { REA_JEB_MCP_TIMEOUT_MS: _jebTimeout, ...appEnvironment } =
environment;
const parsed = parseEnvironment(appEnvironment);
if (!parsed.ok) return parsed;
const env = parsed.value;
return ok({
+203 -76
View File
@@ -146,6 +146,195 @@ const inputGroup = (branch: Readonly<Record<string, unknown>>): string => {
return [...required, ...optional].join(" + ");
};
type SchemaNormalization = {
readonly schema: unknown;
readonly changed: boolean;
};
const isBooleanLiteralSchema = (schema: unknown): boolean =>
isObject(schema) &&
(typeof schema.const === "boolean" ||
(Array.isArray(schema.enum) &&
schema.enum.length > 0 &&
schema.enum.every((value) => typeof value === "boolean")));
const isBooleanSchema = (schema: unknown): boolean =>
isObject(schema) &&
(schema.type === "boolean" || isBooleanLiteralSchema(schema));
const preserveSharedDefault = (
schema: Record<string, unknown>,
branches: readonly unknown[],
): void => {
if (schema.default !== undefined) return;
const defaults = branches
.filter(isObject)
.map((branch) => branch.default)
.filter((value) => value !== undefined);
if (
defaults.length > 0 &&
defaults.every((value) => isDeepStrictEqual(value, defaults[0]))
)
schema.default = defaults[0];
};
const normalizeAdvertisedSchema = (value: unknown): SchemaNormalization => {
if (!isObject(value)) return { schema: value, changed: false };
const node: Record<string, unknown> = { ...value };
let changed = false;
const normalizeMap = (map: Readonly<Record<string, unknown>>) =>
Object.fromEntries(
Object.entries(map).map(([name, child]) => {
const result = normalizeAdvertisedSchema(child);
changed ||= result.changed;
return [name, result.schema];
}),
);
const normalizeList = (list: readonly unknown[]) =>
list.map((child) => {
const result = normalizeAdvertisedSchema(child);
changed ||= result.changed;
return result;
});
for (const [key, child] of Object.entries(value)) {
if (schemaMaps.has(key) && isObject(child)) {
node[key] = normalizeMap(child);
} else if (
schemaArrays.has(key) &&
key !== "anyOf" &&
key !== "oneOf" &&
Array.isArray(child)
) {
node[key] = normalizeList(child).map((result) => result.schema);
} else if (
schemaChildren.has(key) &&
key !== "not" &&
key !== "if" &&
key !== "then" &&
key !== "else"
) {
const results = Array.isArray(child)
? normalizeList(child)
: [normalizeAdvertisedSchema(child)];
const nestedChanged = results.some((result) => result.changed);
if (
key === "contains" &&
nestedChanged &&
value.maxContains !== undefined
) {
delete node.contains;
delete node.minContains;
delete node.maxContains;
changed = true;
} else {
node[key] = Array.isArray(child)
? results.map((result) => result.schema)
: results[0]?.schema;
changed ||= nestedChanged;
}
}
}
if (typeof node.const === "boolean") {
delete node.const;
if (node.type === undefined) node.type = "boolean";
changed = true;
}
if (
Array.isArray(node.enum) &&
node.enum.length > 0 &&
node.enum.every((item) => typeof item === "boolean")
) {
delete node.enum;
if (node.type === undefined) node.type = "boolean";
changed = true;
}
const negated = Object.hasOwn(value, "not")
? normalizeAdvertisedSchema(value.not)
: undefined;
if (negated?.changed) {
// Widening a predicate under `not` narrows the containing schema.
changed = true;
delete node.not;
} else if (negated !== undefined) {
node.not = negated.schema;
}
const conditional = Object.hasOwn(value, "if")
? normalizeAdvertisedSchema(value.if)
: undefined;
const thenBranch = Object.hasOwn(value, "then")
? normalizeAdvertisedSchema(value.then)
: undefined;
const elseBranch = Object.hasOwn(value, "else")
? normalizeAdvertisedSchema(value.else)
: undefined;
if (conditional?.changed) {
// Changing the predicate can route canonical-valid inputs into a rejecting branch.
changed = true;
delete node.if;
delete node.then;
delete node.else;
} else {
if (conditional !== undefined) node.if = conditional.schema;
if (thenBranch !== undefined) {
node.then = thenBranch.schema;
changed ||= thenBranch.changed;
}
if (elseBranch !== undefined) {
node.else = elseBranch.schema;
changed ||= elseBranch.changed;
}
}
for (const combinator of ["anyOf", "oneOf"] as const) {
const branches = value[combinator];
if (!Array.isArray(branches)) continue;
const normalized = normalizeList(branches);
const schemas = normalized.map((result) => result.schema);
const branchesChanged = normalized.some((result) => result.changed);
if (combinator === "oneOf" && branchesChanged) {
const booleanLiteralsOnly = branches.every(isBooleanLiteralSchema);
if (booleanLiteralsOnly) {
node.type = "boolean";
preserveSharedDefault(node, schemas);
delete node.oneOf;
changed = true;
continue;
}
// `anyOf` retains branch guidance without allowing widened branches to
// reject inputs that were accepted by exactly one canonical branch.
delete node.oneOf;
if (Array.isArray(node.anyOf)) {
const existingAllOf = Array.isArray(node.allOf) ? node.allOf : [];
node.allOf = [...existingAllOf, { anyOf: schemas }];
} else {
node.anyOf = schemas;
}
changed = true;
continue;
}
if (
branchesChanged &&
schemas.length > 0 &&
schemas.every(isBooleanSchema)
) {
node.type = "boolean";
preserveSharedDefault(node, schemas);
delete node[combinator];
changed = true;
continue;
}
node[combinator] = schemas;
changed ||= branchesChanged;
}
return { schema: node, changed };
};
// Anthropic tool input schemas reject anyOf, oneOf, and allOf at the root, and
// clients then drop the tool or advertise it without parameters. Present a
// root union of strict object groups as one object whose properties are the
@@ -215,7 +404,11 @@ export const presentInputJsonSchema = (
): Record<string, unknown> => {
const flattened = flattenRootObjectUnion(root);
const described = describeSchema(flattened, flattened, describeProperty);
if (!isObject(described)) return flattened;
if (!isObject(described))
return normalizeAdvertisedSchema(flattened).schema as Record<
string,
unknown
>;
const {
dependencies: _dependencies,
dependentRequired: _dependentRequired,
@@ -237,85 +430,19 @@ export const presentInputJsonSchema = (
]
: [],
);
if (dependencyGuidance.length === 0) return presentation;
if (dependencyGuidance.length === 0)
return normalizeAdvertisedSchema(presentation).schema as Record<
string,
unknown
>;
const description = [
typeof presentation.description === "string"
? presentation.description
: undefined,
...dependencyGuidance,
].filter((text) => text !== undefined);
return { ...presentation, description: description.join(" ") };
return normalizeAdvertisedSchema({
...presentation,
description: description.join(" "),
}).schema as Record<string, unknown>;
};
/**
* Clients that render advertised inputs into SDK declarations expand
* schema-local references with a shared per-input budget: Pi 1.1.0 code-mode
* allows 32 expansions and degrades every later reference to `unknown`
* (morluto/rea#1632), hiding the fields those references describe. The budget
* counts conservatively — every resolvable, non-cyclic reference occurrence
* in every schema position — so any reference-resolving client stays within
* its measured limit.
*/
export const advertisedReferenceExpansionBudget = 32;
const definitionKeys = new Set(["$defs", "definitions"]);
const countReferenceExpansions = (
node: unknown,
root: Readonly<Record<string, unknown>>,
active: ReadonlySet<string>,
count: { value: number },
): void => {
if (Array.isArray(node)) {
node.forEach((child) =>
countReferenceExpansions(child, root, active, count),
);
return;
}
if (!isObject(node)) return;
if (typeof node.$ref === "string" && !active.has(node.$ref)) {
const target = resolveReference(root, node.$ref);
if (target !== undefined) {
count.value += 1;
countReferenceExpansions(
target,
root,
new Set([...active, node.$ref]),
count,
);
return;
}
}
for (const [key, value] of Object.entries(node)) {
if (definitionKeys.has(key)) continue;
if (schemaMaps.has(key) && isObject(value)) {
for (const child of Object.values(value))
countReferenceExpansions(child, root, active, count);
} else if (schemaArrays.has(key) || schemaChildren.has(key)) {
countReferenceExpansions(value, root, active, count);
}
}
};
/** Conservative client expansion count over every schema position. */
export const countLocalReferenceExpansions = (
schema: Record<string, unknown>,
): number => {
const count = { value: 0 };
countReferenceExpansions(schema, schema, new Set(), count);
return count.value;
};
/**
* Keep the shared-definition form of an advertised input while clients can
* still resolve it. Once rendering would exhaust the reference budget, fall
* back to the inline projection, which carries no shared definitions for the
* nonrecursive schemas this repository advertises, so every field renders
* with complete structure in every reference-resolving client.
*/
export const inputSchemaWithinReferenceExpansionBudget = (
shared: Record<string, unknown>,
inline: () => Record<string, unknown>,
budget: number,
): Record<string, unknown> =>
countLocalReferenceExpansions(shared) <= budget ? shared : inline();
+203 -117
View File
@@ -3,12 +3,7 @@ import { Ajv2020 } from "ajv/dist/2020.js";
import { describe, expect, it } from "vitest";
import { z } from "zod";
import {
advertisedReferenceExpansionBudget,
countLocalReferenceExpansions,
inputSchemaWithinReferenceExpansionBudget,
presentInputJsonSchema,
} from "./inputSchemaPresentation.js";
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
import { toolContract } from "./toolContracts.js";
import {
toolInputSchemaWithMetadata,
@@ -20,27 +15,27 @@ it.each(["draft-2020-12", "draft-07"] as const)(
"honors projection options independently of cached wire transformations in %s",
(target) => {
const range = z.strictObject({ offset: z.number(), length: z.number() });
const canonical = toolInputSchemaWithMetadata({
...toolContract("open_binary"),
inputSchema: z.strictObject({ before: range, after: range }),
});
const canonical = z.strictObject({ before: range, after: range });
const transform = (schema: Record<string, unknown>) => ({
...schema,
title: "Transformed range inputs",
});
const options = { target, libraryOptions: { reused: "inline" as const } };
for (const inlineFirst of [false, true]) {
const options = { target, libraryOptions: { reused: "ref" as const } };
for (const sharedFirst of [false, true]) {
const wrapped = transformAdvertisedInputJsonSchema(canonical, transform);
const project = wrapped["~standard"].jsonSchema.input;
if (inlineFirst) project(options);
const shared = project({ target });
if (sharedFirst) project(options);
const advertised = project({ target });
expect(JSON.stringify(advertised)).not.toContain('"$ref":');
expect(advertised).toEqual(
transform(canonical["~standard"].jsonSchema.input({ target })),
);
const shared = project(options);
expect(JSON.stringify(shared)).toContain('"$ref":');
const inline = project(options);
expect(inline).toEqual(
expect(shared).toEqual(
transform(canonical["~standard"].jsonSchema.input(options)),
);
expect(JSON.stringify(inline)).not.toContain('"$ref":');
expect(project({ target })).toBe(shared);
expect(project({ target })).toBe(advertised);
}
},
);
@@ -239,105 +234,6 @@ describe("advertised schema metadata and validation", () => {
);
});
describe("advertised reference expansion budget", () => {
it("counts references in keyword-named fields, excluding literals", () => {
const fields = ["default", "enum", "examples", "const", "$defs"];
const schema = {
type: "object",
properties: Object.fromEntries(
fields.map((name) => [name, { $ref: "#/$defs/value" }]),
),
$defs: { value: { type: "string" } },
default: { $ref: "#/$defs/value" },
examples: [{ $ref: "#/$defs/value" }],
};
expect(countLocalReferenceExpansions(schema)).toBe(fields.length);
});
const reusedComponent = z
.object({ kind: z.literal("rec"), value: z.string() })
.strict();
const inputWithReusedFields = (count: number) =>
z.strictObject(
Object.fromEntries(
Array.from({ length: count }, (_, index) => [
`field_${String(index).padStart(2, "0")}`,
reusedComponent,
]),
),
);
const advertisedInput = (
canonical: z.ZodType<Readonly<Record<string, unknown>>>,
) =>
toolInputSchemaWithMetadata({
...toolContract("open_binary"),
inputSchema: canonical,
})["~standard"].jsonSchema.input({ target: "draft-2020-12" });
it("keeps shared definitions while every client can resolve them", () => {
const advertised = advertisedInput(inputWithReusedFields(8));
expect(advertised.$defs).toBeDefined();
expect(countLocalReferenceExpansions(advertised)).toBe(8);
});
it("keeps shared definitions exactly at the budget", () => {
const advertised = advertisedInput(
inputWithReusedFields(advertisedReferenceExpansionBudget),
);
expect(advertised.$defs).toBeDefined();
expect(countLocalReferenceExpansions(advertised)).toBe(
advertisedReferenceExpansionBudget,
);
});
it("re-projects without shared definitions once the budget is exceeded", () => {
const advertised = advertisedInput(
inputWithReusedFields(advertisedReferenceExpansionBudget + 1),
);
expect(advertised.$defs).toBeUndefined();
expect(countLocalReferenceExpansions(advertised)).toBe(0);
const properties = advertised.properties as Record<string, unknown>;
expect(properties.field_00).toMatchObject({
type: "object",
properties: { kind: { const: "rec" }, value: { type: "string" } },
required: ["kind", "value"],
});
});
it("keeps the shared form untouched when it fits and swaps it when not", () => {
const shared = {
type: "object",
properties: { value: { $ref: "#/$defs/shared" } },
$defs: { shared: { type: "string" } },
};
const inlined = {
type: "object",
properties: { value: { type: "string" } },
};
expect(
inputSchemaWithinReferenceExpansionBudget(
shared,
() => inlined,
advertisedReferenceExpansionBudget,
),
).toBe(shared);
expect(
inputSchemaWithinReferenceExpansionBudget(
{
...shared,
properties: Object.fromEntries(
Array.from({ length: 33 }, (_, index) => [
`field_${index}`,
{ $ref: "#/$defs/shared" },
]),
),
},
() => inlined,
advertisedReferenceExpansionBudget,
),
).toBe(inlined);
});
});
describe("root input presentation", () => {
it("preserves prototype-named properties in object-root presentation", () => {
const fields = Object.fromEntries([
@@ -483,3 +379,193 @@ describe("root input presentation", () => {
});
});
});
describe("advertised Boolean literals", () => {
it("keeps canonical boolean-literal inputs accepted after advertisement", () => {
const canonical = {
type: "object",
properties: {
enabled: { type: "boolean", const: false },
},
required: ["enabled"],
additionalProperties: false,
};
const advertised = presentInputJsonSchema(canonical, () => undefined);
const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical);
const advertisedValidate = new Ajv2020({ strict: false }).compile(
advertised,
);
expect(canonicalValidate({ enabled: false })).toBe(true);
expect(canonicalValidate({ enabled: true })).toBe(false);
expect(advertisedValidate({ enabled: false })).toBe(true);
expect(advertisedValidate({ enabled: true })).toBe(true);
expect(advertised.properties).toEqual({ enabled: { type: "boolean" } });
});
it("drops affected negation and conditional groups without rejecting canonical inputs", () => {
const negated = {
type: "object",
properties: {
flag: { type: "boolean", not: { const: false } },
},
};
const negatedAdvertisement = presentInputJsonSchema(
negated,
() => undefined,
);
const canonicalNegated = new Ajv2020({ strict: false }).compile(negated);
const advertisedNegated = new Ajv2020({ strict: false }).compile(
negatedAdvertisement,
);
expect(canonicalNegated({ flag: true })).toBe(true);
expect(canonicalNegated({ flag: false })).toBe(false);
expect(advertisedNegated({ flag: true })).toBe(true);
const conditional = {
type: "object",
properties: {
choice: {
if: { const: false },
then: { type: "string" },
else: { type: "boolean" },
},
},
};
const conditionalAdvertisement = presentInputJsonSchema(
conditional,
() => undefined,
);
const canonicalConditional = new Ajv2020({ strict: false }).compile(
conditional,
);
const advertisedConditional = new Ajv2020({ strict: false }).compile(
conditionalAdvertisement,
);
expect(canonicalConditional({ choice: true })).toBe(true);
expect(advertisedConditional({ choice: true })).toBe(true);
expect(conditionalAdvertisement.properties).toEqual({
choice: {},
});
});
});
describe("advertised Boolean schema composition", () => {
it("widens an affected oneOf to anyOf when branches could overlap", () => {
const canonical = {
type: "object",
properties: {
flag: {
oneOf: [{ type: "boolean", const: false }, {}],
},
},
};
const advertised = presentInputJsonSchema(canonical, () => undefined);
const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical);
const advertisedValidate = new Ajv2020({ strict: false }).compile(
advertised,
);
expect(canonicalValidate({ flag: true })).toBe(true);
expect(canonicalValidate({ flag: false })).toBe(false);
expect(advertisedValidate({ flag: true })).toBe(true);
expect(advertised).toHaveProperty(["properties", "flag", "anyOf"]);
});
it("retains nested branch guidance when widening oneOf to anyOf", () => {
const canonical = {
type: "object",
properties: {
selection: {
oneOf: [
{
type: "object",
properties: {
source: { type: "string" },
mode: { type: "boolean", const: false },
},
required: ["source", "mode"],
additionalProperties: false,
},
{
type: "object",
properties: {
target: { type: "string" },
mode: { type: "boolean", const: true },
},
required: ["target", "mode"],
additionalProperties: false,
},
],
},
},
};
const advertised = presentInputJsonSchema(canonical, () => undefined);
const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical);
const advertisedValidate = new Ajv2020({ strict: false }).compile(
advertised,
);
const example = { selection: { source: "capture.json", mode: false } };
expect(canonicalValidate(example)).toBe(true);
expect(advertisedValidate(example)).toBe(true);
expect(advertised).toMatchObject({
properties: {
selection: {
anyOf: [
{ properties: { source: { type: "string" } } },
{ properties: { target: { type: "string" } } },
],
},
},
});
});
it("removes a nested not when widening a boolean item predicate", () => {
const canonical = {
type: "object",
properties: {
values: {
type: "array",
not: { items: { type: "boolean", const: false } },
},
},
};
const advertised = presentInputJsonSchema(canonical, () => undefined);
const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical);
const advertisedValidate = new Ajv2020({ strict: false }).compile(
advertised,
);
expect(canonicalValidate({ values: [true] })).toBe(true);
expect(advertisedValidate({ values: [true] })).toBe(true);
expect(advertised).toHaveProperty(["properties", "values"], {
type: "array",
});
});
it("normalizes schema keywords only and preserves literal payloads", () => {
const literal = {
const: false,
enum: [false],
properties: { nested: { const: true } },
};
const presented = presentInputJsonSchema(
{
type: "object",
properties: {
default: { type: "boolean", const: false },
allowed: { enum: [false, true] },
kind: { type: "string", const: "exact" },
},
default: literal,
examples: [literal],
},
() => undefined,
);
expect(presented.properties).toEqual({
default: { type: "boolean" },
allowed: { type: "boolean" },
kind: { type: "string", const: "exact" },
});
expect(presented.default).toEqual(literal);
expect(presented.examples).toEqual([literal]);
});
});
@@ -463,6 +463,7 @@ export const expandToolAvailability = (
export const sessionProvider = z
.object({
cleanup_pending: z.boolean(),
capabilities: z.array(providerCapabilitySet),
analysis_run: analysisRun,
analysis_activity: analysisActivity,
+7 -26
View File
@@ -2,11 +2,7 @@ import { z } from "zod";
import type { ToolContract } from "./toolContractTypes.js";
import { advertisedInputExamples } from "./advertisedInputExamples.js";
import {
advertisedReferenceExpansionBudget,
inputSchemaWithinReferenceExpansionBudget,
presentInputJsonSchema,
} from "./inputSchemaPresentation.js";
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
import { digestCanonicalValue } from "../domain/canonicalDigest.js";
const PROPERTY_DESCRIPTIONS: Readonly<Record<string, string>> = {
@@ -64,31 +60,16 @@ export const toolInputSchemaWithMetadata = <Contract extends ToolContract>(
contract.inputSchema,
"input",
(project) => (options) => {
const shared = project({
const inline = project({
...options,
libraryOptions: { reused: "ref", ...options.libraryOptions },
libraryOptions: { ...options.libraryOptions, reused: "inline" },
});
const examples = advertisedInputExamples(contract);
return {
// Sharing definitions only helps clients that resolve them. A client
// rendering declarations from a shared expansion budget degrades
// every reference past it to `unknown`, so an input whose shared form
// exceeds the budget is re-projected without shared definitions.
...inputSchemaWithinReferenceExpansionBudget(
presentInputJsonSchema(shared, fallbackPropertyDescription),
() =>
presentInputJsonSchema(
project({
...options,
libraryOptions: {
...options.libraryOptions,
reused: "inline",
},
}),
fallbackPropertyDescription,
),
advertisedReferenceExpansionBudget,
),
// Keep advertised parameters self-contained for clients that do not
// resolve local references. The registered canonical schema remains
// responsible for runtime validation.
...presentInputJsonSchema(inline, fallbackPropertyDescription),
...(examples.length === 0 ? {} : { examples }),
};
},
+1
View File
@@ -19,6 +19,7 @@ export const analysisErrorWithCleanupFailure = (
{ cause: primary },
),
userMessage: projectAnalysisError(primary).message,
remediationAction: projected.remediation.action,
...(primary.capturedOutput === undefined
? {}
: { capturedOutput: primary.capturedOutput }),
+8 -2
View File
@@ -27,6 +27,7 @@ import {
HopperTimeoutError,
} from "./hopperErrors.js";
import { ProviderSelectionError } from "./providerSelectionError.js";
import { ProviderAdapterError } from "./providerAdapterError.js";
import { UnknownRegistryError } from "./unknownRegistryError.js";
import { providerRetryAction } from "./providerOperationHealth.js";
import type { AnalysisError, AnalysisErrorTag } from "./analysisErrorBase.js";
@@ -35,6 +36,11 @@ import type { AnalysisErrorProjection } from "./analysisErrorProjection.js";
export const analysisErrorRemediationAction = (
error: AnalysisError,
): string => {
if (
error instanceof ProviderAdapterError &&
error.remediationAction !== undefined
)
return error.remediationAction;
if (
error instanceof BrowserObservationError &&
error.reason === "service_worker_blocked"
@@ -66,7 +72,7 @@ export const analysisErrorRemediationAction = (
);
if (error instanceof HopperTimeoutError)
return error.operation === undefined
? "Inspect Hopper for a loader or license dialog and review details.launcher. Correct the loader configuration or complete Hopper setup, then open the target again."
? "Review details.launcher and inspect Hopper for ongoing initial analysis or a pending loader, demo, or license dialog. For a large target, increase REA_HOPPER_STARTUP_TIMEOUT_MS in the environment that starts REA and allow at least that long in the MCP client. Resolve retained cleanup before reopening."
: error.providerState === "busy"
? "Check binary_session.analysis_activity, wait for the active Hopper request to finish, then retry."
: "Check binary_session for Hopper health, then retry the operation.";
@@ -287,7 +293,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => {
const hopperErrorUserMessage = (error: AnalysisError): string | undefined => {
if (error instanceof HopperTimeoutError) {
if (error.operation === undefined)
return "REA timed out waiting for Hopper bridge readiness. Hopper may be waiting for a loader or license dialog; inspect its window and the captured launcher outcome before retrying.";
return "REA timed out waiting for Hopper bridge readiness, which includes initial analysis. Hopper may still be analyzing the target or waiting for a loader, demo, or license dialog. Review the captured launcher outcome and resolve retained cleanup before retrying.";
const request = error.operation;
return error.providerState === "busy"
? `Hopper timed out during ${request} while the provider remained busy. Check binary_session.analysis_activity, wait for the active request to finish, then retry.`
@@ -156,6 +156,90 @@ describe("compiled Interface Builder graph projection", () => {
});
});
describe("keyed archive objects with unresolved classes", () => {
it("keeps an archived object whose class reference cannot be resolved", () => {
const parsed = parseInterfaceBuilderRecords({
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: [
"$null",
{ $class: { UID: 99 }, label: "Unresolved object" },
{ $classname: "UIView", $classes: ["UIView", "NSObject"] },
{ $class: { UID: 2 }, "NS.string": "ok" },
"plain scalar",
],
$top: { root: { UID: 3 } },
});
// The class descriptor and the scalar are not archived objects.
expect(parsed.objects).toMatchObject([
{
id: "1",
kind: "other",
class_name: null,
name: "Unresolved object",
attributes: { label: "Unresolved object", $class: { UID: 99 } },
},
{ id: "3", kind: "view", class_name: "UIView" },
]);
expect(parsed.objects[1]?.attributes).not.toHaveProperty("$class");
expect(parsed.objectCount).toBe(2);
});
it("keeps an unresolved hierarchy root's label and coverage in the graph", () => {
const result = buildInterfaceBuilderAnalysis({
targetSha256: hash,
toolVersion: "test",
documents: [
{
relativePath: "Main.nib",
archiveSha256: hash,
documentKind: "nib",
raw: {
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: [
"$null",
{ $class: { UID: 99 }, label: "Unresolved object" },
{ $classname: "UIView", $classes: ["UIView", "NSObject"] },
{ $class: { UID: 2 }, "NS.string": "ok" },
],
$top: { root: { UID: 1 } },
},
},
],
limits: interfaceBuilderLimitsSchema.parse({}),
});
const objectNodes = result.graph.nodes.filter(
({ attributes }) => attributes.interface_builder_object_id !== undefined,
);
expect(objectNodes).toMatchObject([
{
kind: "unknown",
name: "Unresolved object",
attributes: { interface_builder_object_id: "1", class_name: null },
},
{ kind: "view", attributes: { interface_builder_object_id: "3" } },
]);
expect(result.documents[0]?.object_count).toBe(2);
expect(
result.graph.edges.some(
({ relation, to, resolution }) =>
relation === "contains" &&
to === objectNodes[0]?.id &&
resolution === "observed",
),
).toBe(true);
expect(result.graph.coverage).toContainEqual(
expect.objectContaining({
facet: "objects:Main.nib",
status: "complete",
examined: 2,
omitted: 0,
}),
);
});
});
describe("Interface Builder parser bounds", () => {
it("reports connections discarded by the parser's hard ceiling", () => {
const connections = Array.from({ length: 40_001 }, (_, index) => ({
@@ -397,6 +481,93 @@ describe("Interface Builder hierarchy completeness", () => {
});
});
describe("Interface Builder hierarchy record limits", () => {
const analyze = (
hierarchy: unknown[],
maxObjects?: number,
objects: Record<string, unknown> = {},
) =>
buildInterfaceBuilderAnalysis({
targetSha256: hash,
toolVersion: "test",
documents: [
{
relativePath: "Main.nib",
archiveSha256: hash,
documentKind: "nib",
raw: {
"com.apple.ibtool.document.objects": objects,
"com.apple.ibtool.document.hierarchy": hierarchy,
},
},
],
limits: interfaceBuilderLimitsSchema.parse(
maxObjects === undefined ? {} : { max_objects: maxObjects },
),
});
const hierarchyFacet = (result: ReturnType<typeof analyze>) =>
result.graph.coverage.find(({ facet }) => facet === "hierarchy:Main.nib");
it("keeps an identified record after more than 20,000 structural records", () => {
const hierarchy: unknown[] = Array.from({ length: 20_001 }, () => ({
children: [],
}));
hierarchy[20_000] = { objectID: "tail", children: [] };
const result = analyze(hierarchy);
expect(
result.graph.nodes.map(
({ attributes }) => attributes.interface_builder_object_id,
),
).toContain("tail");
expect(result.graph.edges).toHaveLength(1);
expect(result.graph.truncated).toBe(false);
expect(hierarchyFacet(result)).toMatchObject({
status: "complete",
examined: 1,
omitted: 0,
});
});
it("counts identified records beyond the object limit as omitted", () => {
const result = analyze(
["a", "b", "c", "d"].map((objectID) => ({ objectID, children: [] })),
2,
);
expect(result.documents[0]?.hierarchy_complete).toBe(false);
expect(result.graph.truncated).toBe(true);
expect(hierarchyFacet(result)).toMatchObject({
status: "partial",
reason: "graph_limit_reached",
examined: 2,
omitted: 2,
});
});
it("stays complete when the identified records exactly reach the object limit", () => {
// The root and both objects fill the three graph nodes; the third
// hierarchy record reaches the hierarchy limit and has a structural child.
const result = analyze(
[
{ objectID: "a", children: [{ objectID: "b", children: [] }] },
{ objectID: "b", children: [{ children: [] }] },
],
3,
{ a: { class: "UIView" }, b: { class: "UIView" } },
);
expect(result.documents[0]?.hierarchy_complete).toBe(true);
expect(result.graph.truncated).toBe(false);
expect(hierarchyFacet(result)).toMatchObject({
status: "complete",
examined: 3,
omitted: 0,
});
});
});
describe("keyed archive object references", () => {
it("keeps referenced objects as stubs so shared and cyclic references stay finite", () => {
// Each view references its window, superview, and next key view, so
+9 -6
View File
@@ -211,6 +211,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
);
};
let hierarchyCount = 0;
let documentOmittedHierarchyNodes = 0;
const visitHierarchy = (
initial: unknown,
initialParentId: string,
@@ -218,11 +219,6 @@ export const buildInterfaceBuilderAnalysis = (input: {
const pending: { readonly value: unknown; readonly parentId: string }[] =
[{ value: initial, parentId: initialParentId }];
while (pending.length > 0) {
if (hierarchyCount >= input.limits.max_objects) {
truncated = true;
hierarchyTruncated = true;
return;
}
const entry = pending.pop();
if (entry === undefined) break;
const { value, parentId } = entry;
@@ -242,7 +238,13 @@ export const buildInterfaceBuilderAnalysis = (input: {
typeof item.archiveUID === "number" ? String(item.archiveUID) : null,
);
let nextParent = parentId;
if (objectId !== null) {
if (objectId !== null && hierarchyCount >= input.limits.max_objects) {
// The walk continues over the decoded records so the omission is
// counted rather than reported as an unknown remainder.
truncated = true;
hierarchyTruncated = true;
documentOmittedHierarchyNodes += 1;
} else if (objectId !== null) {
const known = objectIds.has(objectId);
const childId = known
? objectNodeId(objectId)
@@ -504,6 +506,7 @@ export const buildInterfaceBuilderAnalysis = (input: {
: null,
examined: hierarchyCount,
omitted:
documentOmittedHierarchyNodes +
documentOmittedHierarchyEdges +
parserOmittedHierarchyReferences +
omittedSerializedHierarchyReferences,
@@ -131,7 +131,7 @@ export const parseInterfaceBuilderRecords = (
return {
objects,
connections,
hierarchy: Array.isArray(hierarchyRaw) ? hierarchyRaw.slice(0, 20_000) : [],
hierarchy: Array.isArray(hierarchyRaw) ? hierarchyRaw : [],
classes: classesRaw,
objectCount: objectEntries.length,
omittedObjects: Math.max(0, objectEntries.length - 20_000),
@@ -327,11 +327,16 @@ const parseKeyedArchive = (
if (index === 0) continue;
const resolved = record(resolve(raw));
const className = firstString(resolved.className);
if (className === null) continue;
// An archived object names its class with its own `$class` reference;
// when that reference cannot be resolved the object is still observed,
// with an unknown class. Scalars and class descriptors are not objects.
if (className === null && !Object.hasOwn(record(raw), "$class")) continue;
objectCount += 1;
const fields = Object.fromEntries(
Object.entries(resolved).filter(([key]) => key !== "className"),
);
// Keep the unresolvable class reference itself, e.g. a dangling UID.
if (className === null) fields.$class = record(raw).$class;
const authoredId = firstString(fields.objectID, fields["object-id"]);
const objectId =
authoredId ??
@@ -357,6 +362,7 @@ const parseKeyedArchive = (
}),
);
if (
className !== null &&
/(?:Outlet|Connection|Segue|ActionConnection|ControlConnector)/iu.test(
className,
)
@@ -16,6 +16,19 @@ export const isTypeOnlyExportDeclaration = (
t.isTSTypeAliasDeclaration(node.declaration) ||
t.isTSDeclareFunction(node.declaration)));
/**
* Runtime external-module target for a value import-equals declaration;
* `import type X = require("m")` is erased and declares no dependency.
*/
export const semanticImportEqualsSpecifier = (
node: t.TSImportEqualsDeclaration,
): string | null =>
node.importKind !== "type" &&
t.isTSExternalModuleReference(node.moduleReference) &&
t.isStringLiteral(node.moduleReference.expression)
? node.moduleReference.expression.value
: null;
/** Whether this import binding is explicitly type-only. */
export const isTypeOnlyImportSpecifier = (node: t.ImportSpecifier): boolean =>
node.importKind === "type" || node.importKind === "typeof";
@@ -14,15 +14,21 @@ export const buildJavaScriptNativeHandoffs = (
edges: readonly ApplicationEdge[],
nativeEvidence: readonly Evidence[],
): NativeHandoff[] => {
const nodeById = new Map(nodes.map((node) => [node.node_id, node]));
let nodeById: Map<string, ApplicationNode> | undefined;
return nodes
.filter(({ kind }) => kind === "native-addon")
.flatMap((node) => {
const digest = artifactDigest(node);
if (digest === null) return [];
// Observations may describe only a containing archive. A native target
// requires the node owner's exact byte identity, independently of its
// path, availability labels or the artifact supporting an observation.
if (node.identity.strategy !== "content-digest") return [];
const digest = node.identity.sha256;
const linked = nativeEvidence.filter(
({ subject }) => subject?.digest.sha256 === digest,
);
nodeById ??= new Map(
nodes.map((candidate) => [candidate.node_id, candidate]),
);
const exportNodes = nativeExportNodes(node, edges, nodeById);
return [
{
@@ -54,16 +60,6 @@ export const buildJavaScriptNativeHandoffs = (
);
};
const artifactDigest = (node: ApplicationNode): string | null => {
if (node.identity.strategy === "content-digest") return node.identity.sha256;
const observation = node.observations.find(
({ evidence }) => evidence.artifact.available,
);
return observation?.evidence.artifact.available === true
? observation.evidence.artifact.sha256
: null;
};
const artifactPath = (node: ApplicationNode): string | null => {
for (const { evidence } of node.observations)
if (
@@ -10,7 +10,6 @@ import {
import {
collectSemanticModuleLink,
collectSemanticCallable,
semanticImportEqualsSpecifier,
collectSemanticReferences,
collectSemanticReferencesSteps,
immutableSemanticBindings,
@@ -47,7 +46,10 @@ import { semanticRequireOrigin } from "./javascriptSemanticRequireOrigin.js";
import { semanticBundledRequireOrigin } from "./javascriptBundledRequireOrigin.js";
import { collectSemanticMemberMutationsSteps } from "./javascriptSemanticMemberMutations.js";
import { indexJavaScriptAmbientWritesSteps } from "./javascriptSemanticAmbientWrites.js";
import { hasUncertainTypeOnlyModuleSideEffect } from "./javascriptModuleDeclaration.js";
import {
hasUncertainTypeOnlyModuleSideEffect,
semanticImportEqualsSpecifier,
} from "./javascriptModuleDeclaration.js";
import {
collectSemanticReturns,
resolveSemanticModuleCallables,
@@ -62,7 +64,6 @@ import {
} from "./javascriptElectronMemberWrites.js";
import { range } from "./javascriptStaticAnalysisHelpers.js";
import { semanticCoverage } from "./javascriptSemanticCoverage.js";
import { semanticResourceLimitsIn } from "./javascriptSemanticResourceLimits.js";
import {
parseJavaScriptSource,
type ParsedJavaScriptSource,
@@ -197,12 +198,7 @@ export function* analyzeParsedJavaScriptSemanticsSteps(
...derived,
coverage: semanticCoverage(
parserPartial,
semanticResourceLimitsIn([
...bindings.map(({ value }) => value),
...callables.flatMap((callable) =>
callable.returnSites.map(({ value }) => value),
),
]),
[...state.valueResourceLimits].sort(),
),
limitations: [
...(parserPartial
@@ -304,25 +300,21 @@ function* classifyDefinitionReceiversSteps(
if (client) roots.add(root);
};
const classifyOpenReceiver = (node: t.Node): void => {
if (
(!t.isCallExpression(node) && !t.isOptionalCallExpression(node)) ||
(!t.isMemberExpression(node.callee) &&
!t.isOptionalMemberExpression(node.callee))
)
if (!t.isCallExpression(node) && !t.isOptionalCallExpression(node)) return;
const callee = unwrapJavaScriptExpression(node.callee).node;
if (!t.isMemberExpression(callee) && !t.isOptionalMemberExpression(callee))
return;
const method = semanticStaticPropertyKey(
node.callee.property,
node.callee.computed,
);
const object = unwrapJavaScriptExpression(callee.object).node;
const method = semanticStaticPropertyKey(callee.property, callee.computed);
if (method === "get") {
if (isConfigurationReceiver(state, node.callee.object))
if (isConfigurationReceiver(state, object))
facts.set(node.start ?? -1, "configuration");
else if (isExtensionStateMember(node.callee.object))
else if (isExtensionStateMember(object))
extensionStateReads.push(node.start ?? -1);
return;
}
if (method !== "open" || !t.isIdentifier(node.callee.object)) return;
const receiver = node.callee.object.name;
if (method !== "open" || !t.isIdentifier(object)) return;
const receiver = object.name;
const binding = resolveSemanticBindingState(state, node, receiver);
if (binding === undefined) {
if (semanticResolutionBlocked(state, node, receiver)) return;
@@ -342,7 +334,9 @@ function* classifyDefinitionReceiversSteps(
return;
}
if (binding.mutable || binding.initializers.length !== 1) return;
const initializer = binding.initializers[0]?.node;
const initializerSource = binding.initializers[0]?.node;
if (initializerSource === undefined) return;
const initializer = unwrapJavaScriptExpression(initializerSource).node;
if (!t.isIdentifier(initializer)) return;
const initializerBinding = resolveSemanticBindingState(
state,
@@ -361,7 +355,9 @@ function* classifyDefinitionReceiversSteps(
initializerBinding.initializers.length !== 1
)
return;
const nested = initializerBinding.initializers[0]?.node;
const nestedSource = initializerBinding.initializers[0]?.node;
if (nestedSource === undefined) return;
const nested = unwrapJavaScriptExpression(nestedSource).node;
if (t.isIdentifier(nested)) {
const nestedFact = resolveGlobalAliasFact(
state,
@@ -424,16 +420,17 @@ const isConfigurationAccessorCall = (
state: JavaScriptSemanticAnalysisState,
node: t.Node,
): boolean => {
node = unwrapJavaScriptExpression(node).node;
if (!t.isCallExpression(node) && !t.isOptionalCallExpression(node))
return false;
const callee = node.callee;
const callee = unwrapJavaScriptExpression(node.callee).node;
if (
(!t.isMemberExpression(callee) && !t.isOptionalMemberExpression(callee)) ||
semanticStaticPropertyKey(callee.property, callee.computed) !==
"getConfiguration"
)
return false;
const host = callee.object;
const host = unwrapJavaScriptExpression(callee.object).node;
const matchesSpelling = t.isIdentifier(host)
? host.name === "workspace"
: (t.isMemberExpression(host) || t.isOptionalMemberExpression(host)) &&
@@ -442,7 +439,7 @@ const isConfigurationAccessorCall = (
const root = t.isIdentifier(host)
? host
: t.isMemberExpression(host) || t.isOptionalMemberExpression(host)
? host.object
? unwrapJavaScriptExpression(host.object).node
: undefined;
if (!t.isIdentifier(root)) return false;
const binding = resolveSemanticBindingState(state, root, root.name);
@@ -698,6 +695,7 @@ const createState = (
return {
program,
staticImportResolutionMode,
valueResourceLimits: new Set(),
parentsByNode: new WeakMap(),
scopes: [root],
scopesById: new Map([[root.scopeId, root]]),
@@ -219,7 +219,120 @@ describe("logical primitive unions", () => {
});
});
describe("semantic primitive resource observations", () => {
it.each([
{
name: "member",
source: (large: string) =>
`const answer = (${JSON.stringify(large)}).field;`,
},
{
name: "destructuring",
source: (large: string) =>
`const { nested: { answer } } = { nested: ${JSON.stringify(large)} };`,
},
])("preserves a limiting value through $name projection", ({ source }) => {
const ir = analyzeJavaScriptSemantics(source("x".repeat(200_000)));
expect(topLevelBinding(ir, "answer").value).toMatchObject({
status: "unknown",
resourceLimit: "primitive-bytes",
});
expect(ir.coverage).toMatchObject({
status: "partial",
resourceLimits: ["primitive-bytes"],
});
});
it("records an encountered bound without weakening a selected healthy sibling", () => {
const ir = analyzeJavaScriptSemantics(
`const answer = ({ ignored: ${JSON.stringify("x".repeat(200_000))}, kept: 7 }).kept;`,
);
expect(topLevelBinding(ir, "answer").value).toEqual({
status: "literal",
value: 7,
});
expect(ir.coverage).toMatchObject({
status: "partial",
resourceLimits: ["primitive-bytes"],
});
});
});
describe("configuration default normalization", () => {
it.each(["quoted", "template"])(
"normalizes a bare %s configuration default",
(syntax) => {
const large = "x".repeat(200_000);
const expression =
syntax === "quoted" ? JSON.stringify(large) : `\`${large}\``;
const ir = analyzeJavaScriptSemantics(
`process.env.MODE ?? ${expression};`,
);
const operation = ir.configurationOperations.find(
({ kind }) => kind === "default",
);
expect(
typeof operation?.value === "string" ? operation.value.length : 0,
).toBeLessThanOrEqual(174_762);
expect(operation).toMatchObject({
value: null,
resolution: "partial",
valueResourceLimit: "primitive-bytes",
});
expect(ir.coverage).toMatchObject({
status: "partial",
resourceLimits: ["primitive-bytes"],
});
},
);
it.each([null, false, 0, ""])(
"keeps an exact bounded configuration default %j",
(value) => {
const ir = analyzeJavaScriptSemantics(
`process.env.MODE ?? ${JSON.stringify(value)};`,
);
expect(
ir.configurationOperations.find(({ kind }) => kind === "default"),
).toMatchObject({ value, resolution: "complete" });
expect(ir.coverage.resourceLimits ?? []).toEqual([]);
},
);
});
describe("semantic primitive allocation and binding reuse", () => {
it.each(["quoted", "template"])(
"applies the normalized byte boundary to %s literals",
(syntax) => {
// Six estimated JSON bytes per UTF-16 code unit plus two quotes:
// 174762 code units fit; one more exceeds the 1 MiB bound.
for (const length of [174_762, 174_763]) {
const text = "x".repeat(length);
const expression =
syntax === "quoted" ? JSON.stringify(text) : `\`${text}\``;
const ir = analyzeJavaScriptSemantics(`const answer = ${expression};`);
const value = topLevelBinding(ir, "answer").value;
if (length === 174_762) {
expect(value.status).toBe("literal");
if (value.status !== "literal")
throw new Error("Expected exact value");
expect(typeof value.value === "string" && value.value.length).toBe(
length,
);
expect(ir.coverage.resourceLimits ?? []).toEqual([]);
} else {
expect(value.status).toBe("unknown");
expect(value).toMatchObject({ resourceLimit: "primitive-bytes" });
expect(ir.coverage).toMatchObject({
status: "partial",
omittedCount: null,
resourceLimits: ["primitive-bytes"],
});
}
}
},
);
it("returns an explicit unknown before expanding a large addition product", () => {
const term = '(choice ? "a" : "b")';
const expression = Array.from({ length: 20 }, () => term).join(" + ");
@@ -256,7 +369,7 @@ describe("semantic primitive allocation and binding reuse", () => {
expect(ir.coverage).toMatchObject({
status: "partial",
omittedCount: null,
resourceLimits: ["primitive-bytes"],
resourceLimits: expect.arrayContaining(["primitive-bytes"]),
});
});
@@ -915,15 +915,6 @@ export const dataEffectArgumentBindingId = (
? (resolveSemanticBindingState(state, node, node.name)?.bindingId ?? null)
: null;
/** Read one JSON primitive default without evaluating code. */
export const dataEffectPrimitive = (
node: t.Node,
): string | number | boolean | null => {
if (t.isStringLiteral(node) || t.isNumericLiteral(node)) return node.value;
if (t.isBooleanLiteral(node)) return node.value;
return null;
};
/**
* Read one static string or expression-free template. A template recovered
* with an invalid escape has no cooked value; its raw spelling is not a
@@ -8,11 +8,13 @@ import type {
} from "./javascriptSemanticIr.js";
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
import {
readExactJavaScriptLiteral,
semanticStaticPropertyKey,
semanticStaticPropertyName,
unwrapJavaScriptExpression,
} from "./javascriptAstValues.js";
import {
recordSemanticValueLimit,
resolveSemanticBindingState,
semanticResolutionBlocked,
type JavaScriptSemanticAnalysisState,
@@ -22,7 +24,6 @@ import {
dataEffectArgumentBindingId as argumentBindingId,
dataEffectLiteralString as literalString,
dataEffectMemberCallee as memberCallee,
dataEffectPrimitive as primitiveValue,
isSemanticProcessObject as isProcessObject,
outerDataEffectBinding as outerAssignedBinding,
assignedSemanticResultBindings,
@@ -31,6 +32,8 @@ import {
type DataEffectTraversalContext as TraversalContext,
} from "./javascriptSemanticDataEffectHelpers.js";
import { range } from "./javascriptStaticAnalysisHelpers.js";
import { semanticPrimitiveSet } from "./javascriptSemanticPrimitives.js";
import type { JavaScriptSemanticValue } from "./javascriptSemanticValueTypes.js";
interface DataEffectAnalysis {
readonly configurationOperations: readonly JavaScriptSemanticConfigurationOperation[];
@@ -175,6 +178,11 @@ const collectDefaults = (
const source = unwrapJavaScriptExpression(node.left).node;
const candidate = configurationsByNode.get(source);
if (candidate === undefined) return;
const literal = readExactJavaScriptLiteral(node.right);
const value: JavaScriptSemanticValue = literal.found
? semanticPrimitiveSet([literal.value])
: { status: "unknown", reason: "Default is not an exact primitive." };
recordSemanticValueLimit(context.state, value);
output.push({
configId: `config:default:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "default",
@@ -183,8 +191,11 @@ const collectDefaults = (
resultBindingId: outerAssignedBinding(node, context),
key: candidate.operation.key,
sourceConfigId: candidate.operation.configId,
value: primitiveValue(node.right),
resolution: primitiveValue(node.right) === null ? "partial" : "complete",
value: value.status === "literal" ? value.value : null,
...(value.status === "unknown" && value.resourceLimit !== undefined
? { valueResourceLimit: value.resourceLimit }
: {}),
resolution: value.status === "literal" ? "complete" : "partial",
});
});
return output;
@@ -1,5 +1,8 @@
import type { JavaScriptSourceRange } from "./javascriptStaticAnalysisTypes.js";
import type { JavaScriptSemanticValue } from "./javascriptSemanticValueTypes.js";
import type {
JavaScriptSemanticResourceLimit,
JavaScriptSemanticValue,
} from "./javascriptSemanticValueTypes.js";
import type {
JavaScriptSemanticCoverage,
JavaScriptSemanticReturnCoverage,
@@ -302,6 +305,7 @@ export interface JavaScriptSemanticConfigurationOperation {
readonly key: string | null;
readonly sourceConfigId: string | null;
readonly value: string | number | boolean | null;
readonly valueResourceLimit?: JavaScriptSemanticResourceLimit;
readonly resolution: "complete" | "partial";
}
Loaded 100 of 211 files, more files were not shown because too many files have changed in this diff. Show more