Compare commits

...
Author SHA1 Message Date
777c990786 fix(ci): configure diagnostic preload in the workflow step
GitHub forbids NODE_OPTIONS writes through GITHUB_ENV. Set the trusted probe
explicitly on the verification step instead. The probe records selected public
HTTP facts and child outcomes without arguments or ambient environment data.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 04:07:49 +00:00
df46735fcf chore(ci): observe Hopper installation failure boundaries
Temporary diagnostic branch: record only public Hopper request status/timing
and owned child lifecycle facts, without arguments, headers, or ambient env.
Run the existing clean installer with unchanged application code and preserve
its outcomes. This diagnostic workflow is not proposed for integration.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 04:02:16 +00:00
0022c9af5b chore(hopper): sync current main for final publisher validation
Integrate current main 4692bb333 while preserving its unrelated source exactly.
The complete static, documentation and focused runtime gates run against this
merged source before the next push.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 03:25:28 +00:00
66fca324d1 fix(hopper): serialize signed cache publication across processes
Reuse Hopper's live socket lease for each canonical release cache entry and
retain its cleanup capability alongside temporary inspection resources. Move
that shared lease owner into process so setup and provider adapters consume the
same implementation without reversing application dependencies.

Only proven malformed or unauthenticated cache evidence permits approved
replacement. Preserve old evidence and roll it back if publication fails; keep
unavailable inspection and permission failures distinct from invalid data.
Cover real cross-process publication/cancellation, rollback, failed lease
release, missing/malformed evidence and the moved provider consumers.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 03:20:40 +00:00
d80d03a913 fix(hopper): preserve cache repair recovery diagnostics
Retain both publication and replacement-verification failures, preserve pending
cleanup classification and expose its producer-owned resource paths. Use a file
URL for the deterministic doctor preload so workspace spaces remain valid.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 02:44:54 +00:00
80ca14f16d fix(hopper): recover approved caches and retain verification cleanup
Normalize empty XDG cache settings at their owner and preserve damaged signed
evidence as a backup when approved setup publishes an authenticated replacement.
Keep read-only admission strict and retain incomplete publisher cleanup through
client cancellation, restart admission and shutdown retry.

Verify offline evidence with isolated module state and observed PGP/native
inspection, remove vendor-network dependence from the foreign-launcher doctor
test, and bound small feed/key requests separately from package downloads.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 02:38:36 +00:00
Kaoru0822andClaude Opus 5.5 4692bb3330 fix(reference): stop attributing overwritten require members to Node (#3801)
* fix(reference): stop attributing overwritten require members to Node

import-reference-source treated require.resolve(...) and require.main(...)
as Node's resolver even after the file replaced that member, so a literal
argument became a parsed internal dependency that the source never loads.

Attribute member calls only while the loader's resolve and main members
keep Node's identity: no write to either member or a computed member
(assignment, update, delete, destructuring) and no use of require as a
value where a mutator or alias could replace them. Bare require(...) calls
are unaffected.

Fixes #3393

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(reference): keep optional and wrapped loader uses intact

An optional call require?.(...) or a TypeScript or parenthesis wrapper
around require was read as require escaping as a value, which disabled
every member-call link in the file. Look through inert wrappers and
accept optional calls before deciding the loader escaped, and add
controls for bare calls after a member overwrite, a local loader's
member write, and optional calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:23:06 -07:00
Kaoru0822andClaude Opus 5.5 af6a998066 fix(javascript): keep string Worker paths unresolved without a document base (#3797)
A plain new Worker("./w.js") or navigator.serviceWorker.register("./sw.js")
resolves against the document base URL in a browser, and a Node
worker_threads string resolves against the working directory. Static
analysis was resolving browser strings against the calling script, which
recorded a resolved mapping to the wrong file.

Only new Worker(new URL(path, import.meta.url)) now produces a specifier.
Other worker and service-worker strings keep their expression and stay
unresolved, matching the existing worker_threads string handling.

Fixes #3309

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:13:17 -07:00
Kaoru0822andClaude Opus 5.5 7a66470385 fix(apple): report plist dates the decoder cannot represent as unknown (#3793)
plutil writes years past 9999 and negative years in a form the pinned
XML decoder cannot parse, and binary plists can hold dates outside the
JavaScript Date range. The decoder returns an Invalid Date. Its
projection's toISOString() call then threw, so inspect_plist failed the
whole plist as "unreadable_output" with a `rea doctor` remediation.
Keyed archive inspection and Interface Builder decoding rejected the
archive in the same way.

projectPlistValue, the shared projection, now reports such a date as
{ "$plist_type": "date", "iso8601": null } and counts it. inspect_plist,
keyed archive inspection and Interface Builder decoding each add a
counted limitation, as they already do for non-finite reals.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:13:01 -07:00
Kaoru0822andClaude Opus 5.5 90c6922905 fix(javascript): keep empty URL path segments significant (#3792)
* fix(javascript): keep empty URL path segments significant

ESM imports and HTML script references were joined with POSIX path
rules, which collapse '//' before applying '..'. URL resolution removes
the empty segment instead, so './assets//../app.js' names assets/app.js,
not app.js, and the graph linked the importer and renderer to the wrong
file. URL references refused every '//' even when dot segments cancel it.

Resolve ESM, HTML (including document base) and URL references through
one URL dot-segment helper. A remaining empty segment stays unavailable,
references above the artifact root are still rejected, and CommonJS keeps
filesystem path semantics.

Fixes #3307

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(javascript): resolve HTML base and reference as one URL path

The base href was turned into a canonical path before the script
reference was applied. A base empty segment that the reference's '..'
would remove was rejected early, and an above-root '..' from the base
became a removable segment that the reference's '..' could cancel.

Keep one URL path state through base and reference resolution, with
above-root escapes held apart from removable segments, and reject empty
segments only in the final path. Remove the superseded htmlBaseDirectory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:12:49 -07:00
Kaoru0822andClaude Opus 5.5 3ded799bc7 fix(reference): never resolve a directory to an extensionless index (#3789)
resolveInternalSpecifier offered posix.join(dir, "index") as the
second candidate, ahead of every extension. With both dep.js and an
extensionless dep/index present, require("./dep") therefore resolved to
dep/index, a file that neither Node nor TypeScript ever selects.

Directory candidates now carry an extension. File and index ordering is
otherwise unchanged, so Node-validated cases such as dep/index.js
winning over dep.cjs keep their meaning.

Fixes #3381

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:12:36 -07:00
41189f88c4 chore(hopper): preserve unrelated main tests during integration
Restore the exact upstream MCP tests after the merge hook's lint autofixes
rewrote their expressions. Keep this PR scoped to Hopper and preserve the
upstream formatting rather than introducing unrelated cleanup.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 02:11:27 +00:00
4572486f4c chore(hopper): integrate latest main before review
Preserve main's precise setup failure type and macOS installation cleanup
while retaining Linux publisher diagnostics and selected-platform handling.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 02:06:59 +00:00
morluto d69acc3455 fix(windows): retain runtime ownership and clear readonly attributes 2026-10-15 09:29:04 +08:00
Ysopwnable 7d1e4d0e2b fix(browser): release attached emulation overrides (#3778)
* fix(browser): release attached emulation overrides

* fix(browser): preserve unowned environment state

* fix(browser): coordinate attached environment cleanup
2026-10-15 09:21:56 +08:00
Kaoru0822andClaude Opus 5.5 949228b606 perf(javascript): check wrapper IPC duplicates by invocation set (#3783)
The endpoint index from #3779 still scanned every earlier finding at the
same endpoint and serialized each invocation location. One wrapper called
N times for one channel shares an endpoint, so duplicate checks stayed
quadratic in N.

Record per endpoint whether a directly observed send/invoke exists and the
set of serialized invocation locations. The duplicate rule is unchanged and
each check is constant time.

Refs #3670

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:20:34 -07:00
Kaoru0822andClaude Opus 5.5 5e8f7c536e fix(cli): stop quietly when the stdout reader closes early (#3781)
* fix(cli): stop quietly when the stdout reader closes early

`rea ... | head` crashed once head closed the pipe. TOON and YAML output
raised an unhandled EPIPE 'error' event with a Node stack trace. JSON
output reported {"code":"UNKNOWN","message":"write EPIPE"} on stderr.
Both exited 1, as if the analysis had failed.

The CLI entry point owns process.stdout, so it now treats EPIPE as the
reader declining the rest of the output. It stops writing, suppresses
the failure report and exits 141 like a SIGPIPE-terminated writer, also
when Incur exits through its own failure path. Other stdout errors are
handled as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): guard mcp doctor output and cover midstream reader closure

The closed-reader guard was only installed for ordinary CLI commands. As a
result `rea mcp doctor | head` still crashed on EPIPE. The guard now lives
in one entry-point function shared by both CLI branches; MCP stdio keeps
its own lifecycle.

The tests now also close the reader after the first chunk of a 4 MiB
streamed JSON result, so later writes in writeJsonOutput are covered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:20:19 -07:00
morluto 18591f78ae Merge remote-tracking branch 'origin/main' into codex/bug-sweep-20261015-final 2026-10-15 09:14:14 +08:00
Kaoru0822andClaude Opus 5.5 44b197f924 fix(javascript): parse JavaScript sources with JavaScript grammar (#3782)
parserPluginsForPath enabled Babel's typescript plugin for .js, .jsx,
.mjs and .cjs sources. TypeScript reads some valid JavaScript
differently: a conditional whose consequent is an arrow with a
parenthesized body (c?x=>(a):y=>(b)) throws as a return-type
annotation, and a<b>(c) becomes a generic call. VS Code's 20 MB
workbench bundles therefore returned no AST at all.

Select JavaScript grammar (JSX plus legacy decorators) for explicit
JavaScript extensions. TypeScript dialects and unnamed sources keep the
existing TypeScript-capable contract.

Fixes #3780
Refs #3670

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:12:30 -07:00
morluto 610f873e91 Merge remote-tracking branch 'origin/main' into codex/bug-sweep-20261015-final 2026-10-15 09:01:38 +08:00
Kaoru0822andClaude Opus 5.5 59f528f0e5 perf(javascript): share one definitions pass and index wrapper IPC dedupe (#3779)
Static analysis rebuilt the full lexical definitions index three times per
file (open receivers, RPC client roots, binding lookup) and the Electron
wrapper pass checked each substituted IPC registration against every prior
finding with JSON.stringify, which is quadratic in registrations.

Classify open receivers and RPC client roots in one traversal over a single
definitions-only index, then extend that same state with ambient writes for
binding lookup. Index IPC findings by side, operation, channel and location
so duplicate checks only consult same-endpoint candidates.

Static facts are byte-identical on the measured corpora.

Refs #3670

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:59:44 -07:00
morluto c553d1138e fix(windows): clear readonly when no other file attributes remain 2026-10-15 08:49:11 +08:00
morluto 930dd5357f Merge remote-tracking branch 'origin/main' into codex/bug-sweep-20261015-final 2026-10-15 08:44:08 +08:00
Kaoru0822andClaude Opus 5.5 084e540414 fix(native): resolve inspect_plist defaults from the selected target's bundle (#3774)
inspect_plist rooted every file target at the file's own directory, so a
program file under App.app/Contents/MacOS defaulted to
Contents/MacOS/Contents/Info.plist and --relative-path resolved from MacOS/
instead of the app root the CLI documents. A selected plist file likewise
looked for <dir>/Contents/Info.plist instead of reading itself.

A program file directly under a bundle's Contents/MacOS now roots at that
bundle, and a plist artifact target is its own default plist. The missing
default message names the default plist rather than Contents/Info.plist.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:40:24 -07:00
morluto db689f3c3c fix(windows): relax owned parent sharing before admission validation 2026-10-15 08:35:11 +08:00
morluto f15ddbea7a Merge remote-tracking branch 'origin/main' into codex/bug-sweep-20261015-final 2026-10-15 08:31:24 +08:00
morluto 68081ac414 fix(windows): grant retained roots the rights required for cleanup 2026-10-15 08:31:22 +08:00
Kaoru0822andClaude Opus 5.5 bddeae5108 fix(managed): charge admitted member facts once in the final projection (#3487)
* fix(managed): charge admitted member facts once in the final projection

inspect_managed_members admits method bodies, signatures, call edges,
field accesses and user strings as they are produced, then admits the
whole result again before returning it. The final pass re-counted every
already-admitted record, so the 64 MiB representation budget was spent
at roughly twice the result's JSON size. Ordinary assemblies such as
MathNet.Numerics, MongoDB.Driver, Npgsql, ClosedXML, System.Reactive and
Azure.Storage.Blobs returned resource-limit with no members.

The budget now remembers each admitted projection. A containing
projection spends that charge once instead of counting the subtree
again; a value that serializes more than once still pays for every
further occurrence. Results that already fit are byte-identical.

Refs #3449

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(managed): check a repeated occurrence pays with fresh credit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:19:33 -07:00
morluto f78d137b42 fix(windows): retain unpublished runtimes after cleanup failure 2026-10-15 08:16:22 +08:00
morluto 1a3a96d36b fix(windows): honor directory readonly semantics during cleanup 2026-10-15 08:16:22 +08:00
Kaoru0822andClaude Opus 5.5 c9764084b8 fix(reconstruction): withhold readiness for manifest references to absent Evidence (#3526)
build_reconstruction_obligation_ledger accepted a manifest original case or
fixture whose Evidence ID was absent from the bundle whenever another entry
of the same case kind was valid. The obligation was reported verified, the
ledger ready, and the dangling ID appeared in evidence_links with no
diagnostic, although the documented contract requires every referenced
Evidence ID to exist in the bundle.

Report each absent original-case or fixture reference as missing-original-case
or missing-case, naming the ID. Any such diagnostic withholds verification and
readiness.

Fixes #3363

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:11:40 -07:00
Kaoru0822andClaude Opus 5.5 0f966e2b44 fix(reconstruction): merge repeated observations into one generated obligation (#3524)
Two capture-process records of the same executable and scenario generate
the same obligation ID. addCandidate added their definition counts, so the
ledger reported duplicate-definition, a closure blocker meant for competing
definitions, merely because the claim was observed twice. The merge also
kept only the first record's unavailable_authority and
required_next_evidence.

Generated IDs digest their derivation key, so merging two generated
candidates keeps one definition. A reviewed definition that reuses an ID is
still a duplicate. The merge now unions unavailable_authority and
required_next_evidence from every observation.

Fixes #3375

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:11:28 -07:00
Kaoru0822andClaude Opus 5.5 1851f1fcfd fix(managed): report ILONLY-clear assemblies as mixed in boundary summaries (#3516)
inspect_managed_native_boundaries derived summary.mixed_mode_or_native_header
only from the managed native header and the native entry-point flag. A C++/CLI
assembly with ILONLY clear and no ReadyToRun signature was classified as
mixed-clr-native by inspect_managed_artifact but summarized as not mixed.

Give the rule one owner, managedCliIsMixedClrNative in ManagedPeReader, and
use it for both the artifact runtime family and the boundary summary. The
summary now reads the CLI header directly instead of the cli_native projection.

Fixes #3366

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:11:12 -07:00
Kaoru0822andClaude Opus 5.5 2b2a5d0f26 fix(native): inspect arm64e.x1 slices by their exact name (#3500)
inspect_macho passed the CPU-family spelling (`arm64e`) for every dotted lipo
slice name to otool, nm, dwarfdump and vtool. That is right for the
pointer-auth ABI version suffix (`arm64e.v1`), but `arm64e.x1` is a distinct
subtype: on a universal x86_64 + arm64e.x1 executable otool failed with
"does not contain architecture: arm64e" and the whole inspection failed.

Strip only the `.v<N>` ABI version suffix and pass other variant names
exactly. vtool has no spelling for a variant subtype, so skip it for such
slices and report that build metadata comes from otool only.

Fixes #3389

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:10:58 -07:00
Kaoru0822andClaude Opus 5.5 2a9252e4e6 fix(managed): report metadata range failures at artifact file offsets (#3508)
ManagedMetadataLayout's range check reported offsets relative to the buffer
it was given: the metadata root for version and stream headers, and the
table stream for the tables header, row counts and table data. Other
diagnostics in the same reader already use artifact file offsets, so a
truncated table stream at file offset 888 was reported at offset 24, inside
the DOS header.

Pass each buffer's artifact file offset to the range check so every
invalid-metadata-root issue carries a file offset.

Fixes #3364

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:06:06 -07:00
Kaoru0822andClaude Opus 5.5 7666a482b6 fix(adb)!: keep the exact Android long version code (#3485)
inspect_adb_package converted the dumpsys versionCode, a signed 64-bit long
(versionCodeMajor << 32 | versionCode), to a JavaScript number. Values above
2^53 were rounded, so 9007199254740993 was reported as 9007199254740992 with
the same Evidence ID, and negative values became null.

Return version_code as the exact decimal string dumpsys prints, matching the
apktool and JADX version_code contracts.

BREAKING CHANGE: inspect_adb_package version_code is now a decimal string
("45") instead of a JSON number (45). Read it as a string, or parse it with
BigInt when numeric comparison is needed.

Fixes #3368

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:01:25 -07:00
morluto fa318995f5 fix(setup): include registered IDA budgets in client deadlines 2026-10-15 07:36:34 +08:00
morluto 9a44fea347 test(mcp): avoid unrelated validator compilation in availability checks 2026-10-15 07:12:53 +08:00
morluto 2819defeea test(cli): verify attributed caveats in composed analysis 2026-10-15 06:56:09 +08:00
morluto 937b4e69c8 fix(cache): invalidate snapshots that lack source caveat semantics 2026-10-15 06:45:53 +08:00
morluto 1b668a527d chore: integrate current main before bug qualification 2026-10-15 06:42:51 +08:00
morluto 1e377e6965 perf(evidence): group repeated caveats with their source operations 2026-10-15 06:39:48 +08:00
morluto fb6e2b07e0 fix(ida): bound readiness requests by the remaining deadline 2026-10-15 06:34:32 +08:00
morluto c4608a2962 fix(evidence): preserve source attribution across composed workflows 2026-10-15 06:33:48 +08:00
morluto 225f26a5f8 refactor(ghidra): give function boundary scans one owner 2026-10-15 06:25:02 +08:00
morluto a004061abc fix(native): preserve provider limitations through call traces 2026-10-15 06:20:20 +08:00
morluto 01a1b3fda6 fix(ghidra): retain function boundary limitations on references 2026-10-15 06:20:20 +08:00
Ysopwnable 3d2846f237 fix(ghidra): invalidate synthetic entry reference snapshots (#3728)
fix(ghidra): exclude synthetic entry references
2026-10-15 06:14:29 +08:00
morluto 709e99d781 fix(windows): admit volume roots with existing write handles 2026-10-15 06:12:23 +08:00
morluto bb49252523 test(ida): verify typed timeout and retained cleanup through the CLI 2026-10-15 05:38:36 +08:00
morluto 4946688c33 refactor(setup): distinguish provider configuration from registration settings 2026-10-15 05:23:04 +08:00
morluto 333e0ab0df fix(setup): propagate selected provider deadlines through CLI registrations 2026-10-15 05:16:08 +08:00
morluto c0cfccb45f fix(ida): preserve SDK request timeout identity and captured output 2026-10-15 05:14:57 +08:00
morluto fbdd040324 docs(mcp): explain absolute SDK deadlines and client timeout configuration 2026-10-15 05:05:30 +08:00
morluto dadf909dce fix(setup): derive MCP tool deadlines and preserve caller timeout choices 2026-10-15 05:05:24 +08:00
morluto 097a4bce6f fix(jeb): make upstream request deadlines configurable and preserve cancellation 2026-10-15 05:05:18 +08:00
morluto c831c23633 docs(mcp): clarify gateway schema recovery without inferring backend constraints 2026-10-15 04:49:27 +08:00
morluto 5a46fbacd5 fix(mcp): advertise explicit target cleanup when investigations finish 2026-10-15 04:49:27 +08:00
morluto fffb074193 chore: integrate current main verification fixes 2026-10-15 04:06:41 +08:00
morluto 09e7f07108 fix(setup): select compact input schemas for Kimi registrations 2026-10-15 04:06:41 +08:00
morluto 1bbf4bd16c fix(process): preserve caller overrides during Windows environment normalization 2026-10-15 04:06:41 +08:00
morluto 07a501e82c fix(mcp): share inspection admission for prompt completion 2026-10-15 04:06:41 +08:00
morluto ebecf06f6e fix(setup): integrate Windows runner and ConPTY corrections 2026-10-15 04:04:17 +08:00
morluto 7e0c7b1244 fix(cutter): resolve bridge paths from the selected caller home 2026-10-15 04:01:36 +08:00
Ysopwnable d9cd239c5e fix(test): honor Ghidra startup budgets in Windows verification (#3417)
* fix(test): honor Ghidra startup budgets in file-offset verification

* fix(test): preserve selected startup budgets in direct Ghidra verification

* test(ghidra): cover verifier timeout budgets

* fix(test): preserve Ghidra MCP request budgets
2026-10-15 03:50:50 +08:00
morluto 11f64f653e test(browser): verify typed worker refusals across navigations 2026-10-15 03:43:29 +08:00
morluto 9b7ec809e6 test(process): use owned host-aware settlement fixtures 2026-10-15 03:43:29 +08:00
morluto 170bfc7b3f fix(mcp): authenticate inventory facts before prompt completion 2026-10-15 03:43:29 +08:00
morluto 591c87c4b6 refactor(artifacts): share inspection validation and prune legacy fixtures 2026-10-15 03:43:29 +08:00
morluto bfd76c5d16 test: isolate host discovery and update current capture fixtures 2026-10-15 03:43:29 +08:00
morluto 9ac91e0c1e fix(artifacts): bind inspection provenance to every retained inventory 2026-10-15 03:43:29 +08:00
morluto 30a9139b87 fix(javascript): keep source-module semantics independent of sibling bytes 2026-10-15 03:43:29 +08:00
morluto a9588be4eb fix(evidence)!: reject missing recorded inventory and capture facts
BREAKING CHANGE: Inventory Evidence must explicitly report integrity_contradictions, and browser capture Evidence must report every resolved budget. Caller input defaults remain supported; stored observations are never filled with inferred defaults.
2026-10-15 03:43:29 +08:00
morluto fc789d19ff style(apple): normalize class parser module formatting 2026-10-15 03:43:29 +08:00
morluto 4ff2dadf66 refactor(apple): remove unused class identity re-export 2026-10-15 03:43:29 +08:00
morluto 1a06c82772 style(test): format coverage regression fixtures 2026-10-15 03:43:29 +08:00
morluto 7d885d632c docs: align workflow recovery and current Evidence contracts 2026-10-15 03:43:29 +08:00
morluto 57a23f0a2d fix(browser)!: expose capture budgets and queue admitted content reads
BREAKING CHANGE: Browser capture results require resolved limits, event families, and omission facts. Caller-selected budgets replace hidden retention ceilings; admitted content reads queue behind bounded concurrency.
2026-10-15 03:43:29 +08:00
morluto d2a53beb3d test(hopper): verify external exit recovery and retained ownership 2026-10-15 03:43:29 +08:00
morluto a8887a9a85 fix(jeb): probe live operations with owned discovery deadlines 2026-10-15 03:43:29 +08:00
morluto 45d5e80144 fix(cli): preserve Cutter choices and reject invalid runtime arguments 2026-10-15 03:43:29 +08:00
morluto 50dd9fcad1 fix(ghidra): preserve MIPS overrides and filter entry metadata 2026-10-15 03:43:29 +08:00
morluto 6add17a317 fix(windows): explain missing native artifacts and verify output drain 2026-10-15 03:43:29 +08:00
morluto 8c0e431895 fix(setup): own Hopper replacement staging and committed outcomes 2026-10-15 03:43:29 +08:00
morluto ceb77f6697 fix(apple): distinguish JVM resources and UIKit event connections 2026-10-15 03:43:29 +08:00
morluto fc7e41ff3b fix(process): attribute finite trace omissions and cleanup states 2026-10-15 03:43:29 +08:00
morluto cb4ff50202 fix(functions): preserve coverage diagnostics separately from blockers 2026-10-15 03:43:28 +08:00
morluto b47055edbe fix(javascript): preserve module identity and boundary fields 2026-10-15 03:43:28 +08:00
morluto 0405c9d361 fix(artifacts)!: authenticate complete inspection and summary inputs
BREAKING CHANGE: Artifact inventories require scoped coverage. Obsolete coverage_status and prose-based coverage inference are removed. Inspection carriers authenticate every retained inventory substep.
2026-10-15 03:43:28 +08:00
morluto db8128dc0d refactor(evidence)!: require current coverage commitments
BREAKING CHANGE: Evidence and analysis snapshots require explicit nullable relationship coverage and include it in their identity commitments. Historical records without the field are rejected; saved files are not migrated.
2026-10-15 03:43:28 +08:00
Swatantra Yadav 2abfe53dbd fix(setup): preserve overridden runner variables and test isolation on Windows
Support case-insensitive environment value lookup in ClientRegistrationIdentity while preserving JavaScript property order precedence with findLast(). Add unit regression coverage for ambient uppercase npm variables on Windows. Isolate USERPROFILE in cliSetup tests so child processes do not touch the host profile directory. Harden JSON decision extraction in runJourney against ConPTY trailing ANSI escape codes. Deterministically sequence multiselect prompt navigation before confirmation. Filter ambient runner variables case-insensitively before launching interactive setup completion tests.
2026-10-15 00:56:31 +05:30
qorvrandqorvr efbc486b9b fix(contracts): reject NUL bytes in keyed archive and extraction paths (#3606)
Co-authored-by: qorvr <330186599+qorvr@users.noreply.github.com>
2026-10-15 03:14:31 +08:00
11b747f29d fix(analysis-view): unify inert inline Evidence admission
Share inline Evidence admission before authentication across analysis views.
Reject executable object behavior before reflection, inspect arrays
incrementally, preserve inert JSON-omitted metadata and existing budgets.
Keep precise error locations with one bounded path stack.

Validated base failures, 48 source tests, 71 related public-boundary tests,
27 final CLI/MCP tests and eleven calls against a genuine Chromium HAR;
source identity, selected output and capacity accounting remain preserved.

Co-authored-by: Zhong-fan <205188959+Zhong-fan@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 03:13:28 +08:00
b0de9a5c72 fix(verify): request JSON for modified Hopper rejection
The real Linux mutation probe parses the structured rejection, so request
JSON explicitly for its Hopper analysis call.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-14 18:12:17 +00:00
fd6f996733 fix(hopper): preserve publisher rejection before application execution
Carry the publisher owner's parsed reason through the launch error and public
CLI/MCP projection without inventing a helper exit. Share exact unsupported
launch checks between package and real-Linux verification, and keep foreign
fixture rejection independent of vendor network availability.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-14 18:02:12 +00:00
0ac43ad604 fix(hopper): retain launcher digests in verification diagnostics
Expose the verified launcher digest in doctor and preserve both digest facts
when signed package bytes disagree with release metadata.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-14 17:31:45 +00:00
23a7131143 fix(hopper)!: verify current Linux releases with the publisher key
Resolve public demo packages from Hopper's release API, authenticate their
complete bytes and launcher digests, and retain signed evidence for offline
verification. Share publisher facts across setup, doctor and the Python launch
guard, with bounded process-local reuse and preserved cleanup ownership.

Fixes #3577

BREAKING CHANGE: Older Linux Hopper builds admitted by the former fixed
allowlist now require retained publisher-signed package evidence or an
approved update to the current public demo. Update explicit
HOPPER_LAUNCHER_PATH values when migrating; the live feed exposes only
its current release.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-14 17:19:18 +00:00
330 changed files with 15383 additions and 2588 deletions

No files matched your search

@@ -0,0 +1,33 @@
import { appendFileSync } from 'node:fs';
import { basename } from 'node:path';
import { ChildProcess } from 'node:child_process';
const log = facts => {
try { appendFileSync(process.env.REA_HOPPER_PROBE_LOG, JSON.stringify({at:Date.now(),pid:process.pid,...facts})+'\n'); }
catch { /* Diagnostic failure must not change installer behavior. */ }
};
log({event:'runtime',uid:process.getuid?.(),version:process.version,execPath:process.execPath});
const originalSpawn = ChildProcess.prototype.spawn;
ChildProcess.prototype.spawn = function(options) {
const started=performance.now();
const file=typeof options?.file==='string'?basename(options.file):'unknown';
this.once('exit',(code,signal)=>log({event:'child_exit',childPid:this.pid,file,code,signal,elapsedMs:performance.now()-started}));
this.once('error',error=>log({event:'child_error',file,name:error?.name,code:error?.code,elapsedMs:performance.now()-started}));
const result=originalSpawn.call(this,options);
log({event:'child_spawn',childPid:this.pid,file});
return result;
};
const originalFetch=globalThis.fetch;
globalThis.fetch=async (input,options)=>{
const url=typeof input==='string'?input:input instanceof URL?input.href:input.url;
if(new URL(url).hostname!=='www.hopperapp.com')return originalFetch(input,options);
const started=performance.now();
log({event:'fetch_start',url});
try {
const response=await originalFetch(input,options);
log({event:'fetch_headers',url,status:response.status,contentLength:response.headers.get('content-length'),contentType:response.headers.get('content-type'),retryAfter:response.headers.get('retry-after'),elapsedMs:performance.now()-started});
return response;
} catch(error) {
log({event:'fetch_error',url,name:error?.name,code:error?.code,causeName:error?.cause?.name,causeCode:error?.cause?.code,elapsedMs:performance.now()-started});
throw error;
}
};
+1
View File
@@ -741,6 +741,7 @@ jobs:
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
tests/boundary/cli/referenceSourceImportStatus.test.ts
+16 -27
View File
@@ -1,4 +1,4 @@
name: Real Hopper Linux E2E
name: Hopper Linux installation diagnostic
on:
workflow_dispatch:
@@ -30,31 +30,20 @@ jobs:
prefix="${RUNNER_TEMP}/rea-prefix"
npm install --global --prefix "${prefix}" "./${tarball}"
echo "REA_VERIFY_SERVER_COMMAND=${prefix}/bin/rea" >> "${GITHUB_ENV}"
- name: Verify clean Linux setup through CLI and MCP
run: sudo --preserve-env=REA_VERIFY_SERVER_COMMAND,PATH npm run verify:setup:linux
verify:
name: Installed package against real Linux Hopper
runs-on: [self-hosted, Linux, hopper]
timeout-minutes: 20
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Install packed CLI in isolation
shell: bash
- name: Record public HTTP and child lifecycle facts
run: |
tarball="$(npm pack --silent)"
prefix="${RUNNER_TEMP}/rea-prefix"
npm install --global --ignore-scripts --prefix "${prefix}" "./${tarball}"
echo "REA_VERIFY_SERVER_COMMAND=${prefix}/bin/rea" >> "${GITHUB_ENV}"
echo 'REA_VERIFY_SERVER_ARGS_JSON=["mcp"]' >> "${GITHUB_ENV}"
- name: Verify source-owned ELF fixtures through an alternate launcher symlink
node --input-type=module <<'JS'
import { appendFileSync, chmodSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
const log = join(process.env.RUNNER_TEMP, 'rea-hopper-installer.jsonl');
writeFileSync(log, '', {mode: 0o666});
chmodSync(log, 0o666);
appendFileSync(process.env.GITHUB_ENV, `REA_HOPPER_PROBE_LOG=${log}\n`);
JS
- name: Verify clean Linux setup through CLI and MCP
env:
REA_HOPPER_CONFORMANCE_MANIFEST_PATH: ${{ github.workspace }}/build/conformance/manifest.json
run: npm run verify:hopper:linux
NODE_OPTIONS: --import=file://${{ github.workspace }}/.github/scripts/trace-hopper-installer.mjs
run: sudo --preserve-env=REA_VERIFY_SERVER_COMMAND,PATH,REA_HOPPER_PROBE_LOG env NODE_OPTIONS="$NODE_OPTIONS" npm run verify:setup:linux
- name: Installer boundary observations
if: always()
run: cat "$REA_HOPPER_PROBE_LOG"
+80 -66
View File
@@ -934,7 +934,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
String value = decompiledText(decompile(function));
JsonObject result = functionObservation(
value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value),
noReturnCallLimitations(
functionBoundaryLimitations(
function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
@@ -1027,7 +1027,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
return functionObservation(
GSON.toJsonTree(renderAssembly(scan.instructions)),
noReturnCallLimitations(function, scan.instructions)
functionBoundaryLimitations(function, scan.instructions)
);
}
@@ -1046,7 +1046,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add(
"The fast path does not invoke the decompiler or scan whole-program names and strings."
);
List<String> boundaryLimitations = noReturnCallLimitations(function, scan.instructions);
List<String> boundaryLimitations = functionBoundaryLimitations(function, scan.instructions);
for (String limitation : boundaryLimitations) limitations.add(limitation);
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
@@ -1069,7 +1069,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add(canonicalAddress(item.getEntryPoint()));
}
if (callers) {
JsonObject observation = functionObservation(result, noReturnCallLimitations(function,
JsonObject observation = functionObservation(result, functionBoundaryLimitations(function,
currentProgram.getListing().getInstructions(function.getBody(), true)
));
observation.add("relationship_coverage", callRelationshipCoverage(null,
@@ -1099,7 +1099,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add("classification", functionClassification(function));
return functionObservation(
result,
noReturnCallLimitations(
functionBoundaryLimitations(
function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
@@ -1131,12 +1131,22 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.addProperty("direction", direction);
result.add("references", edges);
result.addProperty("reference_kinds_available", true);
JsonArray unresolvedCalls = direction.equals("outgoing")
? callBoundaryObservation(
List<String> limitations;
JsonArray unresolvedCalls = new JsonArray();
if (direction.equals("outgoing")) {
CallBoundaryObservation calls = callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
).unresolvedCalls() : new JsonArray();
);
unresolvedCalls = calls.unresolvedCalls();
limitations = calls.limitations();
} else {
limitations = functionBoundaryLimitations(
function,
currentProgram.getListing().getInstructions(function.getBody(), true)
);
}
result.add("unresolved_calls", unresolvedCalls);
return result;
return functionObservation(result, limitations);
}
private JsonObject inspectNativeDataType(JsonObject params) throws Exception {
@@ -1337,11 +1347,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add("references", references);
if (includeIncoming) {
TreeMap<String, Reference> incoming = new TreeMap<>();
ReferenceIterator incomingIterator = currentProgram.getReferenceManager().getReferencesTo(address);
while (incomingIterator.hasNext()) {
monitor.checkCancelled();
addReference(incoming, incomingIterator.next());
}
visitObservedReferencesTo(address, reference -> addReference(incoming, reference));
List<Reference> orderedIncoming = new ArrayList<>(incoming.values());
orderedIncoming.sort(REFERENCE_ORDER);
JsonArray incomingReferences = new JsonArray();
@@ -1393,16 +1399,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
requireKeys(params, Set.of("document", "address"));
requireDocument(params);
Address address = requireAddress(params, "address");
ReferenceIterator references = currentProgram.getReferenceManager().getReferencesTo(address);
Set<Address> sources = new HashSet<>();
while (references.hasNext()) {
monitor.checkCancelled();
Reference reference = references.next();
if (reference.isEntryPointReference()) {
continue;
}
sources.add(reference.getFromAddress());
}
visitObservedReferencesTo(address, reference -> sources.add(reference.getFromAddress()));
List<Address> ordered = new ArrayList<>(sources);
ordered.sort(Address::compareTo);
JsonArray result = new JsonArray();
@@ -3430,51 +3428,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
return omitted[0];
}
private List<String> noReturnCallLimitations(
private List<String> functionBoundaryLimitations(
Function function, Iterable<Instruction> instructions)
throws Exception {
List<String> limitations = new ArrayList<>();
Instruction beforePrevious = null;
Instruction previous = null;
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (beforePrevious != null && previous != null) {
String seh = borlandSehHandlerLimitation(
function, beforePrevious, previous, instruction);
if (seh != null) limitations.add(seh);
}
beforePrevious = previous;
previous = instruction;
}
return limitations;
FunctionBoundaryLimitations boundary = new FunctionBoundaryLimitations(function);
for (Instruction instruction : instructions) boundary.observe(instruction);
return boundary.values();
}
private CallBoundaryObservation callBoundaryObservation(
Function function, Iterable<Instruction> instructions, Set<Function> returnedCallees,
boolean includeUnresolvedSites, List<Reference> observedReferences)
throws Exception {
List<String> limitations = new ArrayList<>();
FunctionBoundaryLimitations boundary = new FunctionBoundaryLimitations(function);
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
Instruction beforePrevious = null;
Instruction previous = null;
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
else for (Reference reference : observedReferences) {
monitor.checkCancelled();
calls.observe(reference);
}
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (beforePrevious != null && previous != null) {
String seh = borlandSehHandlerLimitation(
function, beforePrevious, previous, instruction);
if (seh != null) limitations.add(seh);
}
beforePrevious = previous;
previous = instruction;
boundary.observe(instruction);
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
@@ -3488,10 +3462,38 @@ public final class ReaGhidraBridge extends HeadlessScript {
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
}
return new CallBoundaryObservation(List.copyOf(limitations),
return new CallBoundaryObservation(boundary.values(),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private final class FunctionBoundaryLimitations {
private final Function function;
private final List<String> limitations = new ArrayList<>();
private Instruction beforePrevious;
private Instruction previous;
private FunctionBoundaryLimitations(Function function) {
this.function = function;
}
private void observe(Instruction instruction) throws Exception {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (beforePrevious != null && previous != null) {
String seh = borlandSehHandlerLimitation(
function, beforePrevious, previous, instruction);
if (seh != null) limitations.add(seh);
}
beforePrevious = previous;
previous = instruction;
}
private List<String> values() {
return List.copyOf(limitations);
}
}
private String borlandSehHandlerLimitation(
Function function, Instruction handlerPush, Instruction framePush,
Instruction frameLink) {
@@ -3673,24 +3675,36 @@ public final class ReaGhidraBridge extends HeadlessScript {
monitor.checkCancelled();
Address address = addresses.next();
if (outgoing) {
for (Reference reference : currentProgram.getReferenceManager()
.getReferencesFrom(address)) {
monitor.checkCancelled();
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
visitObservedReferencesFrom(address, visitor);
}
else {
ReferenceIterator iterator = currentProgram.getReferenceManager()
.getReferencesTo(address);
while (iterator.hasNext()) {
monitor.checkCancelled();
Reference reference = iterator.next();
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
visitObservedReferencesTo(address, visitor);
}
}
}
private void visitObservedReferencesFrom(Address address, ReferenceVisitor visitor)
throws Exception {
for (Reference reference : currentProgram.getReferenceManager().getReferencesFrom(address)) {
monitor.checkCancelled();
visitObservedReference(reference, visitor);
}
}
private void visitObservedReferencesTo(Address address, ReferenceVisitor visitor)
throws Exception {
ReferenceIterator references = currentProgram.getReferenceManager().getReferencesTo(address);
while (references.hasNext()) {
monitor.checkCancelled();
visitObservedReference(references.next(), visitor);
}
}
private static void visitObservedReference(Reference reference, ReferenceVisitor visitor)
throws Exception {
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
private static void addReference(TreeMap<String, Reference> observed, Reference reference) {
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
canonicalAddress(reference.getToAddress()) + "\u0000" +
+56 -19
View File
@@ -23,9 +23,9 @@ that rejection does not roll back browser effects. REA removes its owned
future-document scripts, restores live frame methods, then disconnects.
Add secret declarations, storage seeds, and additional artifact or event
capture only when needed. Action, secret, and storage counts are not capped.
Duration, action, and navigation
timeouts remain fixed provider-owned liveness limits; the request does not
accept a caller-controlled `limits` object.
Duration, action, and navigation timeouts remain fixed provider-owned liveness
limits. `capture.limits` controls retained event and network-content budgets;
it does not change those liveness deadlines.
## Minimal launch example
@@ -179,12 +179,44 @@ declared action. Each step reports action status, elapsed time, sanitized URLs,
event bounds, and independently typed capture states for screenshot, DOM,
accessibility, URL, history, and storage. Console, page-error, network,
WebSocket, frame, worker, popup, and cancelled-download events are attributed to
steps and all observed events and requested artifacts are returned inline.
There are no event, frame, worker, popup, WebSocket, DOM-node,
accessibility-node, storage-entry, screenshot-count, or cumulative
metadata-byte caps. Requested artifacts and observed events are returned
inline without application-defined size ceilings. Other missing sections
remain explicit and make the capture ineligible for equality claims.
steps. Retained events and requested artifacts are returned inline; omitted
events are counted by kind and identify affected network transactions.
Event and selected network-content retention have caller-selected limits under
`capture.limits`. Omitted values use the defaults below; every supplied value
must be a positive safe integer, and callers can raise or lower a default.
Byte limits are conservative allocation-cost estimates for retained event
values, including UTF-16 storage and JSON expansion, rather than serialized
bytes or a promise about exact process memory. `events.omissions.by_kind`
reports counts and reasons for omitted event kinds. Its
`network_transactions` rows identify affected transaction IDs and omitted
event kinds; `network_transaction_range` identifies the contiguous IDs beyond
the transaction limit. Omission metadata is bounded by the selected
transaction limit. Other missing sections remain explicit and make the capture
ineligible for equality claims.
The default limits are 10,000 events, 4 MiB estimated allocation for one
event, 16 MiB estimated allocation across retained events, 1,024 network
transactions, 64 KiB of body bytes per transaction phase, 1 MiB estimated
allocation for body redaction/base64, 1 MiB for retained header values, and
1 MiB for network metadata. The same limits are included in the capture
Evidence. Playwright materializes response bodies before REA can inspect their
byte length; `max_network_body_bytes` bounds subsequent retained copies and
does not bound the browser API read itself. A body that exceeds its selected
limit has an explicit `body-size-limit` or `body-allocation-limit` unavailable
state with the observed and allowed values.
Additional implementation bounds are 16 simultaneous network-content reads,
a 5-second deadline for each provider API read, and 2,048 live event-listener
owners. These are not caller-selectable. The concurrency bound queues admitted
content jobs and does not itself omit observations; a provider read timeout
produces a typed unavailable state, while exceeding listener capacity produces
event-kind omission counts. A preflight avoids body redaction and base64 copies
when the currently available event budget already rejects the body.
Concurrent reads may each pass that check before their events are retained;
their retained-copy work is bounded by 16 times the selected per-body
allocation limit. The Playwright response Buffer itself is materialized before
REA can apply that limit. Requested artifacts and retained events are returned
inline within the declared limits.
Attach-mode captures also declare the unavoidable pre-attach event gap.
Selected event families are observed on the root page and discovered descendant
@@ -288,16 +320,21 @@ retained representation after redaction.
Only completed responses are read. At the end of the final step, observation
stops; requests still in flight emit `request-unfinished`, and their observed
responses have explicitly unavailable bodies. This cutoff does not wait for
long-lived streaming connections. Already-started content reads settle before
browser cleanup, with a provider-owned five-second timeout per read and
cancellation support. Failed or unexposed reads retain per-content states and make selected
event coverage incomplete. Body events can arrive during later steps or final
drain; event bounds describe receipt during the step, while the source event
reference establishes the transaction phase. Comparison attributes content, like
every other event of its transaction, to the step that sent the request (the
receipt step when that request was not observed) and excludes the sequence
reference from normalization.
responses have explicitly unavailable bodies. Already-admitted content reads
drain before browser cleanup. At most 16 reads run at once; queued work is
bounded by the selected network-transaction limit and by successful admission
of its request or response event. Each provider API read has its own five-second
timeout once it starts, so queue wait adds to total drain and cleanup time; this
is not a five-second total capture cutoff. Cancellation interrupts active reads
and queued jobs produce cancelled states without calling their provider APIs.
This drain does not wait for long-lived unfinished streaming connections and
does not refetch unfinished response bodies. Failed or unexposed reads retain
per-content states and make selected event coverage incomplete. Body events can
arrive during later steps or final drain; event bounds describe receipt during
the step, while the source event reference establishes the transaction phase.
Comparison attributes content, like every other event of its transaction, to
the step that sent the request (the receipt step when that request was not
observed) and excludes the sequence reference from normalization.
Playwright scenario capture does not supply initiator stacks; passive CDP
observation retains its own initiator evidence.
+43 -5
View File
@@ -319,11 +319,49 @@ redirected, which keeps large results smaller for agents and scripts; pipe
through `jq .` for indented files. Output selection and formatting do not change
operation status.
| Status | Meaning |
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0` | The operation completed. Its result may include partial evidence, warnings or unresolved questions. |
| `1` | The operation could not complete. Structured output identifies invalid input, permissions, cancellation, timeouts or another failure when available. |
| `128 + N` | Signal `N` ended the process, where the shell or runtime preserves the conventional signal-derived status. |
For `cutter-command`, `--json-output` requests command-specific JSON from
Cutter when available. This is separate from `--format json`, which selects the
CLI result representation; the global `--json` flag selects that same CLI
representation.
Cutter still accepts deprecated `--json=true` and `--json=false` compatibility
values. When both forms are present, `--json-output` takes precedence in either
argument order. Bare `--json` retains its global meaning of selecting JSON CLI
output formatting.
| Status | Meaning |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `0` | The operation completed. Its result may include partial evidence, warnings or unresolved questions. |
| `1` | The operation could not complete. Structured output identifies invalid input, permissions, cancellation, timeouts or another failure when available. |
| `128 + N` | Signal-derived status. An unhandled signal may have ended the process, with the shell or runtime preserving the conventional status; a command may instead have handled the signal, waited for its operation and cleanup to settle, and assigned the same status (`130` for SIGINT, `143` for SIGTERM). In the handled case, it may also have printed a structured `cancelled` result. |
A `cancelled` result does not by itself determine the exit status; it depends on
the command's signal handling. Commands routed through the shared
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
`143` for SIGTERM after their operation settles. Current source registrations
use that handler for process capture and comparison (`capture-process`,
`compare-process-captures`), JavaScript application analysis (`analyze` when it
selects the JavaScript lane, and `analyze-javascript-application`), selected
browser, web and Electron commands (including `analyze-web-bundle`,
`observe-web-execution`, `capture-browser-scenario` and
`capture-electron-scenario`), and the platform and provider command families
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
command, objdump, Rizin and GDB. In these paths, a returned structured
`cancelled` result can be printed before the wrapper assigns the
signal-derived status. This describes the current source routing; it is not a
claim that every command or provider runtime has been independently exercised.
When the reader of piped output closes early, as `rea ... | head` does, the
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
SIGPIPE-terminated writer). The operation itself may have completed; only its
output was declined.
`inspect-artifact` also returns a structured cancelled result, but its direct
analysis cancellation path does not set a signal-derived exit status, so it
exits `1`. Setup has its own contract: cancellation exits `0`. Treat the
structured result and process status as separate signals when scripting
cancellation.
`setup --dry-run` returns `planned` and exits `0`; a cancelled setup also exits
`0`. Setup returns `1` for `needs_confirmation` or `needs_human`.
+43 -4
View File
@@ -140,8 +140,35 @@ profiles](mcp-contracts.md#output-schemas).
Setup registers `rea` in `~/.kimi-code/mcp.json`, or
`$KIMI_CODE_HOME/mcp.json` when set, while preserving unrelated settings and
creating a `.rea.backup` before updating an existing file. Kimi Code infers
stdio from the `command` entry, so setup does not add a `type` field. The
registration includes `startupTimeoutMs = 30000` and `toolTimeoutMs = 120000`.
stdio from the `command` entry, so setup does not add a `type` field. New
registrations include `startupTimeoutMs = 30000` and a `toolTimeoutMs` derived
from REA's known provider allowance plus 120 seconds of operation
headroom. The default is 450000 ms, covering the default 330-second Ghidra
startup; an explicit Ghidra startup timeout of 2700000 ms produces a 2820000 ms
tool timeout. The startup allowance follows an explicitly selected Ghidra or
Hopper provider, and otherwise uses the larger known Ghidra/Hopper deadline.
An explicit `REA_JEB_MCP_TIMEOUT_MS` request budget also contributes to that
maximum. For `ida` or `auto` selection, a valid IDA registration's parsed
`timeoutMs` also contributes to that maximum; its default is 300000 ms.
Setup forwards these explicitly selected REA deadlines and the input schema
profile into the registration; unrelated process values are omitted.
This headroom is a practical default, not a guarantee for every analysis
duration. Setup preserves positive timeout values already in the `rea` entry.
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
caller-selected values aligned and reports a non-blocking advisory when a tool
timeout is shorter than the known provider allowance.
Setup selects the compact input-schema advertisement for new or refreshed Kimi
registrations unless their stored configuration or setup environment explicitly
selects `REA_MCP_INPUT_SCHEMA_PROFILE=full` or `compact`. This keeps REA's
canonical tool validation unchanged while avoiding a Moonshot Kimi Code 2.2.0
failure observed with the full catalog: its request was rejected because a
single tool schema exceeded the provider's size limit. The limit is
provider-specific and not an MCP protocol limit. `rea doctor --client kimi`
reports an older registration with no explicit profile as stale. To restore the
compact profile, run `rea setup --client kimi` or add
`REA_MCP_INPUT_SCHEMA_PROFILE=compact` to the registration's `env`, then restart
Kimi Code. An explicit `full` profile remains respected and can still exceed
Moonshot's limit.
Kimi Code also discovers personal skills in `~/.agents/skills` and project-local
`.agents/skills`, so no separate skill root is required; setup still installs
and verifies the matching skill files through the normal shared-skill flow.
@@ -207,8 +234,20 @@ standalone CLI check.
## Codex and Devin
Codex treats an unset or empty `CODEX_HOME` as `~/.codex`. Devin honors
`XDG_CONFIG_HOME` for its `devin/mcp_config.json` on non-Windows systems.
Codex treats an unset or empty `CODEX_HOME` as `~/.codex`. Its MCP server
registration supports `startup_timeout_sec` for connection initialization and
`tool_timeout_sec` for each tool call. New REA registrations use a 30-second
startup timeout and a tool timeout derived from the known provider
allowance plus 120 seconds of operation headroom. This yields 450 seconds for
the default Ghidra allowance; an explicit Ghidra startup timeout of 2700
seconds yields 2820 seconds. Setup preserves positive caller-selected timeout
values in the REA entry, and doctor reports a non-blocking advisory when a tool
timeout is shorter than the known provider allowance. An explicitly selected
JEB request budget also contributes to the allowance. A configured IDA
registration's parsed `timeoutMs` contributes for `ida` or `auto` selection.
These values are client-side deadlines, so increase them in `~/.codex/config.toml` if a
workload needs more time. Devin honors `XDG_CONFIG_HOME` for its
`devin/mcp_config.json` on non-Windows systems.
## OMP
+2 -2
View File
@@ -153,8 +153,8 @@ overview merely because its source artifact is unchanged.
Function extents and instruction completeness refer to Ghidra's analyzed body.
An inferred no-return call can terminate that body before subsequent bytes.
Pseudocode, assembly, procedure info, instruction windows, callee and caller
reads, and dossiers report observed terminal call sites in Evidence limitations,
Pseudocode, assembly, procedure info, instruction windows, incoming and outgoing
references, callee and caller reads, and dossiers report observed terminal call sites in Evidence limitations,
including resolved callee names, addresses, and FunctionManager flags when
available. On x86-32, the same views also report the exact Borland/Delphi-style
FS exception-registration pattern when its handler address falls outside the
+2 -2
View File
@@ -131,9 +131,9 @@ Use a package version that actually includes this adapter; repository main can b
{"name":"close_binary","arguments":{}}
```
Opening an MCP target starts its selected provider session; headless opening waits for auto-analysis. Set the agent's MCP request deadline long enough for the target. The registration's optional `timeoutMs` defaults to 300000 for upstream requests. A freshly opened headless worker can briefly report `status: "busy"` from `server_health`; REA re-reads the health every 100 ms until the worker answers, the caller cancels, or `timeoutMs` has elapsed since the first read. Only expiry while the worker is still busy produces a retryable `provider_timeout`; no further read is started once the deadline has passed. That error names the busy tool of the last observation, or `unknown` when the worker reported none or a value that is not tool-name-shaped. Cancellation during the wait produces `cancelled`. Any other unexpected health payload, including a busy payload with an invalid shape, remains `unreadable_output`. Cancellation drains the current upstream request before cleanup; it does not force-stop GUI analysis. An open request that times out without a completed response retains the private workspace because worker inventory alone cannot prove the request has stopped. `binary_session` reports actual tool availability. Repeated doctor/setup calls are unnecessary when the registration already works.
Opening an MCP target starts its selected provider session; headless opening waits for auto-analysis. Set the agent's MCP request deadline long enough for the target. The registration's optional `timeoutMs` defaults to 300000 for upstream requests. An upstream SDK request that reaches this limit returns `provider_timeout` with its operation and actual request budget. A freshly opened headless worker can briefly report `status: "busy"` from `server_health`; REA re-reads health every 100 ms until the worker answers, the caller cancels, or `timeoutMs` has elapsed since the first read. Each health request is capped by the time remaining in that overall wait, so a stalled read cannot extend the readiness deadline by another full request timeout. Expiry while the worker is still busy produces a retryable `provider_timeout`; no further read is started after the deadline. That error names the busy tool from the last observation, or `unknown` when the worker reported none or a value that is not tool-name-shaped. If a health request itself consumes the remaining wait budget, the same last-busy timeout is returned with the stalled request's captured output. Cancellation during the wait produces `cancelled`. Any other unexpected health payload, including a busy payload with an invalid shape, remains `unreadable_output`. Cancellation drains the current upstream request before cleanup; it does not force-stop GUI analysis. An open request that times out without a completed response retains the private workspace because worker inventory alone cannot prove the request has stopped. `binary_session` reports actual tool availability. Repeated doctor/setup calls are unnecessary when the registration already works.
For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <client>`. Setup includes that file reference in its reviewed plan and preserves it after approval; it does not copy upstream credentials into the agent configuration. `rea doctor --provider ida --json` validates the registration without starting IDA and explicitly leaves runtime compatibility and target binding for analysis.
For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <client>`. Setup includes that file reference in its reviewed plan and preserves it after approval; it does not copy upstream credentials into the agent configuration. For `ida` or `auto` selection, Codex/Kimi setup and doctor use the effective client registration's parsed `timeoutMs` (default 300000 ms) when recommending the agent's per-tool deadline. Existing positive client deadlines remain caller choices. `rea doctor --provider ida --json` validates the registration without starting IDA and explicitly leaves runtime compatibility and target binding for analysis.
## Coverage and evidence
+47 -1
View File
@@ -147,7 +147,8 @@ identity remains available at `identity.catalog`.
Setup pins package-runner MCP registrations to the exact installed REA version,
installs the matching skill and on-demand references in the same plan, and adds
`startup_timeout_sec = 30` for Codex and Grok Build.
`startup_timeout_sec = 30` for Codex and Grok Build. Codex also receives a
`tool_timeout_sec` derived from known provider startup plus operation headroom.
## Limit REA to one project or agent profile
@@ -540,6 +541,37 @@ search permissions (`0700`) on a directory you own before retrying.
On supported Linux distributions, approved setup verifies Hopper's official
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
The official release feed selects the current public demo without a version
pin. REA verifies the package's detached PGP signature against Hopper's stable
primary fingerprint `7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD`, then checks
that the signed package and installed launcher agree with `main_exe_hash`.
Rotating signing subkeys remain bound to that primary identity.
The feed row itself is not PGP-signed; doctor labels its display version as
`reported_release_version`, separately from the authenticated launcher identity.
The approval plan includes the caller-selected cache location. Approved setup
retains the signed archive, public key and release metadata under
`${XDG_CACHE_HOME:-$HOME/.cache}/rea/hopper-releases/<launcher-sha256>`.
An empty `XDG_CACHE_HOME` uses `$HOME/.cache`. Approved setup can repair a
damaged caller-owned entry after authenticating a fresh package; the previous
entry is preserved beside it as `<launcher-sha256>.backup-<id>`. Ordinary doctor
and launches reject damaged evidence and never repair it implicitly.
Approved publications of the same canonical cache entry share a process-backed
lease. Concurrent setup waits without moving another writer's evidence; a
failed replacement restores the previous directory when possible and reports
any retained backup path. Unavailable reads or native inspection tools do not
authorize replacing existing evidence.
Doctor and launches authenticate this evidence for offline use; an editable
digest receipt alone cannot authorize a launcher. Verification reuses at most
eight immutable package proofs per process, keyed by the complete package,
key, signature, package format and expected launcher digest. Cached file reads
still check identity and content. Without retained evidence, inspecting an
existing install requires the matching current official package and network
access. Successful live inspection can be reused within that process after
rehashing the launcher; it does not write a persistent cache or claim that a
previously observed release is still the latest. Older uncached builds require
an approved update while the feed exposes only its current release.
REA runs the supported demo build on a private Xvfb display and selects Hopper's
offered demo mode for each analysis session; it does not require the user's
desktop display. Unattended package-manager access requires
@@ -634,6 +666,20 @@ If startup reports `Cannot launch Hopper! Connection is invalid.` or waits for
bridge readiness, check Hopper for a pending demo, license, or loader dialog.
Complete the dialog and retry. If REA reports `cleanup_incomplete`, follow the
retained-resource guidance below before reopening the target.
On macOS, Hopper can show an `Executing Python Script...` overlay while the REA
bridge serves an active MCP connection. Long-lived MCP clients can keep that
connection open across agent turns, so the overlay may remain visible after the
analysis calls finish; by itself, it does not establish a stalled startup. REA
does not automatically idle-close a still-connected target. When the
investigation is finished, export an Evidence bundle first if you need to keep
the session ledger, then call `close_binary`; set `snapshot_path` to save a
provider-neutral analysis snapshot as part of closing. A successful close
shuts down the REA bridge and clears its overlay; REA does not control the
macOS Hopper application lifetime.
The Hopper GUI Cancel button's behavior for this bridge has not been verified;
`close_binary` is the supported cleanup path.
Hopper serializes analysis requests. Cancelling a wait can leave provider work
running, which the session reports. Successful decompilation is cached until
a relevant rename or comment changes it.
+2
View File
@@ -60,6 +60,8 @@ provider implicitly.
`trace_javascript_semantics` queries the authenticated companion graph without
rereading mutable files. A seed may identify a semantic or application node,
literal, function fingerprint, property, endpoint, event, or boundary field.
For a recovered parse, coercion, or validation boundary, its field value is the
static operation's method name (for example, `Number`).
The query declares backward provenance, forward influence, callers, or
ownership and can restrict admitted relation kinds. REA returns every matching
seed and the complete reachable subgraph; callers do not choose node, relation,
+11 -1
View File
@@ -206,6 +206,9 @@ code unit, matching the original bytes identified by the artifact digest.
Plain `.ts` artifact sources use TypeScript syntax without JSX, including
angle-bracket type assertions. `.tsx` and `.jsx` retain JSX parsing.
JavaScript sources use JavaScript grammar with JSX, never TypeScript syntax,
so valid JavaScript that TypeScript reads differently keeps its JavaScript
meaning.
Relative TypeScript imports can name their emitted `.js`, `.mjs`, or `.cjs`
counterparts while resolving to inventoried source files. This substitution
@@ -220,7 +223,10 @@ Selected bounded text is then parsed as inert data to recover:
- Webpack and Rspack chunk registrations and module factories represented as
AST literals;
- static imports, dynamic imports, CommonJS `require` calls, workers, and
service workers;
service workers (only `new Worker(new URL(path, import.meta.url))` resolves
relative to the calling module; string worker and service-worker paths use
the document base URL or Node's working directory, which static analysis
does not establish, so they keep their expression and stay unresolved);
- route, network endpoint, storage, and vendor-marker observations;
- local source-map declarations and original
source names and content digests;
@@ -626,6 +632,10 @@ href are read without surrounding whitespace or embedded tabs and newlines, and
their percent-encoded path bytes are decoded; encoded dot and separator bytes
are still rejected. CommonJS module lookups retain extension and directory
resolution.
ESM, HTML, and URL references apply URL dot segments to empty path segments:
`./assets//../app.js` selects `assets/app.js`, while a remaining empty segment
such as `./assets//app.js` stays unavailable. CommonJS keeps filesystem path
semantics and selects `app.js` for the same text.
Unresolved HTML references retain their declaration, source range, and resolution
reason in the renderer observations.
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
+15
View File
@@ -16,6 +16,21 @@ URL after validating its syntax. That validation does not establish that the
JEB service is reachable or that a project is open; use the existing
`inspect-jeb-client` command for a read-only service query.
MCP `binary_session` probes the live JEB service and its advertised tool catalog.
Availability is reported per operation: a reachable headless client can support
inspection and decompilation while refusing `open_jeb_project`. Catalog discovery
uses one 60-second deadline across connection and pagination; a responsive server
cannot keep discovery running indefinitely by returning more pages.
Each upstream tool call has an absolute 60-second request deadline by default.
Set `REA_JEB_MCP_TIMEOUT_MS` to a positive decimal number of milliseconds to
select a different per-request budget, for example `REA_JEB_MCP_TIMEOUT_MS=300000`
for a five-minute decompilation. This budget starts after discovery and applies
to each individual `tools/call`; it does not extend the separate catalog deadline
or establish one end-to-end deadline across a whole CLI or MCP operation. A
request that reaches its budget returns a typed timeout, while caller
cancellation remains a cancellation.
Verified against JEB 5.48.0 (Linux x64, licensed build) serving `jeb-mcp-server` 1.3.0 over streamable HTTP.
## Operations
+30 -12
View File
@@ -38,13 +38,10 @@ socket and device entries are not opened. Case-distinct archive members remain
separate observations. Their case-insensitive materialization diagnostic does
not reduce traversal coverage when both spellings were read.
Current output also retains `coverage_status` (`complete` or `partial`) for existing clients, derived from the scoped traversal status. The owning parser normalizes retained `coverage_status` records to scoped coverage and rejects conflicting declarations.
The coverage field is optional when reading older retained Evidence. The owning
inventory parser recognizes historical REA scope notices for compatibility;
unfamiliar legacy limitations remain conservative. Typed records do not derive
coverage from limitation text. Pages from one inventory must carry the same
coverage declaration.
Current inventory Evidence requires scoped `coverage` with a `complete` or
`unknown` status. Consumers use this producer-owned fact and reject records
without it; limitations are explanatory and do not supply or infer coverage.
Pages from one inventory must carry the same coverage declaration.
Function dossiers can declare per-facet coverage in the optional
`normalized_result.facet_availability` map. Each supplied facet is either
@@ -520,6 +517,14 @@ These deadlines have different owners:
| Darwin swiftc helper | REA allows 120,000 ms by default for ownership-helper compilation; `REA_SWIFTC_COMPILE_TIMEOUT_MS` accepts decimal integers from 1–2,147,483,647 ms. Invalid values fail configuration validation. |
| Individual tool call | The client bounds its wait, including cold engine startup. The pinned client SDK 2.3.1 defaults to 60,000 ms and can cancel earlier than REA's startup deadline. |
For [Codex and Kimi registrations](client-configuration.md), setup supplies
missing tool deadlines from the known provider allowance plus 120 seconds of
operation headroom: 450 seconds with the default Ghidra allowance, or 2820
seconds with `REA_GHIDRA_STARTUP_TIMEOUT_MS=2700000`. Positive caller-selected
deadlines remain unchanged. Doctor reports an informational recommendation
when a configured deadline is shorter than the known provider allowance. An
explicit JEB upstream request budget also contributes to that allowance.
For an already connected client using the pinned SDK, request options are the
**second** argument of `callTool`:
@@ -549,10 +554,20 @@ Waiting notifications stop when startup settles or the caller cancels. They do
not measure engine work completed.
The pinned SDK can reset the request's inactivity timeout on progress with
`resetTimeoutOnProgress: true`. Also set `maxTotalTimeout` to bound the complete
request; the provider's startup deadline still applies. These options belong in
the same second argument of `callTool`. Without that explicit client policy, the
default 60-second request deadline still expires despite waiting notifications.
`resetTimeoutOnProgress: true`. These options belong in the same second
argument of `callTool`. Without that explicit client policy, the default
60-second request deadline still expires despite waiting notifications.
SDK 2.3.1 checks `maxTotalTimeout` when progress resets the timer; it does not
schedule a separate absolute timer for that setting. A silent interval can
therefore run beyond the declared total. For an absolute caller deadline,
also pass `signal: AbortSignal.timeout(totalBudgetMs)`. If the caller already
has a cancellation signal, preserve both with
`AbortSignal.any([callerSignal, AbortSignal.timeout(totalBudgetMs)])`.
`maxTotalTimeout` can still bound progress resets, and REA's provider startup
deadline still applies. Choose the total budget for the expected workload;
neither server configuration nor progress can override a remote caller's
deadline.
The 240-second setting is a measured example, not a universal timeout: a
controlled Linux x64/WSL2 fixture took about 64 seconds for its first overview
@@ -693,7 +708,10 @@ reported by strict-depth MCP clients in [#2813](https://github.com/morluto/rea/i
WorkBuddy/CodeBuddy gateways have also rejected full REA catalogs with an
opaque `11133` error while accepting the compact profile
([#2999](https://github.com/morluto/rea/issues/2999)); compatibility depends on
the selected gateway and model.
the selected gateway and model. That report used ZCode through a gateway to a
Kimi-K3 backend, not the WorkBuddy MCP client. Set the profile on REA's
registration in the actual MCP host; the host's name alone does not identify
the downstream model or its schema constraints.
The compact profile renders the same tool set, output schemas, annotations,
and canonical server-side validation. It changes only the advertised input
+9 -2
View File
@@ -481,8 +481,9 @@ user's permissions and may change files, show UI or use the network.
Debugging must be allowed for the user (Developer Tools access). A
hardened-runtime target without `com.apple.security.get-task-allow` cannot be
debugged; `inspect_signature` reports this as the debugger-attach facet, and
the tool fails with a `debugger-attach-denied` reason.
debugged; `inspect_signature` reports the signature's `hardened_runtime` value
and its `entitlements`, including `com.apple.security.get-task-allow` when it
is present. A denied trace reports the `debugger-attach-denied` reason.
## Provider and verification boundaries
@@ -569,6 +570,12 @@ Exact and ambiguous value counts describe emitted root/object value instances;
a root and an object may contain the same value. Archive digests identify the
original bytes.
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`. A
date the decoder cannot represent, such as an XML date with a year beyond 9999
or a negative year, is reported as `{ "$plist_type": "date", "iso8601": null }`
with a counted limitation; the rest of the plist or archive is still decoded.
`inspect_plist` and Interface Builder decoding use the same projection.
## Locate a byte signature
Use `search_bytes` after opening a target with Ghidra to locate a known constant
+6 -1
View File
@@ -2,7 +2,7 @@ REA npm dependency inventory
Prepared from the root package-lock.json; refresh this inventory when locked dependencies change.
License identifiers are upstream lockfile declarations, not a legal assessment.
Runtime includes transitive and optional platform packages; development includes build, test and website tooling.
Locked package entries: 721; runtime: 177.
Locked package entries: 722; runtime: 178.
Full runtime copyright/license/NOTICE texts: npm-runtime-notices.txt.
@algolia/abtesting@1.25.0 | development | MIT
@@ -2740,6 +2740,11 @@ oniguruma-to-es@3.1.1 | development | MIT
Package: https://www.npmjs.com/package/oniguruma-to-es/v/3.1.1
Archive: https://registry.npmjs.org/oniguruma-to-es/-/oniguruma-to-es-3.1.1.tgz
Integrity: sha512-bUH8SDvPkH3ho3dvwJwfonjlQ4R80vjyvrU8YpxuROddv55vAEJrTuCuCVUhhsHbtlD9tGGbaNApGQckXhS8iQ==
openpgp@6.3.2 | runtime | LGPL-3.0+
Lockfile path: node_modules/openpgp
Package: https://www.npmjs.com/package/openpgp/v/6.3.2
Archive: https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz
Integrity: sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==
oxc-parser@0.150.0 | development | MIT
Lockfile path: node_modules/oxc-parser
Package: https://www.npmjs.com/package/oxc-parser/v/0.150.0
+175 -1
View File
@@ -1,7 +1,7 @@
REA third-party npm runtime copyright, license and NOTICE texts
Collected from integrity-verified locked npm archives. Release-specific supplements and upstream notice gaps are identified in their entries.
External engines are separately obtained; see THIRD_PARTY_NOTICES.md.
Runtime dependency fingerprint (SHA-256): 49b181923ab240c68c64d35ecdd8f93cb5cd516c765f2282d0e0eca0bcadd13e
Runtime dependency fingerprint (SHA-256): 131100b2e1e6ad5e55b10a0baf4be2d7a0838fd724fb307b9bafd73446b0d3ec
===== @babel/helper-string-parser@8.0.6 =====
Declared license: MIT
@@ -5171,6 +5171,180 @@ ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
===== openpgp@6.3.2 =====
Declared license: LGPL-3.0+
Archive: https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz
Integrity: sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==
Scope: npm runtime dependency. GPLv3 terms incorporated by this LGPL text are preserved in licenses/GPL-3.0.txt.
--- package/LICENSE ---
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
===== pako@1.0.11 =====
Declared license: (MIT AND Zlib)
Archive: https://registry.npmjs.org/pako/-/pako-1.0.11.tgz
+36 -6
View File
@@ -26,6 +26,29 @@ its manifest. Release CI builds both, and `prepublishOnly` rejects a missing,
stale, mismatched, or wrong-architecture artifact. A development tarball built
without this lane reports Windows native controls as unavailable.
## Restore the artifact in a source checkout
The source tree omits `native/windows/build/` from Git. To run Windows
provider-backed workflows from a checkout without the MinGW build lane, extract
the two files from the published `rea-agents` tarball whose version exactly
matches the checkout's `package.json` version. Do not use `latest` or copy files
from another REA version; the loader verifies the manifest version, ABI,
architecture, and addon digest before it grants authority.
From the checkout root, replace `<matching-version>` with the `version` in
`package.json`:
```sh
npm pack rea-agents@<matching-version>
tar -xzf rea-agents-<matching-version>.tgz
mkdir -p native/windows/build
cp package/native/windows/build/manifest.json native/windows/build/
cp package/native/windows/build/rea-windows-x64.node native/windows/build/
```
This is a manual recovery step. REA does not fetch or install the artifact
automatically. Alternatively, build it using the Linux/MinGW lane above.
## Boundaries
- Opaque Node objects carry a native type tag, resource kind, and closed state.
@@ -88,7 +111,9 @@ without this lane reports Windows native controls as unavailable.
- Cleanup walks locked directory objects, removes files by handle, and unlinks
reparse entries without traversing their targets. Failed removal retains the
root and ancestor leases for retry; a partially removed runtime accepts only
cleanup, and its registry entry remains until removal succeeds. Normal close, cancellation,
cleanup, and its registry entry remains until removal succeeds. The owned root
handle includes attribute-write access so cleanup can clear a read-only marker,
matching the child cleanup handles. Normal close, cancellation,
startup failure, and timeout release provider resources. Abrupt owner death
kills the owned job. The Android provider additionally uses a detached helper
that owns its private runtime directory and cleans it when the owner pipe
@@ -133,8 +158,11 @@ npm run verify:windows-native
npm run verify:ghidra:windows
```
The native inspection probe retains its directory owner if deletion fails. A
later availability query retries that cleanup before creating another probe;
An unpublished runtime retains its directory and parent handles if validation,
result publication, or rollback fails. Errors retain the original failure and
any rollback failure. Later runtime creation or native inspection retries that
exact owner before allocating another directory. The same owner covers the
native inspection probe. A later availability query retries failed probe cleanup;
successful admission and static artifact failures remain cached. Environment
teardown makes one final cleanup attempt and reports a persistent failure to
stderr rather than claiming the directory was removed.
@@ -151,9 +179,11 @@ On Windows, after compiling the runtime:
node scripts/verify-windows-native-inspection.mjs
```
This artifact injects one access-denied disposition result while using a real
NTFS directory. The verifier checks retained ownership, recovery on a later
query, cached successful admission, and cleanup on environment teardown. It
This artifact injects an access-denied disposition result while using real
NTFS directories, and separately denies post-create validation and rollback.
The verifier checks retained ownership after inspection and direct runtime
creation failures, recovery on later admission, cached successful inspection,
and cleanup on environment teardown. It
renames the retained probe and recreates its old path, checking that cleanup
removes the owned inode while preserving the replacement directory.
It does not reproduce or identify the host-dependent cause reported in #1701.
+35 -15
View File
@@ -60,6 +60,15 @@ napi_value wrap(napi_env env, std::unique_ptr<Resource> value) {
value.release();
return result;
}
void publishRuntimeHandle(napi_env env, napi_value observation, std::unique_ptr<Runtime>& value) {
auto handle = object(env);
check(napi_type_tag_object(env, handle, &RESOURCE_TAG));
// Complete the result before transferring cleanup ownership. Any earlier
// N-API failure leaves the unpublished runtime in its environment owner.
set(env, observation, "handle", handle);
check(napi_wrap(env, handle, value.get(), finalize, nullptr, nullptr));
value.release();
}
Resource& resource(napi_env env, napi_value value, Kind kind) {
bool tagged = false;
check(napi_check_object_type_tag(env, value, &RESOURCE_TAG, &tagged));
@@ -77,8 +86,15 @@ napi_value failureValue(napi_env env, const Failure& failure) {
std::wstring detail = message == nullptr ? L"" : message;
if (message != nullptr) LocalFree(message);
const std::wstring constraint(failure.constraint.begin(), failure.constraint.end());
auto text = string(env, constraint + (failure.path.empty() ? L"" : L" [" + failure.path + L"]") +
L" (Win32 " + std::to_wstring(failure.win32) + L"): " + detail);
std::wstring renderedMessage = constraint + (failure.path.empty() ? L"" : L" [" + failure.path + L"]") +
L" (Win32 " + std::to_wstring(failure.win32) + L"): " + detail;
if (!failure.cleanupConstraint.empty()) {
const std::wstring cleanup(failure.cleanupConstraint.begin(), failure.cleanupConstraint.end());
renderedMessage += L"; cleanup also failed: " + cleanup +
(failure.cleanupPath.empty() ? L"" : L" [" + failure.cleanupPath + L"]") +
L" (Win32 " + std::to_wstring(failure.cleanupWin32) + L")";
}
auto text = string(env, renderedMessage);
napi_value error; napi_create_error(env, nullptr, text, &error);
const char* code = failure.win32 == ERROR_FILE_NOT_FOUND || failure.win32 == ERROR_PATH_NOT_FOUND ? "ENOENT"
: failure.win32 == ERROR_ACCESS_DENIED ? "EACCES" : "ERR_REA_WINDOWS_NATIVE";
@@ -87,21 +103,24 @@ napi_value failureValue(napi_env env, const Failure& failure) {
set(env, error, "requestedPath", string(env, failure.path));
set(env, error, "win32Code", number(env, failure.win32));
set(env, error, "win32Message", string(env, detail));
if (!failure.cleanupConstraint.empty()) {
set(env, error, "cleanupConstraint", string(env, failure.cleanupConstraint));
set(env, error, "cleanupRequestedPath", string(env, failure.cleanupPath));
set(env, error, "cleanupWin32Code", number(env, failure.cleanupWin32));
}
if (failure.runtimeDirectoryUnallocated)
set(env, error, "runtimeAllocation", string(env, std::string("not-started")));
return error;
}
struct InspectionContext { std::unique_ptr<Runtime> probe; };
static void cleanupInspection(void* data) noexcept {
std::unique_ptr<InspectionContext> context(static_cast<InspectionContext*>(data));
if (!context->probe) return;
try { closeRuntime(*context->probe); }
static void cleanupRuntimeContext(void* data) noexcept {
std::unique_ptr<RuntimeContext> context(static_cast<RuntimeContext*>(data));
if (!context->pending) return;
try { closeRuntime(*context->pending); }
catch (const Failure& failure) {
// Environment teardown cannot retain an owner or return an error. Report
// terminal cleanup failure instead of claiming the directory was removed.
std::fprintf(stderr, "REA Windows probe cleanup failed (Win32 %lu) path=\"",
std::fprintf(stderr, "REA Windows runtime cleanup failed (Win32 %lu) path=\"",
static_cast<unsigned long>(failure.win32));
// Emit reversible UTF-16 escapes without allocation or CRT locale conversion.
for (wchar_t unit : failure.path) {
@@ -112,10 +131,10 @@ static void cleanupInspection(void* data) noexcept {
std::fprintf(stderr, "\"\n");
}
catch (const std::exception& failure) {
std::fprintf(stderr, "REA Windows probe cleanup failed: %s\n", failure.what());
std::fprintf(stderr, "REA Windows runtime cleanup failed: %s\n", failure.what());
}
catch (...) {
std::fprintf(stderr, "REA Windows probe cleanup failed: unknown native exception\n");
std::fprintf(stderr, "REA Windows runtime cleanup failed: unknown native exception\n");
}
}
@@ -128,12 +147,13 @@ static napi_value call(napi_env env, napi_callback_info info) {
require(count == 2, "Native call requires operation and arguments", L"", ERROR_INVALID_PARAMETER);
const auto operation = wide(env, args[0]);
const auto values = array(env, args[1]);
auto& context = *static_cast<RuntimeContext*>(data);
if (operation == L"inspect") {
require(values.empty(), "Inspect takes no arguments", L"", ERROR_INVALID_PARAMETER);
return inspect(env, static_cast<InspectionContext*>(data)->probe);
return inspect(env, context.pending);
}
if (operation.rfind(L"process_", 0) == 0) return processCall(env, operation, values);
return filesystemCall(env, operation, values);
return filesystemCall(env, operation, values, context);
} catch (const Failure& failure) {
auto error = failureValue(env, failure);
napi_throw(env, error);
@@ -145,8 +165,8 @@ static napi_value call(napi_env env, napi_callback_info info) {
}
static napi_value initialize(napi_env env, napi_value exports) {
auto context = std::make_unique<InspectionContext>();
if (napi_add_env_cleanup_hook(env, cleanupInspection, context.get()) != napi_ok) return nullptr;
auto context = std::make_unique<RuntimeContext>();
if (napi_add_env_cleanup_hook(env, cleanupRuntimeContext, context.get()) != napi_ok) return nullptr;
// The environment owns callback data before a function can expose it. Failed
// initialization still leaves a valid context for the registered cleanup hook.
auto* data = context.release();
+12 -3
View File
@@ -33,6 +33,9 @@ struct Failure : std::runtime_error {
DWORD win32;
std::wstring path;
std::string constraint;
std::string cleanupConstraint;
DWORD cleanupWin32 = ERROR_SUCCESS;
std::wstring cleanupPath;
// Affirmative proof for a failed runtime-parent admission; omission is unknown.
bool runtimeDirectoryUnallocated = false;
Failure(std::string constraint, std::wstring path = L"", DWORD code = GetLastError())
@@ -96,6 +99,9 @@ struct Runtime : Resource {
std::atomic_bool snapshotCancelled{false};
Runtime() : Resource(Kind::Runtime) {}
};
// A failed, not-yet-published runtime remains owned by its N-API environment
// until a later admission or environment teardown can retry cleanup by handle.
struct RuntimeContext { std::unique_ptr<Runtime> pending; };
struct Process : Resource {
Handle job, process, stdoutRead, stderrRead, stdinWrite;
// Present only for a job with a commit limit; receives the job's notifications.
@@ -120,21 +126,24 @@ std::wstring wide(napi_env env, napi_value value);
double numeric(napi_env env, napi_value value);
std::vector<napi_value> array(napi_env env, napi_value value);
napi_value wrap(napi_env env, std::unique_ptr<Resource> resource);
void publishRuntimeHandle(napi_env env, napi_value observation, std::unique_ptr<Runtime>& runtime);
Resource& resource(napi_env env, napi_value value, Kind kind);
std::unique_ptr<File> openFile(const std::wstring& path, DWORD access = GENERIC_READ,
bool directory = false,
DWORD finalSharing = FILE_SHARE_READ);
napi_value identity(napi_env env, HANDLE handle, const std::wstring& path);
std::unique_ptr<Runtime> createRuntime(const std::wstring& parent, const std::wstring& prefix);
void createRuntime(const std::wstring& parent, const std::wstring& prefix,
std::unique_ptr<Runtime>& owner);
void cleanupFailedRuntime(std::unique_ptr<Runtime>& owner, Failure& primary);
void closeRuntime(Runtime& runtime);
napi_value filesystemCall(napi_env env, const std::wstring& operation,
const std::vector<napi_value>& args);
const std::vector<napi_value>& args, RuntimeContext& context);
std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::wstring& commandLine,
const std::wstring& cwd, const std::vector<std::wstring>& environment,
bool protocolStdin = false, ULONGLONG memoryLimit = 0);
napi_value processCall(napi_env env, const std::wstring& operation,
const std::vector<napi_value>& args);
napi_value inspect(napi_env env, std::unique_ptr<Runtime>& probe);
napi_value inspect(napi_env env, std::unique_ptr<Runtime>& pending);
} // namespace rea
+69 -21
View File
@@ -67,7 +67,10 @@ std::unique_ptr<File> openFile(const std::wstring& requested, DWORD access, bool
result->requestedPath = requested;
const auto& path = result->path;
require(directory || path.size() > 3, "Target is not a regular file", requested, ERROR_DIRECTORY);
result->handles.push_back(openComponent(path.substr(0, 3), 0, true));
// The volume root cannot be replaced or deleted. Match the later root
// reopen's write sharing here so a root writer cannot block admission.
result->handles.push_back(openComponent(path.substr(0, 3), 0, true,
FILE_SHARE_READ | FILE_SHARE_WRITE));
wchar_t filesystem[64]{};
DWORD flags = 0;
require(GetVolumeInformationByHandleW(result->handles.front().get(), nullptr, 0, nullptr, nullptr,
@@ -181,6 +184,29 @@ static Handle createRelative(HANDLE parent, const std::wstring& name, DWORD acce
return result;
}
void cleanupFailedRuntime(std::unique_ptr<Runtime>& owner, Failure& primary) {
if (!owner || !owner->directory.valid()) {
owner.reset();
return;
}
try {
closeRuntime(*owner);
owner.reset();
} catch (const Failure& cleanup) {
primary.cleanupConstraint = cleanup.constraint;
primary.cleanupWin32 = cleanup.win32;
primary.cleanupPath = cleanup.path;
} catch (const std::exception& cleanup) {
primary.cleanupConstraint = cleanup.what();
primary.cleanupWin32 = ERROR_GEN_FAILURE;
primary.cleanupPath = owner->path;
} catch (...) {
primary.cleanupConstraint = "Unknown runtime cleanup failure";
primary.cleanupWin32 = ERROR_GEN_FAILURE;
primary.cleanupPath = owner->path;
}
}
// Lookup relative to an already-owned directory; path is diagnostic only.
static Handle openRelative(HANDLE parent, const std::wstring& name, DWORD access,
DWORD sharing, ULONG kind, const std::wstring& path) {
@@ -334,7 +360,11 @@ static napi_value readCutterBridgeDescriptor(napi_env env, const std::vector<nap
return result;
}
std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std::wstring& prefix) {
void createRuntime(const std::wstring& parentPath, const std::wstring& prefix,
std::unique_ptr<Runtime>& owner) {
// A previous failed admission retains the original root handle. Retry that
// owner before allowing this environment to allocate another runtime.
if (owner) { closeRuntime(*owner); owner.reset(); }
std::unique_ptr<File> parent;
try { parent = openFile(parentPath, 0, true); }
catch (Failure& failure) {
@@ -346,24 +376,29 @@ std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std
std::array<unsigned char, 16> random;
require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0,
"Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE);
Security security;
auto result = std::make_unique<Runtime>();
const auto name = prefix + hex(random.data(), random.size());
result->path = parent->path + (parent->path.back() == L'\\' ? L"" : L"\\") + name;
Security security;
result->directory = createRelative(parent->get(), name,
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL,
true, security, result->path);
result->parents = std::move(parent->handles);
owner = std::move(result);
try {
verifyPrivate(result->directory.get(), result->path, true);
// The owned child pins the parent nonempty before write sharing is relaxed.
parent->handles.back() = openComponent(parent->path, 0, true, FILE_SHARE_READ | FILE_SHARE_WRITE);
result->parents = std::move(parent->handles);
} catch (...) {
FILE_DISPOSITION_INFO deletion{TRUE};
SetFileInformationByHandle(result->directory.get(), FileDispositionInfo, &deletion, sizeof(deletion));
owner->directory = createRelative(owner->parents.back().get(), name,
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | FILE_WRITE_ATTRIBUTES | READ_CONTROL,
true, security, owner->path);
// FILE_CREATE proves ownership and pins the parent nonempty independently
// of admission validation. A failed validation must not retain a strict
// parent lease that prevents renaming the still-owned child before retry.
owner->parents.back() = openComponent(parent->path, 0, true, FILE_SHARE_READ | FILE_SHARE_WRITE);
verifyPrivate(owner->directory.get(), owner->path, true);
} catch (Failure& primary) {
cleanupFailedRuntime(owner, primary);
throw;
} catch (const std::exception& error) {
Failure primary(error.what(), owner->path, ERROR_GEN_FAILURE);
cleanupFailedRuntime(owner, primary);
throw primary;
}
return result;
}
static std::wstring relativePath(Runtime& root, const std::wstring& relative) {
@@ -438,6 +473,9 @@ static void dispose(HANDLE handle, const std::wstring& path) {
require(GetFileInformationByHandleEx(handle, FileBasicInfo, &info, sizeof(info)), "Read cleanup attributes failed", path);
if ((info.FileAttributes & FILE_ATTRIBUTE_READONLY) != 0) {
info.FileAttributes &= ~FILE_ATTRIBUTE_READONLY;
// Zero means "leave attributes unchanged" to FileBasicInfo. A file whose
// sole attribute was READONLY needs NORMAL to actually clear that marker.
if (info.FileAttributes == 0) info.FileAttributes = FILE_ATTRIBUTE_NORMAL;
require(SetFileInformationByHandle(handle, FileBasicInfo, &info, sizeof(info)), "Clear owned readonly attribute failed", path);
}
FILE_DISPOSITION_INFO deletion{TRUE};
@@ -654,7 +692,8 @@ static napi_value snapshot(napi_env env, napi_value rootValue, Runtime& root,
return promise;
}
napi_value filesystemCall(napi_env env, const std::wstring& operation, const std::vector<napi_value>& args) {
napi_value filesystemCall(napi_env env, const std::wstring& operation,
const std::vector<napi_value>& args, RuntimeContext& context) {
if (operation == L"cutter_bridge_verify_directory") return verifyCutterBridgeDirectory(env, args);
if (operation == L"cutter_bridge_read_descriptor") return readCutterBridgeDescriptor(env, args);
const auto expected = operation == L"open" || operation == L"close" || operation == L"runtime_close" || operation == L"runtime_snapshot_cancel" ? 1
@@ -688,12 +727,21 @@ napi_value filesystemCall(napi_env env, const std::wstring& operation, const std
return result;
}
if (operation == L"runtime_create") {
auto root = createRuntime(wide(env, args[0]), wide(env, args[1]));
auto result = identity(env, root->directory.get(), root->path);
set(env, result, "path", string(env, root->path));
set(env, result, "privateDacl", boolean(env, true));
set(env, result, "handle", wrap(env, std::move(root)));
return result;
createRuntime(wide(env, args[0]), wide(env, args[1]), context.pending);
try {
auto result = identity(env, context.pending->directory.get(), context.pending->path);
set(env, result, "path", string(env, context.pending->path));
set(env, result, "privateDacl", boolean(env, true));
publishRuntimeHandle(env, result, context.pending);
return result;
} catch (Failure& primary) {
cleanupFailedRuntime(context.pending, primary);
throw;
} catch (const std::exception& error) {
Failure primary(error.what(), context.pending ? context.pending->path : L"", ERROR_GEN_FAILURE);
cleanupFailedRuntime(context.pending, primary);
throw primary;
}
}
auto& root = static_cast<Runtime&>(resource(env, args[0], Kind::Runtime));
require(!root.closing || operation == L"runtime_close", "Runtime cleanup is pending", root.path, ERROR_BUSY);
+16 -6
View File
@@ -454,17 +454,27 @@ napi_value processCall(napi_env env, const std::wstring& operation, const std::v
throw Failure("Unknown native process operation", L"", ERROR_INVALID_PARAMETER);
}
napi_value inspect(napi_env env, std::unique_ptr<Runtime>& probe) {
napi_value inspect(napi_env env, std::unique_ptr<Runtime>& pending) {
// Settle the previous owner before admitting another probe. Failed cleanup
// retains its handles, including when identity construction itself failed.
if (probe) { closeRuntime(*probe); probe.reset(); }
if (pending) { closeRuntime(*pending); pending.reset(); }
wchar_t temp[32768];
const auto length = GetTempPathW(32768, temp);
require(length > 0 && length < 32768, "Read temporary directory coordinate failed");
probe = createRuntime(std::wstring(temp, length), L"rea-native-probe-");
auto filesystem = identity(env, probe->directory.get(), probe->path);
closeRuntime(*probe);
probe.reset();
createRuntime(std::wstring(temp, length), L"rea-native-probe-", pending);
napi_value filesystem;
try {
filesystem = identity(env, pending->directory.get(), pending->path);
} catch (Failure& primary) {
cleanupFailedRuntime(pending, primary);
throw;
} catch (const std::exception& error) {
Failure primary(error.what(), pending ? pending->path : L"", ERROR_GEN_FAILURE);
cleanupFailedRuntime(pending, primary);
throw primary;
}
closeRuntime(*pending);
pending.reset();
auto job = createJob();
Attributes attributes(1);
HANDLE raw = job.get();
+19
View File
@@ -34,6 +34,7 @@
"isomorphic-git": "1.38.7",
"jsonc-parser": "3.3.1",
"lossless-json": "4.3.1",
"openpgp": "6.3.2",
"parse5": "8.0.1",
"pino": "10.4.0",
"playwright-core": "1.58.2",
@@ -7871,6 +7872,24 @@
"regex-recursion": "^6.0.2"
}
},
"node_modules/openpgp": {
"version": "6.3.2",
"resolved": "https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz",
"integrity": "sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==",
"license": "LGPL-3.0+",
"engines": {
"node": ">= 18.0.0",
"typescript": ">= 5.0.0"
},
"peerDependencies": {
"@openpgp/web-stream-tools": "~0.3.0"
},
"peerDependenciesMeta": {
"@openpgp/web-stream-tools": {
"optional": true
}
}
},
"node_modules/oxc-parser": {
"version": "0.150.0",
"resolved": "https://registry.npmjs.org/oxc-parser/-/oxc-parser-0.150.0.tgz",
+1
View File
@@ -337,6 +337,7 @@
"isomorphic-git": "1.38.7",
"jsonc-parser": "3.3.1",
"lossless-json": "4.3.1",
"openpgp": "6.3.2",
"parse5": "8.0.1",
"pino": "10.4.0",
"playwright-core": "1.58.2",
+1
View File
@@ -138,6 +138,7 @@ export const SCOPES = {
"src/domain/javascript/javascriptRuntimeStaticCandidates.ts",
"tests/boundary/filesystem/webScriptManifestPaths.test.ts",
"tests/boundary/process/ProcessEvidence.host.test.ts",
"tests/boundary/process/windowsOutputCapture.test.ts",
"tests/acceptance/webModuleCli.test.ts",
"tests/acceptance/webSourceLocationCli.test.ts",
"tests/boundary/cli/referenceSourceImportStatus.test.ts",
+51 -1
View File
@@ -75,7 +75,57 @@ for (const architecture of ["x86_64", "x86"]) {
: [0xe8, 0x0b, 0, 0, 0, 0xe8, 0x16, 0, 0, 0, 0x31, 0xc0, 0xc3],
0x200,
);
bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
bytes.set(
x86 && !dll
? [0xb8, 0x2a, 0, 0, 0, 0xe8, 0x66, 0, 0, 0, 0xc3]
: [0xb8, 0x2a, 0, 0, 0, 0xc3],
0x210,
);
if (x86 && !dll) {
// A statically analyzed Borland-style SEH registration whose handler
// entry is referenced but is not part of the registering function body.
bytes.set(
[
0x55, // PUSH EBP
0x89,
0xe5, // MOV EBP, ESP
0x68,
0xc0,
0x10,
0x40,
0x00, // PUSH 0x4010c0
0x64,
0xff,
0x35,
0,
0,
0,
0, // PUSH DWORD PTR FS:[0]
0x64,
0x89,
0x25,
0,
0,
0,
0, // MOV DWORD PTR FS:[0], ESP
0x64,
0x8f,
0x05,
0,
0,
0,
0, // POP DWORD PTR FS:[0]
0x83,
0xc4,
0x04, // ADD ESP, 4
0x5d, // POP EBP
0xc3, // RET
],
0x280,
);
bytes.set([0xc3], 0x2c0);
}
if (!dll) {
const dataSection = sectionTable + 40;
+12 -10
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Inspect or activate Hopper's version-pinned demo dialog on a private X display."""
"""Inspect or activate Hopper's authenticated demo dialog on a private X display."""
from __future__ import annotations
@@ -8,6 +8,7 @@ import ctypes
import hashlib
import json
import os
import re
from pathlib import Path
import secrets
import select
@@ -18,12 +19,6 @@ import threading
import time
# Keep this digest aligned with LinuxHopper.ts and rerun real-Hopper verification
# before accepting another vendor build.
SUPPORTED_HOPPER_SHA256 = {
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
}
EXPECTED_SCREEN = (1280, 1024)
DIALOG_TITLES = {"Registration", "Registration — Hopper Disassembler"}
HOPPER_WINDOW_TITLE = "Hopper Disassembler"
@@ -121,6 +116,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument("--probe", action="store_true")
parser.add_argument("--owner-stdin", action="store_true")
parser.add_argument("--hopper")
parser.add_argument("--expected-launcher-sha256")
parser.add_argument("--socket")
parser.add_argument(
"--strategy",
@@ -661,7 +657,11 @@ def run_probe(args: argparse.Namespace) -> dict[str, object]:
def run_launch(args: argparse.Namespace) -> int:
if args.hopper is None or args.socket is None:
if (
args.hopper is None or args.socket is None
or args.expected_launcher_sha256 is None
or re.fullmatch(r"[a-f0-9]{64}", args.expected_launcher_sha256) is None
):
raise AdapterFailure(
INVALID_LAUNCH_COMMAND,
"invalid_launch_command",
@@ -673,7 +673,9 @@ def run_launch(args: argparse.Namespace) -> int:
hopper = Path(args.hopper)
socket = Path(args.socket)
try:
supported = digest(hopper) in SUPPORTED_HOPPER_SHA256
# The parent owns publisher/package interpretation. Recheck its admitted
# launcher bytes immediately before execution, preserving all X11 guards.
supported = digest(hopper) == args.expected_launcher_sha256
except OSError as error:
raise AdapterFailure(
INVALID_LAUNCH_COMMAND,
@@ -684,7 +686,7 @@ def run_launch(args: argparse.Namespace) -> int:
raise AdapterFailure(
UNSUPPORTED_HOPPER_BUILD,
"unsupported_hopper_build",
"unsupported Hopper binary",
"Hopper binary differs from the authenticated release",
)
command = args.command[1:] if args.command[:1] == ["--"] else args.command
if not command or Path(command[0]).resolve() != hopper.resolve():
@@ -144,14 +144,19 @@ export async function verifyScenarioServiceWorkers(endpoint, targetId) {
true,
);
};
const assertRejected = async (scenario) => {
const assertRejected = async (scenario, expectedReason) => {
const result = await provider.captureScenario(scenario);
assert.equal(
result.ok,
false,
"Worker-influenced capture unexpectedly succeeded",
);
assert.equal(result.error.reason, "target_not_allowed");
assert.equal(result.error.reason, expectedReason);
if (expectedReason === "service_worker_blocked")
assert.match(
result.error.userMessage,
/environment\.service_workers to allow/,
);
assert.equal(
result.error.partialObservation.capture.browser.cleanup,
"disconnected-external",
@@ -229,7 +234,7 @@ export async function verifyScenarioServiceWorkers(endpoint, targetId) {
"worker",
);
await page.goto("about:blank");
await assertRejected(scenarioFor(second.origin));
await assertRejected(scenarioFor(second.origin), "service_worker_blocked");
await page.goto("about:blank");
await assertRejected(
scenarioFor(first.origin, [
@@ -240,6 +245,7 @@ export async function verifyScenarioServiceWorkers(endpoint, targetId) {
wait_until: "load",
},
]),
"service_worker_blocked",
);
await page.goto("about:blank");
await assertRejected(
@@ -251,9 +257,13 @@ export async function verifyScenarioServiceWorkers(endpoint, targetId) {
},
{ step_id: "loaded", action: "wait_for_timeout", duration_ms: 100 },
]),
"service_worker_blocked",
);
await page.goto("about:blank");
await assertRejected(scenarioFor(`${first.origin}/frame`));
await assertRejected(
scenarioFor(`${first.origin}/frame`),
"service_worker_blocked",
);
const allowed = await provider.captureScenario(
scenarioFor(second.origin, [], "allow"),
@@ -0,0 +1,33 @@
export interface GhidraVerifierTimeouts {
readonly clientOptions: {
readonly startupTimeoutMs: number;
};
readonly requestOptions: {
readonly timeout: number;
};
}
/** Resolve one verifier-owned timeout policy before target I/O or allocation. */
export function resolveGhidraVerifierTimeouts(
environment: Readonly<Record<string, string | undefined>>,
): GhidraVerifierTimeouts;
export interface GhidraVerifierMcpClient {
connect(
transport: unknown,
options: GhidraVerifierTimeouts["requestOptions"],
): Promise<void>;
callTool(
request: unknown,
options: GhidraVerifierTimeouts["requestOptions"],
): Promise<unknown>;
}
/** Bind every verifier MCP operation to the same owned request budget. */
export function createGhidraVerifierMcpOperations(
client: GhidraVerifierMcpClient,
requestOptions: GhidraVerifierTimeouts["requestOptions"],
): {
connect(transport: unknown): Promise<void>;
callTool(request: unknown): Promise<unknown>;
};
+31
View File
@@ -0,0 +1,31 @@
import { parseConfig } from "../../dist/config/parseConfig.js";
const FILE_OFFSET_OPERATION_TIMEOUT_MS = 240_000;
const MAX_TIMER_DELAY_MS = 2_147_483_647;
/** Resolve one verifier-owned timeout policy before target I/O or allocation. */
export function resolveGhidraVerifierTimeouts(environment) {
const configuration = parseConfig(environment);
if (!configuration.ok) throw configuration.error;
const startupTimeoutMs = configuration.value.ghidraStartupTimeoutMs;
// A cold public query includes provider startup before its analysis request.
// Preserve the operation budget after the caller-selected startup deadline.
const publicRequestTimeoutMs =
startupTimeoutMs + FILE_OFFSET_OPERATION_TIMEOUT_MS;
if (publicRequestTimeoutMs > MAX_TIMER_DELAY_MS)
throw new RangeError(
"The Ghidra startup and verification operation budgets exceed the Node.js timer limit.",
);
return {
clientOptions: { startupTimeoutMs },
requestOptions: { timeout: publicRequestTimeoutMs },
};
}
/** Bind every verifier MCP operation to the same owned request budget. */
export function createGhidraVerifierMcpOperations(client, requestOptions) {
return {
connect: (transport) => client.connect(transport, requestOptions),
callTool: (request) => client.callTool(request, requestOptions),
};
}
+84
View File
@@ -0,0 +1,84 @@
import assert from "node:assert/strict";
const limitationPattern = /Borland\/Delphi-style SEH frame registration/u;
/** Verify Ghidra reports, and does not absorb, the fixture's handler region. */
export async function verifyGhidraBorlandSeh(functionObservation, x86) {
assert.equal(x86, true, "The SEH control exists only in the x86 PE fixture");
const caller = "0x401015";
const procedure = "0x401080";
const handler = "0x4010c0";
const ordinary = await functionObservation("procedure_info", {
procedure: "0x401010",
});
assert.ok(
!ordinary.limitations.some((item) => limitationPattern.test(item)),
"The ordinary return-42 control unexpectedly reports an SEH limitation",
);
const info = await functionObservation("procedure_info", { procedure });
assert.ok(info.limitations.some((item) => limitationPattern.test(item)));
const outgoing = await functionObservation("procedure_references", {
procedure,
direction: "outgoing",
});
assert.equal(outgoing.value.direction, "outgoing");
assert.ok(outgoing.limitations.some((item) => limitationPattern.test(item)));
assert.ok(
outgoing.value.references.some(
(reference) =>
reference.source_address === "0x401083" &&
reference.target_address === handler &&
reference.kind.available &&
reference.kind.data &&
!reference.kind.call,
),
"The pushed handler address must remain an observed data reference, without inventing a call",
);
const incoming = await functionObservation("procedure_references", {
procedure,
direction: "incoming",
});
assert.equal(incoming.value.direction, "incoming");
assert.ok(incoming.limitations.some((item) => limitationPattern.test(item)));
assert.ok(
incoming.value.references.some(
(reference) =>
reference.source_address === caller &&
reference.target_address === procedure,
),
"The caller-to-registration procedure reference was not observed",
);
const dossier = await functionObservation("analyze_function", {
procedure,
});
assert.ok(dossier.limitations.some((item) => limitationPattern.test(item)));
assert.ok(
dossier.value.limitations.some((item) => limitationPattern.test(item)),
"The normalized function dossier lost the SEH limitation",
);
assert.equal(dossier.value.procedure.body.available, true);
assert.ok(dossier.value.procedure.body.ranges.length > 0);
assert.ok(
!dossier.value.procedure.body.ranges.some(
(range) =>
BigInt(range.start) <= BigInt(handler) &&
BigInt(range.end) >= BigInt(handler),
),
"The handler was incorrectly claimed as part of the procedure body",
);
return {
limitation: true,
ordinary_control: true,
procedure_references_incoming_and_outgoing: true,
handler_data_reference: true,
caller_reference: true,
dossier_limitation: true,
handler_body_not_claimed: true,
};
}
@@ -0,0 +1,16 @@
export interface GhidraFileOffsetVerification {
readonly cli_mcp_parity: true;
readonly explicit_and_numeric_offsets: true;
readonly invalid_and_unmapped_offsets: true;
readonly mapped_offset_without_procedure: true;
readonly ambiguous_mappings: "verified" | "unavailable-single-section";
readonly source_unchanged: true;
readonly cleanup: "complete";
}
/** Exercise file coordinates through public CLI/MCP using a source-owned PE fixture. */
export function verifyGhidraFileOffsets(
targetPath: string,
entrypoint: string,
procedure: string,
): Promise<GhidraFileOffsetVerification>;
+24 -20
View File
@@ -14,6 +14,10 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseEvidence } from "../../dist/domain/evidence.js";
import {
createGhidraVerifierMcpOperations,
resolveGhidraVerifierTimeouts,
} from "./ghidra-verifier-timeouts.mjs";
import {
requireMcpOperationResult,
requireMcpToolError,
@@ -26,17 +30,22 @@ export async function verifyGhidraFileOffsets(
entrypoint,
procedure,
) {
const environment = {
...process.env,
REA_ANALYSIS_PROVIDER: "ghidra",
REA_LOG_LEVEL: "silent",
};
const { requestOptions } = resolveGhidraVerifierTimeouts(environment);
const timeoutMs = requestOptions.timeout;
const original = await readFile(targetPath);
const workspace = await mkdtemp(join(tmpdir(), "rea-file-offsets-"));
const runtime = join(workspace, "runtime");
await mkdir(runtime);
const env = {
...process.env,
...environment,
TEMP: runtime,
TMP: runtime,
TMPDIR: runtime,
REA_ANALYSIS_PROVIDER: "ghidra",
REA_LOG_LEVEL: "silent",
};
const client = new Client({ name: "rea-file-offsets", version: "1" });
const transport = new StdioClientTransport({
@@ -45,21 +54,19 @@ export async function verifyGhidraFileOffsets(
env,
stderr: "pipe",
});
const mcp = createGhidraVerifierMcpOperations(client, requestOptions);
let connected = false;
let complete = false;
let primaryFailure;
const call = async (name, arguments_ = {}) => {
const result = await client.callTool(
{ name, arguments: arguments_ },
{ timeout: 240_000 },
);
const result = await mcp.callTool({ name, arguments: arguments_ });
const projected = requireMcpOperationResult(result, name);
return ["open_binary", "binary_session", "close_binary"].includes(name)
? projected
: (result.structuredContent ?? JSON.parse(mcpTextValue(result)));
};
try {
await client.connect(transport);
await mcp.connect(transport);
connected = true;
await call("open_binary", { path: targetPath, provider_id: "ghidra" });
const session = await call("binary_session");
@@ -90,12 +97,12 @@ export async function verifyGhidraFileOffsets(
),
);
}
const invalid = await client.callTool({
const invalid = await mcp.callTool({
name: "procedure_pseudo_code",
arguments: { procedure: "file-offset:0x20000000000000" },
});
assert.equal(requireMcpToolError(invalid).code, "invalid_request");
const unmapped = await client.callTool({
const unmapped = await mcp.callTool({
name: "procedure_pseudo_code",
arguments: { procedure: "file-offset:0x100000" },
});
@@ -122,7 +129,7 @@ export async function verifyGhidraFileOffsets(
assert.equal(headerOffset.normalized_result.file_offset, 0);
for (const selector of ["file-offset:0x0", "0x0"]) {
const missing = requireMcpToolError(
await client.callTool({
await mcp.callTool({
name: "procedure_pseudo_code",
arguments: { procedure: selector },
}),
@@ -147,7 +154,7 @@ export async function verifyGhidraFileOffsets(
"ghidra",
"--json",
],
{ env, timeout: 240_000, maxBuffer: 16 * 1024 * 1024 },
{ env, timeout: timeoutMs, maxBuffer: 16 * 1024 * 1024 },
)
).stdout,
);
@@ -164,7 +171,7 @@ export async function verifyGhidraFileOffsets(
"ghidra",
"--json",
],
{ env, timeout: 240_000, maxBuffer: 16 * 1024 * 1024 },
{ env, timeout: timeoutMs, maxBuffer: 16 * 1024 * 1024 },
).then(
() => {
throw new Error(
@@ -183,13 +190,10 @@ export async function verifyGhidraFileOffsets(
const ambiguousPath = join(workspace, "ambiguous.exe");
await writeFile(ambiguousPath, alias);
await call("open_binary", { path: ambiguousPath, provider_id: "ghidra" });
const ambiguous = await client.callTool(
{
name: "procedure_pseudo_code",
arguments: { procedure: `file-offset:${coordinate}` },
},
{ timeout: 240_000 },
);
const ambiguous = await mcp.callTool({
name: "procedure_pseudo_code",
arguments: { procedure: `file-offset:${coordinate}` },
});
assert.ok(
JSON.stringify(requireMcpToolError(ambiguous)).includes(
"multiple loaded mappings",
@@ -0,0 +1,118 @@
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { once } from "node:events";
import { createInterface } from "node:readline";
import { resolve } from "node:path";
const withDeadline = async (promise, milliseconds, description) => {
let timer;
try {
return await Promise.race([
promise,
new Promise((_, reject) => {
timer = setTimeout(
() =>
reject(new Error(`${description} exceeded ${milliseconds} ms.`)),
milliseconds,
);
}),
]);
} finally {
clearTimeout(timer);
}
};
export async function verifyVolumeRootAppendLease({
fixturePath,
volumeRoot,
verifyAdmission,
}) {
const lease = spawn(
resolve(fixturePath),
["volume-root-append-lease", volumeRoot],
{ stdio: ["pipe", "pipe", "pipe"], windowsHide: true },
);
const lines = createInterface({ input: lease.stdout });
let leaseStderr = "";
lease.stderr.setEncoding("utf8");
lease.stderr.on("data", (chunk) => {
if (leaseStderr.length < 8192) leaseStderr += chunk;
});
const closed = once(lease, "close").then(([code, signal]) => ({
code,
signal,
}));
const leaseLine = once(lines, "line").then(([line]) => ({ line }));
const leaseError = once(lease, "error").then(([error]) => {
throw error;
});
let verificationFailure;
try {
const ready = await withDeadline(
Promise.race([leaseLine, closed, leaseError]),
10_000,
"Volume-root append lease readiness",
);
assert.equal(
"line" in ready,
true,
`Volume-root append lease exited before readiness (code ${ready.code}, signal ${ready.signal}): ${leaseStderr}`,
);
const observation = JSON.parse(ready.line);
assert.equal(
observation.writeAccessGranted,
true,
`Windows test token cannot hold FILE_ADD_SUBDIRECTORY on ${volumeRoot} (Win32 ${observation.win32Code}).`,
);
await verifyAdmission();
} catch (error) {
verificationFailure = error;
}
lines.close();
if (
lease.exitCode === null &&
lease.signalCode === null &&
!lease.stdin.destroyed
)
lease.stdin.end("x");
let releaseFailure;
try {
const result = await withDeadline(
closed,
10_000,
"Volume-root append lease release",
).catch(async (error) => {
lease.kill();
try {
return await withDeadline(
closed,
5_000,
"Volume-root append lease termination",
);
} catch (terminationError) {
throw new AggregateError(
[error, terminationError],
`Volume-root append lease could not be released: ${leaseStderr}`,
);
}
});
assert.equal(
result.code,
0,
`Volume-root append lease exited with code ${result.code}, signal ${result.signal}: ${leaseStderr}`,
);
} catch (error) {
releaseFailure = error;
}
if (verificationFailure !== undefined && releaseFailure !== undefined)
throw new AggregateError(
[verificationFailure, releaseFailure],
"Volume-root append lease verification and cleanup both failed.",
);
if (verificationFailure !== undefined) throw verificationFailure;
if (releaseFailure !== undefined) throw releaseFailure;
}
+36 -5
View File
@@ -14,6 +14,10 @@ process.env.REA_PACKAGE_VERSION = packageJson.version;
const isMcpMode =
args.length === 1 && (args[0] === "--mcp" || args[0] === "mcp");
const isMcpDoctorMode = args[0] === "mcp" && args[1] === "doctor";
/** Exit status of a writer terminated by SIGPIPE (128 + 13). */
const SIGPIPE_EXIT_STATUS = 141;
// Set once the reader of CLI output has closed the pipe.
let stdoutClosed = false;
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Windows home lookup throws for an empty USERPROFILE. Dependencies resolve
@@ -55,13 +59,14 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
const { runEntrypoint } = await import("../dist/mcp/main.js");
await runEntrypoint();
} else if (isMcpDoctorMode) {
guardClosedStdout();
const { runProductionMcpDoctorCli } =
await import("../dist/cli-runtime/mcpDoctor.js");
const result = await runProductionMcpDoctorCli(args.slice(2), {
dispatcherPath: fileURLToPath(import.meta.url),
packageRoot,
});
process.stdout.write(result.output);
writeStdout(result.output);
process.exitCode = result.exitCode;
} else {
const { cliConfigurationError, createCli } =
@@ -73,13 +78,14 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
renderEmptyFilteredCliOutput,
validateCliOutputArguments,
} = await import("../dist/cli-runtime/cliOutput.js");
guardClosedStdout();
const outputArguments = validateCliOutputArguments(args);
const configurationError = cliConfigurationError(process.env);
if (configurationError !== undefined) {
process.stderr.write(`${configurationError}\n`);
process.exitCode = 1;
} else if (!outputArguments.ok) {
process.stdout.write(renderCliOutputArgumentError(outputArguments));
writeStdout(renderCliOutputArgumentError(outputArguments));
process.exitCode = 1;
} else {
let wroteOutput = false;
@@ -87,18 +93,43 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
await createCli(process.env, resultOutput).serve(args, {
stdout: (output) => {
if (resultOutput?.handled) {
if (resultOutput.failed) process.stderr.write(output);
if (resultOutput.failed && !stdoutClosed)
process.stderr.write(output);
return;
}
if (output.length > 0) wroteOutput = true;
process.stdout.write(output);
writeStdout(output);
},
// A command failure caused by the closed pipe keeps the pipe's status.
exit: (code) => process.exit(stdoutClosed ? SIGPIPE_EXIT_STATUS : code),
});
if (!wroteOutput && !resultOutput?.handled)
process.stdout.write(renderEmptyFilteredCliOutput(args) ?? "");
writeStdout(renderEmptyFilteredCliOutput(args) ?? "");
}
}
/**
* A reader that closes the pipe early, as `rea ... | head` does, has declined
* the rest of the output. Stop writing without reporting a failure, and exit
* with the status a SIGPIPE-terminated writer reports. MCP stdio keeps its own
* client lifecycle.
*/
function guardClosedStdout() {
process.stdout.on("error", (cause) => {
if (cause?.code === "EPIPE") {
stdoutClosed = true;
process.exitCode = SIGPIPE_EXIT_STATUS;
return;
}
// Other failures stay unhandled unless a pending JSON write reports them.
if (process.stdout.listenerCount("error") === 1) throw cause;
});
}
function writeStdout(output) {
if (!stdoutClosed) process.stdout.write(output);
}
async function compiledRuntimeExists(paths) {
for (const path of paths) {
try {
+15 -6
View File
@@ -13,6 +13,7 @@ import { promisify } from "node:util";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { assertCleanLinuxHopperRuntime } from "./lib/clean-linux-hopper-runtime.mjs";
import { isUnsupportedHopperLaunchFailure } from "./verify-package-platform.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
@@ -89,16 +90,24 @@ try {
await appendFile(modifiedHopper, new Uint8Array([0]));
await chmod(modifiedHopper, 0o700);
try {
await execFileAsync(command, ["analyze", "/usr/bin/true"], {
env: { ...process.env, HOPPER_LAUNCHER_PATH: modifiedHopper },
timeout: 30_000,
});
await execFileAsync(
command,
["analyze", "/usr/bin/true", "--provider", "hopper", "--json"],
{
env: { ...process.env, HOPPER_LAUNCHER_PATH: modifiedHopper },
timeout: 30_000,
},
);
throw new Error("modified Hopper build was not rejected");
} catch (error) {
const output = `${error?.stdout ?? ""}\n${error?.stderr ?? ""}`;
if (!output.includes("unsupported_hopper_build"))
if (
error?.code !== 1 ||
typeof error.stdout !== "string" ||
!isUnsupportedHopperLaunchFailure(JSON.parse(error.stdout))
)
throw new Error(
"modified Hopper build did not return its stable failure code",
{ cause: error },
);
}
} finally {
+9
View File
@@ -1,5 +1,6 @@
import { chmod, mkdir, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { supportedClients } from "../dist/application/SupportedClients.js";
/** Create an isolated prefix, fake PATH, client configs, and environment. */
@@ -121,6 +122,14 @@ export const packageHopperEnvironment = (root, platform = process.platform) => {
};
return {
HOPPER_LAUNCHER_PATH: "/bin/sh",
// Feed availability must not turn this foreign-launcher rejection test into
// an unrelated networking failure. Successful vendor verification is real E2E coverage.
NODE_OPTIONS: [
process.env.NODE_OPTIONS,
`--import=${pathToFileURL(join(root, "tests/fixtures/hopperRejectionFeed.mjs")).href}`,
]
.filter(Boolean)
.join(" "),
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
"-c",
'node_path=$1; shift; "$node_path" "$@"',
+20 -5
View File
@@ -88,14 +88,29 @@ export async function verifyPackagePlatform({
}
}
/** Require the declared unsupported build or an exact missing-Xvfb diagnostic, never arbitrary provider failure. */
/** One owner for unsupported-launch evidence across package and real-Hopper verification. */
export const isUnsupportedHopperLaunchFailure = (failure) => {
const details = failure?.details;
const unsupportedBuild =
details?.failure_code === "unsupported_hopper_build" &&
details.exit_code === 72;
const publisherRejection =
details?.stage === "launch" &&
details.launch_rejection?.boundary === "publisher_verification" &&
details.launch_rejection.reason === "unsupported_hopper_build" &&
details.exit_code === undefined &&
details.launcher === undefined;
return (
failure?.code === "provider_unavailable" &&
(publisherRejection || unsupportedBuild)
);
};
/** Require an exact pre-execution rejection, observed helper exit, or missing-Xvfb diagnostic. */
export const assertLinuxPackageProviderFailure = (execution) => {
const failure = json(execution.stdout);
const details = failure?.details;
const diagnostic = details?.diagnostics;
const unsupportedBuild =
details?.failure_code === "unsupported_hopper_build" &&
details.exit_code === 72;
const missingDisplayDependency =
details?.failure_code === "runtime_dependency_unavailable" &&
details.exit_code === 79 &&
@@ -108,7 +123,7 @@ export const assertLinuxPackageProviderFailure = (execution) => {
if (
execution.status !== 1 ||
failure?.code !== "provider_unavailable" ||
(!unsupportedBuild && !missingDisplayDependency)
(!isUnsupportedHopperLaunchFailure(failure) && !missingDisplayDependency)
)
throw new Error(
`packaged Linux Hopper verification did not fail closed: ${JSON.stringify(failure)}`,
+15 -1
View File
@@ -27,7 +27,9 @@ import {
} from "../dist/ghidra/GhidraSessionValues.js";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { resolveGhidraVerifierTimeouts } from "./lib/ghidra-verifier-timeouts.mjs";
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
import { verifyGhidraBorlandSeh } from "./lib/real-ghidra-borland-seh.mjs";
import { verifyGhidraFileOffsets } from "./lib/real-ghidra-file-offsets.mjs";
import { verifyWindowsGhidraByteSearch } from "./lib/ghidra-windows-byte-search.mjs";
@@ -47,6 +49,7 @@ if (
)
throw new Error("Windows Ghidra must not advertise database mutation");
const installDir = process.env.GHIDRA_INSTALL_DIR;
const { clientOptions } = resolveGhidraVerifierTimeouts(process.env);
if (installDir === undefined || !isAbsolute(installDir))
throw new Error(
"Set GHIDRA_INSTALL_DIR to the absolute root of Ghidra 12.1.4.",
@@ -116,6 +119,7 @@ const client = new GhidraClient({
transport: "authenticated-loopback-tcp",
providerVersion: SUPPORTED_GHIDRA_VERSION,
profileDigest: profile.value.profile.digest,
...clientOptions,
});
let coordinates;
@@ -270,6 +274,10 @@ try {
functionOperation,
architecture === "x86",
);
const borlandSeh =
architecture === "x86" && !dllFixture
? await verifyGhidraBorlandSeh(functionObservationOperation, true)
: null;
report = {
ok: true,
provider: { id: "ghidra", version: SUPPORTED_GHIDRA_VERSION },
@@ -286,6 +294,7 @@ try {
byte_search: byteSearch,
cleanup: "complete",
compact_references: compactReferences,
borland_seh: borlandSeh,
native_controls: {
job_object: true,
protected_dacl: true,
@@ -335,6 +344,11 @@ async function inventory(operation, parameters) {
}
async function functionOperation(operation, parameters) {
const observation = await functionObservationOperation(operation, parameters);
return observation.value;
}
async function functionObservationOperation(operation, parameters) {
const input = parseGhidraFunctionInput(operation, parameters);
if (!input.ok) throw input.error;
const called = await client.callTool(operation, input.value);
@@ -342,7 +356,7 @@ async function functionOperation(operation, parameters) {
const result = parseGhidraFunctionResult(operation, called.value);
if (!result.ok) throw result.error;
observed.add(operation);
return result.value.value;
return result.value;
}
function assertSession(session, targetSha256, profileDigest) {
+1 -1
View File
@@ -23,7 +23,7 @@ import {
requireMcpOperationResult,
} from "./lib/mcp-verifier-results.mjs";
import { snapshotHopperRuntime } from "./lib/real-hopper-cleanup.mjs";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/hopper/HopperTargetLease.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { parseConfig } from "../dist/config/parseConfig.js";
import {
verifyHopperSourceBinding,
+1 -1
View File
@@ -12,7 +12,7 @@ import {
requirePseudocode,
} from "../dist/application/RealHopperAssertions.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/hopper/HopperTargetLease.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { REA_WORKFLOW_PROVIDER } from "../dist/application/InvestigationProviders.js";
import { loadRealHopperFixtureTargets } from "./lib/real-hopper-fixture.mjs";
import { verifyHopperStringObjects } from "./lib/real-hopper-search.mjs";
+38 -9
View File
@@ -55,14 +55,29 @@ try {
join(workspace, "dist", "windows", "WindowsNativeLoader.js"),
).href;
const source = `import assert from 'node:assert/strict';
import {createRequire} from 'node:module';
import {existsSync,mkdirSync,readFileSync,readdirSync,renameSync,statSync,writeFileSync} from 'node:fs';
import {join} from 'node:path';
import {systemWindowsNativeAuthority} from ${JSON.stringify(loader)};
const temp=process.env.TEMP;
const first=systemWindowsNativeAuthority();
assert.equal(first.available,false);
assert.equal(first.retryable,true);
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
const mode=process.argv[2];
const creation=mode.startsWith('create-');
const native=creation?createRequire(import.meta.url)(${JSON.stringify(join(workspace, "native", "windows", "build", "rea-windows-x64.node"))}):null;
if(creation){
let failure;
try{native.call('runtime_create',[temp,'rea-native-created-'])}catch(error){failure=error}
assert.ok(failure instanceof Error,'runtime_create unexpectedly passed injected post-create validation failure');
assert.equal(failure.constraint,'Read private DACL by handle failed');
assert.equal(failure.win32Code,5);
assert.equal(failure.cleanupConstraint,'Delete owned object by handle failed');
assert.equal(failure.cleanupWin32Code,5);
assert.notEqual(failure.runtimeAllocation,'not-started');
} else {
const first=systemWindowsNativeAuthority();
assert.equal(first.available,false);
assert.equal(first.retryable,true);
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
}
const [name,...extra]=readdirSync(temp);assert.ok(name);assert.equal(extra.length,0);
const path=join(temp,name);const identity=statSync(path,{bigint:true});
// Delete sharing permits rename; ownership follows the retained NTFS handle.
@@ -73,26 +88,40 @@ assert.equal(moved.dev,identity.dev);assert.equal(moved.ino,identity.ino);
mkdirSync(path);const replacement=statSync(path,{bigint:true});
assert.notEqual(replacement.ino,identity.ino);
const sentinel=join(path,'unrelated.txt');writeFileSync(sentinel,'unrelated replacement');
if(process.argv[2]!=='teardown'){
if(mode==='retry'){
const second=systemWindowsNativeAuthority();assert.equal(second.available,true);
assert.equal(existsSync(renamed),false);
assert.deepEqual(readdirSync(temp),[name]);
assert.equal(readFileSync(sentinel,'utf8'),'unrelated replacement');
assert.equal(statSync(path,{bigint:true}).ino,replacement.ino);
assert.equal(systemWindowsNativeAuthority(),second);
} else if(mode==='create-retry') {
const second=native.call('runtime_create',[temp,'rea-native-created-retry-']);
assert.equal(existsSync(renamed),false);
assert.equal(readFileSync(sentinel,'utf8'),'unrelated replacement');
native.call('runtime_close',[second.handle]);
assert.equal(existsSync(second.path),false);
assert.equal(statSync(path,{bigint:true}).ino,replacement.ino);
}
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,name,renamed,device:identity.dev.toString(),inode:identity.ino.toString(),replacementInode:replacement.ino.toString()}));
console.log(JSON.stringify({mode,firstFailed:true,retainedOwner:true,name,renamed,device:identity.dev.toString(),inode:identity.ino.toString(),replacementInode:replacement.ino.toString()}));
`;
const fixture = join(workspace, "probe.mjs");
await writeFile(fixture, source);
for (const mode of ["retry", "teardown"]) {
for (const mode of ["retry", "teardown", "create-retry", "create-teardown"]) {
const temp = join(workspace, mode);
await mkdir(temp);
const child = spawnSync(process.execPath, [fixture, mode], {
encoding: "utf8",
timeout: 20_000,
windowsHide: true,
env: { ...process.env, TEMP: temp, TMP: temp },
env: {
...process.env,
TEMP: temp,
TMP: temp,
...(mode.startsWith("create-")
? { REA_NATIVE_CREATION_FAILURE: "1" }
: {}),
},
});
if (child.error !== undefined) throw child.error;
assert.equal(child.status, 0, child.stderr);
@@ -119,7 +148,7 @@ console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:
);
}
process.stdout.write(
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleansChildren: true, unrelatedReplacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
`${JSON.stringify({ ok: true, inspectionPostCreateValidationFailure: true, runtimeCreatePostCreateValidationFailure: true, controlledRollbackDispositionFailure: true, retainedOwner: true, renamedOwnerCleansChildren: true, unrelatedReplacementPreserved: true, retryRecoversByInspectionAndCreation: true, successfulAdmissionCached: true, environmentTeardownCleansProbeAndCreation: true, issue1701HostCauseReproduced: false })}\n`,
);
} finally {
await rm(workspace, { recursive: true, force: true });
+70 -11
View File
@@ -22,6 +22,7 @@ import { dirname, join, parse, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
import { verifyWindowsJobMemoryLimit } from "./lib/windows-job-memory-limit.mjs";
import { verifyVolumeRootAppendLease } from "./lib/windows-volume-root-append-lease.mjs";
if (process.platform !== "win32")
throw new Error(
@@ -47,6 +48,13 @@ const { PrivateRuntimeRootUnavailableError } = await import(
)
);
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
// A pwsh caller can supply module paths incompatible with Windows PowerShell.
// Each independent observer must discover its own module path.
const windowsPowerShellEnvironment = Object.fromEntries(
Object.entries(process.env).filter(
([name]) => name.toLowerCase() !== "psmodulepath",
),
);
const environment = Object.entries(process.env)
.filter(([, value]) => value !== undefined)
.map(([key, value]) => `${key}=${value}`);
@@ -251,6 +259,17 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
}
const unrelatedFile = join(workspace, "unrelated-file");
await writeFile(unrelatedFile, "preserve unrelated writes");
if (process.argv[3] !== undefined) {
await verifyVolumeRootAppendLease({
fixturePath: process.argv[3],
volumeRoot: parse(source).root,
verifyAdmission: () => {
const rootGuard = native.call("open", [source]);
native.call("close", [rootGuard.handle]);
},
});
report.controls.volumeRootAdmissionWithAppendHandle = true;
}
const admitted = native.call("open", [source]);
try {
await assert.rejects(
@@ -297,6 +316,7 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
]);
const sentinel = join(sourceDirectory, "sentinel");
await writeFile(sentinel, "preserve");
const runtimeVerificationErrors = [];
try {
const acl = JSON.parse(
execFileSync(
@@ -305,18 +325,13 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
"-NoProfile",
"-NonInteractive",
"-Command",
"$a=Get-Acl -LiteralPath $env:REA_TEST_PRIVATE_RUNTIME_PATH; $u=[Security.Principal.WindowsIdentity]::GetCurrent().User; @{protected=$a.AreAccessRulesProtected; ownerMatchesUser=($a.GetOwner([Security.Principal.SecurityIdentifier]).Value -eq $u.Value); trusteesOnlyUserAndSystem=(@($a.GetAccessRules($true,$true,[Security.Principal.SecurityIdentifier]) | Where-Object {$_.IdentityReference.Value -ne $u.Value -and $_.IdentityReference.Value -ne 'S-1-5-18'}).Count -eq 0); count=@($a.Access).Count} | ConvertTo-Json -Compress",
"$ErrorActionPreference='Stop'; $p=$env:REA_TEST_PRIVATE_RUNTIME_PATH; $a=Get-Acl -LiteralPath $p; $u=[Security.Principal.WindowsIdentity]::GetCurrent().User; [IO.File]::SetAttributes($p,[IO.FileAttributes]([int][IO.File]::GetAttributes($p) -bor 1)); @{protected=$a.AreAccessRulesProtected; ownerMatchesUser=($a.GetOwner([Security.Principal.SecurityIdentifier]).Value -eq $u.Value); trusteesOnlyUserAndSystem=(@($a.GetAccessRules($true,$true,[Security.Principal.SecurityIdentifier]) | Where-Object {$_.IdentityReference.Value -ne $u.Value -and $_.IdentityReference.Value -ne 'S-1-5-18'}).Count -eq 0); count=@($a.Access).Count; rootReadonly=(([int][IO.File]::GetAttributes($p) -band 1) -ne 0)} | ConvertTo-Json -Compress",
],
{
encoding: "utf8",
// A pwsh caller can supply module paths incompatible with the
// Windows PowerShell observer. Let that observer build its own path.
timeout: 15_000,
env: {
...Object.fromEntries(
Object.entries(process.env).filter(
([name]) => name.toLowerCase() !== "psmodulepath",
),
),
...windowsPowerShellEnvironment,
REA_TEST_PRIVATE_RUNTIME_PATH: runtime.path,
},
},
@@ -327,6 +342,7 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
ownerMatchesUser: true,
trusteesOnlyUserAndSystem: true,
count: 2,
rootReadonly: true,
});
native.call("runtime_mkdir", [runtime.handle, "inputs"]);
native.call("runtime_write", [
@@ -472,7 +488,10 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
{
encoding: "utf8",
timeout: 15_000,
env: { ...process.env, REA_SNAPSHOT_READER_PATH: snapshotPath },
env: {
...windowsPowerShellEnvironment,
REA_SNAPSHOT_READER_PATH: snapshotPath,
},
},
).trim();
assert.equal(readerDigest, readbackSnapshot.sha256);
@@ -562,12 +581,52 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
asynchronousSnapshot: true,
snapshotCancellation: true,
};
} finally {
native.call("runtime_close", [runtime.handle]);
const readonlyChildren = JSON.parse(
execFileSync(
"powershell.exe",
[
"-NoProfile",
"-NonInteractive",
"-Command",
"$ErrorActionPreference='Stop'; $items=ConvertFrom-Json -InputObject $env:REA_TEST_READONLY_CHILDREN; @($items | ForEach-Object { $current=[int][IO.File]::GetAttributes($_); $next=if(($current -band 16) -ne 0){$current -bor 1}else{1}; [IO.File]::SetAttributes($_,[IO.FileAttributes]$next); $a=[int][IO.File]::GetAttributes($_); @{readonly=(($a -band 1) -ne 0); directory=(($a -band 16) -ne 0); onlyReadonly=($a -eq 1)} }) | ConvertTo-Json -Compress",
],
{
encoding: "utf8",
timeout: 15_000,
env: {
...windowsPowerShellEnvironment,
REA_TEST_READONLY_CHILDREN: JSON.stringify([
join(runtime.path, "inputs"),
join(runtime.path, "inputs", "target.bin"),
]),
},
},
),
);
assert.deepEqual(readonlyChildren, [
{ readonly: true, directory: true, onlyReadonly: false },
{ readonly: true, directory: false, onlyReadonly: true },
]);
} catch (failure) {
runtimeVerificationErrors.push(failure);
}
try {
native.call("runtime_close", [runtime.handle]);
} catch (cleanupFailure) {
runtimeVerificationErrors.push(cleanupFailure);
}
if (runtimeVerificationErrors.length === 1)
throw runtimeVerificationErrors[0];
if (runtimeVerificationErrors.length > 1)
throw new AggregateError(
runtimeVerificationErrors,
"Native runtime verification and cleanup failed",
);
assert.equal(await exists(runtime.path), false);
assert.equal(await readFile(sentinel, "utf8"), "preserve");
report.controls.cleanupDoesNotTraverseJunctions = true;
report.controls.readonlyRootAndChildrenRemoved = true;
report.controls.readonlyFileWithoutArchiveAttributeRemoved = true;
const fixture = join(workspace, "tree.mjs");
await writeFile(
fixture,
+25 -1
View File
@@ -709,7 +709,7 @@ it("projects package details from dumpsys package output", async () => {
});
expect(result).toMatchObject({
version_name: "1.2.3",
version_code: 45,
version_code: "45",
first_install_time: "2026-01-02 03:04:05",
last_update_time: "2026-02-03 04:05:06",
installer_package_name: "com.android.vending",
@@ -719,6 +719,30 @@ it("projects package details from dumpsys package output", async () => {
});
});
// PackageInfo.getLongVersionCode() combines versionCodeMajor and versionCode
// into a signed 64-bit long, which JSON numbers cannot hold exactly.
it.each(["9007199254740993", "-4294967296"])(
"keeps the long version code %s exactly as dumpsys prints it",
async (versionCode) => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:package": `Packages:
Package [com.example] (abcdef):
versionCode=${versionCode} minSdk=24 targetSdk=34
`,
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_package",
input: { serial: "emulator-5554", package: "com.example" },
});
expect(result).toMatchObject({
version_code: versionCode,
coverage: "complete",
});
},
);
// Android 10 and 11 append ": restricted=true" to hard- and soft-restricted
// entries of the requested-permissions list (Settings.dumpPackageLPr).
it("counts requested permissions that the device marks as restricted", async () => {
+5 -4
View File
@@ -193,7 +193,8 @@ export const parseWindowFocusOutput = (
export interface AdbPackageDetails {
readonly versionName: string | null;
readonly versionCode: number | null;
/** Signed 64-bit long version code exactly as Package Manager prints it. */
readonly versionCode: string | null;
readonly firstInstallTime: string | null;
readonly lastUpdateTime: string | null;
readonly installerPackageName: string | null;
@@ -214,7 +215,7 @@ export const parseDumpsysPackageOutput = (
): AdbPackageDetails => {
const details: {
versionName: string | null;
versionCode: number | null;
versionCode: string | null;
firstInstallTime: string | null;
lastUpdateTime: string | null;
installerPackageName: string | null;
@@ -252,8 +253,8 @@ export const parseDumpsysPackageOutput = (
if (key === "versionName" && details.versionName === null)
details.versionName = value === "" ? null : value;
else if (key === "versionCode" && details.versionCode === null) {
const parsed = /^(\d+)/u.exec(value);
if (parsed !== null) details.versionCode = Number(parsed[1]!);
const parsed = /^(-?\d+)/u.exec(value);
if (parsed !== null) details.versionCode = parsed[1]!;
} else if (key === "firstInstallTime" && details.firstInstallTime === null)
details.firstInstallTime = value === "" ? null : value;
else if (key === "lastUpdateTime" && details.lastUpdateTime === null)
+18 -10
View File
@@ -22,6 +22,7 @@ import type { ToolKind } from "../contracts/toolContractTypes.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { isMachOFormat } from "../domain/binaryTargetTypes.js";
import type { ProviderAvailability } from "./AnalysisProvider.js";
import type { JebAnalysisAvailability } from "./jeb/JebAnalysisPort.js";
import { TOOL_CONTRACTS } from "../contracts/toolContracts.js";
import { FRIDA_TOOL_CONTRACTS } from "../contracts/fridaToolContracts.js";
import {
@@ -56,7 +57,7 @@ export type AvailabilityPolicy = {
readonly androidAnalysisAvailability?: ProviderAvailability;
readonly adbAnalysisAvailability?: ProviderAvailability;
readonly apktoolAnalysisAvailability?: ProviderAvailability;
readonly jebAnalysisAvailability?: ProviderAvailability;
readonly jebAnalysisAvailability?: JebAnalysisAvailability;
readonly javascriptRecoveryEnabled?: boolean;
readonly webModuleResolutionEnabled?: boolean;
readonly browserObservationEnabled?: boolean;
@@ -485,15 +486,22 @@ const workflowAvailabilityFor = ({
policy.apktoolAnalysisAvailability,
"Set REA_APKTOOL_COMMAND to a caller-supplied Apktool executable or make Apktool 2.7.0 or newer available on PATH.",
);
if (kind === "jeb-provider")
return policy.jebAnalysisAvailability?.status === "available"
? { reason: "available", remediation: null }
: {
reason: "provider_missing",
remediation:
policy.jebAnalysisAvailability?.reason ??
"Start a JEB client serving MCP (GUI client, or headless via its programmatic launcher) and point REA_JEB_MCP_URL at it (default http://127.0.0.1:8425/mcp). REA does not install or launch JEB.",
};
if (kind === "jeb-provider") {
const availability = policy.jebAnalysisAvailability;
if (availability?.status === "available")
return availability.supportedOperations.some(
(operation) => operation === name,
)
? { reason: "available", remediation: null }
: {
reason: "provider_unavailable",
remediation: `The running JEB endpoint does not advertise support for ${name}. Use a JEB client build that provides this operation.`,
};
return ownerProbeAvailability(
availability,
"Start a JEB client serving MCP (GUI client, or headless via its programmatic launcher) and point REA_JEB_MCP_URL at it (default http://127.0.0.1:8425/mcp). REA does not install or launch JEB.",
);
}
const browser = browserProviderAvailability(name, kind, policy);
if (browser !== null) return browser;
if (name === "capture_process_scenario" && !policy.processCaptureEnabled)
@@ -13,6 +13,18 @@ import {
import { PRODUCT_IDENTITY } from "../identity.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import { dshDocument } from "./DshCordisPatch.js";
import type { McpToolTimeoutInputs } from "../mcpStartupPolicy.js";
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
export type ProviderRequestBudgetResolver = (
analysisProvider: AnalysisProviderSelector,
environment: Readonly<Record<string, string>>,
) => Result<number | undefined, ConfigurationError>;
export interface ClientRegistrationEnvironmentDetails {
readonly values: Readonly<Record<string, string>>;
readonly timeoutInputs: McpToolTimeoutInputs;
}
/** Read explicit REA overrides without dropping peers when one value is invalid. */
export const clientRegistrationEnvironment = (
@@ -22,6 +34,33 @@ export const clientRegistrationEnvironment = (
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
if (parsed?.dialect === "dsh")
return dshDocument(parsed.document).environment(overrides, validation);
const details = clientRegistrationEnvironmentDetails(
parsed,
overrides,
validation,
);
return details.ok ? ok(details.value.values) : details;
};
/** Retain original strings and the parser-owned timeout facts. */
export const clientRegistrationEnvironmentDetails = (
parsed: ClientConfigurationDocument | undefined,
overrides: Readonly<Record<string, string>> = {},
validation: "fragment" | "effective" = "effective",
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Result<ClientRegistrationEnvironmentDetails, ConfigurationError> => {
if (parsed?.dialect === "dsh") {
const environment = dshDocument(parsed.document).environment(
overrides,
validation,
);
if (!environment.ok) return environment;
return validatedDetails(
environment.value,
validation,
resolveProviderRequestBudget,
);
}
const key =
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
? "environment"
@@ -50,16 +89,45 @@ export const clientRegistrationEnvironment = (
// Validate known settings with the runtime's parser, but retain the original
// string overrides, including unknown client-specific keys and no defaults.
const environment = { ...existing.data, ...overrides };
return validatedDetails(
environment,
validation,
resolveProviderRequestBudget,
);
};
const validatedDetails = (
environment: Readonly<Record<string, string>>,
validation: "fragment" | "effective",
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Result<ClientRegistrationEnvironmentDetails, ConfigurationError> => {
const validated =
validation === "fragment"
? parseEnvironmentValues(environment)
: parseEnvironment(environment);
if (!validated.ok) return validated;
return ok(
Object.fromEntries(
const providerRequestBudget =
resolveProviderRequestBudget?.(
validated.value.REA_ANALYSIS_PROVIDER,
environment,
) ?? ok(undefined);
if (!providerRequestBudget.ok) return providerRequestBudget;
return ok({
values: Object.fromEntries(
Object.entries(environment).sort(([left], [right]) =>
compareUnicodeCodePoints(left, right),
),
),
);
timeoutInputs: {
analysisProvider: validated.value.REA_ANALYSIS_PROVIDER,
ghidraStartupTimeoutMs: validated.value.REA_GHIDRA_STARTUP_TIMEOUT_MS,
hopperStartupTimeoutMs: validated.value.REA_HOPPER_STARTUP_TIMEOUT_MS,
...(validated.value.REA_JEB_MCP_TIMEOUT_MS === undefined
? {}
: { jebMcpTimeoutMs: validated.value.REA_JEB_MCP_TIMEOUT_MS }),
...(providerRequestBudget.value === undefined
? {}
: { providerRequestTimeoutMs: providerRequestBudget.value }),
},
});
};
@@ -13,8 +13,9 @@ const environmentValue = (
platform: NodeJS.Platform,
): string | undefined => {
if (platform !== "win32") return environment[name];
const key = Object.keys(environment).find(
(candidate) => candidate.toLowerCase() === name,
const target = name.toLowerCase();
const key = Object.keys(environment).findLast(
(candidate) => candidate.toLowerCase() === target,
);
return key === undefined ? undefined : environment[key];
};
+105 -16
View File
@@ -14,7 +14,13 @@ import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import {
MCP_STARTUP_POLICY,
mcpProviderDeadlineAllowanceMs,
recommendedMcpToolTimeoutMs,
recommendedMcpToolTimeoutSeconds,
type McpToolTimeoutInputs,
} from "../mcpStartupPolicy.js";
import {
clientRegistrationPackageReference,
classifyClientRegistrationCommand,
@@ -28,6 +34,8 @@ import {
import type { SetupClient } from "./SupportedClients.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import { dshDocument } from "./DshCordisPatch.js";
import { clientRegistrationEnvironmentDetails } from "./ClientRegistrationEnvironment.js";
import type { ProviderRequestBudgetResolver } from "./ClientRegistrationEnvironment.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
interface ClientRegistrationStatusBase {
@@ -35,6 +43,8 @@ interface ClientRegistrationStatusBase {
readonly config_path: string;
/** Exact package reference requested by npx/bunx, when registration uses one. */
readonly package_reference?: string;
/** Non-blocking timeout guidance; caller-selected values remain aligned. */
readonly advisory?: string;
}
type RegistrationCommand = readonly [string, ...string[]];
@@ -73,6 +83,7 @@ const registrationSchema = z
command: z.string().min(1),
args: z.array(z.string()).default([]),
startup_timeout_sec: z.number().positive().optional(),
tool_timeout_sec: z.number().positive().optional(),
startupTimeoutMs: z.number().positive().optional(),
toolTimeoutMs: z.number().positive().optional(),
type: z.string().optional(),
@@ -88,6 +99,7 @@ interface ClientRegistrationReadOptions {
readonly platform?: NodeJS.Platform;
readonly environment?: NodeJS.ProcessEnv;
readonly resources?: ArtifactResourceScope;
readonly resolveProviderRequestBudget?: ProviderRequestBudgetResolver;
}
/** Registration facts shared by diagnostics and update maintenance. */
@@ -215,6 +227,28 @@ export const readClientRegistrationInspections = async (
}
: client;
const registration = parseRegistration(raw, effectiveClient);
const registrationEnvironment =
client.name === "kimi" || client.name === "codex"
? clientRegistrationEnvironmentDetails(
parsed,
{},
"effective",
options.resolveProviderRequestBudget,
)
: undefined;
if (
registrationEnvironment !== undefined &&
!registrationEnvironment.ok
) {
record({
client: client.name,
config_path: configPath,
command: [],
state: "invalid",
remediation: registrationEnvironment.error.message,
});
continue;
}
const command: RegistrationCommand = [
registration.command,
...registration.args,
@@ -225,12 +259,16 @@ export const readClientRegistrationInspections = async (
const packageVersionDrift =
packageVersion !== undefined &&
packageVersion !== PRODUCT_IDENTITY.packageVersion;
const alignment = registrationAligned(
const alignment = registrationAligned({
registration,
effectiveClient,
client: effectiveClient,
currentCommandPath,
options.platform ?? process.platform,
);
platform: options.platform ?? process.platform,
kimiInputSchemaProfileSelected:
registrationEnvironment?.ok === true &&
registrationEnvironment.value.values.REA_MCP_INPUT_SCHEMA_PROFILE !==
undefined,
});
record(
configuredStatus(
effectiveClient,
@@ -252,6 +290,13 @@ export const readClientRegistrationInspections = async (
? unknownEntryPointRemediation()
: undefined),
packageReference,
registrationEnvironment?.ok === true
? timeoutAdvisory(
client,
registration,
registrationEnvironment.value.timeoutInputs,
)
: undefined,
),
activation,
);
@@ -284,12 +329,19 @@ const registrationActivation = (
? "disabled"
: "enabled";
const registrationAligned = (
registration: z.output<typeof registrationSchema>,
client: SetupClient,
currentCommandPath: string | undefined,
platform: NodeJS.Platform,
): "aligned" | "stale" | "unknown" => {
const registrationAligned = ({
registration,
client,
currentCommandPath,
platform,
kimiInputSchemaProfileSelected,
}: {
readonly registration: z.output<typeof registrationSchema>;
readonly client: SetupClient;
readonly currentCommandPath: string | undefined;
readonly platform: NodeJS.Platform;
readonly kimiInputSchemaProfileSelected: boolean;
}): "aligned" | "stale" | "unknown" => {
const command = [registration.command, ...registration.args];
const identity = classifyClientRegistrationCommand(
command,
@@ -298,18 +350,24 @@ const registrationAligned = (
if (identity === "unknown") return "unknown";
if (identity === "foreign") return "stale";
if (
(client.name === "codex" || client.name === "grok_build") &&
client.name === "grok_build" &&
registration.startup_timeout_sec !==
MCP_STARTUP_POLICY.codexStartupTimeoutSeconds
)
return "stale";
if (
client.name === "kimi" &&
(registration.startupTimeoutMs !==
MCP_STARTUP_POLICY.kimiStartupTimeoutMs ||
registration.toolTimeoutMs !== MCP_STARTUP_POLICY.kimiToolTimeoutMs)
client.name === "codex" &&
(registration.startup_timeout_sec === undefined ||
registration.tool_timeout_sec === undefined)
)
return "stale";
if (
client.name === "kimi" &&
(registration.startupTimeoutMs === undefined ||
registration.toolTimeoutMs === undefined)
)
return "stale";
if (client.name === "kimi" && !kimiInputSchemaProfileSelected) return "stale";
if (
(client.format === "kimi" ||
client.format === "omp" ||
@@ -435,6 +493,7 @@ const configuredStatus = (
state: "aligned" | "stale" | "unknown",
policyRemediation?: string,
packageReference?: string,
advisory?: string,
): ClientRegistrationStatus =>
state === "aligned"
? {
@@ -446,6 +505,7 @@ const configuredStatus = (
: { package_reference: packageReference }),
state,
remediation: null,
...(advisory === undefined ? {} : { advisory }),
}
: {
client: client.name,
@@ -458,6 +518,35 @@ const configuredStatus = (
remediation: policyRemediation ?? remediation(client),
};
const timeoutAdvisory = (
client: SetupClient,
registration: z.output<typeof registrationSchema>,
timeoutInputs: McpToolTimeoutInputs,
): string | undefined => {
const toolTimeout =
client.name === "kimi"
? registration.toolTimeoutMs
: client.name === "codex"
? registration.tool_timeout_sec
: undefined;
if (toolTimeout === undefined) return undefined;
const providerAllowance = mcpProviderDeadlineAllowanceMs(timeoutInputs);
const isKimi = client.name === "kimi";
const toolTimeoutMs = isKimi ? toolTimeout : toolTimeout * 1000;
if (toolTimeoutMs >= providerAllowance) return undefined;
const setting = isKimi ? "toolTimeoutMs" : "tool_timeout_sec";
const currentValue = isKimi
? `${String(toolTimeout)} ms`
: `${String(toolTimeout)} seconds`;
const recommendedValue = isKimi
? `${String(recommendedMcpToolTimeoutMs(timeoutInputs))} ms`
: `${String(recommendedMcpToolTimeoutSeconds(timeoutInputs))} seconds`;
const headroom = isKimi
? `${String(MCP_STARTUP_POLICY.toolOperationHeadroomMs)} ms`
: `${String(MCP_STARTUP_POLICY.toolOperationHeadroomMs / 1000)} seconds`;
return `The configured ${setting} (${currentValue}) is shorter than REA's known provider allowance (${String(providerAllowance)} ms), so provider startup or an upstream request may exceed this client deadline. REA recommends ${setting} = ${recommendedValue} to cover that allowance plus ${headroom} of operation headroom; this additional headroom is a recommendation, not a guarantee for every analysis.`;
};
const exactRegisteredPackageVersion = (
packageReference: string | undefined,
): string | undefined => {
+11 -1
View File
@@ -1,5 +1,6 @@
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import { ok } from "../domain/result.js";
import type { DoctorHost } from "./Doctor.js";
/** Healthy recording host for focused doctor and projection tests. */
@@ -14,7 +15,16 @@ export const createDoctorHostFixture = (
linuxDistribution: () => Promise.resolve(undefined),
validTarget: (path) => Promise.resolve(path.includes("Hopper")),
executable: (path) => Promise.resolve(path.includes("Hopper")),
supportedLinuxHopper: () => Promise.resolve(true),
inspectLinuxHopper: () =>
Promise.resolve(
ok({
sha256: "0".repeat(64),
version: "fixture",
primaryFingerprint: "0".repeat(40),
signingFingerprint: "1".repeat(40),
source: "signed_package_cache" as const,
}),
),
linuxDemoRuntimeCheck: () =>
Promise.resolve({
name: "hopper-demo-runtime",
+44 -2
View File
@@ -4,6 +4,8 @@ import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
import type { ClientRegistrationStatus } from "./ClientRegistrationStatus.js";
import { createDoctorHostFixture as host } from "./Doctor.fixture.js";
import { runDoctor } from "./Doctor.js";
import { err } from "../domain/result.js";
import { HopperReleaseError } from "./HopperReleaseFeed.js";
describe("doctor runtime check", () => {
it("reports Node 25 as unsupported", async () => {
@@ -363,6 +365,38 @@ describe("doctor scoped readiness", () => {
);
expect(staleSelected.healthy).toBe(false);
});
it("keeps a caller-selected short timeout healthy and reports its advisory as informational", async () => {
const result = await runDoctor(
undefined,
host({
clientRegistrations: () =>
Promise.resolve([
{
client: "kimi",
config_path: "/home/user/.kimi-code/mcp.json",
command: ["/current/rea", "mcp"],
state: "aligned",
remediation: null,
advisory: "The configured toolTimeoutMs is shorter than startup.",
},
]),
}),
{ clients: ["kimi"] },
);
expect(result.healthy).toBe(true);
expect(result.scope_checks).toContainEqual(
expect.objectContaining({ name: "registration:kimi", ok: true }),
);
expect(result.informational_checks).toContainEqual(
expect.objectContaining({
name: "registration_timeout:kimi",
ok: true,
detail: "The configured toolTimeoutMs is shorter than startup.",
}),
);
});
});
describe("Grok Bot readiness", () => {
@@ -585,12 +619,20 @@ describe("doctor Linux and Hopper discovery", () => {
hopperSetupSupported: true,
}),
executable: (candidate) => Promise.resolve(candidate === path),
supportedLinuxHopper: () => Promise.resolve(false),
inspectLinuxHopper: () =>
Promise.resolve(
err(
new HopperReleaseError(
"unsupported_hopper_build",
"Unverified fixture build",
),
),
),
}),
);
expect(
result.checks.find(({ name }) => name === "hopper-version")?.remediation,
).toContain("Unset or update HOPPER_LAUNCHER_PATH");
).toContain("Check HOPPER_LAUNCHER_PATH");
});
});
+42 -4
View File
@@ -19,11 +19,14 @@ import {
macHopperRenamedLauncherCandidates,
} from "./MacHopperPaths.js";
import {
linuxHopperBinarySupported,
inspectLinuxHopperLauncher,
linuxSharedLibrariesAvailable,
readLinuxDistribution,
type LinuxDistribution,
} from "./LinuxHopper.js";
import type { Result } from "../domain/result.js";
import type { HopperReleaseError } from "./HopperReleaseFeed.js";
import type { VerifiedLinuxHopperLauncher } from "./LinuxHopperPublisher.js";
import {
readInstalledSkillIdentity,
type InstalledSkillIdentity,
@@ -36,6 +39,7 @@ import {
type ClientRegistrationStatus,
type UnhealthyClientRegistrationStatus,
} from "./ClientRegistrationStatus.js";
import type { ProviderRequestBudgetResolver } from "./ClientRegistrationEnvironment.js";
import {
collectDoctorDiagnostics,
doctorHealthy,
@@ -100,11 +104,15 @@ export interface DoctorHost {
/** Route-specific target admission details, when supplied by the host. */
inspectTarget?(path: string): Promise<DoctorCheck>;
executable(path: string): Promise<boolean>;
supportedLinuxHopper(path: string): Promise<boolean>;
inspectLinuxHopper(
path: string,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>>;
linuxDemoRuntimeCheck(): Promise<DoctorCheck>;
brewHopperPath(): Promise<string | undefined>;
manualHopperPaths(): Promise<readonly string[]>;
providerInspections?(): Promise<readonly DoctorProviderInspection[]>;
/** Resolve a parsed upstream request budget from the effective client environment. */
resolveProviderRequestBudget?: ProviderRequestBudgetResolver;
/** Validated provider settings preserved by setup, independent of readiness. */
providerRegistrationEnvironment?(): Readonly<Record<string, string>>;
installationPaths?(): Promise<readonly string[]>;
@@ -220,7 +228,11 @@ const inspectDoctor = async (
providerInspections,
} = await collectDoctorDiagnostics(target, host);
const identity = await collectDoctorIdentity(host, scope);
const checks = [...diagnosticChecks, ...identity.checks];
const checks = [
...diagnosticChecks,
...identity.checks,
...registrationAdvisoryChecks(identity.value.registrations),
];
const providers = {
hopperAvailable:
hopperPath !== undefined &&
@@ -252,6 +264,22 @@ const inspectDoctor = async (
};
};
const registrationAdvisoryChecks = (
registrations: readonly ClientRegistrationStatus[],
): readonly DoctorCheck[] =>
registrations.flatMap(({ client, advisory }) =>
advisory === undefined
? []
: [
{
name: `registration_timeout:${client}`,
ok: true,
classification: "healthy" as const,
detail: advisory,
},
],
);
const closeDoctorHost = async (
host: DoctorHost,
): Promise<string | undefined> => {
@@ -411,6 +439,7 @@ export interface SystemDoctorHostOptions {
readonly providerRegistrationEnvironment?: () => Readonly<
Record<string, string>
>;
readonly resolveProviderRequestBudget?: ProviderRequestBudgetResolver;
readonly linuxDemoRuntimeCheck?: () => Promise<DoctorCheck>;
}
@@ -488,7 +517,7 @@ export const systemDoctorHost = (
),
linuxDemoRuntimeCheck:
options.linuxDemoRuntimeCheck ?? uncomposedLinuxDemoRuntimeCheck,
supportedLinuxHopper: linuxHopperBinarySupported,
inspectLinuxHopper: (path) => inspectLinuxHopperLauncher(path, environment),
async brewHopperPath() {
return probeHomebrew(async (command) => {
let caskroom: string;
@@ -527,6 +556,9 @@ export const systemDoctorHost = (
providerRegistrationEnvironment:
options.providerRegistrationEnvironment,
}),
...(options.resolveProviderRequestBudget === undefined
? {}
: { resolveProviderRequestBudget: options.resolveProviderRequestBudget }),
async ilspyCmdVersion(path) {
try {
await access(path, constants.X_OK);
@@ -579,6 +611,12 @@ export const systemDoctorHost = (
platform,
environment,
resources,
...(options.resolveProviderRequestBudget === undefined
? {}
: {
resolveProviderRequestBudget:
options.resolveProviderRequestBudget,
}),
}),
};
};
+30 -13
View File
@@ -209,17 +209,34 @@ const optionalHopperVersionCheck = async (
hopperPath: string | undefined,
): Promise<DoctorCheck | undefined> => {
if (host.platform !== "linux" || hopperPath === undefined) return undefined;
return check(
"hopper-version",
await host.supportedLinuxHopper(hopperPath),
hopperPath,
{
remediation: unsupportedHopperRemediation(
host.configuredHopperPath === hopperPath,
),
classification: "config_drift",
},
);
const inspected = await host.inspectLinuxHopper(hopperPath);
const buildCheck = check("hopper-version", inspected.ok, hopperPath, {
remediation: unsupportedHopperRemediation(
host.configuredHopperPath === hopperPath,
),
classification:
!inspected.ok &&
[
"release_metadata",
"download",
"verification_unavailable",
"cancelled",
"cleanup_failed",
].includes(inspected.error.reason)
? "unknown"
: "config_drift",
});
return {
...buildCheck,
details: inspected.ok
? {
reported_release_version: inspected.value.version,
launcher_sha256: inspected.value.sha256,
primary_fingerprint: inspected.value.primaryFingerprint,
source: inspected.value.source,
}
: { reason: inspected.error.reason, message: inspected.error.message },
};
};
const optionalHopperDemoCheck = async (
@@ -297,8 +314,8 @@ const providerDoctorChecks = (
const unsupportedHopperRemediation = (configured: boolean): string =>
configured
? "Unset or update HOPPER_LAUNCHER_PATH, install the Hopper build supported by this REA release, or update REA."
: "Install the Hopper build supported by this REA release, or update REA for a newer Hopper build.";
? "Check HOPPER_LAUNCHER_PATH and its publisher verification evidence. Run rea setup to install and retain an authenticated official Hopper package for offline use."
: "Run rea setup to install and retain an authenticated official Hopper package for offline use.";
const parseMajor = (version: string | undefined): number =>
Number.parseInt(version?.split(".")[0] ?? "0", 10);
+18 -4
View File
@@ -4,9 +4,23 @@ export const HOPPER_DOWNLOAD_BYTES = 100_000_000;
/** Bound observed response bytes before retaining them, including chunked bodies. */
export const downloadHopperPackage = async (
url: string,
options: { readonly signal?: AbortSignal } = {},
options: {
readonly signal?: AbortSignal;
readonly maxBytes?: number;
readonly timeoutMs?: number;
} = {},
): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }> => {
const deadline = AbortSignal.timeout(120_000);
const timeoutMs = options.timeoutMs ?? 120_000;
if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 120_000)
throw new RangeError("Invalid Hopper response time budget");
const deadline = AbortSignal.timeout(timeoutMs);
const maxBytes = options.maxBytes ?? HOPPER_DOWNLOAD_BYTES;
if (
!Number.isSafeInteger(maxBytes) ||
maxBytes < 1 ||
maxBytes > HOPPER_DOWNLOAD_BYTES
)
throw new RangeError("Invalid Hopper response byte budget");
const response = await fetch(url, {
headers: { accept: "application/json", "user-agent": "rea-installer" },
signal:
@@ -17,7 +31,7 @@ export const downloadHopperPackage = async (
const refused = { ok: false, bytes: new Uint8Array() };
if (
!response.ok ||
Number(response.headers.get("content-length")) > HOPPER_DOWNLOAD_BYTES
Number(response.headers.get("content-length")) > maxBytes
) {
await response.body?.cancel();
return refused;
@@ -30,7 +44,7 @@ export const downloadHopperPackage = async (
while (true) {
const chunk = await reader.read();
if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) };
if (chunk.value.byteLength > HOPPER_DOWNLOAD_BYTES - bytes) {
if (chunk.value.byteLength > maxBytes - bytes) {
await reader.cancel();
return refused;
}
+77
View File
@@ -0,0 +1,77 @@
import { describe, expect, it } from "vitest";
import { parseLinuxHopperReleases } from "./HopperReleaseFeed.js";
const release = {
version: "99.2.1",
filename:
"https://www.hopperapp.com:443/downloader/public/Hopper-99.2.1-Linux-demo.deb",
file_length: "123",
file_hash: "a".repeat(40),
main_exe_hash: "b".repeat(64),
signature: "AQ==",
public: "1",
beta: "0",
published_at: "preserved vendor field",
};
describe("Linux Hopper release feed boundary", () => {
it("preserves future release identity and partial platform facts", () => {
const parsed = parseLinuxHopperReleases({
"Ubuntu / Mint": release,
"OS X": { signature: "Sparkle DSA" },
});
expect(parsed).toMatchObject({
ok: true,
value: {
deb: { ok: true, value: release },
rpm: { ok: false },
arch: { ok: false },
},
});
if (!parsed.ok || !parsed.value.deb.ok) throw new Error("Missing release");
expect(parsed.value.deb.value.filename).toBe(release.filename);
});
it.each([
{ signature: undefined },
{ signature: "AQ==garbage" },
{ main_exe_hash: "b".repeat(40) },
{ file_length: "100000001" },
{ file_length: "NaN" },
{ public: "0" },
{ beta: "1" },
{
filename:
"https://other.example/downloader/public/Hopper-99.2.1-Linux-demo.deb",
},
{ filename: release.filename + "?token=caller" },
{
filename: release.filename.replace("https://", "https://user:password@"),
},
{ filename: release.filename.replace(".deb", ".rpm") },
{ filename: release.filename.replace("demo.deb", "full.deb") },
])(
"rejects metadata outside the selected public package contract: %j",
(changes) => {
expect(
parseLinuxHopperReleases({
"Ubuntu / Mint": { ...release, ...changes },
}),
).toMatchObject({
ok: true,
value: { deb: { ok: false, error: { reason: "release_metadata" } } },
});
},
);
it.each([null, [], "html error page"])(
"keeps malformed feed roots unknown",
(raw) => {
expect(parseLinuxHopperReleases(raw)).toMatchObject({
ok: false,
error: { reason: "release_metadata" },
});
},
);
});
+125
View File
@@ -0,0 +1,125 @@
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
import { HOPPER_DOWNLOAD_BYTES } from "./HopperDownload.js";
/** Vendor endpoints, shared by installation and existing-build inspection. */
export const HOPPER_RELEASES_URL =
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
export const HOPPER_PUBLIC_KEY_URL =
"https://www.hopperapp.com/hopper-public-key.asc";
export const HOPPER_DOWNLOAD_PREFIX =
"https://www.hopperapp.com:443/downloader/public/";
/** Stable publisher identity; signing subkeys are admitted through this primary. */
export const HOPPER_LINUX_PRIMARY_FINGERPRINT =
"7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD";
export type LinuxPackageFamily = "deb" | "rpm" | "arch";
const platforms = {
deb: "Ubuntu / Mint",
rpm: "Fedora",
arch: "Arch",
} as const;
const suffixes = { deb: ".deb", rpm: ".rpm", arch: ".pkg.tar.xz" } as const;
export class HopperReleaseError extends Error {
readonly _tag = "HopperReleaseError";
constructor(
readonly reason:
| "release_metadata"
| "download"
| "integrity"
| "publisher_verification_failed"
| "verification_unavailable"
| "unsupported_hopper_build"
| "cancelled"
| "cleanup_failed"
| "cache_missing",
message: string,
options?: ErrorOptions,
) {
super(message, options);
this.name = this._tag;
}
}
const sha256 = z.string().regex(/^[a-f0-9]{64}$/u);
const releaseSchema = z
.object({
version: z.string().min(1),
filename: z.string().url(),
file_length: z
.string()
.regex(/^[1-9]\d*$/u)
.refine((text) => {
const count = Number(text);
return Number.isSafeInteger(count) && count <= HOPPER_DOWNLOAD_BYTES;
}),
file_hash: z.string().regex(/^[a-f0-9]{40}$/u),
main_exe_hash: sha256,
signature: z
.string()
.min(1)
.refine(
(text) => Buffer.from(text, "base64").toString("base64") === text,
),
public: z.literal("1"),
beta: z.literal("0"),
})
.passthrough();
/** Parsed once at the feed boundary; preserve the vendor's complete release row. */
export type LinuxHopperRelease = z.infer<typeof releaseSchema>;
export type LinuxHopperReleaseFeed = Readonly<
Record<LinuxPackageFamily, Result<LinuxHopperRelease, HopperReleaseError>>
>;
export const parseLinuxHopperRelease = (
raw: unknown,
family: LinuxPackageFamily,
): Result<LinuxHopperRelease, HopperReleaseError> => {
const parsed = releaseSchema.safeParse(raw);
if (!parsed.success)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper release omitted valid package identity or signature metadata.",
),
);
const url = new URL(parsed.data.filename);
if (
url.origin !== "https://www.hopperapp.com" ||
url.username !== "" ||
url.password !== "" ||
url.search !== "" ||
url.hash !== "" ||
!url.pathname.startsWith("/downloader/public/Hopper-") ||
!url.pathname.endsWith(`-Linux-demo${suffixes[family]}`)
)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper release is outside the official public Linux demo channel.",
),
);
return ok(parsed.data);
};
export const parseLinuxHopperReleases = (
raw: unknown,
): Result<LinuxHopperReleaseFeed, HopperReleaseError> => {
const feed = z.record(z.string(), z.unknown()).safeParse(raw);
if (!feed.success)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper releases response is not an object.",
),
);
return ok({
deb: parseLinuxHopperRelease(feed.data[platforms.deb], "deb"),
rpm: parseLinuxHopperRelease(feed.data[platforms.rpm], "rpm"),
arch: parseLinuxHopperRelease(feed.data[platforms.arch], "arch"),
});
};
@@ -18,6 +18,7 @@ export const workflowAnalysisProfile = (
): AnalysisProfileCommitment =>
createAnalysisProfile(REA_WORKFLOW_PROVIDER, {
upstream_analysis_profile: upstream,
source_limitations: "provider-operation-caveats-v1",
...(operation === "binary_overview"
? { workflow_revision: 4 }
: operation === "batch_decompile" ||
+79 -25
View File
@@ -4,7 +4,6 @@ import { describe, expect, it } from "vitest";
import {
formatCommandForDisplay,
installLinuxHopper,
linuxHopperLauncherDigestSupported,
linuxPackageManagerCommands,
linuxSharedLibrariesAvailable,
parseLinuxDistribution,
@@ -18,6 +17,12 @@ import {
type LinuxPackageInstallFailure,
type LinuxPackageInstallOutcome,
} from "./LinuxHopper.js";
import { err, ok } from "../domain/result.js";
import {
HopperReleaseError,
type LinuxHopperRelease,
} from "./HopperReleaseFeed.js";
import type { VerifiedLinuxHopperPackage } from "./LinuxHopperPublisher.js";
describe("Linux package-manager command display", () => {
it("preserves argument boundaries for paths with spaces and quotes", () => {
@@ -43,6 +48,9 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
archiveOk = true;
installSucceeds = true;
cleanupFails = false;
signatureValid = true;
cacheWritable = true;
verifiedBeforeInstall = false;
packageFailure: LinuxPackageInstallFailure = {
command: "pacman -U --noconfirm /tmp/rea test/hopper.arch",
exitCode: 1,
@@ -57,6 +65,8 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
distribution = (): Promise<LinuxDistribution | undefined> =>
Promise.resolve(this.distributionValue);
release = (family: LinuxPackageFamily) =>
Promise.resolve(ok(releasesFor(new Uint8Array([1, 2, 3]))[family]));
download = (url: string): Promise<LinuxHopperDownload> => {
this.downloads.push(url);
return Promise.resolve({ ok: this.archiveOk, bytes: this.archive });
@@ -64,10 +74,52 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
createTemporaryDirectory = (): Promise<string> =>
Promise.resolve("/tmp/rea test");
writeArchive = (): Promise<void> => Promise.resolve();
verifyArchive = (family: LinuxPackageFamily, release: LinuxHopperRelease) => {
if (
createHash("sha1").update(this.archive).digest("hex") !==
release.file_hash
)
return Promise.resolve(
err(new HopperReleaseError("integrity", "Fixture byte mismatch")),
);
if (!this.signatureValid)
return Promise.resolve(
err(
new HopperReleaseError(
"publisher_verification_failed",
"Fixture signature rejection",
),
),
);
this.verifiedBeforeInstall = true;
return Promise.resolve(
ok({
family,
release,
packageSha256: createHash("sha256").update(this.archive).digest("hex"),
publicKey: "fixture",
primaryFingerprint: "0".repeat(40),
signingFingerprint: "1".repeat(40),
}),
);
};
retainVerifiedArchive = (_verified: VerifiedLinuxHopperPackage) =>
Promise.resolve(
this.cacheWritable
? ok(undefined)
: err(
new HopperReleaseError(
"verification_unavailable",
"Fixture cache permission rejection",
),
),
);
installPackage = (
family: LinuxPackageFamily,
archive: string,
): Promise<LinuxPackageInstallOutcome> => {
if (!this.verifiedBeforeInstall)
throw new Error("Package manager ran before publisher verification");
this.installed.push({ family, archive });
const outcome: LinuxPackageInstallOutcome = this.installSucceeds
? { status: "installed" }
@@ -160,20 +212,6 @@ describe("Linux Hopper host classification", () => {
});
describe("Linux Hopper installation", () => {
it.each([
[
"Debian and Arch",
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
],
["RPM", "1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13"],
])("accepts the pinned %s launcher build", (_, digest) => {
expect(linuxHopperLauncherDigestSupported(digest)).toBe(true);
});
it("rejects an unverified launcher build", () => {
expect(linuxHopperLauncherDigestSupported("0".repeat(64))).toBe(false);
});
it.each([
["deb", "apt-get", ["xvfb", "xauth", "python3", "libx11-6", "libxtst6"]],
[
@@ -239,6 +277,23 @@ describe("Linux Hopper package installation", () => {
});
describe("Linux Hopper installation failure classification", () => {
it.each([
["signatureValid", "publisher_verification_failed"],
["cacheWritable", "verification_unavailable"],
] as const)(
"stops before privileged installation when %s fails",
async (field, reason) => {
const host = new RecordingLinuxHost();
host[field] = false;
expect(await installFixture(host)).toMatchObject({
status: "failed",
reason,
verificationFailure: { reason, message: expect.any(String) },
});
expect(host.installed).toEqual([]);
expect(host.cleaned).toEqual(["/tmp/rea test"]);
},
);
it.each([
["unsupported host", "unsupported_host"],
["package request failure", "download"],
@@ -263,7 +318,7 @@ describe("Linux Hopper installation failure classification", () => {
host.launcherStatusValue = "runtime_dependencies";
if (scenario === "unsupported build")
host.launcherStatusValue = "unsupported_hopper_build";
expect(await installFixture(host)).toEqual({
expect(await installFixture(host)).toMatchObject({
status: "failed",
reason,
...(scenario === "package manager failure"
@@ -363,14 +418,16 @@ describe("Linux Hopper temporary cleanup ownership", () => {
function releasesFor(
bytes: Uint8Array,
): Record<
LinuxPackageFamily,
{ filename: string; file_length: string; file_hash: string }
> {
): Record<LinuxPackageFamily, LinuxHopperRelease> {
const release = (extension: string) => ({
filename: `https://www.hopperapp.com:443/downloader/public/Hopper.${extension}`,
version: "future-fixture",
filename: `https://www.hopperapp.com:443/downloader/public/Hopper-future-fixture-Linux-demo.${extension}`,
file_length: String(bytes.byteLength),
file_hash: createHash("sha1").update(bytes).digest("hex"),
main_exe_hash: "0".repeat(64),
signature: "AQ==",
public: "1" as const,
beta: "0" as const,
});
return {
deb: release("deb"),
@@ -379,7 +436,4 @@ function releasesFor(
};
}
const installFixture = (host: RecordingLinuxHost) =>
installLinuxHopper(host, {
releases: releasesFor(new Uint8Array([1, 2, 3])),
});
const installFixture = (host: RecordingLinuxHost) => installLinuxHopper(host);
+175 -102
View File
@@ -1,55 +1,37 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { access, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { execFileBoundedOutput } from "../process/ExecFileBoundedOutput.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { downloadHopperPackage } from "./HopperDownload.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { err, type Result } from "../domain/result.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
HOPPER_RELEASES_URL,
HOPPER_PUBLIC_KEY_URL,
HOPPER_DOWNLOAD_PREFIX,
HOPPER_LINUX_PRIMARY_FINGERPRINT,
HopperReleaseError,
type LinuxHopperRelease,
type LinuxPackageFamily,
} from "./HopperReleaseFeed.js";
import {
LinuxHopperPublisher,
retryLinuxHopperPublisherCleanup,
type VerifiedLinuxHopperPackage,
type VerifiedLinuxHopperLauncher,
} from "./LinuxHopperPublisher.js";
export type { LinuxPackageFamily } from "./HopperReleaseFeed.js";
/** Package-manager output retained per stream without stopping the command. */
const PACKAGE_MANAGER_OUTPUT_BYTES = 64 * 1024;
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const SUPPORTED_HOPPER_SHA256 = new Set([
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
]);
interface LinuxHopperRelease {
readonly filename: string;
readonly file_length: string;
readonly file_hash: string;
}
const SUPPORTED_RELEASES: Readonly<
Record<LinuxPackageFamily, LinuxHopperRelease>
> = {
deb: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.deb`,
file_length: "35755772",
file_hash: "e4f79dff602648a8ff4a88b773875b6bfac0dc65",
},
rpm: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.rpm`,
file_length: "32264475",
file_hash: "b9a65f2f583a386d440d4401d9140aef3bf305de",
},
arch: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.pkg.tar.xz`,
file_length: "29405968",
file_hash: "92d780b61cab38eda95dc4112dee4fbeb0920f46",
},
};
export type LinuxPackageFamily = "deb" | "rpm" | "arch";
/** Exact stable external effects disclosed before a Linux Hopper installation. */
export interface LinuxHopperInstallDisclosure {
readonly downloadUrl: string;
readonly expectedBytes: number;
readonly expectedSha1: string;
readonly networkOrigins: readonly string[];
readonly integrity: string;
readonly commands: readonly string[];
}
@@ -58,11 +40,13 @@ export const linuxHopperInstallDisclosure = (
family: LinuxPackageFamily,
isRoot: boolean,
): LinuxHopperInstallDisclosure => {
const release = SUPPORTED_RELEASES[family];
return {
downloadUrl: release.filename,
expectedBytes: Number(release.file_length),
expectedSha1: release.file_hash,
networkOrigins: [
HOPPER_RELEASES_URL,
HOPPER_PUBLIC_KEY_URL,
HOPPER_DOWNLOAD_PREFIX,
],
integrity: `Detached PGP signature · primary ${HOPPER_LINUX_PRIMARY_FINGERPRINT} · package byte identity and installed launcher SHA-256 from the official feed`,
commands: linuxPackageManagerCommands(
family,
`<verified-hopper.${family}>`,
@@ -133,6 +117,10 @@ export type LinuxPackageInstallOutcome =
/** External effects required to install the official Linux package. */
export interface LinuxHopperInstallHost {
distribution(): Promise<LinuxDistribution | undefined>;
release(
family: LinuxPackageFamily,
signal?: AbortSignal,
): Promise<Result<LinuxHopperRelease, HopperReleaseError>>;
download(
url: string,
options: { readonly signal?: AbortSignal },
@@ -143,8 +131,23 @@ export interface LinuxHopperInstallHost {
family: LinuxPackageFamily,
archive: string,
): Promise<LinuxPackageInstallOutcome>;
launcherStatus(path: string): Promise<LinuxHopperLauncherStatus>;
verifyArchive(
family: LinuxPackageFamily,
release: LinuxHopperRelease,
archive: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>>;
retainVerifiedArchive(
verified: VerifiedLinuxHopperPackage,
archive: string,
signal?: AbortSignal,
): Promise<Result<void, HopperReleaseError>>;
launcherStatus(
path: string,
expectedSha256: string,
): Promise<LinuxHopperLauncherStatus>;
cleanup(path: string): Promise<void>;
close?(): Promise<void>;
}
/** Structured outcome for every expected Linux installation failure. */
@@ -163,6 +166,9 @@ export type LinuxHopperInstallResult =
| "launcher_missing"
| "runtime_dependencies"
| "unsupported_hopper_build"
| "publisher_verification_failed"
| "verification_unavailable"
| "cleanup_failed"
| "cancelled";
/**
* The failing package-manager command and its output. Present only when
@@ -170,6 +176,8 @@ export type LinuxHopperInstallResult =
* package manager said instead of assuming an authorization problem.
*/
readonly packageFailure?: LinuxPackageInstallFailure;
/** Publisher-owned failure, retained for the public setup diagnostic. */
readonly verificationFailure?: HopperReleaseError;
} & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure });
/** Failed removal of an owned temporary install directory. */
@@ -270,9 +278,6 @@ export const installLinuxHopper = async (
host: LinuxHopperInstallHost = systemLinuxHopperInstallHost(process.env),
options: {
readonly signal?: AbortSignal;
readonly releases?: Readonly<
Record<LinuxPackageFamily, LinuxHopperRelease>
>;
} = {},
): Promise<LinuxHopperInstallResult> => {
if (options.signal?.aborted === true)
@@ -284,16 +289,56 @@ export const installLinuxHopper = async (
let result: LinuxHopperInstallResult;
try {
result = await (async (): Promise<LinuxHopperInstallResult> => {
const release = (options.releases ?? SUPPORTED_RELEASES)[
distribution.packageFamily
];
const selected = await host.release(
distribution.packageFamily,
options.signal,
);
if (!selected.ok)
return {
status: "failed",
verificationFailure: selected.error,
reason:
selected.error.reason === "cancelled"
? "cancelled"
: "release_metadata",
};
const release = selected.value;
const archiveDownload = await host.download(release.filename, options);
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
if (!packageIntegrityMatches(archiveDownload.bytes, release))
return { status: "failed", reason: "integrity" };
temporary = await host.createTemporaryDirectory();
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
await host.writeArchive(archive, archiveDownload.bytes);
const verified = await host.verifyArchive(
distribution.packageFamily,
release,
archive,
options.signal,
);
if (!verified.ok)
return {
status: "failed",
verificationFailure: verified.error,
reason:
verified.error.reason === "cache_missing"
? "verification_unavailable"
: verified.error.reason,
};
const retained = await host.retainVerifiedArchive(
verified.value,
archive,
options.signal,
);
if (!retained.ok)
return {
status: "failed",
verificationFailure: retained.error,
reason:
retained.error.reason === "cache_missing"
? "verification_unavailable"
: retained.error.reason,
};
if (options.signal?.aborted === true)
return { status: "failed", reason: "cancelled" };
const install = await host.installPackage(
distribution.packageFamily,
archive,
@@ -305,7 +350,10 @@ export const installLinuxHopper = async (
packageFailure: install.failure,
};
const launcherPath = "/opt/hopper/bin/Hopper";
const launcherStatus = await host.launcherStatus(launcherPath);
const launcherStatus = await host.launcherStatus(
launcherPath,
release.main_exe_hash,
);
if (launcherStatus !== "ready")
return {
status: "failed",
@@ -360,49 +408,82 @@ export const linuxHopperLauncherCandidates = (
/** Compose package-manager and linker checks from the selected caller environment. */
export const systemLinuxHopperInstallHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): LinuxHopperInstallHost => ({
distribution: readLinuxDistribution,
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
installSystemPackage(family, archive, environment),
async launcherStatus(path) {
try {
await access(path);
const linked = await execFileOutput("ldd", [path], { env: environment });
if (!linuxSharedLibrariesAvailable(`${linked.stdout}\n${linked.stderr}`))
return "runtime_dependencies";
return (await linuxHopperBinarySupported(path))
? "ready"
: "unsupported_hopper_build";
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; unreadable means missing.
void cause;
return "missing";
}
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
});
/** Match the Linux Hopper build validated by the packaged demo-dialog adapter. */
export const linuxHopperBinarySupported = async (
path: string,
): Promise<boolean> => {
try {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return linuxHopperLauncherDigestSupported(hash.digest("hex"));
} catch (cause: unknown) {
// best-effort cleanup: optional build probing; unreadable means unsupported.
void cause;
return false;
}
): LinuxHopperInstallHost => {
const publisher = systemLinuxHopperPublisher(environment);
return {
distribution: readLinuxDistribution,
release: (family, signal) => publisher.currentRelease(family, signal),
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
installSystemPackage(family, archive, environment),
verifyArchive: (family, release, archive, signal) =>
publisher.verifyArchive(family, release, archive, signal),
retainVerifiedArchive: (verified, archive, signal) =>
publisher.retainVerifiedArchive(verified, archive, signal),
async launcherStatus(path, expectedSha256) {
try {
await access(path);
const linked = await execFileOutput("ldd", [path], {
env: environment,
});
if (
!linuxSharedLibrariesAvailable(`${linked.stdout}\n${linked.stderr}`)
)
return "runtime_dependencies";
return (await hashStableFile(path)).sha256 === expectedSha256
? "ready"
: "unsupported_hopper_build";
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; unreadable means missing.
void cause;
return "missing";
}
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
close: async () => {
await publisher.close();
await retryLinuxHopperPublisherCleanup();
},
};
};
/** Match a launcher extracted from one of the pinned official Linux packages. */
export const linuxHopperLauncherDigestSupported = (digest: string): boolean =>
SUPPORTED_HOPPER_SHA256.has(digest);
export const systemLinuxHopperPublisher = (
environment: Readonly<NodeJS.ProcessEnv>,
): LinuxHopperPublisher =>
new LinuxHopperPublisher(
environment,
async () => (await readLinuxDistribution())?.packageFamily,
);
/** Read-only authentication; persistent package retention belongs to approved setup. */
export const inspectLinuxHopperLauncher = async (
path: string,
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>> => {
const publisher = systemLinuxHopperPublisher(environment);
const result = await publisher.inspectLauncher(path, signal);
try {
await publisher.close();
await retryLinuxHopperPublisherCleanup();
return result;
} catch (cause: unknown) {
return err(
new HopperReleaseError(
"cleanup_failed",
"Hopper publisher verification retains incomplete cleanup.",
{
cause: new AggregateError(
result.ok ? [cause] : [result.error, cause],
"Publisher verification outcomes",
),
},
),
);
}
};
/** Interpret ldd output conservatively so a present but broken launcher is unhealthy. */
export const linuxSharedLibrariesAvailable = (output: string): boolean =>
@@ -492,14 +573,6 @@ export const linuxPackageManagerCommands = (
});
};
const packageIntegrityMatches = (
bytes: Uint8Array,
release: LinuxHopperRelease,
): boolean =>
bytes.byteLength === Number(release.file_length) &&
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
const packageFamilyFor = (
id: string,
idLike: string | undefined,
+833
View File
@@ -0,0 +1,833 @@
import { createHash, randomUUID } from "node:crypto";
import { copyFile, lstat, mkdir, rename, writeFile } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { z } from "zod";
import {
ArtifactResourceScope,
type ArtifactResourceOwner,
} from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
import { verifyDetachedPgpSignature } from "../process/DetachedPgpSignature.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { waitForAbortableDelay } from "../process/ProviderDeadline.js";
import {
acquireHopperReleaseCacheLease,
type HopperTargetLease,
} from "../process/HopperLease.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
import {
HOPPER_LINUX_PRIMARY_FINGERPRINT,
HOPPER_PUBLIC_KEY_URL,
HOPPER_RELEASES_URL,
HopperReleaseError,
parseLinuxHopperRelease,
parseLinuxHopperReleases,
type LinuxHopperRelease,
type LinuxPackageFamily,
} from "./HopperReleaseFeed.js";
import { withRegularFile } from "./RegularFileRead.js";
/** Metadata/key inspection budget, independent of the compressed package budget. */
const RELEASE_METADATA_BYTES = 1024 * 1024;
/** Small feed/key requests must not consume the full package-download deadline. */
const RELEASE_METADATA_TIMEOUT_MS = 10_000;
const cacheRecordSchema = z.object({
format: z.literal("rea-signed-hopper-release-v1"),
family: z.enum(["deb", "rpm", "arch"]),
release: z.unknown(),
});
export interface VerifiedLinuxHopperPackage {
readonly release: LinuxHopperRelease;
readonly family: LinuxPackageFamily;
readonly packageSha256: string;
readonly publicKey: string;
readonly primaryFingerprint: string;
readonly signingFingerprint: string;
}
export interface VerifiedLinuxHopperLauncher {
readonly sha256: string;
/** Display value from release metadata; the package signature binds bytes, not this row. */
readonly version: string;
readonly primaryFingerprint: string;
readonly signingFingerprint: string;
readonly source:
| "signed_package_cache"
| "signed_vendor_release"
| "verified_process_cache";
}
/** Retain failed ephemeral cleanup independently of the caller's lifetime. */
const pendingCleanup = new Set<ArtifactResourceScope>();
/** Immutable cryptographic facts, bounded to eight packages for this process lifetime. */
const packageProofs = new Map<
string,
Pick<VerifiedLinuxHopperPackage, "primaryFingerprint" | "signingFingerprint">
>();
const liveLauncherProofs = new Map<string, VerifiedLinuxHopperLauncher>();
const retainProof = <T>(cache: Map<string, T>, key: string, proof: T): void => {
cache.delete(key);
cache.set(key, proof);
const oldest = cache.keys().next().value;
if (cache.size > 8 && oldest !== undefined) cache.delete(oldest);
};
/** One owner for the selected cache location, shared with the approval plan. */
export const linuxHopperReleaseCacheDirectory = (
environment: Readonly<NodeJS.ProcessEnv>,
): Result<string, HopperReleaseError> => {
try {
const supplied = environment.XDG_CACHE_HOME || undefined;
if (supplied !== undefined && !isAbsolute(supplied))
return err(
new HopperReleaseError(
"verification_unavailable",
"XDG_CACHE_HOME must be an absolute filesystem path.",
),
);
return ok(
join(
supplied ??
join(homeDirectoryFromEnvironment(environment, "linux"), ".cache"),
"rea",
"hopper-releases",
),
);
} catch (cause: unknown) {
return err(
new HopperReleaseError(
"verification_unavailable",
"The Hopper release cache location could not be resolved.",
{ cause },
),
);
}
};
/** Authenticated package facts shared by setup, doctor, and the launch boundary. */
export class LinuxHopperPublisher {
readonly #resources = new ArtifactResourceScope();
#feed: ReturnType<typeof parseLinuxHopperReleases> | undefined;
constructor(
readonly environment: Readonly<NodeJS.ProcessEnv>,
readonly packageFamily: () => Promise<LinuxPackageFamily | undefined>,
) {}
get cacheRoot(): string {
const location = linuxHopperReleaseCacheDirectory(this.environment);
if (!location.ok) throw location.error;
return location.value;
}
async currentRelease(
family: LinuxPackageFamily,
signal?: AbortSignal,
): Promise<Result<LinuxHopperRelease, HopperReleaseError>> {
try {
signal?.throwIfAborted();
if (this.#feed === undefined) {
const response = await downloadHopperPackage(HOPPER_RELEASES_URL, {
...(signal === undefined ? {} : { signal }),
maxBytes: RELEASE_METADATA_BYTES,
timeoutMs: RELEASE_METADATA_TIMEOUT_MS,
});
if (!response.ok)
throw new HopperReleaseError(
"release_metadata",
"The official Hopper release feed is unavailable.",
);
this.#feed = parseLinuxHopperReleases(
JSON.parse(decodeUtf8(response.bytes)),
);
}
if (!this.#feed.ok) return this.#feed;
return this.#feed.value[family];
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "release_metadata"));
}
}
async verifyArchive(
family: LinuxPackageFamily,
release: LinuxHopperRelease,
archive: string,
signal?: AbortSignal,
suppliedPublicKey?: string,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>> {
try {
const bytes = await this.#read(archive, HOPPER_DOWNLOAD_BYTES, signal);
if (
bytes.length !== Number(release.file_length) ||
createHash("sha1").update(bytes).digest("hex") !== release.file_hash
)
return err(
new HopperReleaseError(
"integrity",
"The Hopper package does not match the selected release bytes.",
),
);
let publicKey = suppliedPublicKey;
if (publicKey === undefined) {
const response = await downloadHopperPackage(HOPPER_PUBLIC_KEY_URL, {
...(signal === undefined ? {} : { signal }),
maxBytes: RELEASE_METADATA_BYTES,
timeoutMs: RELEASE_METADATA_TIMEOUT_MS,
});
if (!response.ok)
throw new HopperReleaseError(
"verification_unavailable",
"The official Hopper public key is unavailable.",
);
publicKey = decodeUtf8(response.bytes);
}
const packageSha256 = createHash("sha256").update(bytes).digest("hex");
// Every read still validates file identity, byte bounds and selected metadata.
// Reuse only proof of the exact signed bytes/key/signature/payload interpretation.
const proofKey = createHash("sha256")
.update(
JSON.stringify([
family,
packageSha256,
publicKey,
release.signature,
release.main_exe_hash,
]),
)
.digest("hex");
signal?.throwIfAborted();
const existingProof = packageProofs.get(proofKey);
if (existingProof !== undefined)
return ok({
release,
family,
packageSha256,
publicKey,
...existingProof,
});
const verified = await verifyDetachedPgpSignature(
bytes,
Buffer.from(release.signature, "base64"),
publicKey,
HOPPER_LINUX_PRIMARY_FINGERPRINT,
signal,
);
if (!verified.ok)
return err(
new HopperReleaseError(
verified.error.reason === "cancelled"
? "cancelled"
: "publisher_verification_failed",
verified.error.message,
{ cause: verified.error },
),
);
const mainExeHash = await this.#withTemporary(async (root) => {
// Extract only after authentication, from this operation's private byte snapshot.
const snapshot = join(root, "package");
await writeFile(snapshot, bytes, { flag: "wx", mode: 0o600 });
return this.#packageLauncherHash(family, snapshot, root, signal);
});
if (mainExeHash !== release.main_exe_hash)
return err(
new HopperReleaseError(
"integrity",
`The signed ${family} package launcher SHA-256 ${mainExeHash} differs from the release main_exe_hash ${release.main_exe_hash}.`,
),
);
retainProof(packageProofs, proofKey, verified.value);
return ok({
release,
family,
packageSha256,
publicKey,
...verified.value,
});
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
/** Durable evidence contains the signed bytes; an editable digest receipt cannot authorize a build. */
async retainVerifiedArchive(
verified: VerifiedLinuxHopperPackage,
archive: string,
signal?: AbortSignal,
): Promise<Result<void, HopperReleaseError>> {
try {
signal?.throwIfAborted();
await mkdir(this.cacheRoot, { recursive: true, mode: 0o700 });
const parent = await lstat(this.cacheRoot);
if (
!parent.isDirectory() ||
parent.isSymbolicLink() ||
parent.uid !== process.getuid?.()
)
throw new HopperReleaseError(
"verification_unavailable",
"The Hopper release cache is not a caller-owned directory.",
);
const target = join(this.cacheRoot, verified.release.main_exe_hash);
const lease = await this.#acquireCacheLease(
verified.release.main_exe_hash,
signal,
);
await this.#withOwnedResource(
{
kind: "lease",
resource: lease.socketPath ?? "hopper-release-cache-lease",
lease,
},
async () => {
signal?.throwIfAborted();
await this.#withTemporary(async (stage) => {
await copyFile(archive, join(stage, "archive"));
if (
(await hashStableFile(join(stage, "archive"), signal)).sha256 !==
verified.packageSha256
)
throw new HopperReleaseError(
"integrity",
"The verified package changed before cache retention.",
);
await writeFile(join(stage, "public-key.asc"), verified.publicKey, {
flag: "wx",
mode: 0o600,
});
await writeFile(
join(stage, "release.json"),
JSON.stringify({
format: "rea-signed-hopper-release-v1",
family: verified.family,
release: verified.release,
}) + "\n",
{ flag: "wx", mode: 0o600 },
);
await this.#publishCacheStage(
stage,
target,
verified.release.main_exe_hash,
signal,
);
}, this.cacheRoot);
},
);
return ok(undefined);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async #publishCacheStage(
stage: string,
target: string,
digest: string,
signal?: AbortSignal,
): Promise<void> {
const existing = await this.#cachedPackage(digest, signal);
if (existing.ok) return;
if (existing.error.reason === "cache_missing") {
await rename(stage, target);
return;
}
if (
![
"integrity",
"release_metadata",
"publisher_verification_failed",
].includes(existing.error.reason)
)
throw existing.error;
signal?.throwIfAborted();
const previous = await lstat(target);
if (
!previous.isDirectory() ||
previous.isSymbolicLink() ||
previous.uid !== process.getuid?.()
)
throw existing.error;
// Only approved retention under the publication lease may repair an entry.
// Preserve malformed evidence without following links or deleting content.
const backup = `${target}.backup-${randomUUID()}`;
await rename(target, backup);
try {
const moved = await lstat(backup);
if (moved.dev !== previous.dev || moved.ino !== previous.ino)
throw new Error("The Hopper cache entry changed during repair.");
await rename(stage, target);
} catch (cause: unknown) {
const replacement = await this.#cachedPackage(digest, signal);
if (replacement.ok) return;
await this.#failedCachePublication(
target,
backup,
[cause, replacement.error],
replacement.error.reason === "cleanup_failed",
);
}
}
async #acquireCacheLease(
digest: string,
signal?: AbortSignal,
): Promise<HopperTargetLease> {
const deadline = performance.now() + 120_000;
const runId = randomUUID();
let delay = 25;
while (true) {
signal?.throwIfAborted();
const acquired = await acquireHopperReleaseCacheLease({
cacheDirectory: this.cacheRoot,
launcherSha256: digest,
runId,
});
if (acquired.acquired) return acquired.lease;
const remaining = deadline - performance.now();
if (remaining <= 0)
throw new HopperReleaseError(
"verification_unavailable",
"Another approved Hopper setup still owns this release cache. Retry after it finishes.",
);
await waitForAbortableDelay(Math.min(delay, remaining), signal);
delay = Math.min(delay * 2, 200);
}
}
async #failedCachePublication(
target: string,
backup: string,
causes: unknown[],
cleanupFailed = false,
): Promise<never> {
let preserved = backup;
try {
// Under the publication lease, restore the previous directory on failure.
// POSIX rename cannot replace another nonempty evidence directory.
await rename(backup, target);
preserved = target;
} catch (cause: unknown) {
causes.push(cause);
}
throw new HopperReleaseError(
cleanupFailed ? "cleanup_failed" : "verification_unavailable",
`The Hopper cache repair did not finish; previous evidence is preserved at ${preserved}. Retry approved setup.`,
{
cause: new AggregateError(
causes,
"Hopper cache publication and recovery outcomes",
),
},
);
}
async inspectLauncher(
path: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>> {
try {
const observed = await hashStableFile(path, signal);
const cached = await this.#cachedPackage(observed.sha256, signal);
if (cached.ok) {
if ((await hashStableFile(path, signal)).sha256 !== observed.sha256)
return err(
new HopperReleaseError(
"integrity",
"The Hopper launcher changed during publisher verification.",
),
);
return ok(launcherIdentity(cached.value, "signed_package_cache"));
}
if (cached.error.reason !== "cache_missing") return cached;
signal?.throwIfAborted();
const processProof = liveLauncherProofs.get(observed.sha256);
if (processProof !== undefined)
return ok({ ...processProof, source: "verified_process_cache" });
const family = await this.packageFamily();
if (family === undefined)
return err(
new HopperReleaseError(
"verification_unavailable",
"No native Linux package family is available for Hopper publisher verification.",
),
);
const selected = await this.currentRelease(family, signal);
if (!selected.ok) return selected;
if (selected.value.main_exe_hash !== observed.sha256)
return err(
new HopperReleaseError(
"unsupported_hopper_build",
"The selected Hopper launcher does not match an authenticated cached release or the official current Linux demo.",
),
);
const verified = await this.#withTemporary(async (root) => {
const response = await downloadHopperPackage(
selected.value.filename,
signal === undefined ? {} : { signal },
);
if (!response.ok)
throw new HopperReleaseError(
"download",
"The official Hopper package is unavailable for publisher verification.",
);
const archive = join(root, "archive");
await writeFile(archive, response.bytes, { flag: "wx", mode: 0o600 });
const verified = await this.verifyArchive(
family,
selected.value,
archive,
signal,
);
if (!verified.ok) throw verified.error;
return verified.value;
});
if ((await hashStableFile(path, signal)).sha256 !== observed.sha256)
return err(
new HopperReleaseError(
"integrity",
"The Hopper launcher changed during publisher verification.",
),
);
const identity = launcherIdentity(verified, "signed_vendor_release");
retainProof(liveLauncherProofs, observed.sha256, identity);
return ok(identity);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async close(): Promise<void> {
try {
await this.#resources.close();
pendingCleanup.delete(this.#resources);
} catch (cause: unknown) {
pendingCleanup.add(this.#resources);
throw cause;
}
}
async #cachedPackage(
digest: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>> {
try {
const root = join(this.cacheRoot, digest);
try {
const directory = await lstat(root);
if (!directory.isDirectory() || directory.isSymbolicLink())
throw new HopperReleaseError(
"integrity",
"The retained Hopper cache entry is not a regular directory.",
);
} catch (cause: unknown) {
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT") {
return err(
new HopperReleaseError(
"cache_missing",
"No retained signed Hopper package identifies this launcher.",
{ cause },
),
);
}
throw cause;
}
// Distinguish proven malformed/missing evidence from unavailable reads or
// inspection tools. Only the former authorize approved cache replacement.
for (const name of ["release.json", "public-key.asc", "archive"])
await this.#assertCacheFile(join(root, name));
const metadata = await this.#read(
join(root, "release.json"),
RELEASE_METADATA_BYTES,
signal,
);
let value: unknown;
try {
value = JSON.parse(decodeUtf8(metadata));
} catch (cause: unknown) {
throw new HopperReleaseError(
"release_metadata",
"The retained Hopper package metadata is malformed.",
{ cause },
);
}
const parsed = cacheRecordSchema.safeParse(value);
if (!parsed.success)
throw new HopperReleaseError(
"release_metadata",
"The retained Hopper package metadata is malformed.",
);
const release = parseLinuxHopperRelease(
parsed.data.release,
parsed.data.family,
);
if (!release.ok) return release;
if (release.value.main_exe_hash !== digest)
throw new HopperReleaseError(
"integrity",
"The retained Hopper metadata does not identify this launcher.",
);
const keyBytes = await this.#read(
join(root, "public-key.asc"),
RELEASE_METADATA_BYTES,
signal,
);
let key: string;
try {
key = decodeUtf8(keyBytes);
} catch (cause: unknown) {
throw new HopperReleaseError(
"publisher_verification_failed",
"The retained Hopper public key is malformed.",
{ cause },
);
}
return this.verifyArchive(
parsed.data.family,
release.value,
join(root, "archive"),
signal,
key,
);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async #assertCacheFile(path: string): Promise<void> {
try {
const stats = await lstat(path);
if (!stats.isFile() || stats.isSymbolicLink())
throw new HopperReleaseError(
"integrity",
`Required Hopper evidence is not a regular file: ${path}.`,
);
} catch (cause: unknown) {
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
throw new HopperReleaseError(
"integrity",
`Required Hopper evidence is missing: ${path}.`,
{ cause },
);
throw cause;
}
}
async #packageLauncherHash(
family: LinuxPackageFamily,
archive: string,
root: string,
signal?: AbortSignal,
): Promise<string> {
const commandOptions = {
env: this.environment,
signal,
timeout: 120_000,
stopSignal: "SIGKILL" as const,
maxBuffer: RELEASE_METADATA_BYTES,
};
if (family === "rpm") {
const output = await execFileOutput(
"rpm",
[
"-qp",
"--nosignature",
"--nodigest",
"--qf",
"%{FILEDIGESTALGO}\n[%{FILENAMES}\t%{FILEDIGESTS}\n]",
archive,
],
commandOptions,
);
const rows = output.stdout.split("\n");
if (rows.shift() !== "8")
throw new HopperReleaseError(
"integrity",
"The signed RPM header does not use SHA-256 file digests.",
);
const matches = rows.filter((row) =>
row.startsWith("/opt/hopper/bin/Hopper\t"),
);
const digest = matches[0]?.split("\t")[1];
if (
matches.length !== 1 ||
digest === undefined ||
!/^[a-f0-9]{64}$/u.test(digest)
)
throw new HopperReleaseError(
"integrity",
"The signed RPM header has no unique Hopper launcher digest.",
);
return digest;
}
const extracted = join(root, "extracted");
await mkdir(extracted, { mode: 0o700 });
if (family === "deb")
await execFileOutput(
"dpkg-deb",
["--extract", archive, extracted],
commandOptions,
);
else
await execFileOutput(
"tar",
["-xf", archive, "-C", extracted, "--", "opt/hopper/bin/Hopper"],
commandOptions,
);
return (
await hashStableFile(
join(extracted, "opt", "hopper", "bin", "Hopper"),
signal,
)
).sha256;
}
#read(path: string, limit: number, signal?: AbortSignal): Promise<Buffer> {
return withRegularFile(
path,
async (handle, stats) => {
if (stats.size > limit)
throw new HopperReleaseError(
"integrity",
"Retained Hopper evidence exceeds its resource byte budget.",
);
const bytes = await readBoundedFileBytes(handle, stats.size, signal);
if (bytes === undefined)
throw new HopperReleaseError(
"integrity",
"Retained Hopper evidence changed during inspection.",
);
return bytes;
},
{ signal, resources: this.#resources, symlinks: "reject" },
);
}
async #withTemporary<T>(
operation: (path: string) => Promise<T>,
parent?: string,
): Promise<T> {
const root = await PrivateRuntimeRoot.create({
prefix: "rea-hopper-publisher-",
...(parent === undefined ? {} : { parent }),
});
return this.#withOwnedResource(
{
kind: "private-runtime-root",
resource: root.path,
root,
},
() => operation(root.path),
);
}
async #withOwnedResource<T>(
owner: ArtifactResourceOwner,
operation: () => Promise<T>,
): Promise<T> {
let outcome:
| { readonly ok: true; readonly value: T }
| { readonly ok: false; readonly cause: unknown };
try {
outcome = { ok: true, value: await operation() };
} catch (cause: unknown) {
outcome = { ok: false, cause };
}
const cleanup = await this.#resources.release(owner);
if (cleanup.kind === "failed") {
pendingCleanup.add(this.#resources);
const causes = outcome.ok
? [cleanup.cause]
: [outcome.cause, cleanup.cause];
throw new HopperReleaseError(
"cleanup_failed",
`Hopper publisher verification retained cleanup ownership for ${owner.resource}.`,
{
cause: new AggregateError(
causes,
"Publisher inspection and cleanup outcomes",
),
},
);
}
if (!outcome.ok) throw outcome.cause;
return outcome.value;
}
}
/** Retry only resources retained by this producer after failed cleanup. */
export const retryLinuxHopperPublisherCleanup = async (): Promise<void> => {
const causes: unknown[] = [];
const observations = [];
for (const scope of pendingCleanup) {
try {
await scope.close();
pendingCleanup.delete(scope);
} catch (cause: unknown) {
causes.push(cause);
observations.push(
ArtifactReaderFailure.cleanupObservation(
cause,
"hopper-publisher-verification",
),
);
}
}
if (causes.length > 0) {
const cleanup = {
reason: observations.map(({ reason }) => reason).join("; "),
resources: [
...new Set(observations.flatMap(({ resources }) => resources)),
],
};
throw new ArtifactReaderFailure(
"unavailable",
`Hopper publisher cleanup remains incomplete: ${cleanup.resources.join(", ")}`,
{
cause: new AggregateError(causes, "Hopper publisher cleanup failures"),
cleanup,
},
);
}
};
const decodeUtf8 = (bytes: Uint8Array): string =>
new TextDecoder("utf-8", { fatal: true }).decode(bytes);
const releaseFailure = (
cause: unknown,
signal: AbortSignal | undefined,
fallback: HopperReleaseError["reason"],
): HopperReleaseError =>
cause instanceof HopperReleaseError && cause.reason === "cleanup_failed"
? cause
: signal?.aborted === true
? new HopperReleaseError(
"cancelled",
"Hopper publisher verification was cancelled.",
{ cause },
)
: cause instanceof HopperReleaseError
? cause
: new HopperReleaseError(
fallback,
"Hopper publisher verification could not complete.",
{ cause },
);
const launcherIdentity = (
verified: VerifiedLinuxHopperPackage,
source: VerifiedLinuxHopperLauncher["source"],
): VerifiedLinuxHopperLauncher => ({
sha256: verified.release.main_exe_hash,
version: verified.release.version,
primaryFingerprint: verified.primaryFingerprint,
signingFingerprint: verified.signingFingerprint,
source,
});
+78 -6
View File
@@ -4,7 +4,9 @@ import { describe, expect, it } from "vitest";
import {
installMacHopper,
installMacHopperBundle,
type MacHopperBundleVerification,
type MacHopperBundleInstallResult,
type MacHopperInstallHost,
} from "./MacHopper.js";
@@ -15,7 +17,8 @@ class FakeMacHopperHost implements MacHopperInstallHost {
existing = false;
mountSucceeds = true;
copySucceeds = true;
launcherSucceeds = true;
mountFailure: Error | undefined;
verificationFailure: Error | undefined;
badHash = false;
unmounts = 0;
cleanups = 0;
@@ -46,25 +49,37 @@ class FakeMacHopperHost implements MacHopperInstallHost {
createTemporaryDirectory = () => Promise.resolve("/tmp/rea-hopper-test");
createMountDirectory = () => Promise.resolve();
writePackage = () => Promise.resolve();
mount = () => Promise.resolve(this.mountSucceeds);
mount = () =>
this.mountFailure === undefined
? Promise.resolve(this.mountSucceeds)
: Promise.reject(this.mountFailure);
unmount = () => {
this.unmounts += 1;
return Promise.resolve();
};
appBundle = () => Promise.resolve("/mounted/Hopper Disassembler.app");
validBundle = () => Promise.resolve(this.verification);
validBundle = () =>
this.verificationFailure === undefined
? Promise.resolve(this.verification)
: Promise.reject(this.verificationFailure);
destinationExists = () => Promise.resolve(this.existing);
installBundle = (
_source: string,
_destination: string,
_stage: string,
replaceExisting: boolean,
) => {
this.installs += 1;
this.replaces = replaceExisting;
return Promise.resolve(this.copySucceeds);
return Promise.resolve<MacHopperBundleInstallResult>(
this.copySucceeds
? { status: "installed" }
: {
status: "failed",
reason: "copy",
diagnostic: "ditto: destination is read-only",
},
);
};
launcherReady = () => Promise.resolve(this.launcherSucceeds);
openApplication = () => {
this.opens += 1;
return Promise.resolve();
@@ -89,6 +104,22 @@ describe("macOS Hopper installation", () => {
expect(host.cleanups).toBe(1);
});
it("preserves installation when macOS cannot open the installed app", async () => {
const host = new FakeMacHopperHost();
host.openApplication = () =>
Promise.reject(new Error("LaunchServices refused"));
await expect(installMacHopper({}, host)).resolves.toEqual({
status: "installed",
launcherPath:
"/home/user/Applications/Hopper Disassembler.app/Contents/MacOS/hopper",
openFailure: "LaunchServices refused",
});
expect(host.installs).toBe(1);
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
it("rejects an integrity mismatch before mounting", async () => {
const host = new FakeMacHopperHost();
host.badHash = true;
@@ -130,6 +161,33 @@ describe("macOS Hopper installation", () => {
expect(host.cleanups).toBe(1);
});
it("preserves a thrown mount-stage diagnostic and cleans the temporary image", async () => {
const host = new FakeMacHopperHost();
host.mountFailure = new Error("hdiutil attach: resource busy");
await expect(installMacHopper({}, host)).resolves.toMatchObject({
status: "failed",
reason: "mount",
diagnostic: "hdiutil attach: resource busy",
});
expect(host.unmounts).toBe(0);
expect(host.cleanups).toBe(1);
});
it("preserves the copy diagnostic from the owned install operation", async () => {
const host = new FakeMacHopperHost();
host.copySucceeds = false;
await expect(installMacHopper({}, host)).resolves.toMatchObject({
status: "failed",
reason: "copy",
diagnostic: "ditto: destination is read-only",
});
expect(host.installs).toBe(1);
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
it("reports a publisher mismatch without copying or launching the bundle", async () => {
const host = new FakeMacHopperHost();
host.verification = "unverified_publisher";
@@ -142,4 +200,18 @@ describe("macOS Hopper installation", () => {
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
it("preserves a thrown signature-check diagnostic as a bundle failure", async () => {
const host = new FakeMacHopperHost();
host.verificationFailure = new Error("codesign: helper unavailable");
await expect(installMacHopper({}, host)).resolves.toMatchObject({
status: "failed",
reason: "bundle",
diagnostic: "codesign: helper unavailable",
});
expect(host.installs).toBe(0);
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
});
+240 -90
View File
@@ -10,7 +10,7 @@ import {
writeFile,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { z } from "zod";
@@ -22,6 +22,7 @@ import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
import type { HopperInstallFailureReason } from "./SetupInstallFailure.js";
const execFileAsync = promisify(execFile);
const RELEASES_URL =
@@ -38,6 +39,38 @@ export type MacHopperBundleVerification =
| "invalid_signature"
| "unverified_publisher";
type MacHopperInstallFailureReason = Extract<
HopperInstallFailureReason,
| "release_metadata"
| "download"
| "integrity"
| "launcher_missing"
| "destination_exists"
| "mount"
| "bundle"
| "publisher_verification_failed"
| "copy"
>;
export type MacHopperBundleInstallResult =
| { readonly status: "installed"; readonly cleanupFailure?: string }
| {
readonly status: "failed";
readonly reason: "copy" | "launcher_missing";
readonly diagnostic: string;
readonly cleanupFailure?: string;
};
/** Filesystem effects for one installer-owned bundle replacement transaction. */
export interface MacHopperBundleInstallOperations {
createStagingRoot(parent: string): Promise<string>;
copyBundle(source: string, stage: string): Promise<void>;
requireLauncher(path: string): Promise<void>;
pathExists(path: string): Promise<boolean>;
rename(source: string, destination: string): Promise<void>;
removeStagingRoot(path: string): Promise<void>;
}
/** Stable network and command effects disclosed before macOS Hopper setup. */
export const macHopperInstallDisclosure = {
networkOrigins: [RELEASES_URL, DOWNLOAD_PREFIX] as const,
@@ -56,19 +89,12 @@ export type MacHopperInstallResult =
readonly status: "installed";
readonly launcherPath: string;
readonly cleanupFailure?: string;
readonly openFailure?: string;
}
| {
readonly status: "failed";
readonly reason:
| "release_metadata"
| "download"
| "integrity"
| "mount"
| "bundle"
| "publisher_verification_failed"
| "destination_exists"
| "copy"
| "launcher_missing";
readonly reason: MacHopperInstallFailureReason;
readonly diagnostic?: string;
readonly cleanupFailure?: string;
};
@@ -88,10 +114,8 @@ export interface MacHopperInstallHost {
installBundle(
source: string,
destination: string,
stage: string,
replaceExisting: boolean,
): Promise<boolean>;
launcherReady(path: string): Promise<boolean>;
): Promise<MacHopperBundleInstallResult>;
openApplication(path: string): Promise<void>;
cleanup(path: string): Promise<void>;
/** Retain the cleanup capability until the owning setup host retries it. */
@@ -109,6 +133,8 @@ export const installMacHopper = async (
): Promise<MacHopperInstallResult> => {
let temporary: string | undefined;
let mounted = false;
let bundleCleanupFailure: string | undefined;
let installStage: MacHopperInstallFailureReason = "release_metadata";
let result: MacHopperInstallResult;
try {
result = await (async (): Promise<MacHopperInstallResult> => {
@@ -120,15 +146,19 @@ export const installMacHopper = async (
const release = parsed.data["OS X"];
if (!release.filename.startsWith(DOWNLOAD_PREFIX))
return { status: "failed", reason: "release_metadata" };
installStage = "download";
const archive = await host.download(release.filename);
if (!archive.ok) return { status: "failed", reason: "download" };
installStage = "integrity";
if (!integrityMatches(archive.bytes, release))
return { status: "failed", reason: "integrity" };
installStage = "destination_exists";
const destination = host.destination();
const destinationExists = await host.destinationExists(destination);
if (destinationExists && options.replaceExisting !== true)
return { status: "failed", reason: "destination_exists" };
installStage = "mount";
temporary = await host.createTemporaryDirectory();
const packagePath = join(temporary, "hopper.dmg");
const mountPath = join(temporary, "mounted");
@@ -137,6 +167,7 @@ export const installMacHopper = async (
if (!(await host.mount(packagePath, mountPath)))
return { status: "failed", reason: "mount" };
mounted = true;
installStage = "bundle";
const source = await host.appBundle(mountPath);
if (source === undefined) return { status: "failed", reason: "bundle" };
const verification = await host.validBundle(source);
@@ -144,26 +175,41 @@ export const installMacHopper = async (
return { status: "failed", reason: "bundle" };
if (verification === "unverified_publisher")
return { status: "failed", reason: "publisher_verification_failed" };
const stage = `${destination}.rea-stage`;
if (
!(await host.installBundle(
source,
destination,
stage,
destinationExists,
))
)
return { status: "failed", reason: "copy" };
installStage = "copy";
const installation = await host.installBundle(
source,
destination,
destinationExists,
);
if (installation.status === "failed")
return {
status: "failed",
reason: installation.reason,
diagnostic: installation.diagnostic,
...(installation.cleanupFailure === undefined
? {}
: { cleanupFailure: installation.cleanupFailure }),
};
bundleCleanupFailure = installation.cleanupFailure;
const launcherPath = join(destination, "Contents/MacOS/hopper");
if (!(await host.launcherReady(launcherPath)))
return { status: "failed", reason: "launcher_missing" };
await host.openApplication(destination);
return { status: "installed", launcherPath };
let openFailure: string | undefined;
try {
await host.openApplication(destination);
} catch (cause: unknown) {
openFailure = failureMessage(cause);
}
return {
status: "installed",
launcherPath,
...(openFailure === undefined ? {} : { openFailure }),
};
})();
} catch (cause: unknown) {
// Install-stage failures share the download reason; the stage is cleaned below.
void cause;
result = { status: "failed", reason: "download" };
result = {
status: "failed",
reason: installStage,
diagnostic: failureMessage(cause),
};
}
let cleanupFailure: string | undefined;
@@ -185,7 +231,106 @@ export const installMacHopper = async (
}
}
return cleanupFailure === undefined ? result : { ...result, cleanupFailure };
const retainedCleanupFailure = [
bundleCleanupFailure,
cleanupFailure,
result.cleanupFailure,
]
.filter((failure): failure is string => failure !== undefined)
.join(" ");
return retainedCleanupFailure === ""
? result
: { ...result, cleanupFailure: retainedCleanupFailure };
};
/** Replace a bundle through a fresh same-volume root owned by this transaction. */
export const installMacHopperBundle = async (
source: string,
destination: string,
replaceExisting: boolean,
operations: MacHopperBundleInstallOperations,
retainCleanup: (cleanup: () => Promise<void>) => void,
): Promise<MacHopperBundleInstallResult> => {
const stagingRoot = await operations.createStagingRoot(dirname(destination));
const stagedBundle = join(stagingRoot, "new.app");
const previousBundle = join(stagingRoot, "previous.app");
let originalBackedUp = false;
let newCommitted = false;
let failureReason: "copy" | "launcher_missing" = "copy";
try {
await operations.copyBundle(source, stagedBundle);
failureReason = "launcher_missing";
await operations.requireLauncher(
join(stagedBundle, "Contents/MacOS/hopper"),
);
failureReason = "copy";
if (replaceExisting) {
await operations.rename(destination, previousBundle);
originalBackedUp = true;
}
await operations.rename(stagedBundle, destination);
newCommitted = true;
} catch (cause: unknown) {
const diagnostics = [failureMessage(cause)];
if (originalBackedUp && !newCommitted) {
try {
if (await operations.pathExists(destination))
throw new Error(
"destination now contains another installation; preserving the previous bundle in its owned staging root",
);
await operations.rename(previousBundle, destination);
originalBackedUp = false;
} catch (rollbackCause: unknown) {
diagnostics.push(`rollback failed: ${failureMessage(rollbackCause)}`);
const cleanupFailure = `The previous Hopper bundle remains owned at ${previousBundle}; cleanup will retry restoration only if the destination is empty.`;
retainCleanup(async () => {
if (originalBackedUp) {
if (await operations.pathExists(destination))
throw new Error(
`Hopper destination is occupied; preserving owned backup ${previousBundle}`,
);
await operations.rename(previousBundle, destination);
originalBackedUp = false;
}
await operations.removeStagingRoot(stagingRoot);
});
return {
status: "failed",
reason: failureReason,
diagnostic: diagnostics.join("; "),
cleanupFailure,
};
}
}
try {
await operations.removeStagingRoot(stagingRoot);
} catch (cleanupCause: unknown) {
const cleanupFailure = `Hopper staging root cleanup failed: ${failureMessage(cleanupCause)}`;
retainCleanup(() => operations.removeStagingRoot(stagingRoot));
diagnostics.push(cleanupFailure);
return {
status: "failed",
reason: failureReason,
diagnostic: diagnostics.join("; "),
cleanupFailure,
};
}
return {
status: "failed",
reason: failureReason,
diagnostic: diagnostics.join("; "),
};
}
try {
await operations.removeStagingRoot(stagingRoot);
return { status: "installed" };
} catch (cause: unknown) {
const cleanupFailure = `Hopper staging root cleanup failed after installation committed: ${failureMessage(cause)}`;
retainCleanup(() => operations.removeStagingRoot(stagingRoot));
return { status: "installed", cleanupFailure };
}
};
/** Compose installer paths and subprocesses from one selected caller context. */
@@ -200,7 +345,7 @@ export const systemMacHopperInstallHost = (
createMountDirectory: (path) => mkdir(path),
writePackage: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
async mount(packagePath, mountPath) {
return commandSucceeds(
await execFileAsync(
"hdiutil",
[
"attach",
@@ -210,20 +355,14 @@ export const systemMacHopperInstallHost = (
mountPath,
packagePath,
],
environment,
{ env: environment },
);
return true;
},
async unmount(mountPath) {
if (
!(await commandSucceeds(
"hdiutil",
["detach", mountPath, "-force"],
environment,
))
)
throw new Error(
`hdiutil detach failed for ${JSON.stringify(mountPath)}`,
);
await execFileAsync("hdiutil", ["detach", mountPath, "-force"], {
env: environment,
});
},
async appBundle(mountPath) {
const entries = await readdir(mountPath, { withFileTypes: true });
@@ -261,49 +400,32 @@ export const systemMacHopperInstallHost = (
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional existence probing; absence means missing.
void cause;
return false;
}
},
async installBundle(source, destination, stage, replaceExisting) {
const previous = `${stage}-previous`;
try {
await mkdir(join(homeDirectory, "Applications"), { recursive: true });
await rm(stage, { recursive: true, force: true });
await rm(previous, { recursive: true, force: true });
await execFileAsync("ditto", [source, stage], { env: environment });
await access(join(stage, "Contents/MacOS/hopper"));
if (replaceExisting) await rename(destination, previous);
await rename(stage, destination);
if (replaceExisting)
await rm(previous, { recursive: true, force: true });
return true;
} catch (cause: unknown) {
// Install failure rolls back below; the false return reports it.
void cause;
await rm(stage, { recursive: true, force: true });
if (replaceExisting) {
try {
await rename(previous, destination);
} catch (rollbackCause: unknown) {
// The original destination was not moved, or rollback is impossible.
void rollbackCause;
}
}
return false;
}
},
async launcherReady(path) {
try {
await access(path);
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; absence means not ready.
void cause;
return false;
if (isMissingPath(cause)) return false;
throw cause;
}
},
installBundle: (source, destination, replaceExisting) =>
installMacHopperBundle(
source,
destination,
replaceExisting,
{
async createStagingRoot(parent) {
await mkdir(parent, { recursive: true });
return mkdtemp(join(parent, ".rea-hopper-stage-"));
},
copyBundle: async (bundle, staged) => {
await execFileAsync("ditto", [bundle, staged], {
env: environment,
});
},
requireLauncher: (path) => access(path).then(() => undefined),
pathExists,
rename,
removeStagingRoot: (path) => rm(path, { recursive: true }),
},
(cleanup) => pendingCleanup.add(cleanup),
),
async openApplication(path) {
await execFileAsync("open", [path], { env: environment });
},
@@ -316,7 +438,7 @@ export const systemMacHopperInstallHost = (
await cleanup();
pendingCleanup.delete(cleanup);
} catch (cause: unknown) {
failure ??= `Hopper DMG cleanup remains incomplete: ${failureMessage(cause)}`;
failure ??= `Hopper installer cleanup remains incomplete: ${failureMessage(cause)}`;
}
}
return failure;
@@ -343,13 +465,41 @@ const commandSucceeds = async (
await execFileAsync(command, args, { env: environment });
return true;
} catch (cause: unknown) {
// best-effort cleanup: optional command probing; failure means unavailable.
void cause;
return false;
if (
typeof cause === "object" &&
cause !== null &&
"code" in cause &&
typeof cause.code === "number"
)
return false;
throw cause;
}
};
const failureMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
cause instanceof Error
? [
cause.message,
"stderr" in cause && typeof cause.stderr === "string"
? cause.stderr.trim()
: "",
]
.filter((value) => value !== "")
.join(": ")
: String(cause);
const isMissingPath = (cause: unknown): boolean =>
typeof cause === "object" &&
cause !== null &&
"code" in cause &&
cause.code === "ENOENT";
const pathExists = async (path: string): Promise<boolean> => {
try {
await access(path);
return true;
} catch (cause: unknown) {
if (isMissingPath(cause)) return false;
throw cause;
}
};
const releaseSchema = z.object({
filename: z.string().url(),
file_length: z.string().regex(/^\d+$/u),
@@ -413,7 +413,22 @@ const addCandidate = (
...current.authority_references,
...candidate.authority_references,
]),
definitionCount: current.definitionCount + candidate.definitionCount,
// Each observation keeps its own unresolved authority and next steps.
unavailable_authority: uniqueBy(
[...current.unavailable_authority, ...candidate.unavailable_authority],
(item) => item,
),
required_next_evidence: uniqueBy(
[...current.required_next_evidence, ...candidate.required_next_evidence],
(item) => item,
),
// A generated ID digests its derivation key, so two generated candidates
// with one ID are the same claim seen by several records. Only another
// reviewed definition can introduce a competing definition.
definitionCount:
current.origin === "generated" && candidate.origin === "generated"
? current.definitionCount
: current.definitionCount + candidate.definitionCount,
});
};
@@ -12,6 +12,62 @@ import {
import { deriveReconstructionObligationCandidates } from "./ReconstructionObligationCandidates.js";
import { evaluateReconstructionObligationLedger } from "./ReconstructionObligationLedgerEvaluation.js";
/** Build a request whose packaged-process obligation closes as ready. */
const readyProcessLedgerRequest = () => {
const capture = processEvidence();
const initial = build(request([capture]));
const obligation = initial.obligations[0];
if (obligation === undefined) throw new Error("Expected process obligation");
const fixtureIds = obligation.required_case_kinds.map(
(caseKind) => `packaged.${caseKind}`,
);
const proof = boundProofEvidence("packaged", {
obligation_ids: [obligation.obligation_id],
fixture_ids: fixtureIds,
case_kinds: obligation.required_case_kinds,
verifier_ids: ["verifier.packaged"],
claim_ids: ["claim.process"],
});
const ledgerRequest = request([capture, proof], {
manifest: {
bindings: [
{
obligation_id: obligation.obligation_id,
owner: {
module_path: "src/processOwner.ts",
symbol: "runProcess",
owner_sha256: "2".repeat(64),
},
parser_type: null,
original_cases: originalCases(obligation, capture.evidence_id),
fixtures: obligation.required_case_kinds.map((caseKind) => ({
fixture_id: `packaged.${caseKind}`,
case_kind: caseKind,
authority: "packaged-process",
evidence_ids: [proof.evidence_id],
})),
verifier: {
verifier_id: "verifier.packaged",
claim_id: "claim.process",
command: "npm run verify:package -- process",
authority: "packaged-process",
status: "pass",
result_evidence_id: proof.evidence_id,
enumerated_obligation_ids: [obligation.obligation_id],
nondeterminism: {
mode: "partial-order",
specification:
"Startup precedes teardown; worker completion is unordered.",
},
},
},
],
contradictions: [],
},
});
return ledgerRequest;
};
describe("reconstruction obligation ledger", () => {
it("emits the complete deterministic ledger and closure identity", () => {
const evidence = proofEvidence("review");
@@ -31,6 +87,49 @@ describe("reconstruction obligation ledger", () => {
expect(first.status).toBe("open");
});
it("merges repeated captures of one process scenario into one definition", () => {
const first = processEvidence();
const second = processEvidence({
residual_unknowns: [{ scope: "network", reason: "Second run only." }],
});
expect(second.evidence_id).not.toBe(first.evidence_id);
const single = build(request([first]));
const repeated = build(request([first, second]));
const obligation = repeated.obligations[0];
expect(repeated.obligations).toHaveLength(1);
expect(obligation?.obligation_id).toBe(
single.obligations[0]?.obligation_id,
);
expect(obligation?.diagnostics.map(({ code }) => code)).not.toContain(
"duplicate-definition",
);
expect(
obligation?.authority_references.map(({ evidence_id: id }) => id).sort(),
).toEqual([first.evidence_id, second.evidence_id].sort());
// The second capture's own uncertainty stays visible after the merge.
expect(obligation?.unavailable_authority).toContain(
"network: Second run only.",
);
});
it("still reports a reviewed definition that reuses a generated ID", () => {
const capture = processEvidence();
const generated = build(request([capture])).obligations[0];
if (generated === undefined) throw new Error("Expected process obligation");
const ledger = build(
request([capture], {
reviewed_obligations: [
reviewedObligation(generated.obligation_id, capture.evidence_id),
],
}),
);
expect(
ledger.obligations[0]?.diagnostics.map(({ code }) => code),
).toContain("duplicate-definition");
});
it("does not let green unit evidence close a packaged-process obligation", () => {
const capture = processEvidence();
const proof = proofEvidence("green-unit");
@@ -181,58 +280,7 @@ describe("reconstruction obligation ledger case closure", () => {
});
it("closes only when every required case and verifier has comparable authority", () => {
const capture = processEvidence();
const initial = build(request([capture]));
const obligation = initial.obligations[0];
if (obligation === undefined)
throw new Error("Expected process obligation");
const fixtureIds = obligation.required_case_kinds.map(
(caseKind) => `packaged.${caseKind}`,
);
const proof = boundProofEvidence("packaged", {
obligation_ids: [obligation.obligation_id],
fixture_ids: fixtureIds,
case_kinds: obligation.required_case_kinds,
verifier_ids: ["verifier.packaged"],
claim_ids: ["claim.process"],
});
const ledgerRequest = request([capture, proof], {
manifest: {
bindings: [
{
obligation_id: obligation.obligation_id,
owner: {
module_path: "src/processOwner.ts",
symbol: "runProcess",
owner_sha256: "2".repeat(64),
},
parser_type: null,
original_cases: originalCases(obligation, capture.evidence_id),
fixtures: obligation.required_case_kinds.map((caseKind) => ({
fixture_id: `packaged.${caseKind}`,
case_kind: caseKind,
authority: "packaged-process",
evidence_ids: [proof.evidence_id],
})),
verifier: {
verifier_id: "verifier.packaged",
claim_id: "claim.process",
command: "npm run verify:package -- process",
authority: "packaged-process",
status: "pass",
result_evidence_id: proof.evidence_id,
enumerated_obligation_ids: [obligation.obligation_id],
nondeterminism: {
mode: "partial-order",
specification:
"Startup precedes teardown; worker completion is unordered.",
},
},
},
],
contradictions: [],
},
});
const ledgerRequest = readyProcessLedgerRequest();
const result = build(ledgerRequest);
expect(result.obligations[0]).toMatchObject({
@@ -257,3 +305,54 @@ describe("reconstruction obligation ledger case closure", () => {
expect(limited.coverage.status).toBe("partial");
});
});
describe("reconstruction obligation manifest references", () => {
it("withholds readiness when a manifest entry names absent Evidence", () => {
const ledgerRequest = readyProcessLedgerRequest();
// A valid entry for the same case kind must not hide an absent reference.
const absent = `ev_${"a".repeat(64)}`;
const binding = ledgerRequest.manifest.bindings[0];
if (binding === undefined) throw new Error("Expected binding");
const variants: {
readonly code: string;
readonly binding: typeof binding;
}[] = [
{
code: "missing-original-case",
binding: {
...binding,
original_cases: [
...binding.original_cases,
{ kind: "positive", evidence_id: absent, location: "/old" },
],
},
},
{
code: "missing-case",
binding: {
...binding,
fixtures: [
...binding.fixtures,
{
fixture_id: "packaged.old",
case_kind: "positive",
authority: "packaged-process",
evidence_ids: [absent],
},
],
},
},
];
for (const { code, binding: dangling } of variants) {
const open = build({
...ledgerRequest,
manifest: { ...ledgerRequest.manifest, bindings: [dangling] },
});
expect(open.status).not.toBe("ready");
expect(open.obligations[0]?.status).toBe("implemented");
expect(open.obligations[0]?.diagnostics).toEqual([
{ code, detail: expect.stringContaining(absent) },
]);
}
});
});
@@ -14,7 +14,10 @@ import {
type ReviewedReconstructionObligation,
} from "../domain/reconstructionObligationLedgerSchemas.js";
import { processCaptureSchema } from "../domain/process/processCapture.js";
import {
processCaptureSchema,
type UnverifiedProcessCapture,
} from "../domain/process/processCapture.js";
/** Build controlled fixture Evidence used by reconstruction-ledger scenarios. */
export const proofEvidence = (id: string) =>
createEvidence(
@@ -31,13 +34,18 @@ export const proofEvidence = (id: string) =>
);
/** Use the canonical empty process capture as a controlled Evidence example. */
export const processEvidence = () =>
export const processEvidence = (
overrides: Partial<UnverifiedProcessCapture> = {},
) =>
createEvidence(undefined, PROCESS_PROVIDER, {
predicateType: "rea.process-capture",
operation: "capture_process_scenario",
parameters: {},
result: jsonValueSchema.parse(
processCaptureSchema.parse(EMPTY_PROCESS_CAPTURE_EXAMPLE),
processCaptureSchema.parse({
...EMPTY_PROCESS_CAPTURE_EXAMPLE,
...overrides,
}),
),
confidence: "observed",
authority: "controlled-replay",
@@ -268,6 +268,14 @@ const evaluateBinding = (
);
continue;
}
for (const fixture of fixtures)
for (const evidenceId of fixture.evidence_ids)
if (!context.evidence.has(evidenceId))
addDiagnostic(
diagnostics,
"missing-case",
`${caseKind} fixture ${fixture.fixture_id} references Evidence ${evidenceId}, which is absent from the bundle.`,
);
if (
!fixtures.some(
(fixture) =>
@@ -45,6 +45,14 @@ export const originalCaseDiagnostics = (
});
continue;
}
// A valid entry for this case kind does not resolve another entry's
// reference; every referenced Evidence ID must be in the bundle.
for (const { evidence_id: evidenceId } of cases)
if (!evidenceById.has(evidenceId))
diagnostics.push({
code: "missing-original-case",
detail: `Original ${caseKind} case references Evidence ${evidenceId}, which is absent from the bundle.`,
});
if (
!cases.some(({ evidence_id: evidenceId }) => {
const evidence = evidenceById.get(evidenceId);
@@ -110,9 +110,11 @@ const resolveInternalSpecifier = (
const directoryReference =
specifier.endsWith("/") || [".", ".."].includes(posix.basename(specifier));
const suffixes = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
// A directory resolves only to an index with an extension; neither Node nor
// TypeScript selects an extensionless `index` file.
const candidates = suffixes.flatMap((suffix) => [
...(directoryReference ? [] : [`${normalized}${suffix}`]),
posix.join(normalized, `index${suffix}`),
...(suffix === "" ? [] : [posix.join(normalized, `index${suffix}`)]),
]);
// Retain present runtime files first; substitute only a missing counterpart.
// Counterparts cover explicit filenames, not directory/index resolution.
+4 -2
View File
@@ -206,9 +206,10 @@ describe("configured Linux Hopper selection", () => {
host.hopperLauncherConfigured = true;
host.linuxDemoRuntimeMissing = true;
const doctor = await host.doctor();
const initial = await host.initialSetupState();
const discovery = await discoverSetupState({
host,
providerEnvironment: { HOPPER_LAUNCHER_PATH: host.hopper },
providerConfigured: initial.providerConfigured,
forceHopperInstall: false,
proposeHopper: true,
hopperLauncherConfigured: true,
@@ -227,9 +228,10 @@ describe("configured Linux Hopper selection", () => {
};
host.hopper = "/opt/hopper/bin/Hopper";
host.unsupportedHopperVersion = true;
const initial = await host.initialSetupState();
const discovery = await discoverSetupState({
host,
providerEnvironment: { HOPPER_LAUNCHER_PATH: host.hopper },
providerConfigured: initial.providerConfigured,
forceHopperInstall: false,
proposeHopper: true,
initialDoctor: await host.doctor(),
+1
View File
@@ -74,6 +74,7 @@ export class FakeSetupHost implements SetupHost {
...(this.hopperLauncherConfigured
? { hopperLauncherConfigured: true }
: {}),
providerConfigured: this.hopper !== undefined || this.ghidra !== undefined,
providerEnvironment: {
...(this.hopper === undefined
? {}
+58
View File
@@ -5,6 +5,7 @@ import type { SetupProgressEvent } from "./SetupTypes.js";
import { FakeSetupHost, options } from "./Setup.fixture.js";
import { SetupSkillInstallError } from "./SetupSkill.js";
import { err, ok } from "../domain/result.js";
import { HopperReleaseError } from "./HopperReleaseFeed.js";
const linkedSkillFailure = () =>
new SetupSkillInstallError({
@@ -16,6 +17,44 @@ const linkedSkillFailure = () =>
cause: undefined,
});
describe("Linux publisher cache approval", () => {
it.each([true, false])(
"discloses the selected cache or blocks an unresolved path before approval: %s",
async (valid) => {
const host = Object.assign(new FakeSetupHost("linux"), {
linuxHopperReleaseCache: valid
? ok("/caller/cache/rea/hopper-releases")
: err(
new HopperReleaseError(
"verification_unavailable",
"Invalid selected cache path",
),
),
});
host.distribution = {
id: "ubuntu",
versionId: "24.04",
packageFamily: "deb",
hopperSetupSupported: true,
};
const result = await runSetup(options(false, true), host);
if (valid)
expect(
result.plannedActions.find(({ kind }) => kind === "install_hopper")
?.detail,
).toContain("/caller/cache/rea/hopper-releases/<launcher-sha256>");
else
expect(result).toMatchObject({
status: "needs_human",
plannedActions: [],
});
expect(host.hopperInstalls).toBe(0);
expect(host.configurations).toBe(0);
expect(host.skillInstalls).toBe(0);
},
);
});
describe("selected skill admission", () => {
it.each(["codex", "shared"] as const)(
"keeps %s choices when an unselected client is linked",
@@ -311,6 +350,25 @@ describe("setup plan application", () => {
expect(result.remediation).toBeUndefined();
});
it("records an installed Hopper when opening the app needs manual recovery", async () => {
const host = new FakeSetupHost("darwin");
host.installHopper = () =>
Promise.resolve({
status: "installed",
launcherPath: "/Applications/Hopper.app/Contents/MacOS/hopper",
warning:
"Hopper was installed, but macOS could not open it. Open it manually.",
});
const result = await runSetup(options(true, true), host);
expect(result).toMatchObject({
status: "needs_human",
appliedActions: ["installed_hopper"],
remediation: expect.stringContaining("Open it manually"),
});
});
it("reuses existing Hopper without selecting every detected client", async () => {
const host = new FakeSetupHost();
host.hopper = "/Applications/Hopper";
+16 -5
View File
@@ -111,7 +111,7 @@ const runSetupWorkflow = async (
let { hopperPath, providerEnvironment } = initialState;
const discovery = await discoverSetupState({
host,
providerEnvironment,
providerConfigured: initialState.providerConfigured,
forceHopperInstall: options.installHopper,
hopperLauncherConfigured: initialState.hopperLauncherConfigured === true,
proposeHopper:
@@ -306,6 +306,7 @@ const runSetupWorkflow = async (
}
let selectedClients: readonly SetupClient[] =
planSelection.clientsToConfigure;
let hopperWarning: string | undefined;
if (
planSelection.installHopper &&
(interactiveApproval || options.installHopper)
@@ -322,7 +323,12 @@ const runSetupWorkflow = async (
clientStates,
});
if ("failure" in install) return install.failure;
({ hopperPath, providerEnvironment, selectedClients } = install);
({
hopperPath,
providerEnvironment,
selectedClients,
warning: hopperWarning,
} = install);
}
const clientFailure = await configureDetectedClients({
host,
@@ -355,6 +361,7 @@ const runSetupWorkflow = async (
doctor.healthy,
hopperPath,
),
hopperWarning,
manualRemediation,
);
return {
@@ -407,14 +414,17 @@ const installHopperAction = async (input: {
};
}
input.appliedActions.push("installed_hopper");
const warning = [installed.warning, installed.cleanupFailure]
.filter((value): value is string => value !== undefined)
.join(" ");
emitProgress(input.options, {
actionId: "install_hopper",
label,
state: installed.cleanupFailure === undefined ? "completed" : "warning",
state: warning === "" ? "completed" : "warning",
detail:
installed.cleanupFailure === undefined
warning === ""
? installed.launcherPath
: `${installed.launcherPath}; ${installed.cleanupFailure}`,
: `${installed.launcherPath}; ${warning}`,
});
const providerEnvironment = {
...input.providerEnvironment,
@@ -423,6 +433,7 @@ const installHopperAction = async (input: {
return {
hopperPath: installed.launcherPath,
providerEnvironment,
...(warning === "" ? {} : { warning }),
selectedClients: await filterClientsNeedingConfigure(
input.host,
input.selectedClients,
+88 -13
View File
@@ -6,6 +6,7 @@ import {
type ClientConfigurationSource,
} from "./ClientConfigurationFiles.js";
import { err, ok, type Result } from "../domain/result.js";
import { ConfigurationError } from "../domain/configurationErrors.js";
import {
clientRegistrationEntry,
clientConfigurationValuesEqual,
@@ -23,11 +24,19 @@ import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js";
import { mkdir, rm, writeFile } from "node:fs/promises";
import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
import {
clientRegistrationEnvironment,
clientRegistrationEnvironmentDetails,
type ProviderRequestBudgetResolver,
} from "./ClientRegistrationEnvironment.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import {
MCP_STARTUP_POLICY,
recommendedMcpToolTimeoutMs,
recommendedMcpToolTimeoutSeconds,
} from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { packageReadTargetChanged } from "./MsixPackageStorage.js";
import {
@@ -74,16 +83,23 @@ export const configureClientConfiguration = (
client: SetupClient,
environment: SetupProviderEnvironment = {},
command: readonly string[] = npxRegistrationCommand(),
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Promise<ClientConfigurationResult> => {
if (client.configPathError !== undefined)
return Promise.resolve({ status: "failed", reason: "path" });
return configureClientDocument(client, environment, command);
return configureClientDocument(
client,
environment,
command,
resolveProviderRequestBudget,
);
};
const configureClientDocument = async (
client: SetupClient,
environment: SetupProviderEnvironment,
command: readonly string[],
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Promise<ClientConfigurationResult> => {
const format = client.format;
if (format === undefined || format === "unsupported")
@@ -111,6 +127,7 @@ const configureClientDocument = async (
command,
existing,
files.value,
resolveProviderRequestBudget,
),
);
if (!policy.ok) {
@@ -149,6 +166,7 @@ const configureClientDocument = async (
command,
parsed,
files.value,
resolveProviderRequestBudget,
),
);
if (!policy.ok) {
@@ -177,6 +195,7 @@ const configureClientDocument = async (
servers: { ...fallback.value, ...parsed.servers },
},
files.value,
resolveProviderRequestBudget,
);
if (registrationCurrent(parsed, desired)) return { status: "unchanged" };
return persistClientConfiguration({
@@ -346,6 +365,7 @@ export const clientConfigurationAligned = async (
client: SetupClient,
providerEnvironment: SetupProviderEnvironment,
command: readonly string[],
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Promise<boolean> => {
if (client.configPathError !== undefined) return false;
try {
@@ -365,6 +385,7 @@ export const clientConfigurationAligned = async (
command,
parsed,
files.value,
resolveProviderRequestBudget,
),
);
if (!policy.ok) {
@@ -381,6 +402,7 @@ export const clientConfigurationAligned = async (
command,
parsed,
files.value,
resolveProviderRequestBudget,
),
)
);
@@ -397,11 +419,13 @@ export const inspectClientConfiguration = async (
client: SetupClient,
providerEnvironment: SetupProviderEnvironment,
command: readonly string[],
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Promise<ClientConfigurationInspection> => {
const inspection = await inspectClientDocument(
client,
providerEnvironment,
command,
resolveProviderRequestBudget,
);
if (inspection.status !== "create" && inspection.status !== "update")
return inspection;
@@ -416,6 +440,7 @@ const inspectClientDocument = async (
client: SetupClient,
providerEnvironment: SetupProviderEnvironment,
command: readonly string[],
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
): Promise<ClientConfigurationInspection> => {
if (client.configPathError !== undefined)
return { status: "invalid", remediation: client.configPathError };
@@ -449,6 +474,7 @@ const inspectClientDocument = async (
command,
existing,
files.value,
resolveProviderRequestBudget,
),
);
if (!policy.ok) {
@@ -498,6 +524,7 @@ const inspectClientDocument = async (
command,
parsed,
files.value,
resolveProviderRequestBudget,
),
);
if (!policy.ok) {
@@ -515,6 +542,7 @@ const inspectClientDocument = async (
command,
parsed,
files.value,
resolveProviderRequestBudget,
);
if (registrationCurrent(parsed, desired))
return { status: "already_current" };
@@ -525,6 +553,8 @@ const inspectClientDocument = async (
}
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (cause instanceof ConfigurationError)
return { status: "invalid", remediation: cause.message };
// Malformed configuration is reported with the invalid remediation.
void cause;
return {
@@ -653,20 +683,21 @@ const clientConfigurationDesired = (
command: readonly string[],
parsed: ClientConfigurationDocument | undefined,
sources: readonly ClientConfigurationSource[],
resolveProviderRequestBudget?: ProviderRequestBudgetResolver,
) => {
const effective = effectiveClientConfiguration(sources, {
client,
document: parsed,
});
const validated = clientRegistrationEnvironment(
effective,
providerEnvironment,
);
if (!validated.ok) throw validated.error;
const format = parsed?.dialect ?? client.format ?? "json";
const launcher =
command.length === 0 ? [PRODUCT_IDENTITY.cliBinary, "mcp"] : command;
if (format === "dsh") {
const validated = clientRegistrationEnvironment(
effective,
providerEnvironment,
);
if (!validated.ok) throw validated.error;
const desired = (
parsed === undefined
? new DshCordisDocument("")
@@ -675,18 +706,36 @@ const clientConfigurationDesired = (
if (!desired.ok) throw desired.error;
return desired.value;
}
const validated = clientRegistrationEnvironmentDetails(
effective,
providerEnvironment,
"effective",
client.name === "codex" || client.name === "kimi"
? resolveProviderRequestBudget
: undefined,
);
if (!validated.ok) throw validated.error;
// Preserve explicitly configured server settings, never the ambient process
// environment. Newly discovered provider paths win only for their own keys.
// environment. The effective layered view validates precedence and supplies
// timeout facts; the target file's fragment remains the value being written.
const environment = clientRegistrationEnvironment(
parsed,
providerEnvironment,
"fragment",
);
if (!environment.ok) throw environment.error;
const registrationEnvironment =
client.name === "kimi" &&
environment.value.REA_MCP_INPUT_SCHEMA_PROFILE === undefined
? {
...environment.value,
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
}
: environment.value;
const registration = clientRegistrationEntry(
format,
launcher,
environment.value,
registrationEnvironment,
);
const previous = parsed?.servers[PRODUCT_IDENTITY.mcpServerKey];
if (
@@ -700,14 +749,40 @@ const clientConfigurationDesired = (
...registration,
...(client.name === "codex" || client.name === "grok_build"
? {
startup_timeout_sec: MCP_STARTUP_POLICY.codexStartupTimeoutSeconds,
startup_timeout_sec:
client.name === "codex"
? (positiveTimeout(previous, "startup_timeout_sec") ??
MCP_STARTUP_POLICY.codexStartupTimeoutSeconds)
: MCP_STARTUP_POLICY.codexStartupTimeoutSeconds,
}
: {}),
...(client.name === "codex"
? {
tool_timeout_sec:
positiveTimeout(previous, "tool_timeout_sec") ??
recommendedMcpToolTimeoutSeconds(validated.value.timeoutInputs),
}
: {}),
...(client.name === "kimi"
? {
startupTimeoutMs: MCP_STARTUP_POLICY.kimiStartupTimeoutMs,
toolTimeoutMs: MCP_STARTUP_POLICY.kimiToolTimeoutMs,
startupTimeoutMs:
positiveTimeout(previous, "startupTimeoutMs") ??
MCP_STARTUP_POLICY.kimiStartupTimeoutMs,
toolTimeoutMs:
positiveTimeout(previous, "toolTimeoutMs") ??
recommendedMcpToolTimeoutMs(validated.value.timeoutInputs),
}
: {}),
};
};
const positiveTimeout = (
registration: unknown,
key: string,
): number | undefined => {
if (typeof registration !== "object" || registration === null) return;
const value: unknown = Reflect.get(registration, key);
return typeof value === "number" && Number.isFinite(value) && value > 0
? value
: undefined;
};
+117 -25
View File
@@ -20,8 +20,10 @@ import {
LinuxHopperTemporaryCleanupOwner,
readLinuxDistribution,
systemLinuxHopperInstallHost,
type LinuxHopperInstallHost,
} from "./LinuxHopper.js";
import { installMacHopper, systemMacHopperInstallHost } from "./MacHopper.js";
import { linuxHopperReleaseCacheDirectory } from "./LinuxHopperPublisher.js";
import {
clientEvidencePaths,
supportedClients,
@@ -45,7 +47,10 @@ import {
RegularFileCleanupFailure,
retryRegularFileCleanup,
} from "./RegularFileRead.js";
import { providerRegistrationEnvironment } from "./SetupRegistrationEnvironment.js";
import {
explicitSetupRegistrationEnvironment,
providerRegistrationEnvironment,
} from "./SetupRegistrationEnvironment.js";
import {
msixPackagesDirectory,
packagePrivateLauncherBlocker,
@@ -57,6 +62,7 @@ import type {
} from "./SetupTypes.js";
import type { DoctorScope } from "./Doctor.js";
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
import { analysisProviderSelectorSchema } from "../contracts/providerSelection.js";
/** Resolve the executable and arguments used in a managed MCP registration. */
export const setupRegistrationCommand = (
@@ -152,6 +158,12 @@ export const systemSetupHost = (
return {
platform,
homeDirectory,
...(platform === "linux"
? {
linuxHopperReleaseCache:
linuxHopperReleaseCacheDirectory(environment),
}
: {}),
skillDestinations: (clientIds) =>
skillDestinations(homeDirectory, clientIds, environment, platform),
...setupRegistration(platform, homeDirectory, environment),
@@ -160,12 +172,23 @@ export const systemSetupHost = (
linuxDistribution: readLinuxDistribution,
close: () =>
closeSetupResources(resources, macHopperHost, doctorHost, () =>
pendingLinuxHopperCleanup.retry(linuxHopperHost.cleanup),
closeLinuxHopperInstallHost(linuxHopperHost, pendingLinuxHopperCleanup),
),
initialSetupState: async (
scope?: DoctorScope,
): Promise<SetupInitialState> => {
const diagnosis = await runDoctor(undefined, doctorHost, scope);
const providerEnvironment: SetupProviderEnvironment = {
...providerRegistrationEnvironment(diagnosis.providerInspections ?? []),
...doctorHost.providerRegistrationEnvironment?.(),
...explicitSetupRegistrationEnvironment(environment),
...(diagnosis.hopperPath === undefined
? {}
: { HOPPER_LAUNCHER_PATH: diagnosis.hopperPath }),
};
const requestedProvider = analysisProviderSelectorSchema.safeParse(
environment.REA_ANALYSIS_PROVIDER,
);
return {
...(diagnosis.hopperPath === undefined
? {}
@@ -175,15 +198,14 @@ export const systemSetupHost = (
diagnosis.hopperPath === environment.HOPPER_LAUNCHER_PATH
? { hopperLauncherConfigured: true }
: {}),
providerEnvironment: {
...providerRegistrationEnvironment(
diagnosis.providerInspections ?? [],
),
...doctorHost.providerRegistrationEnvironment?.(),
...(diagnosis.hopperPath === undefined
? {}
: { HOPPER_LAUNCHER_PATH: diagnosis.hopperPath }),
},
providerConfigured:
diagnosis.hopperPath !== undefined ||
diagnosis.providerInspections?.some(
({ configured }) => configured,
) === true ||
providerEnvironment.REA_JEB_MCP_URL !== undefined ||
(requestedProvider.success && requestedProvider.data !== "auto"),
providerEnvironment,
doctor: diagnosis,
};
},
@@ -192,9 +214,19 @@ export const systemSetupHost = (
const result = await installLinuxHopper(linuxHopperHost);
if (result.cleanupFailure !== undefined)
pendingLinuxHopperCleanup.retain(result.cleanupFailure);
return result.status === "installed"
? { status: "installed", launcherPath: result.launcherPath }
: setupInstallFailure(result.reason, result.packageFailure);
if (result.status === "installed")
return { status: "installed", launcherPath: result.launcherPath };
const failure = setupInstallFailure(
result.reason,
result.packageFailure,
"linux",
);
return result.verificationFailure !== undefined
? {
...failure,
remediation: `${result.verificationFailure.message} ${failure.remediation}`,
}
: failure;
}
if (macHopperHost === undefined)
return setupInstallFailure("unsupported_host");
@@ -203,23 +235,37 @@ export const systemSetupHost = (
return {
status: "installed",
launcherPath: result.launcherPath,
...(result.openFailure === undefined
? {}
: {
warning: `Hopper was installed, but macOS could not open it: ${result.openFailure}. Open ${result.launcherPath} manually to complete first-run setup.`,
}),
...(result.cleanupFailure === undefined
? {}
: { cleanupFailure: result.cleanupFailure }),
};
const failure = setupInstallFailure(result.reason);
if (result.cleanupFailure === undefined) return failure;
return failure.status === "failed"
? {
...failure,
remediation: `${failure.remediation} ${result.cleanupFailure}`,
}
: failure;
const diagnostic = setupInstallerDiagnostic(result.diagnostic);
return {
...failure,
remediation: [
failure.remediation,
...(diagnostic === undefined
? []
: [`Installer detail: ${diagnostic}`]),
...(result.cleanupFailure === undefined
? []
: [result.cleanupFailure]),
].join(" "),
};
},
detectedClients: () => detectClients(homeDirectory, platform, environment),
supportedClients: () =>
Promise.resolve(supportedClients(homeDirectory, platform, environment)),
...clientConfigurationOperations(resources),
...clientConfigurationOperations(
resources,
doctorHost.resolveProviderRequestBudget,
),
skillNeedsInstall: (clientIds) =>
canonicalSkillNeedsInstall(
homeDirectory,
@@ -248,7 +294,25 @@ export const systemSetupHost = (
};
};
const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
const setupInstallerDiagnostic = (
value: string | undefined,
): string | undefined => {
if (value === undefined) return undefined;
const normalized = value
.replace(/\u001b(?:\[[0-9;?]*[ -/]*[@-~]|\([0-9A-Za-z]|[@-Z\\^_])/gu, "")
.replace(/[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f]/gu, "")
.replace(/\s+/gu, " ")
.trim();
if (normalized === "") return undefined;
return normalized.length <= 400
? normalized
: `${normalized.slice(0, 397).trimEnd()}...`;
};
const clientConfigurationOperations = (
resources: ArtifactResourceScope,
resolveProviderRequestBudget: DoctorHost["resolveProviderRequestBudget"],
) => ({
configureClient: (
client: SetupClient,
providerEnvironment: SetupProviderEnvironment,
@@ -257,7 +321,12 @@ const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
runClientConfigurationOperation(resources, async () =>
client.format === "unsupported"
? { status: "skipped" as const }
: configureClientConfiguration(client, providerEnvironment, command),
: configureClientConfiguration(
client,
providerEnvironment,
command,
resolveProviderRequestBudget,
),
),
clientNeedsConfigure: (
client: SetupClient,
@@ -271,6 +340,7 @@ const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
client,
providerEnvironment,
command,
resolveProviderRequestBudget,
)),
),
inspectClientConfiguration: (
@@ -279,7 +349,12 @@ const clientConfigurationOperations = (resources: ArtifactResourceScope) => ({
command: readonly string[],
) =>
runClientConfigurationOperation(resources, () =>
inspectClientConfiguration(client, providerEnvironment, command),
inspectClientConfiguration(
client,
providerEnvironment,
command,
resolveProviderRequestBudget,
),
),
});
@@ -368,6 +443,23 @@ const configurationCleanupError = (
const errorMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const closeLinuxHopperInstallHost = async (
host: LinuxHopperInstallHost,
temporaryOwner: LinuxHopperTemporaryCleanupOwner,
): Promise<string | undefined> => {
const reasons: string[] = [];
const temporary = await temporaryOwner.retry(host.cleanup);
if (temporary !== undefined) reasons.push(temporary);
try {
await host.close?.();
} catch (cause: unknown) {
reasons.push(
`Linux Hopper publisher cleanup failed: ${errorMessage(cause)}`,
);
}
return reasons.length === 0 ? undefined : reasons.join("; ");
};
/** Detect supported agents from their config files or stable installation markers. */
export const detectClients = async (
home: string,
+30 -3
View File
@@ -1,4 +1,5 @@
import { HOPPER_PUBLISHER } from "../domain/hopperPublisher.js";
import { HOPPER_LINUX_PRIMARY_FINGERPRINT } from "./HopperReleaseFeed.js";
import type { LinuxPackageInstallFailure } from "./LinuxHopper.js";
export type SetupFailureCode =
@@ -23,6 +24,7 @@ export type SetupHopperInstallResult =
readonly status: "installed";
readonly launcherPath: string;
readonly cleanupFailure?: string;
readonly warning?: string;
}
| {
readonly status: "failed";
@@ -30,6 +32,11 @@ export type SetupHopperInstallResult =
readonly remediation: string;
};
export type SetupHopperInstallFailure = Extract<
SetupHopperInstallResult,
{ readonly status: "failed" }
>;
export type HopperInstallFailureReason =
| "unsupported_host"
| "release_metadata"
@@ -44,6 +51,8 @@ export type HopperInstallFailureReason =
| "mount"
| "bundle"
| "publisher_verification_failed"
| "verification_unavailable"
| "cleanup_failed"
| "copy";
const normalizePackageOutput = (output: string): string =>
@@ -93,7 +102,8 @@ const packageManagerRemediation = (
export const setupInstallFailure = (
reason: HopperInstallFailureReason,
packageFailure?: LinuxPackageInstallFailure,
): SetupHopperInstallResult => {
platform: NodeJS.Platform = "darwin",
): SetupHopperInstallFailure => {
switch (reason) {
case "unsupported_host":
return {
@@ -133,7 +143,7 @@ export const setupInstallFailure = (
status: "failed",
code: "unsupported_hopper_build",
remediation:
"The installed Hopper launcher does not match the build supported by this REA release. Unset HOPPER_LAUNCHER_PATH, reinstall through rea setup, or update REA.",
"The installed Hopper launcher does not match the authenticated vendor package. Check HOPPER_LAUNCHER_PATH and reinstall the official Hopper demo through rea setup.",
};
case "cancelled":
return {
@@ -167,7 +177,24 @@ export const setupInstallFailure = (
return {
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
remediation:
platform === "linux"
? `The Hopper package signature did not verify under the pinned primary PGP fingerprint ${HOPPER_LINUX_PRIMARY_FINGERPRINT}; the package manager was not run. Download the official package and retry setup.`
: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
};
case "verification_unavailable":
return {
status: "failed",
code: "publisher_verification_failed",
remediation:
"REA could not authenticate or retain the signed Hopper package. Check vendor access, native package inspection tools and caller-owned cache permissions, then retry setup.",
};
case "cleanup_failed":
return {
status: "failed",
code: "cleanup_failed",
remediation:
"Hopper publisher verification retained temporary cleanup ownership. Retry cleanup before running setup again.",
};
case "copy":
return {
+17 -6
View File
@@ -37,7 +37,7 @@ export interface SetupDiscovery {
/** Discover supported clients and current registrations without mutation. */
export const discoverSetupState = async (input: {
readonly host: SetupHost;
readonly providerEnvironment: SetupProviderEnvironment;
readonly providerConfigured: boolean;
readonly forceHopperInstall: boolean;
readonly hopperLauncherConfigured?: boolean;
readonly proposeHopper: boolean;
@@ -65,8 +65,7 @@ export const discoverSetupState = async (input: {
input.forceHopperInstall ||
(hopperInstallSupported &&
input.proposeHopper &&
(Object.keys(input.providerEnvironment).length === 0 ||
linuxHopperRepairNeeded));
(!input.providerConfigured || linuxHopperRepairNeeded));
const [clients, detectedClients] = await Promise.all([
input.host.supportedClients(),
input.host.detectedClients(),
@@ -162,6 +161,14 @@ export const planSetupActions = async (input: {
const client = byId.get(id);
return client === undefined ? [] : [client];
});
const hopperCache = input.host.linuxHopperReleaseCache;
if (input.discovery.installHopper && hopperCache?.ok === false)
return {
plannedActions: [],
selectedClients,
clientsToConfigure: [],
blocker: hopperCache.error.message,
};
if (
selectedClients.length > 0 &&
input.host.registrationCommandBlocker !== undefined
@@ -200,6 +207,9 @@ export const planSetupActions = async (input: {
const plannedActions = setupPlan({
platform: input.host.platform,
homeDirectory: input.host.homeDirectory,
...(hopperCache?.ok === true
? { hopperCacheDirectory: hopperCache.value }
: {}),
installHopper: input.discovery.installHopper,
installSkill: input.installSkill,
skillClientIds: input.skillClientIds,
@@ -265,6 +275,7 @@ const plannedClientWrites = (
const setupPlan = (input: {
readonly platform: NodeJS.Platform;
readonly homeDirectory: string;
readonly hopperCacheDirectory?: string;
readonly installHopper: boolean;
readonly installSkill: boolean;
readonly skillClientIds: readonly string[];
@@ -290,7 +301,7 @@ const setupPlan = (input: {
: "system package manager",
detail:
input.platform === "linux"
? "Download, verify, and install Hopper plus its Xvfb demo-session dependencies. For the supported demo build, REA uses a private display and selects Hopper's offered demo mode for each analysis session."
? `Download the current official Hopper demo, verify its PGP publisher and launcher bytes, and retain archive, public-key.asc and release.json under ${input.hopperCacheDirectory ?? `${input.homeDirectory}/.cache/rea/hopper-releases`}/<launcher-sha256> for offline verification. Preserve replaced damaged evidence beside it as <launcher-sha256>.backup-<id>. Install Hopper plus its Xvfb demo-session dependencies. REA uses a private display and selects Hopper's offered demo mode for each analysis session.`
: "Download the official Hopper package, verify it, and install it. Hopper may show its demo or license prompt when first opened.",
external: true,
operation: "install" as const,
@@ -347,9 +358,9 @@ const linuxDisclosure = (family: LinuxPackageFamily) => {
process.getuid?.() === 0,
);
return {
networkOrigins: [disclosure.downloadUrl],
networkOrigins: disclosure.networkOrigins,
commands: disclosure.commands,
integrity: `${String(disclosure.expectedBytes)} bytes · SHA-1 ${disclosure.expectedSha1}`,
integrity: disclosure.integrity,
};
};
@@ -1,6 +1,25 @@
import type { DoctorProviderInspection } from "./Doctor.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
const SELECTED_REGISTRATION_SETTINGS = [
"REA_ANALYSIS_PROVIDER",
"REA_GHIDRA_STARTUP_TIMEOUT_MS",
"REA_HOPPER_STARTUP_TIMEOUT_MS",
"REA_JEB_MCP_TIMEOUT_MS",
"REA_MCP_INPUT_SCHEMA_PROFILE",
] as const;
/** Preserve selected setup settings without copying unrelated process values. */
export const explicitSetupRegistrationEnvironment = (
environment: Readonly<Record<string, string | undefined>>,
): Readonly<Record<string, string>> =>
Object.fromEntries(
SELECTED_REGISTRATION_SETTINGS.flatMap((key) => {
const value = environment[key];
return value === undefined ? [] : [[key, value]];
}),
);
/** Preserve adapter-published settings independently of runtime readiness. */
export const providerRegistrationEnvironment = (
inspections: readonly DoctorProviderInspection[],
+4
View File
@@ -1,5 +1,6 @@
import type { DoctorReport, DoctorScope, runDoctor } from "./Doctor.js";
import type { LinuxDistribution } from "./LinuxHopper.js";
import type { HopperReleaseError } from "./HopperReleaseFeed.js";
import type {
SkillDestination,
SkillInstallResult,
@@ -20,6 +21,8 @@ export interface SetupInitialState {
readonly hopperPath?: string;
/** Whether the caller explicitly selected a Hopper launcher in its environment. */
readonly hopperLauncherConfigured?: boolean;
/** Whether the caller or host has an analysis provider configuration. */
readonly providerConfigured: boolean;
readonly providerEnvironment: SetupProviderEnvironment;
readonly doctor: DoctorReport;
}
@@ -65,6 +68,7 @@ export type ClientConfigurationInspection =
export interface SetupHost {
readonly platform: NodeJS.Platform;
readonly homeDirectory: string;
readonly linuxHopperReleaseCache?: Result<string, HopperReleaseError>;
skillDestinations(clientIds: readonly string[]): readonly SkillDestination[];
readonly registrationCommand: readonly string[];
/** Why client registration cannot proceed from this host. */
+62
View File
@@ -0,0 +1,62 @@
import type {
AnalysisOperation,
ProviderIdentity,
} from "./AnalysisProvider.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
/** One provider-owned caveat tied to the operation that reported it. */
export interface SourceLimitation {
readonly provider: ProviderIdentity;
readonly operation: AnalysisOperation;
readonly limitation: string;
}
/** Stable exact-provenance key for deduplication before rendering. */
export const sourceLimitationKey = ({
provider,
operation,
limitation,
}: SourceLimitation): string =>
JSON.stringify([
provider.id,
provider.name,
provider.version,
operation,
limitation,
]);
/** Render unique source caveats without losing their provider or operation. */
export const renderSourceLimitations = (
limitations: readonly SourceLimitation[],
): string[] => {
const groups = new Map<
string,
{
readonly provider: ProviderIdentity;
readonly limitation: string;
readonly operations: Set<AnalysisOperation>;
}
>();
for (const { provider, operation, limitation } of limitations) {
const key = JSON.stringify([
provider.id,
provider.name,
provider.version,
limitation,
]);
const group = groups.get(key) ?? {
provider,
limitation,
operations: new Set<AnalysisOperation>(),
};
group.operations.add(operation);
groups.set(key, group);
}
return [...groups.values()]
.map(({ provider, limitation, operations }) => {
const version = provider.version === null ? "" : ` ${provider.version}`;
const operationList = [...operations].sort(compareUnicodeCodePoints);
return `${provider.name} (${provider.id}${version}) ${operationList.join(", ")}: ${limitation}`;
})
.sort(compareUnicodeCodePoints);
};
+48 -5
View File
@@ -95,16 +95,18 @@ it("retains unique upstream limits in canonical order across concurrent observat
let calls = 0;
const options = { signal: new AbortController().signal };
const analysis: AnalysisOperationPort = {
execute: async (_, parameters, forwarded) => {
execute: async (operation, parameters, forwarded) => {
expect(parameters).toEqual({ marker: true });
expect(forwarded).toBe(options);
const call = calls++;
if (call === 0) await gate;
else release();
const limitation =
operation === "list_strings"
? ["first", "shared"]
: ["second", "shared"];
return ok(
createAnalysisExecution([], provider, {
limitations: call === 0 ? ["first", "shared"] : ["second", "shared"],
}),
createAnalysisExecution([], provider, { limitations: limitation }),
);
},
};
@@ -117,7 +119,48 @@ it("retains unique upstream limits in canonical order across concurrent observat
});
expect(calls).toBe(2);
expect(result).toEqual(
ok({ result: { count: 0 }, limitations: ["first", "second", "shared"] }),
ok({
result: { count: 0 },
limitations: [
"Fixture (fixture 1) list_procedures, list_strings: shared",
"Fixture (fixture 1) list_procedures: second",
"Fixture (fixture 1) list_strings: first",
],
}),
);
});
it("deduplicates exact source caveats without merging providers or operations", async () => {
const firstProvider = { id: "first", name: "First", version: "1" };
const secondProvider = { id: "second", name: "Second", version: "1" };
const result = await executeComposedWorkflow(
{
execute: async (operation, parameters) =>
ok(
createAnalysisExecution(
null,
parameters.provider === "second" ? secondProvider : firstProvider,
{ limitations: ["same source caveat"] },
),
),
},
async (observed) => {
await observed.execute("list_strings", { provider: "first" });
await observed.execute("list_strings", { provider: "first" });
await observed.execute("list_strings", { provider: "second" });
await observed.execute("list_procedures", { provider: "first" });
return ok({ count: 0 });
},
);
expect(result).toEqual(
ok({
result: { count: 0 },
limitations: [
"First (first 1) list_procedures, list_strings: same source caveat",
"Second (second 1) list_strings: same source caveat",
],
}),
);
});
+15 -6
View File
@@ -23,7 +23,11 @@ import {
import type { JsonValue } from "../domain/jsonValue.js";
import type { UnknownRegistryPort } from "./investigation/InvestigationRecordPort.js";
import { UnknownRegistryError } from "../domain/unknownRegistryError.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import {
renderSourceLimitations,
sourceLimitationKey,
type SourceLimitation,
} from "./SourceLimitations.js";
/** Retain the limits of exactly the observations used by one composed call. */
export const executeComposedWorkflow = async (
@@ -37,21 +41,26 @@ export const executeComposedWorkflow = async (
AnalysisError
>
> => {
const limitations = new Set<string>();
const limitations = new Map<string, SourceLimitation>();
const result = await execute({
execute: async (operation, parameters, options) => {
const execution = await analysis.execute(operation, parameters, options);
if (execution.ok)
execution.value.limitations.forEach((limitation) =>
limitations.add(limitation),
);
for (const limitation of execution.value.limitations) {
const fact = {
provider: execution.value.provider,
operation,
limitation,
} satisfies SourceLimitation;
limitations.set(sourceLimitationKey(fact), fact);
}
return execution;
},
});
if (!result.ok) return result;
return ok({
result: result.value,
limitations: [...limitations].sort(compareUnicodeCodePoints),
limitations: renderSourceLimitations([...limitations.values()]),
});
};
@@ -7,6 +7,7 @@ import { describe, expect, it } from "vitest";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { createEvidence, parseEvidence } from "../../domain/evidence.js";
import { inspectAnalysisViewInputSchema } from "../../domain/analysisView/analysisView.js";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import {
@@ -432,45 +433,284 @@ it("refuses an object-dense borrowed capture before making a full schema clone",
expect(ledger.forAnalysis(parent.evidence_id)).toBe(borrowed);
});
it("rejects accessor-backed inline Evidence before all view projections", () => {
const views = [
{ kind: "summary" },
{ kind: "capture" },
{ kind: "api-surface" },
] as const;
for (const view of views) {
it.each(["summary", "capture", "api-surface"] as const)(
"rejects inline accessors before parsing a %s view",
(kind) => {
let accessed = false;
const evidence = {
get normalized_result() {
accessed = true;
throw new Error("Evidence accessor must not be invoked");
throw new Error("Evidence must be admitted before parsing");
},
};
const result = inspectAnalysisView({
const input = {
source: { kind: "inline", evidence },
view,
});
expect(result).toMatchObject({
view: { kind },
};
const parsed = inspectAnalysisViewInputSchema.parse(input);
if (parsed.source.kind !== "inline")
throw new Error("Expected inline input");
expect(parsed.source.evidence).toBe(evidence);
expect(inspectAnalysisView(input)).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
error: {
_tag: "AnalysisInputError",
issues: [{ path: ["source", "evidence", "normalized_result"] }],
},
});
expect(accessed).toBe(false);
}
const parent = analysisViewCaptureEvidence();
for (const view of views)
expect(
inspectAnalysisView({
source: { kind: "inline", evidence: parent },
view,
source: { kind: "inline", evidence: analysisViewCaptureEvidence() },
view: { kind },
}).ok,
).toBe(true);
},
);
it.each(["summary", "capture", "api-surface"] as const)(
"rejects nested, hidden and inherited inline accessors for %s",
(kind) => {
const parent = analysisViewCaptureEvidence();
let accessed = false;
const getter = () => {
accessed = true;
throw new Error("Inline accessors must remain inert");
};
const nested = {
get value() {
return getter();
},
};
const hidden = Object.defineProperty({}, "value", { get: getter });
const array = Object.defineProperty([null], 0, { get: getter });
const inherited: unknown = Object.create({
get value() {
return getter();
},
});
const cyclic: Record<string, unknown> = {};
cyclic["self"] = cyclic;
const cases: { data: unknown; path: (string | number)[] }[] = [
{ data: nested, path: ["value"] },
{ data: hidden, path: ["value"] },
{ data: array, path: [0] },
{ data: inherited, path: [] },
{ data: cyclic, path: ["self"] },
];
for (const { data, path } of cases) {
// A complete envelope reaches payload parsing and Evidence authentication.
const input = {
source: {
kind: "inline" as const,
evidence: { ...parent, raw_result: data },
},
view: { kind },
};
for (const result of [
inspectAnalysisView(input),
inspectAnalysisViewValidated(
inspectAnalysisViewInputSchema.parse(input),
),
])
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisInputError",
issues: [{ path: ["source", "evidence", "raw_result", ...path] }],
},
});
}
expect(accessed).toBe(false);
},
);
it.each([
["own map", "map", false],
["own iterator", Symbol.iterator, false],
["own constructor", "constructor", false],
["inherited map", "map", true],
["inherited iterator", Symbol.iterator, true],
] as const)(
"rejects array %s accessors without invoking them",
(_name, key, inherited) => {
const parent = analysisViewCaptureEvidence();
let accessed = false;
const array: unknown[] = [];
const owner: object = inherited ? Object.create(Array.prototype) : array;
Object.defineProperty(owner, key, {
get() {
accessed = true;
throw new Error("Array behavior must remain inert");
},
});
if (inherited) Object.setPrototypeOf(array, owner);
for (const kind of ["summary", "capture", "api-surface"] as const) {
const result = inspectAnalysisView({
source: { kind: "inline", evidence: { ...parent, raw_result: array } },
view: { kind },
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
}
expect(accessed).toBe(false);
},
);
it("rejects callable array overrides and subclass species before parsing", () => {
const parent = analysisViewCaptureEvidence();
let invoked = false;
const callback = () => {
invoked = true;
throw new Error("Array callbacks must remain inert");
};
class CustomArray extends Array<unknown> {
static override get [Symbol.species](): ArrayConstructor {
return callback();
}
}
for (const array of [
Object.defineProperty([], "map", { value: callback }),
Object.defineProperty([], Symbol.iterator, { value: callback }),
new CustomArray(),
])
for (const kind of ["summary", "capture", "api-surface"] as const) {
const result = inspectAnalysisView({
source: { kind: "inline", evidence: { ...parent, raw_result: array } },
view: { kind },
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
}
expect(invoked).toBe(false);
});
it.each(["summary", "capture", "api-surface"] as const)(
"preserves ordinary inline Evidence for %s",
(kind) => {
const parent = analysisViewCaptureEvidence();
const before = JSON.stringify(parent);
const result = inspectAnalysisView({
source: { kind: "inline", evidence: parent },
view: { kind },
});
if (!result.ok) throw result.error;
expect(parseEvidence(result.value)).toEqual(result.value);
expect(result.value.evidence_links).toEqual([parent.evidence_id]);
expect(result.value.normalized_result).toMatchObject({
kind,
parent_evidence_id: parent.evidence_id,
});
expect(JSON.stringify(parent)).toBe(before);
expect(Object.isFrozen(parent)).toBe(false);
},
);
it("rejects executable constructor values before schema plain-object checks", () => {
const parent = analysisViewCaptureEvidence();
let invoked = false;
const constructor = () => undefined;
Object.defineProperty(constructor, "prototype", {
get() {
invoked = true;
throw new Error("Constructor metadata must remain inert");
},
});
for (const kind of ["summary", "capture", "api-surface"] as const) {
expect(
inspectAnalysisView({
source: {
kind: "inline",
evidence: { ...parent, raw_result: { constructor } },
},
view: { kind },
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
}
expect(invoked).toBe(false);
});
it("rejects inline proxy values before reflection or parsing can run traps", () => {
const parent = analysisViewCaptureEvidence();
let invoked = false;
const trap = () => {
invoked = true;
throw new Error("Proxy behavior must remain inert");
};
const revoked = Proxy.revocable([], {});
revoked.revoke();
for (const value of [
new Proxy({}, { getPrototypeOf: trap, ownKeys: trap, get: trap }),
new Proxy(() => undefined, { get: trap }),
revoked.proxy,
])
for (const kind of ["summary", "capture", "api-surface"] as const) {
const result = inspectAnalysisView({
source: { kind: "inline", evidence: { ...parent, raw_result: value } },
view: { kind },
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
}
expect(invoked).toBe(false);
});
it("keeps ignored array metadata inert through parsing and projection", () => {
const capture = analysisViewCaptureFixture();
const parent = createEvidence(
{
path: capture.artifact.path,
sha256: capture.artifact.sha256!,
format: "file",
},
{ id: "fixture", name: "fixture", version: "1" },
{
operation: "inspect_web_network_capture",
parameters: { sensitive_value_count: 0 },
result: capture,
rawResult: [],
},
);
if (!Array.isArray(parent.raw_result)) throw new Error("Expected raw array");
const array = parent.raw_result;
let invoked = false;
const getter = () => {
invoked = true;
throw new Error("Ignored array metadata must remain inert");
};
for (let index = 0; index < 10_000; index += 1)
Object.defineProperty(array, `metadata-${index}`, { get: getter });
Object.defineProperty(array, Symbol("metadata"), { get: getter });
for (const kind of ["summary", "capture", "api-surface"] as const) {
const result = inspectAnalysisView({
source: { kind: "inline", evidence: parent },
view: { kind },
});
if (!result.ok) throw result.error;
expect(result.value.evidence_links).toEqual([parent.evidence_id]);
}
expect(invoked).toBe(false);
expect(parseEvidence(parent).raw_result).toEqual([]);
});
it("keeps inline-envelope admission live alongside capture validation and grouping", () => {
// Each clone alone fits; both graphs coexist while the view is projected.
const capture = analysisViewObjectDenseCapture(40_000);
const parent = analysisViewCaptureEvidence(capture);
for (const kind of ["summary", "capture"] as const) {
const projected = inspectAnalysisView({
source: { kind: "inline", evidence: parent },
view: { kind },
});
if (!projected.ok) throw projected.error;
expect(projected.value.evidence_links).toEqual([parent.evidence_id]);
}
const retained = inspectAnalysisView(
{
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
@@ -6,10 +6,7 @@ import type { EvidenceLookup } from "../EvidenceInputResolver.js";
import { resolveEvidenceInput } from "../EvidenceInputResolver.js";
import type { EvidenceWriter } from "../investigation/InvestigationRecordPort.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import {
@@ -27,6 +24,7 @@ import { jsonValueSchema } from "../../domain/jsonValue.js";
import { isImmutableJsonSnapshot } from "../../domain/immutableJson.js";
import { err, ok, type Result } from "../../domain/result.js";
import { WebNetworkApiSurfaceAccumulator } from "../../domain/analysisView/WebNetworkApiSurfaceAccumulator.js";
import { admitAnalysisViewData } from "../../domain/analysisView/analysisViewDataAdmission.js";
/** Authenticate a selected-view request and project already completed Evidence. */
export const inspectAnalysisView = (
@@ -58,8 +56,11 @@ export const inspectAnalysisViewValidated = (
input.view.include_query_parameter_names === true,
)
: undefined;
if (input.source.kind === "inline" && accumulator !== undefined) {
const admitted = accumulator.admitInlineEvidence(input.source.evidence);
if (input.source.kind === "inline") {
const admitted =
accumulator === undefined
? admitAnalysisViewData(input.source.evidence, true)
: accumulator.admitInlineEvidence(input.source.evidence);
if (!admitted.ok) return admitted;
}
const resolved = resolveAnalysisViewSource(input.source, lookup);
@@ -165,18 +166,6 @@ const resolveAnalysisViewSource = (
): Result<Evidence, AnalysisError> => {
if (source.kind === "retained-evidence")
return resolveEvidenceInput(source, lookup);
const invalidDataPath = inlineEvidenceInvalidDataPath(source.evidence);
if (invalidDataPath !== undefined)
return err(
new AnalysisInputError(INSPECT_ANALYSIS_VIEW_OPERATION, undefined, [
{
path: ["source", "evidence", ...invalidDataPath],
reason: "invalid_type",
message:
"Inline Evidence must contain plain JSON data properties without accessors or cycles.",
},
]),
);
try {
return ok(parseEvidence(source.evidence));
} catch (cause: unknown) {
@@ -203,53 +192,6 @@ const resolveAnalysisViewSource = (
}
};
/** Reject non-JSON object behavior before Zod reads caller-owned Evidence. */
const inlineEvidenceInvalidDataPath = (
root: unknown,
): readonly (string | number)[] | undefined => {
const active = new Set<object>();
const pending: {
readonly value: unknown;
readonly path: readonly (string | number)[];
readonly exit?: object;
}[] = [{ value: root, path: [] }];
while (pending.length > 0) {
const current = pending.pop();
if (current === undefined) continue;
if (current.exit !== undefined) {
active.delete(current.exit);
continue;
}
const { value, path } = current;
if (typeof value !== "object" || value === null) continue;
const isArray = Array.isArray(value);
const prototype = Object.getPrototypeOf(value);
if (
(isArray && prototype !== Array.prototype) ||
(!isArray && prototype !== Object.prototype && prototype !== null) ||
active.has(value)
)
return path;
active.add(value);
pending.push({ value, path, exit: value });
const keys = Object.getOwnPropertyNames(value);
for (let index = keys.length - 1; index >= 0; index -= 1) {
const key = keys[index];
if (key === undefined) continue;
const descriptor = Object.getOwnPropertyDescriptor(value, key);
const childPath = [
...path,
isArray && /^(0|[1-9]\d*)$/u.test(key) ? Number(key) : key,
];
if (descriptor === undefined || !Object.hasOwn(descriptor, "value"))
return childPath;
pending.push({ value: descriptor.value, path: childPath });
}
}
return undefined;
};
const createAnalysisViewEvidence = (
input: InspectAnalysisViewInput,
parent: Evidence,
@@ -0,0 +1,166 @@
import { expect, it } from "vitest";
import { inspectAnalysisView } from "../analysisView/AnalysisViewService.js";
import { ARTIFACT_COMPARISON_EXAMPLE } from "../../contracts/artifactComparisonExample.js";
import { artifactInventorySummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
import {
createEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import {
EvidenceIntegrityError,
EvidenceReferenceError,
} from "../../domain/evidenceErrors.js";
import { resolveArtifactComparisonEvidence } from "./ArtifactComparisonEvidence.js";
const artifactSummary = () => {
const parent = ARTIFACT_COMPARISON_EXAMPLE.left;
const summary = inspectAnalysisView(
{
source: { kind: "inline", evidence: parent },
view: { kind: "summary" },
},
(evidenceId) => (evidenceId === parent.evidence_id ? parent : undefined),
);
if (!summary.ok) throw summary.error;
return { parent, summary: summary.value };
};
const recreateSummary = (
summary: Evidence,
normalizedResult: Evidence["normalized_result"],
parameters: Evidence["parameters"] = summary.parameters,
): Evidence => {
if (summary.subject === null)
throw new Error("artifact summary fixture must identify its subject");
return createEvidence(
{
path: summary.subject.local_path,
sha256: summary.subject.digest.sha256,
format: summary.subject.format,
...(summary.subject.architecture === null
? {}
: { architecture: summary.subject.architecture }),
},
summary.provider,
{
predicateType: summary.predicate_type,
operation: summary.operation,
parameters,
result: normalizedResult,
...(summary.analysis_profile === null
? {}
: { analysisProfile: summary.analysis_profile }),
rawResult: summary.raw_result,
confidence: summary.confidence,
authority: summary.authority,
environment: summary.environment,
limitations: summary.limitations,
...(summary.relationship_coverage === null
? {}
: { relationshipCoverage: summary.relationship_coverage }),
locations: summary.locations,
evidenceLinks: summary.evidence_links,
},
);
};
it("reports a missing retained parent instead of treating summary facts as inventory", () => {
const { summary } = artifactSummary();
const resolved = resolveArtifactComparisonEvidence(summary, undefined);
expect(resolved.ok).toBe(false);
if (!resolved.ok) {
expect(resolved.error).toBeInstanceOf(EvidenceReferenceError);
expect(resolved.error).toMatchObject({ reason: "missing" });
}
});
it("uses retained Evidence resolution to reject a mismatched parent identity", () => {
const { summary } = artifactSummary();
const otherParent = ARTIFACT_COMPARISON_EXAMPLE.right;
const resolved = resolveArtifactComparisonEvidence(
summary,
() => otherParent,
);
expect(resolved.ok).toBe(false);
if (!resolved.ok) {
expect(resolved.error).toBeInstanceOf(EvidenceReferenceError);
expect(resolved.error).toMatchObject({ reason: "identity_mismatch" });
}
});
it("rejects a summary that declares the wrong parent operation", () => {
const { parent, summary } = artifactSummary();
const summaryResult = artifactInventorySummaryViewResultSchema.parse(
summary.normalized_result,
);
const normalizedResult = {
...summaryResult,
parent_operation: "inspect_artifact",
};
const forged = recreateSummary(summary, normalizedResult);
const resolved = resolveArtifactComparisonEvidence(forged, (evidenceId) =>
evidenceId === parent.evidence_id ? parent : undefined,
);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
});
it("rejects summary parameters that do not identify the projected parent", () => {
const { parent, summary } = artifactSummary();
const forged = recreateSummary(summary, summary.normalized_result, {
...summary.parameters,
parent_evidence_id: `ev_${"f".repeat(64)}`,
});
const resolved = resolveArtifactComparisonEvidence(forged, (evidenceId) =>
evidenceId === parent.evidence_id ? parent : undefined,
);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
});
it("rejects summary facts that do not reproject from the complete parent", () => {
const { parent, summary } = artifactSummary();
const summaryResult = artifactInventorySummaryViewResultSchema.parse(
summary.normalized_result,
);
const normalizedResult = {
...summaryResult,
summary: {
...summaryResult.summary,
available_counts: {
...summaryResult.summary.available_counts,
nodes: summaryResult.summary.available_counts.nodes + 1,
},
},
};
const forged = recreateSummary(summary, normalizedResult);
const resolved = resolveArtifactComparisonEvidence(forged, (evidenceId) =>
evidenceId === parent.evidence_id ? parent : undefined,
);
expect(resolved.ok).toBe(false);
if (!resolved.ok)
expect(resolved.error).toBeInstanceOf(EvidenceIntegrityError);
});
it("accepts display-only subject changes excluded from Evidence identity", () => {
const { parent, summary } = artifactSummary();
if (summary.subject === null)
throw new Error("artifact summary fixture must identify its subject");
const displayOnly = parseEvidence({
...summary,
subject: {
...summary.subject,
name: "renamed.app",
local_path: "/display-only/renamed.app",
},
});
const resolved = resolveArtifactComparisonEvidence(
displayOnly,
(evidenceId) => (evidenceId === parent.evidence_id ? parent : undefined),
);
expect(resolved).toEqual({ ok: true, value: parent });
});
@@ -0,0 +1,61 @@
import type { EvidenceLookup } from "../EvidenceInputResolver.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { inspectAnalysisViewValidated } from "../analysisView/AnalysisViewService.js";
import { artifactInventorySummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
import {
EvidenceIntegrityError,
EvidenceReferenceError,
} from "../../domain/evidenceErrors.js";
import type { Evidence } from "../../domain/evidence.js";
import { err, ok, type Result } from "../../domain/result.js";
/** Resolve only an artifact summary's exact retained owner before comparison. */
export const resolveArtifactComparisonEvidence = (
evidence: Evidence,
evidenceById: EvidenceLookup | undefined,
): Result<Evidence, AnalysisError> => {
if (evidence.operation !== "inspect_analysis_view") return ok(evidence);
const summary = artifactInventorySummaryViewResultSchema.safeParse(
evidence.normalized_result,
);
if (!summary.success)
return err(
new AnalysisInputError("compare_artifacts", undefined, [
{
path: ["evidence"],
reason: "invalid_value",
message:
"Artifact comparison accepts only an artifact inventory summary from inspect_analysis_view.",
},
]),
);
const parentId = summary.data.parent_evidence_id;
const parent = evidenceById?.(parentId);
if (parent === undefined)
return err(
new EvidenceReferenceError(
parentId,
"missing",
"retained complete artifact inventory Evidence",
null,
),
);
const reprojected = inspectAnalysisViewValidated(
{
source: { kind: "retained-evidence", evidence_id: parentId },
view: { kind: "summary" },
},
(evidenceId) => (evidenceId === parentId ? parent : undefined),
);
if (!reprojected.ok) return reprojected;
if (reprojected.value.evidence_id !== evidence.evidence_id)
return err(
new EvidenceIntegrityError(
"Artifact summary does not match its retained parent inventory.",
),
);
return ok(parent);
};
@@ -1,6 +1,7 @@
import { describe, expect, it } from "vitest";
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
import type { RelationshipCoverage } from "../../domain/relationshipCoverage.js";
import {
ANALYSIS_SNAPSHOT_PROFILE,
ANALYSIS_SNAPSHOT_PROVIDER,
@@ -502,57 +503,58 @@ describe("analysis snapshot cache partitioning", () => {
});
});
describe("relationship coverage snapshot replay", () => {
it("authenticates relationship coverage and replays legacy reads as unknown", () => {
const cache = new AnalysisSnapshotCache();
const operation = "procedure_callees" as const;
const parameters = { procedure: "0x1000", document: "fixture" };
const coverage = {
status: "complete" as const,
diagnostics: ["The provider retained an unresolved call target."],
};
const execution = createAnalysisExecution(
["0x2000"],
ANALYSIS_SNAPSHOT_PROVIDER,
{
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
rawResult: ["0x2000"],
relationshipCoverage: coverage,
subject: {
path: ANALYSIS_SNAPSHOT_TARGET.path,
sha256: ANALYSIS_SNAPSHOT_TARGET.sha256,
format: ANALYSIS_SNAPSHOT_TARGET.format,
...(ANALYSIS_SNAPSHOT_TARGET.architecture === undefined
? {}
: { architecture: ANALYSIS_SNAPSHOT_TARGET.architecture }),
},
},
);
cache.record({
target: ANALYSIS_SNAPSHOT_TARGET,
profile: ANALYSIS_SNAPSHOT_PROFILE,
const createRelationshipCoverageSnapshot = (
relationshipCoverage?: RelationshipCoverage,
) => {
const operation = "procedure_callees" as const;
const parameters = { procedure: "0x1000", document: "fixture" };
const cache = new AnalysisSnapshotCache();
const execution = createAnalysisExecution(
["0x2000"],
ANALYSIS_SNAPSHOT_PROVIDER,
{
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
rawResult: ["0x2000"],
subject: ANALYSIS_SNAPSHOT_TARGET,
...(relationshipCoverage === undefined ? {} : { relationshipCoverage }),
},
);
cache.record({
target: ANALYSIS_SNAPSHOT_TARGET,
profile: ANALYSIS_SNAPSHOT_PROFILE,
operation,
parameters,
execution,
});
const evidence = createEvidence(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROVIDER,
{
operation,
parameters,
execution,
});
const evidence = createEvidence(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROVIDER,
{
operation,
parameters,
result: ["0x2000"],
rawResult: ["0x2000"],
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
relationshipCoverage: coverage,
},
);
const exported = cache.export(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROFILE,
createEvidenceBundle([evidence]),
);
if (!exported.ok) throw exported.error;
result: ["0x2000"],
rawResult: ["0x2000"],
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
...(relationshipCoverage === undefined ? {} : { relationshipCoverage }),
},
);
const exported = cache.export(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROFILE,
createEvidenceBundle([evidence]),
);
if (!exported.ok) throw exported.error;
return { cache, operation, parameters, snapshot: exported.value };
};
describe("relationship coverage snapshot replay", () => {
it("authenticates coverage and rejects missing serialized fields", () => {
const coverage: RelationshipCoverage = {
status: "complete",
diagnostics: ["The provider retained an unresolved call target."],
};
const { cache, operation, parameters, snapshot } =
createRelationshipCoverageSnapshot(coverage);
expect(
cache.lookup(
ANALYSIS_SNAPSHOT_TARGET,
@@ -562,7 +564,7 @@ describe("relationship coverage snapshot replay", () => {
)?.relationshipCoverage,
).toEqual(coverage);
const tampered = structuredClone(exported.value);
const tampered = structuredClone(snapshot);
const tamperedEntry = tampered.entries[0];
if (tamperedEntry === undefined) throw new Error("snapshot entry missing");
tamperedEntry.execution.relationship_coverage = {
@@ -573,26 +575,36 @@ describe("relationship coverage snapshot replay", () => {
new AnalysisSnapshotCache().import(tampered, undefined, () => ok(0)).ok,
).toBe(false);
const legacy = structuredClone(exported.value);
const legacyEntry = legacy.entries[0];
if (legacyEntry === undefined) throw new Error("snapshot entry missing");
Reflect.deleteProperty(legacyEntry.execution, "relationship_coverage");
const legacyEvidence = createEvidence(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROVIDER,
{
operation,
parameters,
result: ["0x2000"],
rawResult: ["0x2000"],
analysisProfile: ANALYSIS_SNAPSHOT_PROFILE,
},
const missingCoverage = structuredClone(snapshot);
const missingCoverageEntry = missingCoverage.entries[0];
if (missingCoverageEntry === undefined)
throw new Error("snapshot entry missing");
Reflect.deleteProperty(
missingCoverageEntry.execution,
"relationship_coverage",
);
Reflect.deleteProperty(legacyEvidence, "relationship_coverage");
legacy.evidence_bundle = createEvidenceBundle([legacyEvidence]);
parseAnalysisSnapshot(legacy);
const missingCoverageEvidence = missingCoverage.evidence_bundle.records[0];
if (missingCoverageEvidence === undefined)
throw new Error("snapshot Evidence missing");
Reflect.deleteProperty(missingCoverageEvidence, "relationship_coverage");
expect(() => parseAnalysisSnapshot(missingCoverage)).toThrow();
expect(
new AnalysisSnapshotCache().import(missingCoverage, undefined, () =>
ok(0),
).ok,
).toBe(false);
});
it("replays absent coverage as explicitly unknown", () => {
const { operation, parameters, snapshot } =
createRelationshipCoverageSnapshot();
expect(snapshot.entries[0]?.execution.relationship_coverage).toBeNull();
expect(
snapshot.evidence_bundle.records[0]?.relationship_coverage,
).toBeNull();
const replayCache = new AnalysisSnapshotCache();
expect(replayCache.import(legacy, undefined, () => ok(0)).ok).toBe(true);
expect(replayCache.import(snapshot, undefined, () => ok(0)).ok).toBe(true);
expect(
replayCache.lookup(
ANALYSIS_SNAPSHOT_TARGET,
@@ -5,10 +5,7 @@ import { setImmediate } from "node:timers/promises";
import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js";
import { analyzeParsedJavaScriptStaticSourceSteps } from "../../domain/javascript/javascriptStaticAnalysis.js";
import {
analyzeParsedJavaScriptSemanticsSteps,
classifyParsedJavaScriptOpenReceiversSteps,
} from "../../domain/javascript/javascriptSemanticAnalysis.js";
import { analyzeParsedJavaScriptSemanticsSteps } from "../../domain/javascript/javascriptSemanticAnalysis.js";
import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
import { projectJavaScriptModuleSemantics } from "../../domain/javascript/javascriptModuleSemanticIr.js";
import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
@@ -382,12 +379,9 @@ function* analyzeArtifactFileSteps<
state.parseFailures += 1;
return;
}
const openReceiverFacts =
yield* classifyParsedJavaScriptOpenReceiversSteps(parsed);
const analysis = yield* analyzeParsedJavaScriptStaticSourceSteps(
file.text.value,
parsed,
openReceiverFacts,
);
const staticFindings = findingCount(analysis);
yield;
@@ -2495,20 +2495,61 @@ describe("url-reference directory and empty-segment semantics", () => {
});
});
it.each(["./nested//worker.js", "./nested//../worker.js"])(
"does not collapse empty URL path segments: %s",
(declaredPath) => {
it.each([
["./nested//worker.js", null],
["./nested//../worker.js", "dist/nested/worker.js"],
])(
"keeps empty URL path segments significant: %s",
(declaredPath, resolvedPath) => {
// A dot segment removes the preceding empty segment, never `nested`.
expect(new URL(declaredPath, "file:///root/dist/app.mjs").pathname).toBe(
resolvedPath === null
? "/root/dist/nested//worker.js"
: `/root/${resolvedPath}`,
);
for (const context of ["url-reference", "html-reference"] as const)
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context,
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
context,
).toMatchObject({ resolved_path: resolvedPath });
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context: "url-reference",
context: "module-specifier",
moduleKind: "import",
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
).toMatchObject({ resolved_path: null });
).toMatchObject({ resolved_path: resolvedPath });
// CommonJS keeps filesystem path semantics.
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context: "module-specifier",
moduleKind: "require",
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
).toMatchObject({
resolved_path:
declaredPath === "./nested//worker.js"
? "dist/nested/worker.js"
: "dist/worker.js",
});
},
);
@@ -551,22 +551,24 @@ const contextualCandidate = (
return unresolvedCandidate("rejected", [
"The URL reference path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
// Empty URL segments are significant; canonical filesystem paths cannot
// preserve them, including their interaction with following dot segments.
if (decoded.includes("//"))
return unresolvedCandidate("unavailable", [
"Repeated URL path separators cannot be mapped to an exact canonical artifact file.",
]);
return exactUrlPathCandidate(
const candidate = urlRelativeCandidate(
posix.dirname(input.sourcePath),
decoded,
relativeCandidate(input.sourcePath, decoded),
"URL",
);
return typeof candidate === "string"
? exactUrlPathCandidate(decoded, candidate, "URL")
: candidate;
}
const candidate = relativeCandidate(input.sourcePath, declared);
return context === "module-specifier" && input.moduleKind === "import"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
if (context === "module-specifier" && input.moduleKind === "import") {
const candidate = urlRelativeCandidate(
posix.dirname(input.sourcePath),
declared,
);
return typeof candidate === "string"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
}
return relativeCandidate(input.sourcePath, declared);
};
const decodeModuleUrlPath = (
@@ -593,6 +595,78 @@ const relativeCandidate = (sourcePath: string, declared: string): string =>
? declared.slice(1)
: posix.join(posix.dirname(sourcePath), declared);
/**
* A URL path during reference resolution. Above-root `..` markers are kept
* apart from segments, so a later `..` cannot cancel an escape, while empty
* segments stay removable until the final path is formed.
*/
interface UrlPathState {
readonly escapes: number;
readonly segments: readonly string[];
}
const urlDirectoryState = (directory: string): UrlPathState => ({
escapes: 0,
segments: directory === "" || directory === "." ? [] : directory.split("/"),
});
/**
* Apply URL dot segments: `..` removes the previous segment even when it is
* empty, so `assets//../app.js` names `assets/app.js`. A final `.` or `..`
* leaves directory form, as the URL parser does.
*/
const applyUrlReference = (
base: UrlPathState,
reference: string,
): UrlPathState => {
const absolute = reference.startsWith("/");
const segments = absolute ? [] : [...base.segments];
let escapes = absolute ? 0 : base.escapes;
const parts = (absolute ? reference.slice(1) : reference).split("/");
for (const [index, part] of parts.entries()) {
if (part !== "." && part !== "..") {
segments.push(part);
continue;
}
if (part === "..")
if (segments.length > 0) segments.pop();
else escapes += 1;
if (index === parts.length - 1) segments.push("");
}
return { escapes, segments };
};
/** A relative reference replaces the base URL's final path segment. */
const urlBaseDirectory = (base: UrlPathState): UrlPathState => ({
escapes: base.escapes,
segments: base.segments.slice(0, -1),
});
/**
* Form the artifact candidate. A remaining empty segment cannot name a
* canonical artifact file; a trailing one is directory form. Confinement
* rejects a candidate that keeps an above-root marker.
*/
const urlPathCandidate = (path: UrlPathState): string | CandidateResolution => {
const segments =
path.segments.at(-1) === "" ? path.segments.slice(0, -1) : path.segments;
if (segments.includes(""))
return unresolvedCandidate("unavailable", [
"Repeated URL path separators cannot be mapped to an exact canonical artifact file.",
]);
return [
...Array.from({ length: path.escapes }, () => ".."),
...segments,
].join("/");
};
/** Resolve a URL path reference against a canonical directory. */
const urlRelativeCandidate = (
directory: string,
reference: string,
): string | CandidateResolution =>
urlPathCandidate(applyUrlReference(urlDirectoryState(directory), reference));
const bareModuleCandidate = (
input: ResolveArtifactPathInput,
): CandidateResolution => {
@@ -1066,11 +1140,12 @@ const htmlCandidate = (
return unresolvedCandidate("rejected", [
"The HTML reference path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
if (declaredPath.startsWith("/")) return declaredPath.slice(1);
if (declaredPath.startsWith("/"))
return urlRelativeCandidate("", declaredPath);
if (base === undefined || base === null || base === "")
return declaredPath === ""
? input.sourcePath
: posix.join(posix.dirname(input.sourcePath), declaredPath);
: urlRelativeCandidate(posix.dirname(input.sourcePath), declaredPath);
// A local base href is a second untrusted path input; apply the same
// admission rules a declared path gets so it cannot smuggle traversal or
// separator syntax past canonicalization.
@@ -1083,12 +1158,21 @@ const htmlCandidate = (
return unresolvedCandidate("rejected", [
"The document base href path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
const basePath = decodedBase.startsWith("/")
? decodedBase.slice(1)
: posix.join(posix.dirname(input.sourcePath), decodedBase);
if (declaredPath === "")
return exactUrlPathCandidate(decodedBase, basePath, "HTML");
return posix.join(htmlBaseDirectory(decodedBase, basePath), declaredPath);
// Keep the base's empty segments and escapes until the reference is
// applied; only the final path must name a canonical artifact file.
const baseState = applyUrlReference(
urlDirectoryState(posix.dirname(input.sourcePath)),
decodedBase,
);
if (declaredPath === "") {
const basePath = urlPathCandidate(baseState);
return typeof basePath === "string"
? exactUrlPathCandidate(decodedBase, basePath, "HTML")
: basePath;
}
return urlPathCandidate(
applyUrlReference(urlBaseDirectory(baseState), declaredPath),
);
};
const exactUrlPathCandidate = (
@@ -1112,9 +1196,6 @@ const urlPathIsDirectory = (path: string): boolean =>
path === "." ||
path === "..";
const htmlBaseDirectory = (base: string, basePath: string): string =>
urlPathIsDirectory(base) ? basePath : posix.dirname(basePath);
const confineCandidate = (candidate: string): string | CandidateResolution => {
const normalized = normalizeJoinedLogicalPath(candidate);
if (logicalPathEscapesRoot(normalized))
@@ -6,6 +6,7 @@ const paths = [
"assets/app.js",
"renderer/app.js",
"renderer/a/app.js",
"renderer/assets/app.js",
"app.js",
];
const files = new Map<string, JavaScriptArtifactFile>(
@@ -105,6 +106,26 @@ describe("HTML base href dot segments", () => {
},
);
});
describe("HTML base href empty segments", () => {
it("lets the script reference remove an empty segment from the base", () => {
// URL resolution keeps the base's empty segment until `..` removes it.
expect(
new URL("../app.js", new URL("./assets//", documentUrl)).pathname,
).toBe("/renderer/assets/app.js");
expect(resolve("../app.js", "./assets//")).toMatchObject({
resolution_status: "resolved",
resolved_path: "renderer/assets/app.js",
});
});
it("keeps an above-root base escape when the reference also climbs", () => {
expect(resolve("../app.js", "../../")).toMatchObject({
resolution_status: "rejected",
limitations: [
"The resolved candidate escapes the canonical artifact root.",
],
});
});
});
describe("HTML base href path syntax", () => {
it.each(["%2e%2e/secret/", "/a/%2e%2e/", "/a\\b/", "a\0b"])(
"rejects base href %j with the same admission rules a declared path gets",
@@ -105,8 +105,8 @@ export const addJavaScriptSourceModules = (
label: file.path,
properties: {
semantic_role: "source-module",
source_sha256: file.sha256,
path: file.path,
container_sha256: file.container_sha256,
logical_module_key: file.path,
module_format: moduleFormat(file.path, semantic.ir),
import_relationships: semantic.ir.moduleLinks.filter(
@@ -421,6 +421,33 @@ it("keeps duplicate function fingerprints ambiguous", () => {
expect(query.summary.total_seed_matches).toBe(2);
});
it("selects analyzer-produced boundaries by their method field", () => {
const graph = graphFor("const port = Number(process.env.PORT);");
const boundary = graph.nodes.find(({ kind }) => kind === "boundary");
if (boundary === undefined) throw new Error("Expected a boundary node");
expect(boundary.properties).toMatchObject({
boundary_kind: "coerce",
field: "Number",
method: "Number",
});
expect(
queryJavaScriptSemanticGraph(graph, {
seed: { kind: "boundary-field", field: "Number" },
direction: "forward-influence",
}),
).toMatchObject({
seed_node_ids: [boundary.node_id],
summary: { total_seed_matches: 1 },
});
expect(
queryJavaScriptSemanticGraph(graph, {
seed: { kind: "boundary-field", field: "missing-field" },
direction: "forward-influence",
}).summary.total_seed_matches,
).toBe(0);
});
const semanticNodeFor = (roleKey: string) => {
const file: JavaScriptArtifactFile = {
path: "budget.js",
@@ -263,6 +263,7 @@ const addBoundaryNode = (
null),
properties: {
boundary_kind: operation.kind,
field: operation.method,
method: operation.method,
},
});
+9 -2
View File
@@ -3,17 +3,24 @@ import type {
ExecutionOptions,
ProviderAvailability,
} from "../AnalysisProvider.js";
import type { JebRequest } from "../../domain/jeb/jebAnalysis.js";
import type { JebOperation, JebRequest } from "../../domain/jeb/jebAnalysis.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
/** Live engine readiness and the operations advertised by that engine. */
export type JebAnalysisAvailability =
| (Extract<ProviderAvailability, { status: "available" }> & {
readonly supportedOperations: readonly JebOperation[];
})
| Extract<ProviderAvailability, { status: "unavailable" }>;
/**
* JEB inspection boundary served by a caller-supplied running engine client.
* Implementation details belong to the provider adapter.
*/
export interface JebAnalysisPort {
/** Probe the running JEB client without opening a project. */
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
inspectAvailability(signal?: AbortSignal): Promise<JebAnalysisAvailability>;
/** Release engine connection state; the JEB client itself keeps running. */
close(): Promise<void>;
execute(
@@ -27,6 +27,12 @@ const fakePort = (
code: null,
reason: null,
diagnostics: {},
supportedOperations: [
"inspect_jeb_client",
"list_jeb_units",
"open_jeb_project",
"decompile_jeb_item",
],
};
},
async close() {},
@@ -257,6 +257,56 @@ describe("native UI action trace", () => {
}),
);
});
it("carries source-owned reference limitations into trace output without changing coverage", async () => {
const limitation = "SEH handler targets may be outside the function body.";
const run = async (limitations: readonly string[]) => {
const result = await new EnhancedTools(
testAnalysis({
procedure_address: () => execution("procedure_address", "0x1000"),
procedure_references: () =>
ok(
createAnalysisExecution(
{
reference_kinds_available: false,
references: [],
unresolved_calls: [],
},
{ id: "ghidra", name: "Ghidra", version: "fixture" },
{ limitations },
),
),
procedure_callees: (parameters) =>
execution(
"procedure_callees",
parameters.procedure === "0x1000" ? ["0x2000"] : [],
),
}),
).execute("trace_native_ui_action", { action: "0x1000" });
if (!result.ok) throw result.error;
return nativeInvestigationTraceSchema.parse(result.value);
};
const withSourceLimitation = await run([limitation]);
const withoutSourceLimitation = await run([]);
expect(withSourceLimitation.limitations).toContain(
`Ghidra (ghidra fixture) procedure_references: ${limitation}`,
);
expect(
withSourceLimitation.limitations.filter((entry) =>
entry.endsWith(`procedure_references: ${limitation}`),
),
).toHaveLength(1);
expect(withSourceLimitation.coverage).toEqual(
withoutSourceLimitation.coverage,
);
expect(withSourceLimitation.coverage).toContainEqual(
expect.objectContaining({
facet: "static_native_calls",
status: "partial",
}),
);
});
it("accepts an explicit native seed without an authored UI connection", async () => {
const result = await new EnhancedTools(
testAnalysis({
@@ -34,6 +34,12 @@ describe("untyped native call route fallback", () => {
operation === "procedure_references"
? referenceProvider
: fallbackProvider,
{
limitations:
operation === "procedure_references"
? ["Reference-kind metadata is incomplete."]
: ["Callee enumeration is limited to named functions."],
},
),
);
},
@@ -54,6 +60,18 @@ describe("untyped native call route fallback", () => {
resolution: "inferred",
})),
unknowns: [{ ...unknown, provider: referenceProvider }],
limitations: [
{
provider: referenceProvider,
operation: "procedure_references",
limitation: "Reference-kind metadata is incomplete.",
},
{
provider: fallbackProvider,
operation: "procedure_callees",
limitation: "Callee enumeration is limited to named functions.",
},
],
},
});
},
@@ -105,6 +123,7 @@ describe("typed native call routes", () => {
],
},
provider,
{ limitations: ["SEH handler targets may be outside the body."] },
),
),
},
@@ -126,6 +145,13 @@ describe("typed native call routes", () => {
},
],
unknowns: [{ address: "0x1008" }],
limitations: [
{
provider,
operation: "procedure_references",
limitation: "SEH handler targets may be outside the body.",
},
],
},
});
});
@@ -146,12 +172,26 @@ describe("typed native call routes", () => {
});
expect(calls).toBe(1);
const malformed = await readNativeCallRoutes(
{ execute: async () => ok(createAnalysisExecution({}, provider)) },
{
execute: async () =>
ok(
createAnalysisExecution({}, provider, {
limitations: ["Malformed output still has source caveats."],
}),
),
},
"0x1000",
);
expect(malformed).toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
limitations: [
{
provider,
operation: "procedure_references",
limitation: "Malformed output still has source caveats.",
},
],
});
});
it("rejects omitted unresolved-call observations instead of claiming none", async () => {
+61 -28
View File
@@ -9,6 +9,14 @@ import {
referenceKindSchema,
} from "../../domain/hopperValues.js";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import type { SourceLimitation } from "../SourceLimitations.js";
const limitationsFrom = (
operation: SourceLimitation["operation"],
provider: ProviderIdentity,
limitations: readonly string[],
): SourceLimitation[] =>
limitations.map((limitation) => ({ provider, operation, limitation }));
const referencesSchema = z.object({
reference_kinds_available: z.boolean(),
@@ -24,6 +32,33 @@ const referencesSchema = z.object({
z.object({ address: z.string(), reason: z.string() }),
),
});
const typedCallRoutes = (
references: z.infer<typeof referencesSchema>["references"],
) => {
const calls = references.filter(({ kind }) => kind.available && kind.call);
const countBySite = new Map<string, Set<string>>();
for (const call of calls) {
const targets = countBySite.get(call.source_address) ?? new Set<string>();
targets.add(call.target_address);
countBySite.set(call.source_address, targets);
}
return calls.map((call) => ({
address: call.target_procedure?.address ?? call.target_address,
site: call.source_address,
relation:
call.kind.available && (call.kind.computed || call.kind.indirect)
? ("indirect_call" as const)
: ("direct_call" as const),
resolution:
(countBySite.get(call.source_address)?.size ?? 0) > 1
? ("ambiguous" as const)
: call.kind.available && (call.kind.computed || call.kind.indirect)
? ("resolved" as const)
: ("observed" as const),
}));
};
/** Prefer typed static call references, preserving computed targets and targetless sites. */
export const readNativeCallRoutes = async (
analysis: AnalysisOperationPort,
@@ -45,7 +80,13 @@ export const readNativeCallRoutes = async (
reason: string;
provider: ProviderIdentity;
}> = [];
let limitations: SourceLimitation[] = [];
if (execution.ok) {
limitations = limitationsFrom(
"procedure_references",
execution.value.provider,
execution.value.limitations,
);
const parsed = referencesSchema.safeParse(execution.value.result);
if (!parsed.success)
return {
@@ -55,6 +96,7 @@ export const readNativeCallRoutes = async (
"Invalid typed call references",
),
unknowns,
limitations,
};
// Missing detailed reference flags do not invalidate observed targetless
// calls. Keep their producing provider even if another provider supplies
@@ -68,48 +110,38 @@ export const readNativeCallRoutes = async (
parsed.data.reference_kinds_available === true ||
(references.length > 0 && references.every(({ kind }) => kind.available))
) {
const calls = references.filter(
({ kind }) => kind.available && kind.call,
);
const countBySite = new Map<string, Set<string>>();
for (const call of calls) {
const targets =
countBySite.get(call.source_address) ?? new Set<string>();
targets.add(call.target_address);
countBySite.set(call.source_address, targets);
}
return {
ok: true as const,
value: {
provider: execution.value.provider,
routes: calls.map((call) => ({
address: call.target_procedure?.address ?? call.target_address,
site: call.source_address,
relation:
call.kind.available && (call.kind.computed || call.kind.indirect)
? ("indirect_call" as const)
: ("direct_call" as const),
resolution:
(countBySite.get(call.source_address)?.size ?? 0) > 1
? ("ambiguous" as const)
: call.kind.available &&
(call.kind.computed || call.kind.indirect)
? ("resolved" as const)
: ("observed" as const),
})),
routes: typedCallRoutes(references),
unknowns,
limitations,
},
};
}
} else if (execution.error._tag !== "AnalysisCapabilityUnavailableError")
return { ok: false as const, error: execution.error, unknowns };
return {
ok: false as const,
error: execution.error,
unknowns,
limitations,
};
const callees = await analysis.execute(
"procedure_callees",
{ procedure },
options,
);
if (!callees.ok)
return { ok: false as const, error: callees.error, unknowns };
return { ok: false as const, error: callees.error, unknowns, limitations };
limitations = [
...limitations,
...limitationsFrom(
"procedure_callees",
callees.value.provider,
callees.value.limitations,
),
];
const parsed = parseRelatedAddresses(callees.value.result, "callees");
return parsed.ok
? {
@@ -123,7 +155,8 @@ export const readNativeCallRoutes = async (
resolution: "inferred" as const,
})),
unknowns,
limitations,
},
}
: { ok: false as const, error: parsed.error, unknowns };
: { ok: false as const, error: parsed.error, unknowns, limitations };
};
@@ -14,6 +14,10 @@ import { nativeDispatchMetadataResultSchema } from "../../domain/native/objcSwif
import { inspectNativeDispatch } from "./NativeDispatchMetadataInspection.js";
import { jsonValueSchema } from "../../domain/jsonValue.js";
import { readNativeCallRoutes } from "./NativeCallRoutes.js";
import {
renderSourceLimitations,
type SourceLimitation,
} from "../SourceLimitations.js";
import { err, ok } from "../../domain/result.js";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
@@ -244,6 +248,10 @@ export const traceNativeUiAction = async (
: graph.truncated && trace.reason === null
? "native_trace_output_bound_reached"
: trace.reason,
limitations: [
...trace.limitations,
...renderSourceLimitations(expanded.limitations),
],
}),
),
);
@@ -398,6 +406,7 @@ const addDirectCallees = async (input: {
let examined = 0;
let omitted = 0;
let truncated = false;
const sourceLimitations: SourceLimitation[] = [];
while (queueIndex < queue.length) {
const current = queue[queueIndex++];
if (current === undefined || visited.has(current.address)) continue;
@@ -419,6 +428,9 @@ const addDirectCallees = async (input: {
"provider",
);
examined += 1;
sourceLimitations.push(
...(execution.ok ? execution.value.limitations : execution.limitations),
);
if (!execution.ok && execution.error._tag === "AnalysisCancelledError")
return { error: execution.error };
const unknowns = execution.ok
@@ -539,6 +551,7 @@ const addDirectCallees = async (input: {
examined,
omitted,
truncated,
limitations: sourceLimitations,
};
};
+58 -16
View File
@@ -291,31 +291,32 @@ const nativeFlowDossier = (
});
};
const callBindingAnalysis = (
callerAddress: string,
calleeAddress: string,
caller: ReturnType<typeof jsonValueSchema.parse>,
callee: ReturnType<typeof jsonValueSchema.parse>,
): AnalysisOperationPort => ({
const callBindingAnalysis = (input: {
readonly callerAddress: string;
readonly calleeAddress: string;
readonly caller: ReturnType<typeof jsonValueSchema.parse>;
readonly callee: ReturnType<typeof jsonValueSchema.parse>;
readonly targetLimitations?: readonly string[];
}): AnalysisOperationPort => ({
execute: async (operation, parameters) => {
if (operation === "resolve_native_call_targets")
return ok(
createAnalysisExecution(
{
call_site: String(parameters.address),
procedure: calleeAddress,
procedure: input.calleeAddress,
status: "direct",
mechanism: "direct",
targets: [
{
address: calleeAddress,
procedure: calleeAddress,
address: input.calleeAddress,
procedure: input.calleeAddress,
status: "direct",
basis: "provider-reference",
references: [],
},
],
limitations: [],
limitations: [...(input.targetLimitations ?? [])],
},
provider,
{ subject },
@@ -324,13 +325,15 @@ const callBindingAnalysis = (
if (operation !== "analyze_function")
throw new Error(`Unexpected native trace operation: ${operation}`);
if (
parameters.procedure !== callerAddress &&
parameters.procedure !== calleeAddress
parameters.procedure !== input.callerAddress &&
parameters.procedure !== input.calleeAddress
)
throw new Error(`Unexpected procedure: ${String(parameters.procedure)}`);
return ok(
createAnalysisExecution(
parameters.procedure === callerAddress ? caller : callee,
parameters.procedure === input.callerAddress
? input.caller
: input.callee,
provider,
{ subject },
),
@@ -430,6 +433,40 @@ describe("native value failed-callee traversal", () => {
});
describe("native value call-binding fan-out", () => {
it("retains limitations embedded in resolved call-target results", async () => {
const callerAddress = "0x1000";
const calleeAddress = "0x2000";
const base = functionDossierSchema.parse(ghidraFunctionDossier());
const caller = nativeFlowDossier(base, {
address: callerAddress,
name: "caller",
operations: [node("call", callerAddress, "CALL")],
parameters: [],
});
const callee = nativeFlowDossier(base, {
address: calleeAddress,
name: "callee",
operations: [],
parameters: [],
});
const caveat = "Only statically resolved call sites are included.";
const result = await traceNativeValues(
callBindingAnalysis({
callerAddress,
calleeAddress,
caller,
callee,
targetLimitations: [caveat],
}),
{ procedure: callerAddress, max_functions: 2 },
);
if (!result.ok) throw result.error;
expect(nativeValueTraceSchema.parse(result.value).limitations).toContain(
`Fixture (fixture 1) resolve_native_call_targets: ${caveat}`,
);
});
it("preserves binding order and edge-budget behavior for repeated callees", async () => {
const callCount = 128;
const callerAddress = "0x1000";
@@ -455,12 +492,12 @@ describe("native value call-binding fan-out", () => {
operations: [node("return", calleeAddress, "RETURN")],
parameters: [{ ordinal: 0, name: "value", data_type: "/int" }],
});
const fanoutAnalysis = callBindingAnalysis(
const fanoutAnalysis = callBindingAnalysis({
callerAddress,
calleeAddress,
caller,
callee,
);
});
const expectedEdges = [
...Array.from({ length: callCount }, (_, index) => ({
source: `${callerAddress}/call-${index}`,
@@ -803,7 +840,12 @@ describe("native value byte-bound call bindings", () => {
],
});
const result = await traceNativeValues(
callBindingAnalysis("0x1000", "0x2000", caller, callee),
callBindingAnalysis({
callerAddress: "0x1000",
calleeAddress: "0x2000",
caller,
callee,
}),
{ procedure: "0x1000" },
);
if (!result.ok) throw result.error;
Loaded 100 of 330 files, more files were not shown because too many files have changed in this diff. Show more