Compare commits

...
Author SHA1 Message Date
Kaoru0822 98b87e7fd9 Merge remote-tracking branch 'origin/main' into sweep/2644
# Conflicts:
#	tests/process-global/cleanLinuxHopperRuntime.test.ts
2026-10-15 07:30:06 +00:00
Kaoru0822andClaude Opus 5.5 6fcfc5b5e0 fix(browser): keep malformed Electron CDP inventories incomplete (#3901)
inspect-electron-page read missing or non-array DOMSnapshot documents and
node arrays, frame-tree resource lists, and Target.getTargets target lists
as empty inventories while reporting complete_within_window.

The web capture path already validated these replies. Its parsing now has
one shared owner in CdpCaptureDocuments: snapshotDocuments,
snapshotDocumentNodes and frameTreeResources report unreadable parts
through a callback, and both the web and Electron inspections consume
them. Worker discovery records a missing target list or non-record entry as
an invalid_protocol_value exclusion. The Electron copies of this parsing are
removed.

Fixes #3379

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 00:09:38 -07:00
pablodiazjorge 35e17a7172 fix(captures): admit schema-valid HAR numbers that evidence cannot publish exactly (#3159)
* fix(captures): admit schema-valid HAR numbers that evidence cannot publish exactly

* fix(captures): judge overflowing HAR integers by their lexeme
2026-10-15 16:09:07 +09:00
pablodiazjorge f8f3593a8c test: resolve script paths from file URLs with fileURLToPath on Windows (#3266)
* test: resolve worker and CI script paths with fileURLToPath on Windows

* test: skip the fake gh lookup failure on Windows

* test: resolve the release helper and Hopper probe paths with fileURLToPath
2026-10-15 16:07:57 +09:00
Aaditya ed4a1c7d95 docs(installation): list every client that first shipped in 6.5.0 (#3465)
docs(installation): list every client missing from npm 6.4.0

The installation guide named only dsh, or dsh, kimi and workbuddy, as
needing a newer release. The rea-agents-6.4.0 tag also lacks droid,
qoder, trae and zcode, so setup against @latest rejects all seven as an
unsupported agent integration.

List all seven client IDs in both the registry-check note and the
supported-agents table note.
2026-10-15 16:07:42 +09:00
Wade 7a83af3cba fix(process): keep observed symlink targets in filesystem snapshots (#3418)
Snapshots stored resolve(dirname(link), readlink(link)), a lexical
absolute path. Two different targets such as aaa/../target and
bbb/../target collapse to the same value, and with equal lengths the
before/after comparison reported a retargeted link as unchanged even
when the new link is broken. Store the target exactly as readlink
returns it.

Fixes #3303
2026-10-15 16:07:28 +09:00
Ysopwnable f5b1dce881 docs: celebrate 250,000 GitHub stars 🎉 (#3971) 2026-10-15 15:07:07 +08:00
Rudy Mizrahi CelekliandClaude Sonnet 5.5 73c72d5745 fix(mcp): keep a snapshot open ahead of later calls on the same connection (#3661)
open_binary read snapshot_path before it entered the session's lifecycle
order, so a current_document or close_binary dispatched after it could
run first: the read returned Evidence for the previous target and the
close left the snapshot target open. The session now loads the snapshot
inside the open's own turn.

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-15 16:05:44 +09:00
ShuaixinHuang ce89706145 fix(javascript): preserve exact static property keys (#3480)
* fix(javascript): preserve Electron options with static empty keys

* fix(javascript): retain exact computed member keys
2026-10-15 15:59:06 +09:00
Harsh Asalkar 3e88e21705 fix(electron): reject active scenario capture when inspect fuse is disabled (#3453) (#3605)
* feat(electron): add EnableNodeCliInspectArguments fuse pre-flight helper

* fix(electron): reject active scenario capture when inspect fuse is disabled (#3453)

* docs(electron): document inspect fuse pre-flight refusal for scenario capture (#3453)

* test(electron): add PE, ELF, and abort-signal coverage for fuse pre-flight (#3453)

* fix(electron): resolve macOS framework binary and enforce startup deadline on fuse preflight (#3453)
2026-10-15 15:57:42 +09:00
Wade d94ef22d70 fix(cli): route app bundles with trailing separators to native analysis (#3511) 2026-10-15 15:55:44 +09:00
Lucien c61a24b8c5 fix(browser): bound CDP handshakes and command responses (#3555)
* fix(browser): bound CDP handshakes and command responses

* test(browser): expire stalled deadlines after fixture readiness

* docs(browser): explain handshake deadline ownership
2026-10-15 15:55:29 +09:00
Kaoru0822andClaude Opus 5.5 fc712b3750 fix(javascript): keep releases through a replaced handle as candidates (#3897)
A release call was linked to its acquisition through the receiver's lexical
binding alone, so `stream = { destroy() {} }; stream.destroy()` still
produced a resolved `releases` edge to the original stream and entered the
default ownership trace.

A release is now complete only when every value written to the handle
binding is the linked acquisition itself (through parentheses, TypeScript
wrappers or `await`) or a nullish placeholder such as `let h = null`, which
cannot be released. Any other write, conditional replacement, or
destructuring keeps the linked release as a candidate (partial).

Fixes #3313

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:40:14 -07:00
Kaoru0822andClaude Opus 5.5 ab45bf641e fix(ghidra): refuse PSP targets with unexamined ABI declarations (#3904)
The generic Ghidra MIPS lane refuses a target whose ABI declaration
inspection is incomplete, such as a section-header table beyond the
captured file, because unexamined headers may hold declarations that
would refuse it. The PSP Allegrex lane skipped that check. It admitted
the target and resolved a supported profile while target inspection
said those declarations remained unknown.

The incomplete-declaration rule is now one shared helper,
ghidraMipsIncompleteDeclarationsReason, used by both the generic MIPS
and the PSP lanes.

Fixes #3329

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:30:12 -07:00
Kaoru0822andClaude Opus 5.5 57f30029b1 fix(javascript): reject Webpack getters replaced through any identity write (#3860)
The canonical `n.n` getter proof scanned for direct `loader.n =` assignments
and four Object methods on its own. `Reflect.set`, `Reflect.defineProperty`,
`delete`, destructuring and unknown-key writes left the proof intact, so a
replaced or deleted getter still attributed BrowserWindow calls to Electron.
Any Object method call on the loader also rejected the proof, even for an
unrelated key.

The proof now uses the shared identity-write owner
(`collectJavaScriptIdentityWritesSteps` with the module interop's intrinsic
resolution), as ambient writes already do. A write whose path may select `n`
(the key itself, an unknown key, or the whole loader) counts; exactly one
canonical assignment installed before entry dispatch remains required.

Fixes #3315

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:25:18 -07:00
morluto 22ec8bdb2f Merge remote-tracking branch 'origin/main' into codex/ci-followups-20261015 2026-10-15 14:11:13 +08:00
morluto a715fb9e6d test(browser): bound atomic environment polling across navigation 2026-10-15 14:11:13 +08:00
morluto 85788452dd test(cli): batch redundant client lifecycle dry runs 2026-10-15 14:11:13 +08:00
Kaoru0822andClaude Opus 5.5 1b036ecbff fix(flutter): check section-backed symbol tables against PT_LOAD (#3859)
* fix(flutter): check section-backed symbol tables against PT_LOAD

The section-backed ELF path read .dynsym and .dynstr only through their
section file offsets. The stripped PT_DYNAMIC path and snapshot symbols
resolve through the PT_LOAD map. A second PT_LOAD interpretation over an
allocated symbol or string table was therefore ignored, and
inspect-dart-aot reported complete coverage from metadata the loader may
read differently.

An allocated (SHF_ALLOC) dynamic symbol table and its string table now
need a single file-backed PT_LOAD interpretation that matches the section
offset, the same check the dynamic path applies to its tables. Otherwise
the symbol inventory is refused with the mapping reason, and
inspect-dart-aot reports that no recognizable snapshot selection remains.
Non-allocated tables keep their section-offset reading.

Fixes #3346

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(flutter): cover allocated ELF32 symbol-table mapping checks

Add an ELF32 (ARM) counterpart of the synthetic ELF64 image and run the
allocated-table cases for both classes, so the ELF32 section-header reads
in the mapping check are exercised: matching mappings, conflicting symbol
and string mappings, and a disagreeing section offset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:02:49 -07:00
933f216698 feat(kuna): add Kuna readiness, doctor and owned project cache (#3832)
Add caller-supplied Kuna readiness, scoped doctor support and a verified whole-program export cache. Index function byte ranges without introducing analysis-tool responses, and retain unresolved process ownership through cleanup failures. Separate retryable file cleanup from unproven process cleanup.

Verified real Kuna 1.833 on Debian 12 x86-64 with coreutils, SQLite and zlib; 5,207 exported function spans, cold/warm performance, raw-export parity and lifecycle failures. Refs #1957.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 05:56:56 +00:00
Kaoru0822andClaude Opus 5.5 cf0a38ad97 fix(managed): project metadata with empty names without a display label (#3853)
The managed application graph copied metadata names into observation
labels. The graph schema requires a non-empty label or null, so Evidence
whose Module name was the string heap's empty entry made
project-managed-application-graph fail with invalid_request at
observations.0.label. The inspector accepts that Evidence as complete,
and no graph was returned.

Labels derived from metadata names (assembly, module, type, member,
P/Invoke import and native implementation) now become null when the name
is empty. The node's properties keep the source value, so display
formatting no longer rejects identity facts.

Fixes #3352

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:22:59 -07:00
a1ca8c76af fix(website): give navigation room and refresh contributors (#3851)
Give the header its own 1320px container while preserving body/article widths, move the complete navigation below the controls on smaller screens, and refresh the dated public contributor snapshot from 214 to 216 accounts.

Validated shared HTML/translations/links/assets and 16 maintenance tests; local Chromium acceptance covered 20 languages, 15 viewport widths and both root/project paths. CI and Greptile review passed.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-15 05:18:54 +00:00
Kaoru0822andClaude Opus 5.5 034342ba28 fix(reference): select a package main before its directory index (#3844)
* fix(reference): select a package main before its directory index

import-reference-source resolved a relative directory import such as
require("./dep") from path candidates alone, so it reported dep/index.js
as the internal, parsed dependency even when dep/package.json declared
"main": "actual.js". Node loads actual.js.

parseReferenceSourceEntries now reads each inventoried package.json once.
When an internal specifier resolves through a directory index, a
non-empty string main is loaded first, as a file and then as a directory
index, as Node does. A main whose target is absent falls back to the
index, matching Node's legacy fallback. An unreadable or invalid
manifest leaves the relationship unresolved instead of asserting the
index.

Fixes #3383

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(reference): resolve package main the way path.resolve does

A trailing separator in main ("lib/") kept directory intent, so the
declared entry selected lib/index.js where Node loads lib.js. An absolute
main was joined under the package directory instead of leaving the
imported root. Strip trailing separators after joining, and report an
absolute main as unresolved. Both cases join the Node-oracle CLI test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:02:45 -07:00
Kaoru0822andClaude Opus 5.5 788964d52c perf(javascript): index object literal members for exact-key reference lookup (#3836)
semanticObjectPropertyReferences scanned every property of an object
literal on each call, and member-mutation tracking calls it once per
write, so N writes to an N-property literal cost O(N^2).

For an exact string or number key, a property with another static name
only updates override bookkeeping for that other name, which the key
never reads. Index each literal once by static name, keeping spreads,
computed keys and __proto__ data entries for every key, and walk only
the relevant members in reverse source order. Wildcard and selection
keys keep the full scan.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:44:14 -07:00
Kaoru0822andClaude Opus 5.5 acca91d22b fix(javascript): keep members unknown after a write through a local setter (#3831)
* fix(javascript): keep members unknown after a write through a local setter

Assigning to a key that a setter of the binding's own object literal
defines runs that setter, which may replace any member of its receiver.
The member-mutation collector recorded it as an ordinary write to that
one key, so sibling literals such as value: 1 stayed known and
export-shape comparisons reported complete, unchanged shapes.

Record such a write with an unknown key, which invalidates the object's
known members. A computed setter key may match any written key.

Fixes #3312

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(javascript): follow nested setters, skip deletes and cache setter keys

Resolve the written receiver along nested object-literal members so a
write through a nested setter widens that receiver's members. A delete
never calls a setter, so it keeps its exact path. Setter keys are
computed once per object literal, including the no-setter result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(javascript): ignore setters replaced by a later data property

Build cached setter keys from the effective definitions: a later data
property or method with the same static name replaces the accessor, so
assigning that key no longer runs a setter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:39:39 -07:00
Kaoru0822andClaude Opus 5.5 0323ce0a28 fix(javascript): keep the request endpoint on a uniquely consumed response (#3822)
* fix(javascript): keep the request endpoint on a uniquely consumed response

collectResponseConsumers always produced response-consumer operations
with endpoint null, so a json()/text() consumer that resolved to exactly
one request lost the request's literal endpoint in the semantic graph.

When the consumer resolves to one request through that request's own
result binding, the same condition that already makes it complete, it
now carries that request's endpoint. Ambiguous consumers stay null.

Fixes #3328

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(javascript): drop the endpoint when another write can replace the response

Copy the request's endpoint only when the request is the binding's sole
write; any other assignment, update, loop or destructuring target can
replace the response before the consumer reads it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:39:26 -07:00
Kaoru0822andClaude Opus 5.5 089ef732e8 fix(apktool): own the version probe's process group and keep its timeout typed (#3808)
* fix(apktool): own the version probe's process group and keep its timeout typed

The version probe ran its launcher through execFileOutput, which waits
for every inherited output pipe to close. A shell launcher's child that
holds those pipes outlived the stop request, so the probe stayed pending
past its deadline and its grace period, and cancellation hung the same
way. An expired probe also surfaced as a non-retryable unusable-version
refusal.

Run the probe through runOwnedCommand like the decode run, so the owned
process group is stopped with its children. Report its deadline as
AnalysisTimeoutError with timeout_ms and retain an unresolved cleanup
owner for close. Apktool launchers are interpreter scripts, so both the
probe and the decode run prove ownership by run token instead of the
launcher's argv prefix. A close with unresolved process cleanup reports
a typed ProviderCleanupError instead of an untyped AggregateError.

Fixes #3342
Fixes #3347
Fixes #3337

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apktool): carry unresolved probe cleanup through both provider entry points

A probe failure other than cancellation or timeout, such as an output
overflow whose launcher child survives, now keeps its cleanup reason and
resources on ApktoolConfigurationFailure. inspectAvailability and execute
report it as ProviderCleanupError instead of ordinary unavailability or
capability_unavailable.

Add a real-process lifecycle test: an interpreter launcher starts a child
that holds the inherited output pipes, the probe is cancelled after
launch, and both processes must be stopped within the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(apktool): ignore zombies and always clean up the probe lifecycle fixture

Treat a zombie as exited when checking the launcher and child, bound the
cancellation wait so a regressed probe still reaches cleanup, and stop
any remaining fixture processes in a finally block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(apktool): treat a failed process inspection as unknown, not stopped

Only ESRCH, a zombie state or ps reporting no match counts as exited; a
ps launch failure leaves the process presumed live so the assertion
fails and the finally block still stops it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 21:37:52 -07:00
morluto 4c99912e8d chore(main): merge pinned CI checkpoint 2895a18 2026-10-14 03:12:47 +08:00
morluto b3a60b6b03 test(ci): provide proc metadata for Linux hopper fixtures 2026-10-14 03:02:01 +08:00
morluto d5b5441a3a fix(hopper): refresh runtime cleanup against main 2026-10-14 02:27:23 +08:00
morluto 4f83c64da8 fix(hopper): merge current main into runtime cleanup 2026-10-14 02:12:58 +08:00
Morluto 780191226c test(hopper): run Python-named fixture through Node eval 2026-10-14 02:03:25 +08:00
morluto 1738989590 test: isolate hopper scanner process fixtures 2026-10-14 01:26:30 +08:00
morluto 3f18439acd fix(types): declare hopper inventory proc root 2026-10-14 01:24:48 +08:00
morluto dd401a2804 test(process): isolate hopper inventory proc root 2026-10-14 01:06:02 +08:00
morluto 2bc898fae2 chore: merge captured main into Hopper runtime cleanup 2026-10-13 21:45:06 +08:00
N0zoM1z0andKaoru0822 b2e743ef24 test(hopper): reap orphan fixtures independently of container init
Keep a test-owned Linux subreaper until token-verified process-group cleanup completes, then wait for each adopted child. Observe from a separate verifier so the orphan remains outside its lineage. Assert disappearance from procfs after reaping.

Reproduced two failures and eight retained zombies without an init reaper. The corrected fourteen process cases pass with no residual children; typecheck and lint pass.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 12:25:38 +00:00
morluto 51b3418f2b chore: merge upstream CI baseline fixes 2026-10-13 20:13:27 +08:00
N0zoM1z0andKaoru0822 3bb09b8d2d fix(verification): retain command-less runtime observations
Use the kernel command name for processes without argv or an executable link. Preserve uncertainty for command-less Python interpreters and detect Hopper/Xvfb zombies without signaling. Real unreaped-child regressions keep cleanup with their waiting parent.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 3deda4781d fix(hopper): parse Python helper interpreter options
Resolve the script operand using CPython option boundaries, including combined flags, separate or attached option values, and --. Keep command/module arguments distinct and defer unknown invocation failures rather than declaring them clean.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 2ac4795df5 test(hopper): cover a reparented Python helper command
Launch a real Python child with the helper filename, wait for readiness before the Node parent exits, bind its procfs arguments, require non-signaling detection, and clean it through the existing run-token/group identity primitive.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 a588dc6376 fix(hopper): distinguish runtime processes from argument text
Parse procfs argument boundaries rather than searching rendered ps command text. Keep executable identity checks and deferred permission failures, eliminate the extra ps process, and put shared process observations in the serial process-boundary lane.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
124 changed files with 6935 additions and 737 deletions

No files matched your search

+1 -1
View File
@@ -344,7 +344,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **200,000 GitHub stars — thank you!**
🎉 **250,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
+4 -2
View File
@@ -343,8 +343,10 @@ an invalid supplied size remains unknown with incomplete resource coverage while
the resource's other facts are retained. Zero and nonnegative fractional sizes
are preserved.
CDP discovery, connection, and commands wait for the peer or caller cancellation;
a stalled peer can leave an operation waiting. WebSocket messages remain subject
CDP discovery has a 10-second deadline. Each WebSocket handshake and command
response has its own 20-second deadline, separate from the caller-selected
observation window. A stalled handshake or command reports `provider_timeout`;
caller cancellation still reports cancellation. WebSocket messages remain subject
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
pending-command quota. On cancellation, it closes only its own WebSocket,
waiting up to one second before terminating it. Malformed unsolicited events
+6
View File
@@ -116,6 +116,12 @@ If cleanup cannot be verified, the error retains collected observations in
`details.partial_observation` and marks their application cleanup `unverified`.
Cleanup uncertainty does not authorize signaling unrelated processes.
### Target fuse prerequisites
Active scenario capture requires that the target Electron executable honors Node CLI inspector flags (`--inspect` or `--remote-debugging-port`) so that Playwright can establish its DevTools protocol session.
When an Electron executable explicitly disables the `EnableNodeCliInspectArguments` fuse (or removes the capability), `capture-electron-scenario` fails immediately before process launch with an `AnalysisUnsupportedTargetError` ("unsupported_target") rather than waiting for the startup deadline. For targets with disabled inspector fuses, use passive CDP inspection via `inspect_electron_page` or run against an unrestricted development binary with the extracted `app.asar`.
The real verifier exercises compiled CLI and stdio MCP discovery, page/script
inspection, static/runtime reconciliation, active observation, and CLI SIGTERM
cleanup before checking the IPC, renderer crash/reload, utility/child-process,
+5 -4
View File
@@ -398,8 +398,9 @@ running server or its registration.
The npm release checked on **2026-10-11** was **6.4.0**, published from
[`a56f7bb6`](https://github.com/morluto/rea/commit/a56f7bb668ab23fec8d33f7af353ac7ed2fe1895).
The actual registry package accepted dry-run setup plans for the 17 client IDs
in that release, including Qwen Code, Pi and Hermes. DeepSeek Harness (`dsh`)
is included from 6.5.0 and was absent from the checked npm 6.4.0 package.
in that release, including Qwen Code, Pi and Hermes. Seven client IDs are
included from 6.5.0 and were absent from the checked npm 6.4.0 package: `droid`,
`dsh`, `kimi`, `qoder`, `trae`, `workbuddy` and `zcode`.
The 6.4.0 registry check does not establish 6.5.0 publication or live DeepSeek
API integration.
OpenCode's plan selected
@@ -482,8 +483,8 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
check above does not cover them.
`droid`, `dsh`, `kimi`, `qoder`, `trae`, `workbuddy`, and `zcode` require
`rea-agents@6.5.0` or later. The 6.4.0 registry check above does not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
+5
View File
@@ -95,6 +95,11 @@ pauses or resumes execution, calls `Runtime.runIfWaitingForDebugger`, launches
the target, instruments JavaScript, or invokes Electron APIs. Closing a call
closes only REA's WebSocket.
Each WebSocket handshake and command response has its own 20-second deadline,
separate from the caller-selected observation window. A stalled handshake or
command reports `provider_timeout` and retains observations already received;
caller cancellation still reports cancellation.
The direct observations are:
- `Debugger.scriptParsed`, establishing that a bounded script location was
+1 -1
View File
@@ -274,7 +274,7 @@ npx rea-agents@latest setup
## سجل النجوم
🎉 **200,000 نجمة على GitHub — شكرًا لكم!**
🎉 **250,000 نجمة على GitHub — شكرًا لكم!**
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
+1 -1
View File
@@ -272,7 +272,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](../../SECURITY.md).
## Sternverlauf
🎉 **200.000 GitHub-Sterne – vielen Dank!**
🎉 **250.000 GitHub-Sterne – vielen Dank!**
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
+1 -1
View File
@@ -285,7 +285,7 @@ npx rea-agents@latest setup
## Ιστορικό αστεριών
🎉 **200.000 αστέρια στο GitHub — σας ευχαριστούμε!**
🎉 **250.000 αστέρια στο GitHub — σας ευχαριστούμε!**
Ευχαριστούμε όλους όσοι χρησιμοποιούν το REA, αναφέρουν σφάλματα, ζητούν νέες λειτουργίες, δοκιμάζουν builds και συνεισφέρουν διορθώσεις.
+1 -1
View File
@@ -272,7 +272,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](../../SECURITY.md).
## Historial de estrellas
🎉 **200.000 estrellas en GitHub: ¡gracias!**
🎉 **250.000 estrellas en GitHub: ¡gracias!**
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
+1 -1
View File
@@ -257,7 +257,7 @@ npx rea-agents@latest setup
## تاریخچهٔ ستاره‌ها
🎉 **۲۰۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
🎉 **۲۵۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
از همهٔ کسانی که از REA استفاده می‌کنند، خطاها را گزارش می‌دهند، قابلیت‌های جدید پیشنهاد می‌کنند، buildها را آزمایش می‌کنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
+1 -1
View File
@@ -276,7 +276,7 @@ Signalez les vulnérabilités via [SECURITY.md](../../SECURITY.md).
## Historique des étoiles
🎉 **200 000 étoiles GitHub — merci !**
🎉 **250 000 étoiles GitHub — merci !**
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
+1 -1
View File
@@ -297,7 +297,7 @@ Laporkan kerentanan melalui [SECURITY.md](../../SECURITY.md).
## Riwayat Bintang
🎉 **200.000 bintang GitHub, terima kasih!**
🎉 **250.000 bintang GitHub, terima kasih!**
Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
+1 -1
View File
@@ -276,7 +276,7 @@ Segnala vulnerabilità tramite [SECURITY.md](../../SECURITY.md).
## Cronologia delle stelle
🎉 **200.000 stelle su GitHub — grazie!**
🎉 **250.000 stelle su GitHub — grazie!**
Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni.
+1 -1
View File
@@ -274,7 +274,7 @@ npx rea-agents@latest setup
## スター履歴
🎉 **GitHub スター 200,000 件、ありがとうございます!**
🎉 **GitHub スター 250,000 件、ありがとうございます!**
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
+1 -1
View File
@@ -283,7 +283,7 @@ npx rea-agents@latest setup
## 스타 기록
🎉 **GitHub 스타 200,000개, 감사합니다!**
🎉 **GitHub 스타 250,000개, 감사합니다!**
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
+1 -1
View File
@@ -272,7 +272,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](../../SECURITY.md).
## Historia gwiazdek
🎉 **200 000 gwiazdek na GitHubie — dziękujemy!**
🎉 **250 000 gwiazdek na GitHubie — dziękujemy!**
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
+1 -1
View File
@@ -272,7 +272,7 @@ Relate vulnerabilidades conforme [SECURITY.md](../../SECURITY.md).
## Histórico de estrelas
🎉 **200.000 estrelas no GitHub — muito obrigado!**
🎉 **250.000 estrelas no GitHub — muito obrigado!**
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## История звёзд
🎉 **200 000 звёзд на GitHub — спасибо!**
🎉 **250 000 звёзд на GitHub — спасибо!**
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## ประวัติดาว
🎉 **200,000 ดาวบน GitHub — ขอบคุณทุกคน!**
🎉 **250,000 ดาวบน GitHub — ขอบคุณทุกคน!**
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
+1 -1
View File
@@ -277,7 +277,7 @@ Güvenlik açıklarını [SECURITY.md](../../SECURITY.md) üzerinden bildirin.
## Yıldız geçmişi
🎉 **GitHub'da 200.000 yıldız — teşekkürler!**
🎉 **GitHub'da 250.000 yıldız — teşekkürler!**
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## Історія зірок
🎉 **200 000 зірок на GitHub — дякуємо!**
🎉 **250 000 зірок на GitHub — дякуємо!**
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
+1 -1
View File
@@ -272,7 +272,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](../../SECURITY.md).
## Lịch sử sao
🎉 **200.000 sao trên GitHub — xin cảm ơn!**
🎉 **250.000 sao trên GitHub — xin cảm ơn!**
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
+1 -1
View File
@@ -278,7 +278,7 @@ npx rea-agents@latest setup
## Star 歷史
🎉 **GitHub Star 達到 200,000 個,感謝大家!**
🎉 **GitHub Star 達到 250,000 個,感謝大家!**
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
+1 -1
View File
@@ -278,7 +278,7 @@ npx rea-agents@latest setup
## Star 历史
🎉 **GitHub Star 达到 200,000 个,感谢大家!**
🎉 **GitHub Star 达到 250,000 个,感谢大家!**
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
@@ -4,9 +4,11 @@ import { fileURLToPath } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { CdpCommandRejection } from "../../dist/browser/CdpCommandRejection.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { AnalysisCancelledError } from "../../dist/domain/analysisErrorCore.js";
import { runScenarioCli } from "./browser-scenario-verifier.mjs";
import { requireMcpEvidenceResult } from "./mcp-verifier-results.mjs";
@@ -24,12 +26,24 @@ const alternate = {
width: 700,
height: 500,
};
const expression = `JSON.stringify({
const environmentExpression = `{
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
locale: Intl.DateTimeFormat().resolvedOptions().locale,
dpr: devicePixelRatio, width: innerWidth, height: innerHeight
}`;
const expression = `JSON.stringify(${environmentExpression})`;
const scenarioSnapshotExpression = `JSON.stringify({
timeOrigin: performance.timeOrigin,
environment: ${environmentExpression},
run: document.body?.dataset.run ?? null
})`;
const isNavigationTransitionRejection = (cause) =>
cause instanceof CdpCommandRejection &&
cause.command === "Runtime.evaluate" &&
cause.code === -32_000 &&
cause.reportedMessage === "Inspected target navigated or closed";
const environmentScenario = ({ endpoint, targetId, origin, values, run }) =>
browserScenarioSchema.parse({
browser: { mode: "connect", cdp_endpoint: endpoint, target_id: targetId },
@@ -146,7 +160,6 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
endpoint,
targetId,
origin,
readEnvironment,
assertRestored,
});
} finally {
@@ -178,12 +191,7 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
assertCapture(actionFailure.value);
assert.equal(actionFailure.value.steps.at(-1).status, "failed");
await assertRestored();
await verifyCancelledEnvironment(
provider,
connection,
waiting,
readEnvironment,
);
await verifyCancelledEnvironment(provider, connection, waiting);
await assertRestored();
return {
mocked: false,
@@ -208,7 +216,6 @@ const verifyConcurrentMcpCapture = async ({
endpoint,
targetId,
origin,
readEnvironment,
assertRestored,
}) => {
const firstScenario = environmentScenario({
@@ -241,12 +248,13 @@ const verifyConcurrentMcpCapture = async ({
try {
const first = callTool(firstScenario, firstCancellation.signal);
await waitForEnvironment(readEnvironment, connection, expected, "A");
await waitForEnvironment(connection, expected, "A");
const second = callTool(secondScenario, secondCancellation.signal);
await client.ping();
assert.deepEqual(await readEnvironment(), expected);
assert.equal(await readScenarioRun(connection), "A");
const firstSnapshot = await readScenarioSnapshot(connection);
assert.deepEqual(firstSnapshot.environment, expected);
assert.equal(firstSnapshot.run, "A");
const cancelled = callTool(
environmentScenario({
@@ -261,8 +269,9 @@ const verifyConcurrentMcpCapture = async ({
await client.ping();
cancellation.abort(new Error("cancel queued browser scenario"));
await assert.rejects(cancelled, /cancel queued browser scenario/u);
assert.deepEqual(await readEnvironment(), expected);
assert.equal(await readScenarioRun(connection), "A");
const queuedSnapshot = await readScenarioSnapshot(connection);
assert.deepEqual(queuedSnapshot.environment, expected);
assert.equal(queuedSnapshot.run, "A");
await releaseScenario(connection);
assertCapture(
@@ -270,7 +279,7 @@ const verifyConcurrentMcpCapture = async ({
expected,
);
await waitForEnvironment(readEnvironment, connection, alternate, "B");
await waitForEnvironment(connection, alternate, "B");
await releaseScenario(connection);
assertCapture(
requireMcpEvidenceResult(await second, "capture_browser_scenario"),
@@ -287,31 +296,57 @@ const verifyConcurrentMcpCapture = async ({
}
};
const waitForEnvironment = async (
readEnvironment,
connection,
expectedValues,
run,
) => {
const deadline = Date.now() + 10_000;
while (Date.now() < deadline) {
if (
JSON.stringify(await readEnvironment()) ===
JSON.stringify(expectedValues) &&
(await readScenarioRun(connection)) === run
)
return;
await delay(25);
}
throw new Error(`Scenario ${run} never reached its page-controlled gate`);
const waitForEnvironment = async (connection, expectedValues, run) => {
await waitForScenarioSnapshot(
connection,
(snapshot) =>
JSON.stringify(snapshot.environment) === JSON.stringify(expectedValues) &&
snapshot.run === run,
`Scenario ${run} never reached its page-controlled gate`,
);
};
const readScenarioRun = async (connection) => {
const value = await connection.send("Runtime.evaluate", {
expression: "document.body.dataset.run ?? null",
returnByValue: true,
});
return value.result.value;
const waitForScenarioSnapshot = async (connection, accept, message) => {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000);
let lastTransitionRejection;
try {
while (!controller.signal.aborted) {
try {
const snapshot = await readScenarioSnapshot(
connection,
controller.signal,
);
if (controller.signal.aborted) break;
if (accept(snapshot)) return;
} catch (cause) {
if (
controller.signal.aborted &&
cause instanceof AnalysisCancelledError
)
break;
if (!isNavigationTransitionRejection(cause)) throw cause;
lastTransitionRejection = cause;
}
await delay(25);
}
throw new Error(message, {
cause: lastTransitionRejection ?? controller.signal.reason,
});
} finally {
clearTimeout(timer);
}
};
const readScenarioSnapshot = async (connection, signal) => {
const value = await connection.send(
"Runtime.evaluate",
{ expression: scenarioSnapshotExpression, returnByValue: true },
undefined,
signal,
);
assert.equal(value.result.type, "string");
return JSON.parse(value.result.value);
};
const releaseScenario = async (connection) => {
@@ -321,39 +356,26 @@ const releaseScenario = async (connection) => {
});
};
async function verifyCancelledEnvironment(
provider,
connection,
scenario,
readEnvironment,
) {
async function verifyCancelledEnvironment(provider, connection, scenario) {
const controller = new AbortController();
const priorNavigation = await connection.send("Runtime.evaluate", {
expression: "performance.timeOrigin",
returnByValue: true,
});
const priorSnapshot = await readScenarioSnapshot(connection);
const pending = provider.captureScenario(
{ ...scenario, actions: [{ ...scenario.actions[0], timeout_ms: 30_000 }] },
{ signal: controller.signal },
);
const settlement = Promise.allSettled([pending]);
try {
const deadline = Date.now() + 10_000;
while (true) {
const navigation = await connection.send("Runtime.evaluate", {
expression: "performance.timeOrigin",
returnByValue: true,
});
if (
navigation.result.value !== priorNavigation.result.value &&
JSON.stringify(await readEnvironment()) === JSON.stringify(expected)
)
break;
assert.ok(Date.now() < deadline, "Scenario never applied emulation");
await delay(25);
}
await waitForScenarioSnapshot(
connection,
(snapshot) =>
snapshot.timeOrigin !== priorSnapshot.timeOrigin &&
JSON.stringify(snapshot.environment) === JSON.stringify(expected),
"Scenario never applied emulation",
);
await delay(100);
} finally {
controller.abort();
await settlement;
}
const cancelled = await pending;
assert.ok(
+73 -21
View File
@@ -8,10 +8,12 @@ import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import { randomUUID } from "node:crypto";
import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -152,14 +154,23 @@ export type ApktoolConfigurationFailureCode =
| "version_unresolved"
| "unsupported_version";
/** Unresolved cleanup of a launched probe; its owner was retained separately. */
export interface ApktoolProbeCleanup {
readonly reason: string;
readonly resources: readonly string[];
}
export class ApktoolConfigurationFailure extends Error {
readonly cleanup: ApktoolProbeCleanup | undefined;
constructor(
readonly code: ApktoolConfigurationFailureCode,
message: string,
readonly diagnostics: Readonly<Record<string, JsonValue>> = {},
options?: ErrorOptions,
options?: ErrorOptions & { readonly cleanup?: ApktoolProbeCleanup },
) {
super(message, options);
this.cleanup = options?.cleanup;
}
}
@@ -228,6 +239,7 @@ export const inspectApktoolClient = async (
signal?: AbortSignal,
environment: Readonly<Record<string, string | undefined>> = process.env,
platform: NodeJS.Platform = process.platform,
onCleanupOwner?: (owner: ProviderProcessSupervisor) => void,
): Promise<ApktoolClientIdentity> => {
const hostEnvironment = snapshotEnvironment(environment, platform);
let invocation: ApktoolInvocation;
@@ -254,23 +266,57 @@ export const inspectApktoolClient = async (
let stdout: string;
let stderr: string;
try {
({ stdout, stderr } = await execFileOutput(
invocation.command,
invocation.arguments,
// The owned process group stops a shell launcher's children with it; a
// child holding inherited output pipes cannot outlive the deadline.
const output = await runOwnedCommand(
{
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
timeout: VERSION_TIMEOUT_MS,
stopSignal: "SIGTERM",
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
env: hostEnvironment,
...(signal === undefined ? {} : { signal }),
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment,
},
));
{
timeoutMs: VERSION_TIMEOUT_MS,
diagnosticBytes: VERSION_MAX_OUTPUT_BYTES,
},
signal === undefined ? {} : { signal },
);
stdout = output.stdout.text;
stderr = output.stderr.text;
} catch (cause) {
const failure = execFileOutputFailure(cause);
if (signal?.aborted || failure?.code === "ABORT_ERR")
throw new AnalysisCancelledError("inspect_apktool_client", { cause });
if (failure?.code === "ENOENT")
if (
cause instanceof OwnedCommandFailure &&
cause.cleanupOwner !== undefined
)
onCleanupOwner?.(cause.cleanupOwner);
const cleanup =
cause instanceof OwnedCommandFailure && cause.cleanupFailure !== null
? { reason: cause.cleanupFailure, resources: cause.resources }
: undefined;
const cleanupDetails = cleanup === undefined ? {} : { cleanup };
if (
signal?.aborted === true ||
(cause instanceof OwnedCommandFailure && cause.reason === "cancelled")
)
throw new AnalysisCancelledError("inspect_apktool_client", {
cause,
...cleanupDetails,
});
if (cause instanceof OwnedCommandFailure && cause.reason === "timeout")
throw new AnalysisTimeoutError(
"inspect_apktool_client",
VERSION_TIMEOUT_MS,
{ cause, ...cleanupDetails },
);
if (
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "EACCES")
)
throw new ApktoolConfigurationFailure(
"command_missing",
`The selected apktool launcher was not found: ${selection.command}`,
@@ -281,19 +327,25 @@ export const inspectApktoolClient = async (
},
{ cause },
);
const snapshot =
cause instanceof OwnedCommandFailure ? cause.snapshot : null;
throw new ApktoolConfigurationFailure(
"version_unresolved",
"The selected apktool launcher did not report a usable version",
{
selected_command: selection.command,
...(failure === undefined
...(snapshot === null
? {}
: {
exit_code: failure.code,
stderr_excerpt: failure.stderr.slice(0, 512),
exit_code: snapshot.exitCode ?? null,
stderr_excerpt: snapshot.stderr.text.slice(0, 512),
}),
...(cause instanceof OwnedCommandFailure &&
cause.reason === "output-limit"
? { max_output_bytes: VERSION_MAX_OUTPUT_BYTES }
: {}),
},
{ cause },
{ cause, ...cleanupDetails },
);
}
const identity = parseApktoolVersionOutput(stdout);
+65 -4
View File
@@ -56,6 +56,7 @@ import { WindowsBatchCommandError } from "../process/WindowsBatchCommand.js";
import {
APKTOOL_COMMAND_ENV,
ApktoolConfigurationFailure,
type ApktoolProbeCleanup,
apktoolConfigurationAvailability,
apktoolInvocation,
inspectApktoolClient,
@@ -148,6 +149,19 @@ interface DecodeLimits {
readonly maxOutputBytes: number;
}
/** A probe whose launched processes could not be confirmed stopped. */
const probeCleanupFailure = (
operation: string,
failure: ApktoolConfigurationFailure,
cleanup: ApktoolProbeCleanup,
): ProviderCleanupError =>
new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
cleanup.resources,
{ reason: cleanup.reason, probe_failure: failure.message },
{ operation, cause: failure },
);
const runApktool = async (
operation: string,
selection: ApktoolCommandSelection,
@@ -180,6 +194,9 @@ const runApktool = async (
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment: environment,
},
@@ -328,6 +345,10 @@ export class ApktoolProvider {
private readonly workspaces = new Set<string>();
private readonly removeWorkspace: (path: string) => Promise<void>;
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
/** A version probe whose cleanup is unresolved is retried on close. */
readonly #retainProbeOwner = (owner: ProviderProcessSupervisor): void => {
this.#pendingCommandOwners.set(owner, "apktool version probe");
};
readonly #shutdown = new AbortController();
readonly #active = new Set<Promise<void>>();
#closing = false;
@@ -378,7 +399,13 @@ export class ApktoolProvider {
process.platform,
signal,
);
await inspectApktoolClient(selection, signal, environment);
await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
throwIfCancelled("inspect_apktool_client", signal);
return {
status: "available",
@@ -390,8 +417,15 @@ export class ApktoolProvider {
},
};
} catch (cause) {
if (cause instanceof ApktoolConfigurationFailure)
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
throw probeCleanupFailure(
"inspect_apktool_client",
cause,
cause.cleanup,
);
return apktoolConfigurationAvailability(cause);
}
throw cause;
}
}
@@ -437,7 +471,13 @@ export class ApktoolProvider {
process.platform,
signal,
);
identity = await inspectApktoolClient(selection, signal, environment);
identity = await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
} catch (cause) {
if (
cause instanceof AnalysisCancelledError ||
@@ -445,6 +485,10 @@ export class ApktoolProvider {
)
return err(cause);
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
return err(
probeCleanupFailure(request.operation, cause, cause.cleanup),
);
return err(
providerConfigurationRefusal(
APKTOOL_PROVIDER_IDENTITY.id,
@@ -497,12 +541,14 @@ export class ApktoolProvider {
async #drainAndClean(): Promise<void> {
await Promise.all(this.#active);
const failures: unknown[] = [];
const retained: string[] = [];
for (const [owner, root] of this.#pendingCommandOwners) {
try {
const stopped = await owner.stop();
if (stopped.status === "incomplete") throw new Error(stopped.reason);
this.#pendingCommandOwners.delete(owner);
} catch (cause) {
retained.push(root);
failures.push(
new Error(
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
@@ -511,8 +557,23 @@ export class ApktoolProvider {
);
}
}
// Ownership stays retained for a later close; report it as typed cleanup.
if (failures.length > 0)
throw new AggregateError(failures, "Apktool process cleanup failed");
throw new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
retained,
{
reason: failures
.map((failure) =>
failure instanceof Error ? failure.message : String(failure),
)
.join("; "),
},
{
operation: "close",
cause: new AggregateError(failures, "Apktool process cleanup failed"),
},
);
for (const path of this.workspaces) {
try {
await this.removeWorkspace(path);
@@ -0,0 +1,107 @@
import { execFile } from "node:child_process";
import { chmod, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { inspectApktoolClient } from "./ApktoolConfiguration.js";
/**
* Only an exited process counts as stopped: it is gone, a zombie waiting to
* be reaped, or absent from `ps`. A failed inspection is not proof of exit.
*/
const alive = async (pid: number): Promise<boolean> => {
try {
process.kill(pid, 0);
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code === "ESRCH") return false;
return true;
}
return promisify(execFile)("ps", ["-o", "stat=", "-p", `${pid}`]).then(
({ stdout }) => {
const state = stdout.trim();
return state !== "" && !state.startsWith("Z");
},
// `ps` exits 1 when no process matches; any other failure is unknown.
(cause: unknown) => (cause as { code?: unknown }).code !== 1,
);
};
const waitFor = async (check: () => Promise<boolean>, ms: number) => {
const deadline = Date.now() + ms;
while (Date.now() < deadline) {
if (await check()) return true;
await delay(25);
}
return false;
};
const readPids = async (path: string): Promise<readonly number[]> =>
readFile(path, "utf8")
.then((text) => JSON.parse(text) as number[])
.catch(() => []);
// The interpreter is the test runtime itself, so its environment (and the
// ownership run token) is readable on both Linux and macOS.
it.skipIf(process.platform === "win32")(
"stops an interpreter launcher and its pipe-holding child when the probe is cancelled",
async () => {
const root = await createTestTempDirectory("rea-apktool-probe-");
const pids = join(root, "pids.json");
const launcher = join(root, "apktool");
await writeFile(
launcher,
[
`#!${process.execPath}`,
'const { spawn } = require("node:child_process");',
'const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "inherit" });',
`require("node:fs").writeFileSync(${JSON.stringify(pids)}, JSON.stringify([process.pid, child.pid]));`,
"setInterval(() => {}, 1000);",
].join("\n"),
);
await chmod(launcher, 0o755);
const controller = new AbortController();
const started = performance.now();
const probe = inspectApktoolClient(
{ command: launcher, commandSource: "environment" },
controller.signal,
process.env,
).catch((cause: unknown) => cause);
try {
expect(
await waitFor(async () => (await readPids(pids)).length === 2, 10_000),
).toBe(true);
const launched = await readPids(pids);
controller.abort();
// A regressed probe never settles; bound the wait so cleanup still runs.
const outcome = await Promise.race([
probe,
delay(15_000).then(() => "still pending"),
]);
expect(outcome).toBeInstanceOf(AnalysisCancelledError);
expect(performance.now() - started).toBeLessThan(15_000);
expect(
await waitFor(
async () =>
(await Promise.all(launched.map(alive))).every((live) => !live),
5_000,
),
).toBe(true);
} finally {
// A failed assertion must not leave the fixture's processes running.
controller.abort();
await Promise.race([probe, delay(10_000)]);
for (const pid of await readPids(pids))
if (await alive(pid))
try {
process.kill(pid, "SIGKILL");
} catch {
// Already exited between the check and the signal.
}
}
},
30_000,
);
+147
View File
@@ -0,0 +1,147 @@
import { afterEach, expect, it, vi } from "vitest";
import {
AnalysisCancelledError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
ApktoolConfigurationFailure,
inspectApktoolClient,
} from "./ApktoolConfiguration.js";
import { ApktoolProvider } from "./ApktoolProvider.js";
vi.mock("../process/OwnedCommand.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../process/OwnedCommand.js")>()),
runOwnedCommand: vi.fn(),
}));
afterEach(() => vi.mocked(runOwnedCommand).mockReset());
const selection = {
command: "/opt/apktool/apktool",
commandSource: "environment",
} as const;
const owner = {} as ProviderProcessSupervisor;
it("runs the version probe as an owned interpreter-script process group", async () => {
vi.mocked(runOwnedCommand).mockResolvedValueOnce({
stdout: { text: "2.9.3\n" },
stderr: { text: "" },
} as Awaited<ReturnType<typeof runOwnedCommand>>);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).resolves.toEqual({ version: "2.9.3" });
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[0]).toMatchObject({
command: "/opt/apktool/apktool",
arguments: ["--version"],
expectedCommand: null,
});
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[1]).toEqual({
timeoutMs: 20_000,
diagnosticBytes: 64 * 1024,
});
});
it("reports an expired version probe as a typed timeout", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("timeout", "Command deadline elapsed."),
);
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "provider_timeout",
category: "timeout",
details: { timeout_ms: 20_000 },
});
});
it("retains an unresolved probe cleanup owner and reports the cleanup", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure(
"timeout",
"Command deadline elapsed.",
null,
"cleanup incomplete",
undefined,
["pid:1"],
owner,
),
);
const retained: ProviderProcessSupervisor[] = [];
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
(cleanupOwner) => retained.push(cleanupOwner),
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "cleanup_incomplete",
});
expect(retained).toEqual([owner]);
});
it("reports a cancelled version probe as cancelled", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("cancelled", "Command cancelled."),
);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).rejects.toBeInstanceOf(AnalysisCancelledError);
});
it("keeps unresolved probe cleanup typed when the launcher overflows its output", async () => {
const overflow = () =>
new OwnedCommandFailure(
"output-limit",
"Command diagnostic output exceeded its complete-output budget.",
null,
"process group still running",
undefined,
["pid:7", "process-group:7"],
owner,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(ApktoolConfigurationFailure);
expect((failure as ApktoolConfigurationFailure).cleanup).toEqual({
reason: "process group still running",
resources: ["pid:7", "process-group:7"],
});
// Both provider entry points report the unresolved processes as cleanup.
const provider = new ApktoolProvider({
environment: { REA_APKTOOL_COMMAND: "/opt/apktool/apktool" },
});
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
await expect(provider.inspectAvailability()).rejects.toBeInstanceOf(
ProviderCleanupError,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const result = await provider.execute({
operation: "decode_android_resources",
input: {},
} as never);
expect(result.ok ? null : result.error).toBeInstanceOf(ProviderCleanupError);
expect(result.ok ? null : projectAnalysisError(result.error)).toMatchObject({
code: "cleanup_incomplete",
details: { resources: ["pid:7", "process-group:7"] },
});
});
+7 -1
View File
@@ -1,2 +1,8 @@
/** Deep-analysis providers whose readiness can be selected explicitly by doctor. */
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida", "jadx"] as const;
export const DOCTOR_PROVIDER_IDS = [
"hopper",
"ghidra",
"ida",
"jadx",
"kuna",
] as const;
+103 -12
View File
@@ -94,10 +94,95 @@ const failedEntry = (
};
};
/** A package directory's legacy `main` entry point, as its manifest declares it. */
type PackageMain =
| { readonly state: "declared"; readonly main: string }
| { readonly state: "none" }
| { readonly state: "unknown" };
/**
* Read each inventoried `package.json` once. Node selects a non-empty string
* `main` before a directory index; an unreadable or invalid manifest leaves
* that selection unknown.
*/
const readPackageMains = (
read: ReferenceSourceRead,
): ReadonlyMap<string, PackageMain> => {
const mains = new Map<string, PackageMain>();
for (const entry of read.entries) {
if (posix.basename(entry.path) !== "package.json") continue;
const directory = posix.dirname(entry.path);
if (entry.status !== "read" || entry.kind !== "file") {
mains.set(directory, { state: "unknown" });
continue;
}
try {
const manifest: unknown = JSON.parse(
new TextDecoder("utf-8", { fatal: true }).decode(entry.bytes),
);
const main =
typeof manifest === "object" && manifest !== null
? Reflect.get(manifest, "main")
: undefined;
mains.set(
directory,
typeof main === "string" && main !== ""
? { state: "declared", main }
: { state: "none" },
);
} catch {
mains.set(directory, { state: "unknown" });
}
}
return mains;
};
const RESOLUTION_SUFFIXES = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
/** Inventory facts an internal specifier resolves against. */
interface SpecifierContext {
readonly lookupPath: ReferenceSourcePathLookup;
readonly packageMains: ReadonlyMap<string, PackageMain>;
}
/**
* Apply a package `main` to a directory resolved through its index. Node loads
* the declared entry as a file, then as a directory index, and falls back to
* the package's own index only when neither exists.
*/
const selectPackageMain = (
index: string,
context: SpecifierContext,
): { to: string; resolution: "internal" | "unresolved" } => {
const directory = posix.dirname(index);
const manifest = context.packageMains.get(directory);
if (manifest === undefined || manifest.state === "none")
return { to: index, resolution: "internal" };
if (manifest.state === "unknown")
return { to: directory, resolution: "unresolved" };
// Node resolves `main` with path.resolve: an absolute main leaves the
// package, and a trailing separator does not make the entry a directory.
if (posix.isAbsolute(manifest.main))
return { to: posix.normalize(manifest.main), resolution: "unresolved" };
const target = posix
.normalize(posix.join(directory, manifest.main))
.replace(/(?<=.)\/+$/u, "");
// Node would load a declared entry outside the imported root.
if (logicalPathEscapesRoot(target))
return { to: target, resolution: "unresolved" };
const selected = context.lookupPath.resolveCandidates([
...RESOLUTION_SUFFIXES.map((suffix) => `${target}${suffix}`),
...RESOLUTION_SUFFIXES.filter((suffix) => suffix !== "").map((suffix) =>
posix.join(target, `index${suffix}`),
),
]);
return { to: selected ?? index, resolution: "internal" };
};
const resolveInternalSpecifier = (
fromPath: string,
specifier: string,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
): { to: string; resolution: "internal" | "unresolved" } => {
if (posix.isAbsolute(specifier))
return { to: specifier, resolution: "unresolved" };
@@ -109,10 +194,9 @@ const resolveInternalSpecifier = (
// Preserve directory intent before normalization removes terminal dot segments.
const directoryReference =
specifier.endsWith("/") || [".", ".."].includes(posix.basename(specifier));
const suffixes = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
// A directory resolves only to an index with an extension; neither Node nor
// TypeScript selects an extensionless `index` file.
const candidates = suffixes.flatMap((suffix) => [
const candidates = RESOLUTION_SUFFIXES.flatMap((suffix) => [
...(directoryReference ? [] : [`${normalized}${suffix}`]),
...(suffix === "" ? [] : [posix.join(normalized, `index${suffix}`)]),
]);
@@ -124,9 +208,13 @@ const resolveInternalSpecifier = (
({ path }) => path,
),
);
const match = lookupPath.resolveCandidates(candidates);
return match === undefined
? { to: normalized, resolution: "unresolved" }
const match = context.lookupPath.resolveCandidates(candidates);
if (match === undefined) return { to: normalized, resolution: "unresolved" };
const resolvedThroughIndex =
posix.basename(match).startsWith("index.") &&
posix.dirname(match).normalize("NFC") === normalized.normalize("NFC");
return resolvedThroughIndex
? selectPackageMain(match, context)
: { to: match, resolution: "internal" };
};
@@ -134,7 +222,7 @@ type ReadFile = Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>;
const appendFile = (
entry: ReadFile,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
output: ParsedReferenceSourceEntries,
): void => {
const language = detectReferenceSourceLanguage(entry.path);
@@ -161,7 +249,7 @@ const appendFile = (
const resolved = resolveInternalSpecifier(
relationship.from_path,
relationship.to,
lookupPath,
context,
);
output.relationships.push({
...relationship,
@@ -174,11 +262,11 @@ const appendFile = (
const appendReadEntry = (
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
output: ParsedReferenceSourceEntries,
): void => {
if (entry.kind === "file") {
appendFile(entry, lookupPath, output);
appendFile(entry, context, output);
return;
}
const classifications = classifyReferenceSourcePath(entry.path);
@@ -221,11 +309,14 @@ export const parseReferenceSourceEntries = (
parseFailures: [],
limitations: [...read.limitations],
};
const lookupPath = createReferenceSourcePathLookup(filePaths);
const context: SpecifierContext = {
lookupPath: createReferenceSourcePathLookup(filePaths),
packageMains: readPackageMains(read),
};
for (const entry of read.entries) {
if (signal?.aborted === true) break;
if (entry.status === "failed") output.entries.push(failedEntry(entry));
else appendReadEntry(entry, lookupPath, output);
else appendReadEntry(entry, context, output);
}
return output;
};
+28 -7
View File
@@ -216,12 +216,12 @@ export class BinarySession
options: BinarySessionOpenOptions = {},
): Promise<Result<BinaryTarget, AnalysisError>> {
return this.#open(
() =>
(admitted) =>
resolveSessionOpen({
router: this.#providerRouter,
current: this.#active,
path,
options,
options: admitted,
stagedSnapshotMatches: (target, profile) =>
this.matchesSnapshot(target, profile),
}),
@@ -250,12 +250,33 @@ export class BinarySession
}
#open(
resolve: () => Promise<Result<ResolvedSessionOpen, AnalysisError>>,
options: Pick<BinarySessionOpenOptions, "signal" | "snapshot">,
resolve: (
admitted: BinarySessionOpenOptions,
) => Promise<Result<ResolvedSessionOpen, AnalysisError>>,
options: BinarySessionOpenOptions,
): Promise<Result<BinaryTarget, AnalysisError>> {
return this.#serialize((admittedThrough) =>
this.#openWithinTransition(admittedThrough, resolve, options),
);
return this.#serialize(async (admittedThrough) => {
const admitted = await this.#withLoadedSnapshot(options);
if (!admitted.ok) return admitted;
return this.#openWithinTransition(
admittedThrough,
() => resolve(admitted.value),
admitted.value,
);
});
}
/** Read the staged snapshot only after earlier lifecycle transitions have settled. */
async #withLoadedSnapshot(
options: BinarySessionOpenOptions,
): Promise<Result<BinarySessionOpenOptions, AnalysisError>> {
if (options.loadSnapshot === undefined) return ok(options);
if (isAborted(options.signal))
return err(new AnalysisCancelledError("open_binary"));
const loaded = await options.loadSnapshot();
if (!loaded.ok) return loaded;
const { loadSnapshot: _loaded, ...rest } = options;
return ok({ ...rest, snapshot: loaded.value });
}
async #openWithinTransition(
@@ -19,6 +19,11 @@ import { err, type Result } from "../../domain/result.js";
import type { SessionProviderRoute } from "./SessionProviderRouter.js";
import { SessionProviderRouter } from "./SessionProviderRouter.js";
/** Read a staged snapshot once the open holds its place in the session order. */
export type AnalysisSnapshotLoader = () => Promise<
Result<AnalysisSnapshot, AnalysisError>
>;
export interface BinarySessionOpenOptions {
readonly signal?: AbortSignal;
readonly targetKind?: BinaryTarget["kind"];
@@ -26,6 +31,11 @@ export interface BinarySessionOpenOptions {
readonly architecture?: BinaryArchitecture;
readonly entryPoint?: string;
readonly snapshot?: AnalysisSnapshot;
/**
* Load the snapshot inside the open's turn instead of before it, so calls
* dispatched after the open cannot overtake its file read.
*/
readonly loadSnapshot?: AnalysisSnapshotLoader;
readonly providerId?: AnalysisProviderSelector;
}
@@ -11,6 +11,7 @@ import type { AnalysisProviderSelector } from "../../contracts/providerSelection
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { AnalysisSnapshotLoader } from "./BinarySessionOpen.js";
import type { WorkflowSnapshotRecordInput } from "./BinarySessionRecords.js";
import type {
@@ -71,6 +72,7 @@ export interface BinarySessionPort
readonly architecture?: BinaryArchitecture;
readonly entryPoint?: string;
readonly snapshot?: AnalysisSnapshot;
readonly loadSnapshot?: AnalysisSnapshotLoader;
readonly providerId?: AnalysisProviderSelector;
},
): Promise<Result<BinaryTarget, AnalysisError>>;
+119 -69
View File
@@ -99,51 +99,62 @@ export const captureResources = (
return { items: [] };
}
const items: CapturedResource[] = [];
for (const tree of walkFrameTrees(root, (omittedCount) =>
completeness?.exclude("resources", "invalid_protocol_value", omittedCount),
for (const resource of frameTreeResources(root, (count) =>
completeness?.exclude("resources", "invalid_protocol_value", count),
)) {
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
"resources",
exclusionReasonForUrl(cdpStringValue(resource.url)),
);
continue;
}
const type = cdpStringValue(resource.type);
const mimeType = cdpStringValue(resource.mimeType);
if (type === undefined || type.length === 0 || mimeType === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const rawContentSize = numberValue(resource.contentSize);
const contentSize =
rawContentSize !== undefined && rawContentSize >= 0
? rawContentSize
: null;
if (resource.contentSize !== undefined && contentSize === null)
completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
type,
mime_type: mimeType,
content_size: contentSize,
});
}
return { items };
};
/**
* Yield resource records from every frame of a resource tree. A missing or
* non-array `resources` list, or a non-record entry, is reported as invalid
* rather than read as an empty frame.
*/
export const frameTreeResources = function* (
root: UnknownRecord,
onInvalid: (count: number | null) => void,
): Generator<UnknownRecord> {
for (const tree of walkFrameTrees(root, onInvalid)) {
if (!Array.isArray(tree.resources)) {
completeness?.exclude("resources", "invalid_protocol_value", null);
onInvalid(null);
continue;
}
for (const entry of tree.resources) {
const resource = recordValue(entry);
if (resource === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
"resources",
exclusionReasonForUrl(cdpStringValue(resource.url)),
);
continue;
}
const type = cdpStringValue(resource.type);
const mimeType = cdpStringValue(resource.mimeType);
if (type === undefined || type.length === 0 || mimeType === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const rawContentSize = numberValue(resource.contentSize);
const contentSize =
rawContentSize !== undefined && rawContentSize >= 0
? rawContentSize
: null;
if (resource.contentSize !== undefined && contentSize === null)
completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
type,
mime_type: mimeType,
content_size: contentSize,
});
if (resource === undefined) onInvalid(1);
else yield resource;
}
}
return { items };
};
export const walkFrameTrees = function* (
@@ -217,7 +228,9 @@ export function* captureDom(
let unresolvedSvgNodes = 0;
if (!Array.isArray(root.strings))
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
for (const document of snapshotDocuments(root.documents, completeness)) {
for (const document of snapshotDocuments(root.documents, (count) =>
excludeDocumentEvidence(completeness, "invalid_protocol_value", count),
)) {
const documentUrl = cdpSnapshotString(strings, document.documentURL);
if (documentUrl === undefined) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
@@ -236,40 +249,25 @@ export function* captureDom(
continue;
}
const baseUrl = baseUrlValue || documentUrl;
const documentNodes = recordValue(document.nodes);
const documentNodes = snapshotDocumentNodes(document.nodes);
if (documentNodes === undefined) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const rawNodeTypes = numberArray(documentNodes.nodeType);
const rawNodeNames = numberArray(documentNodes.nodeName);
const rawNodeValues = numberArray(documentNodes.nodeValue);
const rawParents = numberArray(documentNodes.parentIndex);
const rawAttributes = Array.isArray(documentNodes.attributes)
? documentNodes.attributes
: undefined;
if (
rawNodeTypes === undefined ||
rawNodeNames === undefined ||
rawNodeValues === undefined ||
rawParents === undefined ||
rawAttributes === undefined ||
rawNodeNames.length !== rawNodeTypes.length ||
rawNodeValues.length !== rawNodeTypes.length ||
rawParents.length !== rawNodeTypes.length
) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (rawAttributes.length > rawNodeTypes.length)
const {
nodeTypes: rawNodeTypes,
nodeNames: rawNodeNames,
nodeValues: rawNodeValues,
parents: rawParents,
attributes: rawAttributes,
backendNodeIds,
} = documentNodes;
if (documentNodes.excessAttributes > 0)
excludeDocumentEvidence(
completeness,
"invalid_protocol_value",
rawAttributes.length - rawNodeTypes.length,
documentNodes.excessAttributes,
);
const backendNodeIds = Array.isArray(documentNodes.backendNodeId)
? documentNodes.backendNodeId
: [];
const baseIndex = total;
total += rawNodeTypes.length;
const ancestry = createDocumentSvgAncestry({
@@ -389,22 +387,74 @@ const excludeDocumentEvidence = (
completeness?.exclude("metadata", reason, count);
};
const snapshotDocuments = (
/**
* Read DOMSnapshot documents. A snapshot always describes at least one
* document, so a missing or empty list is invalid rather than an empty DOM.
*/
export const snapshotDocuments = (
value: unknown,
completeness: CdpCaptureCompleteness | undefined,
onInvalid: (count: number | null) => void,
): readonly UnknownRecord[] => {
if (!Array.isArray(value) || value.length === 0) {
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
onInvalid(null);
return [];
}
return value.flatMap((item) => {
const document = recordValue(item);
if (document === undefined)
excludeDocumentEvidence(completeness, "invalid_protocol_value");
if (document === undefined) onInvalid(1);
return document === undefined ? [] : [document];
});
};
/** One snapshot document's parallel node arrays, validated for every consumer. */
export interface SnapshotDocumentNodes {
readonly nodeTypes: readonly (number | undefined)[];
readonly nodeNames: readonly (number | undefined)[];
readonly nodeValues: readonly (number | undefined)[];
readonly parents: readonly (number | undefined)[];
readonly attributes: readonly unknown[];
readonly backendNodeIds: readonly unknown[];
/** Attribute rows beyond the node count describe no node. */
readonly excessAttributes: number;
}
/** Missing or misaligned node arrays make the document's nodes unknown. */
export const snapshotDocumentNodes = (
value: unknown,
): SnapshotDocumentNodes | undefined => {
const nodes = recordValue(value);
if (nodes === undefined) return undefined;
const nodeTypes = numberArray(nodes.nodeType);
const nodeNames = numberArray(nodes.nodeName);
const nodeValues = numberArray(nodes.nodeValue);
const parents = numberArray(nodes.parentIndex);
const attributes = Array.isArray(nodes.attributes)
? nodes.attributes
: undefined;
if (
nodeTypes === undefined ||
nodeNames === undefined ||
nodeValues === undefined ||
parents === undefined ||
attributes === undefined ||
nodeNames.length !== nodeTypes.length ||
nodeValues.length !== nodeTypes.length ||
parents.length !== nodeTypes.length
)
return undefined;
return {
nodeTypes,
nodeNames,
nodeValues,
parents,
attributes,
backendNodeIds: Array.isArray(nodes.backendNodeId)
? nodes.backendNodeId
: [],
excessAttributes: Math.max(0, attributes.length - nodeTypes.length),
};
};
function* captureDomMetadata(
pending: readonly PendingDomMetadata[],
completeness: CdpCaptureCompleteness | undefined,
+47 -25
View File
@@ -23,7 +23,7 @@ interface PendingCommand {
readonly sessionId?: string;
readonly resolve: (value: unknown) => void;
readonly reject: (error: AnalysisError) => void;
readonly removeAbort: () => void;
readonly dispose: () => void;
}
/** Correlated JSON-RPC transport for one CDP target operation. */
@@ -79,6 +79,8 @@ export class CdpConnection {
`CDP maxPayloadBytes must be a positive safe integer no greater than ${String(MAX_WEBSOCKET_PAYLOAD_BYTES)}.`,
);
const socket = new WebSocket(url, {
// waitForOpen owns the deadline, cancellation, cleanup, and typed errors;
// keep ws's timer disabled so handshake failures have one owner.
handshakeTimeout: 0,
maxPayload: maxPayloadBytes,
perMessageDeflate: false,
@@ -115,7 +117,7 @@ export class CdpConnection {
return () => this.#protocolFailureListeners.delete(listener);
}
/** Execute one command, optionally within a flat target session. */
/** Bound each response separately from the caller-selected observation window. */
async send(
method: string,
params: Readonly<Record<string, unknown>> = {},
@@ -130,41 +132,51 @@ export class CdpConnection {
throw new AnalysisCancelledError(this.operation);
const id = this.#nextId;
this.#nextId += 1;
const message = JSON.stringify({
id,
method,
params,
...(sessionId === undefined ? {} : { sessionId }),
});
return await new Promise((resolve, reject) => {
const onAbort = (): void => {
const pending = this.#pending.get(id);
if (pending === undefined) return;
pending.removeAbort();
this.#pending.delete(id);
this.#complete(id, pending);
reject(new AnalysisCancelledError(this.operation));
};
const timer = setTimeout(() => {
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "timeout", {
detail: `CDP command ${method} exceeded its ${String(WEB_RUNTIME_LIMITS.commandTimeoutMs)} ms response deadline.`,
}),
);
}, WEB_RUNTIME_LIMITS.commandTimeoutMs);
signal?.addEventListener("abort", onAbort, { once: true });
this.#pending.set(id, {
method,
...(sessionId === undefined ? {} : { sessionId }),
resolve,
reject,
removeAbort: () => signal?.removeEventListener("abort", onAbort),
});
this.socket.send(
JSON.stringify({
id,
method,
params,
...(sessionId === undefined ? {} : { sessionId }),
}),
(error) => {
if (error === undefined || error === null) return;
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "disconnected", {
cause: error,
}),
);
dispose: () => {
clearTimeout(timer);
signal?.removeEventListener("abort", onAbort);
},
);
});
this.socket.send(message, (error) => {
if (error === undefined || error === null) return;
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "disconnected", {
cause: error,
}),
);
});
});
}
@@ -282,7 +294,7 @@ export class CdpConnection {
}
#complete(id: number, pending: PendingCommand): void {
pending.removeAbort();
pending.dispose();
this.#pending.delete(id);
}
@@ -370,11 +382,21 @@ const waitForOpen = async (
onFailure(new AnalysisCancelledError(operation));
};
const cleanup = (): void => {
clearTimeout(timer);
socket.off("open", onOpen);
socket.off("error", onFailure);
socket.off("close", onClose);
signal?.removeEventListener("abort", onAbort);
};
const timer = setTimeout(() => {
terminateSocket(socket);
cleanup();
reject(
new BrowserObservationError(operation, "timeout", {
detail: `CDP WebSocket handshake exceeded its ${String(WEB_RUNTIME_LIMITS.commandTimeoutMs)} ms deadline.`,
}),
);
}, WEB_RUNTIME_LIMITS.commandTimeoutMs);
socket.once("open", onOpen);
socket.once("error", onFailure);
socket.once("close", onClose);
+45 -38
View File
@@ -12,11 +12,15 @@ import {
delayWithCancellation,
numberValue,
recordValue,
recordsValue,
requiredRecord,
cdpStringValue,
} from "./CdpCaptureValues.js";
import { walkFrameTrees } from "./CdpCaptureDocuments.js";
import {
frameTreeResources,
snapshotDocumentNodes,
snapshotDocuments,
walkFrameTrees,
} from "./CdpCaptureDocuments.js";
import {
ingestElectronScriptEvent,
type ElectronScriptDraft,
@@ -276,38 +280,34 @@ const captureResources = async (
const root = recordValue(requiredRecord(result).frameTree);
if (root === undefined)
throw new BrowserObservationError("inspect_web_page", "protocol_error");
const pending = [root];
const resources: ElectronPageInspection["resources"] = [];
const seen = new Set<string>();
while (pending.length > 0) {
const tree = pending.shift();
if (tree === undefined) break;
for (const resource of recordsValue(tree.resources)) {
const path = await authorizedElectronFile(
cdpStringValue(resource.url) ?? "",
);
if (path === undefined) {
completeness.exclude("resources", "out_of_target_scope");
continue;
}
const item = {
file_path: path,
type: cdpStringValue(resource.type) ?? "Other",
mime_type: cdpStringValue(resource.mimeType) ?? "",
content_size:
numberValue(resource.contentSize) === undefined
? null
: Math.max(0, numberValue(resource.contentSize) ?? 0),
};
const resourceKey = `electron_resource_${digestCanonicalValue(item, "CDP capture")}`;
if (seen.has(resourceKey)) continue;
seen.add(resourceKey);
resources.push({
resource_key: resourceKey,
...item,
});
for (const resource of frameTreeResources(root, (count) =>
completeness.exclude("resources", "invalid_protocol_value", count),
)) {
const path = await authorizedElectronFile(
cdpStringValue(resource.url) ?? "",
);
if (path === undefined) {
completeness.exclude("resources", "out_of_target_scope");
continue;
}
pending.push(...recordsValue(tree.childFrames));
const item = {
file_path: path,
type: cdpStringValue(resource.type) ?? "Other",
mime_type: cdpStringValue(resource.mimeType) ?? "",
content_size:
numberValue(resource.contentSize) === undefined
? null
: Math.max(0, numberValue(resource.contentSize) ?? 0),
};
const resourceKey = `electron_resource_${digestCanonicalValue(item, "CDP capture")}`;
if (seen.has(resourceKey)) continue;
seen.add(resourceKey);
resources.push({
resource_key: resourceKey,
...item,
});
}
return resources.sort((left, right) =>
compareUnicodeCodePoints(left.resource_key, right.resource_key),
@@ -326,23 +326,30 @@ const captureDom = async (
context.signal,
),
);
const invalid = (count: number | null) =>
completeness.exclude("dom", "invalid_protocol_value", count);
if (!Array.isArray(result.strings)) invalid(null);
const strings = Array.isArray(result.strings)
? result.strings.map((value) => (typeof value === "string" ? value : ""))
: [];
const nodes: ElectronPageInspection["dom"]["nodes"] = [];
let total = 0;
for (const document of recordsValue(result.documents)) {
for (const document of snapshotDocuments(result.documents, invalid)) {
const documentUrl = stringAt(strings, document.documentURL);
if ((await authorizedElectronFile(documentUrl)) === undefined) {
completeness.exclude("dom", "out_of_target_scope");
continue;
}
const nodeData = recordValue(document.nodes);
const nodeTypes = arrayValue(nodeData?.nodeType);
const nodeNames = arrayValue(nodeData?.nodeName);
const nodeValues = arrayValue(nodeData?.nodeValue);
const parents = arrayValue(nodeData?.parentIndex);
const attributes = arrayValue(nodeData?.attributes);
// The shared snapshot owner validates the parallel node arrays.
const documentNodes = snapshotDocumentNodes(document.nodes);
if (documentNodes === undefined) {
invalid(1);
continue;
}
if (documentNodes.excessAttributes > 0)
invalid(documentNodes.excessAttributes);
const { nodeTypes, nodeNames, nodeValues, parents, attributes } =
documentNodes;
const baseIndex = nodes.length;
total += nodeTypes.length;
for (let index = 0; index < nodeTypes.length; index += 1) {
+13 -6
View File
@@ -4,11 +4,7 @@ import type {
} from "../domain/javascript/electronObservation.js";
import type { CdpConnection } from "./CdpConnection.js";
import { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
import {
recordValue,
recordsValue,
cdpStringValue,
} from "./CdpCaptureValues.js";
import { recordValue, cdpStringValue } from "./CdpCaptureValues.js";
import type { CdpEndpointTarget } from "./CdpEndpoint.js";
import { optionalCdpCommand } from "./CdpOptionalCommand.js";
import { authorizedElectronFile } from "./ElectronFileScope.js";
@@ -39,7 +35,18 @@ export const captureElectronWorkers = async (input: {
return [];
}
const workers: ElectronPageInspection["workers"] = [];
for (const target of recordsValue(recordValue(result)?.targetInfos)) {
// A missing target list or a non-record entry is unreadable, not empty.
const targetInfos = recordValue(result)?.targetInfos;
if (!Array.isArray(targetInfos)) {
input.completeness.exclude("workers", "invalid_protocol_value", null);
return [];
}
for (const entry of targetInfos) {
const target = recordValue(entry);
if (target === undefined) {
input.completeness.exclude("workers", "invalid_protocol_value");
continue;
}
const type = cdpStringValue(target.type) ?? "";
if (!type.includes("worker")) continue;
const openerTargetId = boundedTargetField(target.openerId);
@@ -0,0 +1,167 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { ELECTRON_FUSE_STATE_BYTES } from "../domain/electron/electronFuses.js";
import {
elfFuseImage,
OBSIDIAN_FUSE_BYTES,
peFuseImage,
thinFuseImage,
thinMacho,
universalFuseImage,
} from "../../tests/fixtures/electron/fuseImage.js";
import { AnalysisUnsupportedTargetError } from "../domain/analysisErrorCore.js";
import { assertNodeCliInspectArgumentsSupported } from "./PlaywrightElectronActiveFuses.js";
describe("assertNodeCliInspectArgumentsSupported", () => {
it("rejects thin Mach-O binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-app");
const image = thinFuseImage(OBSIDIAN_FUSE_BYTES); // index 3 is disabled
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
try {
await assertNodeCliInspectArgumentsSupported(binaryPath);
} catch (error) {
expect(error).toBeInstanceOf(AnalysisUnsupportedTargetError);
const targetError = error as AnalysisUnsupportedTargetError;
expect(targetError.operation).toBe("capture_electron_scenario");
expect(targetError.path).toBe(binaryPath);
expect(targetError.message).toContain("EnableNodeCliInspectArguments");
expect(targetError.remediationAction).toContain("inspect_electron_page");
}
});
it("passes when EnableNodeCliInspectArguments fuse is enabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "enabled-app");
const enabledFuses = [...OBSIDIAN_FUSE_BYTES];
enabledFuses[3] = ELECTRON_FUSE_STATE_BYTES.enabled;
const image = thinFuseImage(enabledFuses);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
});
it("passes when binary contains no fuse sentinel (fuse state unknown)", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "stock-node");
const image = thinMacho(0x0100000c, Buffer.from("no-sentinel-bytes-here"));
await writeFile(binaryPath, image);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
});
it("passes gracefully when fuse reader throws a format or reading failure", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const nonMachoPath = join(root, "script.sh");
await writeFile(nonMachoPath, "#!/bin/sh\necho hello");
await expect(
assertNodeCliInspectArgumentsSupported(nonMachoPath),
).resolves.toBeUndefined();
});
it("rejects Windows PE binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-pe.exe");
const image = peFuseImage(OBSIDIAN_FUSE_BYTES);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
});
it("rejects Linux ELF binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-elf");
const image = elfFuseImage(OBSIDIAN_FUSE_BYTES);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
});
it("rethrows when signal is aborted during reading", async () => {
const controller = new AbortController();
const readFuseImage = vi.fn(async () => {
controller.abort();
throw new Error("aborted during read");
});
await expect(
assertNodeCliInspectArgumentsSupported("/nonexistent/binary", {
signal: controller.signal,
readFuseImage,
}),
).rejects.toThrow();
expect(readFuseImage).toHaveBeenCalledOnce();
});
it("rejects macOS app bundle when framework binary has disabled inspect fuse", async () => {
const root = await createTestTempDirectory("rea-fuse-mac-bundle-");
const appBundle = join(root, "Obsidian.app");
const launcherBinary = join(appBundle, "Contents", "MacOS", "Obsidian");
const frameworkDir = join(
appBundle,
"Contents",
"Frameworks",
"Electron Framework.framework",
"Versions",
"A",
);
await mkdir(join(appBundle, "Contents", "MacOS"), { recursive: true });
await mkdir(frameworkDir, { recursive: true });
// Launcher binary has no sentinel:
await writeFile(launcherBinary, Buffer.from("dummy launcher binary"));
// Framework binary has the disabled fuse wire:
const frameworkBinary = join(frameworkDir, "Electron Framework");
await writeFile(frameworkBinary, thinFuseImage(OBSIDIAN_FUSE_BYTES).bytes);
await expect(
assertNodeCliInspectArgumentsSupported(launcherBinary),
).rejects.toThrow(AnalysisUnsupportedTargetError);
try {
await assertNodeCliInspectArgumentsSupported(launcherBinary);
} catch (error) {
expect(error).toBeInstanceOf(AnalysisUnsupportedTargetError);
const targetError = error as AnalysisUnsupportedTargetError;
expect(targetError.operation).toBe("capture_electron_scenario");
expect(targetError.path).toBe(launcherBinary);
expect(targetError.message).toContain("EnableNodeCliInspectArguments");
}
});
it("evaluates matching architecture slice in universal Mach-O binary", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "universal-app");
// universalFuseImage has left (x86_64) disabled, right (arm64) enabled
const image = universalFuseImage();
await writeFile(binaryPath, image.bytes);
if (process.arch === "arm64") {
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
} else {
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
}
});
});
@@ -0,0 +1,96 @@
import { basename, dirname } from "node:path";
import { AnalysisUnsupportedTargetError } from "../domain/analysisErrorCore.js";
import { resolveElectronFuseTarget } from "../electron/ElectronFusePath.js";
import { readElectronFuseImage } from "../electron/ElectronFuseReader.js";
const OPERATION = "capture_electron_scenario" as const;
const NODE_CLI_INSPECT_ARGUMENTS_FUSE_INDEX = 3;
export interface InspectArgumentsPreflightOptions {
readonly signal?: AbortSignal;
readonly readFuseImage?: typeof readElectronFuseImage;
readonly resolveFuseTarget?: typeof resolveElectronFuseTarget;
}
const appBundleDirectoryOf = (executablePath: string): string | null => {
let current = dirname(executablePath);
while (true) {
if (basename(current).endsWith(".app")) return current;
const parent = dirname(current);
if (parent === current) return null;
current = parent;
}
};
/**
* Pre-flight verification for active Electron scenario capture.
*
* Playwright attaches to Electron processes using Node CLI inspect arguments
* (--inspect=0 or --remote-debugging-port=0). When an Electron executable disables
* the EnableNodeCliInspectArguments fuse, the runtime silently ignores inspect
* flags and Playwright hangs until the 60-second startup deadline expires.
*
* When the fuse wire is present and index 3 is explicitly disabled or removed,
* this check fast-fails with an AnalysisUnsupportedTargetError before any child
* process is launched. When the fuse state is unknown (missing sentinel, wire
* version != 1, or unsupported container format), launch proceeds normally.
*/
export const assertNodeCliInspectArgumentsSupported = async (
executablePath: string,
options?: InspectArgumentsPreflightOptions,
): Promise<void> => {
options?.signal?.throwIfAborted();
const reader = options?.readFuseImage ?? readElectronFuseImage;
const resolver = options?.resolveFuseTarget ?? resolveElectronFuseTarget;
try {
let targetPath = executablePath;
const bundleDir = appBundleDirectoryOf(executablePath);
if (bundleDir !== null) {
try {
const resolved = await resolver(bundleDir, options?.signal);
targetPath = resolved.file.path;
} catch (cause: unknown) {
if (options?.signal?.aborted) throw cause;
targetPath = executablePath;
}
}
const image = await reader(
targetPath,
options?.signal !== undefined ? { signal: options.signal } : undefined,
);
if (image.slices.length === 0) return;
const hostArchitecture = process.arch === "x64" ? "x86_64" : process.arch;
const targetSlice =
image.slices.find((slice) => slice.architecture === hostArchitecture) ??
image.slices[0];
if (targetSlice === undefined) return;
for (const sentinel of targetSlice.sentinels) {
if (sentinel.wire_version !== 1) continue;
const fuse = sentinel.fuses.find(
(entry) =>
entry.index === NODE_CLI_INSPECT_ARGUMENTS_FUSE_INDEX ||
entry.name === "EnableNodeCliInspectArguments",
);
if (fuse?.state === "disabled" || fuse?.state === "removed") {
throw new AnalysisUnsupportedTargetError(
OPERATION,
executablePath,
"Target binary disables the EnableNodeCliInspectArguments fuse; Node CLI inspector arguments are ignored and active scenario capture cannot attach.",
{
remediationAction:
"Use passive CDP observation tools (inspect_electron_page), or launch active capture against an Electron development build or a stock Electron runtime with the extracted app.",
},
);
}
}
} catch (cause: unknown) {
if (cause instanceof AnalysisUnsupportedTargetError) throw cause;
options?.signal?.throwIfAborted();
// Missing sentinels, format failures, or unreadable fuses leave inspection capability unknown; proceed with launch.
}
};
@@ -1,5 +1,12 @@
import { afterEach, expect, it, vi } from "vitest";
import { dirname } from "node:path";
import { writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import {
OBSIDIAN_FUSE_BYTES,
thinFuseImage,
} from "../../tests/fixtures/electron/fuseImage.js";
import * as processOwnership from "../process/ProcessOwnership.js";
import * as electronActions from "./PlaywrightElectronActiveActions.js";
@@ -253,3 +260,34 @@ it("retains the initial snapshot when the final state read fails", async () => {
},
});
});
it("rejects active capture before launch when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-active-provider-fuse-");
const disabledBinary = join(root, "disabled-electron");
await writeFile(disabledBinary, thinFuseImage(OBSIDIAN_FUSE_BYTES).bytes);
const launchSpy = vi.fn(async () => {
throw new Error("launch should not be called when fuse is disabled");
});
const provider = new PlaywrightElectronActiveProvider({}, launchSpy);
const result = await provider.capture({
...electronActiveObservationInputSchema.parse({
executable_path: disabledBinary,
application_path: disabledBinary,
}),
application_root: root,
});
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected capture to fail");
expect(launchSpy).not.toHaveBeenCalled();
expect(result.error).toMatchObject({
_tag: "AnalysisUnsupportedTargetError",
operation: "capture_electron_scenario",
path: disabledBinary,
message: expect.stringContaining("EnableNodeCliInspectArguments"),
remediationAction: expect.stringContaining("inspect_electron_page"),
});
});
@@ -37,6 +37,8 @@ import {
runWithExecutionLimits,
} from "./PlaywrightElectronActiveActions.js";
import { projectElectronActiveCapture } from "./PlaywrightElectronActiveProjection.js";
import { readElectronFuseImage } from "../electron/ElectronFuseReader.js";
import { assertNodeCliInspectArgumentsSupported } from "./PlaywrightElectronActiveFuses.js";
const OPERATION = "capture_electron_scenario" as const;
const STARTUP_TIMEOUT_MS = 60_000;
@@ -61,6 +63,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
// Load Playwright only when a capture launches; MCP startup stays off its module graph.
private readonly launch: typeof electron.launch = async (options) =>
(await import("playwright-core"))._electron.launch(options),
private readonly readFuseImage: typeof readElectronFuseImage = readElectronFuseImage,
) {}
identity(): ProviderIdentity {
@@ -92,6 +95,14 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
application: input.application_path,
root: input.application_root,
};
await runWithExecutionLimits(
assertNodeCliInspectArgumentsSupported(paths.executable, {
...(options.signal === undefined ? {} : { signal: options.signal }),
readFuseImage: this.readFuseImage,
}),
options.signal,
startupDeadline,
);
const captureBaseline =
await systemProcessOwnershipHost.captureBaseline?.(options.signal);
if (options.signal?.aborted)
+70 -7
View File
@@ -290,7 +290,7 @@ it.each([
},
);
it("keeps unsafe extension numeric lexemes and rejects unsafe mandatory schema numbers", () => {
it("keeps unsafe numeric lexemes in extension and mandatory schema fields", () => {
const text = JSON.stringify(historicalHar());
const result = decodeHarCapture(
text.replace('"cache":{}', '"cache":{},"_big":9007199254740993'),
@@ -302,12 +302,17 @@ it("keeps unsafe extension numeric lexemes and rejects unsafe mandatory schema n
literal: "9007199254740993",
});
expect(result.records[0]?.reported).toMatchObject({ _big: null });
expect(() =>
decodeHarCapture(
text.replace('"bodySize":0', '"bodySize":9007199254740993'),
[],
),
).toThrow("HAR schema validation failed");
// The pinned schema admits a valid integer the evidence cannot publish exactly.
const mandatory = decodeHarCapture(
text.replace('"bodySize":0', '"bodySize":9007199254740993'),
[],
).records[0];
expect(mandatory?.numeric_literals).toContainEqual({
pointer: "/request/bodySize",
producer_type: "json-number",
literal: "9007199254740993",
});
expect(mandatory?.reported).toMatchObject({ request: { bodySize: null } });
});
it.each(["container", "record"] as const)(
@@ -693,3 +698,61 @@ it("ignores one leading byte-order mark, as HAR 1.2 requires of readers", () =>
decodeCaptureText(Buffer.from([0x7b, 0xff, 0x7d]), "HAR"),
).toThrow("HAR is not valid UTF-8 JSON.");
});
const reportedAt = (reported: unknown, pointer: string): unknown =>
pointer
.split("/")
.slice(1)
.reduce<unknown>(
(value, key) =>
value !== null && typeof value === "object"
? Reflect.get(value, key)
: undefined,
reported,
);
it.each([
{
from: '"time":1.25',
literal: "0.123456789012345678901",
pointer: "/time",
},
{ from: '"bodySize":99', literal: "1e400", pointer: "/response/bodySize" },
])(
"admits a schema-valid number that evidence cannot publish exactly: $literal",
({ from, literal, pointer }) => {
const key = from.slice(0, from.indexOf(":") + 1);
const text = JSON.stringify(historicalHar()).replace(
from,
`${key}${literal}`,
);
expect(text).toContain(`${key}${literal}`);
const record = decodeHarCapture(text, []).records[0];
// The exact producer lexeme is retained, and the published value stays unknown.
expect(record?.numeric_literals).toContainEqual({
pointer,
producer_type: "json-number",
literal,
});
expect(reportedAt(record?.reported, pointer)).toBeNull();
},
);
it.each([
"9007199254740993.5",
"1e-400",
"1.0000000000000000001",
`1${"0".repeat(309)}.5`,
`-1${"0".repeat(309)}.5`,
])(
"judges an integer field by the lexeme when rounding or overflow would hide a fraction: %s",
(literal) => {
const text = JSON.stringify(historicalHar()).replace(
'"bodySize":99',
`"bodySize":${literal}`,
);
expect(() => decodeHarCapture(text, [])).toThrow(
"HAR schema validation failed: type (must be integer).",
);
},
);
+37 -10
View File
@@ -101,6 +101,7 @@ export const decodeHarCapture = (
};
};
/** Evidence projection when `redact` is true; the upstream schema validation copy otherwise. */
const projectHar = (
value: unknown,
sensitiveValues: readonly string[],
@@ -109,6 +110,18 @@ const projectHar = (
const redactor = new CaptureRedaction(sensitiveValues);
const numeric_literals: Projection["numeric_literals"] = [];
const binary_fields: Projection["binary_fields"] = [];
const projectNumber = (item: LosslessNumber, pointer: string): JsonValue => {
// Evidence publishes an unsafe number as null; the schema must still judge it as a number.
if (!redact) return validationNumber(item.value);
const number = redactor.number(item, pointer, redact);
if (number === null) return null;
numeric_literals.push({
pointer,
producer_type: "json-number",
literal: number.literal,
});
return number.reported;
};
const visit = (
item: unknown,
pointer: string,
@@ -127,16 +140,7 @@ const projectHar = (
redactor.harCredential(pointer, parent)
)
return redactor.credential(pointer);
if (item instanceof LosslessNumber) {
const number = redactor.number(item, pointer, redact);
if (number === null) return null;
numeric_literals.push({
pointer,
producer_type: "json-number",
literal: number.literal,
});
return number.reported;
}
if (item instanceof LosslessNumber) return projectNumber(item, pointer);
if (typeof item === "string") {
if (
redact &&
@@ -235,6 +239,29 @@ const projectHar = (
};
};
/**
* A JSON number lexeme as the validation copy carries it: its nearest double, unless rounding or
* overflow erased a fractional part, so the schema's `integer` checks the lexeme, not the double.
*/
const validationNumber = (literal: string): number => {
const nearest = Number(literal);
if (integerLexeme(literal)) return nearest;
if (Number.isFinite(nearest) && !Number.isInteger(nearest)) return nearest;
return literal.startsWith("-") ? -0.5 : 0.5;
};
/** Whether a JSON number lexeme denotes an integer, decided from its digits and exponent. */
const integerLexeme = (literal: string): boolean => {
const match = /^-?(\d+)(?:\.(\d+))?(?:[eE]([+-]?\d+))?$/.exec(literal);
if (match === null) return false;
const [, whole = "", fraction = "", exponent = "0"] = match;
const digits = `${whole}${fraction}`;
const significant = digits.replace(/0+$/, "");
if (/^0*$/.test(significant)) return true;
const trailingZeros = digits.length - significant.length;
return Number(exponent) - fraction.length + trailingZeros >= 0;
};
const decodeHarBase64 = (value: string, pointer: string): Buffer => {
const compact = value.replace(/[\t\n\r ]/g, "");
const bytes = Buffer.from(compact, "base64");
+2 -1
View File
@@ -235,7 +235,8 @@ const routesToJavaScriptAnalysis = async (
const isJavaScriptApplicationPath = async (path: string): Promise<boolean> => {
const lower = path.toLowerCase();
if (lower.endsWith(".asar")) return true;
if (lower.endsWith(".app")) return false;
// `A.app/` and `A.app/.` name the same bundle as `A.app`.
if (lower.replace(/(?:\/\.?)+$/u, "").endsWith(".app")) return false;
try {
return (await stat(path)).isDirectory();
} catch (cause: unknown) {
+21
View File
@@ -42,6 +42,27 @@ it("omits REA_JADX_MCP_JAR from the provider registration environment when it is
expect(environment).not.toHaveProperty("REA_JADX_MCP_JAR");
});
it("carries a validated REA_KUNA_COMMAND into the provider registration environment", async () => {
expect(DOCTOR_PROVIDER_IDS).toContain("kuna");
const root = await createTestTempDirectory("rea-kuna-registration-");
const kuna = join(root, "kuna");
await writeFile(kuna, "#!/bin/sh\nprintf 'kuna 1.833\\n'\n", { mode: 0o700 });
const inspections = await createSystemDoctorHost({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
}).providerInspections?.();
expect(inspections?.find(({ id }) => id === "kuna")).toMatchObject({
available: true,
providerVersion: "1.833",
});
expect(providerRegistrationEnvironment(inspections ?? [])).toMatchObject({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
});
const createFakeJdk = async (root: string): Promise<string> => {
const javaHome = join(root, "fake-jdk");
const bin = join(javaHome, "bin");
+2
View File
@@ -7,6 +7,7 @@ import { fileURLToPath } from "node:url";
import { inspectJadxProvider } from "./android/JadxDoctor.js";
import { inspectSystemGhidraProvider } from "./ghidra/GhidraDoctor.js";
import { inspectIdaRegistration } from "./ida/IdaDoctor.js";
import { inspectKunaProvider } from "./kuna/KunaDoctor.js";
import { readIdaConfiguration } from "./ida/IdaConfiguration.js";
import { parseJebMcpEndpoint } from "./jeb/JebMcpEndpoint.js";
import { selectLinuxPrivateDisplayStrategy } from "./hopper/LinuxPrivateDisplayProbe.js";
@@ -45,6 +46,7 @@ export const createSystemDoctorHost = (
await inspectSystemGhidraProvider(environment),
inspectIdaRegistration(environment.REA_IDA_MCP_CONFIG),
await inspectJadxProvider(environment),
await inspectKunaProvider(environment),
],
linuxDemoRuntimeCheck: () => inspectLinuxPrivateDisplay(environment),
});
@@ -48,6 +48,38 @@ describe("Electron identity through Webpack externals", () => {
expect(electronOf(source).browser_windows).toEqual([]);
});
const GETTER_CONSUMER = "const s=n.n(n(45840));new (s().BrowserWindow)({});";
it.each([
["Reflect.set", 'Reflect.set(n,"n",e=>()=>({BrowserWindow:class{}}));'],
[
"Reflect.defineProperty",
'Reflect.defineProperty(n,"n",{value:e=>()=>0});',
],
["delete", "delete n.n;"],
["a destructuring write", "({g:n.n}={g:e=>()=>0});"],
["an unknown key", "n[key]=e=>()=>0;"],
])(
"does not infer Electron after a getter write through %s",
(_name, write) => {
expect(
electronOf(bootstrap(EXTERNAL, GETTER_CONSUMER, `${write}n(100);`))
.browser_windows,
).toEqual([]);
},
);
it("keeps the getter after a reflective write to another loader key", () => {
expect(
electronOf(
bootstrap(
EXTERNAL,
GETTER_CONSUMER,
'Object.defineProperty(n,"d",{value:1});n(100);',
),
).browser_windows,
).toHaveLength(1);
});
it("follows the loader passed to a module factory", () => {
const electron = electronOf(bootstrap(EXTERNAL, CONSUMER));
expect(electron.browser_windows).toMatchObject([
@@ -112,3 +112,137 @@ describe("Electron option evidence", () => {
);
});
});
describe("Electron empty option keys", () => {
describe.each([
["plain empty string", '[""]'],
["empty template", "[``]"],
["wrapped empty string", '["" as const]'],
])("unrelated %s option keys", (_label, key) => {
it("retains an earlier explicit webPreferences object", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow } = require("electron");
new BrowserWindow({
webPreferences: { preload: "./kept.js", sandbox: true },
${key}: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "object-literal",
preload_path: "./kept.js",
preload_resolution_context: "module-specifier",
web_preferences: [
{
name: "preload",
value: { status: "literal", value: "./kept.js" },
},
{ name: "sandbox", value: { status: "literal", value: true } },
],
},
]);
});
it("retains an earlier explicit preload within webPreferences", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow } = require("electron");
new BrowserWindow({ webPreferences: {
preload: "./kept.js", sandbox: true, ${key}: false
} });
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "object-literal",
preload_path: "./kept.js",
preload_resolution_context: "module-specifier",
web_preferences: [
{ name: "", value: { status: "literal", value: false } },
{
name: "preload",
value: { status: "literal", value: "./kept.js" },
},
{ name: "sandbox", value: { status: "literal", value: true } },
],
},
]);
});
it("retains an earlier explicit utility-process service name", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { utilityProcess } = require("electron");
utilityProcess.fork("./worker.js", [], {
serviceName: "kept", ${key}: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.utility_processes).toMatchObject([
{ module_path: "./worker.js", service_name: "kept" },
]);
});
});
it.each([
["dynamic key", "[key]: null"],
["spread", "...options"],
])(
"keeps a preceding %s override unknown after an unrelated empty template key",
(_label, override) => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
new BrowserWindow({
webPreferences: { preload: "./maybe.js" },
${override}, [\`\`]: true
});
new BrowserWindow({ webPreferences: {
preload: "./maybe.js", ${override}, [\`\`]: true
} });
utilityProcess.fork("./worker.js", [], {
serviceName: "maybe", ${override}, [\`\`]: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("partial");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "dynamic",
web_preferences: [],
preload_path: null,
preload_resolution_context: null,
},
{
web_preferences_status: "object-literal",
preload_path: null,
preload_resolution_context: null,
},
]);
expect(analysis.electron.browser_windows[1]?.web_preferences).toEqual(
expect.arrayContaining([
{
name: "",
value: { status: "literal", value: true, expression: null },
},
{
name: "preload",
value: { status: "literal", value: "./maybe.js", expression: null },
},
expect.objectContaining({
name: expect.stringMatching(/^\[(?:dynamic|spread)@/u),
value: expect.objectContaining({ status: "dynamic", value: null }),
}),
]),
);
expect(analysis.electron.utility_processes).toMatchObject([
{ module_path: "./worker.js", service_name: null },
]);
},
);
});
@@ -65,9 +65,8 @@ export const objectProperty = (
if (!t.isObjectExpression(object)) return { status: "dynamic" };
for (const property of object.properties.toReversed()) {
if (t.isSpreadElement(property)) return { status: "dynamic" };
const key = semanticStaticPropertyName(property.key, property.computed);
if (key === "" && !t.isStringLiteral(property.key))
return { status: "dynamic" };
const key = semanticStaticPropertyKey(property.key, property.computed);
if (key === null) return { status: "dynamic" };
if (key !== name) continue;
return t.isObjectProperty(property)
? { status: "explicit", property }
@@ -2,7 +2,12 @@ import * as t from "@babel/types";
import { javascriptBootstrapAt } from "./javascriptBootstrap.js";
import { proveJavaScriptBootstrapIntegrity } from "./javascriptBootstrapIntegrity.js";
import { semanticStaticPropertyKey } from "./javascriptAstValues.js";
import {
semanticStaticPropertyKey,
unwrapJavaScriptExpression,
} from "./javascriptAstValues.js";
import { collectJavaScriptIdentityWritesSteps } from "./javascriptElectronMemberWrites.js";
import { createSemanticModuleInterop } from "./javascriptModuleInterop.js";
import {
semanticMemberChain,
semanticRequireOrigin,
@@ -18,7 +23,10 @@ import {
moduleFactory,
modulePropertyName,
} from "./javascriptStaticAnalysisHelpers.js";
import { traverseJavaScriptAst } from "./javascriptSemanticTraversal.js";
import {
completeSemanticSteps,
traverseJavaScriptAst,
} from "./javascriptSemanticTraversal.js";
/** Loader bindings and literal externals proven for one standalone bootstrap. */
interface BootstrapExternals {
@@ -142,60 +150,38 @@ const canonicalWebpackInteropBindings = (
bootstrapBody: t.BlockStatement,
state: JavaScriptSemanticAnalysisState,
): ReadonlySet<string> => {
const writes = new Map<string, t.AssignmentExpression[]>();
let reflectiveWrite = false;
traverseJavaScriptAst(state.program, {
enter: (node) => {
if (t.isAssignmentExpression(node) && t.isMemberExpression(node.left)) {
const target = node.left;
const bindingId = t.isIdentifier(target.object)
? resolveSemanticBindingState(
state,
target.object,
target.object.name,
)?.bindingId
: undefined;
if (
semanticStaticPropertyKey(target.property, target.computed) === "n" &&
bindingId !== undefined &&
loaderBindingIds.has(bindingId)
)
writes.set(bindingId, [...(writes.get(bindingId) ?? []), node]);
}
if (
!t.isCallExpression(node) ||
!t.isMemberExpression(node.callee) ||
!t.isIdentifier(node.callee.object, { name: "Object" }) ||
![
"defineProperty",
"defineProperties",
"assign",
"setPrototypeOf",
].includes(
semanticStaticPropertyKey(
node.callee.property,
node.callee.computed,
) ?? "",
) ||
!t.isIdentifier(node.arguments[0])
)
return;
reflectiveWrite ||= loaderBindingIds.has(
resolveSemanticBindingState(
state,
node.arguments[0],
node.arguments[0].name,
)?.bindingId ?? "",
);
},
});
// The shared identity-write owner covers assignment, delete, destructuring,
// and Object/Reflect mutators; any write that may select `n` counts.
const writes = new Set<t.Node>();
const interop = createSemanticModuleInterop(state);
completeSemanticSteps(
collectJavaScriptIdentityWritesSteps(
state.program,
(root, kind) => {
const node = unwrapJavaScriptExpression(root).node;
return kind === "member" &&
t.isIdentifier(node) &&
loaderBindingIds.has(
resolveSemanticBindingState(state, node, node.name)?.bindingId ??
"",
)
? [["loader"]]
: [];
},
interop.intrinsicInvocation,
interop.staticString,
interop.isFreshObject,
(node, path) => {
if (path.length < 2 || path[1] === "n" || path[1] === null)
writes.add(node);
},
),
);
const proven = new Set<string>();
if (reflectiveWrite) return proven;
const allWrites = [...writes.values()].flat();
const [write] = allWrites;
const [write] = writes;
if (
allWrites.length === 1 &&
write !== undefined &&
writes.size === 1 &&
t.isAssignmentExpression(write) &&
isInstalledBeforeEntryDispatch(
write,
bootstrapBody,
@@ -0,0 +1,108 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const returned = (source: string) => {
const ir = analyzeJavaScriptSemantics(source);
const callable = ir.callables.find(({ name }) => name === "observe");
return callable?.returnSites[0]?.value;
};
it("keeps members unknown after a write through a local setter", () => {
// The setter runs on assignment and may replace any member of `this`.
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) { this.value = next; } };
state.update = 2;
return state;
}
`),
).toMatchObject({ status: "unknown" });
});
it("keeps sibling literals when the written key is an ordinary property", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps nested members unknown after a write through a nested setter", () => {
const value = returned(`
export function observe() {
const state = {
label: "kept",
nested: { value: 1, set update(next) { this.value = next; } },
};
state.nested.update = 2;
return state;
}
`);
expect(value).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "label",
value: { status: "literal", value: "kept" },
}),
expect.objectContaining({
name: "nested",
value: expect.objectContaining({ status: "unknown" }),
}),
]),
});
});
it("keeps sibling literals when an accessor is deleted rather than assigned", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {} };
delete state.update;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps sibling literals when a later data property replaces the setter", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {}, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
@@ -134,11 +134,19 @@ it("keeps a shadowed require local through a deep member initializer", () => {
it.each([
["root.first.last()", "root.first.last"],
["root[`first`].last()", "root.first.last"],
['root["first" as const].last()', "root.first.last"],
['root[""].last()', "root..last"],
["root[``].last()", "root..last"],
['root["" as const].last()', "root..last"],
['("")[""].last()', "last"],
['("")[""]()', ""],
["root[0].last()", "root.0.last"],
["root[0x1n].last()", "root.1.last"],
["root[-1n].last()", "root.-1.last"],
["root[key].last()", "root.[computed@5].last"],
["root[`first${key}`].last()", "root.[computed@5].last"],
["root[`\\unicode`].last()", "root.[computed@5].last"],
["root?.first.last()", "root.first.last"],
[`root${ordinaryMemberChain}.last()`, `root${ordinaryMemberChain}.last`],
])("preserves exact callee syntax (case %#)", (source, expected) => {
@@ -0,0 +1,59 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const releases = (source: string) =>
analyzeJavaScriptSemantics(source)
.resourceOperations.filter(({ kind }) => kind === "release")
.map(({ method, linkedResourceIds, resolution }) => ({
method,
linked: linkedResourceIds.length,
resolution,
}));
it.each([
[
"a declaration",
`import { createReadStream } from "node:fs";
const stream = createReadStream("input.txt");`,
],
[
"a later assignment",
`import { createReadStream } from "node:fs";
let stream;
stream = createReadStream("input.txt");`,
],
[
"an awaited handle",
`import { open } from "node:fs/promises";
const stream = await open("input.txt");`,
],
[
"a nullish placeholder before the acquisition",
`import { open } from "node:fs/promises";
let stream = null;
try { stream = await open("input.txt"); } finally {}`,
],
])(
"keeps a release complete when the handle holds only %s",
(_name, acquire) => {
expect(releases(`${acquire}\nstream.close();`)).toEqual([
{ method: "close", linked: 1, resolution: "complete" },
]);
},
);
it.each([
["an unconditional replacement", "stream = { destroy() {} };"],
["a conditional replacement", "if (flag) stream = { destroy() {} };"],
["a destructuring replacement", "({ stream } = other);"],
])("keeps a release partial after %s", (_name, replacement) => {
expect(
releases(`
import { createReadStream } from "node:fs";
let stream = createReadStream("input.txt");
${replacement}
stream.destroy();
`),
).toEqual([{ method: "destroy", linked: 1, resolution: "partial" }]);
});
@@ -0,0 +1,59 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const responses = (source: string) =>
analyzeJavaScriptSemantics(source)
.requestOperations.filter(({ kind }) => kind === "response-consumer")
.map(({ method, endpoint, resolution }) => ({
method,
endpoint,
resolution,
}));
it("keeps the endpoint of a uniquely consumed request response", () => {
expect(
responses(`
async function load() {
const a = await fetch("https://example.test/a");
const b = await fetch("https://example.test/b");
return [await a.json(), await b.text()];
}
`),
).toEqual([
{
method: "json",
endpoint: "https://example.test/a",
resolution: "complete",
},
{
method: "text",
endpoint: "https://example.test/b",
resolution: "complete",
},
]);
});
it("leaves an ambiguous response consumer without an endpoint", () => {
expect(
responses(`
async function load(flag) {
let response = await fetch("https://example.test/a");
if (flag) response = await fetch("https://example.test/b");
return response.json();
}
`),
).toEqual([{ method: "json", endpoint: null, resolution: "partial" }]);
});
it("leaves a response replaced by a non-request value without an endpoint", () => {
expect(
responses(`
async function load(cachedResponse) {
let response = await fetch("https://example.test/a");
response = cachedResponse;
return response.json();
}
`),
).toEqual([expect.objectContaining({ method: "json", endpoint: null })]);
});
@@ -270,6 +270,18 @@ const collectResponseConsumers = (
? []
: (requestsByBinding.get(binding.bindingId) ?? []);
if (linked.length === 0) return;
// A consumer of exactly one request through its own result binding
// reads that request's response, so it keeps the request's endpoint.
const uniqueRequest =
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? linked[0]
: undefined;
// Any other write (assignment, update, loop or destructuring target) can
// replace the response before this read, so its endpoint stays unknown.
const soleWrite = binding?.initializers.length === 1;
output.push({
requestId: `response:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "response-consumer",
@@ -287,15 +299,9 @@ const collectResponseConsumers = (
linkedRequestIds: linked
.map(({ operation }) => operation.requestId)
.sort(compareUnicodeCodePoints),
endpoint: null,
endpoint: soleWrite ? (uniqueRequest?.operation.endpoint ?? null) : null,
fields: [],
resolution:
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? "complete"
: "partial",
resolution: uniqueRequest === undefined ? "partial" : "complete",
});
});
return output;
@@ -53,6 +53,82 @@ type PropertyPath = JavaScriptSemanticPropertyPath;
type ValueEffect = "write" | "escape";
/** Preserve uncertainty from property writes and references exposed to calls, yielding during traversal and fixpoint iteration. */
/** Setter keys of one object literal; a computed setter key may match any key. */
interface SetterKeys {
readonly names: ReadonlySet<string>;
readonly computed: boolean;
}
// Writes to one literal repeat; scan its properties once, including the
// common result that it defines no setter.
const setterKeysByObject = new WeakMap<t.ObjectExpression, SetterKeys | null>();
const setterKeys = (object: t.ObjectExpression): SetterKeys | null => {
const cached = setterKeysByObject.get(object);
if (cached !== undefined) return cached;
const names = new Set<string>();
let computed = false;
// Follow definition order: a later data property or method with the same
// static name replaces the accessor, so its setter no longer runs. Spreads
// and unknown computed keys may or may not replace it and keep it.
for (const property of object.properties) {
if (t.isSpreadElement(property)) continue;
const name = semanticStaticPropertyKey(property.key, property.computed);
if (t.isObjectMethod(property) && property.kind === "set") {
if (name === null) computed = true;
else names.add(name);
} else if (
name !== null &&
!(t.isObjectMethod(property) && property.kind === "get")
)
names.delete(name);
}
const keys = names.size === 0 && !computed ? null : { names, computed };
setterKeysByObject.set(object, keys);
return keys;
};
/**
* The receiver path whose own object-literal setter an assignment to `path`
* runs, following nested literal members from the binding's initializer.
*/
const localSetterReceiver = (
initializers: readonly {
readonly node: t.Node;
readonly projection: readonly unknown[];
}[],
path: readonly unknown[],
): readonly string[] | undefined => {
const key = path.at(-1);
if (typeof key !== "string") return undefined;
const receiver = path.slice(0, -1);
if (!receiver.every((segment) => typeof segment === "string"))
return undefined;
for (const { node, projection } of initializers) {
if (projection.length !== 0) continue;
let object: t.Node | undefined = unwrapJavaScriptExpression(node).node;
for (const segment of receiver) {
if (!t.isObjectExpression(object)) break;
const member: t.ObjectExpression["properties"][number] | undefined =
object.properties.findLast(
(property) =>
t.isObjectProperty(property) &&
semanticStaticPropertyKey(property.key, property.computed) ===
segment,
);
object = t.isObjectProperty(member)
? unwrapJavaScriptExpression(member.value).node
: undefined;
}
if (!t.isObjectExpression(object)) continue;
const keys = setterKeys(object);
if (keys !== null && (keys.computed || keys.names.has(key)))
return receiver as readonly string[];
}
return undefined;
};
export function* collectSemanticMemberMutationsSteps(
program: t.Program,
state: JavaScriptSemanticAnalysisState,
@@ -377,6 +453,16 @@ export function* collectSemanticMemberMutationsSteps(
current.originAt,
parents,
);
// A setter on the binding's own literal runs on assignment and may
// replace any member of its receiver, so the written key is unknown.
// Deleting the accessor never calls it.
if (
effect === "write" &&
!t.isUnaryExpression(mutation, { operator: "delete" })
) {
const receiver = localSetterReceiver(origins.initializers, path);
if (receiver !== undefined) path = [...receiver, null];
}
// Keep an antichain of exact paths. Widening distinct member keys would
// also invalidate unrelated siblings that never reach this effect.
// This map is local to the file scan. Escaping the binding id keeps a
@@ -0,0 +1,73 @@
import * as t from "@babel/types";
import { expect, it } from "vitest";
import { indexJavaScriptBindingsSteps } from "./javascriptSemanticAnalysis.js";
import { semanticObjectPropertyReferences } from "./javascriptSemanticObjectReferences.js";
import { completeSemanticSteps } from "./javascriptSemanticTraversal.js";
import { parseJavaScriptSource } from "./javascriptSourceParser.js";
const objectAndState = (source: string) => {
const parsed = parseJavaScriptSource(source, "object.js");
if (parsed === null) throw new Error("Expected a parsed fixture");
const declaration = parsed.program.body[0];
if (!t.isVariableDeclaration(declaration))
throw new Error("Expected a declaration");
const init = declaration.declarations[0]?.init;
if (!t.isObjectExpression(init)) throw new Error("Expected an object");
return {
object: init,
source,
state: completeSemanticSteps(indexJavaScriptBindingsSteps(parsed.program)),
};
};
const spelled = (
fixture: ReturnType<typeof objectAndState>,
key: string | number | null,
) =>
semanticObjectPropertyReferences(fixture.object, [key], fixture.state).map(
({ node, path }) => [
fixture.source.slice(node.start ?? 0, node.end ?? 0),
JSON.stringify(path),
],
);
it("resolves exact keys with source-order overrides, spreads and accessors", () => {
const fixture = objectAndState(
`const o = { a: 1, ...base, b: 2, a: 3, [dynamic]: 4, __proto__: proto, ...{ b: 5 }, get c() { return 6; }, c: 7 };`,
);
// Expectations match the full source-order scan these exact keys replace.
expect(spelled(fixture, "a")).toEqual([
["{ b: 5 }", '["a"]'],
["4", "[]"],
["3", "[]"],
]);
expect(spelled(fixture, "b")).toEqual([
["{ b: 5 }", '["b"]'],
["4", "[]"],
]);
expect(spelled(fixture, "c")).toEqual([
["7", "[]"],
["4", "[]"],
]);
expect(spelled(fixture, "missing")).toEqual([
["{ b: 5 }", '["missing"]'],
["proto", '["missing"]'],
["4", "[]"],
["base", '["missing"]'],
]);
// A wildcard key still scans every property.
expect(spelled(fixture, null)).toEqual([
["7", "[]"],
["{ b: 5 }", '[{"excludedKeys":["c"]}]'],
["proto", "[null]"],
["4", "[]"],
["3", "[]"],
["2", "[]"],
["base", '[{"excludedKeys":["a","b","c"]}]'],
]);
// Numeric keys use the same exact spelling as string keys.
expect(
spelled(objectAndState(`const o = { 1: "one", 2: "two" };`), 1),
).toEqual([['"one"', "[]"]]);
});
@@ -739,6 +739,75 @@ const getterResultReferences = (
return references;
};
type ObjectLiteralMember = t.ObjectExpression["properties"][number];
/** Property positions grouped by whether an exact key can reach them. */
interface ObjectPropertyIndex {
/** Spreads, computed keys and `__proto__` data entries reach every key. */
readonly keyIndependent: readonly number[];
readonly byName: ReadonlyMap<string, readonly number[]>;
}
// Mutation tracking resolves one key of the same literal per write, so the
// index removes the per-write scan of every property.
const propertyIndexes = new WeakMap<t.ObjectExpression, ObjectPropertyIndex>();
const objectPropertyIndex = (node: t.ObjectExpression): ObjectPropertyIndex => {
const cached = propertyIndexes.get(node);
if (cached !== undefined) return cached;
const keyIndependent: number[] = [];
const byName = new Map<string, number[]>();
node.properties.forEach((property, index) => {
const name = t.isSpreadElement(property)
? null
: semanticStaticPropertyKey(property.key, property.computed);
if (
name === null ||
(t.isObjectProperty(property) &&
!property.computed &&
!property.shorthand &&
name === "__proto__")
) {
keyIndependent.push(index);
return;
}
const positions = byName.get(name);
if (positions === undefined) byName.set(name, [index]);
else positions.push(index);
});
const index = { keyIndependent, byName };
propertyIndexes.set(node, index);
return index;
};
/**
* Properties in reverse source order that can affect `key`. For an exact key,
* another static name only updates bookkeeping for that other name, which
* this key never reads, so only same-name and key-independent entries remain.
*/
const relevantPropertiesInReverse = (
node: t.ObjectExpression,
key: PropertyPath[number] | undefined,
): readonly ObjectLiteralMember[] => {
if (typeof key !== "string" && typeof key !== "number")
return [...node.properties].reverse();
const { keyIndependent, byName } = objectPropertyIndex(node);
const named = byName.get(String(key)) ?? [];
const relevant: ObjectLiteralMember[] = [];
let left = keyIndependent.length - 1;
let right = named.length - 1;
while (left >= 0 || right >= 0) {
const fromLeft =
right < 0 ||
(left >= 0 && (keyIndependent[left] ?? -1) > (named[right] ?? -1));
const position = fromLeft ? keyIndependent[left--] : named[right--];
const property =
position === undefined ? undefined : node.properties[position];
if (property !== undefined) relevant.push(property);
}
return relevant;
};
export const semanticObjectPropertyReferences = (
node: t.ObjectExpression,
path: PropertyPath,
@@ -751,7 +820,7 @@ export const semanticObjectPropertyReferences = (
const accessorNames = new Set<string>();
const getterNames = new Set<string>();
const setterNames = new Set<string>();
for (const property of [...node.properties].reverse()) {
for (const property of relevantPropertiesInReverse(node, key)) {
if (t.isSpreadElement(property)) {
const selected = key ?? null;
if (typeof selected !== "object" && overwritten.has(String(selected)))
@@ -8,7 +8,11 @@ import type {
JavaScriptSemanticCallable,
JavaScriptSemanticResourceOperation,
} from "./javascriptSemanticIr.js";
import { semanticStaticPropertyName } from "./javascriptAstValues.js";
import {
semanticStaticPropertyName,
unwrapJavaScriptExpression,
} from "./javascriptAstValues.js";
import { isUnmodifiedGlobalReference } from "./javascriptSemanticAmbientWrites.js";
import {
resolveSemanticBindingState,
type JavaScriptSemanticAnalysisState,
@@ -100,21 +104,58 @@ const collectReleases = (
);
const linked =
binding === undefined ? [] : (byBinding.get(binding.bindingId) ?? []);
if (linked.length === 0) return;
if (binding === undefined || linked.length === 0) return;
output.push({
resourceId: `resource:release:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "release",
method,
location: range(node),
ownerCallableId: context.callableStack.at(-1) ?? null,
resultBindingId: binding?.bindingId ?? null,
resultBindingId: binding.bindingId,
linkedResourceIds: linked.map(resourceCandidateId).sort(),
resolution: linked.length === 1 ? "complete" : "partial",
resolution:
linked.length === 1 &&
holdsOnlyAcquisitions(binding, linked, context.state)
? "complete"
: "partial",
});
});
return output;
};
/**
* A lexical handle identifies its acquisition only while every value written
* to the binding is that acquisition itself. Any replacement, even a
* conditional or later one, leaves the released resource a candidate. A
* nullish placeholder, as in `let h = null; try { h = open() } finally {
* h?.close() }`, releases nothing else: the call is skipped or throws.
*/
const holdsOnlyAcquisitions = (
binding: JavaScriptSemanticBindingState,
linked: readonly ResourceCandidate[],
state: JavaScriptSemanticAnalysisState,
): boolean =>
[...binding.initializers, ...binding.referenceInitializers].every(
({ node, projection }) => {
if (projection.length > 0) return false;
let value = unwrapJavaScriptExpression(node).node;
if (t.isAwaitExpression(value))
value = unwrapJavaScriptExpression(value.argument).node;
return (
isNullishPlaceholder(value, state) ||
linked.some((acquisition) => acquisition.node === value)
);
},
);
const isNullishPlaceholder = (
value: t.Node,
state: JavaScriptSemanticAnalysisState,
): boolean =>
t.isNullLiteral(value) ||
t.isUnaryExpression(value, { operator: "void" }) ||
isUnmodifiedGlobalReference(value, state, "undefined");
const immutableAcquisition = (
candidate: ResourceCandidate,
): JavaScriptSemanticResourceOperation => ({
@@ -2,6 +2,173 @@ import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
it("retains endpoint observations for exact template and TypeScript member keys", () => {
for (const [get, beacon] of [
['"get"', '"sendBeacon"'],
["`get`", "`sendBeacon`"],
['"get" as const', '"sendBeacon" as const'],
]) {
const analysis = analyzeJavaScriptStaticSource(`
import * as vscode from "vscode";
client[${get}]("/api/items");
navigator[${beacon}]("/api/telemetry", payload);
client[${get}]("setting");
new Headers()[${get}]("content-type");
new URLSearchParams()[${get}]("q");
vscode.workspace.getConfiguration("editor")[${get}]("tabSize");
`);
expect.soft(analysis.parse_error_count).toBe(0);
expect.soft(analysis.parse_status).toBe("complete");
expect
.soft(
analysis.endpoints.map(({ kind, value, mechanism, interpretation }) => [
kind,
value,
mechanism,
interpretation ?? null,
]),
)
.toEqual([
["network", "/api/items", "call:client.get", null],
["network", "/api/telemetry", "call:navigator.sendBeacon", null],
["network", "setting", "call:client.get", "candidate"],
]);
}
});
it("preserves preload and utility paths through exact computed join members", () => {
for (const key of ['"join"', "`join`", '"join" as const']) {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
const path = require("node:path");
new BrowserWindow({ webPreferences: {
preload: path[${key}](__dirname, "preload.js")
} });
utilityProcess.fork(path[${key}](__dirname, "worker.js"), [], {
serviceName: "kept"
});
`);
expect.soft(analysis.parse_error_count).toBe(0);
expect.soft(analysis.parse_status).toBe("complete");
expect
.soft(
analysis.role_paths.map(
({ role, path, resolution_context, mechanism }) => [
role,
path,
resolution_context,
mechanism,
],
),
)
.toEqual([
["preload", "preload.js", "filesystem-expression", "property:preload"],
]);
expect
.soft(
analysis.electron.browser_windows.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
options_status: "object-literal",
web_preferences_status: "object-literal",
web_preferences: [
{
name: "preload",
value: {
status: "literal",
value: "preload.js",
expression: null,
},
},
],
preload_path: "preload.js",
preload_resolution_context: "filesystem-expression",
module_key: null,
},
]);
expect
.soft(
analysis.electron.utility_processes.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
module_path: "worker.js",
module_resolution_context: "filesystem-expression",
module_expression: null,
service_name: "kept",
module_key: null,
},
]);
}
});
it("keeps dynamic, interpolated, and invalid template keys unresolved", () => {
for (const { key, errors } of [
{ key: "method", errors: 0 },
{ key: "`join${suffix}`", errors: 0 },
{ key: ["`", String.raw`\unicode`, "`"].join(""), errors: 4 },
]) {
const preload = `path[${key}](__dirname, "preload.js")`;
const worker = `path[${key}](__dirname, "worker.js")`;
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
const path = require("node:path");
client[${key}]("/api/items");
navigator[${key}]("/api/telemetry", payload);
new BrowserWindow({ webPreferences: { preload: ${preload} } });
utilityProcess.fork(${worker}, [], { serviceName: "kept" });
`);
expect.soft(analysis.parse_error_count).toBe(errors);
expect.soft(analysis.parse_status).toBe("partial");
expect.soft(analysis.endpoints).toEqual([]);
expect.soft(analysis.role_paths).toEqual([]);
expect
.soft(
analysis.electron.browser_windows.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
options_status: "object-literal",
web_preferences_status: "object-literal",
web_preferences: [
{
name: "preload",
value: { status: "dynamic", value: null, expression: preload },
},
],
preload_path: null,
preload_resolution_context: null,
module_key: null,
},
]);
expect
.soft(
analysis.electron.utility_processes.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
module_path: null,
module_resolution_context: null,
module_expression: worker,
service_name: "kept",
module_key: null,
},
]);
}
});
describe("static callee boundary matching", () => {
it("matches complete callee members without suffix collisions", () => {
const analysis = analyzeJavaScriptStaticSource(`
@@ -693,11 +693,8 @@ export const calleeName = (node: t.Node): string => {
const memberPropertyName = (
node: t.MemberExpression | t.OptionalMemberExpression,
): string =>
node.computed &&
!t.isStringLiteral(node.property) &&
!t.isNumericLiteral(node.property)
? `[computed@${String(node.property.start ?? -1)}]`
: semanticStaticPropertyName(node.property, node.computed);
semanticStaticPropertyKey(node.property, node.computed) ??
(node.computed ? `[computed@${String(node.property.start ?? -1)}]` : "");
/**
* Return a string literal value without evaluating an expression. A template
@@ -23,6 +23,7 @@ const absoluteParts = [
const anchoredExpressions = [
['path.resolve(__dirname, "preload.cjs")', "preload.cjs"],
['path[`resolve`](__dirname, "preload.js")', "preload.js"],
['path.posix.resolve("ignored", __dirname, "preload.js")', "preload.js"],
['path.resolve("/ignored", __dirname, "preload.js")', "preload.js"],
[
@@ -45,6 +46,8 @@ const anchoredExpressions = [
] as const;
const unknownExpressions = [
'path[method](__dirname, "preload.js")',
'path[`join${suffix}`](__dirname, "preload.js")',
"path.resolve()",
"path.posix.resolve()",
'path.resolve("sub", "../preload.js")',
@@ -106,6 +109,11 @@ describe("inert JavaScript path resolution", () => {
"preload.js",
"filesystem-expression",
],
[
'path[`join`](__dirname, "preload.js")',
"preload.js",
"filesystem-expression",
],
[
'path.join(__dirname, "/preload.js")',
"preload.js",
@@ -24,6 +24,13 @@ type PinvokeImport = ManagedNativeBoundaryInspection["pinvoke_imports"][number];
type NativeImplementation =
ManagedNativeBoundaryInspection["native_implementations"][number];
/**
* Display label for a metadata name. Metadata may hold an empty name; the
* graph then has no label, while the node's properties keep the source value.
*/
const metadataLabel = (name: string): string | null =>
name === "" ? null : name;
/** Options for a managed contains relationship edge. */
export interface ContainsEdgeOptions {
readonly kind: string;
@@ -77,7 +84,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: assemblyFacts.name,
label: metadataLabel(assemblyFacts.name),
properties: {
name: assemblyFacts.name,
version: assemblyFacts.version,
@@ -111,7 +118,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: module.name,
label: metadataLabel(module.name),
properties: {
name: module.name,
mvid: module.mvid,
@@ -237,7 +244,7 @@ const typeNode = (
),
observations: [
{
label: type.full_name,
label: metadataLabel(type.full_name),
properties: {
token: type.token,
namespace: type.namespace,
@@ -266,10 +273,11 @@ const methodNode = (
),
observations: [
{
label:
label: metadataLabel(
method.declaring_type === null
? method.name
: `${method.declaring_type}.${method.name}`,
),
properties: {
token: method.token,
declaring_type_token: method.declaring_type_token,
@@ -302,10 +310,11 @@ const fieldNode = (
),
observations: [
{
label:
label: metadataLabel(
field.declaring_type === null
? field.name
: `${field.declaring_type}.${field.name}`,
),
properties: {
token: field.token,
declaring_type_token: field.declaring_type_token,
@@ -410,7 +419,7 @@ const pinvokeNode = (
),
observations: [
{
label: pinvoke.import_name,
label: metadataLabel(pinvoke.import_name),
properties: {
token: pinvoke.token,
member_token: pinvoke.member_token,
@@ -441,7 +450,7 @@ const nativeImplementationNode = (
),
observations: [
{
label: implementation.name,
label: metadataLabel(implementation.name),
properties: {
token: implementation.token,
name: implementation.name,
+101
View File
@@ -3,6 +3,7 @@ import { describe, expect, it } from "vitest";
import { readElfSymbols, readSnapshotHeader } from "./DartElf.js";
import {
REFERENCE_HASH,
buildElf32Image,
buildElf64Image,
buildStrippedElf64Image,
snapshotSection,
@@ -299,6 +300,106 @@ describe("ELF symbol-table ownership", () => {
});
});
/** Section-header field positions for each ELF class. */
const ELF_CLASSES = {
elf64: {
build: buildElf64Image,
word: {
read: (image: Buffer, at: number) => Number(image.readBigUInt64LE(at)),
write: (image: Buffer, value: number, at: number) =>
image.writeBigUInt64LE(BigInt(value), at),
},
shoff: 0x28,
shent: 64,
flags: 8,
address: 0x10,
offset: 0x18,
},
elf32: {
build: buildElf32Image,
word: {
read: (image: Buffer, at: number) => image.readUInt32LE(at),
write: (image: Buffer, value: number, at: number) =>
image.writeUInt32LE(value, at),
},
shoff: 0x20,
shent: 40,
flags: 8,
address: 0x0c,
offset: 0x10,
},
} as const;
describe.each(Object.entries(ELF_CLASSES))(
"allocated %s symbol tables",
(_, layout) => {
const { word } = layout;
/** Snapshot sections, with dynamic tables allocated at their offsets. */
const allocatedTables = (): {
image: Buffer;
headers: number[];
tables: number[];
} => {
const elf = layout.build([
{ name: "_kDartVmSnapshotData", value: 0x400, size: 52 },
]);
const section = snapshotSection(GALLERY_HASH);
const image = Buffer.concat([
elf,
Buffer.alloc(0x400 - elf.length),
section,
]);
const shoff = word.read(image, layout.shoff);
const headers = [shoff + layout.shent, shoff + 2 * layout.shent];
const tables = headers.map((header) => {
const offset = word.read(image, header + layout.offset);
word.write(image, 2, header + layout.flags);
word.write(image, offset, header + layout.address);
return offset;
});
return { image, headers, tables };
};
it("accepts tables whose mapping matches their sections", async () => {
const { image } = allocatedTables();
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [{ name: "_kDartVmSnapshotData", offset: 0x400, size: 52 }],
failures: [],
});
});
it.each([
[0, "Dynamic symbol table has conflicting PT_LOAD mappings"],
[1, "Dynamic symbol string table has conflicting PT_LOAD mappings"],
])(
"refuses table %i under a conflicting PT_LOAD interpretation",
async (table, reason) => {
const { image, tables } = allocatedTables();
const conflict = appendElfRanges(image, "program", [
{ type: 1, address: tables[table]!, offset: 0x300, size: 4 },
]);
expect(
await readElfSymbols(conflict, ["_kDartVmSnapshotData"]),
).toEqual({ symbols: [], failures: [{ reason }] });
},
);
it("refuses a table whose section offset disagrees with its mapping", async () => {
const { image, headers } = allocatedTables();
word.write(image, 0x300, headers[0]! + layout.address);
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [],
failures: [
{
reason:
"Dynamic symbol table section offset disagrees with its PT_LOAD mapping",
},
],
});
});
},
);
describe("readSnapshotHeader", () => {
it("reads a synthesized data header", async () => {
const header = readSnapshotHeader(snapshotSection(GALLERY_HASH), 0);
+33
View File
@@ -59,6 +59,7 @@ export interface ElfParseFailure {
}
const SHT_DYNSYM = 11;
const SHF_ALLOC = 2;
const PT_LOAD = 1;
const PT_DYNAMIC = 2;
/** Position-independent executable images carry PT_DYNAMIC. */
@@ -225,6 +226,28 @@ export const readElfSymbols = async (
stringOffset: number;
stringSize: number;
}[] = [];
/**
* The loader reads an allocated section through PT_LOAD, so its section
* offset must be the single file interpretation of its address range.
*/
const sectionMappingFailure = (
header: number,
offset: number,
size: number,
): string | undefined => {
const flags = is64 ? reader.u64(header + 8) : reader.u32(header + 8);
const address = is64
? reader.u64(header + 0x10)
: reader.u32(header + 0x0c);
if (flags === null || address === null)
return "has an unrepresentable section header";
if ((flags & SHF_ALLOC) === 0) return undefined;
const mapped = toOffset(address, size);
if (!mapped.ok) return mapped.error.reason;
return mapped.value === offset
? undefined
: "section offset disagrees with its PT_LOAD mapping";
};
if (!dynamicFallback) {
for (let index = 0; index < e_shnum; index += 1) {
if (index % 4096 === 0) {
@@ -269,6 +292,16 @@ export const readElfSymbols = async (
return failure(
"Dynamic symbol strings are out of bounds or not a string table",
);
const symbolMapping = sectionMappingFailure(header, offset, size);
if (symbolMapping !== undefined)
return failure(`Dynamic symbol table ${symbolMapping}`);
const stringMapping = sectionMappingFailure(
stringHeader,
stringOffset,
stringSize,
);
if (stringMapping !== undefined)
return failure(`Dynamic symbol string table ${stringMapping}`);
tables.push({
offset,
count: size / entrySize,
+57
View File
@@ -115,6 +115,63 @@ export const buildElf64Image = (
return image;
};
/** The ELF32 (ARM) counterpart of {@link buildElf64Image}, with the same layout roles. */
export const buildElf32Image = (
symbols: ReadonlyArray<{ name: string; value: number; size: number }>,
): Buffer => {
const ELF_HEADER = 52;
const PHENT = 32;
const SHENT = 40;
const SYMENT = 16;
const phoff = ELF_HEADER;
const shoff = phoff + PHENT;
const dynsymOff = shoff + 3 * SHENT + 16;
const dynstrOff = dynsymOff + symbols.length * SYMENT;
const image = Buffer.alloc(dynstrOff + 256);
image.writeUInt32LE(0x464c457f, 0);
image[4] = 1; // ELFCLASS32
image[5] = 1; // little-endian
image[6] = 1;
image.writeUInt16LE(3, 0x10);
image.writeUInt16LE(0x0028, 0x12); // e_machine ARM
image.writeUInt32LE(1, 0x14);
image.writeUInt32LE(phoff, 0x1c);
image.writeUInt32LE(shoff, 0x20);
image.writeUInt16LE(ELF_HEADER, 0x28);
image.writeUInt16LE(PHENT, 0x2a);
image.writeUInt16LE(1, 0x2c);
image.writeUInt16LE(SHENT, 0x2e);
image.writeUInt16LE(3, 0x30); // null, dynsym, dynstr
// One identity PT_LOAD covering the materialized payload.
image.writeUInt32LE(1, phoff);
image.writeUInt32LE(
Math.max(image.length, ...symbols.map(({ value, size }) => value + size)),
phoff + 0x10,
);
const dyn = shoff + SHENT;
image.writeUInt32LE(11, dyn + 4); // SHT_DYNSYM
image.writeUInt32LE(dynsymOff, dyn + 0x10);
image.writeUInt32LE(symbols.length * SYMENT, dyn + 0x14);
image.writeUInt32LE(2, dyn + 0x18); // sh_link -> strtab
image.writeUInt32LE(SYMENT, dyn + 0x24);
const str = dyn + SHENT;
image.writeUInt32LE(3, str + 4); // SHT_STRTAB
image.writeUInt32LE(dynstrOff, str + 0x10);
image.writeUInt32LE(256, str + 0x14);
let strCursor = dynstrOff + 1;
symbols.forEach((symbol, index) => {
image.write(symbol.name, strCursor, "latin1");
const base = dynsymOff + index * SYMENT;
image.writeUInt32LE(strCursor - dynstrOff, base);
image.writeUInt32LE(symbol.value, base + 4);
image.writeUInt32LE(symbol.size, base + 8);
image[base + 0xc] = 0x10; // GLOBAL
image.writeUInt16LE(1, base + 0xe); // Defined export.
strCursor += symbol.name.length + 1;
});
return image;
};
/**
* Build a section-stripped ELF64 image (e_shnum 0, ET_DYN) whose exported
* symbols are reachable only through the PT_DYNAMIC tag-value pairs, the
+16 -5
View File
@@ -69,6 +69,20 @@ const hasLegacyMipsHeaderShape = (target: BinaryTarget): boolean => {
};
/** Return a refusal for GNU ABI attributes this existing MIPS lane cannot model. */
/**
* Every MIPS lane, including PSP, needs every ABI declaration source examined:
* headers outside the captured file may hold declarations that would refuse it.
*/
export const ghidraMipsIncompleteDeclarationsReason = (
metadata: MipsElfMetadata,
): string | null =>
(metadata.limitations?.length ?? 0) > 0
? [
"MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown.",
...(metadata.limitations ?? []),
].join(" ")
: null;
export const ghidraMipsGnuAttributesUnsupportedReason = (
metadata: MipsElfMetadata,
): string | null => {
@@ -164,11 +178,8 @@ export const ghidraMipsUnsupportedReason = (
if (load.symbolRelativeCount !== 0)
return "Symbol-bearing R_MIPS_REL32 is unsupported: Ghidra normalizes MIPS16 function addresses and can lose the source function-pointer mode tag. Symbol-index-zero REL32 is verified separately.";
}
if ((metadata.limitations?.length ?? 0) > 0)
return [
"MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown.",
...(metadata.limitations ?? []),
].join(" ");
const incomplete = ghidraMipsIncompleteDeclarationsReason(metadata);
if (incomplete !== null) return incomplete;
const abi = metadata.abiFlags;
if (shared && abi === null && (metadata.limitations?.length ?? 0) === 0)
return null; // Inspected absence leaves FP semantics unknown, never inferred.
+3
View File
@@ -1,6 +1,7 @@
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
ghidraMipsGnuAttributesUnsupportedReason,
ghidraMipsIncompleteDeclarationsReason,
ghidraMipsUnsupportedReason,
} from "./GhidraMipsProfile.js";
import { ghidraPowerPcUnsupportedReason } from "./GhidraPowerPcProfile.js";
@@ -33,6 +34,8 @@ export const ghidraProcessorUnsupportedReason = (
const gnuAttributesReason =
ghidraMipsGnuAttributesUnsupportedReason(metadata);
if (gnuAttributesReason !== null) return gnuAttributesReason;
const incomplete = ghidraMipsIncompleteDeclarationsReason(metadata);
if (incomplete !== null) return incomplete;
if (metadata.elfClass !== 32 || metadata.byteOrder !== "little")
return "The PSP Allegrex profile requires little-endian ELF32.";
if (metadata.type !== 2)
+144
View File
@@ -0,0 +1,144 @@
import { isAbsolute, join } from "node:path";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { providerStartupTimeoutSchema } from "../config/providerStartupTimeout.js";
import { err, ok, type Result } from "../domain/result.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
} from "./KunaRelease.js";
/** Kuna caps its own decode lanes and worker pool at 32. */
const MAX_KUNA_JOBS = 32;
/**
* A 1.4 MB sqlite3 shell exported serially in 66-108 s on Apple M-series and
* Linux hosts; thirty minutes admits images an order of magnitude larger
* without an unbounded run. Larger images need a raised deadline or more jobs.
*/
const DEFAULT_PROJECT_TIMEOUT_MS = 30 * 60 * 1000;
/**
* A serial (`--jobs 1`) export is Kuna's reference answer: it decompiles
* callees before callers, which recovers prototypes that parallel workers
* cannot. Measured on coreutils sort, 40-60% of functions differ otherwise.
*/
const DEFAULT_JOBS = 1;
/** Measured sqlite3 exports occupy ~25 MiB; four GiB retains many projects. */
const DEFAULT_CACHE_MAX_BYTES = 4 * 1024 ** 3;
/** One project needs its .c, .h, .asm, inventory and manifest; refuse smaller caps. */
const MIN_CACHE_MAX_BYTES = 1024 ** 2;
/** Caller-selected Kuna installation and REA-owned project cache policy. */
export interface KunaConfiguration {
/** Caller-selected executable path, preserved exactly as configured. */
readonly command: string;
/** REA-owned cache root; Kuna's own spec cache lives beneath it too. */
readonly cacheRoot: string;
readonly cacheMaxBytes: number;
readonly projectTimeoutMs: number;
readonly jobs: number;
}
/** Why the configured environment cannot select a Kuna installation. */
export interface KunaConfigurationFailure {
readonly code: "not_configured" | "open_options_invalid" | "unsupported_host";
readonly reason: string;
readonly setting: string | null;
}
const failure = (
code: KunaConfigurationFailure["code"],
reason: string,
setting: string | null = null,
): Result<never, KunaConfigurationFailure> => err({ code, reason, setting });
const positiveInteger = (value: string): number | undefined =>
/^[1-9]\d*$/u.test(value) && Number.isSafeInteger(Number(value))
? Number(value)
: undefined;
/** Per-user cache directory that REA owns for Kuna on this host. */
const defaultCacheRoot = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): string => {
const home = homeDirectoryFromEnvironment(environment, platform);
if (platform === "darwin")
return join(home, "Library", "Caches", "rea", "kuna");
const xdg = environment.XDG_CACHE_HOME;
return join(
xdg !== undefined && isAbsolute(xdg) ? xdg : join(home, ".cache"),
"rea",
"kuna",
);
};
/** Parse the Kuna settings without touching the filesystem or the executable. */
export const readKunaConfiguration = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Result<KunaConfiguration, KunaConfigurationFailure> => {
const command = environment.REA_KUNA_COMMAND;
if (command === undefined || command.length === 0)
return failure("not_configured", KUNA_COMMAND_REMEDIATION);
if (!isAbsolute(command))
return failure(
"open_options_invalid",
`REA_KUNA_COMMAND must be an absolute path; received ${command}. ${KUNA_COMMAND_REMEDIATION}`,
"REA_KUNA_COMMAND",
);
if (platform !== "darwin" && platform !== "linux")
return failure("unsupported_host", KUNA_UNSUPPORTED_HOST_REMEDIATION);
const cacheRoot = environment.REA_KUNA_CACHE_DIR;
if (cacheRoot !== undefined && !isAbsolute(cacheRoot))
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_DIR must be an absolute directory; received ${cacheRoot}.`,
"REA_KUNA_CACHE_DIR",
);
const rawMaxBytes = environment.REA_KUNA_CACHE_MAX_BYTES;
const cacheMaxBytes =
rawMaxBytes === undefined
? DEFAULT_CACHE_MAX_BYTES
: positiveInteger(rawMaxBytes);
if (cacheMaxBytes === undefined || cacheMaxBytes < MIN_CACHE_MAX_BYTES)
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_MAX_BYTES must be a decimal byte count of at least ${String(MIN_CACHE_MAX_BYTES)}; received ${String(rawMaxBytes)}.`,
"REA_KUNA_CACHE_MAX_BYTES",
);
const rawTimeout = environment.REA_KUNA_PROJECT_TIMEOUT_MS;
const timeout =
rawTimeout === undefined
? undefined
: providerStartupTimeoutSchema.safeParse(rawTimeout);
if (timeout !== undefined && !timeout.success)
return failure(
"open_options_invalid",
`REA_KUNA_PROJECT_TIMEOUT_MS must be a decimal millisecond count from 1 to 2147483647; received ${String(rawTimeout)}.`,
"REA_KUNA_PROJECT_TIMEOUT_MS",
);
const rawJobs = environment.REA_KUNA_JOBS;
const jobs = rawJobs === undefined ? DEFAULT_JOBS : positiveInteger(rawJobs);
if (jobs === undefined || jobs > MAX_KUNA_JOBS)
return failure(
"open_options_invalid",
`REA_KUNA_JOBS must be an integer from 1 to ${String(MAX_KUNA_JOBS)}; received ${String(rawJobs)}.`,
"REA_KUNA_JOBS",
);
return ok({
command,
cacheRoot: cacheRoot ?? defaultCacheRoot(environment, platform),
cacheMaxBytes,
projectTimeoutMs: timeout?.data ?? DEFAULT_PROJECT_TIMEOUT_MS,
jobs,
});
};
+211
View File
@@ -0,0 +1,211 @@
import { constants } from "node:fs";
import { access, stat } from "node:fs/promises";
import { dirname } from "node:path";
import type {
DoctorProviderCheck,
DoctorProviderInspection,
} from "../application/Doctor.js";
import { readKunaConfiguration } from "./KunaConfiguration.js";
import {
inspectKunaInstallation,
type KunaReadinessFailure,
} from "./KunaInstallation.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_PROVIDER_ID,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** Settings a registration may carry; only those the caller set explicitly. */
const REGISTRATION_SETTINGS = [
"REA_KUNA_COMMAND",
"REA_KUNA_CACHE_DIR",
"REA_KUNA_CACHE_MAX_BYTES",
"REA_KUNA_PROJECT_TIMEOUT_MS",
"REA_KUNA_JOBS",
] as const;
/** Project Kuna readiness into the doctor contract without writing anything. */
export const inspectKunaProvider = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Promise<DoctorProviderInspection> => {
const installation = await inspectKunaInstallation(environment, {
platform,
});
const configured =
environment.REA_KUNA_COMMAND !== undefined &&
environment.REA_KUNA_COMMAND.length > 0;
if (!installation.ok)
return {
id: KUNA_PROVIDER_ID,
configured,
available: false,
providerVersion: null,
registrationEnvironment: {},
checks: [installationFailure(installation.error, configured)],
};
const { value } = installation;
return {
id: KUNA_PROVIDER_ID,
configured,
available: true,
providerVersion: value.version,
registrationEnvironment: Object.fromEntries(
REGISTRATION_SETTINGS.flatMap((name) => {
const setting = environment[name];
return setting === undefined ? [] : [[name, setting]];
}),
),
checks: [
{
name: "installation",
ok: true,
code: null,
detail: `${value.executable} reports Kuna ${value.version} (sha256 ${value.sha256})${
value.verified
? "."
: `; REA verified ${KUNA_VERIFIED_VERSION}, so this release's project layout is unverified.`
}`,
remediation: null,
classification: "config_drift",
details: {
executable: value.executable,
version: value.version,
verified_version: KUNA_VERIFIED_VERSION,
sha256: value.sha256,
},
},
await cacheCheck(environment, platform),
],
};
};
const installationFailure = (
failure: KunaReadinessFailure,
configured: boolean,
): DoctorProviderCheck => ({
name: "installation",
ok: false,
code: failure.code,
detail: failure.reason,
remediation:
failure.code === "unsupported_host"
? KUNA_UNSUPPORTED_HOST_REMEDIATION
: failure.code === "version_unresolved"
? KUNA_VERSION_PROBE_REMEDIATION
: KUNA_COMMAND_REMEDIATION,
classification:
failure.code === "unsupported_host"
? "unsupported_host"
: configured
? "config_drift"
: "missing_analysis_engine",
details: failure.diagnostics,
});
/** The cache root may not exist yet; doctor reports it without creating it. */
const cacheCheck = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): Promise<DoctorProviderCheck> => {
const configuration = readKunaConfiguration(environment, platform);
if (!configuration.ok)
return {
name: "cache",
ok: false,
code: configuration.error.code,
detail: configuration.error.reason,
remediation: KUNA_COMMAND_REMEDIATION,
classification: "config_drift",
};
const { cacheRoot, cacheMaxBytes } = configuration.value;
const details = { cache_root: cacheRoot, cache_max_bytes: cacheMaxBytes };
try {
if (!(await stat(cacheRoot)).isDirectory())
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `The Kuna cache root ${cacheRoot} exists but is not a directory.`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory REA may own, or remove the conflicting file.",
classification: "config_drift",
details,
};
await access(cacheRoot, constants.W_OK | constants.X_OK);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT") {
const creatable = await nearestDirectoryWritable(cacheRoot);
return creatable.ok
? {
name: "cache",
ok: true,
code: null,
detail: `REA creates the Kuna project cache at ${cacheRoot} on first use.`,
remediation: null,
classification: "config_drift",
details,
}
: {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot create the Kuna cache root ${cacheRoot}: ${creatable.reason}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can create or write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot write the Kuna cache root ${cacheRoot}: ${cause instanceof Error ? cause.message : String(cause)}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: true,
code: null,
detail: `Kuna project exports are cached in ${cacheRoot} (at most ${String(cacheMaxBytes)} bytes).`,
remediation: null,
classification: "config_drift",
details,
};
};
/** Whether the nearest existing ancestor is a directory REA may create entries in. */
const nearestDirectoryWritable = async (
path: string,
): Promise<
{ readonly ok: true } | { readonly ok: false; readonly reason: string }
> => {
let candidate = dirname(path);
for (;;) {
try {
if (!(await stat(candidate)).isDirectory())
return { ok: false, reason: `${candidate} is not a directory.` };
await access(candidate, constants.W_OK | constants.X_OK);
return { ok: true };
} catch (cause: unknown) {
const missing =
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
const parent = dirname(candidate);
if (!missing || parent === candidate)
return {
ok: false,
reason: `${candidate} is not writable: ${cause instanceof Error ? cause.message : String(cause)}`,
};
candidate = parent;
}
}
};
+191
View File
@@ -0,0 +1,191 @@
import { constants } from "node:fs";
import { access, open, realpath, stat } from "node:fs/promises";
import { join } from "node:path";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import {
readKunaConfiguration,
type KunaConfiguration,
type KunaConfigurationFailure,
} from "./KunaConfiguration.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** `kuna --version` exits before loading specs or an input; it needs no budget beyond this. */
const VERSION_PROBE_TIMEOUT_MS = 10_000;
const VERSION_PROBE_MAX_OUTPUT_BYTES = 64 * 1024;
/** Release CI prints exactly `kuna <MAJOR.MINOR>`; source builds print the Cargo version. */
const VERSION_LINE = /^kuna (\d+\.\d+(?:\.\d+)?)\r?\n?$/u;
/** One observed Kuna executable, identified by its bytes as well as its version. */
export interface KunaInstallation {
readonly configuration: KunaConfiguration;
/** Canonical path of the probed executable. */
readonly executable: string;
readonly version: string;
/** Whether this release is the one REA verified. */
readonly verified: boolean;
readonly sha256: string;
readonly bytes: number;
/**
* Whether the executable is a `#!` script, such as a package-manager shim.
* Its process then reports the interpreter as its command, so ownership
* checks cannot match the configured path by argv prefix.
*/
readonly script: boolean;
}
/** Why the selected Kuna installation cannot be used. */
export interface KunaReadinessFailure {
readonly code:
| KunaConfigurationFailure["code"]
| "executable_missing"
| "version_unresolved";
readonly reason: string;
readonly diagnostics: Readonly<Record<string, JsonValue>>;
}
/**
* Environment for every Kuna child. Kuna unpacks its built-in SLEIGH tree into
* the per-user cache under HOME (macOS) or XDG_CACHE_HOME (Linux), so both
* point inside REA's own cache root instead of the user's.
*/
export const kunaChildEnvironment = (
configuration: KunaConfiguration,
): NodeJS.ProcessEnv => {
const home = join(configuration.cacheRoot, "home");
return {
HOME: home,
XDG_CACHE_HOME: join(home, ".cache"),
NO_COLOR: "1",
};
};
const failure = (
code: KunaReadinessFailure["code"],
reason: string,
diagnostics: Readonly<Record<string, JsonValue>>,
): Result<never, KunaReadinessFailure> => err({ code, reason, diagnostics });
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Resolve, version-probe and fingerprint the configured executable. */
export const inspectKunaInstallation = async (
environment: Readonly<NodeJS.ProcessEnv>,
options: {
readonly platform?: NodeJS.Platform;
readonly signal?: AbortSignal;
/** Probe deadline; tests shorten it. */
readonly probeTimeoutMs?: number;
} = {},
): Promise<Result<KunaInstallation, KunaReadinessFailure>> => {
const configuration = readKunaConfiguration(environment, options.platform);
if (!configuration.ok)
return failure(configuration.error.code, configuration.error.reason, {
configured: configuration.error.code !== "not_configured",
...(configuration.error.setting === null
? {}
: { setting: configuration.error.setting }),
});
const { command } = configuration.value;
let executable: string;
try {
executable = await realpath(command);
await access(executable, constants.X_OK);
if (!(await stat(executable)).isFile())
return failure(
"executable_missing",
`REA_KUNA_COMMAND is not a regular file: ${command}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot execute REA_KUNA_COMMAND ${command}: ${message(cause)}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
}
const deadline = options.probeTimeoutMs ?? VERSION_PROBE_TIMEOUT_MS;
let stdout: string;
try {
({ stdout } = await execFileOutput(executable, ["--version"], {
env: kunaChildEnvironment(configuration.value),
timeout: deadline,
maxBuffer: VERSION_PROBE_MAX_OUTPUT_BYTES,
// Escalate to SIGKILL so an executable that ignores SIGTERM cannot hang doctor.
stopSignal: "SIGTERM",
...(options.signal === undefined ? {} : { signal: options.signal }),
}));
} catch (cause: unknown) {
options.signal?.throwIfAborted();
const output = execFileOutputFailure(cause);
return failure(
"version_unresolved",
`${executable} --version failed: ${output?.stderr.trim() || message(cause)}. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{
command,
executable,
exit_code: typeof output?.code === "number" ? output.code : null,
signal: output?.signal ?? null,
killed: output?.killed ?? false,
output_truncated: output?.outputTruncated ?? false,
deadline_ms: deadline,
},
);
}
const version = VERSION_LINE.exec(stdout)?.[1];
if (version === undefined)
return failure(
"version_unresolved",
`${executable} --version printed no recognizable Kuna version. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{ command, executable, stdout: stdout.slice(0, 256) },
);
let fingerprint: { readonly sha256: string; readonly bytes: number };
let script: boolean;
try {
fingerprint = await hashStableFile(executable, options.signal);
script = await startsWithShebang(executable);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot fingerprint ${executable}: ${message(cause)}`,
{ command, executable },
);
}
return ok({
configuration: configuration.value,
executable,
version,
verified: version === KUNA_VERIFIED_VERSION,
...fingerprint,
script,
});
};
const startsWithShebang = async (path: string): Promise<boolean> => {
const handle = await open(path, "r");
try {
const prefix = Buffer.alloc(2);
const { bytesRead } = await handle.read(prefix, 0, 2, 0);
return bytesRead === 2 && prefix.toString("latin1") === "#!";
} finally {
await handle.close();
}
};
+690
View File
@@ -0,0 +1,690 @@
import { createHash, randomUUID } from "node:crypto";
import {
mkdir,
mkdtemp,
readdir,
readFile,
rename,
rm,
stat,
utimes,
} from "node:fs/promises";
import { join } from "node:path";
import canonicalize from "canonicalize";
import writeFileAtomic from "write-file-atomic";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import type { KunaInstallation } from "./KunaInstallation.js";
import {
exportKunaProject,
KUNA_EXPORT_OPERATION,
kunaExportFailure,
readKunaExport,
type KunaExport,
type KunaExportStatus,
type KunaProcessLauncher,
} from "./KunaProjectExport.js";
import type {
KunaInventoryFunction,
KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Bumped whenever the entry layout or manifest meaning changes. */
const CACHE_LAYOUT = 1;
/** A manifest names five files and their digests; anything larger is not ours. */
const MANIFEST_MAX_BYTES = 64 * 1024;
const KEY_PATTERN = /^[0-9a-f]{64}$/u;
const SHA256_PATTERN = /^[0-9a-f]{64}$/u;
/** Inputs that determine an export's content. Each one is part of the cache key. */
export interface KunaProjectKeyInputs {
readonly targetSha256: string;
readonly kunaSha256: string;
readonly kunaVersion: string;
/**
* `--jobs 1` is Kuna's reference answer; more jobs decompile without
* callee-first prototypes and change some bodies.
*/
readonly jobs: number;
}
/** Content address of one export: every input that can change its bytes. */
export const kunaProjectKey = (inputs: KunaProjectKeyInputs): string =>
createHash("sha256")
.update(
canonicalize({
layout: CACHE_LAYOUT,
target_sha256: inputs.targetSha256,
kuna_sha256: inputs.kunaSha256,
kuna_version: inputs.kunaVersion,
export: { command: "decompile-project", jobs: inputs.jobs },
}) ?? "",
)
.digest("hex");
const fileSchema = z.strictObject({
path: z.string().min(1),
bytes: z.number().int().nonnegative(),
sha256: z.string().regex(SHA256_PATTERN),
});
const manifestSchema = z.strictObject({
layout: z.literal(CACHE_LAYOUT),
key: z.string().regex(KEY_PATTERN),
inputs: z.strictObject({
target_sha256: z.string().regex(SHA256_PATTERN),
kuna_sha256: z.string().regex(SHA256_PATTERN),
kuna_version: z.string().min(1),
jobs: z.number().int().positive(),
}),
files: z.strictObject({
c: fileSchema,
h: fileSchema,
asm: fileSchema,
readme: fileSchema,
inventory: fileSchema,
}),
summary: z.strictObject({
decompiled: z.number().int().nonnegative(),
failed: z.number().int().nonnegative(),
}),
created_at: z.string().datetime(),
});
/** Immutable description of one published export, written once at publish. */
export type KunaProjectManifest = z.infer<typeof manifestSchema>;
/** One verified, published export. */
export interface KunaProject {
readonly key: string;
/** Directory holding the export's `.c`, `.h`, `.asm`, README and inventory. */
readonly directory: string;
readonly manifest: KunaProjectManifest;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** One request to produce or reuse the export for a target. */
export interface KunaProjectRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; REA decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
readonly signal?: AbortSignal;
readonly onStatus?: (status: KunaExportStatus) => void;
}
interface StagedResult {
readonly result: Result<KunaProject, AnalysisError>;
/** Set when an owned process may still write into staging. */
readonly retainStaging: boolean;
}
interface InflightBuild {
readonly key: string;
readonly promise: Promise<Result<KunaProject, AnalysisError>>;
readonly controller: AbortController;
readonly listeners: Set<(status: KunaExportStatus) => void>;
waiters: number;
}
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** A file an entry's manifest names is gone, replaced or changed under it. */
const isDamaged = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
["ENOENT", "ENOTDIR", "EISDIR", "ENOTFILE", "ESTALE"].includes(
String(cause.code),
);
const isPublishedByAnother = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "EEXIST" || cause.code === "ENOTEMPTY");
const parseManifest = (text: string): KunaProjectManifest | undefined => {
try {
const parsed = manifestSchema.safeParse(JSON.parse(text));
return parsed.success ? parsed.data : undefined;
} catch {
return undefined;
}
};
const manifestBytes = (manifest: KunaProjectManifest): number =>
Object.values(manifest.files).reduce((sum, file) => sum + file.bytes, 0);
/**
* REA-owned on-disk cache of `kuna decompile-project` exports.
*
* Layout under the root: `projects/<key>/` holds published entries
* (`manifest.json` plus the export and inventory in `project/`), `staging/`
* holds builds whose processes may still be live, `trash/` holds retired
* workspaces and evicted entries awaiting removal,
* and `home/` is Kuna's own HOME so its spec cache stays inside REA's
* ownership.
*
* Entries are content-addressed and immutable. A build runs in private
* staging and is published by one directory rename, so a reader sees a
* complete entry or none, and concurrent publishers of one key keep the first.
* File digests are recorded once at publish and checked on every first read.
* Recency is the entry directory's mtime; publishing evicts the least recently
* used entries above the size cap, never an entry pinned by this process.
*/
export class KunaProjectCache {
readonly #root: string;
readonly #maxBytes: number;
readonly #launcher: KunaProcessLauncher | undefined;
/** Builds new requests may join, at most one per key. */
readonly #inflight = new Map<string, InflightBuild>();
/** Every build not yet settled, including abandoned ones a retry replaced. */
readonly #unfinished = new Set<InflightBuild>();
readonly #verified = new Map<string, KunaProject>();
readonly #pins = new Map<string, number>();
/** One eviction decision at a time; exports themselves remain concurrent. */
#eviction: Promise<void> = Promise.resolve();
#capacity: "within_limit" | "pinned" | "building" = "within_limit";
#maintenance: Promise<void> | undefined;
#maintenanceRequested = false;
readonly #retainedProcesses: ProviderCleanupError[] = [];
/** Only resources whose process cleanup is proven may be retried as files. */
readonly #retainedFiles = new Map<string, ProviderCleanupError>();
constructor(
root: string,
options: {
readonly maxBytes: number;
readonly launcher?: KunaProcessLauncher;
},
) {
this.#root = root;
this.#maxBytes = options.maxBytes;
this.#launcher = options.launcher;
}
/** Keep an entry from eviction while this process serves it. */
pin(key: string): () => void {
this.#pins.set(key, (this.#pins.get(key) ?? 0) + 1);
let released = false;
return () => {
if (released) return;
released = true;
const count = (this.#pins.get(key) ?? 1) - 1;
if (count === 0) {
this.#pins.delete(key);
if (this.#capacity !== "within_limit") this.#scheduleEviction();
} else this.#pins.set(key, count);
};
}
/**
* Return the verified published entry for a key, or undefined when none is
* usable. A corrupt or tampered entry is removed and reported as absent.
*/
async lookup(
key: string,
signal?: AbortSignal,
): Promise<KunaProject | undefined> {
const verified = this.#verified.get(key);
if (verified !== undefined) {
// Another process may have evicted it since this one verified it.
if (await this.#touch(key)) return verified;
this.#verified.delete(key);
return undefined;
}
const entry = join(this.#root, "projects", key);
let text: string;
try {
text = await this.#readBounded(join(entry, "manifest.json"));
} catch (cause: unknown) {
if (isMissing(cause)) return undefined;
if (!(cause instanceof RangeError)) throw cause;
await this.#discard(key);
return undefined;
}
const manifest = parseManifest(text);
if (manifest?.key !== key) {
await this.#discard(key);
return undefined;
}
const directory = join(entry, "project");
for (const file of Object.values(manifest.files)) {
let observed: { readonly sha256: string; readonly bytes: number };
try {
observed = await hashStableFile(join(directory, file.path), signal);
} catch (cause: unknown) {
signal?.throwIfAborted();
if (!isDamaged(cause)) throw cause;
await this.#discard(key);
return undefined;
}
if (observed.sha256 !== file.sha256 || observed.bytes !== file.bytes) {
await this.#discard(key);
return undefined;
}
}
const parsed = await readKunaExport(
directory,
manifest.files,
manifest.summary,
signal,
);
if (!parsed.ok) {
await this.#discard(key);
return undefined;
}
const project = { key, directory, manifest, ...parsed.value };
if (!(await this.#touch(key))) return undefined;
this.#verified.set(key, project);
return project;
}
/**
* Return the entry for a request, building and publishing it when absent.
* Concurrent requests for one key share one Kuna run; the run is cancelled
* only when every waiting request has been cancelled.
*/
async build(
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
if (request.signal?.aborted === true)
return err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION));
const key = kunaProjectKey({
targetSha256: request.targetSha256,
kunaSha256: request.installation.sha256,
kunaVersion: request.installation.version,
jobs: request.installation.configuration.jobs,
});
const running = this.#inflight.get(key);
// A build whose waiters all left is winding down; a new request starts afresh.
if (running !== undefined && !running.controller.signal.aborted)
return this.#await(running, request);
const controller = new AbortController();
const listeners = new Set<(status: KunaExportStatus) => void>();
const promise = this.#produce(key, request, controller.signal, (status) => {
for (const listener of listeners) listener(status);
});
const created: InflightBuild = {
key,
promise,
controller,
listeners,
waiters: 0,
};
this.#inflight.set(key, created);
this.#unfinished.add(created);
void promise.then(() => {
this.#unfinished.delete(created);
if (this.#inflight.get(key) === created) this.#inflight.delete(key);
if (this.#unfinished.size === 0 && this.#capacity === "building")
this.#scheduleEviction();
});
return this.#await(created, request);
}
/** Wait for builds, retry safe file removal, and report unresolved cleanup. */
async close(): Promise<Result<null, ProviderCleanupError>> {
await Promise.allSettled(
[...this.#unfinished].map((entry) => entry.promise),
);
while (this.#maintenance !== undefined) await this.#maintenance;
for (const path of [...this.#retainedFiles.keys()])
await this.#removeRetired(path).catch(() => undefined);
const retained = [
...this.#retainedProcesses,
...this.#retainedFiles.values(),
];
if (retained.length === 0) return ok(null);
return err(
new ProviderCleanupError(
KUNA_PROVIDER_ID,
retained.flatMap((entry) => entry.cleanupResources),
{
reason: retained
.map((entry) =>
String(entry.diagnostics?.["reason"] ?? entry.message),
)
.join("; "),
},
{ operation: KUNA_EXPORT_OPERATION },
),
);
}
#await(
entry: InflightBuild,
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
entry.waiters += 1;
const onStatus = request.onStatus;
// Each waiter owns its registration, even when callers reuse a callback.
const listener =
onStatus === undefined
? undefined
: (status: KunaExportStatus) => onStatus(status);
if (listener !== undefined) entry.listeners.add(listener);
return new Promise((resolve) => {
let settled = false;
const leave = () => {
if (listener !== undefined) entry.listeners.delete(listener);
entry.waiters -= 1;
};
const onAbort = () => {
if (settled) return;
settled = true;
leave();
if (entry.waiters === 0) entry.controller.abort();
resolve(err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION)));
};
request.signal?.addEventListener("abort", onAbort, { once: true });
void entry.promise.then((result) => {
request.signal?.removeEventListener("abort", onAbort);
if (settled) return;
settled = true;
leave();
resolve(result);
});
});
}
async #produce(
key: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<Result<KunaProject, AnalysisError>> {
let staging: string;
try {
const existing = await this.lookup(key, signal);
if (existing !== undefined) return ok(existing);
await Promise.all(
["projects", "staging", "trash", "home"].map((name) =>
mkdir(join(this.#root, name), { recursive: true, mode: 0o700 }),
),
);
await this.#sweep(signal);
staging = await mkdtemp(join(this.#root, "staging", "build-"));
} catch (cause: unknown) {
return err(kunaExportFailure(cause, signal));
}
const built = await this.#exportAndPublish(
key,
staging,
request,
signal,
onStatus,
).catch((cause: unknown): StagedResult => ({
result: err(kunaExportFailure(cause, signal)),
retainStaging: false,
}));
if (built.retainStaging) return built.result;
let cleanupPath = staging;
try {
// Only a proven-stopped build may enter trash. A failed write, crash or
// lost cleanup identity therefore never makes live staging reclaimable.
const retired = join(this.#root, "trash", `build-${randomUUID()}`);
await rename(staging, retired);
cleanupPath = retired;
await this.#removeRetired(retired);
} catch (cause: unknown) {
if (cause instanceof ProviderCleanupError) return err(cause);
return err(this.#retainFileCleanup(cleanupPath, cause));
}
if (built.result.ok)
// The entry is published either way; a failed eviction retries on the next publish.
await this.#evict(key).catch(() => undefined);
return built.result;
}
async #exportAndPublish(
key: string,
staging: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<StagedResult> {
const outcome = await exportKunaProject({
installation: request.installation,
targetPath: request.targetPath,
targetSha256: request.targetSha256,
workspace: staging,
output: join(staging, "entry", "project"),
signal,
onStatus,
launcher: this.#launcher,
});
if (outcome.kind === "retained") {
this.#retainedProcesses.push(outcome.error);
return { result: err(outcome.error), retainStaging: true };
}
return {
result: outcome.result.ok
? await this.#publish(
key,
staging,
request,
outcome.result.value,
signal,
)
: outcome.result,
retainStaging: false,
};
}
/** Record the manifest and publish the staged export under its key. */
async #publish(
key: string,
staging: string,
request: KunaProjectRequest,
exported: KunaExport,
signal: AbortSignal,
): Promise<Result<KunaProject, AnalysisError>> {
const manifest: KunaProjectManifest = {
layout: CACHE_LAYOUT,
key,
inputs: {
target_sha256: request.targetSha256,
kuna_sha256: request.installation.sha256,
kuna_version: request.installation.version,
jobs: request.installation.configuration.jobs,
},
files: exported.files,
summary: exported.summary,
created_at: new Date().toISOString(),
};
const total = manifestBytes(manifest);
if (total > this.#maxBytes)
return err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`This export occupies ${String(total)} bytes, more than the whole Kuna cache may hold.`,
{ cache_max_bytes: this.#maxBytes, export_bytes: total },
{
remediationAction:
"Raise REA_KUNA_CACHE_MAX_BYTES above the export size, or analyze this target with another provider.",
},
),
);
await writeFileAtomic(
join(staging, "entry", "manifest.json"),
`${JSON.stringify(manifest, null, 2)}\n`,
{ mode: 0o600 },
);
const destination = join(this.#root, "projects", key);
try {
await rename(join(staging, "entry"), destination);
} catch (cause: unknown) {
if (!isPublishedByAnother(cause)) throw cause;
const winner = await this.lookup(key, signal);
if (winner !== undefined) return ok(winner);
throw cause;
}
const project = {
key,
directory: join(destination, "project"),
manifest,
functions: exported.functions,
inventory: exported.inventory,
};
this.#verified.set(key, project);
return ok(project);
}
/**
* Remove retired resources only. Age and a missing marker cannot establish
* that a process stopped, so unresolved staging is never swept.
*/
async #sweep(signal: AbortSignal): Promise<void> {
for (const name of await readdir(join(this.#root, "trash"))) {
signal.throwIfAborted();
await this.#removeRetired(join(this.#root, "trash", name)).catch(
() => undefined,
);
}
}
/** Evict least recently used entries above the cap, keeping pinned ones. */
#evict(keep: string | null): Promise<void> {
const eviction = this.#eviction.then(() => this.#evictUnpinned(keep));
this.#eviction = eviction.catch(() => undefined);
return eviction;
}
/** Coalesce deferred reclamation after callers have had a turn to pin results. */
#scheduleEviction(): void {
this.#maintenanceRequested = true;
if (this.#maintenance !== undefined) return;
const maintenance = (async () => {
while (this.#maintenanceRequested) {
await new Promise<void>((resolve) => setImmediate(resolve));
this.#maintenanceRequested = false;
await this.#evict(null).catch(() => undefined);
}
})();
this.#maintenance = maintenance;
void maintenance.then(() => {
if (this.#maintenance === maintenance) this.#maintenance = undefined;
if (this.#maintenanceRequested) this.#scheduleEviction();
});
}
async #evictUnpinned(keep: string | null): Promise<void> {
// A successful build must reach its callers before they can pin its entry.
// Protect every unfinished build while reading this eviction snapshot.
const building = new Set([...this.#unfinished].map(({ key }) => key));
const entries: { key: string; bytes: number; mtimeMs: number }[] = [];
for (const name of await readdir(join(this.#root, "projects"))) {
if (!KEY_PATTERN.test(name)) continue;
const path = join(this.#root, "projects", name);
const status = await stat(path).catch(() => undefined);
if (status === undefined) continue;
const manifest = await this.#readBounded(join(path, "manifest.json"))
.then(parseManifest)
.catch(() => undefined);
// An owned entry without a valid manifest is unusable; evict it first.
entries.push(
manifest === undefined
? { key: name, bytes: 0, mtimeMs: -1 }
: {
key: name,
bytes: manifestBytes(manifest),
mtimeMs: status.mtimeMs,
},
);
}
let total = entries.reduce((sum, entry) => sum + entry.bytes, 0);
let deferredByBuild = false;
for (const entry of entries.sort((a, b) => a.mtimeMs - b.mtimeMs)) {
if (total <= this.#maxBytes && entry.mtimeMs >= 0) break;
if (building.has(entry.key) || this.#inflight.has(entry.key)) {
deferredByBuild = true;
continue;
}
if (entry.key === keep || this.#pins.has(entry.key)) continue;
await this.#discard(entry.key);
total -= entry.bytes;
}
this.#capacity =
total <= this.#maxBytes
? "within_limit"
: deferredByBuild
? "building"
: "pinned";
}
/** Atomically retire one owned entry, then remove it. */
async #discard(key: string): Promise<void> {
this.#verified.delete(key);
const trash = join(this.#root, "trash", `${key}-${randomUUID()}`);
try {
await mkdir(join(this.#root, "trash"), { recursive: true, mode: 0o700 });
await rename(join(this.#root, "projects", key), trash);
} catch (cause: unknown) {
if (isMissing(cause)) return;
throw cause;
}
await this.#removeRetired(trash);
}
/** Keep each failed removal owned until that path is successfully removed. */
async #removeRetired(path: string): Promise<void> {
try {
await rm(path, { recursive: true, force: true });
this.#retainedFiles.delete(path);
} catch (cause: unknown) {
throw this.#retainFileCleanup(path, cause);
}
}
/** Mark an entry recently used; false when it is no longer published. */
async #touch(key: string): Promise<boolean> {
const now = new Date();
try {
await utimes(join(this.#root, "projects", key), now, now);
return true;
} catch (cause: unknown) {
// Recency is advisory; only a missing entry changes the answer.
return !isMissing(cause);
}
}
async #readBounded(path: string): Promise<string> {
const status = await stat(path);
if (status.size > MANIFEST_MAX_BYTES)
throw new RangeError(
`${path} exceeds ${String(MANIFEST_MAX_BYTES)} bytes`,
);
return readFile(path, "utf8");
}
#retainFileCleanup(path: string, cause: unknown): ProviderCleanupError {
const error = new ProviderCleanupError(
KUNA_PROVIDER_ID,
[path],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation: KUNA_EXPORT_OPERATION, cause },
);
this.#retainedFiles.set(path, error);
return error;
}
}
+466
View File
@@ -0,0 +1,466 @@
import { createHash, randomUUID } from "node:crypto";
import { createReadStream, createWriteStream } from "node:fs";
import { lstat, mkdir, readFile, writeFile } from "node:fs/promises";
import { basename, join } from "node:path";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisArtifactChangedError,
AnalysisCancelledError,
AnalysisResourceConstraintError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js";
import type {
OwnedProviderProcessSpawnOptions,
ProviderProcessSnapshot,
SpawnedOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import {
kunaChildEnvironment,
type KunaInstallation,
} from "./KunaInstallation.js";
import {
parseKunaExportSummary,
parseKunaInventory,
scanKunaProjectSource,
type KunaExportSummary,
type KunaInventoryFunction,
type KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Operation name every export and cache failure reports. */
export const KUNA_EXPORT_OPERATION = "kuna:decompile-project";
/** Kuna prints one summary line per export; notes on a large image stay far below this. */
const DIAGNOSTIC_BYTES = 8 * 1024 * 1024;
/**
* `kuna functions --json` costs ~210 bytes per function; this admits about
* 600,000 functions, above the 392,814 of the largest image Kuna documents.
*/
const INVENTORY_BYTES = 128 * 1024 * 1024;
const INVENTORY_FILE = "functions.json";
/** Identity of one file an export consists of. */
export interface KunaExportFile {
readonly path: string;
readonly bytes: number;
readonly sha256: string;
}
/** The files of one export, by role. */
export interface KunaExportFiles {
readonly c: KunaExportFile;
readonly h: KunaExportFile;
readonly asm: KunaExportFile;
readonly readme: KunaExportFile;
readonly inventory: KunaExportFile;
}
/** One validated export: its files, Kuna's counts and the facts parsed from them. */
export interface KunaExport {
readonly files: KunaExportFiles;
readonly summary: KunaExportSummary;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** The step an export is in; Kuna itself reports no progress until it finishes. */
export interface KunaExportStatus {
readonly phase: "inventory" | "decompiling" | "sealing";
/** Callable entries found by the inventory, once known. */
readonly inventoryFunctions: number | null;
}
/** Test seam for the owned process launcher; production uses the default. */
export type KunaProcessLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/**
* How an export ended. `retained` means an owned Kuna process could not be
* proven stopped, so it may still write into the workspace, which therefore
* stays with the caller's cleanup owner.
*/
export type KunaExportOutcome =
| {
readonly kind: "done";
readonly result: Result<KunaExport, AnalysisError>;
}
| { readonly kind: "retained"; readonly error: ProviderCleanupError };
/** One export of a target into a private workspace. */
export interface KunaExportRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; Kuna decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
/** Private directory for the copy, Kuna's temporary files and its output. */
readonly workspace: string;
/** Directory, inside the workspace, that receives the export's files. */
readonly output: string;
readonly signal: AbortSignal;
readonly onStatus: (status: KunaExportStatus) => void;
readonly launcher?: KunaProcessLauncher | undefined;
}
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** Map an unexpected failure to a typed one, preferring cancellation. */
export const kunaExportFailure = (
cause: unknown,
signal: AbortSignal,
): AnalysisError => {
if (signal.aborted)
return new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause });
if (cause instanceof AnalysisError) return cause;
return new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Kuna project cache failed: ${message(cause)}`,
});
};
const unusable = (reason: string): KunaExportOutcome => ({
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `Kuna's project export is not usable: ${reason}`,
}),
),
});
/** Verify the selected installation at each execution and publication boundary. */
const verifyKunaExecutable = async (
installation: KunaInstallation,
signal: AbortSignal,
): Promise<Result<null, ProviderAdapterError>> => {
const executable = await hashStableFile(installation.executable, signal);
if (executable.sha256 === installation.sha256) return ok(null);
return err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `${installation.executable} changed after readiness inspection; reopen the target so REA probes the current Kuna executable.`,
diagnostics: {
expected_sha256: installation.sha256,
observed_sha256: executable.sha256,
},
}),
);
};
/**
* Inventory and export one target with Kuna, then validate the result.
*
* Kuna runs on a copy hashed while copying, so the export always describes
* the digest the caller selected. Both Kuna runs share one deadline.
*/
export const exportKunaProject = async (
request: KunaExportRequest,
): Promise<KunaExportOutcome> => {
const { installation, signal } = request;
const { configuration } = installation;
const deadline = Date.now() + configuration.projectTimeoutMs;
const input = join(request.workspace, "input", basename(request.targetPath));
await mkdir(join(request.workspace, "input"), { mode: 0o700 });
await mkdir(request.output, { recursive: true, mode: 0o700 });
const copied = await copyAndHash(request.targetPath, input, signal);
if (copied !== request.targetSha256)
return {
kind: "done",
result: err(
new AnalysisArtifactChangedError(
KUNA_EXPORT_OPERATION,
request.targetPath,
"the target no longer matches the digest selected for analysis",
),
),
};
await prepareProcessOwnershipInspection(signal);
const run = (arguments_: readonly string[], diagnosticBytes: number) =>
runKuna(request, arguments_, { deadline, diagnosticBytes });
request.onStatus({ phase: "inventory", inventoryFunctions: null });
const listed = await run(["functions", input, "--json"], INVENTORY_BYTES);
if (!listed.ok) return listed.outcome;
const inventory = parseKunaInventory(listed.snapshot.stdout.text);
if (!inventory.ok) return unusable(inventory.error);
if (listed.snapshot.stdout.invalidUtf8 === true)
return unusable("the function inventory is not valid UTF-8");
await writeFile(
join(request.output, INVENTORY_FILE),
listed.snapshot.stdout.text,
{ mode: 0o600 },
);
request.onStatus({
phase: "decompiling",
inventoryFunctions: inventory.value.length,
});
const exported = await run(
[
"decompile-project",
input,
"-o",
request.output,
"--jobs",
String(configuration.jobs),
],
DIAGNOSTIC_BYTES,
);
if (!exported.ok) return exported.outcome;
request.onStatus({
phase: "sealing",
inventoryFunctions: inventory.value.length,
});
const summary = parseKunaExportSummary(exported.snapshot.stdout.text);
if (summary === undefined)
return unusable(
"Kuna did not print its `functions: N ok, M failed` summary.",
);
const stem = basename(request.targetPath);
const names = {
c: `${stem}.c`,
h: `${stem}.h`,
asm: `${stem}.asm`,
readme: "README.md",
inventory: INVENTORY_FILE,
} as const;
const files: Partial<Record<keyof KunaExportFiles, KunaExportFile>> = {};
for (const [role, name] of Object.entries(names) as [
keyof KunaExportFiles,
string,
][]) {
const path = join(request.output, name);
const status = await lstat(path).catch((cause: unknown) => {
if (isMissing(cause)) return undefined;
throw cause;
});
if (status === undefined || !status.isFile())
return unusable(`Kuna's export has no regular ${name}.`);
files[role] = { path: name, ...(await hashStableFile(path, signal)) };
}
const complete = files as KunaExportFiles;
const parsed = await readKunaExport(
request.output,
complete,
summary,
signal,
);
if (!parsed.ok) return unusable(parsed.error);
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok) return { kind: "done", result: identity };
return {
kind: "done",
result: ok({ files: complete, summary, ...parsed.value }),
};
};
/** Parse an export's function blocks and inventory from its own files. */
export const readKunaExport = async (
directory: string,
files: KunaExportFiles,
summary: KunaExportSummary,
signal?: AbortSignal,
): Promise<
Result<
{
readonly functions: readonly KunaProjectFunction[];
readonly inventory: readonly KunaInventoryFunction[];
},
string
>
> => {
const functions = await scanKunaProjectSource(
createReadStream(
join(directory, files.c.path),
signal === undefined ? {} : { signal },
),
summary,
);
if (!functions.ok) return functions;
const inventory = parseKunaInventory(
await readFile(join(directory, files.inventory.path), {
encoding: "utf8",
...(signal === undefined ? {} : { signal }),
}),
);
if (!inventory.ok) return inventory;
return ok({ functions: functions.value, inventory: inventory.value });
};
const runKuna = async (
request: KunaExportRequest,
arguments_: readonly string[],
limits: { readonly deadline: number; readonly diagnosticBytes: number },
): Promise<
| { readonly ok: true; readonly snapshot: ProviderProcessSnapshot }
| { readonly ok: false; readonly outcome: KunaExportOutcome }
> => {
const { installation, signal } = request;
try {
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok)
return { ok: false, outcome: { kind: "done", result: identity } };
const snapshot = await runOwnedCommand(
{
command: installation.executable,
arguments: arguments_,
runId: `rea-kuna-${randomUUID()}`,
// A script's process reports its interpreter, not the configured path.
...(installation.script ? { expectedCommand: null } : {}),
cwd: request.workspace,
hostEnvironment: kunaChildEnvironment(installation.configuration),
env: { TMPDIR: request.workspace },
},
{
timeoutMs: Math.max(1, limits.deadline - Date.now()),
diagnosticBytes: limits.diagnosticBytes,
},
{
signal,
...(request.launcher === undefined
? {}
: { launcher: request.launcher }),
},
);
return { ok: true, snapshot };
} catch (cause: unknown) {
return {
ok: false,
outcome: commandFailure(cause, request, arguments_[0] ?? "", limits),
};
}
};
/** Copy the target into the workspace and return the SHA-256 of exactly the bytes copied. */
const copyAndHash = async (
source: string,
destination: string,
signal: AbortSignal,
): Promise<string> => {
const hash = createHash("sha256");
await pipeline(
createReadStream(source),
new Transform({
transform(chunk: Buffer, _encoding, callback) {
hash.update(chunk);
callback(null, chunk);
},
}),
createWriteStream(destination, { flags: "wx", mode: 0o600 }),
{ signal },
);
return hash.digest("hex");
};
/** Map an owned-command failure to its typed error, keeping unproven cleanup owned. */
const commandFailure = (
cause: unknown,
request: KunaExportRequest,
subcommand: string,
limits: { readonly diagnosticBytes: number },
): KunaExportOutcome => {
const { installation, signal } = request;
if (!(cause instanceof OwnedCommandFailure))
return {
kind: "done",
result: err(
signal.aborted
? new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause })
: new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Cannot run ${installation.executable}: ${message(cause)}`,
}),
),
};
const snapshot = cause.snapshot;
const capturedOutput =
snapshot === null
? undefined
: {
// An inventory is evidence-sized, not diagnostic-sized; keep only its tail.
stdout: snapshot.stdout.text.slice(-4096),
stderr: snapshot.stderr.text,
truncated:
snapshot.diagnosticTruncated === true ||
snapshot.outputIncomplete === true ||
snapshot.stdout.text.length > 4096,
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
};
const details = {
cause,
...(capturedOutput === undefined ? {} : { capturedOutput }),
};
if (cause.cleanupFailure !== null)
return {
kind: "retained",
error: new ProviderCleanupError(
KUNA_PROVIDER_ID,
[...cause.resources, request.workspace],
{ reason: cause.cleanupFailure },
{ operation: KUNA_EXPORT_OPERATION, cause },
),
};
if (cause.reason === "cancelled")
return {
kind: "done",
result: err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION, details)),
};
if (cause.reason === "timeout")
return {
kind: "done",
result: err(
new AnalysisTimeoutError(
KUNA_EXPORT_OPERATION,
installation.configuration.projectTimeoutMs,
details,
),
),
};
if (cause.reason === "output-limit")
return {
kind: "done",
result: err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`kuna ${subcommand} wrote more than ${String(limits.diagnosticBytes)} bytes of output; the export was not retained.`,
{ output_limit_bytes: limits.diagnosticBytes },
details,
),
),
};
const exitCode = capturedOutput?.exit_code;
return {
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
...details,
userMessage: `kuna ${subcommand} failed${exitCode === undefined || exitCode === null ? "" : ` with exit code ${String(exitCode)}`}: ${capturedOutput?.stderr.trim().split("\n").at(-1) ?? cause.message}`,
}),
),
};
};
+189
View File
@@ -0,0 +1,189 @@
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
/**
* One function block in a `kuna decompile-project` `.c` file. The span covers
* its `// Function:` header through the blank line that ends the block, which
* is how REA serves one function without reading the whole file.
*/
export interface KunaProjectFunction {
readonly name: string;
/** Kuna's display address, a lowercase `0x…` virtual address. */
readonly address: string;
readonly cOffset: number;
readonly cLength: number;
/** Kuna's own failure for this function; its block is then the header alone. */
readonly error: string | null;
}
// Rendered by kuna-console's `render_c`: `// Function: {name} @ 0x{addr:x}`,
// or `... @ 0x{addr:x} (error: {message})` for a failed function. A name
// can itself contain " @ ", so the address is the last such field.
const HEADER = /^\/\/ Function: (.+) @ (0x[0-9a-f]+)(?: {2}\(error: (.*)\))?$/u;
const HEADER_PREFIX = "// Function: ";
/** Kuna's completed-export summary on stdout: `functions: N ok, M failed`. */
const SUMMARY = /functions: (\d+) ok, (\d+) failed/u;
/** Function counts Kuna reported for a finished export. */
export interface KunaExportSummary {
readonly decompiled: number;
readonly failed: number;
}
/** Read the completed-file summary Kuna prints after finalizing an export. */
export const parseKunaExportSummary = (
stdout: string,
): KunaExportSummary | undefined => {
const match = SUMMARY.exec(stdout);
if (match?.[1] === undefined || match[2] === undefined) return undefined;
return { decompiled: Number(match[1]), failed: Number(match[2]) };
};
/** A header line longer than this is not one Kuna wrote. */
const MAX_HEADER_BYTES = 64 * 1024;
const PREFIX_BYTES = Buffer.from(HEADER_PREFIX);
interface Header {
readonly name: string;
readonly address: string;
readonly error: string | null;
readonly offset: number;
}
/**
* Split a finished export's `.c` into its function blocks, reading it as a
* stream so a very large export is never held in memory.
*
* The `#include` preamble precedes the first header. The header count must
* equal the functions Kuna reported, and every address must be distinct; an
* export that fails either check is not a usable function inventory.
*/
export const scanKunaProjectSource = async (
chunks: AsyncIterable<Buffer>,
summary: KunaExportSummary,
): Promise<Result<readonly KunaProjectFunction[], string>> => {
const headers: Header[] = [];
let offset = 0;
let lineStart = 0;
// Bytes of the current line, kept only while it may still be a header.
let line: Buffer[] | undefined = [];
let lineBytes = 0;
const endLine = (): string | undefined => {
if (line === undefined) return undefined;
const text = Buffer.concat(line).toString("utf8");
if (!text.startsWith(HEADER_PREFIX)) return undefined;
const match = HEADER.exec(text);
if (match?.[1] === undefined || match[2] === undefined)
return `unrecognized function header at byte ${String(lineStart)}`;
headers.push({
name: match[1],
address: match[2],
error: match[3] ?? null,
offset: lineStart,
});
return undefined;
};
for await (const chunk of chunks) {
let position = 0;
while (position < chunk.length) {
const newline = chunk.indexOf(0x0a, position);
const end = newline === -1 ? chunk.length : newline;
if (line !== undefined) {
const segment = chunk.subarray(position, end);
line.push(segment);
lineBytes += segment.length;
const seen = Buffer.concat(line).subarray(0, PREFIX_BYTES.length);
if (!PREFIX_BYTES.subarray(0, seen.length).equals(seen))
line = undefined;
else if (lineBytes > MAX_HEADER_BYTES)
return err(
`function header at byte ${String(lineStart)} is too long`,
);
}
if (newline === -1) break;
const failure = endLine();
if (failure !== undefined) return err(failure);
position = newline + 1;
lineStart = offset + position;
line = [];
lineBytes = 0;
}
offset += chunk.length;
}
const failure = endLine();
if (failure !== undefined) return err(failure);
const total = offset;
const expected = summary.decompiled + summary.failed;
if (headers.length !== expected)
return err(
`the .c holds ${String(headers.length)} function headers but Kuna reported ${String(expected)} functions`,
);
const failed = headers.filter(({ error }) => error !== null).length;
if (failed !== summary.failed)
return err(
`the .c marks ${String(failed)} failed functions but Kuna reported ${String(summary.failed)}`,
);
const addresses = new Set<string>();
const functions: KunaProjectFunction[] = [];
for (const [index, header] of headers.entries()) {
if (addresses.has(header.address))
return err(`the .c repeats address ${header.address}`);
addresses.add(header.address);
const end = headers[index + 1]?.offset ?? total;
functions.push({
name: header.name,
address: header.address,
cOffset: header.offset,
cLength: end - header.offset,
error: header.error,
});
}
return ok(functions);
};
const inventorySchema = z.object({
functions: z.array(
z.object({
name: z.string(),
address_hex: z.string().regex(/^0x[0-9a-f]+$/u),
aliases: z.array(z.string()),
size: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
}),
),
});
/** One callable entry from `kuna functions --json`, including imports Kuna does not decompile. */
export interface KunaInventoryFunction {
readonly name: string;
readonly address: string;
readonly aliases: readonly string[];
/** Inventory extent: an upper bound clipped at the next entry, or 0 when unmeasured. */
readonly size: number;
}
/** Parse the callable-symbol inventory `kuna functions --json` writes. */
export const parseKunaInventory = (
text: string,
): Result<readonly KunaInventoryFunction[], string> => {
let value: unknown;
try {
value = JSON.parse(text);
} catch {
return err("the function inventory is not JSON");
}
const parsed = inventorySchema.safeParse(value);
if (!parsed.success)
return err(
`the function inventory does not match Kuna's document: ${parsed.error.issues[0]?.message ?? "invalid"}`,
);
return ok(
parsed.data.functions.map((entry) => ({
name: entry.name,
address: entry.address_hex,
aliases: entry.aliases,
size: entry.size,
})),
);
};
+16
View File
@@ -0,0 +1,16 @@
/** Kuna release whose CLI and project layout REA verified; others report as unverified. */
export const KUNA_VERIFIED_VERSION = "1.833";
/** Stable provider ID shared by readiness, doctor and Evidence. */
export const KUNA_PROVIDER_ID = "kuna";
/** Remediation when `REA_KUNA_COMMAND` is missing or not an absolute path. */
export const KUNA_COMMAND_REMEDIATION = `Set REA_KUNA_COMMAND to the absolute path of a caller-supplied kuna executable (verified with ${KUNA_VERIFIED_VERSION}). REA does not download, install or update Kuna.`;
/** Remediation when the host cannot supervise a Kuna process tree. */
export const KUNA_UNSUPPORTED_HOST_REMEDIATION =
"Use Linux or macOS for the Kuna provider. Windows process-tree ownership for Kuna is not verified.";
/** Remediation when `kuna --version` cannot establish the installed release. */
export const KUNA_VERSION_PROBE_REMEDIATION =
"Run the selected executable with --version: it must exit promptly and print `kuna <major>.<minor>`.";
@@ -339,6 +339,38 @@ it.skipIf(process.platform === "win32")(
},
);
it.skipIf(process.platform === "win32")(
"keeps the observed symlink target when dot segments resolve to the same path",
async () => {
const root = await createTestTempDirectory("rea-fs-symlink-spelling-");
await mkdir(join(root, "aaa"));
await writeFile(join(root, "target"), "content");
// Equal length, so only the target spelling distinguishes the links;
// "bbb" does not exist, so the replacement link is broken.
await symlink("aaa/../target", join(root, "link"));
const scenario = parseProcessScenario({
executable: process.execPath,
working_directory: root,
filesystem_observation_paths: [root],
limits: { file_bytes: 1 },
});
const before = await snapshotRoots(scenario);
await rm(join(root, "link"));
await symlink("bbb/../target", join(root, "link"));
const after = await snapshotRoots(scenario);
const link = (snapshot: typeof before) =>
snapshot.files.find(({ path }) => path === "root_0:link");
expect(link(before)?.symlink_target).toBe("aaa/../target");
expect(link(after)?.symlink_target).toBe("bbb/../target");
expect(
classifyFilesystemEffects(before, after).find(
({ path }) => path === "root_0:link",
)?.status,
).toBe("modified");
},
);
it("preserves creation when the selected root was observed missing before capture", async () => {
const root = await createTestTempDirectory("rea-fs-missing-root-");
const file = join(root, "created.txt");
+3 -2
View File
@@ -1,7 +1,7 @@
import { createHash } from "node:crypto";
import { constants as fsConstants } from "node:fs";
import { lstat, open, opendir, readlink } from "node:fs/promises";
import { dirname, join, relative, resolve } from "node:path";
import { join, relative } from "node:path";
import type { FileHandle } from "node:fs/promises";
import type { ProcessScenario } from "../../domain/process/processScenario.js";
import type { FilesystemCoverage } from "../../domain/process/processCaptureCoverage.js";
@@ -224,7 +224,8 @@ export const snapshotRoots = async (
if (stats === undefined) return true;
const relativePath = relative(root, path) || ".";
if (stats.isSymbolicLink()) {
const target = resolve(dirname(path), await readlink(path));
// Keep the target as stored: lexical resolution can merge distinct targets.
const target = await readlink(path);
const afterRead = await lstat(path);
if (!hasSameIdentity(stats, afterRead, "symlink")) {
truncated = true;
+13 -13
View File
@@ -12,7 +12,6 @@ import { captureProcessScenario } from "../process/capture/ProcessHarness.js";
import { ProcessCaptureResourceScope } from "../process/capture/ProcessCaptureLifecycle.js";
import { ProcessCaptureError } from "../process/capture/ProcessCaptureError.js";
import { toolContract } from "../contracts/toolContracts.js";
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import { binaryTargetSummary } from "../application/binary/BinaryTargetSummary.js";
import { UnknownRegistryError } from "../domain/unknownRegistryError.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
@@ -178,17 +177,7 @@ const registerOpenLifecycleTool = (
openContract.name,
toolRegistrationOptions(openContract),
async (input, context) => {
let snapshot: AnalysisSnapshot | undefined;
if (input.snapshot_path !== undefined) {
const loaded = await readAnalysisSnapshot(
input.snapshot_path,
context.mcpReq.signal,
artifactResources,
);
if (!loaded.ok)
return server.delivery.toCallToolResult(loaded, openContract);
snapshot = loaded.value;
}
const snapshotPath = input.snapshot_path;
const opened = await logToolExecution(logger, openContract.name, () =>
session.open(input.path, {
signal: context.mcpReq.signal,
@@ -202,7 +191,18 @@ const registerOpenLifecycleTool = (
...(input.provider_id === undefined
? {}
: { providerId: input.provider_id }),
...(snapshot === undefined ? {} : { snapshot }),
...(snapshotPath === undefined
? {}
: {
// Read inside the open's turn: calls issued after it must not
// overtake a file read that precedes its place in the order.
loadSnapshot: () =>
readAnalysisSnapshot(
snapshotPath,
context.mcpReq.signal,
artifactResources,
),
}),
}),
);
return opened.ok
@@ -348,6 +348,63 @@ describe("CdpElectronProvider malformed source replies", () => {
);
});
describe("CdpElectronProvider malformed inventory replies", () => {
const inspectWith = async (
method: string,
reply: (fileUrl: string) => Readonly<Record<string, unknown>>,
) => {
const root = await electronFixture();
const fileUrl = pathToFileURL(join(root, "index.html")).href;
const browser = await startFakeCdpBrowser({
electronFileUrl: fileUrl,
commandResult: (command) =>
command.method === method ? reply(fileUrl) : undefined,
});
browsers.push(browser);
const result = await new CdpElectronProvider().inspectPage(
inspectElectronPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "electron-page",
observation_ms: 0,
}),
);
if (!result.ok) throw result.error;
return result.value;
};
it.each([
["dom", "DOMSnapshot.captureSnapshot", () => ({ strings: [] })],
[
"dom",
"DOMSnapshot.captureSnapshot",
(fileUrl: string) => ({
strings: [fileUrl],
documents: [{ documentURL: 0, nodes: { nodeType: null } }],
}),
],
[
"resources",
"Page.getResourceTree",
(fileUrl: string) => ({
frameTree: { frame: { id: "main", url: fileUrl }, resources: null },
}),
],
["workers", "Target.getTargets", () => ({ targetInfos: null })],
] as const)(
"keeps a malformed %s inventory incomplete (%s)",
async (section, method, reply) => {
const inspected = await inspectWith(method, reply);
expect(inspected.completeness.status).not.toBe("complete_within_window");
expect(inspected.completeness.excluded).toContainEqual(
expect.objectContaining({
section,
reason: "invalid_protocol_value",
}),
);
},
);
});
describe("CdpElectronProvider script source capture", () => {
it("retains an explicitly empty Electron producer script", async () => {
const root = await electronFixture();
@@ -0,0 +1,277 @@
import { createServer, type Socket } from "node:net";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
import { CdpConnection } from "../../../src/browser/CdpConnection.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
import { WEB_RUNTIME_LIMITS } from "../../../src/domain/webRuntime.js";
import { V8InspectorProvider } from "../../../src/inspector/V8InspectorProvider.js";
import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
import { startFakeV8Inspector } from "../../fixtures/inspector/fakeV8Inspector.js";
// Keep real socket scheduling; expire only a timer whose owning phase is ready.
beforeEach(() => {
vi.spyOn(globalThis, "setTimeout");
vi.spyOn(globalThis, "clearTimeout");
});
afterEach(() => vi.restoreAllMocks());
const activeTimers = (milliseconds: number) => {
const scheduled = vi.mocked(globalThis.setTimeout);
const cleared = vi.mocked(globalThis.clearTimeout);
return scheduled.mock.calls.flatMap(([callback, delay], index) => {
const result = scheduled.mock.results[index];
if (
delay !== milliseconds ||
result?.type !== "return" ||
cleared.mock.calls.some(([timer]) => timer === result.value)
)
return [];
return [{ callback, timer: result.value }];
});
};
const expireTimer = (milliseconds: number): void => {
const timers = activeTimers(milliseconds);
expect(timers).toHaveLength(1);
const timer = timers[0];
if (timer === undefined) throw new Error("Expected one active timer");
try {
timer.callback();
if (milliseconds === WEB_RUNTIME_LIMITS.commandTimeoutMs)
expect(globalThis.clearTimeout).toHaveBeenCalledWith(timer.timer);
} finally {
// The observation callback resolves naturally without cancelling its timer.
clearTimeout(timer.timer);
}
};
it("bounds a stalled WebSocket handshake and releases its socket", async () => {
const sockets = new Set<Socket>();
const server = createServer((socket) => {
sockets.add(socket);
socket.on("close", () => sockets.delete(socket));
socket.resume();
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("Expected a TCP address");
try {
const rejection = expect(
CdpConnection.connect(
`ws://127.0.0.1:${String(address.port)}/target`,
"inspect_web_page",
AbortSignal.timeout(10_000),
),
).rejects.toMatchObject({ reason: "timeout" });
await vi.waitFor(() => expect(sockets.size).toBe(1), { timeout: 5_000 });
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
await rejection;
await vi.waitFor(() => expect(sockets.size).toBe(0));
} finally {
for (const socket of sockets) socket.destroy();
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
it.each(["Target.attachToTarget", "Page.enable", "Runtime.enable"])(
"times out %s after HTTP discovery and closes the target session",
async (hangOnMethod) => {
const browser = await startFakeCdpBrowser({ hangOnMethod });
try {
const pending = new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "allowed-page",
observation_ms: 0,
}),
{ signal: AbortSignal.timeout(10_000) },
);
await vi.waitFor(
() =>
expect(browser.commands.map(({ method }) => method)).toContain(
hangOnMethod,
),
{ timeout: 5_000 },
);
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
const result = await pending;
if (result.ok) throw new Error("Expected a target command timeout");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "provider_timeout",
category: "timeout",
});
expect(browser.commands.map(({ method }) => method)).toContain(
hangOnMethod,
);
await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0));
} finally {
await browser.close();
}
},
);
it("ignores a late timed-out reply and keeps following commands correlated", async () => {
const browser = await startFakeCdpBrowser({ hangOnMethod: "Page.enable" });
const connection = await CdpConnection.connect(
browser.browserWebSocketUrl,
"inspect_web_page",
);
try {
const rejection = expect(
connection.send(
"Page.enable",
{},
undefined,
AbortSignal.timeout(10_000),
),
).rejects.toMatchObject({ reason: "timeout" });
await vi.waitFor(() => expect(browser.commands).toHaveLength(1), {
timeout: 5_000,
});
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
await rejection;
const command = browser.commands[0];
if (command === undefined)
throw new Error("Expected the timed-out command");
browser.emitRawMessage(JSON.stringify({ id: command.id, result: {} }));
await expect(connection.send("Runtime.enable")).resolves.toEqual({});
} finally {
await connection.close();
await browser.close();
}
});
it.each(["success", "rejection", "abort", "disconnect"])(
"releases handshake and command timers on %s",
async (outcome) => {
const cleared = vi.mocked(globalThis.clearTimeout);
const browser = await startFakeCdpBrowser({
...(outcome === "abort" || outcome === "success"
? { hangOnMethod: "Runtime.enable" }
: {}),
...(outcome === "disconnect" ? { closeOnMethod: "Runtime.enable" } : {}),
});
const controller = new AbortController();
const removeAbort = vi.spyOn(controller.signal, "removeEventListener");
const connection = await CdpConnection.connect(
browser.browserWebSocketUrl,
"inspect_web_page",
controller.signal,
);
try {
const pending = connection.send(
outcome === "rejection" ? "Fixture.unmodeledMethod" : "Runtime.enable",
{},
undefined,
controller.signal,
);
const assertion =
outcome === "success"
? expect(pending).resolves.toEqual({})
: expect(pending).rejects.toMatchObject({
_tag:
outcome === "abort"
? "AnalysisCancelledError"
: "BrowserObservationError",
});
if (outcome === "success") {
await vi.waitFor(() => expect(browser.commands).toHaveLength(1), {
timeout: 5_000,
});
// A healthy reply may take longer than the former 100 ms test budget.
await new Promise<void>((resolve) => setTimeout(resolve, 250));
const command = browser.commands[0];
if (command === undefined)
throw new Error("Expected a pending command");
browser.emitRawMessage(JSON.stringify({ id: command.id, result: {} }));
}
if (outcome === "abort") controller.abort();
await assertion;
const scheduled = vi.mocked(globalThis.setTimeout);
const deadlines = scheduled.mock.calls.flatMap((call, index) =>
call[1] === WEB_RUNTIME_LIMITS.commandTimeoutMs
? [scheduled.mock.results[index]?.value]
: [],
);
expect(deadlines).toHaveLength(2);
for (const timer of deadlines)
expect(cleared).toHaveBeenCalledWith(timer);
expect(removeAbort).toHaveBeenCalledTimes(2);
} finally {
await connection.close();
await browser.close();
}
},
);
it.each(["Runtime.enable", "Debugger.enable"])(
"times out Inspector %s and preserves observations received before expiry",
async (hangOnMethod) => {
const inspector = await startFakeV8Inspector({
targetUrl: "https://fixture.example/app.js",
hangOnMethod,
});
try {
const pending = new V8InspectorProvider().observe(
{
inspector_endpoint: inspector.endpoint,
target_id: inspector.targetId,
observation_ms: 0,
},
{ signal: AbortSignal.timeout(10_000) },
);
await vi.waitFor(
() =>
expect(inspector.commands.map(({ method }) => method)).toContain(
hangOnMethod,
),
{ timeout: 5_000 },
);
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
const result = await pending;
if (result.ok) throw new Error("Expected an Inspector command timeout");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "provider_timeout",
category: "timeout",
});
expect(result.error.partialObservation).toMatchObject({
capture: {
events_observed: hangOnMethod === "Debugger.enable" ? 1 : 0,
},
});
} finally {
await inspector.close();
}
},
);
it("keeps the chosen observation window independent of command deadlines", async () => {
const browser = await startFakeCdpBrowser();
const observationMs = WEB_RUNTIME_LIMITS.commandTimeoutMs + 250;
try {
const pending = new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "allowed-page",
observation_ms: observationMs,
}),
);
await vi.waitFor(
() => expect(activeTimers(observationMs)).toHaveLength(1),
{
timeout: 5_000,
},
);
expect(activeTimers(WEB_RUNTIME_LIMITS.commandTimeoutMs)).toHaveLength(0);
expireTimer(observationMs);
const result = await pending;
expect(result.ok).toBe(true);
} finally {
await browser.close();
}
});
@@ -0,0 +1,52 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect } from "vitest";
import { thinMach } from "../../../src/domain/binaryTarget.fixture.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
// An unknown standing provider stops native analysis before any engine runs.
const environment = {
REA_LOG_LEVEL: "silent",
REA_ANALYSIS_PROVIDER: "missing-provider",
} as const;
const fixtureApp = async () => {
const directory = await createTestTempDirectory("rea-analyze-app-path-");
const contents = join(directory, "Sample.app", "Contents");
await mkdir(join(contents, "MacOS"), { recursive: true });
await writeFile(
join(contents, "MacOS/Sample"),
thinMach(0xfeedfacf, 0x100000c),
);
await writeFile(
join(contents, "Info.plist"),
"<plist><dict><key>CFBundleExecutable</key><string>Sample</string></dict></plist>",
);
return directory;
};
describe("analyze app bundle path spellings", () => {
cliTest.for(["Sample.app", "Sample.app/", "Sample.app/.", "Sample.app//"])(
"routes %s to native provider selection",
async (path, { cli }) => {
const cwd = await fixtureApp();
const result = await cli.run({
arguments: ["analyze", path, "--json"],
cwd,
environment,
});
expect(result.exitCode).toBe(1);
expect(result.json).toMatchObject({
code: "capability_unavailable",
details: {
operation: "open_binary",
selection_reason: "unknown_provider",
requested_provider_id: "missing-provider",
},
});
},
);
});
@@ -163,31 +163,33 @@ for (const group of groups) {
environment,
timeoutMs: 20_000,
});
for (const { client, config, skillDirectory } of profiles) {
const planned = await run([
"setup",
"--client",
client,
"--dry-run",
"--json",
]);
expect(planned.exitCode, client).toBe(0);
expect(planned.json, client).toMatchObject({
status: "planned",
plannedActions: expect.arrayContaining([
const planned = await run([
"setup",
...clientArguments,
"--dry-run",
"--json",
]);
expect(planned.exitCode, planned.stderr).toBe(0);
expect(planned.json).toMatchObject({
status: "planned",
plannedActions: expect.arrayContaining([
...profiles.map(({ client, config }) =>
expect.objectContaining({
id: `configure_client:${client}`,
target: config,
}),
expect.objectContaining({ target: skillDirectory }),
]),
});
),
...[
...new Set(profiles.map(({ skillDirectory }) => skillDirectory)),
].map((target) => expect.objectContaining({ target })),
]),
});
for (const { config, skillDirectory } of profiles) {
await expect(access(config)).rejects.toMatchObject({ code: "ENOENT" });
await expect(access(skillDirectory)).rejects.toMatchObject({
code: "ENOENT",
});
}
for (const { config } of profiles)
await expect(access(config)).rejects.toMatchObject({ code: "ENOENT" });
const setup = await run(["setup", ...clientArguments, "--yes", "--json"]);
expect(setup.exitCode).toBe(0);
expect(setup.json).toMatchObject({ status: "ready" });
@@ -19,6 +19,7 @@ describe.skipIf(process.platform === "win32")(
"rea-reference-directory-imports-",
);
const root = join(workspace, "source");
const outside = join(workspace, "outside", "index.js");
const imports = [
["slash.cjs", "./lib/", "lib/index.js"],
["dot.cjs", "./lib/.", "lib/index.js"],
@@ -30,12 +31,17 @@ describe.skipIf(process.platform === "win32")(
["fallback.cjs", "./fallback", "fallback/index.js"],
["legacy-order.cjs", "./dep", "dep/index.js"],
["bare-index.cjs", "./bare", "bare.js"],
["package-main.cjs", "./pkg", "pkg/actual.js"],
["package-main-fallback.cjs", "./stale", "stale/index.js"],
["package-main-slash.cjs", "./slashed", "slashed/lib.js"],
] as const;
const unresolved = [
["missing.cjs", "./missing/", "missing/"],
["escape.cjs", "../outside/", "../outside/"],
["main.ts", "./folder.js/.", "folder.js"],
["only-bare.cjs", "./only/", "only/"],
["invalid-manifest.cjs", "./broken", "broken"],
["absolute-main.cjs", "./absolute", outside],
] as const;
const files = new Map<string, string>([
["index.js", 'module.exports = "root index";\n'],
@@ -53,6 +59,22 @@ describe.skipIf(process.platform === "win32")(
["bare.js", 'module.exports = "file before directory";\n'],
["bare/index", 'module.exports = "extensionless index";\n'],
["only/index", 'module.exports = "extensionless index";\n'],
// A package main outranks the index; a missing main falls back to it.
["pkg/package.json", '{"main":"actual.js"}\n'],
["pkg/actual.js", 'module.exports = "declared main";\n'],
["pkg/index.js", 'module.exports = "ignored index";\n'],
["stale/package.json", '{"main":"gone.js"}\n'],
["stale/index.js", 'module.exports = "legacy fallback";\n'],
["broken/package.json", '{"main":\n'],
["broken/index.js", 'module.exports = "unknown selection";\n'],
// Node resolves main with path.resolve: no trailing-slash directory
// intent, and an absolute main leaves the imported root.
["slashed/package.json", '{"main":"lib/"}\n'],
["slashed/lib.js", 'module.exports = "file main";\n'],
["slashed/lib/index.js", 'module.exports = "not selected";\n'],
["slashed/index.js", 'module.exports = "not selected";\n'],
["absolute/package.json", `${JSON.stringify({ main: outside })}\n`],
["absolute/index.js", 'module.exports = "not selected";\n'],
...[...imports, ...unresolved].map(
([path, specifier]) =>
[path, `require(${JSON.stringify(specifier)});\n`] as const,
@@ -65,7 +87,6 @@ describe.skipIf(process.platform === "win32")(
await writeFile(target, contents);
}),
);
const outside = join(workspace, "outside", "index.js");
await mkdir(dirname(outside), { recursive: true });
await writeFile(outside, 'module.exports = "outside import root";\n');
@@ -83,9 +104,15 @@ describe.skipIf(process.platform === "win32")(
expect(() =>
createRequire(join(root, "only-bare.cjs")).resolve("./only/"),
).toThrow();
expect(() =>
createRequire(join(root, "invalid-manifest.cjs")).resolve("./broken"),
).toThrow();
expect(
createRequire(join(root, "escape.cjs")).resolve("../outside/"),
).toBe(outside);
expect(
createRequire(join(root, "absolute-main.cjs")).resolve("./absolute"),
).toBe(outside);
const result = await cli.run({
arguments: ["import-reference-source", root, "--json"],
@@ -1,6 +1,7 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { solidityMetadataSchema } from "../../../src/domain/evm/solidityMetadata.js";
@@ -38,8 +39,12 @@ it.each([
[
"--max-old-space-size=256",
"--disable-wasm-trap-handler",
new URL("../../../dist/evm/EvmoleInterfaceWorker.js", import.meta.url)
.pathname,
fileURLToPath(
new URL(
"../../../dist/evm/EvmoleInterfaceWorker.js",
import.meta.url,
),
),
request,
],
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
@@ -416,7 +416,10 @@ it.each(["", "not-a-commit"])(
},
);
it("selects only a trusted successful ancestor from recent CI push runs", async () => {
it("selects only a trusted successful ancestor from recent CI push runs", async ({
skip,
}) => {
skip(process.platform === "win32", "the fake gh is a POSIX shell script");
const { directory, base } = await fixture();
const head = await change(directory, ["src/evm/client.ts"]);
const bin = join(directory, "fake-bin");
@@ -449,7 +452,11 @@ it("selects only a trusted successful ancestor from recent CI push runs", async
const output = join(directory, "baseline-output");
const result = await execFileAsync(
process.execPath,
[new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname],
[
fileURLToPath(
new URL("../../../scripts/ci/baseline.mjs", import.meta.url),
),
],
{
cwd: directory,
env: {
@@ -467,7 +474,8 @@ it("selects only a trusted successful ancestor from recent CI push runs", async
expect(await readFile(output, "utf8")).toBe(`base=${base}\nreason=\n`);
});
it("returns an empty baseline when GitHub lookup fails", async () => {
it("returns an empty baseline when GitHub lookup fails", async ({ skip }) => {
skip(process.platform === "win32", "the fake gh is a POSIX shell script");
const { directory, base } = await fixture();
const head = await change(directory, ["src/evm/client.ts"]);
const bin = join(directory, "fake-bin");
@@ -477,7 +485,11 @@ it("returns an empty baseline when GitHub lookup fails", async () => {
await chmod(gh, 0o755);
const result = await execFileAsync(
process.execPath,
[new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname],
[
fileURLToPath(
new URL("../../../scripts/ci/baseline.mjs", import.meta.url),
),
],
{
cwd: directory,
env: {
@@ -326,31 +326,72 @@ describe.skipIf(process.platform === "win32")(
},
);
/**
* A PSPSDK-shaped ELF32 whose PT_MIPS_ABIFLAGS record matches the PSP profile.
* Optionally declare a section-header table beyond the captured file.
*/
const pspElfBytes = ({ sectionHeadersBeyondEof = false } = {}): Buffer => {
const abiOffset = 8192;
const bytes = Buffer.alloc(abiOffset + 24);
bytes.set([0x7f, 0x45, 0x4c, 0x46, 1, 1, 1]);
bytes.writeUInt16LE(2, 16);
bytes.writeUInt16LE(8, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt32LE(52, 28);
bytes.writeUInt32LE(0x10a23001, 36);
bytes.writeUInt16LE(52, 40);
bytes.writeUInt16LE(32, 42);
bytes.writeUInt16LE(1, 44);
if (sectionHeadersBeyondEof) {
bytes.writeUInt32LE(bytes.length + 64, 32); // e_shoff
bytes.writeUInt16LE(40, 46); // e_shentsize
bytes.writeUInt16LE(2, 48); // e_shnum
}
bytes.writeUInt32LE(0x70000003, 52);
bytes.writeUInt32LE(abiOffset, 56);
bytes.writeUInt32LE(24, 68);
bytes.writeUInt32LE(24, 72);
// GNU readelf -x .MIPS.abiflags, not bytes generated by REA.
bytes.set(
Buffer.from("000002000101000200000000000000000100000000000000", "hex"),
abiOffset,
);
return bytes;
};
describe.skipIf(process.platform === "win32")(
"PSP inspected ABI declaration boundary",
() => {
it("refuses a PSP image whose section headers lie beyond the captured file", async () => {
const { directory, provider } = await installation();
const path = join(directory, "psp-truncated-sections.elf");
await writeFile(path, pspElfBytes({ sectionHeadersBeyondEof: true }));
const parsed = await parseBinaryTarget(path);
if (!parsed.ok) throw parsed.error;
if (
parsed.value.kind !== "executable" ||
parsed.value.architecture !== "mips"
)
throw new Error("PSP ELF was not recognized as MIPS");
expect(parsed.value.mips.abiFlags).toEqual(pspAbiFlags);
expect(parsed.value.mips.limitations).toContainEqual(
expect.stringContaining("ABI declarations in unavailable headers"),
);
expect(provider.inspectTargetSupport(parsed.value).status).toBe(
"unsupported",
);
const result = await provider.resolveAnalysisProfile(parsed.value);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Incomplete ABI inspection was admitted");
expect(result.error._tag).toBe("AnalysisUnsupportedTargetError");
expect(result.error.message).toContain(
"MIPS ABI declaration inspection is incomplete",
);
});
it("retains independently inspected PSP ABI bytes through the real target reader", async () => {
const { directory, provider } = await installation();
const abiOffset = 8192;
const bytes = Buffer.alloc(abiOffset + 24);
bytes.set([0x7f, 0x45, 0x4c, 0x46, 1, 1, 1]);
bytes.writeUInt16LE(2, 16);
bytes.writeUInt16LE(8, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt32LE(52, 28);
bytes.writeUInt32LE(0x10a23001, 36);
bytes.writeUInt16LE(52, 40);
bytes.writeUInt16LE(32, 42);
bytes.writeUInt16LE(1, 44);
bytes.writeUInt32LE(0x70000003, 52);
bytes.writeUInt32LE(abiOffset, 56);
bytes.writeUInt32LE(24, 68);
bytes.writeUInt32LE(24, 72);
// GNU readelf -x .MIPS.abiflags, not bytes generated by REA.
bytes.set(
Buffer.from("000002000101000200000000000000000100000000000000", "hex"),
abiOffset,
);
const bytes = pspElfBytes();
const path = join(directory, "psp-with-abi.elf");
await writeFile(path, bytes);
const parsed = await parseBinaryTarget(path);
@@ -54,6 +54,8 @@ const graphIdentities = async (source: string) => {
// source locations and every derived identity stay comparable.
it.each([
"fetch('https://api.example.com/v1/users');",
"client['get']('/api/items');",
"navigator['sendBeacon']('/api/events', payload);",
"localStorage.setItem('token', 1);",
"new Worker(new URL('./worker.js', import.meta.url));",
"const dep = require('./dep.js'); dep();",
@@ -1,6 +1,7 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { delimiter, join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { parse } from "yaml";
@@ -348,7 +349,7 @@ it("binds npm and MCP publication to the same immutable release SHA", async () =
import.meta.url,
);
const result = await execFileAsync(process.execPath, [
helper.pathname,
fileURLToPath(helper),
version,
]);
expect(result.stdout).toBe(tag);
+39
View File
@@ -481,6 +481,45 @@ describe("target-free MCP workflow", () => {
}, 10_000);
});
describe("MCP snapshot open ordering", () => {
it("keeps a snapshot open ahead of later reads and closes on the same connection", async () => {
directory = await createTestTempDirectory("rea-mcp-snapshot-open-order-");
const { mcp, first, second } = await createSessionMcpHarness(
directory,
provider,
resources,
);
const snapshotPath = join(directory, "analysis.json");
await mcp.callTool({ name: "open_binary", arguments: { path: second } });
expect(
(
await mcp.callTool({
name: "close_binary",
arguments: { snapshot_path: snapshotPath },
})
).isError,
).not.toBe(true);
await mcp.callTool({ name: "open_binary", arguments: { path: first } });
// Dispatched without awaiting, like a client issuing one turn's calls.
const opening = mcp.callTool({
name: "open_binary",
arguments: { path: second, snapshot_path: snapshotPath },
});
const reading = mcp.callTool({ name: "current_document", arguments: {} });
const closing = mcp.callTool({ name: "close_binary", arguments: {} });
expect((await opening).isError, JSON.stringify(await opening)).not.toBe(
true,
);
expect(text(await reading)).toContain("second.hop");
expect((await closing).isError).not.toBe(true);
expect(
text(await mcp.callTool({ name: "binary_session", arguments: {} })),
).toContain('"open":false');
}, 10_000);
});
describe("session filesystem path boundaries over MCP", () => {
it("rejects relative snapshot and export paths with an absolute-path error", async () => {
directory = await createTestTempDirectory("rea-mcp-path-boundary-");
@@ -1,4 +1,5 @@
import { execFile } from "node:child_process";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
@@ -133,7 +134,7 @@ describe("Hopper API facade", () => {
it("imports without Hopper globals and gates exhaustive work during analysis", async () => {
const { stdout } = await execute(
"python3",
[probePath.pathname, bridgePath.pathname],
[fileURLToPath(probePath), fileURLToPath(bridgePath)],
{
encoding: "utf8",
timeout: 3_000,
+125
View File
@@ -0,0 +1,125 @@
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
/** One function the fake exports; `error` renders Kuna's failed-function header. */
export interface FakeKunaFunction {
readonly name: string;
readonly addr: string;
readonly error?: string;
}
/** Scripted behavior of the fake `kuna` executable for one test. */
export interface FakeKunaBehavior {
/** Text printed by `--version`; `exit` fails the probe instead. */
readonly version?:
| { readonly stdout: string }
| { readonly exit: number }
| { readonly ignoreTerm: true };
readonly inventory?: "ok" | "not-json" | "hang";
readonly project?:
| "ok"
| "exit-failure"
| "hang"
| "miscounted"
| "silent"
| "block-manifest";
readonly functions?: readonly FakeKunaFunction[];
/** Extra `.asm` bytes, for capacity tests. */
readonly asmPadding?: number;
/** Milliseconds the export takes before writing its files. */
readonly delayMs?: number;
}
/**
* The fake reads its behavior from `<script>.json` and appends one line per
* invocation to `<script>.calls`. REA gives Kuna children no PATH, so the
* shebang names this Node binary directly. Output formats follow Kuna 1.833:
* `functions --json` and the `// Function:` headers of `render_c`.
*/
const SCRIPT = String.raw`
const fs = require("node:fs");
const path = require("node:path");
const behavior = JSON.parse(fs.readFileSync(__filename + ".json", "utf8"));
fs.appendFileSync(__filename + ".calls", JSON.stringify({ pid: process.pid, argv: process.argv.slice(2), home: process.env.HOME, xdg: process.env.XDG_CACHE_HOME, path: process.env.PATH ?? null }) + "\n");
const args = process.argv.slice(2);
const functions = behavior.functions ?? [{ name: "main", addr: "0x1000" }, { name: "helper", addr: "0x1040" }];
const hang = () => setInterval(() => {}, 1000);
if (args[0] === "--version") {
const version = behavior.version ?? { stdout: "kuna 1.833\n" };
if ("exit" in version) { process.stderr.write("version probe failed\n"); process.exit(version.exit); }
if ("ignoreTerm" in version) { process.on("SIGTERM", () => {}); hang(); }
else process.stdout.write(version.stdout);
} else if (args[0] === "functions") {
const mode = behavior.inventory ?? "ok";
if (mode === "hang") hang();
else if (mode === "not-json") process.stdout.write("functions: 2\n");
else {
const listed = [...functions, { name: "puts", addr: "0x2000" }].map((fn) => ({ name: fn.name, address: parseInt(fn.addr, 16), address_hex: fn.addr, aliases: [], object_location: null, size: fn.name === "puts" ? 0 : 16 }));
process.stdout.write(JSON.stringify({ binary: args[1], count: listed.length, total: listed.length, error: null, functions: listed }, null, 2) + "\n");
}
} else if (args[0] === "decompile-project") {
const input = args[1];
const output = args[args.indexOf("-o") + 1];
const stem = path.basename(input);
const mode = behavior.project ?? "ok";
if (mode === "hang") hang();
else if (mode === "exit-failure") { process.stderr.write("error: not in recognized object file format\n"); process.exit(1); }
else setTimeout(() => {
fs.mkdirSync(output, { recursive: true });
let c = '#include "' + stem + '.h"\n\n';
functions.forEach((fn, index) => {
c += fn.error === undefined
? "// Function: " + fn.name + " @ " + fn.addr + "\nint " + fn.name + "(void)\n{\n return " + index + ";\n}\n\n"
: "// Function: " + fn.name + " @ " + fn.addr + " (error: " + fn.error + ")\n\n";
});
fs.writeFileSync(path.join(output, stem + ".c"), c);
fs.writeFileSync(path.join(output, stem + ".h"), "#pragma once\n");
fs.writeFileSync(path.join(output, stem + ".asm"), "; asm\n" + "x".repeat(behavior.asmPadding ?? 0));
fs.writeFileSync(path.join(output, "README.md"), "# " + stem + "\n");
// Occupy the manifest path so REA's publication write fails.
if (mode === "block-manifest") fs.mkdirSync(path.join(output, "..", "manifest.json"));
const failed = functions.filter((fn) => fn.error !== undefined).length;
const reported = mode === "miscounted" ? functions.length + 1 : functions.length - failed;
if (mode !== "silent") process.stdout.write("wrote " + output + ": " + stem + ".c (" + c.length + " bytes); functions: " + reported + " ok, " + failed + " failed\n");
}, behavior.delayMs ?? 0);
} else {
process.stderr.write("unexpected command\n");
process.exit(2);
}
`;
/** Write a fake `kuna` into a directory and return its path. */
export const writeFakeKuna = async (
directory: string,
behavior: FakeKunaBehavior = {},
): Promise<string> => {
const path = join(directory, "kuna");
await writeFile(path, `#!${process.execPath}\n${SCRIPT}`, { mode: 0o755 });
await writeFile(`${path}.json`, JSON.stringify(behavior));
return path;
};
/** Replace the scripted behavior of an existing fake. */
export const setFakeKunaBehavior = (
path: string,
behavior: FakeKunaBehavior,
): Promise<void> => writeFile(`${path}.json`, JSON.stringify(behavior));
/** Invocations the fake recorded, oldest first. */
export const fakeKunaCalls = async (
path: string,
): Promise<
readonly {
readonly pid: number;
readonly argv: readonly string[];
readonly home: string;
readonly xdg: string;
readonly path: string | null;
}[]
> => {
const text = await readFile(`${path}.calls`, "utf8").catch(() => "");
return text
.split("\n")
.filter((line) => line.length > 0)
.map((line) => JSON.parse(line) as never);
};
@@ -0,0 +1,136 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { inspectKunaProvider } from "../../../../src/kuna/KunaDoctor.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { writeFakeKuna } from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
it("reports an unconfigured Kuna as a missing optional engine", async () => {
expect(await inspectKunaProvider({})).toMatchObject({
id: "kuna",
configured: false,
available: false,
providerVersion: null,
registrationEnvironment: {},
checks: [
{
name: "installation",
ok: false,
code: "not_configured",
classification: "missing_analysis_engine",
},
],
});
});
it("publishes the caller's explicit settings once the installation validates", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-");
const kuna = await writeFakeKuna(root);
const cache = join(root, "cache");
const inspection = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: cache,
REA_KUNA_JOBS: "4",
UNRELATED: "not published",
});
expect(inspection).toMatchObject({
configured: true,
available: true,
providerVersion: "1.833",
registrationEnvironment: {
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: cache,
REA_KUNA_JOBS: "4",
},
checks: [
{ name: "installation", ok: true, details: { version: "1.833" } },
{
name: "cache",
ok: true,
detail: `REA creates the Kuna project cache at ${cache} on first use.`,
},
],
});
expect(Object.keys(inspection.registrationEnvironment)).toHaveLength(3);
});
it("flags an unverified release without failing the installation", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-version-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "kuna 1.950\n" },
});
const inspection = await inspectKunaProvider({ REA_KUNA_COMMAND: kuna });
expect(inspection.checks[0]).toMatchObject({
ok: true,
detail: expect.stringContaining(
"this release's project layout is unverified",
),
});
});
it("reports a cache root REA cannot use without creating anything", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-cache-");
const kuna = await writeFakeKuna(root);
const occupied = join(root, "occupied");
await writeFile(occupied, "not a directory");
const existing = join(root, "existing");
await mkdir(existing);
const blocked = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: occupied,
});
const ready = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: existing,
});
expect(blocked.checks[1]).toMatchObject({
name: "cache",
ok: false,
code: "open_options_invalid",
});
expect(ready.checks[1]).toMatchObject({
name: "cache",
ok: true,
detail: expect.stringContaining(`cached in ${existing}`),
});
});
it("reports a cache root REA cannot create under its nearest existing parent", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-create-");
const kuna = await writeFakeKuna(root);
const readOnly = join(root, "read-only");
await mkdir(readOnly, { mode: 0o500 });
const occupied = join(root, "occupied");
await writeFile(occupied, "not a directory");
const underReadOnly = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(readOnly, "nested", "cache"),
});
const underFile = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(occupied, "cache"),
});
expect(underReadOnly.checks[1]).toMatchObject({
name: "cache",
ok: false,
detail: expect.stringContaining(`${readOnly} is not writable`),
});
expect(underFile.checks[1]).toMatchObject({
name: "cache",
ok: false,
// stat reports ENOTDIR for a path beneath a regular file.
detail: expect.stringContaining("not a directory"),
});
});
@@ -0,0 +1,212 @@
import { createHash } from "node:crypto";
import { chmod, readFile, realpath, symlink } from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { readKunaConfiguration } from "../../../../src/kuna/KunaConfiguration.js";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
it("identifies a caller-selected kuna by version and digest", async () => {
const root = await createTestTempDirectory("rea-kuna-install-");
const kuna = await writeFakeKuna(root);
const link = join(root, "current-kuna");
await symlink(kuna, link);
const result = await inspectKunaInstallation({
REA_KUNA_COMMAND: link,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
expect(result).toMatchObject({
ok: true,
value: {
configuration: { command: link, cacheRoot: join(root, "cache") },
executable: await realpath(kuna),
version: "1.833",
verified: true,
// A `#!` launcher reports its interpreter; ownership checks must not expect the path.
script: true,
sha256: createHash("sha256")
.update(await readFile(kuna))
.digest("hex"),
},
});
// The probe runs inside REA's cache home with no inherited PATH.
expect(await fakeKunaCalls(kuna)).toEqual([
{
pid: expect.any(Number),
argv: ["--version"],
home: join(root, "cache", "home"),
xdg: join(root, "cache", "home", ".cache"),
path: null,
},
]);
});
it("accepts another release but reports it as unverified", async () => {
const root = await createTestTempDirectory("rea-kuna-unverified-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "kuna 1.900\n" },
});
const result = await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna });
expect(result).toMatchObject({
ok: true,
value: { version: "1.900", verified: false },
});
});
it("refuses a version line it cannot recognize", async () => {
const root = await createTestTempDirectory("rea-kuna-garbled-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "Kuna decompiler, built today\n" },
});
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna }),
).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
diagnostics: { stdout: "Kuna decompiler, built today\n" },
},
});
});
it("reports a failed version probe with its exit status", async () => {
const root = await createTestTempDirectory("rea-kuna-probe-exit-");
const kuna = await writeFakeKuna(root, { version: { exit: 3 } });
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna }),
).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
reason: expect.stringContaining("version probe failed"),
diagnostics: { exit_code: 3, killed: false },
},
});
});
it("reports a missing or non-executable command as executable_missing", async () => {
const root = await createTestTempDirectory("rea-kuna-missing-");
const kuna = await writeFakeKuna(root);
await chmod(kuna, 0o644);
for (const command of [kuna, join(root, "absent"), root])
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: command }),
).toMatchObject({
ok: false,
error: { code: "executable_missing", diagnostics: { command } },
});
});
it("distinguishes an unset command from an invalid one", async () => {
expect(await inspectKunaInstallation({})).toMatchObject({
ok: false,
error: { code: "not_configured", diagnostics: { configured: false } },
});
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: "bin/kuna" }),
).toMatchObject({
ok: false,
error: {
code: "open_options_invalid",
diagnostics: { configured: true, setting: "REA_KUNA_COMMAND" },
},
});
});
it("declares Windows hosts unsupported before touching the command", async () => {
expect(
await inspectKunaInstallation(
{ REA_KUNA_COMMAND: "/opt/kuna/kuna" },
{ platform: "win32" },
),
).toMatchObject({ ok: false, error: { code: "unsupported_host" } });
});
it("parses cache, timeout and job settings at the configuration boundary", () => {
const environment = {
REA_KUNA_COMMAND: "/opt/kuna/kuna",
HOME: "/home/analyst",
};
expect(readKunaConfiguration(environment, "linux")).toMatchObject({
ok: true,
value: {
cacheRoot: "/home/analyst/.cache/rea/kuna",
cacheMaxBytes: 4 * 1024 ** 3,
projectTimeoutMs: 1_800_000,
// Kuna's reference answer is a serial export; more jobs change results.
jobs: 1,
},
});
expect(
readKunaConfiguration(
{ ...environment, XDG_CACHE_HOME: "/var/cache/analyst" },
"linux",
),
).toMatchObject({ value: { cacheRoot: "/var/cache/analyst/rea/kuna" } });
expect(readKunaConfiguration(environment, "darwin")).toMatchObject({
value: { cacheRoot: "/home/analyst/Library/Caches/rea/kuna" },
});
expect(
readKunaConfiguration(
{
...environment,
REA_KUNA_CACHE_MAX_BYTES: "2097152",
REA_KUNA_PROJECT_TIMEOUT_MS: "60000",
REA_KUNA_JOBS: "3",
},
"linux",
),
).toMatchObject({
value: { cacheMaxBytes: 2097152, projectTimeoutMs: 60000, jobs: 3 },
});
for (const [setting, value] of [
["REA_KUNA_CACHE_DIR", "relative/cache"],
["REA_KUNA_CACHE_MAX_BYTES", "1024"],
["REA_KUNA_CACHE_MAX_BYTES", "4GiB"],
["REA_KUNA_PROJECT_TIMEOUT_MS", "0"],
["REA_KUNA_JOBS", "33"],
["REA_KUNA_JOBS", "0"],
] as const)
expect(
readKunaConfiguration({ ...environment, [setting]: value }, "linux"),
).toMatchObject({
ok: false,
error: { code: "open_options_invalid", setting },
});
});
it("stops a version probe that ignores SIGTERM instead of waiting on it", async () => {
const root = await createTestTempDirectory("rea-kuna-probe-term-");
const kuna = await writeFakeKuna(root, { version: { ignoreTerm: true } });
const started = Date.now();
const result = await inspectKunaInstallation(
{ REA_KUNA_COMMAND: kuna },
// Long enough for Node to start the fake and install its SIGTERM handler.
{ probeTimeoutMs: 1500 },
);
expect(result).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
diagnostics: { killed: true, deadline_ms: 1500 },
},
});
// The deadline plus ExecFileOutput's bounded kill grace, never an open-ended wait.
expect(Date.now() - started).toBeLessThan(15_000);
const pid = (await fakeKunaCalls(kuna))[0]?.pid ?? 0;
expect(() => process.kill(pid, 0)).toThrow();
}, 20_000);
@@ -0,0 +1,704 @@
import { createHash } from "node:crypto";
import {
appendFile,
chmod,
mkdir,
readdir,
readFile,
rename,
rm,
utimes,
writeFile,
} from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { cleanupOwnedProcessGroup } from "../../../../src/process/ProcessOwnership.js";
import { spawnOwnedProviderProcess } from "../../../../src/process/ProviderProcess.js";
import type { KunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import {
KunaProjectCache,
kunaProjectKey,
} from "../../../../src/kuna/KunaProjectCache.js";
import type { KunaExportStatus } from "../../../../src/kuna/KunaProjectExport.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import {
fakeKunaCalls,
setFakeKunaBehavior,
writeFakeKuna,
type FakeKunaBehavior,
} from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
interface Target {
readonly targetPath: string;
readonly targetSha256: string;
}
const fixture = async (
prefix: string,
behavior: FakeKunaBehavior = {},
options: {
readonly maxBytes?: number;
readonly environment?: Readonly<Record<string, string>>;
} = {},
) => {
const root = await createTestTempDirectory(prefix);
const kuna = await writeFakeKuna(root, behavior);
const installed = await inspectKunaInstallation({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
...options.environment,
});
if (!installed.ok) throw new Error(installed.error.reason);
return {
kuna,
installation: installed.value,
cacheRoot: installed.value.configuration.cacheRoot,
cache: newCache(installed.value, options.maxBytes),
target: async (name: string, content = name): Promise<Target> => {
const targetPath = join(root, name);
await writeFile(targetPath, content);
return {
targetPath,
targetSha256: createHash("sha256").update(content).digest("hex"),
};
},
};
};
const newCache = (installation: KunaInstallation, maxBytes = 64 * 1024 ** 2) =>
new KunaProjectCache(installation.configuration.cacheRoot, {
maxBytes,
});
const keyOf = (installation: KunaInstallation, target: Target) =>
kunaProjectKey({
targetSha256: target.targetSha256,
kunaSha256: installation.sha256,
kunaVersion: installation.version,
jobs: installation.configuration.jobs,
});
const callsOf = async (kuna: string, subcommand: string) =>
(await fakeKunaCalls(kuna)).filter(({ argv }) => argv[0] === subcommand);
const entries = (cacheRoot: string, area: string) =>
readdir(join(cacheRoot, area)).catch(() => []);
const processGone = (pid: number): boolean => {
try {
process.kill(pid, 0);
return false;
} catch {
return true;
}
};
it("publishes a verified export and serves later requests from the cache", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-cache-",
{
functions: [
{ name: "main", addr: "0x1000" },
{ name: "parse @ table", addr: "0x1040" },
{
name: "broken",
addr: "0x1080",
error: "Exceeded the instruction budget",
},
],
},
{ environment: { REA_KUNA_JOBS: "1" } },
);
const input = await target("program");
const built = await cache.build({ installation, ...input });
expect(built).toMatchObject({
ok: true,
value: {
key: keyOf(installation, input),
manifest: {
layout: 1,
inputs: {
target_sha256: input.targetSha256,
kuna_sha256: installation.sha256,
kuna_version: "1.833",
jobs: 1,
},
summary: { decompiled: 2, failed: 1 },
},
functions: [
{ name: "main", address: "0x1000", error: null },
// A name may itself contain " @ "; the address is the last field.
{ name: "parse @ table", address: "0x1040", error: null },
{
name: "broken",
address: "0x1080",
error: "Exceeded the instruction budget",
},
],
// The inventory also names imports Kuna does not decompile.
inventory: [
{ name: "main", address: "0x1000", size: 16 },
{ name: "parse @ table", address: "0x1040", size: 16 },
{ name: "broken", address: "0x1080", size: 16 },
{ name: "puts", address: "0x2000", size: 0 },
],
},
});
if (!built.ok) throw built.error;
const [main, , broken] = built.value.functions;
const c = await readFile(join(built.value.directory, "program.c"));
const slice = (span: typeof main) =>
c
.subarray(span?.cOffset, (span?.cOffset ?? 0) + (span?.cLength ?? 0))
.toString("utf8");
expect(slice(main)).toBe(
"// Function: main @ 0x1000\nint main(void)\n{\n return 0;\n}\n\n",
);
expect(slice(broken)).toBe(
"// Function: broken @ 0x1080 (error: Exceeded the instruction budget)\n\n",
);
// Kuna listed and decompiled a staged copy from REA's own HOME, in reference mode.
const [listing] = await callsOf(kuna, "functions");
const [exported] = await callsOf(kuna, "decompile-project");
expect(listing?.argv).toEqual(["functions", exported?.argv[1], "--json"]);
expect(exported?.argv.slice(2)).toEqual([
"-o",
expect.stringContaining(join("cache", "staging")),
"--jobs",
"1",
]);
expect(exported?.argv[1]).not.toBe(input.targetPath);
expect(exported?.home).toBe(join(cacheRoot, "home"));
expect(await entries(cacheRoot, "staging")).toEqual([]);
// The same process and a fresh one both reuse the published entry.
await cache.build({ installation, ...input });
expect(await newCache(installation).lookup(built.value.key)).toMatchObject({
functions: built.value.functions,
inventory: built.value.inventory,
});
expect(await callsOf(kuna, "decompile-project")).toHaveLength(1);
});
it("shares one Kuna run between concurrent requests for one export", async () => {
const { kuna, installation, cache, target } = await fixture(
"rea-kuna-singleflight-",
{ delayMs: 300 },
);
const input = await target("program");
const [first, second] = await Promise.all([
cache.build({ installation, ...input }),
cache.build({ installation, ...input }),
]);
expect(first.ok && second.ok).toBe(true);
if (first.ok && second.ok)
expect(first.value.directory).toBe(second.value.directory);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(1);
});
it("stops the run only when every waiting request is cancelled", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-cancel-",
{ project: "hang" },
);
const input = await target("program");
const firstCaller = new AbortController();
const secondCaller = new AbortController();
const first = cache.build({
installation,
...input,
signal: firstCaller.signal,
});
const second = cache.build({
installation,
...input,
signal: secondCaller.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
const pid = (await callsOf(kuna, "decompile-project"))[0]?.pid ?? 0;
firstCaller.abort();
expect(await first).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
// The other request still owns the run.
expect(processGone(pid)).toBe(false);
secondCaller.abort();
expect(await second).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(processGone(pid)).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([]);
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("applies one deadline to the inventory and the export together", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-timeout-",
{ project: "hang" },
{ environment: { REA_KUNA_PROJECT_TIMEOUT_MS: "800" } },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisTimeoutError", timeoutMs: 800 },
});
const [exported] = await callsOf(kuna, "decompile-project");
expect(processGone(exported?.pid ?? 0)).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([]);
await setFakeKunaBehavior(kuna, { inventory: "hang" });
expect(
await cache.build({ installation, ...(await target("other")) }),
).toMatchObject({ ok: false, error: { _tag: "AnalysisTimeoutError" } });
});
it("reports a failed export with Kuna's diagnostics and retains nothing", async () => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-failure-",
{ project: "exit-failure" },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage:
"kuna decompile-project failed with exit code 1: error: not in recognized object file format",
capturedOutput: { exit_code: 1 },
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it.each([
[
"an inventory that is not Kuna's JSON document",
{ inventory: "not-json" },
"inventory is not JSON",
],
[
"an export without Kuna's completed-file summary",
{ project: "silent" },
"summary",
],
[
"an export whose headers disagree with Kuna's summary",
{ project: "miscounted" },
"2 function headers but Kuna reported 3 functions",
],
] as const)("refuses %s", async (_case, behavior, reason) => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-refused-",
behavior,
);
expect(
await cache.build({ installation, ...(await target("program")) }),
).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage: expect.stringContaining(reason),
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("refuses to decompile a target that no longer matches its digest", async () => {
const { kuna, installation, cache, target } =
await fixture("rea-kuna-changed-");
const input = await target("program");
await appendFile(input.targetPath, "patched");
expect(await cache.build({ installation, ...input })).toMatchObject({
ok: false,
error: { _tag: "AnalysisArtifactChangedError", path: input.targetPath },
});
expect(await fakeKunaCalls(kuna)).toHaveLength(1);
});
it("removes its staging when the target disappears before the copy", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-vanished-");
const input = await target("program");
const result = await cache.build({
installation,
...input,
targetPath: `${input.targetPath}.gone`,
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage: expect.stringContaining("ENOENT"),
},
});
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("discards an entry whose files no longer match their manifest", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-tamper-");
const input = await target("program");
const built = await cache.build({ installation, ...input });
if (!built.ok) throw built.error;
await appendFile(join(built.value.directory, "program.c"), "// edited\n");
expect(await newCache(installation).lookup(built.value.key)).toBeUndefined();
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("evicts least recently used entries above the cap, never a pinned one", async () => {
const { installation, cacheRoot, target } = await fixture("rea-kuna-evict-", {
asmPadding: 400 * 1024,
});
const cache = newCache(installation, 1024 * 1024);
const build = async (name: string, ageSeconds: number) => {
const result = await cache.build({ installation, ...(await target(name)) });
if (!result.ok) throw result.error;
const when = new Date(Date.now() - ageSeconds * 1000);
await utimes(join(cacheRoot, "projects", result.value.key), when, when);
return result.value.key;
};
const oldest = await build("oldest", 30);
const pinned = await build("pinned", 40);
const release = cache.pin(pinned);
const newest = await build("newest", 0);
// Three ~400 KiB entries exceed 1 MiB: the oldest unpinned entry goes.
expect((await entries(cacheRoot, "projects")).sort()).toEqual(
[pinned, newest].sort(),
);
expect(await cache.lookup(oldest)).toBeUndefined();
release();
await build("latest", 0);
expect(await entries(cacheRoot, "projects")).not.toContain(pinned);
});
it("refuses an export larger than the whole cache may hold", async () => {
const { installation, cacheRoot, target } = await fixture(
"rea-kuna-capacity-",
{ asmPadding: 8 * 1024 },
);
const result = await newCache(installation, 4096).build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisResourceConstraintError",
resource: "capacity",
reportedLimits: { cache_max_bytes: 4096 },
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("keeps the first publication when another process exports the same key", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-race-",
{ delayMs: 300 },
);
const input = await target("program");
// Two cache instances stand in for two REA processes sharing one root.
const [first, second] = await Promise.all([
newCache(installation).build({ installation, ...input }),
newCache(installation).build({ installation, ...input }),
]);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(2);
expect(first.ok && second.ok).toBe(true);
if (first.ok && second.ok)
expect(first.value.manifest).toEqual(second.value.manifest);
expect(await entries(cacheRoot, "projects")).toEqual([
keyOf(installation, input),
]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("reports each build phase to every waiting request", async () => {
const { installation, cache, target } = await fixture("rea-kuna-phases-");
const statuses: KunaExportStatus[] = [];
const result = await cache.build({
installation,
...(await target("program")),
onStatus: (status) => statuses.push(status),
});
expect(result.ok).toBe(true);
expect(statuses).toEqual([
{ phase: "inventory", inventoryFunctions: null },
{ phase: "decompiling", inventoryFunctions: 3 },
{ phase: "sealing", inventoryFunctions: 3 },
]);
});
it("sweeps retired entries but preserves staging with unresolved process ownership", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-sweep-");
const abandoned = join(cacheRoot, "staging", "build-abandoned");
const recent = join(cacheRoot, "staging", "build-recent");
const evicted = join(cacheRoot, "trash", "evicted");
for (const path of [abandoned, recent, evicted])
await mkdir(path, { recursive: true });
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000);
await utimes(abandoned, twoDaysAgo, twoDaysAgo);
expect(
(await cache.build({ installation, ...(await target("program")) })).ok,
).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([
"build-abandoned",
"build-recent",
]);
expect(await entries(cacheRoot, "trash")).toEqual([]);
});
it.each(["sweep", "close"] as const)(
"keeps failed file cleanup owned until recovery through %s",
async (recovery) => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-trash-cleanup-",
);
const retired = join(cacheRoot, "trash", "retired");
await mkdir(retired, { recursive: true });
await writeFile(join(retired, "evidence"), "retained export");
await chmod(retired, 0o500);
let built;
let closed;
try {
built = await cache.build({ installation, ...(await target("program")) });
closed = await cache.close();
} finally {
await chmod(retired, 0o700);
}
if (!built.ok) throw built.error;
expect(closed).toMatchObject({
ok: false,
error: { cleanupIncomplete: true, cleanupResources: [retired] },
});
expect(await readFile(join(retired, "evidence"), "utf8")).toBe(
"retained export",
);
if (recovery === "sweep") {
const next = await cache.build({
installation,
...(await target("other")),
});
if (!next.ok) throw next.error;
}
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(await entries(cacheRoot, "trash")).toEqual([]);
},
);
it("keeps staging owned when a cancelled Kuna process cannot be proven stopped", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-retained-",
{ project: "hang" },
);
// Stop the process for real, but report that ownership could not be revalidated.
const cache = new KunaProjectCache(cacheRoot, {
maxBytes: 64 * 1024 ** 2,
launcher: async (options) => {
const spawned = await spawnOwnedProviderProcess(options);
if (options.arguments[0] !== "decompile-project") return spawned;
const workspace = options.cwd;
if (workspace === undefined) throw new Error("Kuna workspace is missing");
return {
...spawned,
cleanup: async () => {
await cleanupOwnedProcessGroup(spawned.ownership);
// A full disk or lost directory write access can prevent a final
// lifecycle marker. Cleanup must remain fail-closed across instances.
await chmod(workspace, 0o500);
return {
cleaned: false,
reason: "ownership could not be revalidated",
};
},
};
},
});
const caller = new AbortController();
const build = cache.build({
installation,
...(await target("program")),
signal: caller.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
caller.abort();
expect(await build).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
const closed = await cache.close();
const staged = await entries(cacheRoot, "staging");
const retained = join(cacheRoot, "staging", staged[0] ?? "missing");
// Restore access before assertions so the fixture can always remove it.
await chmod(retained, 0o700);
expect(closed).toMatchObject({
ok: false,
error: {
cleanupIncomplete: true,
diagnostics: { reason: "ownership could not be revalidated" },
},
});
expect(staged).toHaveLength(1);
if (!closed.ok) expect(closed.error.cleanupResources).toContain(retained);
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000);
await utimes(retained, twoDaysAgo, twoDaysAgo);
await setFakeKunaBehavior(kuna, {});
const other = newCache(installation);
const built = await other.build({
installation,
...(await target("other")),
});
if (!built.ok) throw built.error;
expect(await entries(cacheRoot, "staging")).toEqual(staged);
expect(await readFile(join(retained, "input", "program"), "utf8")).toBe(
"program",
);
expect(await other.close()).toEqual({ ok: true, value: null });
// The fake process really stopped, so the test can remove its files. File
// absence still does not resolve the deliberately unproven process record.
await rm(retained, { recursive: true });
expect(await cache.close()).toMatchObject({
ok: false,
error: { diagnostics: { reason: "ownership could not be revalidated" } },
});
});
it("rebuilds an entry another cache instance evicted after this one verified it", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-evicted-elsewhere-",
);
const input = await target("program");
const built = await cache.build({ installation, ...input });
if (!built.ok) throw built.error;
// Another process retires the entry the way #discard does.
await mkdir(join(cacheRoot, "trash"), { recursive: true });
await rename(
join(cacheRoot, "projects", built.value.key),
join(cacheRoot, "trash", "evicted"),
);
expect(await cache.lookup(built.value.key)).toBeUndefined();
const rebuilt = await cache.build({ installation, ...input });
expect(rebuilt.ok).toBe(true);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(2);
expect(await entries(cacheRoot, "projects")).toEqual([built.value.key]);
});
it("reports an abandoned build's unproven cleanup even after a retry replaced it", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-replaced-",
{ project: "hang" },
);
let exports = 0;
const cache = new KunaProjectCache(cacheRoot, {
maxBytes: 64 * 1024 ** 2,
launcher: async (options) => {
const spawned = await spawnOwnedProviderProcess(options);
if (options.arguments[0] !== "decompile-project" || ++exports > 1)
return spawned;
// The first export stops slowly and cannot prove its group gone.
return {
...spawned,
cleanup: async () => {
await new Promise((resolve) => setTimeout(resolve, 500));
await cleanupOwnedProcessGroup(spawned.ownership);
return { cleaned: false, reason: "first export unverified" };
},
};
},
});
const input = await target("program");
const first = new AbortController();
const firstBuild = cache.build({
installation,
...input,
signal: first.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
first.abort();
await firstBuild;
// A retry for the same key starts while the first is still stopping.
const second = new AbortController();
const secondBuild = cache.build({
installation,
...input,
signal: second.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(2);
second.abort();
await secondBuild;
expect(await cache.close()).toMatchObject({
ok: false,
error: { diagnostics: { reason: "first export unverified" } },
});
});
it("removes its staging when publication itself fails", async () => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-publish-failure-",
{ project: "block-manifest" },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "ProviderAdapterError" },
});
expect(await entries(cacheRoot, "staging")).toEqual([]);
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
@@ -0,0 +1,252 @@
import { createHash } from "node:crypto";
import { appendFileSync } from "node:fs";
import { readFile, readdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, expect, it as test, onTestFinished, vi } from "vitest";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { KunaProjectCache } from "../../../../src/kuna/KunaProjectCache.js";
import type { KunaExportStatus } from "../../../../src/kuna/KunaProjectExport.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
// Reproduce concurrent eviction's shared directory snapshot while retaining
// real exports, publication, hashing and file removal underneath it.
const listing = vi.hoisted(() => ({
directory: "",
names: [] as string[],
gate: undefined as Promise<void> | undefined,
arrived: 0,
touchPath: "",
touchGate: undefined as Promise<void> | undefined,
touched: false,
}));
vi.mock("node:fs/promises", async (importOriginal) => {
const actual = await importOriginal<typeof import("node:fs/promises")>();
return {
...actual,
readdir: async (...args: Parameters<typeof actual.readdir>) => {
if (String(args[0]) === listing.directory && listing.gate !== undefined) {
listing.arrived += 1;
await listing.gate;
return listing.names;
}
return actual.readdir(...args);
},
utimes: async (...args: Parameters<typeof actual.utimes>) => {
await actual.utimes(...args);
if (
String(args[0]) === listing.touchPath &&
listing.touchGate !== undefined
) {
listing.touched = true;
await listing.touchGate;
}
},
};
});
afterEach(() => {
listing.directory = "";
listing.gate = undefined;
listing.names = [];
listing.arrived = 0;
listing.touchPath = "";
listing.touchGate = undefined;
listing.touched = false;
});
const it = test.skipIf(process.platform === "win32");
const barrier = () => {
let release = () => {};
const promise = new Promise<void>((resolve) => {
release = resolve;
});
return { promise, release };
};
const fixture = async (maxBytes = 64 * 1024 ** 2) => {
const root = await createTestTempDirectory("rea-kuna-concurrency-");
const kuna = await writeFakeKuna(root, {
delayMs: 300,
asmPadding: 400 * 1024,
});
const inspected = await inspectKunaInstallation({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
if (!inspected.ok) throw new Error(inspected.error.reason);
const installation = inspected.value;
const cache = new KunaProjectCache(installation.configuration.cacheRoot, {
maxBytes,
});
const request = async (name: string) => {
const targetPath = join(root, name);
await writeFile(targetPath, name);
return {
installation,
targetPath,
targetSha256: createHash("sha256").update(name).digest("hex"),
};
};
return {
kuna,
cache,
request,
cacheRoot: installation.configuration.cacheRoot,
};
};
it.each([true, false])(
"preserves concurrent results and reclaims capacity (pin: %s)",
async (pin) => {
const { cache, request, cacheRoot } = await fixture(600 * 1024);
const actual =
await vi.importActual<typeof import("node:fs/promises")>(
"node:fs/promises",
);
let releaseListing = () => {};
const gate = new Promise<void>((resolve) => {
releaseListing = resolve;
});
listing.directory = join(cacheRoot, "projects");
listing.gate = gate;
const releases: (() => void)[] = [];
const inputs = await Promise.all([request("first"), request("second")]);
const builds = inputs.map(async (input) => {
const result = await cache.build(input);
if (!result.ok) throw result.error;
if (pin) releases.push(cache.pin(result.value.key));
return result.value;
});
onTestFinished(async () => {
releaseListing();
await Promise.allSettled(builds);
for (const release of releases) release();
const closed = await cache.close();
if (!closed.ok) throw closed.error;
});
await expect
.poll(() => actual.readdir(listing.directory).catch(() => []))
.toHaveLength(2);
listing.names = await actual.readdir(listing.directory);
releaseListing();
const projects = await Promise.all(builds);
if (!pin) {
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(await actual.readdir(listing.directory)).toHaveLength(1);
return;
}
for (const project of projects) {
expect(
await readFile(
join(project.directory, project.manifest.files.c.path),
"utf8",
),
).toContain("// Function: main @ 0x1000");
}
},
);
it.each(["decompiling", "sealing"] as const)(
"rejects an executable changed during %s",
async (phase) => {
const { kuna, cache, request, cacheRoot } = await fixture();
const input = await request("program");
const result = await cache.build({
...input,
onStatus: (status) => {
if (status.phase === phase)
appendFileSync(kuna, "\n// installation replaced\n");
},
});
expect(result).toMatchObject({
ok: false,
error: {
diagnostics: {
expected_sha256: input.installation.sha256,
observed_sha256: expect.any(String),
},
},
});
expect(await readdir(join(cacheRoot, "projects"))).toEqual([]);
expect(await cache.close()).toEqual({ ok: true, value: null });
},
);
it("protects a cache request that arrives after an eviction snapshot starts", async () => {
const { cache, cacheRoot, request } = await fixture(600 * 1024);
const actual =
await vi.importActual<typeof import("node:fs/promises")>(
"node:fs/promises",
);
const firstInput = await request("first");
const first = await cache.build(firstInput);
if (!first.ok) throw first.error;
const snapshot = barrier();
const touch = barrier();
listing.directory = join(cacheRoot, "projects");
listing.gate = snapshot.promise;
listing.touchPath = join(listing.directory, first.value.key);
listing.touchGate = touch.promise;
const releases: (() => void)[] = [];
const pinResult = async (pending: ReturnType<KunaProjectCache["build"]>) => {
const result = await pending;
if (!result.ok) throw result.error;
releases.push(cache.pin(result.value.key));
return result.value;
};
const second = pinResult(cache.build(await request("second")));
let warm: ReturnType<typeof pinResult> | undefined;
onTestFinished(async () => {
snapshot.release();
touch.release();
await Promise.allSettled(warm === undefined ? [second] : [second, warm]);
for (const release of releases) release();
const closed = await cache.close();
if (!closed.ok) throw closed.error;
});
await expect.poll(() => listing.arrived).toBe(1);
warm = pinResult(cache.build(firstInput));
await expect.poll(() => listing.touched).toBe(true);
listing.names = await actual.readdir(listing.directory);
snapshot.release();
await second;
touch.release();
const reused = await warm;
expect(
await readFile(
join(reused.directory, reused.manifest.files.c.path),
"utf8",
),
).toContain("// Function: main @ 0x1000");
});
it("keeps a shared progress callback registered for the remaining waiter", async () => {
const { kuna, cache, request } = await fixture();
const input = await request("program");
const controller = new AbortController();
const statuses: KunaExportStatus[] = [];
const onStatus = (status: KunaExportStatus) => {
statuses.push(status);
if (status.phase === "decompiling") controller.abort();
};
const first = cache.build({ ...input, signal: controller.signal, onStatus });
const second = cache.build({ ...input, onStatus });
expect(await first).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
const result = await second;
if (!result.ok) throw result.error;
expect(statuses.map(({ phase }) => phase)).toContain("sealing");
expect(
(await fakeKunaCalls(kuna)).filter(
({ argv }) => argv[0] === "decompile-project",
),
).toHaveLength(1);
expect(await cache.close()).toEqual({ ok: true, value: null });
});
@@ -0,0 +1,164 @@
import { expect, it } from "vitest";
import {
parseKunaExportSummary,
parseKunaInventory,
scanKunaProjectSource,
} from "../../../../src/kuna/KunaProjectFiles.js";
const SOURCE = [
'#include "a.out.h"',
"",
"// Function: main @ 0x1000",
"int main(void)",
"{",
' return puts("// Function: not a header");',
"}",
"",
"// Function: operator new(unsigned long) @ 0x1040",
"void *operator new(unsigned long a0) { return malloc(a0); }",
"",
"// Function: sub_1080 @ 0x1080 (error: Flow exceeded maximum allowable instructions)",
"",
"",
].join("\n");
/** Feed the text in fixed-size pieces so headers straddle chunk boundaries. */
async function* chunked(text: string, size: number): AsyncGenerator<Buffer> {
const bytes = Buffer.from(text);
for (let offset = 0; offset < bytes.length; offset += size)
yield bytes.subarray(offset, offset + size);
}
it("splits Kuna's .c into function blocks at any chunk size", async () => {
// SOURCE is ASCII, so string offsets are byte offsets.
const starts = [
SOURCE.indexOf("// Function: main"),
SOURCE.indexOf("// Function: operator new"),
SOURCE.indexOf("// Function: sub_1080"),
SOURCE.length,
];
const span = (index: number) => ({
cOffset: starts[index] ?? -1,
cLength: (starts[index + 1] ?? -1) - (starts[index] ?? -1),
});
const expected = [
{ name: "main", address: "0x1000", ...span(0), error: null },
{
name: "operator new(unsigned long)",
address: "0x1040",
...span(1),
error: null,
},
{
name: "sub_1080",
address: "0x1080",
...span(2),
error: "Flow exceeded maximum allowable instructions",
},
];
expect(SOURCE.slice(starts[0], starts[1])).toBe(
'// Function: main @ 0x1000\nint main(void)\n{\n return puts("// Function: not a header");\n}\n\n',
);
for (const size of [1, 7, 64, 4096])
expect(
await scanKunaProjectSource(chunked(SOURCE, size), {
decompiled: 2,
failed: 1,
}),
).toEqual({ ok: true, value: expected });
const block = Buffer.from(SOURCE)
.subarray(expected[2]?.cOffset, SOURCE.length)
.toString();
expect(block.startsWith("// Function: sub_1080 @ 0x1080 (error:")).toBe(
true,
);
});
it.each([
[
"a header count that disagrees with Kuna's summary",
SOURCE,
{ decompiled: 3, failed: 1 },
"3 function headers but Kuna reported 4",
],
[
"a failure count that disagrees with Kuna's summary",
SOURCE,
{ decompiled: 3, failed: 0 },
"marks 1 failed functions but Kuna reported 0",
],
[
"a malformed header",
"// Function: main at 0x1000\n",
{ decompiled: 1, failed: 0 },
"unrecognized function header at byte 0",
],
[
"a repeated address",
"// Function: a @ 0x10\n\n// Function: b @ 0x10\n\n",
{ decompiled: 2, failed: 0 },
"repeats address 0x10",
],
[
"an overlong header line",
`// Function: ${"x".repeat(70_000)} @ 0x10\n`,
{ decompiled: 1, failed: 0 },
"too long",
],
] as const)("refuses %s", async (_case, source, summary, reason) => {
expect(await scanKunaProjectSource(chunked(source, 4096), summary)).toEqual({
ok: false,
error: expect.stringContaining(reason),
});
});
it("reads Kuna's completed-export summary from its stdout", () => {
expect(
parseKunaExportSummary(
"wrote ./a.out.kuna: a.out.c (325163 bytes), a.out.h (21366 bytes); functions: 172 ok, 3 failed\n",
),
).toEqual({ decompiled: 172, failed: 3 });
expect(parseKunaExportSummary("wrote ./a.out.kuna\n")).toBeUndefined();
});
it("reads the callable inventory, ignoring fields a later release adds", () => {
expect(
parseKunaInventory(
JSON.stringify({
binary: "a.out",
count: 1,
total: 1,
error: null,
functions: [
{
name: "_inflate",
address: 4294995696,
address_hex: "0x100006ef0",
aliases: ["inflate"],
object_location: null,
size: 252,
interior: false,
},
],
}),
),
).toEqual({
ok: true,
value: [
{
name: "_inflate",
address: "0x100006ef0",
aliases: ["inflate"],
size: 252,
},
],
});
expect(parseKunaInventory("functions: 1")).toEqual({
ok: false,
error: "the function inventory is not JSON",
});
expect(
parseKunaInventory(JSON.stringify({ functions: [{ name: "x" }] })),
).toMatchObject({ ok: false });
});
@@ -301,6 +301,51 @@ describe("managed application graph identity facts", () => {
),
).toEqual([]);
});
it("projects a module whose metadata name is empty without a label", () => {
const bytes = buildManagedPeFixture({ ilBody: Buffer.from([0x06, 0x2a]) });
const control = inspectManagedMembersBytes(
bytes,
managedPeFixtureTarget(bytes, "/fixture/EmptyModuleName.dll"),
);
if (control.module === null) throw new Error("Expected a Module row");
// Point the Module row's Name at the string heap's empty entry.
bytes.writeUInt16LE(0, control.module.row_offset + 2);
const binary = managedPeFixtureTarget(
bytes,
"/fixture/EmptyModuleName.dll",
);
const members = inspectManagedMembersBytes(bytes, binary);
expect(members.module?.name).toBe("");
const projected = projectManagedApplicationGraphEvidence({
managed_members: createEvidence(binary, MANAGED_STATIC_PROVIDER, {
operation: "inspect_managed_members",
parameters: {},
result: members,
rawResult: null,
limitations: members.limitations,
locations: [{ kind: "artifact-path", path: binary.path }],
}),
});
if (!projected.ok) throw projected.error;
const graph = parseJavaScriptApplicationGraph(
managedApplicationGraphResultSchema.parse(
parseEvidence(projected.value).normalized_result,
).graph,
);
expect(
graph.nodes.find(({ kind }) => kind === "managed-module")
?.observations[0],
).toMatchObject({ label: null, properties: { name: "" } });
expect(graph.nodes.map(({ kind }) => kind)).toEqual(
expect.arrayContaining([
"managed-module",
"managed-type",
"managed-method",
"managed-field",
]),
);
});
});
it("normalizes MVID casing for node identity while preserving observed values", () => {
+2
View File
@@ -28,6 +28,7 @@ interface FakeV8InspectorOptions {
readonly additionalTargetUrl?: string;
readonly additionalTargetCount?: number;
readonly closeOnMethod?: string;
readonly hangOnMethod?: string;
}
export interface FakeV8Inspector {
@@ -147,6 +148,7 @@ const registerInspectorSocket = (
socket.close();
return;
}
if (options.hangOnMethod === command.method) return;
respondToInspectorCommand(socket, command, options);
socket.send(JSON.stringify({ id: command.id, result: {} }));
});
+17 -12
View File
@@ -1,18 +1,26 @@
import { afterEach, expect, it, onTestFinished, vi } from "vitest";
import { ApktoolProvider } from "../../src/apktool/ApktoolProvider.js";
import { execFileOutput } from "../../src/process/ExecFileOutput.js";
import { runOwnedCommand } from "../../src/process/OwnedCommand.js";
import type { ProviderProcessSnapshot } from "../../src/process/ProviderProcess.js";
vi.mock("../../src/process/ExecFileOutput.js", async (importOriginal) => ({
vi.mock("../../src/process/OwnedCommand.js", async (importOriginal) => ({
...(await importOriginal<
typeof import("../../src/process/ExecFileOutput.js")
typeof import("../../src/process/OwnedCommand.js")
>()),
execFileOutput: vi.fn(async () => ({ stdout: "2.7.0-stub\n", stderr: "" })),
runOwnedCommand: vi.fn(
async () =>
({
stdout: { text: "2.7.0-stub\n" },
stderr: { text: "" },
}) as unknown as ProviderProcessSnapshot,
),
}));
afterEach(() => {
vi.unstubAllEnvs();
vi.mocked(execFileOutput).mockClear();
vi.mocked(runOwnedCommand).mockClear();
});
const lastSpawn = () => vi.mocked(runOwnedCommand).mock.lastCall?.[0];
it.skipIf(process.platform === "win32")(
"keeps an explicit environment's PATH choice and samples ambient inheritance per operation",
@@ -24,16 +32,15 @@ it.skipIf(process.platform === "win32")(
vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/changed-apktool");
vi.stubEnv("REA_APKTOOL_TEST_INHERITED_VALUE", "changed");
await provider.inspectAvailability();
expect(vi.mocked(execFileOutput).mock.lastCall?.[0]).toBe("apktool");
expect(vi.mocked(execFileOutput).mock.lastCall?.[2]?.env).toMatchObject({
expect(lastSpawn()?.command).toBe("apktool");
expect(lastSpawn()?.hostEnvironment).toMatchObject({
REA_APKTOOL_COMMAND: undefined,
REA_APKTOOL_TEST_INHERITED_VALUE: "changed",
});
vi.stubEnv("REA_APKTOOL_TEST_INHERITED_VALUE", undefined);
await provider.inspectAvailability();
expect(
vi.mocked(execFileOutput).mock.lastCall?.[2]?.env
?.REA_APKTOOL_TEST_INHERITED_VALUE,
lastSpawn()?.hostEnvironment?.REA_APKTOOL_TEST_INHERITED_VALUE,
).toBeUndefined();
},
);
@@ -46,8 +53,6 @@ it.skipIf(process.platform === "win32")(
onTestFinished(() => provider.close());
vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/later-apktool");
await provider.inspectAvailability();
expect(vi.mocked(execFileOutput).mock.lastCall?.[0]).toBe(
"/ambient/later-apktool",
);
expect(lastSpawn()?.command).toBe("/ambient/later-apktool");
},
);
Loaded 100 of 124 files, more files were not shown because too many files have changed in this diff. Show more