Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
528da7cc31 | ||
|
|
e9a71efa31 | ||
|
|
fe7bcc3fbe | ||
|
|
736aa946bc | ||
|
|
d929543198 | ||
|
|
e598b4656a | ||
|
|
ec394a2bdf | ||
|
|
7054bbb9ab | ||
|
|
b2257653ca | ||
|
|
b3b5c8b133 | ||
|
|
befbc024c9 | ||
|
|
fc41b24dc4 | ||
|
|
2c90082a69 | ||
|
|
82763824dc | ||
|
|
41e4c06397 | ||
|
|
a5024a9710 | ||
|
|
fc791bb606 | ||
|
|
8ed3d1df72 | ||
|
|
2b5ee72d26 | ||
|
|
2a2a33ad92 | ||
|
|
c7697595d8 | ||
|
|
c9f9b83c6f | ||
|
|
55cef2566a |
No files matched your search
@@ -217,7 +217,12 @@ facets remain unknown; live IDA results are not replayed from snapshots.
|
||||
|
||||
With `REA_KUNA_COMMAND` configured, `provider_id: "kuna"` answers procedure
|
||||
lists, lookups, pseudocode, assembly and `analyze_function` from Kuna's cached
|
||||
whole-program export; it has no cross-references, callers or CFG. A cold first
|
||||
whole-program export. Callers/callees use a separate body-free graph shared by
|
||||
the session; `xrefs` returns exact-address incoming sites and filters forwarding
|
||||
aliases. These queries and inventory lookups do not trigger a C export. Indirect
|
||||
targets remain unknown and can make relationship coverage partial. Reference
|
||||
caches expire on close; a fresh CLI process repeats the native reference walk.
|
||||
Typed references, CFG, raw-memory tools and annotations remain unsupported. A cold first
|
||||
answer can come from a single-function run (its limitations say so); assembly
|
||||
waits for the export. For pseudocode and assembly, `raw_result.source` names the
|
||||
cached `.c`/`.asm` file and byte span, which you may search directly instead of
|
||||
|
||||
@@ -94,6 +94,36 @@ recorded observations, identity, hashes, exports and source ranges; it does not
|
||||
include every child syntax observation, related edge or original source file.
|
||||
An absent item fact is not evidence of absence from the whole application.
|
||||
|
||||
For canonical identities and retained source-range counts, use a typed reference
|
||||
page instead of reading every full item:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "javascript",
|
||||
"collection": "modules",
|
||||
"projection": "references",
|
||||
"where": { "path": "main.js", "kind": "javascript-module" },
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Filters match exact retained strings; omit a filter to include all values, use
|
||||
`path: null` for unavailable paths, and `path: ""` for a recorded empty path.
|
||||
Rows preserve canonical `identity` and distinct observation support states in
|
||||
`basis`. `retained_source_range_count` counts retained source-range observations,
|
||||
not runtime imports or unique reference sites. Use a row's `node_id` with the item
|
||||
view above for exact locations and full supporting observations. Follow
|
||||
`coverage.next_offset`; page exhaustion describes matching retained rows only.
|
||||
Check `source_coverage` for source selection, partial/unknown graph coverage and
|
||||
unknown counts before drawing a negative conclusion.
|
||||
|
||||
## Trace only the remaining question
|
||||
|
||||
Use `trace_application_feature` on the parent application Evidence for one
|
||||
|
||||
@@ -144,6 +144,15 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
|
||||
from an external CDP browser. Real-browser verification checks both outcomes and
|
||||
confirms that an attached external browser remains alive.
|
||||
|
||||
`environment.locale` sets Intl formatting in both modes. Launch also sets
|
||||
`navigator.language`, `navigator.languages`, and the request `Accept-Language`.
|
||||
Connect preserves the caller's page and request languages; every attached
|
||||
capture identifies this locale limitation, including retained partial
|
||||
observations. Use launch mode when the application must select the requested
|
||||
language through navigator values or request headers. Artifact completeness
|
||||
covers the selected observations, not equivalence of launch and connect
|
||||
language environments.
|
||||
|
||||
HTTP(S) URLs may include ordinary query values and fragments directly. Use
|
||||
ordered structured query entries when a value needs to reference a declared
|
||||
environment-backed secret. Form, storage, and cookie values are either literal
|
||||
|
||||
@@ -69,6 +69,12 @@ Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE
|
||||
query also derives the snapshot metadata from the authenticated Ghidra session
|
||||
handshake, so it does not require a preceding load-image call. In a persistent
|
||||
MCP session, load-image followed by type inspection reuses that same derivation.
|
||||
The bridge resolves the caller's address spelling once; metadata lookup uses
|
||||
its canonical default-space identity. Bare hexadecimal, prefixed hexadecimal,
|
||||
and encoded default-space selectors therefore select the same MethodTable.
|
||||
Other address spaces and unresolved identities are not matched by offset alone.
|
||||
Metadata already supplied by the engine is retained without reading or deriving
|
||||
an additional source snapshot.
|
||||
The database result remains `unavailable` when Ghidra has no DataType at that
|
||||
address; its `metadata_recovery.source` is `read-only-derived-overlay` and
|
||||
carries related type, interface, slot, and target observations. Generated
|
||||
|
||||
@@ -193,7 +193,11 @@ these tools before compilation. Keeping this matrix separate lets Linux
|
||||
host/provider acceptance run with only the host compiler.
|
||||
|
||||
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
|
||||
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
|
||||
`-- --x86`, `-- --arm`, or `-- --arm64` selects another target architecture;
|
||||
`-- --dll` selects DLLs. Choose one architecture and optionally combine it with
|
||||
`--dll`. These static fixtures contain source-owned encoded instructions; they
|
||||
are not compiler-produced programs. See the [Windows verification scope](windows-ghidra-p0.md#native-authority-and-target-identity)
|
||||
for real-provider coverage and exclusions.
|
||||
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
and project, endpoint, process, and runtime cleanup. The independent native
|
||||
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
||||
|
||||
@@ -797,7 +797,7 @@ and `set_addresses_names` name a function entry or any other mapped address
|
||||
session metadata edits leave executable bytes unchanged and are discarded on close. GUI controls
|
||||
require Hopper; Windows P0 remains read-only.
|
||||
|
||||
Windows P0 admits native x86 and x86-64 PE applications and DLLs, plus explicitly
|
||||
Windows P0 admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs, plus explicitly
|
||||
selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied
|
||||
entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain
|
||||
unsupported on Windows.
|
||||
|
||||
@@ -6,7 +6,7 @@ are `inspect_analysis_view`, `trace_application_feature`,
|
||||
`trace_javascript_semantics`, `compare_application_versions`,
|
||||
`compare_source_to_bundle`, and `compare_javascript_export_shapes`; their CLI
|
||||
equivalents use the same names with hyphens. `inspect_analysis_view` projects a
|
||||
summary, module page, or one module from already completed application
|
||||
summary, module page, typed module references, or one module from already completed application
|
||||
Evidence; it does not walk relationships.
|
||||
|
||||
`inspect_analysis_view` also projects retained `inspect_binary_layout`
|
||||
@@ -40,6 +40,46 @@ combine its fields with the fact's location. Semantic trace results include the
|
||||
canonical context subset referenced by their returned facts, so those facts
|
||||
remain self-contained in the response.
|
||||
|
||||
## Retained module facts
|
||||
|
||||
For canonical identities and retained source-range counts, query the existing
|
||||
`inspect_analysis_view` tool with a typed module-reference projection:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "javascript",
|
||||
"collection": "modules",
|
||||
"projection": "references",
|
||||
"where": { "path": "main.js", "kind": "javascript-module" },
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace the parent ID with the actual analysis summary's `parent_evidence_id`.
|
||||
Filters match exact retained paths and kinds. Omission includes all values;
|
||||
`path: null` selects an unavailable path and `path: ""` selects a recorded empty
|
||||
path. Multiple identities can share a path. The default offset is 0 and limit
|
||||
is 64; follow `coverage.next_offset` for later matching rows in retained order.
|
||||
|
||||
Each row contains its `node_id`, kind, path, canonical identity, retained
|
||||
source-range observation count, and distinct supporting authority/state/confidence
|
||||
combinations in `basis`. These counts do not prove unique reference sites or
|
||||
runtime loading. Use the existing module item view with the row's `node_id` for
|
||||
exact source ranges and complete observations. The result retains parent/query
|
||||
Evidence identity, artifact identity, limitations and unknowns. `source_coverage`
|
||||
keeps selected/unselected sources, graph coverage and unknown counts separate
|
||||
from page exhaustion; an exhausted or empty page cannot prove absence from an
|
||||
incompletely analyzed application. CLI `inspect-analysis-view` reads the same
|
||||
request from a JSON file; portable input uses `source.kind: "inline"` with the
|
||||
complete parent Evidence.
|
||||
|
||||
## Feature tracing
|
||||
|
||||
Select one literal seed kind: node ID, route, string, API, IPC channel, module,
|
||||
|
||||
+37
-14
@@ -18,34 +18,37 @@ Select it explicitly with `--provider kuna`, MCP's `open_binary.provider_id: "ku
|
||||
| `REA_KUNA_COMMAND` | none (required) | Absolute path of the `kuna` executable. |
|
||||
| `REA_KUNA_CACHE_DIR` | `~/Library/Caches/rea/kuna` (macOS), `$XDG_CACHE_HOME/rea/kuna` (Linux; `~/.cache/rea/kuna` when `XDG_CACHE_HOME` is unset or relative) | Absolute directory REA owns for project exports. Kuna's own spec cache also lives here, not in your Kuna cache. |
|
||||
| `REA_KUNA_CACHE_MAX_BYTES` | 4 GiB | Size cap; least recently used exports are evicted above it. |
|
||||
| `REA_KUNA_PROJECT_TIMEOUT_MS` | 30 minutes | Deadline for one project export. |
|
||||
| `REA_KUNA_TIMEOUT_MS` | 120 s | Deadline for one single-function decompile. |
|
||||
| `REA_KUNA_PROJECT_TIMEOUT_MS` | 30 minutes | Deadline for whole-program inventory/export, reference and string scans. |
|
||||
| `REA_KUNA_TIMEOUT_MS` | 120 s | Deadline for one single-function decompile, and for the inventory run REA makes when the export ends without one. |
|
||||
| `REA_KUNA_JOBS` | 1 | `decompile-project --jobs`. 1 is Kuna's reference answer; more jobs finish sooner but lose callee-first prototypes, and change about 40–60% of function bodies on the measured binaries. |
|
||||
|
||||
Windows is not supported yet. Supported targets are ELF and thin Mach-O executables for x86-64, AArch64 and ARM; PE and universal Mach-O are rejected with a typed reason.
|
||||
|
||||
## What Kuna answers
|
||||
|
||||
| Tool | Source |
|
||||
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots it does not decompile |
|
||||
| `address_name` | Function entries, then the data labels Kuna's export assigns |
|
||||
| `procedure_pseudo_code`, `batch_decompile` | The project export, or a single-function run while the export builds |
|
||||
| `procedure_assembly` | The project export's labeled listing |
|
||||
| `analyze_function` | Pseudocode and assembly from the export; callers, callees, references, comments, strings and CFG are declared unavailable |
|
||||
| Tool | Source |
|
||||
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots it does not decompile |
|
||||
| `address_name` | Function entries, then the data labels Kuna's export assigns |
|
||||
| `procedure_pseudo_code`, `batch_decompile` | The project export, or a single-function run while the export builds |
|
||||
| `procedure_assembly` | The project export's labeled listing |
|
||||
| `procedure_callers`, `procedure_callees` | Call-kind edges from a lazy, body-free Kuna graph; synthetic import nodes are accepted by address |
|
||||
| `list_strings`, `search_strings` | Typed string candidates from one cached, reference-free scan; exact-address, literal and ECMAScript regex selection |
|
||||
| `xrefs` | Exact-address incoming reference sites, with forwarding aliases filtered explicitly |
|
||||
| `analyze_function` | Pseudocode and assembly from the export, plus call relationships; typed references, comments, strings and CFG remain unavailable |
|
||||
|
||||
Cross-references, callers, callees and basic blocks are not available from this provider. Use another provider when you need them.
|
||||
Typed procedure references, basic blocks, raw-memory tools, annotations and type recovery are not available from this provider. Use another provider for those operations.
|
||||
|
||||
## How answers are produced
|
||||
|
||||
The first analysis request in a session looks for this target's export in the cache. `list_documents` reads the selected path without initializing analysis. The export is keyed by the target's SHA-256, the Kuna executable's SHA-256, its version and `--jobs`.
|
||||
The first text or assembly request in a session looks for this target's export in the cache. `list_documents` reads the selected path without initializing analysis. Inventory, function lookup, string queries, call relationships and xrefs do not start a C export. A data-name lookup starts it only when the address is absent from the function inventory. The export is keyed by the target's SHA-256, the Kuna executable's SHA-256, its version and `--jobs`.
|
||||
|
||||
- **Warm cache:** analysis reads reuse the verified export. A fresh server verifies its file digests once, even for concurrent first requests; that cost scales with export size.
|
||||
- **Cold cache:**
|
||||
- REA starts `kuna decompile-project` in the background on a private copy of the target.
|
||||
- Meanwhile, pseudocode comes from one `kuna decompile --addr` run, and listings from `kuna functions --json`.
|
||||
- Meanwhile, pseudocode comes from one `kuna decompile --addr` run. Name and address lookups, and listings, use the inventory the export takes with `kuna functions --json` before it decompiles. The session runs `kuna functions` itself, under `REA_KUNA_TIMEOUT_MS`, only when no export is building (an inventory-only request does not start one) or the export ends without an inventory.
|
||||
- Requests for the same function share one run; it stops when its last waiting request is cancelled. Each run analyzes the whole image again in its own process, so one REA process runs at most as many at once as the host has cores beyond `REA_KUNA_JOBS` (at least one). Further requests wait in order, and a waiting request is answered from the export if it finishes first. A run whose stop REA cannot prove keeps its slot until `close_binary` proves it stopped.
|
||||
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds.
|
||||
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
|
||||
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
|
||||
|
||||
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
|
||||
@@ -54,10 +57,30 @@ The background export belongs to the session. `close_binary`, or the end of a CL
|
||||
|
||||
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
|
||||
|
||||
Regex procedure search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline. Literal matching remains the default. Cancellation or a matching deadline leaves the session available.
|
||||
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline. Literal matching remains the default. Cancellation or a matching deadline leaves the session available.
|
||||
|
||||
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
|
||||
|
||||
## Strings
|
||||
|
||||
`list_strings` and `search_strings` share one lazy `kuna strings --json --encoding all --min-length 5 --termination any --no-xrefs` scan of the digest-checked private input. Skipping references avoids program lifting, a call graph and a C export. REA validates the input, scan policy, every row and the total before caching any facts. Encoding, character count, byte length and terminator facts must agree; supplementary UTF-8 characters count as Unicode scalars. Malformed or oversized output is an error, not an empty search. JSON admits at most 16 MiB before parsing. String scans use the whole-program `REA_KUNA_PROJECT_TIMEOUT_MS` deadline; `REA_KUNA_TIMEOUT_MS` applies to single-function decompilation. The parsed inventory belongs to one selected target/executable profile and expires on session close; deterministic output-buffer failures are retained until close, while other failures are not cached.
|
||||
|
||||
Results keep exact u64 virtual addresses, decoded text, native encoding, terminator status and byte length including any terminator. Kuna scans mapped sections or segments for runs of at least five characters; unterminated fragments can include instruction bytes. `all` includes ASCII, UTF-8 and the engine's Latin-range UTF-16LE reading. Other text, including short strings, non-Latin wide text and UTF-16BE, can remain undiscovered. An empty result does not prove absence.
|
||||
|
||||
Use `search_strings` for the matching rows or `list_strings` with an exact address for that location. Literal and regex matching run in REA, preserving the existing case and ECMAScript Unicode semantics instead of substituting Kuna's different regex dialect. Procedure and string regex queries share one worker slot per session. Input projections are materialized only after admission, so queued queries retain no per-query inventory arrays. Close cancels and drains matching. Native metadata queries also share one admission slot, including references; a failed stop retains admission until cleanup is proven.
|
||||
|
||||
`raw_result.strings` reports the source JSON digest/size, scanned scope, extraction policy, complete inventory count and selected count without copying unrequested text into the answer. Omit the address from `list_strings` to retrieve the complete inventory as Evidence; normal session retention and bundle export apply, including transport-capacity errors for oversized inline replies. These operations preserve their complete-result contracts and do not add paging. Function-referenced string facets in `analyze_function` remain unavailable; call `xrefs` explicitly when references to one returned string are needed.
|
||||
|
||||
## Calls and references
|
||||
|
||||
The first call query runs `kuna decompile-graph --functions ''`, admitting no C or assembly bodies. One validated graph and forward/reverse indexes serve later call queries and existing `get_call_graph` / `trace_call_path` workflows. Numeric u64 addresses are parsed from their original JSON tokens. Only `call` edges become callers/callees; jumps and data references retain their distinct meaning. Graph counts, edge endpoints and ordering are validated before any relationship is returned. A forwarding address may be an instruction inside a function, so it is retained separately and never promoted to a function node or call edge. Reported indirect calls make affected relationship coverage partial; no target is invented for them.
|
||||
|
||||
`xrefs` runs native `xrefs --to ADDRESS --json` on demand. Kuna unifies forwarding aliases and suppresses edges internal to their class; REA filters to the requested address and records the excluded-row count. Duplicate source sites appear once. An empty result cannot establish the absence of references. Addresses are explicit u64 values or exact, unambiguous procedure names; Kuna has no cursor.
|
||||
|
||||
Graph JSON admits at most 32 MiB before parsing; xref JSON at most 16 MiB. A large image can exceed the graph budget: an arm64 Firefox `XUL` (404k functions) does, while a 14 MB Sublime Text binary produced 31.8 MB. Output beyond a buffer is reported as a `memory` resource constraint with `retryable: false`, because the same target and executable produce the same output. The session remembers a graph that overflowed, so later call queries fail at once instead of repeating the whole-program walk. The graph is cached for the selected target and executable profile within one session. Xref reuse is an LRU charged by source bytes plus metadata, limited to 16 MiB and 128 entries; at most eight distinct native xref queries may be pending. Reference commands run one at a time per session and share the provider's foreground budget with single-function decompiles. An unproven process stop retains its admission slots until cleanup succeeds. Concurrent callers share work, individual cancellation releases only that waiter, and the last departing waiter cancels its producer and receives any unproven cleanup. Session close aborts and drains producers, releases parsed caches and retries any retained process/file cleanup.
|
||||
|
||||
Evidence includes only selected addresses or dossier neighbors, the producing JSON's SHA-256/byte count, coverage and relevant function facts; unrequested program-wide JSON and instruction text are omitted. Reference caches are in memory and expire on close: a fresh CLI process or MCP server rebuilds them. A first call query therefore costs a whole-program reference walk even when a C export is warm. These operations do not establish full Ghidra/IDA feature parity.
|
||||
|
||||
## Provenance and limitations
|
||||
|
||||
- Addresses and bytes are Kuna's observations of the image. Procedure entries and extents are its derivations: extents are upper bounds clipped at the next entry, and Kuna can split one function at an entry its own analysis infers. Pseudocode is its reconstruction, never original source.
|
||||
|
||||
@@ -352,7 +352,7 @@ install it or its toolchain. The [NativeAOT guide](ghidra-nativeaot.md) defines
|
||||
these routes, their layout/host restrictions, derived-memory provenance, and
|
||||
verification coverage.
|
||||
|
||||
REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
|
||||
REA's experimental Windows Ghidra boundary admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE
|
||||
applications and DLLs for static analysis. For x64 PE NativeAOT, the read-only
|
||||
parser also runs there; the [NativeAOT guide](ghidra-nativeaot.md) lists what a
|
||||
Windows host covers.
|
||||
|
||||
@@ -497,7 +497,10 @@ declared by their tools. Artifact extraction materializes the selected regular
|
||||
files into a fresh REA-chosen temporary directory and reports its path.
|
||||
|
||||
`analyze_javascript_application` reads the supplied local directory or ASAR path
|
||||
and returns its result and Evidence inline.
|
||||
and returns its result and Evidence inline. Its owned analysis worker and private
|
||||
temporary source files are declared as `launchesProcess` and `writesFilesystem`,
|
||||
including for a single JavaScript source. This internal analysis does not execute
|
||||
the supplied application code.
|
||||
|
||||
## Ghidra first-query deadlines and recovery
|
||||
|
||||
|
||||
+6
-2
@@ -111,8 +111,12 @@ list does not establish that the target has no procedures.
|
||||
Suggestions are Unicode-normalized, case-insensitive prefix matches. The server
|
||||
deduplicates them and sorts by code point for deterministic results. Procedure
|
||||
completion derives candidates from the complete inventory, so a name is offered
|
||||
only when it identifies one address in that inventory. Completion returns all
|
||||
matching values; it does not expose paging controls.
|
||||
only when it identifies one address in that inventory. A completion reply
|
||||
carries at most 100 values, the MCP limit. `total` counts every match and
|
||||
`hasMore` is `true` when values were omitted. Completion has no paging
|
||||
controls. A longer prefix reaches an omitted value only when it excludes other
|
||||
matches; matching uses normalized, case-insensitive forms, so a value can stay
|
||||
unreachable when more than 100 matches share its normalized prefix.
|
||||
|
||||
## Lifecycle and safety
|
||||
|
||||
|
||||
@@ -497,7 +497,7 @@ is present. A denied trace reports the `debugger-attach-denied` reason.
|
||||
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
|
||||
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
|
||||
the matching native decompiler. Experimental Windows x64 P0 admits native
|
||||
x86 and x86-64 PE applications and DLLs, plus explicitly selected
|
||||
x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs, plus explicitly selected
|
||||
[raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied load address,
|
||||
on fixed local NTFS volumes. Other raw CPU families remain unsupported on Windows.
|
||||
The boundary uses bundled Job Object ownership, protected runtime DACLs, and
|
||||
|
||||
+6
-2
@@ -821,7 +821,10 @@ The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local naviga
|
||||
|
||||
`npm run verify:kuna` requires a caller-supplied `kuna` in `REA_KUNA_COMMAND`
|
||||
and a host C compiler (`REA_CC`, default `cc`) with `strip` and `nm`. It builds a
|
||||
source-owned fixture with symbols and stripped, then verifies through MCP that
|
||||
source-owned fixture with non-inlined calls, with symbols and stripped, then verifies through MCP that
|
||||
callers, callees and exact incoming xrefs agree with those calls, graph/path
|
||||
composition reaches the callee, reference-only queries leave project/staging
|
||||
entries unchanged, repeated relationships reuse the graph, and
|
||||
the first cold pseudocode answer comes from a single-function run or an export
|
||||
that already finished, assembly waits for and serves from that export, and
|
||||
pseudocode and both dossier facets match their recorded file spans and digests,
|
||||
@@ -834,7 +837,8 @@ caller-built functions through Ghidra (it refuses to run without `--targets` and
|
||||
`--functions`, an empty matrix, or names missing from every supplied target) and
|
||||
reports latency, sampled process-tree CPU and peak memory, UTF-8 response size,
|
||||
and goto and warning-marker counts. Those counts are not a correctness judgement,
|
||||
and model tokens are not measured. `--report PATH` writes the JSON report.
|
||||
and model tokens are not measured. Reference timings and response sizes are
|
||||
reported separately; reference caches expire when the server closes. `--report PATH` writes the JSON report.
|
||||
|
||||
The first matrix run used Kuna 1.833 and Ghidra 12.1.4 with JDK 21 on macOS
|
||||
arm64. It covered zlib 1.3.2 `minigzip`, coreutils 9.9 `sort` and the sqlite
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
{
|
||||
"base_revision": "e598b4656ad90d59ede2d10454a0ac34f3813558",
|
||||
"scope": "Canonical effect metadata only; real static Obsidian reads, no target execution",
|
||||
"baseline_source_sha256": {
|
||||
"src/contracts/toolEffects.ts": "39505b5f1501768515229a37ff67e8a1a512faedfb6669aee7a690a2ac3ac633",
|
||||
"src/javascript/analysis/JavaScriptAnalysisWorker.ts": "49ea4e6344060fae3529ffe6e14bfadc534132067cfa5c370263dc7476375be0",
|
||||
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
|
||||
},
|
||||
"final_source_sha256": {
|
||||
"src/contracts/toolEffects.ts": "feb66f1b71c4a6849d1730227c9c74e4efbb67b4537d75fd41ce9ff615c25753",
|
||||
"src/javascript/analysis/JavaScriptAnalysisWorker.ts": "49ea4e6344060fae3529ffe6e14bfadc534132067cfa5c370263dc7476375be0",
|
||||
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
|
||||
"tests/boundary/mcp/electronMcp.test.ts": "e483bec8512cfeafa7f951c8faf25e1aecd6e6987ea74aead14230501912928d",
|
||||
"tests/boundary/mcp/toolSchemaValidity.test.ts": "ab6d9ba37035a6ff9279b820aa8660c50a3ef9f31d25f838c0ba40929148e89c"
|
||||
},
|
||||
"cohorts": {
|
||||
"baseline": {
|
||||
"metadata": {
|
||||
"tool": "analyze_javascript_application",
|
||||
"advertised_annotations": {
|
||||
"readOnlyHint": false,
|
||||
"destructiveHint": false,
|
||||
"idempotentHint": true,
|
||||
"openWorldHint": false
|
||||
},
|
||||
"session_effects": ["mutatesSession", "idempotent"],
|
||||
"session_annotations": ["idempotent"],
|
||||
"catalog_tool_count": 207
|
||||
},
|
||||
"metadata_probe_resources": {
|
||||
"elapsed_seconds": 10.711,
|
||||
"cpu_seconds": 10.503415,
|
||||
"peak_cgroup_bytes": 842891264,
|
||||
"payload_cleanup_verified": true,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"real_workflow": {
|
||||
"artifact": {
|
||||
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
|
||||
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
|
||||
},
|
||||
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 109.19401700000162,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"token_costs": {
|
||||
"tokenizer": "cl100k_base",
|
||||
"tiktoken_version": "0.12.0",
|
||||
"costs": {
|
||||
"references": {
|
||||
"selected_catalog": 7977,
|
||||
"full_catalog": 195522,
|
||||
"common_request": 53,
|
||||
"common_summary": 2675,
|
||||
"requests": 102,
|
||||
"structured_replies": 5741,
|
||||
"selected_catalog_task_total": 16548,
|
||||
"full_catalog_task_total": 204093
|
||||
}
|
||||
}
|
||||
},
|
||||
"cli_parity": {
|
||||
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"exact_cli_mcp_view_parity": true
|
||||
}
|
||||
},
|
||||
"real_workflow_resources": {
|
||||
"elapsed_seconds": 30.47,
|
||||
"cpu_seconds": 29.939807,
|
||||
"peak_cgroup_bytes": 1551282176,
|
||||
"payload_cleanup_verified": true,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"final": {
|
||||
"metadata": {
|
||||
"tool": "analyze_javascript_application",
|
||||
"advertised_annotations": {
|
||||
"readOnlyHint": false,
|
||||
"destructiveHint": false,
|
||||
"idempotentHint": true,
|
||||
"openWorldHint": true
|
||||
},
|
||||
"session_effects": [
|
||||
"mutatesSession",
|
||||
"writesFilesystem",
|
||||
"launchesProcess",
|
||||
"idempotent"
|
||||
],
|
||||
"session_annotations": ["idempotent", "open_world"],
|
||||
"catalog_tool_count": 207
|
||||
},
|
||||
"metadata_probe_resources": {
|
||||
"elapsed_seconds": 10.909,
|
||||
"cpu_seconds": 10.664997,
|
||||
"peak_cgroup_bytes": 853958656,
|
||||
"payload_cleanup_verified": true,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"real_workflow": {
|
||||
"artifact": {
|
||||
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
|
||||
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
|
||||
},
|
||||
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 110.17946600000141,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"token_costs": {
|
||||
"tokenizer": "cl100k_base",
|
||||
"tiktoken_version": "0.12.0",
|
||||
"costs": {
|
||||
"references": {
|
||||
"selected_catalog": 7977,
|
||||
"full_catalog": 195522,
|
||||
"common_request": 53,
|
||||
"common_summary": 2675,
|
||||
"requests": 102,
|
||||
"structured_replies": 5741,
|
||||
"selected_catalog_task_total": 16548,
|
||||
"full_catalog_task_total": 204093
|
||||
}
|
||||
}
|
||||
},
|
||||
"cli_parity": {
|
||||
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"exact_cli_mcp_view_parity": true
|
||||
}
|
||||
},
|
||||
"real_workflow_resources": {
|
||||
"elapsed_seconds": 31.059,
|
||||
"cpu_seconds": 30.663163,
|
||||
"peak_cgroup_bytes": 1584574464,
|
||||
"payload_cleanup_verified": true,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"conditions": "Separate fresh MCP sessions, uncontrolled warm filesystem/build and shared-host scheduling. One CPU, 8 GiB hard cap, 6 GiB early stop, no swap. Resource counters are at payload exit with the observer alive, not final whole-scope counters; memory includes charged file cache, not summed RSS. Whole real-verifier resources include analysis, export, CLI parity and tokenizer; metadata probe resources are separate. Static BPE uses name/description/inputSchema catalog projections plus requests/structured replies, excludes annotations, metadata-probe calls, rendering/replay/reasoning/final-answer/model usage. No performance improvement or equivalence bound established by one before/after pair.",
|
||||
"regressions": {
|
||||
"baseline": "20 executed: 2 expected metadata failures, 18 passed",
|
||||
"final": "27 passed after the test ownership cleanup across canonical effects and MCP/catalog boundaries",
|
||||
"interim_correction": "26 passed before metadata assertions were split into their own target-free case to satisfy function-size lint"
|
||||
},
|
||||
"selected_input": {
|
||||
"input_path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
|
||||
"detail": "summary",
|
||||
"source_paths": ["package.json", "main.js"]
|
||||
},
|
||||
"selected_view": {
|
||||
"kind": "javascript",
|
||||
"collection": "modules",
|
||||
"projection": "references",
|
||||
"where": {
|
||||
"path": "main.js",
|
||||
"kind": "javascript-module"
|
||||
}
|
||||
},
|
||||
"observer_sources_sha256": {
|
||||
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
|
||||
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
|
||||
},
|
||||
"runtime": {
|
||||
"platform": "Linux x64",
|
||||
"node": "24.18.0",
|
||||
"npm": "11.16.0",
|
||||
"tiktoken": "0.12.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
# JavaScript analysis effect metadata verification
|
||||
|
||||
Tracks #3867 under #4027. The canonical effect record omitted existing internal
|
||||
worker launches and source staging. `CapabilityInventory` and SDK registration
|
||||
already consume that owner directly; no adapter override is required. The
|
||||
[worker](../../src/javascript/analysis/JavaScriptAnalysisWorker.ts) creates a
|
||||
private runtime root, stages `.source` files and launches an owned REA worker.
|
||||
These are source-derived facts; the analyzed application code is not executed.
|
||||
|
||||
## Executed results
|
||||
|
||||
Public stdio MCP before/after:
|
||||
|
||||
| Surface | Before | After |
|
||||
| ---------------------------- | ------------------------------ | ------------------------------------------ |
|
||||
| `tools/list` `openWorldHint` | `false` | `true` |
|
||||
| `binary_session` effects | `mutatesSession`, `idempotent` | also `writesFilesystem`, `launchesProcess` |
|
||||
| Session annotation flags | `idempotent` | also `open_world` |
|
||||
| Advertised tools | 207 | 207 |
|
||||
|
||||
Regression assertions failed on the original contract: **2 failed,
|
||||
18 passed**. After correction and separating the target-free metadata case, **27 passed** across canonical effects, public
|
||||
Electron/JavaScript MCP and SDK-converted catalog validity. The tests retain the
|
||||
non-destructive, idempotent hints and exact session effects; no foreign target
|
||||
execution is declared.
|
||||
|
||||
The existing real verifier analyzed official Obsidian 1.14.4 with `package.json`
|
||||
and `main.js` selected, then read the typed module references and verified exact
|
||||
public CLI/MCP parity. Both revisions returned the same parent/view Evidence
|
||||
identities and 23-row fact digest. Query replies remained 22,158 bytes and the
|
||||
selected-schema task remained 16,548 `cl100k_base` BPE tokens (204,093 with the full
|
||||
name/description/inputSchema catalog projection). These are static counts, not
|
||||
model billing; annotations, the separate metadata probe, host rendering/replay,
|
||||
reasoning and final answers are outside this projection. No hosted agent was run.
|
||||
|
||||
| Complete real verifier | Before | After |
|
||||
| ---------------------- | ------------: | ------------: |
|
||||
| Elapsed seconds | 30.470 | 31.059 |
|
||||
| CPU seconds | 29.939807 | 30.663163 |
|
||||
| Peak cgroup bytes | 1,551,282,176 | 1,584,574,464 |
|
||||
|
||||
The scope includes shared analysis, temporary export, CLI parity and tokenizer.
|
||||
The metadata-only probe separately used 10.711/10.909 s wall,
|
||||
10.503415/10.664997 s CPU and 842,891,264/853,958,656 B peak cgroup memory.
|
||||
All scopes used one CPU, an 8 GiB hard cap, 6 GiB early stop and no swap; cleanup
|
||||
was verified with no OOM. Memory includes charged file cache, not summed RSS.
|
||||
Counters were observed at payload exit while the observer remained alive.
|
||||
Fresh MCP sessions do not establish an OS-cold cache. Shared-host scheduling and
|
||||
filesystem/build warmness were uncontrolled; one pair establishes neither a
|
||||
performance improvement nor an equivalence bound. Linux x64 static ASAR analysis
|
||||
was executed; live Electron and other platforms were not exercised locally.
|
||||
[Compact records and source hashes](javascript-analysis-effects-2026-10-15.json)
|
||||
preserve these distinctions without archiving the downloadable application.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Use pinned Node/npm, `npm ci`, then `npm run build:cached`. Run the following as
|
||||
an ES module from the checkout root (`node --input-type=module < probe.mjs`):
|
||||
|
||||
```javascript
|
||||
import assert from "node:assert/strict";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { toolAvailabilityInventory } from "./dist/contracts/toolOutputSchemaPrimitives.js";
|
||||
import { closeVerifierResources } from "./scripts/lib/verifier-cleanup.mjs";
|
||||
|
||||
const client = new Client({
|
||||
name: "javascript-effect-verification",
|
||||
version: "1",
|
||||
});
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: ["scripts/rea.mjs", "mcp"],
|
||||
stderr: "pipe",
|
||||
});
|
||||
let primaryError;
|
||||
try {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.resume();
|
||||
const { tools } = await client.listTools();
|
||||
const tool = tools.find(
|
||||
({ name }) => name === "analyze_javascript_application",
|
||||
);
|
||||
assert.ok(tool);
|
||||
const response = await client.callTool({
|
||||
name: "binary_session",
|
||||
arguments: {},
|
||||
});
|
||||
assert.notEqual(response.isError, true);
|
||||
const inventory = toolAvailabilityInventory.parse(
|
||||
response.structuredContent?.result?.tool_availability,
|
||||
);
|
||||
const group = inventory.groups.find(({ names }) => names.includes(tool.name));
|
||||
assert.ok(group);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
tool: tool.name,
|
||||
advertised_annotations: tool.annotations,
|
||||
session_effects: group.effects,
|
||||
session_annotations: group.annotations,
|
||||
catalog_tool_count: tools.length,
|
||||
}),
|
||||
);
|
||||
} catch (cause) {
|
||||
primaryError = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
await closeVerifierResources(
|
||||
[() => client.close(), () => transport.close()],
|
||||
primaryError,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
For the real workflow, follow the artifact acquisition/digest instructions in
|
||||
[module-reference verification](javascript-module-references-2026-10-15.md).
|
||||
Run its existing command with `--mode references --rounds 1 --cli-parity` and the
|
||||
same selected source paths and tokenizer. That verifier is unchanged by this
|
||||
fix. The preserved 8 GiB cgroup observer and its source hashes are documented in
|
||||
the same guide. The machine-readable records separate metadata-probe resources
|
||||
from the complete real-workflow resources.
|
||||
@@ -0,0 +1,691 @@
|
||||
{
|
||||
"base_revision": "c9f9b83c6f5878f1d5b102ab206a2ce214749258",
|
||||
"source_sha256": {
|
||||
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
|
||||
"src/contracts/analysisViewToolContracts.ts": "f7c64b3472e7ec1090f9d47ede864b8c9630d506d896454a800747f2c90a8960",
|
||||
"src/domain/analysisView/analysisView.ts": "854b03988791c19642b01d17813b379b51d8edc10d2190a3c236c9a1b2feb365",
|
||||
"src/domain/analysisView/javascriptApplicationView.ts": "f2c41ad15fdc9f891412c9161a832b60baf3dae55e3c4d60159098d07f7d1ef1",
|
||||
"src/domain/analysisView/javascriptModuleReferences.ts": "34c7a7ff466e9c49d43f1b11b72f878041de61a1bc09086ec190a5833b454806",
|
||||
"scripts/fixtures/tokenizer-family-wrapper.py": "2e9b7092ee539ae1c6a477492f05bb6007d4612dadfff9b90051d621de43049b"
|
||||
},
|
||||
"verification": {
|
||||
"scope": "Completed module-fact reads from identical retained Evidence; no hosted-model performance claim",
|
||||
"artifact": {
|
||||
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
|
||||
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
|
||||
},
|
||||
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
|
||||
"source_selection": {
|
||||
"requested_paths": ["package.json", "main.js"],
|
||||
"unselected_files": 36
|
||||
},
|
||||
"source_coverage": {
|
||||
"application": {
|
||||
"status": "partial",
|
||||
"truncated": false,
|
||||
"omitted_count": null,
|
||||
"limits": []
|
||||
},
|
||||
"semantic": {
|
||||
"status": "partial",
|
||||
"truncated": false,
|
||||
"omitted_nodes": null,
|
||||
"omitted_relations": null,
|
||||
"limits": [],
|
||||
"families": [
|
||||
{
|
||||
"family": "boundary",
|
||||
"status": "partial",
|
||||
"retained_relations": 14,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "call-flow",
|
||||
"status": "partial",
|
||||
"retained_relations": 845,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 1278
|
||||
},
|
||||
{
|
||||
"family": "child-process",
|
||||
"status": "partial",
|
||||
"retained_relations": 0,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "closure",
|
||||
"status": "partial",
|
||||
"retained_relations": 1187,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "configuration",
|
||||
"status": "partial",
|
||||
"retained_relations": 0,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "data-flow",
|
||||
"status": "partial",
|
||||
"retained_relations": 9388,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 1135
|
||||
},
|
||||
{
|
||||
"family": "event",
|
||||
"status": "partial",
|
||||
"retained_relations": 127,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 89
|
||||
},
|
||||
{
|
||||
"family": "object-flow",
|
||||
"status": "partial",
|
||||
"retained_relations": 1462,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 2292
|
||||
},
|
||||
{
|
||||
"family": "promise-ownership",
|
||||
"status": "partial",
|
||||
"retained_relations": 80,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 98
|
||||
},
|
||||
{
|
||||
"family": "request",
|
||||
"status": "partial",
|
||||
"retained_relations": 0,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "resource-lifecycle",
|
||||
"status": "partial",
|
||||
"retained_relations": 4,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
},
|
||||
{
|
||||
"family": "timer",
|
||||
"status": "partial",
|
||||
"retained_relations": 8,
|
||||
"omitted_relations": null,
|
||||
"unknown_count": 31
|
||||
}
|
||||
]
|
||||
},
|
||||
"application_unknowns": 36,
|
||||
"semantic_unknowns": 5109
|
||||
},
|
||||
"where": {
|
||||
"path": "main.js",
|
||||
"kind": "javascript-module"
|
||||
},
|
||||
"preparation_ms": 20978.249922,
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 363.02500200000213,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 14.895039999999426,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 12.951290000000881,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 232.7533779999976,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 235.16672899999685,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 10.566158999998152,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"token_costs": {
|
||||
"tokenizer": "cl100k_base",
|
||||
"tiktoken_version": "0.12.0",
|
||||
"costs": {
|
||||
"legacy": {
|
||||
"selected_catalog": 7977,
|
||||
"full_catalog": 195446,
|
||||
"common_request": 53,
|
||||
"common_summary": 2675,
|
||||
"requests": 2836,
|
||||
"structured_replies": 71761,
|
||||
"selected_catalog_task_total": 85302,
|
||||
"full_catalog_task_total": 272771
|
||||
},
|
||||
"references": {
|
||||
"selected_catalog": 7977,
|
||||
"full_catalog": 195446,
|
||||
"common_request": 53,
|
||||
"common_summary": 2675,
|
||||
"requests": 102,
|
||||
"structured_replies": 5741,
|
||||
"selected_catalog_task_total": 16548,
|
||||
"full_catalog_task_total": 204017
|
||||
}
|
||||
}
|
||||
},
|
||||
"cli_parity": {
|
||||
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"exact_cli_mcp_view_parity": true
|
||||
},
|
||||
"conditions": "First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children."
|
||||
},
|
||||
"resources": {
|
||||
"scope": "Complete verifier, MCP/analysis descendants, temporary export, CLI parity and tokenizer; cgroup memory includes charged file cache and is not summed RSS",
|
||||
"elapsed_seconds": 35.797,
|
||||
"cpu_seconds": 35.197074,
|
||||
"peak_cgroup_bytes": 1938313216,
|
||||
"memory_max_bytes": 8589934592,
|
||||
"memory_early_stop_bytes": 6442450944,
|
||||
"cpu_quota_percent": 100,
|
||||
"payload_exit_code": 0,
|
||||
"payload_cleanup_verified": true,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
},
|
||||
"observer_sources_sha256": {
|
||||
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
|
||||
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
|
||||
}
|
||||
},
|
||||
"prior_attempts": [
|
||||
{
|
||||
"scope": "Initial verifier incorrectly requested full MCP delivery; completed analysis exceeded existing response admission",
|
||||
"elapsed_seconds": 21.971,
|
||||
"cpu_seconds": 21.78546,
|
||||
"peak_cgroup_bytes": 1080860672,
|
||||
"outcome": "resource_constraint transport; verifier corrected to summary-first; no performance credit"
|
||||
},
|
||||
{
|
||||
"scope": "Interim successful parity run before eliminating verifier assertion serialization and counting the bootstrap request",
|
||||
"elapsed_seconds": 31.335,
|
||||
"cpu_seconds": 31.07044,
|
||||
"peak_cgroup_bytes": 1652293632,
|
||||
"outcome": "parity passed; superseded measurement, not pooled with final samples"
|
||||
},
|
||||
{
|
||||
"scope": "Successful three-pair probe before adding the shared analyze_javascript_application schema to selected-catalog accounting",
|
||||
"preparation_ms": 18317.110148,
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 313.59318599999824,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 13.595478999999614,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 11.230690000000322,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 206.19418099999893,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 242.2037199999977,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 13.734700000000885,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"elapsed_seconds": 31.037,
|
||||
"cpu_seconds": 30.786998,
|
||||
"peak_cgroup_bytes": 1886543872,
|
||||
"outcome": "Facts and CLI/MCP parity passed. Selected-catalog totals omitted the common analysis-tool schema; superseded, not pooled. Shared-host timing differed substantially from the final run."
|
||||
},
|
||||
{
|
||||
"scope": "Completed three-pair probe with full token accounting before the review-added tokenizer deadline",
|
||||
"preparation_ms": 66696.731143,
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 1114.080965000001,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 46.572758999987855,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 37.09324900001229,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 675.7124039999908,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 777.2660329999926,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 36.31178899999941,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"elapsed_seconds": 108.372,
|
||||
"cpu_seconds": 81.551676,
|
||||
"peak_cgroup_bytes": 1884962816,
|
||||
"outcome": "All facts/costs and CLI/MCP parity passed. Superseded execution source after adding a tokenizer deadline; retained separately, not pooled."
|
||||
},
|
||||
{
|
||||
"scope": "Successful data comparison with direct-child tokenizer timeout before adopting shared owned-process supervision",
|
||||
"preparation_ms": 19051.368892000002,
|
||||
"samples": [
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 330.7313439999998,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 0,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 13.62184999999954,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 12.475999999998749,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 187.18667300000016,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "legacy",
|
||||
"elapsed_ms": 293.4882039999975,
|
||||
"calls": 24,
|
||||
"structured_reply_bytes": 325201,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 55,
|
||||
"total": 55,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"round": 2,
|
||||
"mode": "references",
|
||||
"elapsed_ms": 8.146389999998064,
|
||||
"calls": 1,
|
||||
"structured_reply_bytes": 22158,
|
||||
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
|
||||
"matching_modules": 23,
|
||||
"coverage": {
|
||||
"status": "page",
|
||||
"examined": 23,
|
||||
"total": 23,
|
||||
"next_offset": null,
|
||||
"exhausted": true
|
||||
}
|
||||
}
|
||||
],
|
||||
"elapsed_seconds": 30.742,
|
||||
"cpu_seconds": 30.425218,
|
||||
"peak_cgroup_bytes": 1592938496,
|
||||
"outcome": "Fact and cost parity passed. That harness only killed its immediate tokenizer child; superseded by process-family supervision and detached-helper verification."
|
||||
}
|
||||
],
|
||||
"prior_tokenizer_timeout_verifications": [
|
||||
{
|
||||
"artifact_sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f",
|
||||
"scenario": "Real Obsidian references plus CLI parity, then a deliberately stalled tokenizer",
|
||||
"injected_tokenizer_source": "#!/bin/sh\nexec /bin/sleep 30\n",
|
||||
"timeout_ms": 20,
|
||||
"observed_error_code": "ETIMEDOUT",
|
||||
"expected_payload_exit_code": 1,
|
||||
"payload_cleanup_verified": true,
|
||||
"remaining_owned_temporary_files": 0,
|
||||
"elapsed_seconds": 31.258,
|
||||
"cpu_seconds": 31.00025,
|
||||
"peak_cgroup_bytes": 1644077056,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
},
|
||||
"scope": "Robustness check; no efficiency credit",
|
||||
"limitation": "Immediate child only; did not test detached descendants and is superseded by the owned-family check."
|
||||
}
|
||||
],
|
||||
"tokenizer_timeout_verification": {
|
||||
"artifact_sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f",
|
||||
"scenario": "Real Obsidian references and CLI parity, then an owned tokenizer wrapper with a detached SIGTERM-ignoring helper",
|
||||
"fixture": "scripts/fixtures/tokenizer-family-wrapper.py",
|
||||
"fixture_sha256": "2e9b7092ee539ae1c6a477492f05bb6007d4612dadfff9b90051d621de43049b",
|
||||
"timeout_ms": 2000,
|
||||
"observed_failure_type": "OwnedCommandFailure",
|
||||
"observed_reason": "timeout",
|
||||
"cleanup_failure": null,
|
||||
"expected_payload_exit_code": 1,
|
||||
"payload_cleanup_verified": true,
|
||||
"remaining_owned_temporary_files": 0,
|
||||
"helper_observation": {
|
||||
"pid": 2871108,
|
||||
"group": 2871108,
|
||||
"parent": 2871107,
|
||||
"run_id": "rea-module-token-f139b74a-dd83-498b-b663-e765104159ec",
|
||||
"detached": true,
|
||||
"ignores_sigterm": true
|
||||
},
|
||||
"elapsed_seconds": 55.08,
|
||||
"cpu_seconds": 45.245603,
|
||||
"peak_cgroup_bytes": 1563475968,
|
||||
"events_at_payload_exit": {
|
||||
"memory.events": {
|
||||
"low": 0,
|
||||
"high": 0,
|
||||
"max": 0,
|
||||
"oom": 0,
|
||||
"oom_kill": 0,
|
||||
"oom_group_kill": 0
|
||||
},
|
||||
"pids.events": {
|
||||
"max": 0
|
||||
}
|
||||
},
|
||||
"scope": "Robustness check; no efficiency credit. PIDs are observations; signaling authority comes from the shared process owner."
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
# Retained JavaScript module-reference verification
|
||||
|
||||
Tracks #4092 under #4027 and the decision in #4091. This verifies an existing
|
||||
tool's data projection, without a code-mode host or SDK. [Machine-readable
|
||||
results](javascript-module-references-2026-10-15.json) pin the base revision,
|
||||
executed source hashes, samples, tokenizer and resource observations.
|
||||
|
||||
## Real application and question
|
||||
|
||||
Official Obsidian 1.14.4 ASAR, 27,477,966 bytes, SHA-256
|
||||
`146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f`.
|
||||
The [official gzip](https://github.com/obsidianmd/obsidian-releases/releases/download/v1.14.4/obsidian-1.14.4.asar.gz)
|
||||
has SHA-256 `d1ed428c363968774f0f3906e67d8a53a058ece0b6cdd924b07865977b2ada21`.
|
||||
Select `package.json` and `main.js`; the other 36 inventoried files remain
|
||||
explicitly unselected. No target code is executed.
|
||||
|
||||
Question: retrieve every retained `javascript-module` identity at exact path
|
||||
`main.js`, its retained source-range observation count, and distinct support
|
||||
states. Compare the existing module page plus full item reads with one filtered
|
||||
typed reference view. All six completed query samples use one identical retained
|
||||
parent. All 23 selected rows, identities, counts and support states match. Public
|
||||
CLI inline input and MCP retained input produce the exact same view and Evidence
|
||||
ID. This establishes producer-fact consistency, not independent extractor
|
||||
correctness or proof of runtime loading.
|
||||
|
||||
## Completed read costs
|
||||
|
||||
| Measurement | Existing page + items | Typed reference view |
|
||||
| ------------------------------------------------------------- | --------------------: | -------------------: |
|
||||
| Calls per query task | 24 | 1 |
|
||||
| Structured reply bytes | 325,201 | 22,158 |
|
||||
| First query in its arm, after preparation | 363.03 ms | 14.90 ms |
|
||||
| Subsequent same-session query times | 232.75 / 235.17 ms | 12.95 / 10.57 ms |
|
||||
| Request + structured-reply BPE tokens | 74,597 | 5,843 |
|
||||
| With selected tool catalog + shared bootstrap request/summary | 85,302 | 16,548 |
|
||||
| With complete tool catalog + shared bootstrap request/summary | 272,771 | 204,017 |
|
||||
|
||||
Order alternates: existing/reference, reference/existing, existing/reference.
|
||||
Times include verifier-side fact normalization and equality checks; byte/token
|
||||
accounting follows the timed reads. The initial artifact analysis and summary
|
||||
took 20.978 seconds and is shared preparation, not a query-speed improvement.
|
||||
The retained source remains partial; unknown counts and source selection are
|
||||
preserved independently of exhausted pages. No query cache was introduced.
|
||||
|
||||
Tokens are `cl100k_base` counts with tiktoken 0.12.0 over serialized values, not
|
||||
hosted-model usage. Catalog projections contain name/description/inputSchema;
|
||||
both workflows use the same post-change catalog. Each request/reply is counted
|
||||
once. Output schemas, host-specific rendering/replay, reasoning, final answers,
|
||||
agent success rates and model latency are unmeasured. This is a completed
|
||||
data-retrieval comparison; it does not establish a whole-agent speedup or a
|
||||
before/after catalog-size improvement. Legacy item reads are serial; a concurrent
|
||||
legacy baseline was not measured. Selected discovery includes both the
|
||||
analysis and view tool schemas. Builds/filesystem are warm or uncontrolled,
|
||||
so the first query is not an OS-cold analysis measurement. Shared-host scheduling
|
||||
was uncontrolled: an earlier successful probe had substantially higher wall/CPU
|
||||
times. Its samples are retained separately; neither run establishes a portable
|
||||
latency guarantee.
|
||||
|
||||
The complete final verifier scope used 35.797 seconds elapsed, 35.197 seconds
|
||||
CPU and 1,938,313,216 bytes peak cgroup memory. This includes shared analysis,
|
||||
both query arms, temporary export, CLI parity and tokenization; CPU/memory were
|
||||
not isolated by arm. Cgroup memory includes charged file cache and is not summed
|
||||
process RSS. The whole scope had one CPU, an 8 GiB hard memory cap, 6 GiB early
|
||||
stop and no swap. Payload cleanup was verified; no OOM/capacity event occurred.
|
||||
An earlier full-response transport failure and interim probes are recorded
|
||||
separately in the JSON and earn no performance credit.
|
||||
|
||||
A review follow-up exercised a stalled tokenizer after real Obsidian reference
|
||||
reads and CLI parity. The committed POSIX fault fixture forks a helper into a
|
||||
separate process group, ignores SIGTERM and stalls. With a caller-selected
|
||||
2,000 ms deadline, the shared owned-command supervisor reported
|
||||
`OwnedCommandFailure` with reason `timeout` and no cleanup failure. The verifier
|
||||
failed as expected, its complete owned scope was cleaned, and its selected
|
||||
temporary root was empty. The separate robustness run used 55.080 s wall /
|
||||
45.246 s CPU / 1,563,475,968 B peak cgroup memory, with cleanup verified and no
|
||||
OOM. It earns no efficiency credit. An earlier 20 ms immediate-child timeout
|
||||
probe did not establish descendant cleanup; it remains a separate prior attempt.
|
||||
The detached-helper scenario was executed on Linux, not Windows or macOS.
|
||||
|
||||
A deterministic CI regression also asserts two retained
|
||||
observations at the same source range, excludes an unavailable location, and
|
||||
checks agreement with the full item view; the count deliberately measures
|
||||
observations rather than unique source sites.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Use the pinned Node/npm toolchain and `npm ci`, then `npm run build:cached`.
|
||||
Download/decompress the versioned application outside the checkout and check its
|
||||
digest. Run with an operator-selected absolute artifact path:
|
||||
|
||||
```bash
|
||||
node scripts/verify/javascript/module-references.mjs \
|
||||
--artifact /absolute/path/obsidian-1.14.4.asar \
|
||||
--source-path package.json --source-path main.js \
|
||||
--path main.js --kind javascript-module --rounds 3 --cli-parity \
|
||||
--tokenizer-python /absolute/path/to/tiktoken-venv/bin/python
|
||||
```
|
||||
|
||||
The optional Python interpreter needs tiktoken 0.12.0; omit that flag to skip
|
||||
token accounting. It has a 60-second wall-time deadline, configurable with the
|
||||
positive integer `--tokenizer-timeout-ms`. The tokenizer uses the existing
|
||||
owned-command supervisor, with native identity checks and inherited run-token
|
||||
sweeping for descendants. Inputs and temporary Evidence use the existing private
|
||||
runtime root. Cleanup attempts preserve the primary error; incomplete command
|
||||
cleanup retains its owner and private files for recovery. `--mode legacy` or
|
||||
`--mode references` runs one arm. Fresh runs
|
||||
reanalyze the artifact and can produce different parent identities/resource
|
||||
coverage; the verifier requires equal facts within its paired run. The measured
|
||||
run additionally used the preserved [8 GiB cgroup observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
|
||||
and its adjacent `resource_payload.py`; observer hashes are in the JSON.
|
||||
No application, complete bundle, raw observations or ambient environment is
|
||||
archived. Temporary exported/inline Evidence files are removed after successful
|
||||
owned-command cleanup.
|
||||
|
||||
For the Linux descendant-timeout check, create an operator-selected private
|
||||
directory and set `REA_TOKENIZER_PROBE_RECORD` to a file within it. Run the same
|
||||
real-artifact command with `--mode references --rounds 1`, replace the tokenizer
|
||||
with the absolute path to `scripts/fixtures/tokenizer-family-wrapper.py`, and add
|
||||
`--tokenizer-timeout-ms 2000`. Expect a timeout failure. The explicit marker
|
||||
confirms the detached helper started; the resource observer must separately
|
||||
verify the complete owned scope is empty, and the selected temporary root must
|
||||
contain no verifier files. The recorded PID is an observation, not authority to
|
||||
signal that process.
|
||||
@@ -17,7 +17,7 @@ additional permission flag or degraded mode is required.
|
||||
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
|
||||
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
|
||||
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
|
||||
- an explicit native, non-managed x86 or x86-64 PE application/DLL, or a
|
||||
- an explicit native, non-managed x86, x86-64, ARMNT (Thumb-2) or ARM64 PE application/DLL, or a
|
||||
[raw NMOS 6502 image](raw-binary.md#nmos-6502) with a caller-supplied entry; and
|
||||
- the 26 read-only Ghidra inventory, memory, and function-analysis operations.
|
||||
|
||||
@@ -135,9 +135,55 @@ and artifact SHA-256 before loading the package-owned addon. Native failure
|
||||
reasons remain distinct from a valid installation and an unsupported target.
|
||||
|
||||
PE header classification and SHA-256 come from the same open file. The Windows
|
||||
provider admits native x86 and x86-64 PE applications and DLLs and rejects unsupported roles,
|
||||
provider admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs and rejects unsupported roles,
|
||||
architectures, managed images, and malformed headers. The original selected
|
||||
source coordinate is preserved for native admission.
|
||||
source coordinate is preserved for native admission. The original COFF machine is
|
||||
retained separately from the normalized CPU family. ARM admission requires the
|
||||
observed ARMNT machine 0x01C4; ARM64 requires 0xAA64. Legacy ARM (0x01C0), Thumb
|
||||
(0x01C2), and unavailable machine facts do not select this Windows lane.
|
||||
Without a caller-selected processor override, Ghidra's PE loader
|
||||
selects the language from the PE machine field: ARM (`IMAGE_FILE_MACHINE_ARMNT`)
|
||||
imports as `ARM:LE:32:v8` (Thumb), ARM64 (`IMAGE_FILE_MACHINE_ARM64`) as
|
||||
`AARCH64:LE:64:v8A`. Explicit language/compiler selections remain caller choices
|
||||
in the analysis profile and are recorded as explicit imports. The source
|
||||
fixture verifier compares the complete observed language ID, not just its CPU
|
||||
family prefix. ARM/ARM64 admission is a bounded extension of the x86/x64
|
||||
lane. `npm run verify:ghidra:windows -- --arm` or `-- --arm64`, optionally
|
||||
combined with `--dll`, selects encoded Thumb or ARM64 PE fixtures on a Windows
|
||||
x64 host. The packaged verifier accepts the same flags. These fixtures are not
|
||||
compiler-produced programs. ARM64EC (machine 0xA641) and ARM64X are not admitted;
|
||||
this target support does not establish Windows-on-Arm host support.
|
||||
|
||||
Real Windows x64 verification with Ghidra 12.1.4 and JDK 21 completed the ARM64
|
||||
application and ARMNT/Thumb DLL fixtures' 26 read-only operations, constant-return
|
||||
decompilation, byte searches, CLI/MCP file-offset parity, source identity, and
|
||||
owned cleanup. The DLL fixture also verifies the exported caller/callee
|
||||
relationship. The x86/x64 compact-reference value-flow battery does not cover
|
||||
ARM or ARM64 instructions. ARM64 EXE file-offset verification includes ambiguous
|
||||
mappings; the single-section ARM32 DLL cannot exercise that case.
|
||||
|
||||
The runs used two logical CPUs, 512 MiB Node/JVM heaps, a 900-second startup
|
||||
budget, and an explicit `REA_GHIDRA_DISABLED_ANALYZERS=WindowsResourceReference`
|
||||
test configuration. Default-analyzer runs exceeded the selected startup budget;
|
||||
resource-reference analysis and default-configuration completion are not
|
||||
established by these results. A Microsoft-signed Coreinfo ARM64 application also
|
||||
completed focused MCP decompilation, repeated readback, source-mapped memory
|
||||
reads, unchanged-input and cleanup checks. On this constrained host the first
|
||||
function query, including cold analysis, took about 434 seconds; its identical
|
||||
same-session read took about 23 ms and returned about 4.6 KiB of Evidence. These
|
||||
are individual observations, not general latency guarantees. Public ARM32
|
||||
DLL verification also completed against Microsoft Visual C++ 2013 ARM
|
||||
`msvcr120.dll`, including decompilation of its exported `abs`, repeated readback,
|
||||
source-mapped bytes, unchanged-input and cleanup checks. Its first function
|
||||
query took about 601 seconds; the same-session repeat took about 11 ms and
|
||||
returned about 4.8 KiB of Evidence. After preserving original COFF machines in the
|
||||
admission/profile, a further real Windows check rejected legacy ARM/Thumb and
|
||||
retained ARMNT/ARM64 admission. A rebuilt-candidate MSVCR120 query completed
|
||||
in about 407 seconds with a 15 ms repeat (4,830 bytes of Evidence), unchanged
|
||||
input and complete cleanup. That retry selected an 1,800-second startup budget
|
||||
after an additional 900-second attempt timed out. The resource bounds were
|
||||
unchanged; these observations do not establish a speedup. Configured processor
|
||||
selection is covered by profile tests, not these real-provider runs.
|
||||
|
||||
Admission opens and checks every path component without following a reparse
|
||||
point. It retains handles, verifies local NTFS semantics, and observes volume
|
||||
@@ -200,6 +246,8 @@ npm run verify:ghidra:windows
|
||||
npm run verify:ghidra:windows -- --x86
|
||||
npm run verify:ghidra:windows -- --dll
|
||||
npm run verify:ghidra:windows -- --x86 --dll
|
||||
npm run verify:ghidra:windows -- --arm64
|
||||
npm run verify:ghidra:windows -- --arm64 --dll
|
||||
npm run verify:ghidra:windows:package -- --x86
|
||||
npm run verify:ghidra:windows:package -- --dll
|
||||
npm run verify:ghidra:windows:package -- --x86 --dll
|
||||
|
||||
@@ -213,6 +213,7 @@
|
||||
"verify:javascript-runtime": "npm run build:cached && node scripts/verify/javascript/runtime-observation.mjs",
|
||||
"verify:javascript:digests": "npm run build:cached && node scripts/verify/javascript/digests.mjs",
|
||||
"verify:javascript:output": "npm run build:cached && node scripts/verify/javascript/output.mjs",
|
||||
"verify:javascript:module-references": "npm run build:cached && node scripts/verify/javascript/module-references.mjs",
|
||||
"verify:application-workflows": "npm run build:cached && node scripts/verify/javascript/application-workflows.mjs",
|
||||
"verify:managed": "npm run build:cached && node scripts/verify/managed/conformance.mjs",
|
||||
"verify:firmware": "npm run build:cached && node scripts/verify/firmware/analysis.mjs",
|
||||
|
||||
@@ -3,42 +3,52 @@ import { dirname, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { writeWindowsGhidraByteSearchFixture } from "./lib/ghidra-windows-byte-search.mjs";
|
||||
|
||||
import {
|
||||
WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES,
|
||||
windowsGhidraFixtureOptions,
|
||||
} from "./lib/ghidra-windows-fixture-options.mjs";
|
||||
|
||||
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
// Source-owned static fixtures: no compiler, target loading or execution.
|
||||
for (const architecture of ["x86_64", "x86"]) {
|
||||
for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
|
||||
for (const dll of [false, true]) {
|
||||
const x86 = architecture === "x86";
|
||||
const output = resolve(
|
||||
root,
|
||||
"build",
|
||||
"fixtures",
|
||||
`rea-ghidra-windows${x86 ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
|
||||
);
|
||||
const arm = architecture === "arm";
|
||||
const arm64 = architecture === "arm64";
|
||||
const fixture = windowsGhidraFixtureOptions([
|
||||
...(architecture === "x86_64" ? [] : [`--${architecture}`]),
|
||||
...(dll ? ["--dll"] : []),
|
||||
]);
|
||||
const pe32 = fixture.pe32;
|
||||
const output = resolve(root, "build", "fixtures", fixture.filename);
|
||||
const bytes = Buffer.alloc(dll ? 1024 : 1536);
|
||||
const peOffset = 0x80;
|
||||
const optionalHeader = peOffset + 24;
|
||||
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
|
||||
const optionalHeaderSize = pe32 ? 0xe0 : 0xf0;
|
||||
const sectionTable = optionalHeader + optionalHeaderSize;
|
||||
const directories = optionalHeader + (x86 ? 96 : 112);
|
||||
const directories = optionalHeader + (pe32 ? 96 : 112);
|
||||
|
||||
bytes.write("MZ", 0, "ascii");
|
||||
bytes.writeUInt32LE(peOffset, 0x3c);
|
||||
bytes.write("PE\0\0", peOffset, "binary");
|
||||
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
|
||||
// The fixture descriptor owns COFF machine, width and function coordinates.
|
||||
bytes.writeUInt16LE(fixture.machine, peOffset + 4);
|
||||
bytes.writeUInt16LE(dll ? 1 : 2, peOffset + 6);
|
||||
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
|
||||
bytes.writeUInt16LE(
|
||||
(x86 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
|
||||
(pe32 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
|
||||
peOffset + 22,
|
||||
);
|
||||
|
||||
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
|
||||
// ARM64 PE uses the PE32+ optional header (magic 0x20b), like x86-64.
|
||||
bytes.writeUInt16LE(pe32 ? 0x10b : 0x20b, optionalHeader);
|
||||
bytes.writeUInt32LE(0x200, optionalHeader + 4);
|
||||
if (!dll) bytes.writeUInt32LE(0x200, optionalHeader + 8);
|
||||
bytes.writeUInt32LE(dll ? 0 : 0x1000, optionalHeader + 16);
|
||||
bytes.writeUInt32LE(dll ? 0 : arm ? 0x1001 : 0x1000, optionalHeader + 16);
|
||||
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
|
||||
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
|
||||
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
|
||||
if (pe32)
|
||||
bytes.writeUInt32LE(Number(fixture.imageBase), optionalHeader + 28);
|
||||
else bytes.writeBigUInt64LE(fixture.imageBase, optionalHeader + 24);
|
||||
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
|
||||
bytes.writeUInt32LE(0x200, optionalHeader + 36);
|
||||
bytes.writeUInt16LE(6, optionalHeader + 40);
|
||||
@@ -46,8 +56,8 @@ for (const architecture of ["x86_64", "x86"]) {
|
||||
bytes.writeUInt32LE(dll ? 0x2000 : 0x3000, optionalHeader + 56);
|
||||
bytes.writeUInt32LE(0x200, optionalHeader + 60);
|
||||
bytes.writeUInt16LE(3, optionalHeader + 68);
|
||||
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
|
||||
if (x86) {
|
||||
bytes.writeUInt16LE(pe32 ? 0x8140 : 0x8160, optionalHeader + 70);
|
||||
if (pe32) {
|
||||
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
|
||||
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
|
||||
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
|
||||
@@ -69,18 +79,52 @@ for (const architecture of ["x86_64", "x86"]) {
|
||||
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
|
||||
|
||||
// EXE entry returns zero. DLL caller adds one to the callee's 42.
|
||||
bytes.set(
|
||||
dll
|
||||
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
|
||||
: [0xe8, 0x0b, 0, 0, 0, 0xe8, 0x16, 0, 0, 0, 0x31, 0xc0, 0xc3],
|
||||
0x200,
|
||||
);
|
||||
bytes.set(
|
||||
x86 && !dll
|
||||
? [0xb8, 0x2a, 0, 0, 0, 0xe8, 0x66, 0, 0, 0, 0xc3]
|
||||
: [0xb8, 0x2a, 0, 0, 0, 0xc3],
|
||||
0x210,
|
||||
);
|
||||
if (arm) {
|
||||
// ARMNT uses Thumb instructions and low-bit-tagged PE entry/export RVAs.
|
||||
// PUSH keeps SP aligned and preserves LR across BL from 0x1002 to 0x1020.
|
||||
bytes.set(
|
||||
[
|
||||
0x10,
|
||||
0xb5,
|
||||
0x00,
|
||||
0xf0,
|
||||
0x0d,
|
||||
0xf8,
|
||||
dll ? 0x01 : 0x00,
|
||||
dll ? 0x30 : 0x20,
|
||||
0x10,
|
||||
0xbd,
|
||||
],
|
||||
0x200,
|
||||
);
|
||||
bytes.set([0x2a, 0x20, 0x70, 0x47], 0x220);
|
||||
} else if (arm64) {
|
||||
// Preserve the caller's frame/link registers across BL. The callee
|
||||
// starts at RVA 0x1020, outside the five-instruction caller.
|
||||
const instructions = [
|
||||
0xa9bf7bfd, // STP X29, X30, [SP, #-16]!
|
||||
0x94000007, // BL +0x1c from RVA 0x1004 to RVA 0x1020
|
||||
dll ? 0x11000400 : 0x52800000, // ADD W0,W0,#1 / MOV W0,#0
|
||||
0xa8c17bfd, // LDP X29, X30, [SP], #16
|
||||
0xd65f03c0, // RET
|
||||
];
|
||||
for (const [index, instruction] of instructions.entries())
|
||||
bytes.writeUInt32LE(instruction, 0x200 + index * 4);
|
||||
bytes.set([0x40, 0x05, 0x80, 0x52, 0xc0, 0x03, 0x5f, 0xd6], 0x220);
|
||||
} else {
|
||||
bytes.set(
|
||||
dll
|
||||
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
|
||||
: [0xe8, 0x0b, 0, 0, 0, 0xe8, 0x16, 0, 0, 0, 0x31, 0xc0, 0xc3],
|
||||
0x200,
|
||||
);
|
||||
bytes.set(
|
||||
x86 && !dll
|
||||
? [0xb8, 0x2a, 0, 0, 0, 0xe8, 0x66, 0, 0, 0, 0xc3]
|
||||
: [0xb8, 0x2a, 0, 0, 0, 0xc3],
|
||||
0x210,
|
||||
);
|
||||
}
|
||||
|
||||
if (x86 && !dll) {
|
||||
// A statically analyzed Borland-style SEH registration whose handler
|
||||
@@ -149,19 +193,31 @@ for (const architecture of ["x86_64", "x86"]) {
|
||||
"ffd7", // CALL EDI: unknown target.
|
||||
"c3",
|
||||
]
|
||||
: [
|
||||
"488d15d90f0000", // LEA RDX, [RIP + displacement]: DATA.
|
||||
"8b05d30f0000", // MOV EAX, [RIP + displacement]: READ.
|
||||
"8905d10f0000", // MOV [RIP + displacement], EAX: WRITE.
|
||||
"488d15d60f0000", // Indexed base, unknown RCX.
|
||||
"85db", // TEST EBX, EBX: two candidate bases.
|
||||
"7407", // JZ over the alternate base.
|
||||
"488d15d30f0000", // LEA RDX, alternate base.
|
||||
"c6040a01", // MOV byte [RDX + RCX], 1.
|
||||
"41ffd0", // CALL R8: unknown target.
|
||||
"c3",
|
||||
];
|
||||
bytes.set(Buffer.from(referenceInstructions.join(""), "hex"), 0x220);
|
||||
: arm
|
||||
? ["7047"] // Thumb return; x86 reference-flow controls do not apply.
|
||||
: arm64
|
||||
? [
|
||||
"010000b0", // ADRP X1, .data page (RVA 0x2000): DATA.
|
||||
"200040b9", // LDR W0, [X1]: READ.
|
||||
"200000b9", // STR W0, [X1]: WRITE.
|
||||
"c0035fd6", // RET.
|
||||
]
|
||||
: [
|
||||
"488d15d90f0000", // LEA RDX, [RIP + displacement]: DATA.
|
||||
"8b05d30f0000", // MOV EAX, [RIP + displacement]: READ.
|
||||
"8905d10f0000", // MOV [RIP + displacement], EAX: WRITE.
|
||||
"488d15d60f0000", // Indexed base, unknown RCX.
|
||||
"85db", // TEST EBX, EBX: two candidate bases.
|
||||
"7407", // JZ over the alternate base.
|
||||
"488d15d30f0000", // LEA RDX, alternate base.
|
||||
"c6040a01", // MOV byte [RDX + RCX], 1.
|
||||
"41ffd0", // CALL R8: unknown target.
|
||||
"c3",
|
||||
];
|
||||
bytes.set(
|
||||
Buffer.from(referenceInstructions.join(""), "hex"),
|
||||
arm || arm64 ? 0x230 : 0x220,
|
||||
);
|
||||
}
|
||||
|
||||
if (dll) {
|
||||
@@ -176,8 +232,8 @@ for (const architecture of ["x86_64", "x86"]) {
|
||||
bytes.writeUInt32LE(0x1080, exports + 28);
|
||||
bytes.writeUInt32LE(0x1088, exports + 32);
|
||||
bytes.writeUInt32LE(0x1090, exports + 36);
|
||||
bytes.writeUInt32LE(0x1000, 0x280);
|
||||
bytes.writeUInt32LE(0x1010, 0x284);
|
||||
bytes.writeUInt32LE(arm ? 0x1001 : 0x1000, 0x280);
|
||||
bytes.writeUInt32LE(fixture.calleeRva + (arm ? 1 : 0), 0x284);
|
||||
bytes.writeUInt32LE(0x1120, 0x288);
|
||||
bytes.writeUInt32LE(0x1140, 0x28c);
|
||||
bytes.writeUInt16LE(0, 0x290);
|
||||
@@ -191,6 +247,6 @@ for (const architecture of ["x86_64", "x86"]) {
|
||||
|
||||
await mkdir(dirname(output), { recursive: true });
|
||||
await writeFile(output, bytes);
|
||||
if (!x86 && !dll) process.stdout.write(`${output}\n`);
|
||||
if (architecture === "x86_64" && !dll) process.stdout.write(`${output}\n`);
|
||||
}
|
||||
}
|
||||
@@ -12,13 +12,22 @@ export async function startBrowserVerifierSite() {
|
||||
return;
|
||||
}
|
||||
const requestUrl = new URL(request.url ?? "/", "http://127.0.0.1");
|
||||
if (requestUrl.pathname === "/scenario-request-language") {
|
||||
response.setHeader("content-type", "application/json");
|
||||
response.setHeader("cache-control", "no-store");
|
||||
response.end(JSON.stringify(request.headers["accept-language"] ?? null));
|
||||
return;
|
||||
}
|
||||
if (requestUrl.pathname === "/scenario-environment") {
|
||||
const requestedRun = requestUrl.searchParams.get("run");
|
||||
const run = ["A", "B", "C"].includes(requestedRun)
|
||||
? requestedRun
|
||||
: "default";
|
||||
response.setHeader("content-type", "text/html");
|
||||
response.end(`<!doctype html><html><body data-run="${run}"><pre id="environment"></pre>
|
||||
const requestLanguage = encodeURIComponent(
|
||||
JSON.stringify(request.headers["accept-language"] ?? null),
|
||||
);
|
||||
response.end(`<!doctype html><html><body data-run="${run}" data-request-language="${requestLanguage}"><pre id="environment"></pre><pre id="language"></pre>
|
||||
<button id="refresh-environment">Refresh environment</button><script>
|
||||
const refresh = () => document.querySelector('#environment').textContent = JSON.stringify({
|
||||
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
|
||||
@@ -26,6 +35,10 @@ export async function startBrowserVerifierSite() {
|
||||
dpr: devicePixelRatio, width: innerWidth, height: innerHeight
|
||||
});
|
||||
document.querySelector('#refresh-environment').onclick = refresh;
|
||||
document.querySelector('#language').textContent = JSON.stringify({
|
||||
language: navigator.language, languages: [...navigator.languages],
|
||||
request_language: JSON.parse(decodeURIComponent(document.body.dataset.requestLanguage))
|
||||
});
|
||||
window.releaseScenario = () => {
|
||||
const release = document.createElement('span');
|
||||
release.id = 'release-' + document.body.dataset.run;
|
||||
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env python3
|
||||
"""POSIX fault fixture for the optional module-reference token counter.
|
||||
|
||||
Use as --tokenizer-python with an explicit REA_TOKENIZER_PROBE_RECORD path.
|
||||
It deliberately ignores the counter arguments, forks a detached helper that
|
||||
ignores SIGTERM, and stalls. The production owned-command supervisor must
|
||||
terminate the verified family. Recorded PIDs are observations, not authority
|
||||
to signal a process; verification also checks the owned scope after exit.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import signal
|
||||
import time
|
||||
|
||||
|
||||
def main():
|
||||
record = Path(os.environ["REA_TOKENIZER_PROBE_RECORD"])
|
||||
child = os.fork()
|
||||
if child == 0:
|
||||
os.setsid()
|
||||
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||
record.write_text(json.dumps({
|
||||
"pid": os.getpid(),
|
||||
"group": os.getpgrp(),
|
||||
"parent": os.getppid(),
|
||||
"run_id": os.environ.get("REA_PROCESS_RUN_ID"),
|
||||
"detached": True,
|
||||
"ignores_sigterm": True,
|
||||
}) + "\n", encoding="utf-8")
|
||||
time.sleep(30)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -97,8 +97,32 @@ const assertCapture = (capture, values = expected) => {
|
||||
}
|
||||
};
|
||||
|
||||
const assertCallerLanguage = (capture, baseline) => {
|
||||
assert.ok(
|
||||
capture.limitations.some((value) =>
|
||||
value.includes("Attached locale fr-FR applies to Intl formatting only"),
|
||||
),
|
||||
"Attached capture must identify its page/request language limitation",
|
||||
);
|
||||
for (const step of capture.steps) {
|
||||
const text = step.artifacts.dom.value.text;
|
||||
const observed = /<pre id="language">([^<]+)<\/pre>/u.exec(text);
|
||||
assert.ok(
|
||||
observed,
|
||||
"DOM omitted the navigator/request language observations",
|
||||
);
|
||||
assert.deepEqual(JSON.parse(observed[1]), baseline);
|
||||
}
|
||||
};
|
||||
|
||||
/** Verify actual attached-page emulation and restoration across scenario outcomes. */
|
||||
export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
const languageExpression = `(async () => JSON.stringify({
|
||||
language: navigator.language, languages: [...navigator.languages],
|
||||
request_language: await (await fetch(
|
||||
${JSON.stringify(`${origin}/scenario-request-language`)}, { cache: "no-store" }
|
||||
)).json()
|
||||
}))()`;
|
||||
const targets = await (await fetch(`${endpoint}/json/list`)).json();
|
||||
const target = targets.find(({ id }) => id === targetId);
|
||||
assert.ok(target, "External fixture target is missing");
|
||||
@@ -106,10 +130,11 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
target.webSocketDebuggerUrl,
|
||||
"capture_browser_scenario",
|
||||
);
|
||||
const readEnvironment = async () => {
|
||||
const readEnvironment = async (expressionToRead = expression) => {
|
||||
const value = await connection.send("Runtime.evaluate", {
|
||||
expression,
|
||||
expression: expressionToRead,
|
||||
returnByValue: true,
|
||||
awaitPromise: true,
|
||||
});
|
||||
assert.equal(value.result.type, "string");
|
||||
return JSON.parse(value.result.value);
|
||||
@@ -118,6 +143,7 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
let verificationError;
|
||||
try {
|
||||
const baseline = await readEnvironment();
|
||||
const callerLanguage = await readEnvironment(languageExpression);
|
||||
const scenario = environmentScenario({
|
||||
endpoint,
|
||||
targetId,
|
||||
@@ -126,10 +152,16 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
});
|
||||
const assertRestored = async () => {
|
||||
assert.deepEqual(await readEnvironment(), baseline);
|
||||
assert.deepEqual(
|
||||
await readEnvironment(languageExpression),
|
||||
callerLanguage,
|
||||
);
|
||||
const remaining = await (await fetch(`${endpoint}/json/list`)).json();
|
||||
assert.ok(remaining.some(({ id }) => id === targetId));
|
||||
};
|
||||
assertCapture((await runScenarioCli(scenario)).normalized_result);
|
||||
const cliCapture = (await runScenarioCli(scenario)).normalized_result;
|
||||
assertCapture(cliCapture);
|
||||
assertCallerLanguage(cliCapture, callerLanguage);
|
||||
await assertRestored();
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -147,15 +179,15 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
({ name }) => name === "capture_browser_scenario",
|
||||
);
|
||||
assert.ok(definition, "Scenario tool is absent from the current catalog");
|
||||
assertCapture(
|
||||
requireMcpEvidenceResult(
|
||||
await client.callTool(
|
||||
{ name: "capture_browser_scenario", arguments: scenario },
|
||||
{ toolDefinition: definition },
|
||||
),
|
||||
"capture_browser_scenario",
|
||||
const mcpCapture = requireMcpEvidenceResult(
|
||||
await client.callTool(
|
||||
{ name: "capture_browser_scenario", arguments: scenario },
|
||||
{ toolDefinition: definition },
|
||||
),
|
||||
"capture_browser_scenario",
|
||||
);
|
||||
assertCapture(mcpCapture);
|
||||
assertCallerLanguage(mcpCapture, callerLanguage);
|
||||
await verifyConcurrentMcpCapture({
|
||||
client,
|
||||
definition,
|
||||
@@ -201,6 +233,8 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
|
||||
stdio_mcp: true,
|
||||
observed: expected,
|
||||
restored: baseline,
|
||||
caller_language_preserved: callerLanguage,
|
||||
attached_locale_limitation: true,
|
||||
initialization_failure_cleanup: true,
|
||||
action_failure_cleanup: true,
|
||||
cancellation_cleanup: true,
|
||||
|
||||
@@ -626,6 +626,7 @@ const verifyCancelledStorage = async (
|
||||
const pending = provider.captureScenario(cancelledScenario, {
|
||||
signal: controller.signal,
|
||||
});
|
||||
let failure;
|
||||
try {
|
||||
await waitForNewDocument(
|
||||
observer,
|
||||
@@ -645,19 +646,23 @@ const verifyCancelledStorage = async (
|
||||
});
|
||||
controller.abort(new Error("cancel storage seed scenario"));
|
||||
const result = await pending;
|
||||
assert.ok(
|
||||
!result.ok ||
|
||||
if (result.ok)
|
||||
assert.ok(
|
||||
result.value.steps.some(({ status }) => status === "cancelled"),
|
||||
"Cancellation unexpectedly completed every scenario step",
|
||||
);
|
||||
"Cancellation unexpectedly completed every scenario step",
|
||||
);
|
||||
else assert.equal(result.error._tag, "AnalysisCancelledError");
|
||||
const targets = await (await fetch(`${endpoint}/json/list`)).json();
|
||||
assert.ok(
|
||||
targets.some(({ id }) => id === targetId),
|
||||
"Cancelled connect-mode storage cleanup closed the external target",
|
||||
);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
controller.abort();
|
||||
await provider.close();
|
||||
controller.abort(new Error("cancel failed storage verification"));
|
||||
await closeVerifierResources([() => provider.close()], failure);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -18,13 +18,12 @@ export function writeWindowsGhidraByteSearchFixture(bytes) {
|
||||
export async function verifyWindowsGhidraByteSearch({
|
||||
invoke,
|
||||
sourceBytes,
|
||||
architecture,
|
||||
imageBase,
|
||||
sourceOwnedFixture = true,
|
||||
fallbackAddress,
|
||||
fallbackFileOffset,
|
||||
runCliPage,
|
||||
}) {
|
||||
const imageBase = architecture === "x86" ? 0x400000n : 0x140000000n;
|
||||
const query = sourceOwnedFixture
|
||||
? {
|
||||
pattern: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.pattern,
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
export const WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES: readonly [
|
||||
"x86_64",
|
||||
"x86",
|
||||
"arm",
|
||||
"arm64",
|
||||
];
|
||||
export const WINDOWS_GHIDRA_FIXTURE_FLAGS: readonly string[];
|
||||
|
||||
/** Select one architecture without interpreting positional paths as flags. */
|
||||
export function windowsGhidraFixtureOptions(arguments_: readonly string[]): {
|
||||
architecture: (typeof WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES)[number];
|
||||
dll: boolean;
|
||||
pe32: boolean;
|
||||
imageBase: bigint;
|
||||
calleeRva: number;
|
||||
machine: number;
|
||||
filename: string;
|
||||
languageId: string;
|
||||
callerAddress: string;
|
||||
calleeAddress: string;
|
||||
};
|
||||
@@ -0,0 +1,50 @@
|
||||
/** Source-owned PE fixtures; these are encoded bytes, not compiled programs. */
|
||||
/** @type {readonly ["x86_64", "x86", "arm", "arm64"]} */
|
||||
export const WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES = Object.freeze([
|
||||
"x86_64",
|
||||
"x86",
|
||||
"arm",
|
||||
"arm64",
|
||||
]);
|
||||
|
||||
export const WINDOWS_GHIDRA_FIXTURE_FLAGS = Object.freeze([
|
||||
"--x86",
|
||||
"--arm",
|
||||
"--arm64",
|
||||
"--dll",
|
||||
]);
|
||||
|
||||
/** Select one architecture without interpreting positional paths as flags. */
|
||||
/** @param {readonly string[]} arguments_ */
|
||||
export const windowsGhidraFixtureOptions = (arguments_) => {
|
||||
const selected = WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES.filter(
|
||||
(architecture) =>
|
||||
architecture !== "x86_64" && arguments_.includes(`--${architecture}`),
|
||||
);
|
||||
if (selected.length > 1)
|
||||
throw new Error("Choose one Windows PE fixture architecture.");
|
||||
const architecture = selected[0] ?? "x86_64";
|
||||
const dll = arguments_.includes("--dll");
|
||||
const pe32 = architecture === "x86" || architecture === "arm";
|
||||
const imageBase = pe32 ? 0x400000n : 0x140000000n;
|
||||
const calleeRva = architecture.startsWith("arm") ? 0x1020 : 0x1010;
|
||||
return {
|
||||
architecture,
|
||||
dll,
|
||||
pe32,
|
||||
imageBase,
|
||||
calleeRva,
|
||||
machine: { x86: 0x14c, x86_64: 0x8664, arm: 0x1c4, arm64: 0xaa64 }[
|
||||
architecture
|
||||
],
|
||||
filename: `rea-ghidra-windows${architecture === "x86_64" ? "" : `-${architecture}`}.${dll ? "dll" : "exe"}`,
|
||||
languageId:
|
||||
architecture === "arm"
|
||||
? "ARM:LE:32:v8"
|
||||
: architecture === "arm64"
|
||||
? "AARCH64:LE:64:v8A"
|
||||
: `x86:LE:${pe32 ? 32 : 64}:default`,
|
||||
callerAddress: `0x${(imageBase + 0x1000n).toString(16)}`,
|
||||
calleeAddress: `0x${(imageBase + BigInt(calleeRva)).toString(16)}`,
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,46 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { realpath } from "node:fs/promises";
|
||||
|
||||
import { runKunaCommand } from "../../dist/kuna/KunaCommand.js";
|
||||
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
|
||||
import { KUNA_XREF_OUTPUT_BYTES } from "../../dist/kuna/KunaReferenceFiles.js";
|
||||
|
||||
/** Independent projection of native exact-target rows, bypassing REA's parser. */
|
||||
export const nativeKunaIncomingSites = async ({
|
||||
executable,
|
||||
cacheRoot,
|
||||
workspace,
|
||||
binary,
|
||||
address,
|
||||
retain,
|
||||
}) => {
|
||||
const installed = await inspectKunaInstallation({
|
||||
REA_KUNA_COMMAND: executable,
|
||||
REA_KUNA_CACHE_DIR: cacheRoot,
|
||||
});
|
||||
assert.ok(installed.ok, "Native reference oracle could not inspect Kuna");
|
||||
const input = await realpath(binary);
|
||||
const timeoutMs = installed.value.configuration.functionTimeoutMs;
|
||||
const outcome = await runKunaCommand({
|
||||
installation: installed.value,
|
||||
arguments: ["xrefs", input, "--to", address, "--json"],
|
||||
workspace,
|
||||
operation: "verify:kuna-native-xrefs",
|
||||
timeoutMs,
|
||||
reportedTimeoutMs: timeoutMs,
|
||||
outputBytes: KUNA_XREF_OUTPUT_BYTES,
|
||||
signal: new AbortController().signal,
|
||||
});
|
||||
if (outcome.kind === "retained") retain(outcome);
|
||||
if (outcome.kind !== "ok") throw outcome.error;
|
||||
const document = JSON.parse(outcome.snapshot.stdout.text);
|
||||
assert.equal(document.binary, input);
|
||||
assert.equal(document.direction, "to");
|
||||
assert.equal(document.target.address_hex, address);
|
||||
assert.equal(document.count, document.xrefs.length);
|
||||
const sites = document.xrefs
|
||||
.filter((row) => row.to_address_hex === address)
|
||||
.map((row) => row.from_address_hex);
|
||||
assert.ok(sites.every((site) => /^0x[0-9a-f]+$/u.test(site)));
|
||||
return [...new Set(sites)].sort();
|
||||
};
|
||||
@@ -0,0 +1,109 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readdir } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { combineFunctionRelationshipCoverage } from "../../dist/domain/functionCallCoverage.js";
|
||||
|
||||
const entries = async (path) => {
|
||||
try {
|
||||
return (await readdir(path)).sort();
|
||||
} catch (cause) {
|
||||
if (cause.code === "ENOENT") return [];
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
/** Verify selected public reference answers without requesting source text. */
|
||||
export const verifyKunaReferences = async ({
|
||||
call,
|
||||
cacheRoot,
|
||||
name,
|
||||
address,
|
||||
expectedCaller,
|
||||
nativeIncoming,
|
||||
}) => {
|
||||
const before = await Promise.all([
|
||||
entries(join(cacheRoot, "projects")),
|
||||
entries(join(cacheRoot, "staging")),
|
||||
]);
|
||||
const callers = await call("procedure_callers", { procedure: address });
|
||||
const callees = await call("procedure_callees", { procedure: address });
|
||||
const incoming = await call("xrefs", { address });
|
||||
for (const [label, result] of [
|
||||
["callers", callers],
|
||||
["callees", callees],
|
||||
["xrefs", incoming],
|
||||
]) {
|
||||
assert.ok(
|
||||
Array.isArray(result.value),
|
||||
`${name} ${label} omitted its address list`,
|
||||
);
|
||||
assert.equal(
|
||||
new Set(result.value).size,
|
||||
result.value.length,
|
||||
`${label} repeats an address`,
|
||||
);
|
||||
assert.ok(
|
||||
result.value.every((value) =>
|
||||
/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u.test(value),
|
||||
),
|
||||
`${label} returned a noncanonical u64 address`,
|
||||
);
|
||||
assert.equal(result.structured.provider.id, "kuna");
|
||||
const source = result.structured.raw_result.references;
|
||||
assert.match(source.sha256, /^[0-9a-f]{64}$/u);
|
||||
assert.ok(source.bytes > 0);
|
||||
}
|
||||
let composed;
|
||||
if (expectedCaller !== undefined) {
|
||||
assert.ok(
|
||||
callers.value.includes(expectedCaller),
|
||||
`${name} lost its source-owned caller`,
|
||||
);
|
||||
assert.ok(
|
||||
incoming.value.length > 0,
|
||||
`${name} lost its source-owned call site`,
|
||||
);
|
||||
assert.deepEqual(
|
||||
[...incoming.value].sort(),
|
||||
await nativeIncoming(address),
|
||||
`${name} incoming references differ from exact-target native sites`,
|
||||
);
|
||||
const graph = await call("get_call_graph", {
|
||||
address: expectedCaller,
|
||||
direction: "forward",
|
||||
});
|
||||
assert.ok(graph.value.graph["0"][0].calls.includes(address));
|
||||
const path = await call("trace_call_path", {
|
||||
start: expectedCaller,
|
||||
goal: address,
|
||||
direction: "forward",
|
||||
});
|
||||
assert.equal(path.value.goal_status, "reached");
|
||||
composed = { graph_ms: graph.ms, path_ms: path.ms };
|
||||
}
|
||||
assert.deepEqual(
|
||||
await Promise.all([
|
||||
entries(join(cacheRoot, "projects")),
|
||||
entries(join(cacheRoot, "staging")),
|
||||
]),
|
||||
before,
|
||||
"A reference-only workflow started a C project export",
|
||||
);
|
||||
const repeated = await call("procedure_callees", { procedure: address });
|
||||
assert.deepEqual(repeated.value, callees.value);
|
||||
return {
|
||||
callers: callers.value.length,
|
||||
callees: callees.value.length,
|
||||
xrefs: incoming.value.length,
|
||||
first_graph_ms: callers.ms,
|
||||
warm_graph_ms: repeated.ms,
|
||||
xrefs_ms: incoming.ms,
|
||||
response_bytes: callers.bytes + callees.bytes + incoming.bytes,
|
||||
relationship_coverage: combineFunctionRelationshipCoverage(
|
||||
callers.structured.relationship_coverage,
|
||||
callees.structured.relationship_coverage,
|
||||
),
|
||||
...composed,
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,166 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readdir, realpath } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { runKunaCommand } from "../../dist/kuna/KunaCommand.js";
|
||||
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
|
||||
|
||||
const entries = (path) =>
|
||||
readdir(path).catch((cause) => {
|
||||
if (cause.code === "ENOENT") return [];
|
||||
throw cause;
|
||||
});
|
||||
|
||||
/** Independent native oracle; uses source hex fields rather than JS numeric addresses. */
|
||||
export const nativeKunaStrings = async ({
|
||||
executable,
|
||||
cacheRoot,
|
||||
workspace,
|
||||
binary,
|
||||
retain,
|
||||
}) => {
|
||||
const installed = await inspectKunaInstallation({
|
||||
REA_KUNA_COMMAND: executable,
|
||||
REA_KUNA_CACHE_DIR: cacheRoot,
|
||||
});
|
||||
assert.ok(installed.ok, "Native string oracle could not inspect Kuna");
|
||||
const input = await realpath(binary);
|
||||
const outcome = await runKunaCommand({
|
||||
installation: installed.value,
|
||||
arguments: [
|
||||
"strings",
|
||||
input,
|
||||
"--json",
|
||||
"--encoding",
|
||||
"all",
|
||||
"--min-length",
|
||||
"5",
|
||||
"--termination",
|
||||
"any",
|
||||
"--no-xrefs",
|
||||
],
|
||||
workspace,
|
||||
operation: "verify:kuna-native-strings",
|
||||
timeoutMs: installed.value.configuration.projectTimeoutMs,
|
||||
reportedTimeoutMs: installed.value.configuration.projectTimeoutMs,
|
||||
outputBytes: 16 * 1024 * 1024,
|
||||
signal: new AbortController().signal,
|
||||
});
|
||||
if (outcome.kind === "retained") retain(outcome);
|
||||
if (outcome.kind !== "ok") throw outcome.error;
|
||||
const document = JSON.parse(outcome.snapshot.stdout.text);
|
||||
assert.equal(document.binary, input);
|
||||
assert.equal(document.count, document.strings.length);
|
||||
assert.equal(document.xrefs, false);
|
||||
return document.strings
|
||||
.map((row) => ({
|
||||
address: row.address_hex,
|
||||
value: row.text,
|
||||
string: {
|
||||
encoding: row.encoding,
|
||||
encoding_status: "observed",
|
||||
termination: row.nul_terminated ? "present_or_not_required" : "missing",
|
||||
byte_length: row.byte_length,
|
||||
},
|
||||
}))
|
||||
.sort((left, right) => {
|
||||
const a = BigInt(left.address),
|
||||
b = BigInt(right.address);
|
||||
return a < b
|
||||
? -1
|
||||
: a > b
|
||||
? 1
|
||||
: left.string.encoding.localeCompare(right.string.encoding, "en");
|
||||
});
|
||||
};
|
||||
|
||||
/** Verify focused/complete string operations before any procedure analysis starts. */
|
||||
export const verifyKunaStrings = async ({
|
||||
call,
|
||||
cacheRoot,
|
||||
nativeStrings,
|
||||
expectedStrings = [],
|
||||
}) => {
|
||||
const expected = await nativeStrings();
|
||||
if (expectedStrings.length > 0)
|
||||
assert.ok(
|
||||
expected.length > 0,
|
||||
"Source-owned string fixture did not exercise a positive result",
|
||||
);
|
||||
const pattern =
|
||||
expected.find(({ value }) => value.length >= 8 && value.length <= 100)
|
||||
?.value ??
|
||||
expected[0]?.value ??
|
||||
"REA-empty-string-inventory";
|
||||
const selected = expected.filter(({ value }) =>
|
||||
value.toLowerCase().includes(pattern.toLowerCase()),
|
||||
);
|
||||
const before = await Promise.all([
|
||||
entries(join(cacheRoot, "projects")),
|
||||
entries(join(cacheRoot, "staging")),
|
||||
]);
|
||||
const cold = await call("search_strings", { pattern });
|
||||
assert.deepEqual(
|
||||
cold.value,
|
||||
selected,
|
||||
"String search differs from native scan",
|
||||
);
|
||||
const source = cold.structured.raw_result.strings;
|
||||
assert.equal(source.inventory_count, expected.length);
|
||||
assert.equal(source.selected_count, selected.length);
|
||||
assert.equal(source.xrefs, false);
|
||||
assert.match(source.sha256, /^[0-9a-f]{64}$/u);
|
||||
assert.ok(source.bytes > 0);
|
||||
assert.equal(cold.structured.provider.id, "kuna");
|
||||
assert.ok(cold.structured.evidence_id);
|
||||
const complete = await call("list_strings");
|
||||
assert.deepEqual(complete.value, expected);
|
||||
for (const value of expectedStrings)
|
||||
assert.ok(
|
||||
complete.value.some((row) => row.value === value),
|
||||
`Required string ${JSON.stringify(value)} was not recovered`,
|
||||
);
|
||||
assert.equal(complete.structured.raw_result.strings.sha256, source.sha256);
|
||||
const repeated = await call("search_strings", { pattern });
|
||||
assert.deepEqual(repeated.value, selected);
|
||||
let exact;
|
||||
if (selected.length > 0) {
|
||||
const address = selected[0].address;
|
||||
exact = await call("list_strings", { address });
|
||||
assert.deepEqual(
|
||||
exact.value,
|
||||
expected.filter((item) => item.address === address),
|
||||
);
|
||||
}
|
||||
const regex = await call("search_strings", {
|
||||
pattern: "^.{5,12}$",
|
||||
mode: "regex",
|
||||
case_sensitive: true,
|
||||
});
|
||||
assert.deepEqual(
|
||||
regex.value,
|
||||
expected.filter(({ value }) => /^.{5,12}$/u.test(value)),
|
||||
);
|
||||
const missing = await call("search_strings", {
|
||||
pattern: "\u0000REA-negative-string-query\u0000",
|
||||
});
|
||||
assert.deepEqual(missing.value, []);
|
||||
assert.deepEqual(
|
||||
await Promise.all([
|
||||
entries(join(cacheRoot, "projects")),
|
||||
entries(join(cacheRoot, "staging")),
|
||||
]),
|
||||
before,
|
||||
"String reads started a project export",
|
||||
);
|
||||
return {
|
||||
inventory_count: expected.length,
|
||||
selected_count: selected.length,
|
||||
cold_scan_ms: cold.ms,
|
||||
warm_search_ms: repeated.ms,
|
||||
exact_address_ms: exact?.ms ?? null,
|
||||
cold_response_bytes: cold.bytes,
|
||||
complete_response_bytes: complete.bytes,
|
||||
regex_response_bytes: regex.bytes,
|
||||
};
|
||||
};
|
||||
@@ -146,6 +146,9 @@ async function verifySourceReadFailures({
|
||||
...(permissionDenied ? ["permission"] : []),
|
||||
];
|
||||
for (const scenario of scenarios) {
|
||||
// Same-target reopen may admit restored bytes through retained-client health.
|
||||
// Each rejection needs a cold client, without an existing private snapshot.
|
||||
await call("close_binary");
|
||||
await call("open_binary", { path: source, provider_id: "ghidra" });
|
||||
const before = await call("binary_session");
|
||||
if (scenario === "permission") await chmod(source, 0);
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
#!/usr/bin/env node
|
||||
import {
|
||||
WINDOWS_GHIDRA_FIXTURE_FLAGS,
|
||||
windowsGhidraFixtureOptions,
|
||||
} from "./lib/ghidra-windows-fixture-options.mjs";
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
@@ -33,11 +37,11 @@ const exec = promisify(execFile);
|
||||
const arguments_ = process.argv.slice(2);
|
||||
const internalWorkspace = arguments_[0] === "--internal-workspace";
|
||||
const fixtureArguments = internalWorkspace ? arguments_.slice(2) : arguments_;
|
||||
const x86Fixture = fixtureArguments.includes("--x86");
|
||||
const dllFixture = fixtureArguments.includes("--dll");
|
||||
const fixture = windowsGhidraFixtureOptions(fixtureArguments);
|
||||
const dllFixture = fixture.dll;
|
||||
const [packageRootArgument, targetArgument, reportArgument] =
|
||||
fixtureArguments.filter(
|
||||
(argument) => argument !== "--x86" && argument !== "--dll",
|
||||
(argument) => !WINDOWS_GHIDRA_FIXTURE_FLAGS.includes(argument),
|
||||
);
|
||||
// Windows locks the loaded package-owned addon until the verifier exits.
|
||||
// Keep installation cleanup in a parent that never loads that addon.
|
||||
@@ -125,10 +129,7 @@ if (packageRootArgument === undefined) {
|
||||
});
|
||||
}
|
||||
}
|
||||
const target = resolve(
|
||||
targetArgument ??
|
||||
`build/fixtures/rea-ghidra-windows${x86Fixture ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
|
||||
);
|
||||
const target = resolve(targetArgument ?? `build/fixtures/${fixture.filename}`);
|
||||
let TOOL_CONTRACTS, native, token, sha256;
|
||||
let parsedTarget;
|
||||
try {
|
||||
@@ -160,10 +161,7 @@ try {
|
||||
assert.equal(parsedTarget.value.format, "pe");
|
||||
assert.equal(parsedTarget.value.managed, false);
|
||||
if (targetArgument === undefined) {
|
||||
assert.equal(
|
||||
parsedTarget.value.architecture,
|
||||
x86Fixture ? "x86" : "x86_64",
|
||||
);
|
||||
assert.equal(parsedTarget.value.architecture, fixture.architecture);
|
||||
assert.equal(
|
||||
parsedTarget.value.executableRole,
|
||||
dllFixture ? "shared-library" : "application",
|
||||
@@ -321,13 +319,13 @@ try {
|
||||
const procedures = await call("list_procedures", {});
|
||||
assert.ok(procedures.length > 0);
|
||||
if (targetArgument === undefined) {
|
||||
const calleeAddress = x86Fixture ? "0x401010" : "0x140001010";
|
||||
const calleeAddress = fixture.calleeAddress;
|
||||
const calleeCode = await call("procedure_pseudo_code", {
|
||||
procedure: calleeAddress,
|
||||
});
|
||||
assert.match(calleeCode, /\b(?:42|0x2a)\b/iu);
|
||||
if (dllFixture) {
|
||||
const callerAddress = x86Fixture ? "0x401000" : "0x140001000";
|
||||
const callerAddress = fixture.callerAddress;
|
||||
assert.ok(
|
||||
(
|
||||
await call("procedure_callees", { procedure: callerAddress })
|
||||
@@ -379,7 +377,7 @@ try {
|
||||
report.byteSearch = await verifyWindowsGhidraByteSearch({
|
||||
invoke: call,
|
||||
sourceBytes,
|
||||
architecture: parsedTarget.value.architecture,
|
||||
imageBase: fixture.imageBase,
|
||||
sourceOwnedFixture: targetArgument === undefined,
|
||||
fallbackAddress: procedure.address,
|
||||
fallbackFileOffset: mapping.file_offset,
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
#!/usr/bin/env node
|
||||
import { windowsGhidraFixtureOptions } from "./lib/ghidra-windows-fixture-options.mjs";
|
||||
|
||||
import { access, readFile } from "node:fs/promises";
|
||||
import { isAbsolute, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { parseConfig } from "../dist/config/parseConfig.js";
|
||||
import { resolveGhidraAnalysisProfile } from "../dist/ghidra/GhidraAnalysisProfile.js";
|
||||
import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
|
||||
import {
|
||||
@@ -20,7 +22,10 @@ import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js"
|
||||
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
|
||||
import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
|
||||
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderCapabilities.js";
|
||||
import { windowsP0Capabilities } from "../dist/ghidra/GhidraProviderCapabilities.js";
|
||||
import {
|
||||
WINDOWS_P0_NATIVE_PE_ARCHITECTURES,
|
||||
windowsP0Capabilities,
|
||||
} from "../dist/ghidra/GhidraProviderCapabilities.js";
|
||||
import {
|
||||
GHIDRA_MUTATING_OPERATIONS,
|
||||
GHIDRA_SESSION_CAPABILITIES,
|
||||
@@ -71,15 +76,11 @@ if (
|
||||
);
|
||||
|
||||
const root = resolve(fileURLToPath(new URL("..", import.meta.url)));
|
||||
const architecture = process.argv.includes("--x86") ? "x86" : "x86_64";
|
||||
const dllFixture = process.argv.includes("--dll");
|
||||
const fixture = windowsGhidraFixtureOptions(process.argv.slice(2));
|
||||
const architecture = fixture.architecture;
|
||||
const dllFixture = fixture.dll;
|
||||
const role = dllFixture ? "shared-library" : "application";
|
||||
const targetPath = resolve(
|
||||
root,
|
||||
"build",
|
||||
"fixtures",
|
||||
`rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
|
||||
);
|
||||
const targetPath = resolve(root, "build", "fixtures", fixture.filename);
|
||||
const target = await parseBinaryTarget(targetPath);
|
||||
if (
|
||||
!target.ok ||
|
||||
@@ -91,10 +92,15 @@ if (
|
||||
throw new Error(
|
||||
`Controlled Windows fixture failed admission: ${JSON.stringify(target)}`,
|
||||
);
|
||||
const configuration = parseConfig(process.env);
|
||||
if (!configuration.ok) throw configuration.error;
|
||||
const profile = await resolveGhidraAnalysisProfile(
|
||||
target.value,
|
||||
GHIDRA_PROVIDER_IDENTITY,
|
||||
installation,
|
||||
undefined,
|
||||
undefined,
|
||||
configuration.value.ghidraDisabledAnalyzers,
|
||||
);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Windows Ghidra profile could not be committed");
|
||||
@@ -113,6 +119,9 @@ const client = new GhidraClient({
|
||||
new URL("../bridge/ghidra/ReaGhidraBridge.java", import.meta.url),
|
||||
),
|
||||
platform: "win32",
|
||||
...(configuration.value.ghidraDisabledAnalyzers === undefined
|
||||
? {}
|
||||
: { disabledAnalyzers: configuration.value.ghidraDisabledAnalyzers }),
|
||||
}),
|
||||
targetPath: target.value.path,
|
||||
targetSha256: target.value.sha256,
|
||||
@@ -134,8 +143,8 @@ try {
|
||||
target.value.sha256,
|
||||
profile.value.profile.digest,
|
||||
);
|
||||
const languagePrefix = architecture === "x86" ? "x86:LE:32:" : "x86:LE:64:";
|
||||
if (!started.value.target.language_id.startsWith(languagePrefix))
|
||||
const languageId = fixture.languageId;
|
||||
if (started.value.target.language_id !== languageId)
|
||||
throw new Error("Windows fixture imported with the wrong Ghidra language");
|
||||
coordinates = client.diagnostics();
|
||||
assertRuntimeCoordinates(coordinates, target.value.sha256);
|
||||
@@ -161,7 +170,7 @@ try {
|
||||
);
|
||||
if (procedure === undefined)
|
||||
throw new Error("Windows Ghidra fixture exposed no local procedure");
|
||||
const calleeAddress = architecture === "x86" ? 0x401010n : 0x140001010n;
|
||||
const calleeAddress = BigInt(fixture.calleeAddress);
|
||||
const callee = procedures.find(
|
||||
(item) => BigInt(item.address) === calleeAddress,
|
||||
);
|
||||
@@ -175,7 +184,7 @@ try {
|
||||
"Windows fixture decompilation lost its known return value",
|
||||
);
|
||||
if (dllFixture) {
|
||||
const callerAddress = architecture === "x86" ? "0x401000" : "0x140001000";
|
||||
const callerAddress = fixture.callerAddress;
|
||||
const callees = await functionOperation("procedure_callees", {
|
||||
procedure: callerAddress,
|
||||
});
|
||||
@@ -223,7 +232,7 @@ try {
|
||||
const byteSearch = await verifyWindowsGhidraByteSearch({
|
||||
invoke: inventory,
|
||||
sourceBytes,
|
||||
architecture,
|
||||
imageBase: fixture.imageBase,
|
||||
});
|
||||
|
||||
for (const operation of [
|
||||
@@ -268,12 +277,16 @@ try {
|
||||
|
||||
// The reference controls occupy the EXE fixture's additional function.
|
||||
// DLL fixtures retain their independent exported-caller/return controls.
|
||||
const compactReferences = dllFixture
|
||||
? null
|
||||
: await verifyGhidraCompactReferences(
|
||||
functionOperation,
|
||||
architecture === "x86",
|
||||
);
|
||||
// ARM/ARM64 fixtures verify function entry, decompiler readback and the
|
||||
// exported caller/callee relationship; the x86/x64 compact-reference
|
||||
// value-flow battery has no ARM/ARM64 instruction encoding in this fixture.
|
||||
const compactReferences =
|
||||
dllFixture || architecture === "arm" || architecture === "arm64"
|
||||
? null
|
||||
: await verifyGhidraCompactReferences(
|
||||
functionOperation,
|
||||
architecture === "x86",
|
||||
);
|
||||
const borlandSeh =
|
||||
architecture === "x86" && !dllFixture
|
||||
? await verifyGhidraBorlandSeh(functionObservationOperation, true)
|
||||
@@ -300,10 +313,11 @@ try {
|
||||
protected_dacl: true,
|
||||
handle_admission: true,
|
||||
},
|
||||
supported_target_architectures: [...WINDOWS_P0_NATIVE_PE_ARCHITECTURES],
|
||||
limitations: [
|
||||
"approved-non-sensitive-fixtures-only",
|
||||
"windows-x64-local-ntfs-only",
|
||||
"native-x86-and-x86-64-pe-applications-and-dlls-only",
|
||||
"native-pe-applications-and-dlls-only",
|
||||
"no-gui-or-mutation-authority",
|
||||
],
|
||||
};
|
||||
@@ -323,11 +337,24 @@ try {
|
||||
}
|
||||
if (report === undefined)
|
||||
throw new Error("Windows Ghidra report was not produced");
|
||||
report.file_offsets = await verifyGhidraFileOffsets(
|
||||
targetPath,
|
||||
resolve(root, "scripts", "rea.mjs"),
|
||||
architecture === "x86" ? "0x401010" : "0x140001010",
|
||||
);
|
||||
try {
|
||||
report.file_offsets = await verifyGhidraFileOffsets(
|
||||
targetPath,
|
||||
resolve(root, "scripts", "rea.mjs"),
|
||||
fixture.calleeAddress,
|
||||
);
|
||||
} catch (cause) {
|
||||
// Preserve completed real-provider facts if a separate adapter proof fails.
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({
|
||||
...report,
|
||||
ok: false,
|
||||
primary_verification: "complete",
|
||||
file_offsets: { status: "incomplete", failure_code: cause.code ?? null },
|
||||
})}\n`,
|
||||
);
|
||||
throw cause;
|
||||
}
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
|
||||
);
|
||||
|
||||
@@ -27,6 +27,12 @@ import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { execFileOutput } from "../dist/process/ExecFileOutput.js";
|
||||
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
|
||||
import { verifyKunaReferences } from "./lib/kuna-reference-verifier.mjs";
|
||||
import {
|
||||
nativeKunaStrings,
|
||||
verifyKunaStrings,
|
||||
} from "./lib/kuna-string-verifier.mjs";
|
||||
import { nativeKunaIncomingSites } from "./lib/kuna-native-reference-verifier.mjs";
|
||||
import {
|
||||
assertVerifierCleanupComplete,
|
||||
completeVerifierRun,
|
||||
@@ -69,18 +75,24 @@ const cacheRoot = join(workspace, "cache");
|
||||
// A server retains its workspace until both provider and transport cleanup
|
||||
// have positively completed, even if later lineage observation is empty.
|
||||
const pendingServers = new Set();
|
||||
// Unproven native-oracle stops retain their supervisors and this workspace.
|
||||
const retainedNative = new Set();
|
||||
let kunaVersion;
|
||||
|
||||
const FIXTURE_SOURCE = String.raw`
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
static const char rea_kuna_utf8[] __attribute__((used)) = "_φ( °-°)/ REA unicode marker";
|
||||
static const unsigned short rea_kuna_wide[] __attribute__((used)) = {
|
||||
'R','E','A',' ','w','i','d','e',' ','m','a','r','k','e','r',0
|
||||
};
|
||||
#include <string.h>
|
||||
unsigned rea_checksum(const unsigned char *p, size_t n) {
|
||||
__attribute__((noinline)) unsigned rea_checksum(const unsigned char *p, size_t n) {
|
||||
unsigned h = 2166136261u;
|
||||
for (size_t i = 0; i < n; i++) h = (h ^ p[i]) * 16777619u;
|
||||
return h;
|
||||
}
|
||||
int rea_classify(int c) {
|
||||
__attribute__((noinline)) int rea_classify(int c) {
|
||||
switch (c) {
|
||||
case 'a': return 1; case 'e': return 2; case 'i': return 3;
|
||||
case 'o': return 4; case 'u': return 5; default: return c > 'z' ? -1 : 0;
|
||||
@@ -252,7 +264,12 @@ const quality = (text) => ({
|
||||
});
|
||||
|
||||
/** Verify Kuna on one binary and measure its cold and warm answers. */
|
||||
const verifyKuna = async (path, functions) => {
|
||||
const verifyKuna = async (
|
||||
path,
|
||||
functions,
|
||||
expectedCaller,
|
||||
expectedStrings = [],
|
||||
) => {
|
||||
const sha256 = createHash("sha256")
|
||||
.update(await readFile(path))
|
||||
.digest("hex");
|
||||
@@ -260,12 +277,43 @@ const verifyKuna = async (path, functions) => {
|
||||
const answers = [];
|
||||
let coldFailure;
|
||||
try {
|
||||
answers.push({
|
||||
strings: await verifyKunaStrings({
|
||||
call: cold.call,
|
||||
cacheRoot,
|
||||
expectedStrings,
|
||||
nativeStrings: () =>
|
||||
nativeKunaStrings({
|
||||
executable: kuna,
|
||||
cacheRoot,
|
||||
workspace,
|
||||
binary: path,
|
||||
retain: (owner) => retainedNative.add(owner),
|
||||
}),
|
||||
}),
|
||||
});
|
||||
const listed = await cold.call("list_procedures");
|
||||
for (const [name, address] of functions) {
|
||||
assert.ok(
|
||||
listed.value.some((entry) => entry.address === address),
|
||||
`${name} at ${address} is missing from Kuna's procedure list for ${path}`,
|
||||
);
|
||||
const references = await verifyKunaReferences({
|
||||
call: cold.call,
|
||||
cacheRoot,
|
||||
name,
|
||||
address,
|
||||
expectedCaller,
|
||||
nativeIncoming: (targetAddress) =>
|
||||
nativeKunaIncomingSites({
|
||||
executable: kuna,
|
||||
cacheRoot,
|
||||
workspace,
|
||||
binary: path,
|
||||
address: targetAddress,
|
||||
retain: (outcome) => retainedNative.add(outcome),
|
||||
}),
|
||||
});
|
||||
const pseudocode = await cold.call("procedure_pseudo_code", {
|
||||
procedure: address,
|
||||
});
|
||||
@@ -305,6 +353,11 @@ const verifyKuna = async (path, functions) => {
|
||||
const dossier = await cold.call("analyze_function", {
|
||||
procedure: address,
|
||||
});
|
||||
assert.deepEqual(
|
||||
dossier.structured.relationship_coverage,
|
||||
references.relationship_coverage,
|
||||
"Function dossier lost the directional call coverage",
|
||||
);
|
||||
const combined = dossier.structured.raw_result.source;
|
||||
assert.equal(combined.kind, "project-export");
|
||||
assert.deepEqual(
|
||||
@@ -335,6 +388,7 @@ const verifyKuna = async (path, functions) => {
|
||||
response_bytes: exported.bytes,
|
||||
dossier_response_bytes: dossier.bytes,
|
||||
...quality(exported.value),
|
||||
references,
|
||||
});
|
||||
}
|
||||
} catch (cause) {
|
||||
@@ -466,12 +520,18 @@ try {
|
||||
const fixtureFunctions = await symbolAddresses(built, [
|
||||
"rea_checksum",
|
||||
"rea_classify",
|
||||
"main",
|
||||
]);
|
||||
assert.equal(fixtureFunctions.size, 2, "Fixture symbols were not found");
|
||||
assert.equal(fixtureFunctions.size, 3, "Fixture symbols were not found");
|
||||
const fixtureMain = fixtureFunctions.get("main");
|
||||
fixtureFunctions.delete("main");
|
||||
for (const binary of [built, stripped])
|
||||
results.push({
|
||||
binary: binary.slice(workspace.length + 1),
|
||||
kuna: await verifyKuna(binary, [...fixtureFunctions]),
|
||||
kuna: await verifyKuna(binary, [...fixtureFunctions], fixtureMain, [
|
||||
"_φ( °-°)/ REA unicode marker",
|
||||
"REA wide marker",
|
||||
]),
|
||||
});
|
||||
|
||||
// Caller-built matrix.
|
||||
@@ -482,8 +542,17 @@ try {
|
||||
failures.push(cause);
|
||||
}
|
||||
// Failure paths need the same process evidence as successful verification.
|
||||
for (const retained of [...retainedNative]) {
|
||||
try {
|
||||
const stopped = await retained.owner?.stop();
|
||||
if (stopped !== undefined && stopped.status !== "incomplete")
|
||||
retainedNative.delete(retained);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
const lifecycle = await completeVerifierRun(run);
|
||||
let cleanupComplete = pendingServers.size === 0;
|
||||
let cleanupComplete = pendingServers.size === 0 && retainedNative.size === 0;
|
||||
try {
|
||||
assertVerifierCleanupComplete(lifecycle.process_lineage);
|
||||
} catch (cause) {
|
||||
|
||||
@@ -0,0 +1,415 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { parseArgs } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { artifactCliEvidence } from "../../lib/artifact-e2e.mjs";
|
||||
import { closeVerifierResources } from "../../lib/verifier-cleanup.mjs";
|
||||
import { PrivateRuntimeRoot } from "../../../dist/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../../../dist/process/OwnedCommand.js";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
artifact: { type: "string" },
|
||||
"source-path": { type: "string", multiple: true },
|
||||
path: { type: "string" },
|
||||
kind: { type: "string" },
|
||||
rounds: { type: "string", default: "3" },
|
||||
mode: { type: "string", default: "parity" },
|
||||
"cli-parity": { type: "boolean", default: false },
|
||||
"tokenizer-python": { type: "string" },
|
||||
"tokenizer-timeout-ms": { type: "string", default: "60000" },
|
||||
},
|
||||
strict: true,
|
||||
});
|
||||
assert.ok(values.artifact?.startsWith("/"), "--artifact must be absolute");
|
||||
assert.ok(["parity", "legacy", "references"].includes(values.mode));
|
||||
const rounds = Number(values.rounds);
|
||||
assert.ok(Number.isSafeInteger(rounds) && rounds > 0);
|
||||
const tokenizerTimeoutMs = Number(values["tokenizer-timeout-ms"]);
|
||||
assert.ok(
|
||||
Number.isSafeInteger(tokenizerTimeoutMs) && tokenizerTimeoutMs > 0,
|
||||
"--tokenizer-timeout-ms must be a positive safe integer",
|
||||
);
|
||||
const where = {
|
||||
...(values.path === undefined ? {} : { path: values.path }),
|
||||
...(values.kind === undefined ? {} : { kind: values.kind }),
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [fileURLToPath(new URL("../../rea.mjs", import.meta.url)), "mcp"],
|
||||
env: {
|
||||
PATH: process.env.PATH ?? "/usr/bin:/bin",
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_ANALYSIS_PROVIDER: "auto",
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
const client = new Client({
|
||||
name: "javascript-module-reference-verifier",
|
||||
version: "1",
|
||||
});
|
||||
const serialize = (value) => JSON.stringify(value);
|
||||
const bytes = (value) => Buffer.byteLength(serialize(value));
|
||||
const digest = (value) =>
|
||||
createHash("sha256").update(serialize(value)).digest("hex");
|
||||
const support = (observations) => [
|
||||
...new Map(
|
||||
observations.map(({ evidence }) => {
|
||||
const basis = {
|
||||
authority: evidence.authority,
|
||||
state: evidence.state,
|
||||
confidence: evidence.confidence,
|
||||
};
|
||||
return [serialize(basis), basis];
|
||||
}),
|
||||
).values(),
|
||||
];
|
||||
const normalize = (rows) =>
|
||||
rows.map((row) => ({
|
||||
...row,
|
||||
basis: [...row.basis].sort((a, b) =>
|
||||
serialize(a).localeCompare(serialize(b)),
|
||||
),
|
||||
}));
|
||||
let temporary;
|
||||
let primaryError;
|
||||
let retainTemporary = false;
|
||||
try {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.resume();
|
||||
const tools = (await client.listTools()).tools;
|
||||
const relevant = tools.filter(({ name }) =>
|
||||
["analyze_javascript_application", "inspect_analysis_view"].includes(name),
|
||||
);
|
||||
assert.equal(relevant.length, 2);
|
||||
const call = async (name, args, transcript) => {
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ timeout: 360_000 },
|
||||
);
|
||||
if (response.isError) throw new Error(serialize(response));
|
||||
const evidence = response.structuredContent;
|
||||
assert.ok(evidence?.evidence_id && evidence.normalized_result);
|
||||
transcript?.push({
|
||||
request: { name, arguments: args },
|
||||
response: evidence,
|
||||
});
|
||||
return evidence;
|
||||
};
|
||||
const preparationStart = performance.now();
|
||||
const preparationRequest = {
|
||||
input_path: values.artifact,
|
||||
detail: "summary",
|
||||
...(values["source-path"] === undefined
|
||||
? {}
|
||||
: { source_paths: values["source-path"] }),
|
||||
};
|
||||
const summary = await call(
|
||||
"analyze_javascript_application",
|
||||
preparationRequest,
|
||||
);
|
||||
const source = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: summary.normalized_result.parent_evidence_id,
|
||||
};
|
||||
const preparationMs = performance.now() - preparationStart;
|
||||
const samples = [];
|
||||
const costInputs = {};
|
||||
let canonical;
|
||||
for (let round = 0; round < rounds; round += 1) {
|
||||
const modes =
|
||||
values.mode === "parity"
|
||||
? round % 2 === 0
|
||||
? ["legacy", "references"]
|
||||
: ["references", "legacy"]
|
||||
: [values.mode];
|
||||
for (const mode of modes) {
|
||||
const transcript = [];
|
||||
const started = performance.now();
|
||||
let rows;
|
||||
let coverage;
|
||||
if (mode === "legacy") {
|
||||
const page = await call(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "page",
|
||||
collection: "modules",
|
||||
offset: 0,
|
||||
limit: Number.MAX_SAFE_INTEGER,
|
||||
},
|
||||
},
|
||||
transcript,
|
||||
);
|
||||
assert.equal(page.normalized_result.coverage.exhausted, true);
|
||||
rows = [];
|
||||
for (const row of page.normalized_result.items) {
|
||||
if (where.path !== undefined && row.path !== where.path) continue;
|
||||
if (where.kind !== undefined && row.kind !== where.kind) continue;
|
||||
const itemEvidence = await call(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "item",
|
||||
collection: "modules",
|
||||
selector: { node_id: row.node_id },
|
||||
},
|
||||
},
|
||||
transcript,
|
||||
);
|
||||
const item = itemEvidence.normalized_result.item;
|
||||
rows.push({
|
||||
node_id: item.node_id,
|
||||
kind: item.kind,
|
||||
path: item.path,
|
||||
identity: item.identity,
|
||||
retained_source_range_count: item.source_ranges.length,
|
||||
basis: support(item.observations),
|
||||
});
|
||||
}
|
||||
coverage = page.normalized_result.coverage;
|
||||
} else {
|
||||
const references = await call(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where,
|
||||
offset: 0,
|
||||
limit: Number.MAX_SAFE_INTEGER,
|
||||
},
|
||||
},
|
||||
transcript,
|
||||
);
|
||||
const result = references.normalized_result;
|
||||
rows = result.rows;
|
||||
coverage = result.coverage;
|
||||
assert.equal(coverage.exhausted, true);
|
||||
assert.deepEqual(
|
||||
result.source_coverage.application,
|
||||
summary.normalized_result.summary.coverage.application,
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.source_coverage.source_selection,
|
||||
summary.normalized_result.summary.source_selection,
|
||||
);
|
||||
assert.equal(
|
||||
result.source_coverage.semantic.status,
|
||||
summary.normalized_result.summary.coverage.semantic.status,
|
||||
);
|
||||
assert.equal(
|
||||
result.source_coverage.application_unknown_count,
|
||||
summary.normalized_result.summary.coverage.application_unknowns,
|
||||
);
|
||||
assert.equal(
|
||||
result.source_coverage.semantic_unknown_count,
|
||||
summary.normalized_result.summary.coverage.semantic_unknowns,
|
||||
);
|
||||
}
|
||||
assert.ok(
|
||||
rows.length > 0,
|
||||
"No matching modules; select a representative nonempty task",
|
||||
);
|
||||
const normalized = normalize(rows);
|
||||
canonical ??= normalized;
|
||||
assert.deepEqual(
|
||||
normalized,
|
||||
canonical,
|
||||
"Module facts changed across workflows/rounds",
|
||||
);
|
||||
samples.push({
|
||||
round,
|
||||
mode,
|
||||
elapsed_ms: performance.now() - started,
|
||||
calls: transcript.length,
|
||||
structured_reply_bytes: transcript.reduce(
|
||||
(total, entry) => total + bytes(entry.response),
|
||||
0,
|
||||
),
|
||||
facts_sha256: digest(normalized),
|
||||
matching_modules: rows.length,
|
||||
coverage,
|
||||
});
|
||||
costInputs[mode] ??= {
|
||||
catalog: relevant.map(({ name, description, inputSchema }) => ({
|
||||
name,
|
||||
description,
|
||||
inputSchema,
|
||||
})),
|
||||
full_catalog: tools.map(({ name, description, inputSchema }) => ({
|
||||
name,
|
||||
description,
|
||||
inputSchema,
|
||||
})),
|
||||
common_summary: summary,
|
||||
common_request: {
|
||||
name: "analyze_javascript_application",
|
||||
arguments: preparationRequest,
|
||||
},
|
||||
transcript,
|
||||
};
|
||||
}
|
||||
}
|
||||
let cliParity = null;
|
||||
if (values["cli-parity"]) {
|
||||
temporary = await PrivateRuntimeRoot.create({
|
||||
prefix: "rea-module-reference-verifier-",
|
||||
});
|
||||
const bundlePath = join(temporary.path, "bundle.json");
|
||||
const exported = await client.callTool({
|
||||
name: "export_evidence_bundle",
|
||||
arguments: { path: bundlePath },
|
||||
});
|
||||
if (exported.isError) throw new Error(serialize(exported));
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find(
|
||||
({ evidence_id }) => evidence_id === source.evidence_id,
|
||||
);
|
||||
assert.ok(parent, "Export omitted the retained analysis parent");
|
||||
const request = join(temporary.path, "view.json");
|
||||
await writeFile(
|
||||
request,
|
||||
serialize({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where,
|
||||
offset: 0,
|
||||
limit: Number.MAX_SAFE_INTEGER,
|
||||
},
|
||||
}),
|
||||
);
|
||||
const cli = await artifactCliEvidence("inspect-analysis-view", request);
|
||||
assert.deepEqual(normalize(cli.normalized_result.rows), canonical);
|
||||
assert.equal(cli.normalized_result.parent_evidence_id, parent.evidence_id);
|
||||
const retained = await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: cli.normalized_result.view,
|
||||
});
|
||||
assert.deepEqual(cli.normalized_result, retained.normalized_result);
|
||||
assert.equal(cli.evidence_id, retained.evidence_id);
|
||||
cliParity = {
|
||||
evidence_id: cli.evidence_id,
|
||||
facts_sha256: digest(canonical),
|
||||
exact_cli_mcp_view_parity: true,
|
||||
};
|
||||
}
|
||||
let tokenCosts = null;
|
||||
if (values["tokenizer-python"]) {
|
||||
temporary ??= await PrivateRuntimeRoot.create({
|
||||
prefix: "rea-module-reference-verifier-",
|
||||
});
|
||||
const input = join(temporary.path, "token-input.json");
|
||||
await writeFile(input, serialize(costInputs));
|
||||
let result;
|
||||
try {
|
||||
result = await runOwnedCommand(
|
||||
{
|
||||
command: values["tokenizer-python"],
|
||||
arguments: [
|
||||
"-c",
|
||||
`
|
||||
import json, sys, tiktoken
|
||||
with open(sys.argv[1], encoding='utf-8') as source:
|
||||
data=json.load(source)
|
||||
enc=tiktoken.get_encoding('cl100k_base')
|
||||
def count(value):
|
||||
return len(enc.encode(json.dumps(value,ensure_ascii=False,separators=(',',':')),disallowed_special=()))
|
||||
out={}
|
||||
for mode, task in data.items():
|
||||
replies=sum(count(entry['response']) for entry in task['transcript'])
|
||||
requests=sum(count(entry['request']) for entry in task['transcript'])
|
||||
common=count(task['common_summary'])
|
||||
common_request=count(task['common_request'])
|
||||
selected=count(task['catalog'])
|
||||
full=count(task['full_catalog'])
|
||||
out[mode]=dict(selected_catalog=selected,full_catalog=full,common_request=common_request,common_summary=common,requests=requests,structured_replies=replies,selected_catalog_task_total=selected+common_request+common+requests+replies,full_catalog_task_total=full+common_request+common+requests+replies)
|
||||
print(json.dumps(dict(tokenizer='cl100k_base',tiktoken_version=tiktoken.__version__,costs=out)))
|
||||
`,
|
||||
input,
|
||||
],
|
||||
cwd: temporary.path,
|
||||
runId: `rea-module-token-${randomUUID()}`,
|
||||
// An interpreter wrapper may exec another command. The shared owner
|
||||
// still verifies native identities and the exact inherited run token.
|
||||
expectedCommand: null,
|
||||
sweepTokenOwnedProcesses: true,
|
||||
},
|
||||
{ timeoutMs: tokenizerTimeoutMs, diagnosticBytes: 1024 * 1024 },
|
||||
);
|
||||
} catch (cause) {
|
||||
// An incomplete cleanup carries its supervisor and identity resources in
|
||||
// the error. Preserve its private files for that owner instead of erasing
|
||||
// recovery data while a command may still be using it.
|
||||
retainTemporary =
|
||||
cause instanceof OwnedCommandFailure &&
|
||||
cause.cleanupOwner !== undefined;
|
||||
throw new Error(
|
||||
"Token accounting failed; inspect the owned-command outcome and cleanup status.",
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
assert.notEqual(
|
||||
result.stdout.invalidUtf8,
|
||||
true,
|
||||
"Tokenizer output must be UTF-8 JSON",
|
||||
);
|
||||
tokenCosts = JSON.parse(result.stdout.text);
|
||||
}
|
||||
process.stdout.write(
|
||||
`${serialize({
|
||||
scope:
|
||||
"Completed module-fact reads from identical retained Evidence; no hosted-model performance claim",
|
||||
artifact: {
|
||||
path: values.artifact,
|
||||
sha256: summary.normalized_result.artifact.sha256,
|
||||
},
|
||||
parent_evidence_id: source.evidence_id,
|
||||
source_selection: summary.normalized_result.summary.source_selection,
|
||||
source_coverage: summary.normalized_result.summary.coverage,
|
||||
where,
|
||||
preparation_ms: preparationMs,
|
||||
samples,
|
||||
token_costs: tokenCosts,
|
||||
cli_parity: cliParity,
|
||||
conditions:
|
||||
"First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children.",
|
||||
})}\n`,
|
||||
);
|
||||
} catch (cause) {
|
||||
primaryError = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
await closeVerifierResources(
|
||||
[
|
||||
() => client.close(),
|
||||
() => transport.close(),
|
||||
async () => {
|
||||
if (temporary === undefined) return;
|
||||
if (retainTemporary)
|
||||
throw new Error(
|
||||
`Command cleanup is incomplete; retaining its private input/Evidence files at ${temporary.path}.`,
|
||||
);
|
||||
await temporary.close();
|
||||
},
|
||||
],
|
||||
primaryError,
|
||||
);
|
||||
}
|
||||
@@ -7,7 +7,7 @@ import {
|
||||
stat,
|
||||
type FileHandle,
|
||||
} from "node:fs/promises";
|
||||
import { extname, isAbsolute, resolve } from "node:path";
|
||||
import { extname } from "node:path";
|
||||
|
||||
import { BinaryTargetError } from "../domain/configurationErrors.js";
|
||||
import {
|
||||
@@ -52,6 +52,7 @@ import {
|
||||
type ExecutableMetadata,
|
||||
} from "../domain/binaryTarget.js";
|
||||
import type { PeCodeViewRecord } from "../domain/binaryTargetTypes.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
const XML_PLIST_PROBE_CHUNK_BYTES = 64 * 1024;
|
||||
|
||||
@@ -80,7 +81,7 @@ export const parseBinaryTarget = async (
|
||||
const cwd = options.cwd ?? process.cwd();
|
||||
const hostArchitecture = options.hostArchitecture ?? process.arch;
|
||||
const { targetKind, formatHint, signal, architecture, entryPoint } = options;
|
||||
const candidate = isAbsolute(input) ? input : resolve(cwd, input);
|
||||
const candidate = absoluteCallerPath(input, { cwd });
|
||||
try {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
await access(candidate, constants.R_OK);
|
||||
|
||||
@@ -95,6 +95,46 @@ const firstAnalysisCancellation = <Value>(
|
||||
* Direct callers use `execute`; adapters that own validation use
|
||||
* `executeValidated` so an MCP input is parsed exactly once.
|
||||
*/
|
||||
|
||||
const NO_PSEUDOCODE =
|
||||
"The provider reports no pseudocode for this procedure; the batch limitations for procedure_pseudo_code say why.";
|
||||
|
||||
/** One batch item from its pseudocode observation, keeping null distinct from unreadable output. */
|
||||
const batchPseudocodeItem = (
|
||||
address: string,
|
||||
procedure: BatchProcedureIdentity,
|
||||
observed: Result<AnalysisExecution, AnalysisError>,
|
||||
): BatchDecompileItem => {
|
||||
const failed = (error: AnalysisError): BatchDecompileItem => ({
|
||||
address,
|
||||
procedure,
|
||||
status: "error",
|
||||
error: projectAnalysisError(error),
|
||||
});
|
||||
if (!observed.ok) return failed(observed.error);
|
||||
const pseudocode = observed.value.result;
|
||||
// The contract's null is the provider's own answer that this procedure has
|
||||
// no pseudocode (an import slot, a failed body); the provider's reason is
|
||||
// in the batch limitations, so it is not reported as unreadable output.
|
||||
if (pseudocode === null)
|
||||
return failed(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
observed.value.provider.id,
|
||||
"procedure_pseudo_code",
|
||||
NO_PSEUDOCODE,
|
||||
{ userMessage: NO_PSEUDOCODE },
|
||||
),
|
||||
);
|
||||
if (typeof pseudocode !== "string" || pseudocode.length === 0)
|
||||
return failed(
|
||||
new AnalysisOutputError(
|
||||
"procedure_pseudo_code",
|
||||
"provider returned empty pseudocode",
|
||||
),
|
||||
);
|
||||
return { address, procedure, status: "ok", pseudocode };
|
||||
};
|
||||
|
||||
export class EnhancedTools {
|
||||
constructor(private readonly analysis: AnalysisOperationPort) {}
|
||||
|
||||
@@ -375,7 +415,7 @@ export class EnhancedTools {
|
||||
): Promise<Result<BatchDecompileItem, AnalysisCancelledError>[]> {
|
||||
return Promise.all(
|
||||
ready.map(async ({ address, procedure, providerProcedure }) => {
|
||||
const result = await this.#call(
|
||||
const result = await this.#observe(
|
||||
"procedure_pseudo_code",
|
||||
{ procedure: providerProcedure },
|
||||
signal,
|
||||
@@ -383,32 +423,7 @@ export class EnhancedTools {
|
||||
);
|
||||
if (!result.ok && isAnalysisCancellation(result.error))
|
||||
return err(result.error);
|
||||
const item: BatchDecompileItem = !result.ok
|
||||
? {
|
||||
address,
|
||||
procedure,
|
||||
status: "error",
|
||||
error: projectAnalysisError(result.error),
|
||||
}
|
||||
: typeof result.value !== "string" || result.value.length === 0
|
||||
? {
|
||||
address,
|
||||
procedure,
|
||||
status: "error",
|
||||
error: projectAnalysisError(
|
||||
new AnalysisOutputError(
|
||||
"procedure_pseudo_code",
|
||||
"provider returned empty pseudocode",
|
||||
),
|
||||
),
|
||||
}
|
||||
: {
|
||||
address,
|
||||
procedure,
|
||||
status: "ok",
|
||||
pseudocode: result.value,
|
||||
};
|
||||
return ok(item);
|
||||
return ok(batchPseudocodeItem(address, procedure, result));
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -2,7 +2,6 @@ import { createHash, randomUUID } from "node:crypto";
|
||||
import { createReadStream } from "node:fs";
|
||||
import { stat as statPath } from "node:fs/promises";
|
||||
import type { Stats } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import {
|
||||
AnalysisAccessDeniedError,
|
||||
@@ -37,6 +36,7 @@ import {
|
||||
runOwnedCommand,
|
||||
} from "../../process/OwnedCommand.js";
|
||||
import type { ProviderProcessSupervisor } from "../../process/ProviderProcess.js";
|
||||
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
|
||||
|
||||
export type ReverseEngineeringOperation =
|
||||
| "inspect_with_objdump"
|
||||
@@ -291,7 +291,8 @@ export class ReverseEngineeringService {
|
||||
if (signal?.aborted) return err(new AnalysisCancelledError(operation));
|
||||
const cleanup = await this.retryCleanup(operation);
|
||||
if (!cleanup.ok) return cleanup;
|
||||
const absolutePath = path === undefined ? undefined : resolve(path);
|
||||
const absolutePath =
|
||||
path === undefined ? undefined : absoluteCallerPath(path);
|
||||
let digest: string | undefined;
|
||||
try {
|
||||
if (absolutePath !== undefined)
|
||||
|
||||
@@ -7,7 +7,7 @@ import {
|
||||
realpath,
|
||||
type FileHandle,
|
||||
} from "node:fs/promises";
|
||||
import { basename, dirname, isAbsolute, join, resolve } from "node:path";
|
||||
import { basename, dirname, isAbsolute, join } from "node:path";
|
||||
import type { Readable } from "node:stream";
|
||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
||||
import { streamChunkToBuffer } from "./StreamBytes.js";
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
destinationCaseCollisionMessage,
|
||||
normalizeArtifactPath,
|
||||
} from "./ArtifactPaths.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||
import { SafeOutputTreeCreationFailure } from "./SafeOutputTreeCreationFailure.js";
|
||||
import { hashOutputFile } from "./SafeOutputTreeReadback.js";
|
||||
@@ -92,7 +93,8 @@ export class SafeOutputTree {
|
||||
"path",
|
||||
"Extraction output root must be absolute",
|
||||
);
|
||||
const requested = resolve(outputRoot);
|
||||
// Path syntax follows the host; `platform` selects only the name policy.
|
||||
const requested = absoluteCallerPath(outputRoot);
|
||||
const name = basename(requested);
|
||||
if (name === "." || name === "..")
|
||||
throw new ArtifactReaderFailure("path", "Invalid extraction output root");
|
||||
|
||||
@@ -68,6 +68,9 @@ describe("Apple asset catalog application workflow", () => {
|
||||
const bundle = await createTestTempDirectory("rea-asset-catalog-absent-");
|
||||
await mkdir(join(bundle, "Contents", "Resources"), { recursive: true });
|
||||
await writeFile(join(bundle, "Contents", "Resources", "icon.icns"), "");
|
||||
// Names that merely end in Assets.car are ordinary resources.
|
||||
await mkdir(join(bundle, "Contents", "Resources", "NotAssets.car"));
|
||||
await writeFile(join(bundle, "Contents", "Resources", "OldAssets.car"), "");
|
||||
const result = await analyzeAppleAssetCatalogs({
|
||||
environment: {},
|
||||
bundlePath: bundle,
|
||||
|
||||
@@ -122,7 +122,9 @@ const collectAssetCatalogs = async (input: {
|
||||
let totalMetadataBytes = 0;
|
||||
try {
|
||||
for await (const entry of reader.entries(input.signal)) {
|
||||
if (!entry.path.endsWith("Assets.car")) continue;
|
||||
// Reader paths are `/`-separated; only an entry named exactly
|
||||
// Assets.car is a compiled catalog, not e.g. NotAssets.car.
|
||||
if (entry.path.split("/").at(-1) !== "Assets.car") continue;
|
||||
if (entry.kind !== "file") {
|
||||
uninspected.push(`${entry.path} (${entry.kind})`);
|
||||
continue;
|
||||
|
||||
@@ -56,6 +56,7 @@ const browserScenarioLimitations = (
|
||||
...(session.mode === "connect"
|
||||
? [
|
||||
"CDP attachment cannot recover pre-attach events or guarantee launch-time context options.",
|
||||
`Attached locale ${scenario.environment.locale} applies to Intl formatting only; navigator.language(s) and Accept-Language remain caller-controlled. Use launch mode for matching page and request languages.`,
|
||||
]
|
||||
: []),
|
||||
];
|
||||
|
||||
@@ -254,10 +254,20 @@ export class CdpConnection {
|
||||
if (pending === undefined) return;
|
||||
const hasResult = "result" in message;
|
||||
const hasError = "error" in message;
|
||||
// Chromium cannot route a rejection through a flat session after that
|
||||
// session disappears. Its browser-level reply omits sessionId, but the
|
||||
// connection-global command id still binds this exact terminal error.
|
||||
const detachedSessionError =
|
||||
pending.sessionId !== undefined &&
|
||||
!("sessionId" in message) &&
|
||||
hasError &&
|
||||
isRecord(message.error) &&
|
||||
message.error.code === -32_001 &&
|
||||
message.error.message === "Session with given id not found.";
|
||||
if (
|
||||
(pending.sessionId === undefined
|
||||
? "sessionId" in message
|
||||
: message.sessionId !== pending.sessionId) ||
|
||||
: message.sessionId !== pending.sessionId && !detachedSessionError) ||
|
||||
hasResult === hasError
|
||||
) {
|
||||
this.#failPending("protocol_error");
|
||||
|
||||
@@ -730,3 +730,49 @@ it("keeps failed event finalization failed across cleanup retries", async () =>
|
||||
expect(finishEvents).toHaveBeenCalledTimes(1);
|
||||
expect(closeBrowser).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it.each(["launch", "connect"] as const)(
|
||||
"reports locale coverage in %s success and retained operation/cleanup failures",
|
||||
async (mode) => {
|
||||
const input = scenario({ mode });
|
||||
input.environment.locale = "fr-FR";
|
||||
for (const failures of [
|
||||
{},
|
||||
{
|
||||
capture: {
|
||||
call: 1,
|
||||
error: new AnalysisCancelledError("capture_browser_scenario"),
|
||||
},
|
||||
},
|
||||
{ close: new Error("fixture cleanup interrupted") },
|
||||
]) {
|
||||
const session = new FakeSession(mode, failures);
|
||||
const provider = new PlaywrightBrowserScenarioProvider(
|
||||
async () => session,
|
||||
);
|
||||
const result = await provider.captureScenario(input);
|
||||
expect(result.ok).toBe(!("capture" in failures || "close" in failures));
|
||||
const expected = {
|
||||
limitations:
|
||||
mode === "connect"
|
||||
? expect.arrayContaining([
|
||||
"Attached locale fr-FR applies to Intl formatting only; navigator.language(s) and Accept-Language remain caller-controlled. Use launch mode for matching page and request languages.",
|
||||
])
|
||||
: expect.not.arrayContaining([
|
||||
expect.stringContaining("Attached locale"),
|
||||
]),
|
||||
};
|
||||
if (result.ok) expect(result.value).toMatchObject(expected);
|
||||
else
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
details: {
|
||||
partial_observation: {
|
||||
kind: "browser-scenario-observation",
|
||||
capture: expected,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(session.closeCalls).toBe(1);
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -1,7 +1,7 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
|
||||
import { z } from "zod";
|
||||
import type { Page } from "playwright-core";
|
||||
import type { Frame, Page } from "playwright-core";
|
||||
|
||||
import { browserObservationEndpointSchema } from "../domain/browserObservation.js";
|
||||
import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
@@ -15,7 +15,14 @@ import {
|
||||
import { withPlaywrightExecutionBoundary } from "./PlaywrightExecutionBoundary.js";
|
||||
|
||||
const OPERATION = "capture_browser_scenario" as const;
|
||||
// Restoration performs several sequential CDP mutations before the transport's
|
||||
// own bounded close. A single round-trip deadline is too short for the complete
|
||||
// chain on a loaded Windows browser, while retries must still stay bounded.
|
||||
const POLICY_CLEANUP_CONFIRMATION_TIMEOUT_MS = 5_000;
|
||||
const scriptResponse = z.object({ identifier: z.string() });
|
||||
const restorationResponse = z.object({
|
||||
result: z.object({ value: z.literal("policy_owned").nullable() }),
|
||||
});
|
||||
const attachedEvent = z.object({
|
||||
sessionId: z.string(),
|
||||
targetInfo: z.object({ type: z.string() }),
|
||||
@@ -23,6 +30,8 @@ const attachedEvent = z.object({
|
||||
const detachedEvent = z.object({ sessionId: z.string() });
|
||||
type PolicyTarget = {
|
||||
readonly id: string;
|
||||
readonly detached: Promise<void>;
|
||||
readonly markDetached: () => void;
|
||||
ready: Promise<void>;
|
||||
identifier?: string;
|
||||
};
|
||||
@@ -41,6 +50,7 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
#closed = false;
|
||||
#released = false;
|
||||
#detached = false;
|
||||
#loadingStopped = false;
|
||||
#removeListener: (() => void) | undefined;
|
||||
#setup = new AbortController();
|
||||
|
||||
@@ -101,6 +111,7 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
if (targetInfo.type === "iframe") this.addTarget(sessionId);
|
||||
} else if (event.method === "Target.detachedFromTarget") {
|
||||
const id = detachedEvent.parse(event.params).sessionId;
|
||||
this.#targets.get(id)?.markDetached();
|
||||
this.#targets.delete(id);
|
||||
if (id === this.#session?.sessionId) this.#detached = true;
|
||||
}
|
||||
@@ -119,7 +130,17 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
}
|
||||
|
||||
private addTarget(id: string): void {
|
||||
const target: PolicyTarget = { id, ready: Promise.resolve() };
|
||||
this.#targets.get(id)?.markDetached();
|
||||
let markDetached: () => void = () => undefined;
|
||||
const detached = new Promise<void>((resolve) => {
|
||||
markDetached = resolve;
|
||||
});
|
||||
const target: PolicyTarget = {
|
||||
id,
|
||||
detached,
|
||||
markDetached,
|
||||
ready: Promise.resolve(),
|
||||
};
|
||||
this.#targets.set(id, target);
|
||||
const ready = this.initializeTarget(target);
|
||||
target.ready = ready;
|
||||
@@ -187,15 +208,7 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
private async settle(): Promise<void> {
|
||||
for (;;) {
|
||||
const targets = [...this.#targets.values()];
|
||||
await Promise.all(
|
||||
targets.map(async (target) => {
|
||||
try {
|
||||
await target.ready;
|
||||
} catch (cause: unknown) {
|
||||
if (this.#targets.get(target.id) === target) throw cause;
|
||||
}
|
||||
}),
|
||||
);
|
||||
await Promise.all(targets.map((target) => this.settleTarget(target)));
|
||||
if (
|
||||
targets.length === this.#targets.size &&
|
||||
targets.every((target) => this.#targets.get(target.id) === target)
|
||||
@@ -204,6 +217,71 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
}
|
||||
}
|
||||
|
||||
private async settleTarget(target: PolicyTarget): Promise<void> {
|
||||
try {
|
||||
await Promise.race([target.ready, target.detached]);
|
||||
} catch (cause: unknown) {
|
||||
if (this.#targets.get(target.id) === target) throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
private async settleTargetForRestore(target: PolicyTarget): Promise<void> {
|
||||
try {
|
||||
await Promise.race([target.ready, target.detached]);
|
||||
} catch {
|
||||
// Setup already reports its own failure. Cleanup must continue from the
|
||||
// side effects that were acquired before it failed, such as an init
|
||||
// script whose identifier was returned before auto-attach was rejected.
|
||||
}
|
||||
}
|
||||
|
||||
private restorationScript(): string {
|
||||
return `(() => {
|
||||
const restore = globalThis[${this.restoreKey}];
|
||||
if (typeof restore === "function" && restore(${this.secret}) === true) return null;
|
||||
const register = globalThis.ServiceWorkerContainer?.prototype.register;
|
||||
return typeof register === "function" && Object.hasOwn(register, Symbol.for(${this.marker}))
|
||||
? "policy_owned"
|
||||
: null;
|
||||
})()`;
|
||||
}
|
||||
|
||||
private async restoreMainFrame(id: string): Promise<void> {
|
||||
const { result } = restorationResponse.parse(
|
||||
await this.send(id, "Runtime.evaluate", {
|
||||
expression: this.restorationScript(),
|
||||
returnByValue: true,
|
||||
}),
|
||||
);
|
||||
if (result.value === "policy_owned")
|
||||
throw new Error(
|
||||
"service-worker policy restoration could not be confirmed",
|
||||
);
|
||||
}
|
||||
|
||||
private async restorePlaywrightFrame(frame: Frame): Promise<void> {
|
||||
try {
|
||||
const failure = await frame.evaluate(this.restorationScript());
|
||||
if (failure === "policy_owned")
|
||||
throw new Error(
|
||||
"service-worker policy restoration could not be confirmed",
|
||||
);
|
||||
if (failure !== null)
|
||||
throw new BrowserObservationError(OPERATION, "protocol_error");
|
||||
} catch (cause: unknown) {
|
||||
if (!this.page.isClosed() && !frame.isDetached()) throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
private async stopMainFrame(frame: Frame): Promise<void> {
|
||||
try {
|
||||
await frame.evaluate("globalThis.stop()");
|
||||
this.#loadingStopped = true;
|
||||
} catch (cause: unknown) {
|
||||
if (!this.page.isClosed() && !frame.isDetached()) throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
async assertBlocked(): Promise<void> {
|
||||
if (this.#session !== undefined) await this.settle();
|
||||
for (const frame of this.page.frames()) {
|
||||
@@ -246,7 +324,10 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
if (this.#restoration === restoration) this.#restoration = undefined;
|
||||
},
|
||||
);
|
||||
return withPlaywrightExecutionBoundary(() => restoration, 1_000);
|
||||
return withPlaywrightExecutionBoundary(
|
||||
() => restoration,
|
||||
POLICY_CLEANUP_CONFIRMATION_TIMEOUT_MS,
|
||||
);
|
||||
}
|
||||
|
||||
private async restore(stopLoading: boolean): Promise<void> {
|
||||
@@ -255,13 +336,20 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
const session = this.#session;
|
||||
if (session === undefined) return;
|
||||
if (!this.page.isClosed()) {
|
||||
const mainFrame = this.page.mainFrame();
|
||||
// This owner exists even with empty storage seeds. Stopped navigation is
|
||||
// independent of storage and restoration never waits for a load event.
|
||||
if (stopLoading && !this.#detached && session.sessionId !== undefined)
|
||||
await this.send(session.sessionId, "Page.stopLoading");
|
||||
if (stopLoading && !this.#loadingStopped) {
|
||||
if (!this.#detached && session.sessionId !== undefined) {
|
||||
await this.send(session.sessionId, "Page.stopLoading");
|
||||
this.#loadingStopped = true;
|
||||
} else {
|
||||
await this.stopMainFrame(mainFrame);
|
||||
}
|
||||
}
|
||||
await Promise.all(
|
||||
[...this.#targets.values()].map(({ ready }) =>
|
||||
ready.catch(() => undefined),
|
||||
[...this.#targets.values()].map((target) =>
|
||||
this.settleTargetForRestore(target),
|
||||
),
|
||||
);
|
||||
for (const target of this.#targets.values()) {
|
||||
@@ -280,19 +368,20 @@ export class PlaywrightScenarioServiceWorkers {
|
||||
}
|
||||
// Future documents cannot install our wrapper after the removals above.
|
||||
// A realm lost to navigation is not assumed restored: retain and retry.
|
||||
if (!this.#detached && session.sessionId !== undefined) {
|
||||
await this.restoreMainFrame(session.sessionId);
|
||||
} else {
|
||||
await this.restorePlaywrightFrame(mainFrame);
|
||||
}
|
||||
for (const frame of this.page.frames()) {
|
||||
// The owned CDP session evaluates the selected main realm directly,
|
||||
// without Playwright waiting on an in-flight document transition.
|
||||
if (frame === mainFrame) continue;
|
||||
if (frame.isDetached()) continue;
|
||||
try {
|
||||
await frame.evaluate(`(() => {
|
||||
const restore = globalThis[${this.restoreKey}];
|
||||
if (typeof restore === "function" && restore(${this.secret}) === true) return;
|
||||
const register = globalThis.ServiceWorkerContainer?.prototype.register;
|
||||
if (typeof register === "function" && Object.hasOwn(register, Symbol.for(${this.marker})))
|
||||
throw new Error("service-worker policy restoration could not be confirmed");
|
||||
})()`);
|
||||
} catch (cause: unknown) {
|
||||
if (!frame.isDetached()) throw cause;
|
||||
}
|
||||
// An empty URL has no committed document realm to restore. Future
|
||||
// documents cannot install the wrapper after the script removals.
|
||||
if (frame.url().length === 0) continue;
|
||||
await this.restorePlaywrightFrame(frame);
|
||||
}
|
||||
}
|
||||
// Detaching the exact owned session also releases its iframe auto-attach.
|
||||
|
||||
@@ -15,7 +15,7 @@ const mocks = vi.hoisted(() => ({
|
||||
policySend: vi.fn(),
|
||||
policyTransportClose: vi.fn(),
|
||||
policyEvent: undefined as
|
||||
| ((event: { method: string; params: unknown }) => void)
|
||||
| ((event: { method: string; params: unknown; sessionId?: string }) => void)
|
||||
| undefined,
|
||||
}));
|
||||
|
||||
@@ -118,6 +118,7 @@ const fixture = () => {
|
||||
addInitScript: vi.fn(async (_script: unknown) => undefined),
|
||||
context: () => context,
|
||||
frames: () => [page],
|
||||
mainFrame: () => page,
|
||||
isDetached: () => false,
|
||||
isClosed: (): boolean => false,
|
||||
waitForLoadState: vi.fn(async () => undefined),
|
||||
@@ -152,20 +153,27 @@ const fixture = () => {
|
||||
mocks.storage = {
|
||||
initialize: vi.fn(async () => undefined),
|
||||
settle: vi.fn(async () => undefined),
|
||||
close: vi.fn(async () => undefined),
|
||||
close: vi.fn(async (stopLoading = false) => ({
|
||||
loadingStopped: stopLoading,
|
||||
})),
|
||||
};
|
||||
return { cleanup, page, storage: mocks.storage };
|
||||
};
|
||||
|
||||
const policyResponse = (method: string): unknown =>
|
||||
method === "Page.addScriptToEvaluateOnNewDocument"
|
||||
? { identifier: "owned-policy-script" }
|
||||
: method === "Runtime.evaluate"
|
||||
? { result: { value: null } }
|
||||
: {};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
mocks.policySend.mockReset();
|
||||
mocks.policyTransportClose.mockReset();
|
||||
mocks.policyEvent = undefined;
|
||||
mocks.policySend.mockImplementation(async (method: string) =>
|
||||
method === "Page.addScriptToEvaluateOnNewDocument"
|
||||
? { identifier: "owned-policy-script" }
|
||||
: {},
|
||||
policyResponse(method),
|
||||
);
|
||||
mocks.policyTransportClose.mockResolvedValue(undefined);
|
||||
});
|
||||
@@ -197,6 +205,292 @@ it("stops attached navigation without storage and retries rejected policy cleanu
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("assigns attached navigation stopping to the storage owner once", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
|
||||
await expect(session.close(true)).resolves.toBe("disconnected-external");
|
||||
expect(storage?.close).toHaveBeenCalledWith(true);
|
||||
expect(
|
||||
mocks.policySend.mock.calls.filter(
|
||||
([method]) => method === "Page.stopLoading",
|
||||
),
|
||||
).toHaveLength(0);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("transfers attached navigation stopping after storage cleanup fails", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const storageFailure = new Error("storage stop unconfirmed");
|
||||
storage?.close.mockRejectedValueOnce(storageFailure);
|
||||
const policyFailure = new Error("policy stop unconfirmed");
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
mocks.policySend.mockRejectedValueOnce(policyFailure);
|
||||
|
||||
await expect(session.close(true)).rejects.toMatchObject({
|
||||
errors: [policyFailure, storageFailure],
|
||||
});
|
||||
expect(cleanup.close).not.toHaveBeenCalled();
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
expect(storage?.close).toHaveBeenCalledWith(true);
|
||||
expect(
|
||||
mocks.policySend.mock.calls.filter(
|
||||
([method]) => method === "Page.stopLoading",
|
||||
),
|
||||
).toHaveLength(2);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("transfers a late stop request after storage has already released", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const cancellation = new AbortController();
|
||||
storage?.close.mockResolvedValue({ loadingStopped: false });
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(
|
||||
attached,
|
||||
{},
|
||||
{
|
||||
signal: cancellation.signal,
|
||||
},
|
||||
);
|
||||
const restorationFailure = new Error("registration restoration failed");
|
||||
let failRestoration = true;
|
||||
mocks.policySend.mockImplementation(async (method: string) => {
|
||||
if (method === "Runtime.evaluate" && failRestoration) {
|
||||
failRestoration = false;
|
||||
throw restorationFailure;
|
||||
}
|
||||
return policyResponse(method);
|
||||
});
|
||||
|
||||
await expect(session.close(false)).rejects.toBe(restorationFailure);
|
||||
expect(cleanup.close).not.toHaveBeenCalled();
|
||||
cancellation.abort();
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
expect(storage?.close.mock.calls).toEqual([[false], [true]]);
|
||||
expect(
|
||||
mocks.policySend.mock.calls.filter(
|
||||
([method]) => method === "Page.stopLoading",
|
||||
),
|
||||
).toHaveLength(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("retains transferred navigation stopping after setup cleanup fails", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const cancellation = new AbortController();
|
||||
const setupFailure = new Error("storage initialization failed");
|
||||
storage?.initialize.mockImplementationOnce(async () => {
|
||||
cancellation.abort();
|
||||
throw setupFailure;
|
||||
});
|
||||
storage?.close.mockRejectedValueOnce(new Error("storage stop unconfirmed"));
|
||||
const policyFailure = new Error("policy stop unconfirmed");
|
||||
let failPolicyStop = true;
|
||||
mocks.policySend.mockImplementation(async (method: string) => {
|
||||
if (method === "Page.stopLoading" && failPolicyStop) {
|
||||
failPolicyStop = false;
|
||||
throw policyFailure;
|
||||
}
|
||||
return policyResponse(method);
|
||||
});
|
||||
let retainedCleanup: (() => Promise<unknown>) | undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
});
|
||||
|
||||
await expect(
|
||||
PlaywrightScenarioSession.open(
|
||||
attached,
|
||||
{},
|
||||
{
|
||||
signal: cancellation.signal,
|
||||
retainCleanup: (close) => {
|
||||
retainedCleanup = close;
|
||||
},
|
||||
},
|
||||
),
|
||||
).rejects.toBeInstanceOf(AggregateError);
|
||||
expect(cleanup.close).not.toHaveBeenCalled();
|
||||
if (retainedCleanup === undefined)
|
||||
throw new Error("failed cleanup was not retained");
|
||||
await expect(retainedCleanup()).resolves.toBeUndefined();
|
||||
expect(
|
||||
mocks.policySend.mock.calls.filter(
|
||||
([method]) => method === "Page.stopLoading",
|
||||
),
|
||||
).toHaveLength(2);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("allows the bounded policy cleanup chain to cross one second", async () => {
|
||||
const { cleanup } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
let releaseStop: () => void = () => undefined;
|
||||
let stopStarted: () => void = () => undefined;
|
||||
const stopped = new Promise<void>((resolve) => {
|
||||
releaseStop = resolve;
|
||||
});
|
||||
const started = new Promise<void>((resolve) => {
|
||||
stopStarted = resolve;
|
||||
});
|
||||
mocks.policySend.mockImplementation(async (method: string) => {
|
||||
if (method === "Page.stopLoading") {
|
||||
stopStarted();
|
||||
await stopped;
|
||||
}
|
||||
return policyResponse(method);
|
||||
});
|
||||
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
let settled = false;
|
||||
const closing = session.close(true).then(
|
||||
() => {
|
||||
settled = true;
|
||||
return { ok: true as const };
|
||||
},
|
||||
(cause: unknown) => {
|
||||
settled = true;
|
||||
return { ok: false as const, cause };
|
||||
},
|
||||
);
|
||||
await started;
|
||||
await vi.advanceTimersByTimeAsync(1_100);
|
||||
expect(settled).toBe(false);
|
||||
releaseStop();
|
||||
await expect(closing).resolves.toEqual({ ok: true });
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not await policy initialization after its child target detaches", async () => {
|
||||
const { cleanup } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
let childStarted: () => void = () => undefined;
|
||||
let releaseChild: () => void = () => undefined;
|
||||
const started = new Promise<void>((resolve) => {
|
||||
childStarted = resolve;
|
||||
});
|
||||
const held = new Promise<void>((resolve) => {
|
||||
releaseChild = resolve;
|
||||
});
|
||||
mocks.policySend.mockImplementation(
|
||||
async (method: string, _params: unknown, target: string) => {
|
||||
if (method === "Page.enable" && target === "child-session") {
|
||||
childStarted();
|
||||
await held;
|
||||
}
|
||||
return policyResponse(method);
|
||||
},
|
||||
);
|
||||
mocks.policyEvent?.({
|
||||
method: "Target.attachedToTarget",
|
||||
params: {
|
||||
sessionId: "child-session",
|
||||
targetInfo: { type: "iframe" },
|
||||
},
|
||||
sessionId: "policy-session",
|
||||
});
|
||||
await started;
|
||||
const closing = session.close();
|
||||
mocks.policyEvent?.({
|
||||
method: "Target.detachedFromTarget",
|
||||
params: { sessionId: "child-session" },
|
||||
});
|
||||
try {
|
||||
await expect(closing).resolves.toBe("disconnected-external");
|
||||
} finally {
|
||||
releaseChild();
|
||||
}
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("skips policy restoration for a child without a committed document", async () => {
|
||||
const { cleanup, page } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const pendingChild = {
|
||||
url: () => "",
|
||||
isDetached: () => false,
|
||||
evaluate: vi.fn(() => new Promise(() => undefined)),
|
||||
};
|
||||
const session = await PlaywrightScenarioSession.open(
|
||||
browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
}),
|
||||
{},
|
||||
{},
|
||||
);
|
||||
page.frames = () => [page, pendingChild as unknown as typeof page];
|
||||
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
expect(pendingChild.evaluate).not.toHaveBeenCalled();
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("rejects an init script accepted by CDP when the live realm is unprotected", async () => {
|
||||
const { page, cleanup } = fixture();
|
||||
page.evaluate
|
||||
@@ -229,6 +523,40 @@ it("rejects an init script accepted by CDP when the live realm is unprotected",
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("restores an owned policy script after later setup fails", async () => {
|
||||
const { cleanup } = fixture();
|
||||
const setupFailure = new Error("auto-attach rejected");
|
||||
mocks.policySend.mockImplementation(async (method: string) => {
|
||||
if (method === "Target.setAutoAttach") throw setupFailure;
|
||||
return policyResponse(method);
|
||||
});
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
});
|
||||
|
||||
await expect(PlaywrightScenarioSession.open(attached, {}, {})).rejects.toBe(
|
||||
setupFailure,
|
||||
);
|
||||
expect(mocks.policySend).toHaveBeenCalledWith(
|
||||
"Page.removeScriptToEvaluateOnNewDocument",
|
||||
{ identifier: "owned-policy-script" },
|
||||
"policy-session",
|
||||
);
|
||||
expect(
|
||||
mocks.policySend.mock.calls.some(
|
||||
([method]) => method === "Runtime.evaluate",
|
||||
),
|
||||
).toBe(true);
|
||||
expect(mocks.policyTransportClose).toHaveBeenCalledTimes(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("retains an owned script identifier after rejected removal", async () => {
|
||||
const { cleanup } = fixture();
|
||||
const attached = browserScenarioSchema.parse({
|
||||
@@ -283,6 +611,7 @@ it.each(["closed page", "detached target"])(
|
||||
"releases its policy transport after a %s",
|
||||
async (terminal) => {
|
||||
const { page } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
@@ -291,6 +620,7 @@ it.each(["closed page", "detached target"])(
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
if (terminal === "closed page") page.isClosed = () => true;
|
||||
@@ -299,16 +629,61 @@ it.each(["closed page", "detached target"])(
|
||||
method: "Target.detachedFromTarget",
|
||||
params: { sessionId: "policy-session" },
|
||||
});
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
await expect(session.close(true)).resolves.toBe("disconnected-external");
|
||||
expect(
|
||||
mocks.policySend.mock.calls.some(
|
||||
([method]) => method === "Target.detachFromTarget",
|
||||
),
|
||||
).toBe(false);
|
||||
expect(mocks.policyTransportClose).toHaveBeenCalledTimes(1);
|
||||
if (terminal === "detached target") {
|
||||
expect(page.evaluate).toHaveBeenCalledWith("globalThis.stop()");
|
||||
expect(page.evaluate).toHaveBeenCalledWith(
|
||||
expect.stringContaining("policy_owned"),
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("retains a detached policy connection until the main realm is restored", async () => {
|
||||
const { cleanup, page } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: "http://127.0.0.1:9222",
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
mocks.policyEvent?.({
|
||||
method: "Target.detachedFromTarget",
|
||||
params: { sessionId: "policy-session" },
|
||||
});
|
||||
page.evaluate
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockResolvedValueOnce("policy_owned")
|
||||
.mockResolvedValueOnce(null);
|
||||
|
||||
await expect(session.close(true)).rejects.toThrow(
|
||||
"service-worker policy restoration could not be confirmed",
|
||||
);
|
||||
expect(cleanup.close).not.toHaveBeenCalled();
|
||||
expect(mocks.policyTransportClose).not.toHaveBeenCalled();
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
expect(page.evaluate).toHaveBeenCalledWith("globalThis.stop()");
|
||||
expect(
|
||||
page.evaluate.mock.calls.filter(
|
||||
([script]) => script === "globalThis.stop()",
|
||||
),
|
||||
).toHaveLength(1);
|
||||
expect(mocks.policyTransportClose).toHaveBeenCalledTimes(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("uses browser disconnect as terminal storage release after detach failure", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const detachFailure = new Error("CDP detach rejected");
|
||||
@@ -379,7 +754,8 @@ it("blocks new attached service-worker registration without removing existing re
|
||||
});
|
||||
|
||||
it("retains the attached connection when registration restoration fails", async () => {
|
||||
const { cleanup, page } = fixture();
|
||||
const { cleanup } = fixture();
|
||||
mocks.storage = undefined;
|
||||
const attached = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
browser: {
|
||||
@@ -388,13 +764,26 @@ it("retains the attached connection when registration restoration fails", async
|
||||
target_id: "target",
|
||||
},
|
||||
environment: { service_workers: "block" },
|
||||
storage: {},
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(attached, {}, {});
|
||||
const failure = new Error("registration restoration failed");
|
||||
page.evaluate.mockRejectedValueOnce(failure);
|
||||
await expect(session.close()).rejects.toBe(failure);
|
||||
let failRestoration = true;
|
||||
mocks.policySend.mockImplementation(async (method: string) => {
|
||||
if (method === "Runtime.evaluate" && failRestoration) {
|
||||
failRestoration = false;
|
||||
throw failure;
|
||||
}
|
||||
return policyResponse(method);
|
||||
});
|
||||
await expect(session.close(true)).rejects.toBe(failure);
|
||||
expect(cleanup.close).not.toHaveBeenCalled();
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
expect(
|
||||
mocks.policySend.mock.calls.filter(
|
||||
([method]) => method === "Page.stopLoading",
|
||||
),
|
||||
).toHaveLength(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
await expect(session.close()).resolves.toBe("disconnected-external");
|
||||
});
|
||||
@@ -456,7 +845,7 @@ it("waits for a pending registration installation before cancelled cleanup", asy
|
||||
await installed;
|
||||
return { identifier: "owned-policy-script" };
|
||||
}
|
||||
return {};
|
||||
return policyResponse(method);
|
||||
});
|
||||
const cancellation = new AbortController();
|
||||
const opening = PlaywrightScenarioSession.open(
|
||||
@@ -482,9 +871,12 @@ it("waits for a pending registration installation before cancelled cleanup", asy
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(1);
|
||||
releaseInstallation();
|
||||
await outcome;
|
||||
expect(page.evaluate.mock.calls.at(-1)?.[0]).toContain(
|
||||
"policy restoration could not be confirmed",
|
||||
);
|
||||
const restoration = mocks.policySend.mock.calls.find(
|
||||
([method]) => method === "Runtime.evaluate",
|
||||
)?.[1];
|
||||
expect(restoration).toMatchObject({
|
||||
expression: expect.stringContaining("policy_owned"),
|
||||
});
|
||||
expect(mocks.policySend).toHaveBeenCalledWith(
|
||||
"Page.stopLoading",
|
||||
{},
|
||||
|
||||
@@ -73,6 +73,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
private readonly serviceWorkers: PlaywrightScenarioServiceWorkers | undefined;
|
||||
private readonly signal: AbortSignal | undefined;
|
||||
#storageReleased = false;
|
||||
#policyStopsLoading = false;
|
||||
|
||||
private constructor(
|
||||
private readonly opened: OpenedScenarioBrowser,
|
||||
@@ -260,19 +261,29 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
let storageReleased = false;
|
||||
const storageStopsLoading =
|
||||
storage !== undefined && options.signal?.aborted === true;
|
||||
let policyStopsLoading = !storageStopsLoading;
|
||||
const cleanup = async () => {
|
||||
let storageFailure: { readonly cause: unknown } | undefined;
|
||||
let policyFailure: { readonly cause: unknown } | undefined;
|
||||
let browserFailure: { readonly cause: unknown } | undefined;
|
||||
if (!storageReleased) {
|
||||
try {
|
||||
await storage?.close(options.signal?.aborted === true);
|
||||
const result = await storage?.close(storageStopsLoading);
|
||||
if (storageStopsLoading && result?.loadingStopped !== true)
|
||||
policyStopsLoading = true;
|
||||
} catch (failure: unknown) {
|
||||
storageFailure = { cause: failure };
|
||||
// Ownership transfers only after the storage attempt is no longer
|
||||
// in flight. Its failure does not establish that navigation stop
|
||||
// succeeded, so the policy owner must confirm it before release.
|
||||
policyStopsLoading = true;
|
||||
}
|
||||
}
|
||||
try {
|
||||
if (!opened.cleanup.browserClosed) await serviceWorkers?.close(true);
|
||||
if (!opened.cleanup.browserClosed)
|
||||
await serviceWorkers?.close(policyStopsLoading);
|
||||
} catch (failure: unknown) {
|
||||
policyFailure = { cause: failure };
|
||||
}
|
||||
@@ -394,16 +405,21 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
let storageFailure: { readonly cause: unknown } | undefined;
|
||||
let policyFailure: { readonly cause: unknown } | undefined;
|
||||
let browserFailure: { readonly cause: unknown } | undefined;
|
||||
if (stopLoading && this.storage === undefined)
|
||||
this.#policyStopsLoading = true;
|
||||
if (!this.#storageReleased) {
|
||||
try {
|
||||
await this.storage?.close(stopLoading);
|
||||
const result = await this.storage?.close(stopLoading);
|
||||
if (stopLoading && result?.loadingStopped !== true)
|
||||
this.#policyStopsLoading = true;
|
||||
} catch (failure: unknown) {
|
||||
storageFailure = { cause: failure };
|
||||
if (stopLoading) this.#policyStopsLoading = true;
|
||||
}
|
||||
}
|
||||
try {
|
||||
if (!this.opened.cleanup.browserClosed)
|
||||
await this.serviceWorkers?.close(stopLoading);
|
||||
await this.serviceWorkers?.close(this.#policyStopsLoading);
|
||||
} catch (failure: unknown) {
|
||||
policyFailure = { cause: failure };
|
||||
}
|
||||
|
||||
@@ -25,13 +25,15 @@ class FakeCdp extends EventEmitter {
|
||||
this.calls.push({ method, params });
|
||||
if (method === this.holdMethod) {
|
||||
const response =
|
||||
method === "Page.addScriptToEvaluateOnNewDocument"
|
||||
? { identifier: "late-script" }
|
||||
: method === "Debugger.setBreakpointByUrl"
|
||||
? { breakpointId: "late-breakpoint" }
|
||||
: method === "Storage.getStorageKey"
|
||||
? { storageKey: this.storageKey }
|
||||
: {};
|
||||
method === "Target.getTargetInfo"
|
||||
? { targetInfo: { targetId: "root-target" } }
|
||||
: method === "Page.addScriptToEvaluateOnNewDocument"
|
||||
? { identifier: "late-script" }
|
||||
: method === "Debugger.setBreakpointByUrl"
|
||||
? { breakpointId: "late-breakpoint" }
|
||||
: method === "Storage.getStorageKey"
|
||||
? { storageKey: this.storageKey }
|
||||
: {};
|
||||
return new Promise((resolve) => {
|
||||
this.heldReply = () => resolve(response);
|
||||
});
|
||||
@@ -273,16 +275,61 @@ it("retries the owning CDP detach after a failed attempt", async () => {
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
cdp.failDetachCount = 1;
|
||||
|
||||
await expect(owner?.close()).rejects.toMatchObject({
|
||||
await expect(owner?.close(true)).rejects.toMatchObject({
|
||||
_tag: "BrowserObservationError",
|
||||
reason: "cleanup_failed",
|
||||
});
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
expect(cdp.listenerCount("Debugger.paused")).toBe(1);
|
||||
|
||||
await owner?.close();
|
||||
await expect(owner?.close(true)).resolves.toEqual({
|
||||
loadingStopped: true,
|
||||
});
|
||||
expect(cdp.detachCalls).toBe(2);
|
||||
expect(cdp.listenerCount("Debugger.paused")).toBe(0);
|
||||
expect(
|
||||
cdp.calls.filter(({ method }) => method === "Page.stopLoading"),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("reports an unconfirmed stop when root initialization is still pending", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
cdp.holdMethod = "Target.getTargetInfo";
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const input = scenario();
|
||||
const owner = await PlaywrightScenarioStorage.create(
|
||||
{ context: contextApi as never, page: page as never },
|
||||
input,
|
||||
BrowserScenarioSecrets.resolve(input, {})!,
|
||||
);
|
||||
expect(owner).toBeDefined();
|
||||
const initialization = owner?.initialize(page as never, contextApi as never);
|
||||
await waitForCall(cdp, "Target.getTargetInfo");
|
||||
|
||||
const closing = owner?.close(true);
|
||||
cdp.releaseHeldReply();
|
||||
await expect(initialization).resolves.toBeUndefined();
|
||||
await expect(closing).resolves.toEqual({ loadingStopped: false });
|
||||
expect(
|
||||
cdp.calls.filter(({ method }) => method === "Page.stopLoading"),
|
||||
).toHaveLength(0);
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
});
|
||||
|
||||
it("reports and retains a confirmed navigation stop", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
|
||||
await expect(owner?.close(true)).resolves.toEqual({
|
||||
loadingStopped: true,
|
||||
});
|
||||
await expect(owner?.close(true)).resolves.toEqual({
|
||||
loadingStopped: true,
|
||||
});
|
||||
expect(
|
||||
cdp.calls.filter(({ method }) => method === "Page.stopLoading"),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("does not create storage instrumentation when there are no storage seeds", async () => {
|
||||
|
||||
@@ -106,6 +106,9 @@ type StorageInstallationOptions = {
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
};
|
||||
type StoragePage = { readonly context: BrowserContext; readonly page: Page };
|
||||
export type PlaywrightScenarioStorageCleanup = {
|
||||
readonly loadingStopped: boolean;
|
||||
};
|
||||
|
||||
/** Owns pre-script storage seeding and its CDP instrumentation for one scenario. */
|
||||
export class PlaywrightScenarioStorage {
|
||||
@@ -123,8 +126,9 @@ export class PlaywrightScenarioStorage {
|
||||
#generation = 0;
|
||||
#closed = false;
|
||||
#cleanupCompleted = false;
|
||||
#loadingStopped = false;
|
||||
#failure: BrowserObservationError | undefined;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#closePromise: Promise<PlaywrightScenarioStorageCleanup> | undefined;
|
||||
#detachPromise: Promise<void> | undefined;
|
||||
#rootInitialization: Promise<void> | undefined;
|
||||
#work: Promise<void> = Promise.resolve();
|
||||
@@ -203,6 +207,7 @@ export class PlaywrightScenarioStorage {
|
||||
const rootInfo = targetInfoResponse.parse(
|
||||
await this.cdp.send("Target.getTargetInfo"),
|
||||
);
|
||||
if (this.#closed) return;
|
||||
const root = this.createTarget(
|
||||
undefined,
|
||||
rootInfo.targetInfo.targetId,
|
||||
@@ -210,7 +215,6 @@ export class PlaywrightScenarioStorage {
|
||||
this.cdp.send(method as Parameters<CDPSession["send"]>[0], params),
|
||||
);
|
||||
this.#targets.set("", root);
|
||||
if (this.#closed) return;
|
||||
this.listen(page, context);
|
||||
const ready = this.initializeTarget(root);
|
||||
root.ready = ready;
|
||||
@@ -591,8 +595,9 @@ export class PlaywrightScenarioStorage {
|
||||
}
|
||||
}
|
||||
|
||||
close(stopLoading = false): Promise<void> {
|
||||
if (this.#cleanupCompleted) return Promise.resolve();
|
||||
close(stopLoading = false): Promise<PlaywrightScenarioStorageCleanup> {
|
||||
if (this.#cleanupCompleted)
|
||||
return Promise.resolve({ loadingStopped: this.#loadingStopped });
|
||||
if (this.#closePromise !== undefined) return this.#closePromise;
|
||||
const closing = this.closeResources(stopLoading).catch((cause: unknown) => {
|
||||
throw new BrowserObservationError(
|
||||
@@ -614,17 +619,21 @@ export class PlaywrightScenarioStorage {
|
||||
return closing;
|
||||
}
|
||||
|
||||
private async closeResources(stopLoading: boolean): Promise<void> {
|
||||
private async closeResources(
|
||||
stopLoading: boolean,
|
||||
): Promise<PlaywrightScenarioStorageCleanup> {
|
||||
this.#closed = true;
|
||||
let stopLoadingFailure: { readonly cause: unknown } | undefined;
|
||||
if (stopLoading) {
|
||||
if (stopLoading && !this.#loadingStopped) {
|
||||
try {
|
||||
const root = this.#targets.get("");
|
||||
if (root !== undefined)
|
||||
if (root !== undefined) {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => root.send("Page.stopLoading"),
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
this.#loadingStopped = true;
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
stopLoadingFailure = { cause };
|
||||
}
|
||||
@@ -674,5 +683,6 @@ export class PlaywrightScenarioStorage {
|
||||
this.#replies.clear();
|
||||
this.#cleanupCompleted = true;
|
||||
if (stopLoadingFailure !== undefined) throw stopLoadingFailure.cause;
|
||||
return { loadingStopped: this.#loadingStopped };
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import {
|
||||
createRecordedCrashService,
|
||||
createBinaryLayoutService,
|
||||
@@ -13,6 +12,7 @@ import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { inspectPeResourcesInputSchema } from "../domain/native/peResources.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Inspect operator-selected files through the same shared workflow as MCP. */
|
||||
export const registerBinaryDiagnosticsCommands = (
|
||||
@@ -39,7 +39,7 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
() =>
|
||||
pe.inspect(
|
||||
{
|
||||
path: resolve(args.path),
|
||||
path: absoluteCallerPath(args.path),
|
||||
max_file_bytes: options.maxFileBytes,
|
||||
max_entries: options.maxEntries,
|
||||
},
|
||||
@@ -63,7 +63,7 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectBinaryLayout, async () => {
|
||||
const result = await service.inspectAndClose(
|
||||
{ path: resolve(args.path) },
|
||||
{ path: absoluteCallerPath(args.path) },
|
||||
{ signal },
|
||||
);
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
@@ -90,7 +90,7 @@ export const registerBinaryDiagnosticsCommands = (
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectRecordedCrash, async () => {
|
||||
const result = await recorded.inspectAndClose(
|
||||
{
|
||||
path: resolve(args.path),
|
||||
path: absoluteCallerPath(args.path),
|
||||
include_debugger_context: options.debuggerContext,
|
||||
},
|
||||
{ signal },
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { inspectPacketFieldsInputSchema } from "../domain/captures/packetFields.js";
|
||||
import { z } from "incur";
|
||||
import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js";
|
||||
import { resolve } from "node:path";
|
||||
import { createPacketCaptureService } from "../composition/captures.js";
|
||||
import {
|
||||
packetCaptureOptionsSchema,
|
||||
@@ -14,6 +13,7 @@ import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
export const registerCaptureCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
@@ -105,7 +105,7 @@ export const registerCaptureCommands = (
|
||||
{
|
||||
...options,
|
||||
...(all_matches ? { match_limit: null } : {}),
|
||||
path: resolve(args.path),
|
||||
path: absoluteCallerPath(args.path),
|
||||
},
|
||||
{ signal },
|
||||
),
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { z } from "incur";
|
||||
import { stat } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import { resolveCliAnalysisSelector } from "./analysisSelector.js";
|
||||
import type { DirectAnalysis } from "../composition/directAnalysis.js";
|
||||
@@ -16,6 +15,7 @@ import { runCliJavaScriptApplicationAnalysis } from "./javascriptApplicationAnal
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { javascriptApplicationOptions } from "../cliObservationOptions.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Register provider-neutral binary overview and procedure CLI commands. */
|
||||
export const registerCoreBinaryCommands = (
|
||||
@@ -84,7 +84,7 @@ const registerOverviewCommands = (
|
||||
logCliCommand(logger, "analyze", () =>
|
||||
runCliJavaScriptApplicationAnalysis(
|
||||
{
|
||||
input_path: resolve(args.path),
|
||||
input_path: absoluteCallerPath(args.path),
|
||||
integrity_policy:
|
||||
options.integrityPolicy ?? defaultIntegrityPolicy,
|
||||
...(options.sourcePath === undefined
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { evmInterfaceModule } from "../composition/evm.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Resolve operator paths and preserve explicit encoding through the shared workflow. */
|
||||
export const registerEvmCommands = (
|
||||
@@ -30,7 +30,10 @@ export const registerEvmCommands = (
|
||||
evmInterfaceModule.cliCommand,
|
||||
async () => {
|
||||
const result = await service.inspect(
|
||||
{ path: resolve(args.path), encoding: args.encoding },
|
||||
{
|
||||
path: absoluteCallerPath(args.path),
|
||||
encoding: args.encoding,
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
return result.ok
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { FirmwareAnalysisService } from "../application/firmware/FirmwareAnalysisService.js";
|
||||
import { createFirmwareAnalysisProvider } from "../composition/firmware.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
@@ -11,6 +10,7 @@ import type { FirmwareOperation } from "../domain/firmware/firmwareAnalysis.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** One-shot firmware CLI commands backed by the same workflows as MCP. */
|
||||
export const registerFirmwareCommands = (
|
||||
@@ -50,7 +50,7 @@ export const registerFirmwareCommands = (
|
||||
CLI_COMMANDS.inspectFirmwareRegions,
|
||||
"inspect_firmware_regions",
|
||||
// CLI paths are operator-relative; resolve before shared validation.
|
||||
{ path: resolve(args.path) },
|
||||
{ path: absoluteCallerPath(args.path) },
|
||||
),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.extractFirmware, {
|
||||
@@ -75,8 +75,8 @@ export const registerFirmwareCommands = (
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
execute(CLI_COMMANDS.extractFirmware, "extract_firmware", {
|
||||
path: resolve(args.path),
|
||||
output_directory: resolve(args.outputDirectory),
|
||||
path: absoluteCallerPath(args.path),
|
||||
output_directory: absoluteCallerPath(args.outputDirectory),
|
||||
...(options.offset === undefined && options.length === undefined
|
||||
? {}
|
||||
: { range: { offset: options.offset, length: options.length } }),
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { createGoBinaryService } from "../composition/go.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -7,6 +6,7 @@ import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Resolve caller paths and inspect static build metadata through the shared workflow. */
|
||||
export const registerGoCommands = (cli: CliInstance, logger: Logger): void => {
|
||||
@@ -24,7 +24,11 @@ export const registerGoCommands = (cli: CliInstance, logger: Logger): void => {
|
||||
CLI_COMMANDS.inspectGoBinary,
|
||||
async () =>
|
||||
finishAnalysisCommand(
|
||||
() => service.inspect({ path: resolve(args.path) }, { signal }),
|
||||
() =>
|
||||
service.inspect(
|
||||
{ path: absoluteCallerPath(args.path) },
|
||||
{ signal },
|
||||
),
|
||||
service,
|
||||
"go-build-info",
|
||||
"inspect_go_binary",
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { createRustBinaryService } from "../composition/rust.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -7,6 +6,7 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Resolve caller paths and inspect embedded Rust dependencies through the shared workflow. */
|
||||
export const registerRustCommands = (
|
||||
@@ -24,7 +24,7 @@ export const registerRustCommands = (
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectRustBinary, async () => {
|
||||
const result = await service.inspect(
|
||||
{ path: resolve(args.path) },
|
||||
{ path: absoluteCallerPath(args.path) },
|
||||
{ signal },
|
||||
);
|
||||
return result.ok ? result.value : projectAnalysisError(result.error);
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { inspectSignalDecodersInputSchema } from "../domain/signals/signalDecoders.js";
|
||||
import { z } from "incur";
|
||||
import type { SignalRecordingService } from "../application/signals/SignalRecordingService.js";
|
||||
import { resolve } from "node:path";
|
||||
import { createSignalRecordingService } from "../composition/signals.js";
|
||||
import {
|
||||
signalRecordingOptionsSchema,
|
||||
@@ -14,6 +13,7 @@ import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
export const registerSignalCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
@@ -107,7 +107,7 @@ export const registerSignalCommands = (
|
||||
{
|
||||
...options,
|
||||
...(all_lines ? { line_limit: null } : {}),
|
||||
path: resolve(args.path),
|
||||
path: absoluteCallerPath(args.path),
|
||||
},
|
||||
{ signal },
|
||||
),
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { createSqliteDatabaseService } from "../composition/sqlite.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -9,6 +8,7 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Resolve an operator-selected SQLite snapshot through the shared inspection workflow. */
|
||||
export const registerSqliteCommands = (
|
||||
@@ -55,7 +55,7 @@ export const registerSqliteCommands = (
|
||||
const inspection = await service
|
||||
.inspect(
|
||||
{
|
||||
path: resolve(args.path),
|
||||
path: absoluteCallerPath(args.path),
|
||||
...(options.table === undefined
|
||||
? {}
|
||||
: { table: options.table }),
|
||||
|
||||
@@ -2,13 +2,13 @@ import { compareWasmInterfaces } from "../application/wasm/WasmInterfaceComparis
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { z } from "incur";
|
||||
import { resolve } from "node:path";
|
||||
import { wasmArtifactModule } from "../composition/wasm.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { withCommandCancellation } from "./commandCancellation.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
|
||||
/** Resolve all caller-selected artifact paths through the shared workflow. */
|
||||
export const registerWasmCommands = (
|
||||
@@ -64,10 +64,12 @@ export const registerWasmCommands = (
|
||||
async () => {
|
||||
const result = await service.inspect(
|
||||
{
|
||||
path: resolve(args.path),
|
||||
glue_paths: options.glue_paths?.map((path) => resolve(path)),
|
||||
path: absoluteCallerPath(args.path),
|
||||
glue_paths: options.glue_paths?.map((path) =>
|
||||
absoluteCallerPath(path),
|
||||
),
|
||||
candidate_paths: options.candidate_paths?.map((path) =>
|
||||
resolve(path),
|
||||
absoluteCallerPath(path),
|
||||
),
|
||||
},
|
||||
{ signal },
|
||||
|
||||
@@ -16,7 +16,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_analysis_view"),
|
||||
kind: "application",
|
||||
description:
|
||||
'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidence_id>"}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: <complete Evidence>}. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_accesses/user_strings and carry the source selection and coverage. Android package summaries retain all declarations, engine identity and counts without decoded manifest text. Android manifest pages use kind android, facet manifest, offset and limit in UTF-16 code units; source_status preserves partial decoded coverage even when the retained text page is exhausted. Dart AOT summary preserves target/ABI/library identity, snapshot headers and pool lower bounds. StringLine truncated
|
||||
'Inspect selected views of completed binary-layout, JavaScript application, native analyze_function, inspect_artifact, inventory_artifact, inspect_wasm_artifact, inspect_macho, decode_interface_builder, inspect_evm_interface, inspect_managed_artifact, inspect_managed_members, inspect_android_package, inspect_dart_aot or inspect_web_network_capture Evidence without repeating analysis. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. For a retained record use source: {kind: "retained-evidence", evidence_id: "<parent_evidence_id>"}. Use normalized_result.parent_evidence_id from a summary receipt. Source kind is required. Portable source uses {kind: "inline", evidence: <complete Evidence>}. For layout, JavaScript, inventory, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package and Dart AOT sources, request a summary with view: {kind: "summary"}; summary is a view kind and has no facet. The WASM summary gives validation, counts, WAT digest/length, tool profile and execution/network states without reading WAT text. Select a layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Typed module references use view {kind: "javascript", collection: "modules", projection: "references", where: {path: "main.js", kind: "javascript-module"}, offset: 0, limit: 64}; where matches exact retained strings (null selects unavailable paths). Rows include canonical identity, retained source-range count and observation support states. Source coverage and selection remain explicit; exhausting matching rows does not prove runtime loading or complete analysis. Inventory uses view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json"], offset: 0, limit: 64}. Inventory views page nodes, occurrences, edges or integrity contradictions; occurrence logical_paths matches exact recorded strings (omitted means all available, [] means none). source_complete distinguishes complete inventories from partial source records; page exhaustion does not prove absence in a partial source. WASM views use kind wasm and facet sections/imports/exports/candidates/glue/wat/headers/details/tool_profile; text facets use UTF-16 code units and source forms remain exact. Mach-O views use kind macho and facet entrypoints/architectures/build_metadata/load_commands/dependencies/imports/exports/segments/provenance; source_exhaustive false means unreturned rows may exist. Interface Builder views use kind interface-builder and collection documents/nodes/edges/coverage; nodes and edges accept an exact document relative_path. EVM summary returns all inferred interface candidates, carrier/decoded identities and explicit unknowns without hex or duplicate stdout. EVM views use kind evm and facet functions/bytecode_hex/stdout/stderr; function pages preserve the original candidate rows, and text offsets/limits count UTF-16 code units (hex has two characters per decoded byte). Exhausting candidate pages does not establish global interface completeness. Managed artifact summaries preserve complete identity/classification, metadata status, module/MVID, frameworks, coverage and limitations without unrelated inventories. Managed pages use kind managed and collection references/resources/attributes, retaining source coverage and build-local token scope; page exhaustion does not establish complete metadata. Managed member summaries keep identity, selection with unmatched names, coverage and per-collection counts; managed-members pages use collection types/exported_types/fields/methods/member_refs/call_edges/field_acLine truncated
|
||||
inputSchema: inspectAnalysisViewInputSchema,
|
||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||
examples: [
|
||||
@@ -94,6 +94,24 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
title:
|
||||
"Read typed module reference facts from retained application Evidence",
|
||||
input: {
|
||||
source: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: RETAINED_EXAMPLE_ID,
|
||||
},
|
||||
view: {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { path: "main.js", kind: "javascript-module" },
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
] as const satisfies readonly ToolContract[];
|
||||
@@ -124,7 +124,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"list_strings",
|
||||
"List every provider-defined string object, or filter to one address, as address/value pairs with encoding, terminator status, and byte length when available. Hopper reads complete typed objects and retains its original display text as provider_value; long literals may span adjacent unterminated objects.",
|
||||
"List every provider-reported string object or candidate, or filter to one address, as address/value pairs with encoding, terminator status, and byte length when available. Hopper reads complete typed objects and retains its original display text as provider_value; long literals may span adjacent unterminated objects. Kuna scans mapped printable sequences of at least five characters, including unterminated fragments; references are not computed.",
|
||||
z.object({ document, address: optionalAddress }),
|
||||
),
|
||||
official(
|
||||
@@ -196,7 +196,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"search_strings",
|
||||
"Search every provider-defined string object using literal matching by default or regex when requested. Results are deterministic and complete over those defined objects, not every text sequence in artifact bytes. Ghidra may omit packed or undefined literals, including Rust strings; an empty result does not prove text is absent. If an address is known, read_bytes and xrefs inspect its bytes and references. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker; a matching deadline leaves its API usable.",
|
||||
"Search every provider-reported string object or candidate using literal matching by default or regex when requested. Results are deterministic and complete over that provider inventory, not every text sequence in artifact bytes. Kuna scans mapped sequences of at least five characters, including unterminated fragments, without computing references. Ghidra may omit packed or undefined literals, including Rust strings; an empty result does not prove text is absent. If an address is known, read_bytes and xrefs inspect its bytes and references. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker; a matching deadline leaves its API usable.",
|
||||
z.object(analysisSearchInput),
|
||||
),
|
||||
official(
|
||||
|
||||
@@ -647,7 +647,11 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
|
||||
}),
|
||||
list_javascript_runtime_targets: browserEvidence,
|
||||
observe_javascript_runtime: browserEvidence,
|
||||
analyze_javascript_application: evidence,
|
||||
analyze_javascript_application: effects({
|
||||
mutatesSession: true,
|
||||
writesFilesystem: true,
|
||||
launchesProcess: true,
|
||||
}),
|
||||
inspect_electron_fuses: evidence,
|
||||
export_web_scripts: effects({
|
||||
mutatesSession: true,
|
||||
|
||||
@@ -481,6 +481,240 @@ it("projects JavaScript summary and preserves selected module observations", ()
|
||||
});
|
||||
});
|
||||
|
||||
it("projects typed module facts without source bodies and keeps source coverage independent of paging", () => {
|
||||
const analysis = analysisViewJavaScriptAnalysisWithSource();
|
||||
const parent = javascriptParent(analysis);
|
||||
const input = inspectAnalysisViewInputSchema.parse({
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidenceId },
|
||||
view: {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { path: "renderer.js" },
|
||||
limit: 1,
|
||||
},
|
||||
});
|
||||
const projected = projectAnalysisView(parent, input.view);
|
||||
if (!projected.ok) throw projected.error;
|
||||
if (projected.value.kind !== "javascript")
|
||||
throw new Error("expected JavaScript view");
|
||||
const node = analysis.graph.nodes[0];
|
||||
if (!node) throw new Error("missing module");
|
||||
expect(projected.value.rows).toEqual([
|
||||
{
|
||||
node_id: node.node_id,
|
||||
kind: node.kind,
|
||||
path: "renderer.js",
|
||||
identity: node.identity,
|
||||
retained_source_range_count: 0,
|
||||
basis: [
|
||||
{ authority: "artifact-bytes", state: "observed", confidence: "exact" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
expect(projected.value.coverage).toMatchObject({ total: 1, exhausted: true });
|
||||
expect(projected.value.source_coverage).toMatchObject({
|
||||
application: analysis.graph.coverage,
|
||||
semantic: { status: analysis.semantic_graph.coverage.status },
|
||||
});
|
||||
expect(JSON.stringify(projected.value.rows)).not.toMatch(
|
||||
/secret|should not leak|observations/,
|
||||
);
|
||||
const row = projected.value.rows[0];
|
||||
if (!row) throw new Error("missing reference");
|
||||
const retainedIdentity = structuredClone(node.identity);
|
||||
if (row.identity.strategy !== "content-digest")
|
||||
throw new Error("expected digest identity");
|
||||
row.identity.sha256 = "e".repeat(64);
|
||||
expect(node.identity).toEqual(retainedIdentity);
|
||||
const empty = projectAnalysisView(parent, {
|
||||
...input.view,
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { kind: "javascript-module" },
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
});
|
||||
if (!empty.ok) throw empty.error;
|
||||
expect(empty.value).toMatchObject({
|
||||
rows: [],
|
||||
coverage: { total: 0, exhausted: true },
|
||||
});
|
||||
});
|
||||
|
||||
it("counts retained range observations including repeats and skips unavailable locations", () => {
|
||||
const analysis = analysisViewJavaScriptAnalysis();
|
||||
const original = analysis.graph.nodes[0];
|
||||
const observed = original?.observations[0];
|
||||
if (!original || !observed) throw new Error("missing module observation");
|
||||
const range = {
|
||||
kind: "source-range",
|
||||
source: "renderer.js",
|
||||
start: { line: 1, column: 0 },
|
||||
end: { line: 1, column: 8 },
|
||||
};
|
||||
const node = createJavaScriptApplicationNode({
|
||||
kind: original.kind,
|
||||
identity: original.identity,
|
||||
observations: [
|
||||
...["first retained observation", "second retained observation"].map(
|
||||
(label) => ({
|
||||
label,
|
||||
properties: {},
|
||||
evidence: {
|
||||
...observed.evidence,
|
||||
location: { available: true, value: range },
|
||||
},
|
||||
}),
|
||||
),
|
||||
{
|
||||
label: "unavailable location",
|
||||
properties: {},
|
||||
evidence: {
|
||||
...observed.evidence,
|
||||
authority: "unknown",
|
||||
state: "unavailable",
|
||||
confidence: "unknown",
|
||||
location: {
|
||||
available: false,
|
||||
reason: "not-observed",
|
||||
detail: "Location was not recorded.",
|
||||
},
|
||||
limitations: ["Location was not recorded."],
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
const { graph_id: _graphId, ...application } = analysis.graph;
|
||||
const graph = createJavaScriptApplicationGraph({
|
||||
...application,
|
||||
nodes: [node],
|
||||
root_node_ids: [node.node_id],
|
||||
});
|
||||
const parent = javascriptParent(
|
||||
analysisViewBindJavaScriptGraphs(analysis, graph),
|
||||
);
|
||||
const references = projectAnalysisView(parent, {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: {},
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
});
|
||||
const item = projectAnalysisView(parent, {
|
||||
kind: "item",
|
||||
collection: "modules",
|
||||
selector: { node_id: node.node_id },
|
||||
});
|
||||
if (!references.ok) throw references.error;
|
||||
if (!item.ok) throw item.error;
|
||||
expect(references.value).toMatchObject({
|
||||
rows: [{ node_id: node.node_id, retained_source_range_count: 2 }],
|
||||
});
|
||||
expect(item.value).toMatchObject({ item: { source_ranges: [range, range] } });
|
||||
});
|
||||
|
||||
it("validates typed module selectors without coercing null or empty paths", () => {
|
||||
const source = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: javascriptParent().evidenceId,
|
||||
};
|
||||
const view = {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
};
|
||||
for (const path of [null, "", "renderer.js"])
|
||||
expect(
|
||||
inspectAnalysisViewInputSchema.safeParse({
|
||||
source,
|
||||
view: { ...view, where: { path } },
|
||||
}).success,
|
||||
).toBe(true);
|
||||
for (const where of [
|
||||
{ path: 1 },
|
||||
{ kind: "electron-main" },
|
||||
{ glob: "*.js" },
|
||||
])
|
||||
expect(
|
||||
inspectAnalysisViewInputSchema.safeParse({
|
||||
source,
|
||||
view: { ...view, where },
|
||||
}).success,
|
||||
).toBe(false);
|
||||
expect(
|
||||
projectAnalysisView(layoutParent(), {
|
||||
...view,
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: {},
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
}).ok,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps mixed observation support distinct instead of promoting a module to observed", () => {
|
||||
const analysis = analysisViewJavaScriptAnalysis();
|
||||
const original = analysis.graph.nodes[0];
|
||||
const observed = original?.observations[0];
|
||||
if (!original || !observed) throw new Error("missing module observation");
|
||||
const inferred = {
|
||||
...observed.evidence,
|
||||
authority: "static-relationship-inference" as const,
|
||||
state: "inferred" as const,
|
||||
confidence: "low" as const,
|
||||
limitations: ["Static candidate; runtime loading is unknown."],
|
||||
};
|
||||
const node = createJavaScriptApplicationNode({
|
||||
kind: original.kind,
|
||||
identity: original.identity,
|
||||
observations: [
|
||||
{
|
||||
label: observed.label,
|
||||
properties: observed.properties,
|
||||
evidence: observed.evidence,
|
||||
},
|
||||
{ label: "candidate one", properties: {}, evidence: inferred },
|
||||
{ label: "candidate two", properties: {}, evidence: inferred },
|
||||
],
|
||||
});
|
||||
const { graph_id: _graphId, ...application } = analysis.graph;
|
||||
const graph = createJavaScriptApplicationGraph({
|
||||
...application,
|
||||
nodes: [node],
|
||||
root_node_ids: [node.node_id],
|
||||
});
|
||||
const result = projectAnalysisView(
|
||||
javascriptParent(analysisViewBindJavaScriptGraphs(analysis, graph)),
|
||||
{
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: {},
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
},
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
if (result.value.kind !== "javascript")
|
||||
throw new Error("expected references");
|
||||
expect(result.value.rows[0]?.basis).toHaveLength(2);
|
||||
expect(result.value.rows[0]?.basis).toEqual(
|
||||
expect.arrayContaining([
|
||||
{ authority: "artifact-bytes", state: "observed", confidence: "exact" },
|
||||
{
|
||||
authority: "static-relationship-inference",
|
||||
state: "inferred",
|
||||
confidence: "low",
|
||||
},
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects ambiguous JavaScript module paths with candidate node ids", () => {
|
||||
const analysis = analysisViewJavaScriptAnalysis();
|
||||
const first = analysis.graph.nodes[0];
|
||||
@@ -535,6 +769,28 @@ it("rejects ambiguous JavaScript module paths with candidate node ids", () => {
|
||||
},
|
||||
],
|
||||
});
|
||||
const view = {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { path: "renderer.js" },
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
} as const;
|
||||
const ids: string[] = [];
|
||||
for (const offset of [0, 1, 2, Number.MAX_SAFE_INTEGER]) {
|
||||
const page = projectAnalysisView(parent, { ...view, offset });
|
||||
if (!page.ok) throw page.error;
|
||||
if (page.value.kind !== "javascript")
|
||||
throw new Error("expected references");
|
||||
ids.push(...page.value.rows.map(({ node_id }) => node_id));
|
||||
expect(page.value.coverage).toMatchObject({
|
||||
total: 2,
|
||||
next_offset: offset === 0 ? 1 : null,
|
||||
exhausted: offset !== 0,
|
||||
});
|
||||
}
|
||||
expect(ids).toEqual(graph.nodes.map(({ node_id }) => node_id));
|
||||
});
|
||||
|
||||
it("pages actual modules, including unknown and empty paths, without unrelated roles", () => {
|
||||
@@ -644,6 +900,24 @@ it("pages actual modules, including unknown and empty paths, without unrelated r
|
||||
collection: "modules",
|
||||
selector: { path: "" },
|
||||
});
|
||||
for (const [path, kind] of [
|
||||
[null, "javascript-module"],
|
||||
["", "source-module"],
|
||||
] as const) {
|
||||
const references = projectAnalysisView(parent, {
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { path },
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
});
|
||||
if (!references.ok) throw references.error;
|
||||
expect(references.value).toMatchObject({
|
||||
rows: [{ path, kind }],
|
||||
coverage: { total: 1, exhausted: true },
|
||||
});
|
||||
}
|
||||
expect(emptyPath.ok).toBe(true);
|
||||
const role = nodes.find((node) => node.kind === "electron-renderer");
|
||||
if (role === undefined) throw new Error("missing role");
|
||||
@@ -875,6 +1149,22 @@ it.each(["partial", "unknown"] as const)(
|
||||
});
|
||||
for (const view of [
|
||||
{ kind: "page", collection: "modules", offset: 0, limit: 10 },
|
||||
{
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: {},
|
||||
offset: 0,
|
||||
limit: 10,
|
||||
},
|
||||
{
|
||||
kind: "javascript",
|
||||
collection: "modules",
|
||||
projection: "references",
|
||||
where: { path: "not-retained.js" },
|
||||
offset: 0,
|
||||
limit: 10,
|
||||
},
|
||||
{
|
||||
kind: "item",
|
||||
collection: "modules",
|
||||
@@ -884,6 +1174,12 @@ it.each(["partial", "unknown"] as const)(
|
||||
const result = projectAnalysisView(parent, view);
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value.coverage.exhausted).toBe(true);
|
||||
if (result.value.kind === "javascript")
|
||||
expect(result.value.source_coverage).toMatchObject({
|
||||
application: { status, omitted_count: null },
|
||||
semantic: { status: "unknown" },
|
||||
application_unknown_count: 1,
|
||||
});
|
||||
expect(result.value.unknowns).toEqual([
|
||||
"1 application graph unknowns remain in the parent Evidence.",
|
||||
...messages.slice(0, 65),
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { z } from "zod";
|
||||
import {
|
||||
javascriptModuleReferencesRequestSchema,
|
||||
javascriptModuleReferenceSchema,
|
||||
javascriptModuleSourceCoverageSchema,
|
||||
} from "./javascriptModuleReferences.js";
|
||||
import { javaScriptSourceSelectionSchema } from "../javascript/javascriptSourceSelection.js";
|
||||
|
||||
import {
|
||||
@@ -216,6 +221,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||
facetViewSchema,
|
||||
itemViewSchema,
|
||||
pageViewSchema,
|
||||
javascriptModuleReferencesRequestSchema,
|
||||
nativeViewSchema,
|
||||
inventoryViewSchema,
|
||||
wasmViewRequestSchema,
|
||||
@@ -379,6 +385,16 @@ export const javascriptApplicationSummaryViewResultSchema = z.strictObject({
|
||||
parent_operation: z.literal("analyze_javascript_application"),
|
||||
});
|
||||
|
||||
/** Typed reference page; full item observations remain available separately. */
|
||||
export const javascriptModuleReferencesViewResultSchema = z.strictObject({
|
||||
...viewResultBase,
|
||||
kind: z.literal("javascript"),
|
||||
view: javascriptModuleReferencesRequestSchema,
|
||||
parent_operation: z.literal("analyze_javascript_application"),
|
||||
rows: z.array(javascriptModuleReferenceSchema),
|
||||
source_coverage: javascriptModuleSourceCoverageSchema,
|
||||
});
|
||||
|
||||
/** Exact summary carrier for artifact inspection's optional delivery contract. */
|
||||
export const artifactInventorySummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
@@ -527,6 +543,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
items: z.array(jsonValueSchema),
|
||||
...viewResultBase,
|
||||
}),
|
||||
javascriptModuleReferencesViewResultSchema,
|
||||
z.strictObject({
|
||||
kind: z.literal("native"),
|
||||
view: nativeViewSchema,
|
||||
@@ -647,6 +664,10 @@ export type UnsignedAnalysisView =
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "facet" }>, "view_digest">
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "item" }>, "view_digest">
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "page" }>, "view_digest">
|
||||
| Omit<
|
||||
Extract<AnalysisViewResult, { readonly kind: "javascript" }>,
|
||||
"view_digest"
|
||||
>
|
||||
| Omit<
|
||||
Extract<AnalysisViewResult, { readonly kind: "native" }>,
|
||||
"view_digest"
|
||||
|
||||
@@ -7,6 +7,7 @@ import { jsonObjectSchema, type JsonValue } from "../jsonValue.js";
|
||||
import type { JavaScriptApplicationAnalysisResult } from "../javascript/javascriptApplicationAnalysis.js";
|
||||
import type { ApplicationNode } from "../javascript/javascriptApplicationGraphSchemas.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import type { JavaScriptModuleReference } from "./javascriptModuleReferences.js";
|
||||
import {
|
||||
boundedViewMessages,
|
||||
completeWithinViewCoverage as completeCoverage,
|
||||
@@ -50,20 +51,52 @@ export const javascriptModulePath = (node: ApplicationNode): string | null => {
|
||||
return null;
|
||||
};
|
||||
|
||||
const moduleEntries = (
|
||||
nodes: readonly ApplicationNode[],
|
||||
): readonly {
|
||||
readonly node: ApplicationNode;
|
||||
interface ModuleEntry {
|
||||
readonly node: ApplicationNode & {
|
||||
readonly kind: JavaScriptModuleReference["kind"];
|
||||
};
|
||||
readonly path: string | null;
|
||||
}[] =>
|
||||
nodes
|
||||
.filter(
|
||||
(node) =>
|
||||
node.kind === "javascript-asset" ||
|
||||
node.kind === "javascript-module" ||
|
||||
node.kind === "source-module",
|
||||
)
|
||||
.map((node) => ({ node, path: javascriptModulePath(node) }));
|
||||
}
|
||||
|
||||
const isModuleNode = (node: ApplicationNode): node is ModuleEntry["node"] =>
|
||||
node.kind === "javascript-asset" ||
|
||||
node.kind === "javascript-module" ||
|
||||
node.kind === "source-module";
|
||||
|
||||
/** One owner for the module kinds and path semantics of every module view. */
|
||||
function* moduleEntries(
|
||||
nodes: readonly ApplicationNode[],
|
||||
): Generator<ModuleEntry> {
|
||||
for (const node of nodes) {
|
||||
if (isModuleNode(node)) yield { node, path: javascriptModulePath(node) };
|
||||
}
|
||||
}
|
||||
|
||||
/** Scan retained order once; materialize only the requested matching page. */
|
||||
const modulePage = (
|
||||
nodes: readonly ApplicationNode[],
|
||||
view: {
|
||||
readonly offset: number;
|
||||
readonly limit: number;
|
||||
readonly where?: Extract<
|
||||
AnalysisViewRequest,
|
||||
{ kind: "javascript" }
|
||||
>["where"];
|
||||
},
|
||||
): { entries: ModuleEntry[]; total: number } => {
|
||||
const entries: ModuleEntry[] = [];
|
||||
let total = 0;
|
||||
for (const entry of moduleEntries(nodes)) {
|
||||
if (view.where?.kind !== undefined && entry.node.kind !== view.where.kind)
|
||||
continue;
|
||||
if (view.where?.path !== undefined && entry.path !== view.where.path)
|
||||
continue;
|
||||
if (total >= view.offset && entries.length < view.limit)
|
||||
entries.push(entry);
|
||||
total += 1;
|
||||
}
|
||||
return { entries, total };
|
||||
};
|
||||
|
||||
const exportNames = (node: ApplicationNode): readonly string[] => {
|
||||
const names = new Set<string>();
|
||||
@@ -128,6 +161,35 @@ const moduleItem = (node: ApplicationNode): JsonValue => ({
|
||||
})),
|
||||
});
|
||||
|
||||
const moduleReference = ({
|
||||
node,
|
||||
path,
|
||||
}: ModuleEntry): JavaScriptModuleReference => {
|
||||
let ranges = 0;
|
||||
const support = new Map<string, JavaScriptModuleReference["basis"][number]>();
|
||||
for (const { evidence } of node.observations) {
|
||||
if (
|
||||
evidence.location.available &&
|
||||
evidence.location.value.kind === "source-range"
|
||||
)
|
||||
ranges += 1;
|
||||
const { authority, state, confidence } = evidence;
|
||||
support.set(`${authority}:${state}:${confidence}`, {
|
||||
authority,
|
||||
state,
|
||||
confidence,
|
||||
});
|
||||
}
|
||||
return {
|
||||
node_id: node.node_id,
|
||||
kind: node.kind,
|
||||
path,
|
||||
identity: node.identity,
|
||||
retained_source_range_count: ranges,
|
||||
basis: [...support.values()],
|
||||
};
|
||||
};
|
||||
|
||||
const applicationUnknownCount = (
|
||||
analysis: JavaScriptApplicationAnalysisResult,
|
||||
): number =>
|
||||
@@ -190,7 +252,7 @@ const parentFields = (
|
||||
};
|
||||
|
||||
const selectModule = (
|
||||
modules: ReturnType<typeof moduleEntries>,
|
||||
modules: readonly ModuleEntry[],
|
||||
selector: Extract<AnalysisViewRequest, { readonly kind: "item" }>["selector"],
|
||||
): Result<ApplicationNode, AnalysisError> => {
|
||||
if ("node_id" in selector) {
|
||||
@@ -249,15 +311,45 @@ export const projectJavaScriptApplicationView = (
|
||||
analysis: JavaScriptApplicationAnalysisResult,
|
||||
view: Exclude<AnalysisViewRequest, { kind: "api-surface" }>,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
if (view.kind !== "summary" && view.kind !== "item" && view.kind !== "page")
|
||||
if (
|
||||
view.kind !== "summary" &&
|
||||
view.kind !== "item" &&
|
||||
view.kind !== "page" &&
|
||||
view.kind !== "javascript"
|
||||
)
|
||||
return err(
|
||||
jsIncompatible(
|
||||
"JavaScript application Evidence supports summary, module item or module page views.",
|
||||
"JavaScript application Evidence supports summary, module item, module page or typed JavaScript reference views.",
|
||||
),
|
||||
);
|
||||
const shared = parentFields(parent, analysis);
|
||||
const modules = moduleEntries(analysis.graph.nodes);
|
||||
if (view.kind === "summary")
|
||||
if (view.kind === "javascript") {
|
||||
const { entries, total } = modulePage(analysis.graph.nodes, view);
|
||||
const rows = entries.map(moduleReference);
|
||||
const { families: _families, ...semantic } =
|
||||
analysis.semantic_graph.coverage;
|
||||
return ok({
|
||||
...shared,
|
||||
kind: "javascript",
|
||||
view,
|
||||
rows,
|
||||
source_coverage: {
|
||||
...(analysis.source_selection === undefined
|
||||
? {}
|
||||
: { source_selection: analysis.source_selection }),
|
||||
application: analysis.graph.coverage,
|
||||
semantic,
|
||||
application_unknown_count: applicationUnknownCount(analysis),
|
||||
semantic_unknown_count: analysis.semantic_graph.unknowns.length,
|
||||
},
|
||||
coverage: pageCoverage(view.offset, rows.length, total),
|
||||
});
|
||||
}
|
||||
if (view.kind === "summary") {
|
||||
const moduleCount = analysis.graph.nodes.reduce(
|
||||
(count, node) => count + Number(isModuleNode(node)),
|
||||
0,
|
||||
);
|
||||
return ok({
|
||||
kind: "summary",
|
||||
view,
|
||||
@@ -288,9 +380,10 @@ export const projectJavaScriptApplicationView = (
|
||||
}),
|
||||
limitation_count: analysis.limitations.length,
|
||||
},
|
||||
coverage: completeCoverage(modules.length, analysis.graph.nodes.length),
|
||||
coverage: completeCoverage(moduleCount, analysis.graph.nodes.length),
|
||||
...shared,
|
||||
});
|
||||
}
|
||||
if (view.kind === "item") {
|
||||
if (view.collection !== "modules")
|
||||
return err(
|
||||
@@ -298,6 +391,7 @@ export const projectJavaScriptApplicationView = (
|
||||
"Section and symbol collections apply to inspect_binary_layout Evidence.",
|
||||
),
|
||||
);
|
||||
const modules = [...moduleEntries(analysis.graph.nodes)];
|
||||
const selected = selectModule(modules, view.selector);
|
||||
if (!selected.ok) return selected;
|
||||
return ok({
|
||||
@@ -314,18 +408,17 @@ export const projectJavaScriptApplicationView = (
|
||||
"Section and symbol collections apply to inspect_binary_layout Evidence.",
|
||||
),
|
||||
);
|
||||
const items = modules
|
||||
.slice(view.offset, view.offset + view.limit)
|
||||
.map(({ node, path }) => ({
|
||||
node_id: node.node_id,
|
||||
kind: node.kind,
|
||||
path,
|
||||
}));
|
||||
const { entries, total } = modulePage(analysis.graph.nodes, view);
|
||||
const items = entries.map(({ node, path }) => ({
|
||||
node_id: node.node_id,
|
||||
kind: node.kind,
|
||||
path,
|
||||
}));
|
||||
return ok({
|
||||
kind: "page",
|
||||
view,
|
||||
items,
|
||||
coverage: pageCoverage(view.offset, items.length, modules.length),
|
||||
coverage: pageCoverage(view.offset, items.length, total),
|
||||
...shared,
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,61 @@
|
||||
import { z } from "zod";
|
||||
import { prefixedDigestSchema } from "../digests.js";
|
||||
import {
|
||||
applicationCoverageSchema,
|
||||
applicationGraphEvidenceSchema,
|
||||
applicationNodeIdentitySchema,
|
||||
} from "../javascript/javascriptApplicationEvidenceSchemas.js";
|
||||
import { javaScriptSemanticGraphInputSchema } from "../javascript/javascriptSemanticGraphSchemas.js";
|
||||
import { javaScriptSourceSelectionSchema } from "../javascript/javascriptSourceSelection.js";
|
||||
|
||||
const moduleKindSchema = z.enum([
|
||||
"javascript-asset",
|
||||
"javascript-module",
|
||||
"source-module",
|
||||
]);
|
||||
|
||||
/** Exact filters over retained owner facts; paths are not filesystem lookups. */
|
||||
export const javascriptModuleReferencesRequestSchema = z.strictObject({
|
||||
kind: z.literal("javascript"),
|
||||
collection: z.literal("modules"),
|
||||
projection: z.literal("references"),
|
||||
where: z
|
||||
.strictObject({
|
||||
path: z.string().nullable().optional(),
|
||||
kind: moduleKindSchema.optional(),
|
||||
})
|
||||
.default({}),
|
||||
offset: z.number().int().nonnegative().default(0),
|
||||
limit: z.number().int().positive().default(64),
|
||||
});
|
||||
|
||||
/** Unique support states, without promoting mixed observations to one fact. */
|
||||
const basisSchema = z.strictObject({
|
||||
authority: applicationGraphEvidenceSchema.shape.authority,
|
||||
state: applicationGraphEvidenceSchema.shape.state,
|
||||
confidence: applicationGraphEvidenceSchema.shape.confidence,
|
||||
});
|
||||
|
||||
/** Reusable facts rather than task-specific dependency interpretations. */
|
||||
export const javascriptModuleReferenceSchema = z.strictObject({
|
||||
node_id: prefixedDigestSchema("jag_node"),
|
||||
kind: moduleKindSchema,
|
||||
path: z.string().nullable(),
|
||||
identity: applicationNodeIdentitySchema,
|
||||
retained_source_range_count: z.number().int().nonnegative(),
|
||||
basis: z.array(basisSchema).min(1),
|
||||
});
|
||||
|
||||
export const javascriptModuleSourceCoverageSchema = z.strictObject({
|
||||
source_selection: javaScriptSourceSelectionSchema.exactOptional(),
|
||||
application: applicationCoverageSchema,
|
||||
semantic: javaScriptSemanticGraphInputSchema.shape.coverage.omit({
|
||||
families: true,
|
||||
}),
|
||||
application_unknown_count: z.number().int().nonnegative(),
|
||||
semantic_unknown_count: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
export type JavaScriptModuleReference = z.infer<
|
||||
typeof javascriptModuleReferenceSchema
|
||||
>;
|
||||
@@ -483,16 +483,19 @@ const summarize = (
|
||||
contradiction: 0,
|
||||
};
|
||||
for (const change of changes) counts[change.classification] += 1;
|
||||
// Change rows are keyed by logical path, and one unknown row for a duplicate
|
||||
// path stands for every left occurrence on it. Only occurrences on paths
|
||||
// without a left-side change are affirmatively unchanged.
|
||||
const changedPaths = new Set(
|
||||
changes
|
||||
.filter(({ classification }) => classification !== "added")
|
||||
.map(({ logical_path: path }) => path),
|
||||
);
|
||||
return {
|
||||
unchanged: covered
|
||||
? Math.max(
|
||||
0,
|
||||
left.occurrences.length -
|
||||
counts.removed -
|
||||
counts.changed -
|
||||
counts.contradiction -
|
||||
counts.unknown,
|
||||
)
|
||||
? left.occurrences.filter(
|
||||
({ logical_path: path }) => !changedPaths.has(path),
|
||||
).length
|
||||
: 0,
|
||||
...counts,
|
||||
unknown: covered
|
||||
|
||||
@@ -51,6 +51,8 @@ export type ExecutableMetadata =
|
||||
}
|
||||
| {
|
||||
readonly format: "pe";
|
||||
/** Original unsigned COFF machine, before CPU-family normalization. */
|
||||
readonly peMachine: number;
|
||||
readonly architecture: HeaderArchitecture;
|
||||
readonly availableArchitectures: readonly HeaderArchitecture[];
|
||||
readonly executableRole:
|
||||
@@ -318,6 +320,7 @@ export const parsePeRecord = (
|
||||
const characteristics = record.readUInt16LE(22);
|
||||
return ok({
|
||||
format: "pe",
|
||||
peMachine: machineType,
|
||||
architecture,
|
||||
availableArchitectures: [architecture],
|
||||
executableRole:
|
||||
|
||||
@@ -195,6 +195,8 @@ export type BinaryTarget =
|
||||
})
|
||||
| (ExecutableTarget & {
|
||||
readonly format: "pe";
|
||||
/** Original COFF machine; undefined means this header fact is unavailable. */
|
||||
readonly peMachine?: number;
|
||||
/** Provider-neutral PE image role observed from COFF characteristics. */
|
||||
readonly executableRole:
|
||||
| "application"
|
||||
|
||||
@@ -107,7 +107,10 @@ export const browserScenarioEnvironmentSchema = z
|
||||
.string()
|
||||
.min(2)
|
||||
.regex(/^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$/u)
|
||||
.default("en-US"),
|
||||
.default("en-US")
|
||||
.describe(
|
||||
"Formatting locale. Launch also sets navigator.language(s) and Accept-Language; connect retains the caller's page and request languages and reports this limitation.",
|
||||
),
|
||||
timezone: z
|
||||
.string()
|
||||
.min(1)
|
||||
|
||||
@@ -99,6 +99,40 @@ it("preserves OpenStep dictionary and array autodetection", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("counts OpenStep __proto__ keys the decoder silently drops", () => {
|
||||
// Unquoted, quoted, escape-spelled, and object-valued keys all decode to
|
||||
// __proto__; values, other keys, and `\137` (decoded as "137") do not.
|
||||
const result = parseXmlPropertyList(
|
||||
'{ __proto__ = a; "__proto__" = b; "\\_\\_proto__" = c; ' +
|
||||
'nested = { "__proto__" = { polluted = yes; }; }; ' +
|
||||
'kept = __proto__; list = (__proto__); "\\137_proto__" = d; }',
|
||||
);
|
||||
expect(result).toEqual({
|
||||
value: {
|
||||
nested: {},
|
||||
kept: "__proto__",
|
||||
list: ["__proto__"],
|
||||
"137_proto__": "d",
|
||||
},
|
||||
omittedPrototypeKeys: 4,
|
||||
});
|
||||
const nested = (result.value as { nested: object }).nested;
|
||||
expect(Object.getPrototypeOf(nested)).toBe(Object.prototype);
|
||||
});
|
||||
|
||||
it("counts OpenStep compound keys that coerce to __proto__", () => {
|
||||
// `obj[key]` coerces an array key to its comma-joined string.
|
||||
expect(
|
||||
parseXmlPropertyList(
|
||||
'{ (__proto__) = a; (("__proto__")) = b; ("__pro", "to__") = c; ' +
|
||||
"list = (__proto__); }",
|
||||
),
|
||||
).toEqual({
|
||||
value: { "__pro,to__": "c", list: ["__proto__"] },
|
||||
omittedPrototypeKeys: 2,
|
||||
});
|
||||
});
|
||||
|
||||
/** The single-member chain of a nested dictionary, down to its deepest value. */
|
||||
const deepestValue = (value: unknown): unknown => {
|
||||
let cursor = value;
|
||||
|
||||
@@ -4,7 +4,7 @@ import {
|
||||
type Document,
|
||||
type Node,
|
||||
} from "@xmldom/xmldom";
|
||||
import { parse } from "plist";
|
||||
import { parse, parseOpenStep } from "plist";
|
||||
|
||||
import { MAX_PLIST_DEPTH } from "./apple/plistValue.js";
|
||||
|
||||
@@ -95,8 +95,14 @@ export const parseXmlPropertyList = (
|
||||
// by app Info.plist resolution. Only XML needs the key-element substitution.
|
||||
const first = text.trimStart()[0];
|
||||
if (first === "{" || first === "(") {
|
||||
assertOpenStepDepthWithinLimit(text);
|
||||
return omitPrototypeKeys(parse(text));
|
||||
// The pinned decoder assigns a `__proto__` key to the object prototype, so
|
||||
// the entry is gone before the decoded value can be counted. Count keys in
|
||||
// the source instead; the rebuild still drops any prototype it replaced.
|
||||
const prototypeKeys = scanOpenStep(text);
|
||||
return {
|
||||
value: omitPrototypeKeys(parse(text)).value,
|
||||
omittedPrototypeKeys: prototypeKeys,
|
||||
};
|
||||
}
|
||||
// Normalize only the literal line endings the pinned decoder normalizes.
|
||||
// Preserve character references in the source: serializing a DOM would turn
|
||||
@@ -117,6 +123,8 @@ export const parseXmlPropertyList = (
|
||||
interface OpenStepContainer {
|
||||
end: string;
|
||||
next: "key" | "equals" | "value" | "separator" | "after-value";
|
||||
/** Source offset of a container that is itself a dictionary key. */
|
||||
keyStart?: number;
|
||||
}
|
||||
|
||||
/** Consume only the punctuation expected between the pinned decoder's values. */
|
||||
@@ -143,8 +151,12 @@ const admitsOpenStepCloser = (
|
||||
parent.next !== "equals" &&
|
||||
!(parent.end === "}" && parent.next === "value");
|
||||
|
||||
/** Bound OpenStep value nesting before the pinned decoder recursively reads it. */
|
||||
const assertOpenStepDepthWithinLimit = (text: string): void => {
|
||||
/**
|
||||
* Bound OpenStep value nesting before the pinned decoder recursively reads it,
|
||||
* and count the dictionary keys that decode to `__proto__`.
|
||||
*/
|
||||
const scanOpenStep = (text: string): number => {
|
||||
let prototypeKeys = 0;
|
||||
const containers: OpenStepContainer[] = [];
|
||||
const completeValue = (): void => {
|
||||
const parent = containers.at(-1);
|
||||
@@ -174,16 +186,21 @@ const assertOpenStepDepthWithinLimit = (text: string): void => {
|
||||
// Stop at the first impossible token: only the pinned decoder owns syntax
|
||||
// diagnostics. A later deep suffix cannot change that failure into a limit.
|
||||
if (character === "}" || character === ")") {
|
||||
if (!admitsOpenStepCloser(parent, character)) return;
|
||||
containers.pop();
|
||||
if (!admitsOpenStepCloser(parent, character)) return prototypeKeys;
|
||||
const closed = containers.pop();
|
||||
if (
|
||||
closed?.keyStart !== undefined &&
|
||||
isOpenStepPrototypeKey(text.slice(closed.keyStart, cursor + 1))
|
||||
)
|
||||
prototypeKeys += 1;
|
||||
// parseOpenStep returns the first root value without reading trailing text.
|
||||
if (containers.length === 0) return;
|
||||
if (containers.length === 0) return prototypeKeys;
|
||||
completeValue();
|
||||
cursor += 1;
|
||||
continue;
|
||||
}
|
||||
const separator = consumeOpenStepSeparator(parent, character);
|
||||
if (separator === "invalid") return;
|
||||
if (separator === "invalid") return prototypeKeys;
|
||||
if (separator === "consumed") {
|
||||
cursor += 1;
|
||||
continue;
|
||||
@@ -193,16 +210,39 @@ const assertOpenStepDepthWithinLimit = (text: string): void => {
|
||||
containers.push({
|
||||
end: character === "{" ? "}" : ")",
|
||||
next: character === "{" ? "key" : "value",
|
||||
// A compound key is coerced to a string, e.g. `(__proto__)`.
|
||||
...(parent?.next === "key" ? { keyStart: cursor } : {}),
|
||||
});
|
||||
cursor += 1;
|
||||
continue;
|
||||
}
|
||||
const end = skipOpenStepScalar(text, cursor);
|
||||
if (end === cursor) return;
|
||||
if (parent?.next !== "key") assertDepth();
|
||||
if (end === cursor) return prototypeKeys;
|
||||
if (parent?.next === "key") {
|
||||
if (isOpenStepPrototypeKey(text.slice(cursor, end))) prototypeKeys += 1;
|
||||
} else assertDepth();
|
||||
cursor = end;
|
||||
completeValue();
|
||||
}
|
||||
return prototypeKeys;
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether one OpenStep key, scalar or compound, becomes the property name
|
||||
* `__proto__` when the decoder assigns `obj[key]`. Plain strings compare
|
||||
* directly; any other key is decoded by the pinned decoder itself and
|
||||
* coerced to a property name the same way that assignment does.
|
||||
*/
|
||||
const isOpenStepPrototypeKey = (token: string): boolean => {
|
||||
if (!/^["({<]/u.test(token)) return token === PROTOTYPE_KEY;
|
||||
if (token.startsWith('"') && !token.includes("\\"))
|
||||
return token === `"${PROTOTYPE_KEY}"`;
|
||||
try {
|
||||
return String(parseOpenStep(token)) === PROTOTYPE_KEY;
|
||||
} catch {
|
||||
// An unterminated token is the decoder's syntax error to report.
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
/** Skip grammar trivia without interpreting delimiter examples inside comments. */
|
||||
|
||||
@@ -136,4 +136,34 @@ describe("Swift alias ambiguity", () => {
|
||||
]);
|
||||
expect(result.categories.other?.items).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([undefined, "classes"] as const)(
|
||||
"keeps a decoded alias category beside an undecoded Swift alias (filter %s)",
|
||||
(category) => {
|
||||
const result = categorizeSwiftTypes(
|
||||
[{ address: "0x1", name: "main::Service::init" }],
|
||||
category === undefined ? {} : { category },
|
||||
[
|
||||
{ address: "0x1", name: "_$s4main7ServiceCMa" },
|
||||
{ address: "0x1", name: "_$s4main04PairVMa" },
|
||||
],
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
total: 1,
|
||||
categories: {
|
||||
classes: {
|
||||
count: 1,
|
||||
items: [
|
||||
{
|
||||
name: "main::Service::init",
|
||||
mangled_names: ["_$s4main7ServiceCMa", "_$s4main04PairVMa"],
|
||||
},
|
||||
],
|
||||
},
|
||||
other: { count: 0 },
|
||||
},
|
||||
unclassified: [],
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -115,13 +115,13 @@ export const categorizeSwiftTypes = (
|
||||
![entry.name, ...mangledNames].some((name) => name.includes(pattern))
|
||||
)
|
||||
continue;
|
||||
// Only decoded names vote. Unresolved aliases remain evidence without
|
||||
// erasing category evidence from a name the decoder did recognize.
|
||||
const kinds = new Set(
|
||||
mangledNames
|
||||
.filter((name) => classifySwiftSymbol(name) !== undefined)
|
||||
.map((name) => classifySwiftSymbol(name)?.category ?? null),
|
||||
);
|
||||
// Only decoded categories vote. A Swift-looking alias whose category is
|
||||
// not decoded remains evidence without erasing a decoded category.
|
||||
const kinds = new Set<SwiftTypeCategory>();
|
||||
for (const name of mangledNames) {
|
||||
const decoded = classifySwiftSymbol(name)?.category;
|
||||
if (decoded !== undefined && decoded !== null) kinds.add(decoded);
|
||||
}
|
||||
const category = kinds.size === 1 ? ([...kinds][0] ?? null) : null;
|
||||
const item = {
|
||||
address: entry.address,
|
||||
@@ -133,9 +133,7 @@ export const categorizeSwiftTypes = (
|
||||
unclassified.push({
|
||||
...item,
|
||||
reason:
|
||||
kinds.size > 1 && !kinds.has(null)
|
||||
? "conflicting_categories"
|
||||
: "category_not_decoded",
|
||||
kinds.size > 1 ? "conflicting_categories" : "category_not_decoded",
|
||||
});
|
||||
}
|
||||
const bucket = category ?? "other";
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import { posix, win32 } from "node:path";
|
||||
|
||||
/**
|
||||
* Make a caller-selected local path absolute without changing which file it
|
||||
* names. POSIX resolves `..` after following the preceding symlink, so lexical
|
||||
* normalization of `link/../file` can select a different file. On POSIX, only
|
||||
* interior `.` segments and repeated separators are removed, which never
|
||||
* change the named file; every `..` is kept for the filesystem to resolve, and
|
||||
* a trailing `/` or `/.` keeps its directory requirement as one trailing `/`.
|
||||
* A relative `cwd` is made absolute the same way. Win32 resolves `..`
|
||||
* lexically, so normalizing there is exact.
|
||||
*/
|
||||
export const absoluteCallerPath = (
|
||||
path: string,
|
||||
{
|
||||
cwd = process.cwd(),
|
||||
platform = process.platform,
|
||||
}: { readonly cwd?: string; readonly platform?: NodeJS.Platform } = {},
|
||||
): string => {
|
||||
if (platform === "win32") return win32.resolve(cwd, path);
|
||||
if (!posix.isAbsolute(path)) {
|
||||
const base = posix.isAbsolute(cwd)
|
||||
? cwd
|
||||
: absoluteCallerPath(cwd, { platform });
|
||||
return absoluteCallerPath(`${base}/${path}`, { platform });
|
||||
}
|
||||
const segments = path
|
||||
.split("/")
|
||||
.filter((segment) => segment !== "" && segment !== ".");
|
||||
const directory = /\/\.?$/u.test(path) && segments.length > 0;
|
||||
return `/${segments.join("/")}${directory ? "/" : ""}`;
|
||||
};
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
readlink,
|
||||
realpath,
|
||||
} from "node:fs/promises";
|
||||
import { isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { isAbsolute, join, relative, sep } from "node:path";
|
||||
import { Transform } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import {
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
AnalysisOutputError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { FirmwareRequest } from "../domain/firmware/firmwareAnalysis.js";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
||||
import { FIRMWARE_LIMITS } from "./FirmwareRelease.js";
|
||||
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
|
||||
@@ -63,7 +64,7 @@ export const snapshotFirmware = async (
|
||||
root: string,
|
||||
signal: AbortSignal,
|
||||
) => {
|
||||
const sourcePath = resolve(request.input.path);
|
||||
const sourcePath = absoluteCallerPath(request.input.path);
|
||||
const source = await realpath(sourcePath);
|
||||
const handle = await open(source, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||
const fullPath = join(root, "firmware.bin");
|
||||
|
||||
@@ -227,6 +227,17 @@ export const resolveGhidraAnalysisProfile = async (
|
||||
...(target.format === "pe"
|
||||
? { pdb_observations: "program-info-v5" }
|
||||
: {}),
|
||||
...(installation.platform === "win32" &&
|
||||
target.format === "pe" &&
|
||||
(target.architecture === "arm" || target.architecture === "arm64")
|
||||
? {
|
||||
windows_arm_pe_import:
|
||||
selectedLanguage === undefined
|
||||
? "loader-selected-language-v1"
|
||||
: "explicit-language-v1",
|
||||
windows_arm_pe_machine: target.peMachine ?? null,
|
||||
}
|
||||
: {}),
|
||||
annotation_policy: "atomic-function-entry-metadata-v1",
|
||||
load_image_observations: "source-mappings-entry-context-v2",
|
||||
function_body_evidence: "complete-inclusive-ranges-v1",
|
||||
|
||||
@@ -3,7 +3,7 @@ import { createHash } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
// Fake-backed provider coverage; real Ghidra verification lives in
|
||||
// `npm run verify:ghidra` and its focused variants.
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
@@ -376,6 +376,82 @@ describe("Ghidra Mach-O slice support", () => {
|
||||
});
|
||||
|
||||
describe("Ghidra platform support", () => {
|
||||
it.each([
|
||||
{ architecture: "arm", languageId: "ARM:LE:32:v7" },
|
||||
{ architecture: "arm64", languageId: "AARCH64:LE:64:v8A" },
|
||||
] as const)(
|
||||
"records explicit Windows $architecture processor selection without claiming loader selection",
|
||||
async ({ architecture, languageId }) => {
|
||||
const directory = await mkdtemp(
|
||||
join(tmpdir(), "rea-arm-profile-language-"),
|
||||
);
|
||||
onTestFinished(() => rm(directory, { recursive: true, force: true }));
|
||||
const languages = join(
|
||||
directory,
|
||||
"Ghidra/Processors/fixture/data/languages",
|
||||
);
|
||||
await mkdir(languages, { recursive: true });
|
||||
await writeFile(
|
||||
join(languages, "fixture.ldefs"),
|
||||
`<language_definitions><language processor="${architecture === "arm" ? "ARM" : "AARCH64"}" endian="little" size="${architecture === "arm" ? 32 : 64}" variant="fixture" id="${languageId}"><compiler name="Visual Studio" spec="fixture.cspec" id="windows"/></language></language_definitions>`,
|
||||
);
|
||||
const config = parseConfig({
|
||||
GHIDRA_INSTALL_DIR: directory,
|
||||
REA_GHIDRA_LANGUAGE_ID: languageId,
|
||||
REA_GHIDRA_COMPILER_SPEC_ID: "windows",
|
||||
});
|
||||
if (!config.ok) throw config.error;
|
||||
const ghidra = new GhidraProvider(
|
||||
config.value,
|
||||
silentLogger,
|
||||
{},
|
||||
installationHost("win32"),
|
||||
);
|
||||
const resolved = await ghidra.resolveAnalysisProfile(
|
||||
peTarget(architecture),
|
||||
);
|
||||
if (!resolved.ok) throw resolved.error;
|
||||
expect(resolved.value.profile?.parameters).toMatchObject({
|
||||
windows_arm_pe_import: "explicit-language-v1",
|
||||
windows_arm_pe_machine: peTarget(architecture).peMachine,
|
||||
language_id: languageId,
|
||||
compiler_spec_id: "windows",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("versions the Windows ARM PE import without changing other profile lanes", async () => {
|
||||
for (const platform of ["win32", "linux"] as const) {
|
||||
const ghidra = provider(installationHost(platform));
|
||||
for (const architecture of ["x86", "x86_64", "arm", "arm64"] as const) {
|
||||
const resolved = await ghidra.resolveAnalysisProfile(
|
||||
peTarget(architecture),
|
||||
);
|
||||
if (!resolved.ok) throw resolved.error;
|
||||
if (resolved.value.profile === null)
|
||||
throw new Error("Expected a PE analysis profile");
|
||||
const parameters = resolved.value.profile.parameters;
|
||||
if (
|
||||
platform === "win32" &&
|
||||
(architecture === "arm" || architecture === "arm64")
|
||||
)
|
||||
expect(parameters).toHaveProperty(
|
||||
"windows_arm_pe_import",
|
||||
"loader-selected-language-v1",
|
||||
);
|
||||
else expect(parameters).not.toHaveProperty("windows_arm_pe_import");
|
||||
if (
|
||||
platform === "win32" &&
|
||||
(architecture === "arm" || architecture === "arm64")
|
||||
)
|
||||
expect(parameters.windows_arm_pe_machine).toBe(
|
||||
peTarget(architecture).peMachine,
|
||||
);
|
||||
else expect(parameters).not.toHaveProperty("windows_arm_pe_machine");
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("commits the PE role and managed classification to profile identity", async () => {
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
const application = peTarget("x86_64");
|
||||
@@ -520,12 +596,13 @@ describe("Ghidra Windows platform support", () => {
|
||||
status: "unsupported",
|
||||
code: "target_format_unsupported",
|
||||
});
|
||||
// ARM64 PE is admitted on Windows P0 (bounded; see docs/windows-ghidra-p0.md).
|
||||
expect(
|
||||
ghidra.inspectTargetSupport(executableTarget("pe", "arm64")),
|
||||
).toMatchObject({
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
});
|
||||
ghidra.inspectTargetSupport(executableTarget("pe", "arm64")).status,
|
||||
).toBe("supported");
|
||||
expect(
|
||||
ghidra.inspectTargetSupport(executableTarget("pe", "arm")).status,
|
||||
).toBe("supported");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -625,12 +702,6 @@ describe("Windows Ghidra native failure diagnostics", () => {
|
||||
status: "unsupported",
|
||||
code: "target_format_unsupported",
|
||||
});
|
||||
expect(
|
||||
ghidra.inspectTargetSupport(executableTarget("pe", "arm64")),
|
||||
).toMatchObject({
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
});
|
||||
const client = ghidra.createClient(nativeApplication);
|
||||
const health = await client.execute("health", {});
|
||||
if (health.ok)
|
||||
@@ -649,18 +720,34 @@ describe("Windows Ghidra native failure diagnostics", () => {
|
||||
});
|
||||
|
||||
describe("Ghidra Windows PE fixture admission", () => {
|
||||
it.each(["arm", "arm64"] as const)(
|
||||
"does not infer a COFF machine from an unobserved %s family",
|
||||
(architecture) => {
|
||||
const { peMachine, ...unobserved } = peTarget(architecture);
|
||||
expect(peMachine).toBeDefined();
|
||||
expect(
|
||||
provider(installationHost("win32")).inspectTargetSupport(unobserved),
|
||||
).toMatchObject({
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
diagnostics: { pe_machine: null },
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("admits generated PE32 and PE32+ DLLs and applications while preserving native-only Windows admission", async () => {
|
||||
const root = fileURLToPath(new URL("../../", import.meta.url));
|
||||
await promisify(execFile)(process.execPath, [
|
||||
join(root, "scripts/create-ghidra-windows-fixture.mjs"),
|
||||
]);
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
for (const architecture of ["x86", "x86_64"] as const) {
|
||||
for (const architecture of ["x86", "x86_64", "arm", "arm64"] as const) {
|
||||
for (const dll of [false, true]) {
|
||||
const suffix = architecture === "x86_64" ? "" : `-${architecture}`;
|
||||
const path = join(
|
||||
root,
|
||||
"build/fixtures",
|
||||
`rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
|
||||
`rea-ghidra-windows${suffix}.${dll ? "dll" : "exe"}`,
|
||||
);
|
||||
const bytes = await readFile(path);
|
||||
const metadata = parseExecutableHeader(bytes, "x64");
|
||||
@@ -684,13 +771,34 @@ describe("Ghidra Windows PE fixture admission", () => {
|
||||
...metadata.value,
|
||||
};
|
||||
expect(ghidra.inspectTargetSupport(target).status).toBe("supported");
|
||||
if (architecture === "arm") {
|
||||
for (const peMachine of [0x01c0, 0x01c2]) {
|
||||
const legacy = Buffer.from(bytes);
|
||||
legacy.writeUInt16LE(peMachine, legacy.readUInt32LE(0x3c) + 4);
|
||||
const parsed = parseExecutableHeader(legacy, "x64");
|
||||
if (!parsed.ok) throw new Error(parsed.error);
|
||||
if (parsed.value.format !== "pe")
|
||||
throw new Error("Expected a PE fixture");
|
||||
expect(parsed.value).toMatchObject({ architecture, peMachine });
|
||||
expect(
|
||||
ghidra.inspectTargetSupport({
|
||||
...target,
|
||||
...parsed.value,
|
||||
sha256: createHash("sha256").update(legacy).digest("hex"),
|
||||
}),
|
||||
).toMatchObject({
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
diagnostics: { pe_machine: peMachine },
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const [candidate, code] of [
|
||||
[
|
||||
{ ...target, executableRole: "non-executable" },
|
||||
"target_role_unsupported",
|
||||
],
|
||||
[{ ...target, managed: true }, "managed_target_unsupported"],
|
||||
[{ ...target, architecture: "arm64" }, "architecture_unsupported"],
|
||||
] as const) {
|
||||
expect(ghidra.inspectTargetSupport(candidate)).toMatchObject({
|
||||
status: "unsupported",
|
||||
@@ -1300,8 +1408,34 @@ describe("Ghidra measured DOS load-image projection", () => {
|
||||
);
|
||||
});
|
||||
|
||||
const unavailableNativeAotDatabaseType = (address: string | null) => ({
|
||||
status: "unavailable",
|
||||
reason:
|
||||
"No exact database type or defined typed data exists at the selected identity",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
});
|
||||
|
||||
it("returns read-only NativeAOT metadata inline and drills down by derived MethodTable address", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const readSnapshot = vi.fn(async () =>
|
||||
ok({ kind: "captured" as const, bytes }),
|
||||
);
|
||||
const observations = fixtureDosLoadImage().observation;
|
||||
observations.image_base = "0x140000000";
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
@@ -1318,31 +1452,11 @@ it("returns read-only NativeAOT metadata inline and drills down by derived Metho
|
||||
jsonValueSchema.parse(
|
||||
operation === "inspect_native_load_image"
|
||||
? observations
|
||||
: {
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
},
|
||||
: unavailableNativeAotDatabaseType("0x140002280"),
|
||||
),
|
||||
),
|
||||
),
|
||||
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
|
||||
readTargetSnapshot: readSnapshot,
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
@@ -1371,67 +1485,155 @@ it("returns read-only NativeAOT metadata inline and drills down by derived Metho
|
||||
rawResult: observations,
|
||||
},
|
||||
});
|
||||
const detail = await client.execute("inspect_native_data_type", {
|
||||
address: "0x140002280",
|
||||
});
|
||||
expect(detail).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
status: "unavailable",
|
||||
source: "analysis-database",
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
for (const address of [
|
||||
"0x140002280",
|
||||
"140002280",
|
||||
"ram:0x140002280",
|
||||
"%72am:0x140002280",
|
||||
"0X0140002280",
|
||||
]) {
|
||||
const detail = await client.execute("inspect_native_data_type", {
|
||||
address,
|
||||
});
|
||||
expect(detail).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
status: "unavailable",
|
||||
source: "analysis-database",
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
}
|
||||
expect(readSnapshot).toHaveBeenCalledOnce();
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("derives PE metadata for a direct type-address query without a load-image call", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const operations: string[] = [];
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () =>
|
||||
Promise.resolve(
|
||||
ok({
|
||||
...sessionInfo(),
|
||||
target: { ...sessionInfo().target, image_base: "0x140000000" },
|
||||
}),
|
||||
),
|
||||
callTool: (operation) => {
|
||||
operations.push(operation);
|
||||
return Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse({
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
it.each([
|
||||
["0x140002280", "0x140002280", "0x140002280"],
|
||||
["140002280", "0x140002280", "0x140002280"],
|
||||
["ram:0x140002280", "0x140002280", "0x140002280"],
|
||||
["%72am:0x140002280", "0x140002280", "0x140002280"],
|
||||
["0X0140002280", "0x140002280", "0x140002280"],
|
||||
["0x140002280", "0x140002200", "0x140002200"],
|
||||
["0x140002280", null, null],
|
||||
["0x140002280", "register:0x140002280", null],
|
||||
["0x140002280", "overlay%3Aspace:0x140002280", null],
|
||||
["0x140002280", "0x140002288", null],
|
||||
["register:0x140002280", "register:0x140002280", null],
|
||||
] as const)(
|
||||
"looks up NativeAOT type %s using resolved identity %s",
|
||||
async (address, resolvedAddress, expectedTable) => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const operations: string[] = [];
|
||||
const callerAddresses: unknown[] = [];
|
||||
const readSnapshot = vi.fn(async () =>
|
||||
ok({ kind: "captured" as const, bytes }),
|
||||
);
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () =>
|
||||
Promise.resolve(
|
||||
ok({
|
||||
...sessionInfo(),
|
||||
target: { ...sessionInfo().target, image_base: "0x140000000" },
|
||||
}),
|
||||
),
|
||||
);
|
||||
},
|
||||
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
callTool: (operation, parameters) => {
|
||||
operations.push(operation);
|
||||
callerAddresses.push(parameters.address);
|
||||
return Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse(
|
||||
unavailableNativeAotDatabaseType(resolvedAddress),
|
||||
),
|
||||
),
|
||||
);
|
||||
},
|
||||
readTargetSnapshot: readSnapshot,
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const detail = await client.execute("inspect_native_data_type", {
|
||||
address,
|
||||
});
|
||||
expect(detail.ok).toBe(true);
|
||||
if (!detail.ok) throw detail.error;
|
||||
expect(detail.value.result).toMatchObject({ address: resolvedAddress });
|
||||
if (expectedTable === null) {
|
||||
expect(detail.value.result).not.toHaveProperty("metadata_recovery");
|
||||
} else {
|
||||
expect(detail.value.result).toMatchObject({
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: expectedTable,
|
||||
},
|
||||
});
|
||||
}
|
||||
expect(callerAddresses).toEqual([address]);
|
||||
expect(readSnapshot).toHaveBeenCalledTimes(
|
||||
resolvedAddress === null ? 0 : 1,
|
||||
);
|
||||
expect(operations).toEqual(["inspect_native_data_type"]);
|
||||
await client.close();
|
||||
},
|
||||
);
|
||||
|
||||
it("retains engine-provided NativeAOT metadata without deriving another snapshot", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const metadata = {
|
||||
format: "dotnet-nativeaot",
|
||||
source: "analysis-database",
|
||||
status: "partial",
|
||||
header_address: "0x140002000",
|
||||
format_major: 9,
|
||||
format_minor: 1,
|
||||
method_table_address: "0x140002280",
|
||||
name_origin: "generated",
|
||||
original_name: null,
|
||||
base_size_bytes: 32,
|
||||
related_type: null,
|
||||
interfaces: [],
|
||||
virtual_slots: [],
|
||||
derived_memory: null,
|
||||
diagnostics: ["Engine-provided fixture metadata"],
|
||||
limitations: ["Fixture has no original managed names"],
|
||||
};
|
||||
const readSnapshot = vi.fn(async () => {
|
||||
throw new Error("Unexpected source snapshot read");
|
||||
});
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: async () =>
|
||||
ok({
|
||||
...sessionInfo(),
|
||||
target: { ...sessionInfo().target, image_base: "0x140000000" },
|
||||
}),
|
||||
callTool: async () =>
|
||||
ok(
|
||||
jsonValueSchema.parse({
|
||||
...unavailableNativeAotDatabaseType("0x140002280"),
|
||||
status: "available",
|
||||
reason: null,
|
||||
id: "/NativeAOT/Example_MT",
|
||||
name: "Example_MT",
|
||||
kind: "struct",
|
||||
size_bytes: 32,
|
||||
alignment_bytes: 8,
|
||||
packing_enabled: false,
|
||||
metadata_recovery: metadata,
|
||||
}),
|
||||
),
|
||||
readTargetSnapshot: readSnapshot,
|
||||
close: async () => ok(null),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
@@ -1444,17 +1646,9 @@ it("derives PE metadata for a direct type-address query without a load-image cal
|
||||
});
|
||||
expect(detail).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
},
|
||||
},
|
||||
},
|
||||
value: { result: { status: "available", metadata_recovery: metadata } },
|
||||
});
|
||||
expect(operations).toEqual(["inspect_native_data_type"]);
|
||||
expect(readSnapshot).not.toHaveBeenCalled();
|
||||
await client.close();
|
||||
});
|
||||
|
||||
@@ -1492,27 +1686,7 @@ it("shares one session snapshot across callers while isolating cancellation and
|
||||
jsonValueSchema.parse(
|
||||
operation === "inspect_native_load_image"
|
||||
? observation
|
||||
: {
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
},
|
||||
: unavailableNativeAotDatabaseType("0x140002280"),
|
||||
),
|
||||
),
|
||||
);
|
||||
@@ -1947,6 +2121,9 @@ const peTarget = (
|
||||
sha256: "a".repeat(64),
|
||||
kind: "executable",
|
||||
format: "pe",
|
||||
peMachine: { x86: 0x014c, x86_64: 0x8664, arm: 0x01c4, arm64: 0xaa64 }[
|
||||
architecture
|
||||
],
|
||||
architecture,
|
||||
availableArchitectures: [architecture],
|
||||
executableRole: "application",
|
||||
|
||||
@@ -52,6 +52,7 @@ import {
|
||||
CAPABILITIES,
|
||||
windowsP0Capabilities,
|
||||
GHIDRA_PROVIDER_IDENTITY,
|
||||
WINDOWS_P0_NATIVE_PE_ARCHITECTURES,
|
||||
} from "./GhidraProviderCapabilities.js";
|
||||
import {
|
||||
createGhidraProviderClient,
|
||||
@@ -360,11 +361,12 @@ const inspectWindowsP0TargetSupport = (
|
||||
"Windows Ghidra P0 accepts native PE and explicitly selected raw 6502 targets only.",
|
||||
diagnostics,
|
||||
};
|
||||
if (target.architecture !== "x86" && target.architecture !== "x86_64")
|
||||
if (!WINDOWS_P0_NATIVE_PE_ARCHITECTURES.includes(target.architecture))
|
||||
return {
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
reason: "Windows Ghidra P0 accepts x86 and x86-64 PE targets only.",
|
||||
reason:
|
||||
"Windows Ghidra P0 accepts x86, x86-64, ARM and ARM64 PE targets only.",
|
||||
diagnostics,
|
||||
};
|
||||
if (
|
||||
@@ -386,6 +388,17 @@ const inspectWindowsP0TargetSupport = (
|
||||
"Windows Ghidra P0 accepts native PE applications and DLLs; managed or unclassified PE targets are unsupported.",
|
||||
diagnostics,
|
||||
};
|
||||
if (
|
||||
(target.architecture === "arm" && target.peMachine !== 0x01c4) ||
|
||||
(target.architecture === "arm64" && target.peMachine !== 0xaa64)
|
||||
)
|
||||
return {
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
reason:
|
||||
"Windows Ghidra P0 ARM PE targets require observed ARMNT (0x01c4) or ARM64 (0xaa64) COFF machines; legacy ARM/Thumb and unknown machines are unsupported.",
|
||||
diagnostics: { ...diagnostics, pe_machine: target.peMachine ?? null },
|
||||
};
|
||||
return {
|
||||
status: "supported",
|
||||
code: null,
|
||||
|
||||
@@ -10,6 +10,10 @@ import { GHIDRA_FUNCTION_OPERATIONS } from "./GhidraFunctionValues.js";
|
||||
import { GHIDRA_INVENTORY_OPERATIONS } from "./GhidraInventoryValues.js";
|
||||
import { GHIDRA_MUTATING_OPERATIONS } from "./GhidraSessionValues.js";
|
||||
|
||||
/** Target architectures admitted by the Windows P0 native PE boundary. */
|
||||
export const WINDOWS_P0_NATIVE_PE_ARCHITECTURES: readonly string[] =
|
||||
Object.freeze(["x86", "x86_64", "arm", "arm64"]);
|
||||
|
||||
/** Public identity committed by every Ghidra-backed observation. */
|
||||
export const GHIDRA_PROVIDER_IDENTITY: ProviderIdentity = Object.freeze({
|
||||
id: "ghidra",
|
||||
@@ -32,7 +36,7 @@ export const healthLimitations = Object.freeze([
|
||||
|
||||
/** Additional limitations applied to the experimental Windows x64 P0 boundary. */
|
||||
export const windowsP0Limitations = Object.freeze([
|
||||
"Windows Ghidra P0 accepts native x86 and x86-64 PE applications and DLLs, and explicitly selected raw 6502 images; managed, hostile, sensitive, and mutable-path targets are unsupported.",
|
||||
"Windows Ghidra P0 accepts native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs, and explicitly selected raw 6502 images; managed, hostile, sensitive, and mutable-path targets are unsupported.",
|
||||
"The Windows bridge uses authenticated IPv4 loopback because Node path-based IPC does not expose Java AF_UNIX sockets; the endpoint file contains no bearer token.",
|
||||
"Windows sessions require the matching packaged Windows x64 native addon, local NTFS targets and runtimes, and Windows 10 or later. Native handles enforce path admission, private DACLs, and Job Object ownership automatically.",
|
||||
]);
|
||||
|
||||
@@ -858,8 +858,16 @@ export const createGhidraProviderClient = (input: {
|
||||
if (!result.ok) return result;
|
||||
normalized = result.value;
|
||||
}
|
||||
if (operation === "inspect_native_data_type") {
|
||||
const address = input.value.address;
|
||||
if (
|
||||
operation === "inspect_native_data_type" &&
|
||||
normalized !== null &&
|
||||
typeof normalized === "object" &&
|
||||
!Array.isArray(normalized) &&
|
||||
normalized.metadata_recovery === undefined
|
||||
) {
|
||||
// The bridge owns address-space resolution. Enrich its parsed identity;
|
||||
// request parameters retain the caller's spelling.
|
||||
const address = normalized.address;
|
||||
if (typeof address === "string") {
|
||||
if (target.format === "pe" && nativeAotOwner !== undefined) {
|
||||
const started = await client.start(
|
||||
@@ -883,23 +891,12 @@ export const createGhidraProviderClient = (input: {
|
||||
);
|
||||
if (!derived.ok) return derived;
|
||||
}
|
||||
let metadata:
|
||||
| ReturnType<NativeAotPeResult["readTypeDetail"]>
|
||||
| undefined;
|
||||
metadata = nativeAotOwner?.readTypeDetail(address);
|
||||
if (
|
||||
metadata !== undefined &&
|
||||
normalized !== null &&
|
||||
typeof normalized === "object" &&
|
||||
!Array.isArray(normalized)
|
||||
) {
|
||||
const nativeResult = normalized as Record<string, JsonValue>;
|
||||
if (nativeResult.metadata_recovery === undefined)
|
||||
normalized = jsonValueSchema.parse({
|
||||
...nativeResult,
|
||||
metadata_recovery: metadata,
|
||||
});
|
||||
}
|
||||
const metadata = nativeAotOwner?.readTypeDetail(address);
|
||||
if (metadata !== undefined)
|
||||
normalized = jsonValueSchema.parse({
|
||||
...normalized,
|
||||
metadata_recovery: metadata,
|
||||
});
|
||||
}
|
||||
}
|
||||
if (operation === "inspect_native_load_image") {
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
/** Normalize a u64 selector without allocating a BigInt for unbounded digits. */
|
||||
export const normalizeKunaAddress = (value: unknown): string | undefined => {
|
||||
if (typeof value !== "string") return undefined;
|
||||
const match = /^0x0*([0-9a-f]{1,16})$/iu.exec(value);
|
||||
return match?.[1] === undefined
|
||||
? undefined
|
||||
: `0x${BigInt(`0x${match[1]}`).toString(16)}`;
|
||||
};
|
||||
|
||||
/** Sort canonical hexadecimal addresses numerically without BigInt allocations. */
|
||||
export const compareKunaAddresses = (left: string, right: string): number =>
|
||||
left.length - right.length || (left < right ? -1 : left > right ? 1 : 0);
|
||||
+14
-4
@@ -206,8 +206,13 @@ const commandFailure = (
|
||||
snapshot === null
|
||||
? undefined
|
||||
: {
|
||||
// Output can be evidence-sized, not diagnostic-sized; keep only its tail.
|
||||
stdout: snapshot.stdout.text.slice(-4096),
|
||||
// Output can be evidence-sized, not diagnostic-sized; keep only its
|
||||
// tail. Output that overflowed its buffer is a cut-off document
|
||||
// whose tail explains nothing, so none of it is kept.
|
||||
stdout:
|
||||
cause.reason === "output-limit"
|
||||
? ""
|
||||
: snapshot.stdout.text.slice(-4096),
|
||||
stderr: snapshot.stderr.text,
|
||||
truncated:
|
||||
snapshot.diagnosticTruncated === true ||
|
||||
@@ -245,15 +250,20 @@ const commandFailure = (
|
||||
details,
|
||||
),
|
||||
};
|
||||
// The same target and executable always produce the same output, so a
|
||||
// fixed buffer it overflows is a memory bound, not a busy admission queue.
|
||||
if (cause.reason === "output-limit")
|
||||
return {
|
||||
kind: "failed",
|
||||
error: new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"capacity",
|
||||
"memory",
|
||||
`kuna ${subcommand} wrote more than ${String(request.outputBytes)} bytes of output.`,
|
||||
{ output_limit_bytes: request.outputBytes },
|
||||
details,
|
||||
{
|
||||
...details,
|
||||
remediationAction: `REA buffers at most ${String(request.outputBytes)} bytes of kuna ${subcommand} output, so repeating this request for this target fails the same way. Use another provider for it.`,
|
||||
},
|
||||
),
|
||||
};
|
||||
const exitCode = capturedOutput?.exit_code;
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
/** Parse numeric u64 addresses from their exact JSON tokens, before rounding. */
|
||||
export const parseKunaJson = (
|
||||
text: string,
|
||||
limit: number,
|
||||
addressKeys: ReadonlySet<string>,
|
||||
): unknown => {
|
||||
if (Buffer.byteLength(text) > limit)
|
||||
throw new Error("Kuna JSON exceeds its buffered capacity");
|
||||
return JSON.parse(
|
||||
text,
|
||||
(
|
||||
key: string,
|
||||
value: unknown,
|
||||
context?: { readonly source?: string },
|
||||
): unknown => {
|
||||
if (!addressKeys.has(key) || value === null) return value;
|
||||
const source = context?.source;
|
||||
if (
|
||||
typeof value !== "number" ||
|
||||
source === undefined ||
|
||||
source.length > 20 ||
|
||||
!/^(?:0|[1-9][0-9]*)$/u.test(source)
|
||||
)
|
||||
throw new Error(`invalid numeric address in ${key}`);
|
||||
const exact = BigInt(source);
|
||||
if (exact > 0xffffffffffffffffn)
|
||||
throw new Error(`address in ${key} exceeds u64`);
|
||||
return `0x${exact.toString(16)}`;
|
||||
},
|
||||
);
|
||||
};
|
||||
@@ -1,2 +1,6 @@
|
||||
/** Shared buffered-text capacity for a subprocess answer or one cached source facet. */
|
||||
export const KUNA_FUNCTION_TEXT_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
/** One buffered, retained string scan; independent of decompilation. */
|
||||
export const KUNA_STRING_OUTPUT_BYTES = 16 * 1024 * 1024;
|
||||
export const KUNA_STRING_MIN_LENGTH = 5;
|
||||
@@ -150,6 +150,8 @@ interface InflightBuild {
|
||||
readonly promise: Promise<Result<KunaProject, AnalysisError>>;
|
||||
readonly controller: AbortController;
|
||||
readonly listeners: Set<(status: KunaExportStatus) => void>;
|
||||
/** The latest status, replayed to requests that join after it was reported. */
|
||||
status: KunaExportStatus | undefined;
|
||||
waiters: number;
|
||||
}
|
||||
|
||||
@@ -334,7 +336,9 @@ export class KunaProjectCache {
|
||||
return this.#await(running, request);
|
||||
const controller = new AbortController();
|
||||
const listeners = new Set<(status: KunaExportStatus) => void>();
|
||||
// #produce reports its first status only after an await, once `created` exists.
|
||||
const promise = this.#produce(key, request, controller.signal, (status) => {
|
||||
created.status = status;
|
||||
for (const listener of listeners) listener(status);
|
||||
});
|
||||
const created: InflightBuild = {
|
||||
@@ -342,6 +346,7 @@ export class KunaProjectCache {
|
||||
promise,
|
||||
controller,
|
||||
listeners,
|
||||
status: undefined,
|
||||
waiters: 0,
|
||||
};
|
||||
this.#inflight.set(key, created);
|
||||
@@ -465,6 +470,9 @@ export class KunaProjectCache {
|
||||
leave();
|
||||
resolve(result);
|
||||
});
|
||||
// Replay only once cancellation is registered, so a listener may abort.
|
||||
if (listener !== undefined && entry.status !== undefined)
|
||||
listener(entry.status);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -67,8 +67,11 @@ export interface KunaExport {
|
||||
/** The step an export is in; Kuna itself reports no progress until it finishes. */
|
||||
export interface KunaExportStatus {
|
||||
readonly phase: "inventory" | "decompiling" | "sealing";
|
||||
/** Callable entries found by the inventory, once known. */
|
||||
readonly inventoryFunctions: number | null;
|
||||
/**
|
||||
* Every callable entry the export's inventory found, once known. Listings
|
||||
* use it while the export still decompiles, so no second inventory runs.
|
||||
*/
|
||||
readonly inventory: readonly KunaInventoryFunction[] | null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -207,7 +210,7 @@ export const exportKunaProject = async (
|
||||
};
|
||||
};
|
||||
|
||||
request.onStatus({ phase: "inventory", inventoryFunctions: null });
|
||||
request.onStatus({ phase: "inventory", inventory: null });
|
||||
const listed = await run(
|
||||
["functions", input, "--json"],
|
||||
KUNA_INVENTORY_OUTPUT_BYTES,
|
||||
@@ -223,10 +226,7 @@ export const exportKunaProject = async (
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
|
||||
request.onStatus({
|
||||
phase: "decompiling",
|
||||
inventoryFunctions: inventory.value.length,
|
||||
});
|
||||
request.onStatus({ phase: "decompiling", inventory: inventory.value });
|
||||
const exported = await run(
|
||||
[
|
||||
"decompile-project",
|
||||
@@ -240,10 +240,7 @@ export const exportKunaProject = async (
|
||||
);
|
||||
if (!exported.ok) return exported.outcome;
|
||||
|
||||
request.onStatus({
|
||||
phase: "sealing",
|
||||
inventoryFunctions: inventory.value.length,
|
||||
});
|
||||
request.onStatus({ phase: "sealing", inventory: inventory.value });
|
||||
const summary = parseKunaExportSummary(exported.snapshot.stdout.text);
|
||||
if (summary === undefined)
|
||||
return unusable(
|
||||
@@ -272,22 +269,44 @@ export const exportKunaProject = async (
|
||||
files[role] = { path: name, ...(await hashStableFile(path, signal)) };
|
||||
}
|
||||
const complete = files as KunaExportFiles;
|
||||
const parsed = await readKunaExport(
|
||||
const functions = await readKunaExportSource(
|
||||
request.output,
|
||||
complete,
|
||||
complete.c,
|
||||
summary,
|
||||
signal,
|
||||
);
|
||||
if (!parsed.ok) return unusable(parsed.error);
|
||||
if (!functions.ok) return unusable(functions.error);
|
||||
const identity = await verifyKunaExecutable(installation, signal);
|
||||
if (!identity.ok) return { kind: "done", result: identity };
|
||||
// `functions.json` holds exactly the text this inventory was parsed from, so
|
||||
// publishing reuses it rather than holding a second parsed copy.
|
||||
return {
|
||||
kind: "done",
|
||||
result: ok({ files: complete, summary, ...parsed.value }),
|
||||
result: ok({
|
||||
files: complete,
|
||||
summary,
|
||||
functions: functions.value,
|
||||
inventory: inventory.value,
|
||||
}),
|
||||
};
|
||||
};
|
||||
|
||||
/** Parse an export's function blocks and inventory from its own files. */
|
||||
/** Split an export's `.c` into its function blocks without buffering the file. */
|
||||
const readKunaExportSource = (
|
||||
directory: string,
|
||||
file: KunaExportFile,
|
||||
summary: KunaExportSummary,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<readonly KunaProjectFunction[], string>> =>
|
||||
scanKunaProjectSource(
|
||||
createReadStream(
|
||||
join(directory, file.path),
|
||||
signal === undefined ? {} : { signal },
|
||||
),
|
||||
summary,
|
||||
);
|
||||
|
||||
/** Parse a published export's function blocks and inventory from its own files. */
|
||||
export const readKunaExport = async (
|
||||
directory: string,
|
||||
files: KunaExportFiles,
|
||||
@@ -302,12 +321,11 @@ export const readKunaExport = async (
|
||||
string
|
||||
>
|
||||
> => {
|
||||
const functions = await scanKunaProjectSource(
|
||||
createReadStream(
|
||||
join(directory, files.c.path),
|
||||
signal === undefined ? {} : { signal },
|
||||
),
|
||||
const functions = await readKunaExportSource(
|
||||
directory,
|
||||
files.c,
|
||||
summary,
|
||||
signal,
|
||||
);
|
||||
if (!functions.ok) return functions;
|
||||
const inventory = parseKunaInventory(
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { KUNA_STRING_MIN_LENGTH } from "./KunaLimits.js";
|
||||
import { accessSync, constants, statSync } from "node:fs";
|
||||
|
||||
import type {
|
||||
@@ -57,7 +58,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
options: {
|
||||
readonly platform?: NodeJS.Platform;
|
||||
readonly launcher?: KunaProcessLauncher;
|
||||
/** Test seam for concurrent single-function runs; production derives it from the host. */
|
||||
/** Test seam for foreground native runs; production derives it from the host. */
|
||||
readonly singleFunctionRuns?: number;
|
||||
} = {},
|
||||
) {
|
||||
@@ -186,6 +187,20 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
verified_release: value.verified,
|
||||
jobs: value.configuration.jobs,
|
||||
export: "decompile-project",
|
||||
reference_graph: {
|
||||
command: "decompile-graph",
|
||||
schema_version: 4,
|
||||
bodies: false,
|
||||
edge_kind: "call",
|
||||
},
|
||||
xrefs: { command: "xrefs", direction: "to", exact_address: true },
|
||||
strings: {
|
||||
command: "strings",
|
||||
encoding: "all",
|
||||
min_length: KUNA_STRING_MIN_LENGTH,
|
||||
termination: "any",
|
||||
xrefs: false,
|
||||
},
|
||||
cache_policy: "live",
|
||||
},
|
||||
),
|
||||
@@ -214,7 +229,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
};
|
||||
return new KunaSessionClient(target, profile, {
|
||||
cache: this.#projectCache(configuration.value),
|
||||
runSlots: this.#singleFunctionSlots(configuration.value),
|
||||
runSlots: this.#foregroundSlots(configuration.value),
|
||||
environment: this.#environment,
|
||||
...(this.#launcher === undefined ? {} : { launcher: this.#launcher }),
|
||||
});
|
||||
@@ -230,7 +245,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
|
||||
}
|
||||
|
||||
/** One admission pool per provider, so concurrent sessions share the host's cores. */
|
||||
#singleFunctionSlots(configuration: KunaConfiguration): KunaRunSlots {
|
||||
#foregroundSlots(configuration: KunaConfiguration): KunaRunSlots {
|
||||
this.#runSlots ??=
|
||||
this.#singleFunctionRuns === undefined
|
||||
? KunaRunSlots.forHost(configuration.jobs)
|
||||
|
||||
@@ -19,14 +19,27 @@ export const KUNA_OPERATIONS = Object.freeze([
|
||||
"list_documents",
|
||||
"list_procedures",
|
||||
"search_procedures",
|
||||
"list_strings",
|
||||
"search_strings",
|
||||
"procedure_address",
|
||||
"address_name",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_assembly",
|
||||
"analyze_function",
|
||||
"procedure_callers",
|
||||
"procedure_callees",
|
||||
"xrefs",
|
||||
] as const satisfies readonly Exclude<AnalysisOperation, "health">[]);
|
||||
export type KunaOperation = (typeof KUNA_OPERATIONS)[number];
|
||||
|
||||
export const KUNA_RELATIONSHIP_LIMITATION =
|
||||
"Kuna reports call-kind edges between its discovered function nodes. Jump/data edges are excluded; unresolved indirect calls and targets outside that inventory remain unknown.";
|
||||
export const KUNA_STRING_LIMITATION =
|
||||
"Kuna scans its mapped sections or segments for printable runs of at least five characters, including unterminated fragments. Encodings are ASCII, UTF-8 and Latin-range UTF-16LE; other text can remain undiscovered. String scanning skips references; an empty search is not proof of absence.";
|
||||
|
||||
export const KUNA_XREF_LIMITATION =
|
||||
"Kuna's static reference walk unifies forwarding aliases and omits references internal to that alias class. REA keeps only exact-address incoming edges; an empty result does not prove no reference exists.";
|
||||
|
||||
/** Whether an operation is one Kuna answers. */
|
||||
export const isKunaOperation = (
|
||||
operation: AnalysisOperation,
|
||||
@@ -53,6 +66,14 @@ export const kunaLimitations = (
|
||||
switch (operation) {
|
||||
case "list_documents":
|
||||
return [];
|
||||
case "list_strings":
|
||||
return [KUNA_PROVENANCE, KUNA_STRING_LIMITATION];
|
||||
case "search_strings":
|
||||
return [
|
||||
KUNA_PROVENANCE,
|
||||
KUNA_STRING_LIMITATION,
|
||||
REGEX_MATCHING_LIMITATION,
|
||||
];
|
||||
case "list_procedures":
|
||||
case "procedure_address":
|
||||
case "address_name":
|
||||
@@ -63,6 +84,11 @@ export const kunaLimitations = (
|
||||
KUNA_ENTRY_LIMITATION,
|
||||
REGEX_MATCHING_LIMITATION,
|
||||
];
|
||||
case "procedure_callers":
|
||||
case "procedure_callees":
|
||||
return [KUNA_PROVENANCE, KUNA_RELATIONSHIP_LIMITATION];
|
||||
case "xrefs":
|
||||
return [KUNA_PROVENANCE, KUNA_XREF_LIMITATION];
|
||||
case "procedure_pseudo_code":
|
||||
case "procedure_assembly":
|
||||
case "analyze_function":
|
||||
@@ -98,9 +124,8 @@ export const kunaCapabilities = (
|
||||
reason: availability.reason,
|
||||
availabilityCode: availability.code,
|
||||
} as const)),
|
||||
// A cold answer can come from the single-function path and differ from the
|
||||
// export's later answer, so results are re-read rather than replayed. The
|
||||
// project cache already serves warm reads in milliseconds.
|
||||
// Text can change when the export finishes. References own bounded session
|
||||
// reuse; generic snapshot replay must not bypass that lifecycle.
|
||||
cachePolicy: "live",
|
||||
effects: {
|
||||
mutatesArtifact: false,
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
import type {
|
||||
AnalysisError,
|
||||
AnalysisErrorOptions,
|
||||
} from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisArtifactChangedError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisOutputError,
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisTimeoutError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
|
||||
/** Keep one producer failure immutable while each waiter receives its context. */
|
||||
export const kunaQueryFailureForCaller = (
|
||||
source: AnalysisError,
|
||||
operation: string,
|
||||
): AnalysisError => {
|
||||
if (
|
||||
!("operation" in source) ||
|
||||
typeof source.operation !== "string" ||
|
||||
source.operation === operation
|
||||
)
|
||||
return source;
|
||||
const diagnostics = {
|
||||
...source.diagnostics,
|
||||
producer_operation: source.operation,
|
||||
};
|
||||
const options: AnalysisErrorOptions & {
|
||||
userMessage?: string;
|
||||
remediationAction?: string;
|
||||
} = {
|
||||
cause: source,
|
||||
diagnostics,
|
||||
...(source.capturedOutput === undefined
|
||||
? {}
|
||||
: { capturedOutput: source.capturedOutput }),
|
||||
...(source.cleanup === undefined ? {} : { cleanup: source.cleanup }),
|
||||
...(source.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: source.partialObservation }),
|
||||
...(source.userMessage === undefined
|
||||
? {}
|
||||
: { userMessage: source.userMessage }),
|
||||
...("remediationAction" in source &&
|
||||
typeof source.remediationAction === "string"
|
||||
? { remediationAction: source.remediationAction }
|
||||
: {}),
|
||||
};
|
||||
// The command owner keeps the original cleanup failure and its supervisor.
|
||||
// This contextual diagnostic retains that source as its cause and every resource.
|
||||
if (source instanceof ProviderCleanupError)
|
||||
return new ProviderCleanupError(
|
||||
source.providerId,
|
||||
source.cleanupResources,
|
||||
diagnostics,
|
||||
{
|
||||
operation,
|
||||
cause: source,
|
||||
...(source.partialObservation === undefined
|
||||
? {}
|
||||
: { partialObservation: source.partialObservation }),
|
||||
},
|
||||
);
|
||||
if (source instanceof AnalysisTimeoutError)
|
||||
return new AnalysisTimeoutError(operation, source.timeoutMs, options);
|
||||
if (source instanceof AnalysisResourceConstraintError)
|
||||
return new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
source.resource,
|
||||
source.reason,
|
||||
source.reportedLimits,
|
||||
options,
|
||||
);
|
||||
if (source instanceof AnalysisOutputError)
|
||||
return new AnalysisOutputError(operation, source.reason, {
|
||||
...options,
|
||||
cleanupOnly: source.cleanupOnly,
|
||||
});
|
||||
if (source instanceof ProviderAdapterError)
|
||||
return new ProviderAdapterError(source.providerId, operation, options);
|
||||
if (source instanceof AnalysisCancelledError)
|
||||
return new AnalysisCancelledError(operation, options);
|
||||
if (source instanceof AnalysisArtifactChangedError)
|
||||
return new AnalysisArtifactChangedError(
|
||||
operation,
|
||||
source.path,
|
||||
source.reason,
|
||||
options,
|
||||
);
|
||||
// Preserve unfamiliar error variants rather than flattening their semantics.
|
||||
return source;
|
||||
};
|
||||
@@ -0,0 +1,138 @@
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js";
|
||||
import { matchRegexValues } from "../process/RegexMatcher.js";
|
||||
import { KunaRunSlots } from "./KunaRunSlots.js";
|
||||
import type {
|
||||
KunaTargetCommands,
|
||||
KunaTargetCommandRequest,
|
||||
KunaTargetCommandSnapshot,
|
||||
} from "./KunaTargetCommands.js";
|
||||
|
||||
/** One shared producer; individual callers own only their wait. */
|
||||
export interface KunaPendingQuery<T> {
|
||||
readonly promise: Promise<Result<T, AnalysisError>>;
|
||||
readonly controller: AbortController;
|
||||
waiters: number;
|
||||
settled: boolean;
|
||||
}
|
||||
|
||||
type RegexQuery = Omit<Parameters<typeof matchRegexValues>[0], "values"> & {
|
||||
readonly values: () => readonly string[];
|
||||
};
|
||||
|
||||
/** Shared admission, cancellation and cleanup for session metadata queries. */
|
||||
export class KunaQueryWork {
|
||||
readonly #nativeSlots = new KunaRunSlots(1);
|
||||
readonly #regexSlots = new KunaRunSlots(1);
|
||||
|
||||
constructor(
|
||||
private readonly commands: KunaTargetCommands,
|
||||
private readonly lifetime: {
|
||||
readonly signal: AbortSignal;
|
||||
readonly own: <T>(work: Promise<T>) => Promise<T>;
|
||||
readonly runSlots: KunaRunSlots;
|
||||
},
|
||||
) {}
|
||||
|
||||
/** Unproven process cleanup retains both leases in the native command owner. */
|
||||
async run(
|
||||
request: Omit<KunaTargetCommandRequest, "releaseRun">,
|
||||
): Promise<Result<KunaTargetCommandSnapshot, AnalysisError>> {
|
||||
const local = await this.#nativeSlots.acquire(request.signal);
|
||||
if (local === undefined)
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
let release: (() => void) | undefined;
|
||||
try {
|
||||
const foreground = await this.lifetime.runSlots.acquire(request.signal);
|
||||
if (foreground === undefined) {
|
||||
local();
|
||||
return err(new AnalysisCancelledError(request.operation));
|
||||
}
|
||||
release = () => {
|
||||
foreground();
|
||||
local();
|
||||
};
|
||||
return await this.commands.run({ ...request, releaseRun: release });
|
||||
} catch (cause: unknown) {
|
||||
if (release === undefined) local();
|
||||
else release();
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
/** Bound inventory copies and regex workers; close cancels and drains both. */
|
||||
match(input: RegexQuery): ReturnType<typeof matchRegexValues> {
|
||||
const signal = AbortSignal.any([
|
||||
this.lifetime.signal,
|
||||
...(input.signal === undefined ? [] : [input.signal]),
|
||||
]);
|
||||
return this.lifetime.own(this.#match(input, signal));
|
||||
}
|
||||
|
||||
async #match(
|
||||
input: RegexQuery,
|
||||
signal: AbortSignal,
|
||||
): ReturnType<typeof matchRegexValues> {
|
||||
const release = await this.#regexSlots.acquire(signal);
|
||||
if (release === undefined)
|
||||
return err(new AnalysisCancelledError(input.operation));
|
||||
try {
|
||||
return await matchRegexValues({
|
||||
...input,
|
||||
values: input.values(),
|
||||
signal,
|
||||
});
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
start<T>(
|
||||
load: (signal: AbortSignal) => Promise<Result<T, AnalysisError>>,
|
||||
accept: (value: T) => void,
|
||||
): KunaPendingQuery<T> {
|
||||
const controller = new AbortController();
|
||||
const signal = AbortSignal.any([this.lifetime.signal, controller.signal]);
|
||||
const pending: KunaPendingQuery<T> = {
|
||||
promise: this.lifetime.own(load(signal)),
|
||||
controller,
|
||||
settled: false,
|
||||
waiters: 0,
|
||||
};
|
||||
void pending.promise.then(
|
||||
(result) => {
|
||||
pending.settled = true;
|
||||
if (result.ok && !signal.aborted) accept(result.value);
|
||||
},
|
||||
() => {
|
||||
pending.settled = true;
|
||||
},
|
||||
);
|
||||
return pending;
|
||||
}
|
||||
|
||||
async wait<T>(
|
||||
pending: KunaPendingQuery<T>,
|
||||
signal: AbortSignal | undefined,
|
||||
operation: string,
|
||||
): Promise<Result<T, AnalysisError>> {
|
||||
pending.waiters += 1;
|
||||
try {
|
||||
const result = await waitForAbortable(pending.promise, signal);
|
||||
if (result === ABORTED && pending.waiters === 1 && !pending.settled) {
|
||||
pending.controller.abort();
|
||||
const stopped = await pending.promise;
|
||||
if (!stopped.ok && !(stopped.error instanceof AnalysisCancelledError))
|
||||
return stopped;
|
||||
}
|
||||
return result === ABORTED
|
||||
? err(new AnalysisCancelledError(operation))
|
||||
: result;
|
||||
} finally {
|
||||
pending.waiters -= 1;
|
||||
if (pending.waiters === 0 && !pending.settled) pending.controller.abort();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { compareKunaAddresses } from "./KunaAddress.js";
|
||||
import { parseKunaJson } from "./KunaJson.js";
|
||||
|
||||
/** Raw JSON budgets also bound the materialized indexes; no decompiled bodies are admitted. */
|
||||
export const KUNA_GRAPH_OUTPUT_BYTES = 32 * 1024 * 1024;
|
||||
export const KUNA_XREF_OUTPUT_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
const addressKeys = new Set([
|
||||
"address",
|
||||
"callerAddress",
|
||||
"calleeAddress",
|
||||
"forwardsTo",
|
||||
"from_address",
|
||||
"to_address",
|
||||
]);
|
||||
const address = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
|
||||
const count = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const kind = z.enum(["call", "jump", "data", "read", "write"]);
|
||||
|
||||
const graphHeader = z.object({
|
||||
schemaVersion: z.literal(4),
|
||||
binary: z.object({
|
||||
sourcePath: z.string(),
|
||||
functionCount: count,
|
||||
edgeCount: count,
|
||||
}),
|
||||
functions: z.array(z.unknown()),
|
||||
edges: z.array(z.unknown()),
|
||||
});
|
||||
const graphNode = z.object({
|
||||
address,
|
||||
name: z.string().min(1),
|
||||
kind: z.string().min(1),
|
||||
hasIndirectCalls: z.boolean(),
|
||||
forwardsTo: address.nullable(),
|
||||
codeC: z.null(),
|
||||
assembly: z.null(),
|
||||
error: z.null(),
|
||||
});
|
||||
const graphEdge = z.object({
|
||||
callerAddress: address,
|
||||
calleeAddress: address,
|
||||
kind,
|
||||
calleeOrder: count,
|
||||
});
|
||||
|
||||
export type KunaGraphNode = z.infer<typeof graphNode>;
|
||||
export interface KunaCallGraph {
|
||||
readonly sha256: string;
|
||||
readonly bytes: number;
|
||||
readonly edgeCount: number;
|
||||
readonly hasIndirectCalls: boolean;
|
||||
readonly nodes: ReadonlyMap<string, KunaGraphNode>;
|
||||
readonly callers: ReadonlyMap<string, readonly string[]>;
|
||||
readonly callees: ReadonlyMap<string, readonly string[]>;
|
||||
}
|
||||
|
||||
/** Validate once at the producer boundary and retain only facts used by queries. */
|
||||
export const parseKunaCallGraph = (
|
||||
text: string,
|
||||
input: string,
|
||||
): Result<KunaCallGraph, string> => {
|
||||
try {
|
||||
const header = graphHeader.parse(
|
||||
parseKunaJson(text, KUNA_GRAPH_OUTPUT_BYTES, addressKeys),
|
||||
);
|
||||
if (header.binary.sourcePath !== input)
|
||||
throw new Error("graph names a different input");
|
||||
if (
|
||||
header.binary.functionCount !== header.functions.length ||
|
||||
header.binary.edgeCount !== header.edges.length
|
||||
)
|
||||
throw new Error("graph counts disagree with examined rows");
|
||||
const nodes = new Map<string, KunaGraphNode>();
|
||||
let hasIndirectCalls = false;
|
||||
for (const raw of header.functions) {
|
||||
const node = graphNode.parse(raw);
|
||||
if (nodes.has(node.address))
|
||||
throw new Error("graph repeats a function address");
|
||||
nodes.set(node.address, node);
|
||||
hasIndirectCalls ||= node.hasIndirectCalls;
|
||||
}
|
||||
const callers = new Map<string, string[]>();
|
||||
const callees = new Map<string, string[]>();
|
||||
const pairs = new Set<string>();
|
||||
const orders = new Map<string, Set<number>>();
|
||||
for (const raw of header.edges) {
|
||||
const edge = graphEdge.parse(raw);
|
||||
if (!nodes.has(edge.callerAddress) || !nodes.has(edge.calleeAddress))
|
||||
throw new Error("graph edge has an unknown endpoint");
|
||||
const pair = `${edge.callerAddress}:${edge.calleeAddress}`;
|
||||
if (pairs.has(pair)) throw new Error("graph repeats an edge");
|
||||
pairs.add(pair);
|
||||
const order = orders.get(edge.callerAddress) ?? new Set<number>();
|
||||
if (order.has(edge.calleeOrder))
|
||||
throw new Error("graph repeats a callee order");
|
||||
order.add(edge.calleeOrder);
|
||||
orders.set(edge.callerAddress, order);
|
||||
if (edge.kind !== "call") continue;
|
||||
const incoming = callers.get(edge.calleeAddress) ?? [];
|
||||
incoming.push(edge.callerAddress);
|
||||
callers.set(edge.calleeAddress, incoming);
|
||||
const outgoing = callees.get(edge.callerAddress) ?? [];
|
||||
outgoing.push(edge.calleeAddress);
|
||||
callees.set(edge.callerAddress, outgoing);
|
||||
}
|
||||
for (const order of orders.values())
|
||||
for (let index = 0; index < order.size; index += 1)
|
||||
if (!order.has(index))
|
||||
throw new Error("graph callee orders are incomplete");
|
||||
for (const list of [...callers.values(), ...callees.values()])
|
||||
list.sort(compareKunaAddresses);
|
||||
return ok({
|
||||
nodes,
|
||||
callers,
|
||||
callees,
|
||||
edgeCount: header.edges.length,
|
||||
hasIndirectCalls,
|
||||
bytes: Buffer.byteLength(text),
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(cause instanceof Error ? cause.message : String(cause));
|
||||
}
|
||||
};
|
||||
|
||||
const xrefFunction = z.object({
|
||||
name: z.string().min(1),
|
||||
address,
|
||||
address_hex: address,
|
||||
});
|
||||
const xrefRow = z.object({
|
||||
address,
|
||||
address_hex: address,
|
||||
kind,
|
||||
from_address: address,
|
||||
from_address_hex: address,
|
||||
to_address: address,
|
||||
to_address_hex: address,
|
||||
from_function: xrefFunction.nullable(),
|
||||
to_function: xrefFunction.nullable(),
|
||||
instruction: z.string(),
|
||||
});
|
||||
const xrefsDocument = z.object({
|
||||
binary: z.string(),
|
||||
direction: z.literal("to"),
|
||||
count,
|
||||
target: z.object({
|
||||
address,
|
||||
address_hex: address,
|
||||
name: z.string().nullable(),
|
||||
aliases: z.array(xrefFunction),
|
||||
}),
|
||||
xrefs: z.array(z.unknown()),
|
||||
});
|
||||
|
||||
export interface KunaIncomingReferences {
|
||||
readonly addresses: readonly string[];
|
||||
readonly sha256: string;
|
||||
readonly bytes: number;
|
||||
readonly rows: number;
|
||||
readonly excludedAliasRows: number;
|
||||
}
|
||||
|
||||
/** Native --to unifies forwarding aliases; REA xrefs asks for the exact address. */
|
||||
export const parseKunaIncomingReferences = (
|
||||
text: string,
|
||||
input: string,
|
||||
target: string,
|
||||
): Result<KunaIncomingReferences, string> => {
|
||||
try {
|
||||
const document = xrefsDocument.parse(
|
||||
parseKunaJson(text, KUNA_XREF_OUTPUT_BYTES, addressKeys),
|
||||
);
|
||||
if (
|
||||
document.binary !== input ||
|
||||
document.target.address !== target ||
|
||||
document.target.address_hex !== target
|
||||
)
|
||||
throw new Error("xref document names a different input or target");
|
||||
if (document.count !== document.xrefs.length)
|
||||
throw new Error("xref count disagrees with examined rows");
|
||||
const aliases = new Set([target]);
|
||||
for (const alias of document.target.aliases) {
|
||||
if (alias.address !== alias.address_hex)
|
||||
throw new Error("xref alias address disagrees with its hex field");
|
||||
aliases.add(alias.address);
|
||||
}
|
||||
const addresses = new Set<string>();
|
||||
let excludedAliasRows = 0;
|
||||
for (const raw of document.xrefs) {
|
||||
const row = xrefRow.parse(raw);
|
||||
if (
|
||||
row.address !== row.address_hex ||
|
||||
row.address !== row.from_address ||
|
||||
row.from_address !== row.from_address_hex ||
|
||||
row.to_address !== row.to_address_hex
|
||||
)
|
||||
throw new Error(
|
||||
"xref endpoints disagree with numeric or directional fields",
|
||||
);
|
||||
for (const fn of [row.from_function, row.to_function])
|
||||
if (fn !== null && fn.address !== fn.address_hex)
|
||||
throw new Error("xref function address disagrees with its hex field");
|
||||
if (!aliases.has(row.to_address))
|
||||
throw new Error(
|
||||
"xref row targets neither the requested address nor an advertised alias",
|
||||
);
|
||||
if (row.to_address === target) addresses.add(row.from_address);
|
||||
else excludedAliasRows += 1;
|
||||
}
|
||||
return ok({
|
||||
addresses: [...addresses].sort(compareKunaAddresses),
|
||||
rows: document.xrefs.length,
|
||||
excludedAliasRows,
|
||||
bytes: Buffer.byteLength(text),
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(cause instanceof Error ? cause.message : String(cause));
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,325 @@
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import type { FunctionDossier } from "../domain/hopperValues.js";
|
||||
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { KunaInstallation } from "./KunaInstallation.js";
|
||||
import {
|
||||
KUNA_GRAPH_OUTPUT_BYTES,
|
||||
KUNA_XREF_OUTPUT_BYTES,
|
||||
parseKunaCallGraph,
|
||||
parseKunaIncomingReferences,
|
||||
type KunaCallGraph,
|
||||
type KunaGraphNode,
|
||||
type KunaIncomingReferences,
|
||||
} from "./KunaReferenceFiles.js";
|
||||
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
|
||||
import type { KunaPendingQuery, KunaQueryWork } from "./KunaQueryWork.js";
|
||||
|
||||
/** Charge raw JSON bytes conservatively; retained projections are smaller. */
|
||||
const XREF_CACHE_BYTES = 16 * 1024 * 1024;
|
||||
/** Also bound metadata for empty results and queued native queries. */
|
||||
const XREF_CACHE_ENTRIES = 128;
|
||||
const PENDING_XREF_QUERIES = 8;
|
||||
|
||||
export interface KunaFunctionRelationships {
|
||||
readonly callers: readonly FunctionDossier["callers"][number][];
|
||||
readonly callees: readonly FunctionDossier["callees"][number][];
|
||||
readonly source: JsonValue;
|
||||
readonly callersCoverage: RelationshipCoverage;
|
||||
readonly calleesCoverage: RelationshipCoverage;
|
||||
}
|
||||
|
||||
/** Lazy session facts, shared producers and bounded exact-address xref reuse. */
|
||||
export class KunaReferenceQueries {
|
||||
#graph: KunaCallGraph | undefined;
|
||||
/** Deterministic buffer failures belong to the same session as its graph. */
|
||||
#graphFailure: AnalysisError | undefined;
|
||||
#pendingGraph: KunaPendingQuery<KunaCallGraph> | undefined;
|
||||
readonly #xrefs = new Map<string, KunaIncomingReferences>();
|
||||
readonly #pendingXrefs = new Map<
|
||||
string,
|
||||
KunaPendingQuery<KunaIncomingReferences>
|
||||
>();
|
||||
#xrefBytes = 0;
|
||||
constructor(private readonly work: KunaQueryWork) {}
|
||||
|
||||
async relationships(
|
||||
installation: KunaInstallation,
|
||||
address: string,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<KunaFunctionRelationships, AnalysisError>> {
|
||||
const graph = await this.#loadGraph(installation, operation, signal);
|
||||
return graph.ok
|
||||
? this.#relationships(graph.value, address, operation)
|
||||
: graph;
|
||||
}
|
||||
|
||||
/** Only a validated graph can establish that an inventory-missing entry exists. */
|
||||
async functionAt(
|
||||
installation: KunaInstallation,
|
||||
address: string,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<
|
||||
Result<
|
||||
{ readonly address: string; readonly source: JsonValue } | undefined,
|
||||
AnalysisError
|
||||
>
|
||||
> {
|
||||
const graph = await this.#loadGraph(installation, operation, signal);
|
||||
if (!graph.ok) return graph;
|
||||
const node = graph.value.nodes.get(address);
|
||||
return ok(
|
||||
node === undefined
|
||||
? undefined
|
||||
: {
|
||||
address: node.address,
|
||||
source: this.#source(graph.value, node),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
async #loadGraph(
|
||||
installation: KunaInstallation,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<KunaCallGraph, AnalysisError>> {
|
||||
if (this.#graphFailure !== undefined) return err(this.#graphFailure);
|
||||
if (this.#graph === undefined) {
|
||||
if (this.#pendingGraph?.controller.signal.aborted === true) {
|
||||
const retired = await this.work.wait(
|
||||
this.#pendingGraph,
|
||||
signal,
|
||||
operation,
|
||||
);
|
||||
if (!retired.ok && !(retired.error instanceof AnalysisCancelledError))
|
||||
return retired;
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
return this.#loadGraph(installation, operation, signal);
|
||||
}
|
||||
if (this.#pendingGraph === undefined) {
|
||||
const pending = this.work.start<KunaCallGraph>(
|
||||
async (producerSignal) => {
|
||||
const run = await this.work.run({
|
||||
installation,
|
||||
operation: "kuna:call-graph",
|
||||
arguments: (input) => [
|
||||
"decompile-graph",
|
||||
input,
|
||||
"--functions",
|
||||
"",
|
||||
],
|
||||
timeoutMs: installation.configuration.projectTimeoutMs,
|
||||
outputBytes: KUNA_GRAPH_OUTPUT_BYTES,
|
||||
signal: producerSignal,
|
||||
});
|
||||
if (!run.ok) {
|
||||
if (
|
||||
run.error instanceof AnalysisResourceConstraintError &&
|
||||
run.error.resource === "memory"
|
||||
)
|
||||
this.#graphFailure = run.error;
|
||||
return run;
|
||||
}
|
||||
return this.#parsed(
|
||||
parseKunaCallGraph(run.value.stdout.text, run.value.input),
|
||||
"call graph",
|
||||
);
|
||||
},
|
||||
(value) => {
|
||||
this.#graph = value;
|
||||
},
|
||||
);
|
||||
this.#pendingGraph = pending;
|
||||
const release = () => {
|
||||
if (this.#pendingGraph === pending) this.#pendingGraph = undefined;
|
||||
};
|
||||
void pending.promise.then(release, release);
|
||||
}
|
||||
return this.work.wait(this.#pendingGraph, signal, operation);
|
||||
}
|
||||
return ok(this.#graph);
|
||||
}
|
||||
|
||||
#relationships(
|
||||
graph: KunaCallGraph,
|
||||
address: string,
|
||||
operation: string,
|
||||
): Result<KunaFunctionRelationships, AnalysisError> {
|
||||
const node = graph.nodes.get(address);
|
||||
if (node === undefined)
|
||||
return err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
KUNA_PROVIDER_ID,
|
||||
operation,
|
||||
`Kuna's graph has no function node at ${address}; its relationships are unknown.`,
|
||||
),
|
||||
);
|
||||
const identities = (
|
||||
addresses: readonly string[],
|
||||
): readonly FunctionDossier["callers"][number][] =>
|
||||
addresses.map((entryAddress) => {
|
||||
const entry = graph.nodes.get(entryAddress)!; // The parser proved both endpoints.
|
||||
return {
|
||||
address: entry.address,
|
||||
name: entry.name,
|
||||
classification: null,
|
||||
body: {
|
||||
available: false,
|
||||
reason: "Kuna's graph reports no observed body ranges.",
|
||||
},
|
||||
};
|
||||
});
|
||||
return ok({
|
||||
callers: identities(graph.callers.get(address) ?? []),
|
||||
callees: identities(graph.callees.get(address) ?? []),
|
||||
callersCoverage: graph.hasIndirectCalls
|
||||
? {
|
||||
status: "partial",
|
||||
limited_directions: ["callers"],
|
||||
limitations: [
|
||||
"Kuna reports unresolved indirect calls in the image; additional callers cannot be excluded.",
|
||||
],
|
||||
}
|
||||
: { status: "complete" },
|
||||
calleesCoverage: node.hasIndirectCalls
|
||||
? {
|
||||
status: "partial",
|
||||
limited_directions: ["callees"],
|
||||
limitations: [
|
||||
"Kuna reports indirect calls in this function without every target; additional callees remain unknown.",
|
||||
],
|
||||
}
|
||||
: { status: "complete" },
|
||||
source: this.#source(graph, node),
|
||||
});
|
||||
}
|
||||
|
||||
#source(graph: KunaCallGraph, node: KunaGraphNode): JsonValue {
|
||||
return {
|
||||
kind: "kuna-call-graph",
|
||||
schema_version: 4,
|
||||
sha256: graph.sha256,
|
||||
bytes: graph.bytes,
|
||||
functions: graph.nodes.size,
|
||||
edges: graph.edgeCount,
|
||||
has_indirect_calls: graph.hasIndirectCalls,
|
||||
function: {
|
||||
address: node.address,
|
||||
has_indirect_calls: node.hasIndirectCalls,
|
||||
forwards_to: node.forwardsTo,
|
||||
forwarding_target_in_graph:
|
||||
node.forwardsTo === null ? null : graph.nodes.has(node.forwardsTo),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async incoming(
|
||||
installation: KunaInstallation,
|
||||
address: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<KunaIncomingReferences, AnalysisError>> {
|
||||
const cached = this.#xrefs.get(address);
|
||||
if (cached !== undefined) {
|
||||
this.#xrefs.delete(address);
|
||||
this.#xrefs.set(address, cached);
|
||||
return ok(cached);
|
||||
}
|
||||
let pending = this.#pendingXrefs.get(address);
|
||||
if (pending?.controller.signal.aborted === true) {
|
||||
const retired = await this.work.wait(pending, signal, "xrefs");
|
||||
if (!retired.ok && !(retired.error instanceof AnalysisCancelledError))
|
||||
return retired;
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("xrefs"));
|
||||
return this.incoming(installation, address, signal);
|
||||
}
|
||||
if (pending === undefined) {
|
||||
if (this.#pendingXrefs.size >= PENDING_XREF_QUERIES)
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
"xrefs",
|
||||
"capacity",
|
||||
"The session's native reference-query queue is full; await an existing query before starting another.",
|
||||
{ limit_queries: PENDING_XREF_QUERIES },
|
||||
),
|
||||
);
|
||||
pending = this.work.start<KunaIncomingReferences>(
|
||||
async (producerSignal) => {
|
||||
const run = await this.work.run({
|
||||
installation,
|
||||
operation: "xrefs",
|
||||
arguments: (input) => ["xrefs", input, "--to", address, "--json"],
|
||||
timeoutMs: installation.configuration.projectTimeoutMs,
|
||||
outputBytes: KUNA_XREF_OUTPUT_BYTES,
|
||||
signal: producerSignal,
|
||||
});
|
||||
if (!run.ok) return run;
|
||||
return this.#parsed(
|
||||
parseKunaIncomingReferences(
|
||||
run.value.stdout.text,
|
||||
run.value.input,
|
||||
address,
|
||||
),
|
||||
"incoming references",
|
||||
);
|
||||
},
|
||||
(value) => this.#cache(address, value),
|
||||
);
|
||||
this.#pendingXrefs.set(address, pending);
|
||||
const current = pending;
|
||||
const release = () => {
|
||||
if (this.#pendingXrefs.get(address) === current)
|
||||
this.#pendingXrefs.delete(address);
|
||||
};
|
||||
void pending.promise.then(release, release);
|
||||
}
|
||||
return this.work.wait(pending, signal, "xrefs");
|
||||
}
|
||||
|
||||
#cache(address: string, value: KunaIncomingReferences): void {
|
||||
this.#xrefs.set(address, value);
|
||||
this.#xrefBytes += value.bytes + 256;
|
||||
while (
|
||||
this.#xrefBytes > XREF_CACHE_BYTES ||
|
||||
this.#xrefs.size > XREF_CACHE_ENTRIES
|
||||
) {
|
||||
const oldest = this.#xrefs.entries().next().value;
|
||||
if (oldest === undefined) break;
|
||||
this.#xrefs.delete(oldest[0]);
|
||||
this.#xrefBytes -= oldest[1].bytes + 256;
|
||||
}
|
||||
}
|
||||
|
||||
/** Release parsed facts after the session has drained all owned producers. */
|
||||
clear(): void {
|
||||
this.#graph = undefined;
|
||||
this.#graphFailure = undefined;
|
||||
this.#pendingGraph = undefined;
|
||||
this.#xrefs.clear();
|
||||
this.#pendingXrefs.clear();
|
||||
this.#xrefBytes = 0;
|
||||
}
|
||||
|
||||
#parsed<T>(
|
||||
parsed: Result<T, string>,
|
||||
label: string,
|
||||
): Result<T, AnalysisError> {
|
||||
return parsed.ok
|
||||
? parsed
|
||||
: err(
|
||||
new ProviderAdapterError(KUNA_PROVIDER_ID, "kuna:references", {
|
||||
userMessage: `Kuna's ${label} cannot be used: ${parsed.error}`,
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -4,12 +4,13 @@ import { availableParallelism } from "node:os";
|
||||
export type KunaRunRelease = () => void;
|
||||
|
||||
/**
|
||||
* Process-wide admission for single-function `kuna decompile` runs.
|
||||
* Provider-wide admission for foreground decompile and reference walks.
|
||||
*
|
||||
* Each run analyzes the whole image again in its own process tree, which held
|
||||
* about 360 MiB resident and close to one core on a 200 KiB Mach-O. These runs only
|
||||
* happen while the project export builds, so they share the cores its
|
||||
* `--jobs` workers leave free, and requests beyond that wait in FIFO order.
|
||||
* A single-function decompile analyzes the whole image again; one measured
|
||||
* run held about 360 MiB resident and close to one core on a 200 KiB Mach-O.
|
||||
* Foreground walks share the cores reserved beyond an export's `--jobs` workers.
|
||||
* Requests beyond that budget wait in FIFO order. A native cleanup owner
|
||||
* keeps its slot while it cannot prove that the process stopped.
|
||||
*/
|
||||
export class KunaRunSlots {
|
||||
readonly #capacity: number;
|
||||
|
||||
+441
-229
@@ -1,5 +1,3 @@
|
||||
import { basename, join } from "node:path";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import {
|
||||
@@ -11,10 +9,11 @@ import {
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisArtifactChangedError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisTimeoutError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
|
||||
import {
|
||||
@@ -23,19 +22,18 @@ import {
|
||||
} from "../domain/analysisSearch.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import type { EvidenceLocation } from "../domain/evidence.js";
|
||||
import { combineFunctionRelationshipCoverage } from "../domain/functionCallCoverage.js";
|
||||
import { functionDossierSchema } from "../domain/hopperValues.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import { matchRegexValues } from "../process/RegexMatcher.js";
|
||||
import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js";
|
||||
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
|
||||
import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js";
|
||||
import {
|
||||
copyAndHash,
|
||||
KUNA_INVENTORY_OUTPUT_BYTES,
|
||||
runKunaCommand,
|
||||
type KunaProcessLauncher,
|
||||
} from "./KunaCommand.js";
|
||||
import {
|
||||
@@ -54,11 +52,17 @@ import {
|
||||
} from "./KunaProjectFiles.js";
|
||||
import {
|
||||
isKunaOperation,
|
||||
KUNA_RELATIONSHIP_LIMITATION,
|
||||
KUNA_SINGLE_FUNCTION_LIMITATION,
|
||||
kunaLimitations,
|
||||
type KunaOperation,
|
||||
} from "./KunaProviderCapabilities.js";
|
||||
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
|
||||
import { KunaTargetCommands } from "./KunaTargetCommands.js";
|
||||
import { KunaReferenceQueries } from "./KunaReferenceQueries.js";
|
||||
import { KunaQueryWork } from "./KunaQueryWork.js";
|
||||
import { KunaStringQueries } from "./KunaStringQueries.js";
|
||||
import { normalizeKunaAddress } from "./KunaAddress.js";
|
||||
import { KUNA_FUNCTION_TEXT_BYTES } from "./KunaLimits.js";
|
||||
import type { KunaRunRelease, KunaRunSlots } from "./KunaRunSlots.js";
|
||||
import {
|
||||
@@ -69,8 +73,6 @@ import {
|
||||
/** How often a request waiting on the project export reports that it still waits. */
|
||||
const WAIT_PROGRESS_INTERVAL_MS = 10_000;
|
||||
|
||||
const HEX_ADDRESS = /^0x[0-9a-f]+$/iu;
|
||||
|
||||
/** Where one answer's text came from, recorded in its raw result. */
|
||||
type AnswerSource =
|
||||
| { readonly kind: "project-export"; readonly project: KunaProject }
|
||||
@@ -103,28 +105,24 @@ interface Answer {
|
||||
readonly files?: readonly KunaProjectFileSpan[];
|
||||
readonly locations: readonly EvidenceLocation[];
|
||||
readonly limitations: readonly string[];
|
||||
readonly referenceSource?: JsonValue;
|
||||
readonly stringSource?: JsonValue;
|
||||
readonly relationshipCoverage?: RelationshipCoverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for work this session owns, stopping only this caller's wait when its
|
||||
* signal aborts; the work itself continues for other callers.
|
||||
*/
|
||||
const waitFor = <T>(
|
||||
const waitFor = async <T>(
|
||||
work: Promise<Result<T, AnalysisError>>,
|
||||
signal: AbortSignal | undefined,
|
||||
operation: string,
|
||||
): Promise<Result<T, AnalysisError>> => {
|
||||
if (signal === undefined) return work;
|
||||
if (signal.aborted)
|
||||
return Promise.resolve(err(new AnalysisCancelledError(operation)));
|
||||
return new Promise((resolve) => {
|
||||
const onAbort = () => resolve(err(new AnalysisCancelledError(operation)));
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
void work.then((result) => {
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
resolve(result);
|
||||
});
|
||||
});
|
||||
const result = await waitForAbortable(work, signal);
|
||||
return result === ABORTED
|
||||
? err(new AnalysisCancelledError(operation))
|
||||
: result;
|
||||
};
|
||||
|
||||
/** A file the adopted export named is gone, e.g. evicted by another REA process. */
|
||||
@@ -148,6 +146,52 @@ const failure = (operation: string, cause: unknown): AnalysisError =>
|
||||
userMessage: `Kuna ${operation} failed: ${message(cause)}`,
|
||||
});
|
||||
|
||||
/**
|
||||
* How a request that needs the export reports the export's failure. A session
|
||||
* never rebuilds a failed export, so repeating the request cannot succeed; the
|
||||
* error says what to change before reopening the target. Kuna settings are read
|
||||
* when REA starts, so changing one also needs a restart. Cancellation and
|
||||
* unproven cleanup keep their own errors, which callers must still observe.
|
||||
*/
|
||||
const exportUnavailable = (
|
||||
operation: string,
|
||||
failure: AnalysisError,
|
||||
): AnalysisError => {
|
||||
if (failure instanceof AnalysisCancelledError || failure.cleanupIncomplete)
|
||||
return failure;
|
||||
const reason = failure.userMessage ?? failure.message;
|
||||
const exported = projectAnalysisError(failure);
|
||||
const restart =
|
||||
"REA reads Kuna settings when it starts, so restart REA, then reopen the target.";
|
||||
const next =
|
||||
failure instanceof AnalysisTimeoutError
|
||||
? `To retry, set REA_KUNA_PROJECT_TIMEOUT_MS above ${String(failure.timeoutMs)} ms or raise REA_KUNA_JOBS in the environment that starts REA. ${restart}`
|
||||
: failure instanceof AnalysisResourceConstraintError &&
|
||||
failure.remediationAction !== undefined
|
||||
? // A fixed buffer (`memory`) is no setting, so a restart cannot help.
|
||||
`${failure.remediationAction.replace(/\.?$/u, ".")}${failure.resource === "memory" ? "" : ` ${restart}`}`
|
||||
: "To retry, resolve the reported failure, then reopen the target.";
|
||||
return new AnalysisCapabilityUnavailableError(
|
||||
KUNA_PROVIDER_ID,
|
||||
operation,
|
||||
reason,
|
||||
{
|
||||
cause: failure,
|
||||
// Callers see only the top-level projection, so the export failure's
|
||||
// own projected code and details (its limits, target path, captured
|
||||
// output and diagnostics) travel whole rather than field by field.
|
||||
diagnostics: {
|
||||
export_error: failure._tag,
|
||||
export_code: exported.code,
|
||||
...(exported.details === undefined
|
||||
? {}
|
||||
: { export_details: exported.details }),
|
||||
},
|
||||
userMessage: `Kuna's project export failed, so ${operation} is unavailable in this session: ${reason.replace(/\.?$/u, ".")} ${next}`,
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
/** Kuna names a procedure by its inventory entry or by an exact entry address. */
|
||||
const procedureSelectorSchema = z.string().min(1);
|
||||
|
||||
@@ -155,7 +199,7 @@ const procedureSelectorSchema = z.string().min(1);
|
||||
* One Kuna analysis session for one target.
|
||||
*
|
||||
* Reads come from the verified project export in REA's cache. Until that
|
||||
* export exists, listings use a quick `kuna functions` inventory and
|
||||
* export exists, listings use the inventory the export takes first and
|
||||
* pseudocode a single-function `kuna decompile`, while the export builds in
|
||||
* the background; assembly waits for it. Kuna always runs on a private copy
|
||||
* whose digest matches the selected target.
|
||||
@@ -165,7 +209,6 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
readonly #profile: AnalysisProfileCommitment | undefined;
|
||||
readonly #cache: KunaProjectCache;
|
||||
readonly #environment: Readonly<NodeJS.ProcessEnv>;
|
||||
readonly #launcher: KunaProcessLauncher | undefined;
|
||||
readonly #runSlots: KunaRunSlots;
|
||||
/** Single-function runs in flight, by entry address. */
|
||||
readonly #singleFunctionRuns = new Map<string, SingleFunctionRun>();
|
||||
@@ -173,32 +216,21 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
readonly #session = new AbortController();
|
||||
/** Requests and shared readers retain their resources until their work settles. */
|
||||
readonly #active = new Set<Promise<unknown>>();
|
||||
/**
|
||||
* Kuna runs that could not be proven stopped, with their retry owners and,
|
||||
* for single-function runs, the run slot their process tree still holds.
|
||||
*/
|
||||
readonly #retained: {
|
||||
readonly error: ProviderCleanupError;
|
||||
owner: ProviderProcessSupervisor | undefined;
|
||||
readonly slot?: KunaRunRelease;
|
||||
}[] = [];
|
||||
/** Private roots whose removal failed during setup or close. */
|
||||
readonly #leakedRoots: PrivateRuntimeRoot[] = [];
|
||||
readonly #commands: KunaTargetCommands;
|
||||
readonly #queryWork: KunaQueryWork;
|
||||
readonly #references: KunaReferenceQueries;
|
||||
readonly #strings: KunaStringQueries;
|
||||
#installation: Promise<Result<KunaInstallation, AnalysisError>> | undefined;
|
||||
#workspace:
|
||||
| Promise<
|
||||
Result<
|
||||
{ readonly root: PrivateRuntimeRoot; readonly input: string },
|
||||
AnalysisError
|
||||
>
|
||||
>
|
||||
| undefined;
|
||||
#inventory:
|
||||
| Promise<Result<readonly KunaInventoryFunction[], AnalysisError>>
|
||||
| undefined;
|
||||
#project: Promise<Result<KunaProject, AnalysisError>> | undefined;
|
||||
/** The cache lookup that decides #project, shared by concurrent first requests. */
|
||||
#starting: Promise<void> | undefined;
|
||||
/** The building export's inventory; undefined when it settles without one. */
|
||||
#exportInventory:
|
||||
| Promise<readonly KunaInventoryFunction[] | undefined>
|
||||
| undefined;
|
||||
#reader: KunaProjectReader | undefined;
|
||||
#inventoryIndex: InventoryIndex | undefined;
|
||||
#status: KunaExportStatus | undefined;
|
||||
@@ -225,7 +257,19 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
this.#cache = dependencies.cache;
|
||||
this.#runSlots = dependencies.runSlots;
|
||||
this.#environment = dependencies.environment;
|
||||
this.#launcher = dependencies.launcher;
|
||||
this.#commands = new KunaTargetCommands(target, {
|
||||
signal: this.#session.signal,
|
||||
...(dependencies.launcher === undefined
|
||||
? {}
|
||||
: { launcher: dependencies.launcher }),
|
||||
});
|
||||
this.#queryWork = new KunaQueryWork(this.#commands, {
|
||||
signal: this.#session.signal,
|
||||
own: (work) => this.#own(work),
|
||||
runSlots: dependencies.runSlots,
|
||||
});
|
||||
this.#references = new KunaReferenceQueries(this.#queryWork);
|
||||
this.#strings = new KunaStringQueries(this.#queryWork);
|
||||
}
|
||||
|
||||
async execute(
|
||||
@@ -271,7 +315,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
KUNA_PROVIDER_ID,
|
||||
operation,
|
||||
"Kuna answers procedure listings, lookups, pseudocode and assembly; this operation needs another provider.",
|
||||
"Kuna answers procedure and string queries, pseudocode, assembly and references; this operation needs another provider.",
|
||||
),
|
||||
);
|
||||
const document = parameters.document;
|
||||
@@ -311,7 +355,11 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
limitations: [],
|
||||
}),
|
||||
);
|
||||
if (operation !== "list_documents")
|
||||
if (
|
||||
operation === "procedure_pseudo_code" ||
|
||||
operation === "procedure_assembly" ||
|
||||
operation === "analyze_function"
|
||||
)
|
||||
await this.#startProject(installation.value);
|
||||
const answer = await this.#answer(
|
||||
operation,
|
||||
@@ -350,6 +398,8 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
// is safe to remove private copies and exposes any retained cleanup owner.
|
||||
// An admitted request may create a shared reader after shutdown begins.
|
||||
while (this.#active.size > 0) await Promise.allSettled(this.#active);
|
||||
this.#references.clear();
|
||||
this.#strings.clear();
|
||||
await this.#project;
|
||||
await this.#inventory;
|
||||
const failures: ProviderCleanupError[] = [];
|
||||
@@ -359,36 +409,8 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
const settled = await this.#cache.settled(this.#key);
|
||||
if (!settled.ok) failures.push(settled.error);
|
||||
}
|
||||
for (const retained of [...this.#retained]) {
|
||||
const stopped = await retained.owner?.stop();
|
||||
if (stopped !== undefined && stopped.status !== "incomplete") {
|
||||
this.#retained.splice(this.#retained.indexOf(retained), 1);
|
||||
retained.slot?.();
|
||||
}
|
||||
}
|
||||
failures.push(...this.#retained.map(({ error }) => error));
|
||||
const workspace = await this.#workspace;
|
||||
// A single-function run that may still be running keeps its copy.
|
||||
if (this.#retained.length === 0) {
|
||||
if (workspace?.ok === true) {
|
||||
this.#leakedRoots.push(workspace.value.root);
|
||||
this.#workspace = undefined;
|
||||
}
|
||||
for (const root of [...this.#leakedRoots])
|
||||
try {
|
||||
await root.close();
|
||||
this.#leakedRoots.splice(this.#leakedRoots.indexOf(root), 1);
|
||||
} catch (cause: unknown) {
|
||||
failures.push(
|
||||
new ProviderCleanupError(
|
||||
KUNA_PROVIDER_ID,
|
||||
[root.path],
|
||||
{ reason: message(cause) },
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
const commands = await this.#commands.close();
|
||||
if (!commands.ok) failures.push(commands.error);
|
||||
if (failures.length > 0)
|
||||
return err(
|
||||
new ProviderCleanupError(
|
||||
@@ -442,23 +464,38 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
{
|
||||
// Kuna's identity is in the analysis profile; the raw result records
|
||||
// only where file-backed text came from.
|
||||
...(source === null
|
||||
...(source === null &&
|
||||
answer.referenceSource === undefined &&
|
||||
answer.stringSource === undefined
|
||||
? {}
|
||||
: {
|
||||
rawResult: {
|
||||
source:
|
||||
source.kind === "single-function"
|
||||
? { kind: "single-function" }
|
||||
: {
|
||||
kind: "project-export",
|
||||
key: source.project.key,
|
||||
directory: source.project.directory,
|
||||
...recordedFiles,
|
||||
},
|
||||
project_export: this.#exportState(),
|
||||
...(source === null
|
||||
? {}
|
||||
: {
|
||||
source:
|
||||
source.kind === "single-function"
|
||||
? { kind: "single-function" }
|
||||
: {
|
||||
kind: "project-export",
|
||||
key: source.project.key,
|
||||
directory: source.project.directory,
|
||||
...recordedFiles,
|
||||
},
|
||||
project_export: this.#exportState(),
|
||||
}),
|
||||
...(answer.referenceSource === undefined
|
||||
? {}
|
||||
: { references: answer.referenceSource }),
|
||||
...(answer.stringSource === undefined
|
||||
? {}
|
||||
: { strings: answer.stringSource }),
|
||||
},
|
||||
}),
|
||||
limitations: [...operationLimitations, ...answer.limitations],
|
||||
...(answer.relationshipCoverage === undefined
|
||||
? {}
|
||||
: { relationshipCoverage: answer.relationshipCoverage }),
|
||||
locations: [
|
||||
{ kind: "artifact-path", path: this.#target.path },
|
||||
...answer.locations,
|
||||
@@ -480,7 +517,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
return {
|
||||
state: "building",
|
||||
phase: this.#status?.phase ?? null,
|
||||
inventory_functions: this.#status?.inventoryFunctions ?? null,
|
||||
inventory_functions: this.#status?.inventory?.length ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -497,39 +534,23 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
locations: [],
|
||||
limitations: [],
|
||||
});
|
||||
if (
|
||||
operation === "xrefs" ||
|
||||
operation === "procedure_callers" ||
|
||||
operation === "procedure_callees"
|
||||
)
|
||||
return this.#referenceAnswer(
|
||||
operation,
|
||||
parameters,
|
||||
installation,
|
||||
options,
|
||||
);
|
||||
if (operation === "list_strings" || operation === "search_strings")
|
||||
return this.#stringAnswer(operation, parameters, installation, options);
|
||||
if (operation === "list_procedures" || operation === "search_procedures")
|
||||
return this.#procedureList(operation, parameters, installation, options);
|
||||
const inventory = await this.#ensureInventory(installation, options);
|
||||
if (!inventory.ok) return inventory;
|
||||
if (operation === "list_procedures" || operation === "search_procedures") {
|
||||
const items = inventory.value.map(({ address, name }) => ({
|
||||
address,
|
||||
value: name,
|
||||
}));
|
||||
let selected = items;
|
||||
if (operation === "search_procedures") {
|
||||
const query = analysisSearchQuerySchema.parse(parameters);
|
||||
if (query.mode === "regex") {
|
||||
const matches = await matchRegexValues({
|
||||
operation,
|
||||
pattern: query.pattern,
|
||||
caseSensitive: query.case_sensitive,
|
||||
values: items.map(({ value }) => value),
|
||||
signal: options.signal,
|
||||
});
|
||||
if (!matches.ok) return matches;
|
||||
const indices = new Set(matches.value);
|
||||
selected = items.filter((_, index) => indices.has(index));
|
||||
} else {
|
||||
const matches = analysisSearchPredicate(query);
|
||||
selected = items.filter(({ value }) => matches(value));
|
||||
}
|
||||
}
|
||||
return ok({
|
||||
result: selected,
|
||||
source: null,
|
||||
locations: [],
|
||||
limitations: [],
|
||||
});
|
||||
}
|
||||
if (operation === "address_name")
|
||||
return this.#addressName(
|
||||
parameters,
|
||||
@@ -537,8 +558,33 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
installation,
|
||||
options,
|
||||
);
|
||||
const entry = this.#resolve(operation, parameters, inventory.value);
|
||||
if (!entry.ok) return entry;
|
||||
const entry = this.#resolve(
|
||||
operation,
|
||||
parameters.procedure,
|
||||
inventory.value,
|
||||
);
|
||||
if (!entry.ok) {
|
||||
const selectedAddress =
|
||||
operation === "procedure_address"
|
||||
? normalizeKunaAddress(parameters.procedure)
|
||||
: undefined;
|
||||
if (selectedAddress === undefined) return entry;
|
||||
const graph = await this.#references.functionAt(
|
||||
installation,
|
||||
selectedAddress,
|
||||
operation,
|
||||
options.signal,
|
||||
);
|
||||
if (!graph.ok) return graph;
|
||||
if (graph.value === undefined) return entry;
|
||||
return ok({
|
||||
result: graph.value.address,
|
||||
source: null,
|
||||
locations: [{ kind: "address", address: selectedAddress }],
|
||||
limitations: [],
|
||||
referenceSource: graph.value.source,
|
||||
});
|
||||
}
|
||||
const locations = [
|
||||
{ kind: "address", address: entry.value.address } as const,
|
||||
];
|
||||
@@ -572,29 +618,202 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
}
|
||||
case "analyze_function":
|
||||
return this.#withProject(operation, installation, options, (reader) =>
|
||||
this.#dossier(operation, entry.value, reader, options.signal),
|
||||
this.#dossier(
|
||||
operation,
|
||||
entry.value,
|
||||
reader,
|
||||
installation,
|
||||
options.signal,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async #stringAnswer(
|
||||
operation: "list_strings" | "search_strings",
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
installation: KunaInstallation,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<Answer, AnalysisError>> {
|
||||
const answer = await this.#own(
|
||||
this.#strings.execute(
|
||||
operation,
|
||||
parameters,
|
||||
installation,
|
||||
AbortSignal.any([
|
||||
this.#session.signal,
|
||||
...(options.signal === undefined ? [] : [options.signal]),
|
||||
]),
|
||||
),
|
||||
);
|
||||
return answer.ok
|
||||
? ok({
|
||||
result: answer.value.result,
|
||||
source: null,
|
||||
stringSource: answer.value.source,
|
||||
locations: [],
|
||||
limitations: [],
|
||||
})
|
||||
: answer;
|
||||
}
|
||||
|
||||
async #procedureList(
|
||||
operation: "list_procedures" | "search_procedures",
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
installation: KunaInstallation,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<Answer, AnalysisError>> {
|
||||
const query =
|
||||
operation === "search_procedures"
|
||||
? analysisSearchQuerySchema.safeParse(parameters)
|
||||
: undefined;
|
||||
if (query !== undefined && !query.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(operation, query.error.issues, parameters),
|
||||
);
|
||||
const inventory = await this.#ensureInventory(installation, options);
|
||||
if (!inventory.ok) return inventory;
|
||||
let selected = inventory.value;
|
||||
if (query?.success === true) {
|
||||
if (query.data.mode === "regex") {
|
||||
const matches = await this.#queryWork.match({
|
||||
operation,
|
||||
pattern: query.data.pattern,
|
||||
caseSensitive: query.data.case_sensitive,
|
||||
values: () => inventory.value.map(({ name }) => name),
|
||||
signal: options.signal,
|
||||
});
|
||||
if (!matches.ok) return matches;
|
||||
selected = matches.value.map((index) => inventory.value[index]!);
|
||||
} else {
|
||||
const matches = analysisSearchPredicate(query.data);
|
||||
selected = inventory.value.filter(({ name }) => matches(name));
|
||||
}
|
||||
}
|
||||
return ok({
|
||||
result: selected.map(({ address, name }) => ({ address, value: name })),
|
||||
source: null,
|
||||
locations: [],
|
||||
limitations: [],
|
||||
});
|
||||
}
|
||||
|
||||
async #referenceAnswer(
|
||||
operation: "xrefs" | "procedure_callers" | "procedure_callees",
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
installation: KunaInstallation,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<Answer, AnalysisError>> {
|
||||
const field = operation === "xrefs" ? "address" : "procedure";
|
||||
const selected = await this.#referenceAddress(
|
||||
operation,
|
||||
parameters[field],
|
||||
field,
|
||||
installation,
|
||||
options,
|
||||
);
|
||||
if (!selected.ok) return selected;
|
||||
const address = selected.value;
|
||||
const locations = [{ kind: "address" as const, address }];
|
||||
if (operation === "xrefs") {
|
||||
const references = await this.#references.incoming(
|
||||
installation,
|
||||
address,
|
||||
options.signal,
|
||||
);
|
||||
if (!references.ok) return references;
|
||||
return ok({
|
||||
result: [...references.value.addresses],
|
||||
source: null,
|
||||
locations,
|
||||
limitations: [],
|
||||
referenceSource: {
|
||||
kind: "kuna-xrefs",
|
||||
direction: "to",
|
||||
target: address,
|
||||
sha256: references.value.sha256,
|
||||
bytes: references.value.bytes,
|
||||
examined_rows: references.value.rows,
|
||||
excluded_alias_rows: references.value.excludedAliasRows,
|
||||
},
|
||||
});
|
||||
}
|
||||
const relationships = await this.#references.relationships(
|
||||
installation,
|
||||
address,
|
||||
operation,
|
||||
options.signal,
|
||||
);
|
||||
if (!relationships.ok) return relationships;
|
||||
const entries =
|
||||
operation === "procedure_callers"
|
||||
? relationships.value.callers
|
||||
: relationships.value.callees;
|
||||
return ok({
|
||||
result: entries.map(({ address }) => address),
|
||||
source: null,
|
||||
locations,
|
||||
referenceSource: relationships.value.source,
|
||||
limitations: [],
|
||||
relationshipCoverage:
|
||||
operation === "procedure_callers"
|
||||
? relationships.value.callersCoverage
|
||||
: relationships.value.calleesCoverage,
|
||||
});
|
||||
}
|
||||
|
||||
/** Exact addresses are validated by the producing graph, which also owns
|
||||
* synthetic import nodes absent from the quick function inventory. */
|
||||
async #referenceAddress(
|
||||
operation: string,
|
||||
raw: unknown,
|
||||
field: "address" | "procedure",
|
||||
installation: KunaInstallation,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<string, AnalysisError>> {
|
||||
if (typeof raw !== "string" || raw.length === 0)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
{ userMessage: `Kuna requires ${field}; it has no cursor.` },
|
||||
[{ path: [field], reason: "missing_argument" }],
|
||||
),
|
||||
);
|
||||
const address = normalizeKunaAddress(raw);
|
||||
if (address !== undefined) return ok(address);
|
||||
if (/^0x[0-9a-f]+$/iu.test(raw))
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
{ userMessage: "The address exceeds Kuna's u64 address space." },
|
||||
[{ path: [field], reason: "invalid_value" }],
|
||||
),
|
||||
);
|
||||
const inventory = await this.#ensureInventory(installation, options);
|
||||
if (!inventory.ok) return inventory;
|
||||
const entry = this.#resolve(operation, raw, inventory.value, field);
|
||||
return entry.ok ? ok(entry.value.address) : entry;
|
||||
}
|
||||
|
||||
/** Resolve an exact inventory name, alias or entry address. */
|
||||
#resolve(
|
||||
operation: string,
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
raw: unknown,
|
||||
inventory: readonly KunaInventoryFunction[],
|
||||
field: "procedure" | "address" = "procedure",
|
||||
): Result<KunaInventoryFunction, AnalysisError> {
|
||||
const parsed = procedureSelectorSchema.safeParse(parameters.procedure);
|
||||
const parsed = procedureSelectorSchema.safeParse(raw);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
{ userMessage: "procedure is required." },
|
||||
[{ path: ["procedure"], reason: "missing_argument" }],
|
||||
{ userMessage: `${field} is required.` },
|
||||
[{ path: [field], reason: "missing_argument" }],
|
||||
),
|
||||
);
|
||||
const selector = parsed.data;
|
||||
if (HEX_ADDRESS.test(selector)) {
|
||||
const address = `0x${BigInt(selector).toString(16)}`;
|
||||
const address = normalizeKunaAddress(selector);
|
||||
if (address !== undefined) {
|
||||
const entry = this.#index(inventory).byAddress.get(address);
|
||||
return entry === undefined
|
||||
? err(
|
||||
@@ -623,7 +842,8 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<Answer, AnalysisError>> {
|
||||
const raw = parameters.address;
|
||||
if (typeof raw !== "string" || !HEX_ADDRESS.test(raw))
|
||||
const address = normalizeKunaAddress(raw);
|
||||
if (address === undefined)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
"address_name",
|
||||
@@ -634,7 +854,6 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
[{ path: ["address"], reason: "invalid_format" }],
|
||||
),
|
||||
);
|
||||
const address = `0x${BigInt(raw).toString(16)}`;
|
||||
const locations = [{ kind: "address", address } as const];
|
||||
const entry = this.#index(inventory).byAddress.get(address);
|
||||
if (entry !== undefined)
|
||||
@@ -644,6 +863,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
locations,
|
||||
limitations: [],
|
||||
});
|
||||
await this.#startProject(installation);
|
||||
return this.#withProject(
|
||||
"address_name",
|
||||
installation,
|
||||
@@ -787,36 +1007,22 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
installation: KunaInstallation,
|
||||
signal: AbortSignal,
|
||||
): Promise<SingleFunctionOutcome> {
|
||||
const workspace = await this.#ensureWorkspace();
|
||||
if (!workspace.ok) return workspace;
|
||||
const prepared = await this.#commands.prepare();
|
||||
if (!prepared.ok) return prepared;
|
||||
const admitted = await this.#admitSingleFunction(signal);
|
||||
if (admitted === EXPORT_READY) return admitted;
|
||||
if (admitted === undefined)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
let slot: KunaRunRelease | undefined = admitted;
|
||||
try {
|
||||
const run = await runKunaCommand({
|
||||
installation,
|
||||
arguments: ["decompile", workspace.value.input, address, "--addr"],
|
||||
workspace: workspace.value.root.path,
|
||||
operation,
|
||||
timeoutMs: installation.configuration.functionTimeoutMs,
|
||||
reportedTimeoutMs: installation.configuration.functionTimeoutMs,
|
||||
outputBytes: KUNA_FUNCTION_TEXT_BYTES,
|
||||
signal,
|
||||
launcher: this.#launcher,
|
||||
});
|
||||
if (run.kind === "retained") {
|
||||
// The tree may still run, so it keeps its slot until close proves it stopped.
|
||||
this.#retained.push({ error: run.error, owner: run.owner, slot });
|
||||
slot = undefined;
|
||||
return err(run.error);
|
||||
}
|
||||
if (run.kind === "failed") return err(run.error);
|
||||
return ok(run.snapshot.stdout.text.replace(/\n+$/u, ""));
|
||||
} finally {
|
||||
slot?.();
|
||||
}
|
||||
const run = await this.#commands.run({
|
||||
installation,
|
||||
arguments: (input) => ["decompile", input, address, "--addr"],
|
||||
operation,
|
||||
timeoutMs: installation.configuration.functionTimeoutMs,
|
||||
outputBytes: KUNA_FUNCTION_TEXT_BYTES,
|
||||
signal,
|
||||
releaseRun: admitted,
|
||||
});
|
||||
return run.ok ? ok(run.value.stdout.text.replace(/\n+$/u, "")) : run;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -879,6 +1085,7 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
operation: string,
|
||||
entry: KunaInventoryFunction,
|
||||
reader: KunaProjectReader,
|
||||
installation: KunaInstallation,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<Answer, AnalysisError>> {
|
||||
const assembly = await reader.assembly(entry.address, operation, signal);
|
||||
@@ -891,6 +1098,22 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
: null;
|
||||
const unavailable = (reason: string) =>
|
||||
({ available: false, reason }) as const;
|
||||
const relationships = await this.#references.relationships(
|
||||
installation,
|
||||
entry.address,
|
||||
operation,
|
||||
signal,
|
||||
);
|
||||
if (
|
||||
!relationships.ok &&
|
||||
(relationships.error.cleanupIncomplete || signal?.aborted === true)
|
||||
)
|
||||
return relationships;
|
||||
const callAvailability = relationships.ok
|
||||
? { available: true as const }
|
||||
: unavailable(
|
||||
relationships.error.userMessage ?? relationships.error.message,
|
||||
);
|
||||
const dossier = functionDossierSchema.parse({
|
||||
procedure: {
|
||||
address: entry.address,
|
||||
@@ -908,8 +1131,8 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
? assembly.value.result.split("\n")
|
||||
: [],
|
||||
comments: [],
|
||||
callers: [],
|
||||
callees: [],
|
||||
callers: relationships.ok ? relationships.value.callers : [],
|
||||
callees: relationships.ok ? relationships.value.callees : [],
|
||||
incoming_references: [],
|
||||
outgoing_references: [],
|
||||
referenced_strings: [],
|
||||
@@ -927,12 +1150,8 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
: { available: true },
|
||||
assembly: { available: true },
|
||||
comments: unavailable("Kuna exports no analyst comments."),
|
||||
callers: unavailable(
|
||||
"This provider does not report Kuna cross-references.",
|
||||
),
|
||||
callees: unavailable(
|
||||
"This provider does not report Kuna cross-references.",
|
||||
),
|
||||
callers: callAvailability,
|
||||
callees: callAvailability,
|
||||
references: unavailable(
|
||||
"This provider does not report typed references.",
|
||||
),
|
||||
@@ -944,6 +1163,10 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
limitations: [
|
||||
...kunaLimitations("analyze_function"),
|
||||
...pseudocode.value.limitations,
|
||||
KUNA_RELATIONSHIP_LIMITATION,
|
||||
...(relationships.ok
|
||||
? []
|
||||
: [relationships.error.userMessage ?? relationships.error.message]),
|
||||
],
|
||||
});
|
||||
return ok({
|
||||
@@ -954,7 +1177,31 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
...(assembly.value.files ?? []),
|
||||
],
|
||||
locations: [{ kind: "address", address: entry.address }],
|
||||
limitations: pseudocode.value.limitations,
|
||||
limitations: [
|
||||
...pseudocode.value.limitations,
|
||||
KUNA_RELATIONSHIP_LIMITATION,
|
||||
...(relationships.ok
|
||||
? []
|
||||
: [relationships.error.userMessage ?? relationships.error.message]),
|
||||
],
|
||||
referenceSource: relationships.ok
|
||||
? relationships.value.source
|
||||
: {
|
||||
available: false,
|
||||
reason:
|
||||
relationships.error.userMessage ?? relationships.error.message,
|
||||
},
|
||||
relationshipCoverage: relationships.ok
|
||||
? combineFunctionRelationshipCoverage(
|
||||
relationships.value.callersCoverage,
|
||||
relationships.value.calleesCoverage,
|
||||
)
|
||||
: {
|
||||
status: "unknown",
|
||||
reason:
|
||||
relationships.error.userMessage ?? relationships.error.message,
|
||||
limited_directions: ["callers", "callees"],
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1033,6 +1280,12 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
this.#project = Promise.resolve(ok(cached));
|
||||
return;
|
||||
}
|
||||
let listed!: (
|
||||
inventory: readonly KunaInventoryFunction[] | undefined,
|
||||
) => void;
|
||||
this.#exportInventory = new Promise((resolve) => {
|
||||
listed = resolve;
|
||||
});
|
||||
this.#project = this.#cache
|
||||
.build({
|
||||
installation,
|
||||
@@ -1041,9 +1294,11 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
signal: this.#session.signal,
|
||||
onStatus: (status) => {
|
||||
this.#status = status;
|
||||
if (status.inventory !== null) listed(status.inventory);
|
||||
},
|
||||
})
|
||||
.then((result) => {
|
||||
listed(result.ok ? result.value.inventory : undefined);
|
||||
if (result.ok && !this.#closing) this.#adopt(result.value);
|
||||
else if (!result.ok && !this.#closing)
|
||||
this.#exportFailure =
|
||||
@@ -1135,53 +1390,57 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
}, WAIT_PROGRESS_INTERVAL_MS);
|
||||
try {
|
||||
const built = await waitFor(project, options.signal, operation);
|
||||
if (!built.ok) return built;
|
||||
if (!built.ok) return err(exportUnavailable(operation, built.error));
|
||||
return ok(this.#readerFor(built.value));
|
||||
} finally {
|
||||
clearInterval(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/** The export's inventory when ready, otherwise one quick `kuna functions` run. */
|
||||
/** The published or building export's inventory, otherwise one `kuna functions` run. */
|
||||
async #ensureInventory(
|
||||
installation: KunaInstallation,
|
||||
options: ExecutionOptions,
|
||||
): Promise<Result<readonly KunaInventoryFunction[], AnalysisError>> {
|
||||
if (this.#reader !== undefined) return ok(this.#reader.project.inventory);
|
||||
const exported = this.#status?.inventory;
|
||||
if (exported !== undefined && exported !== null) return ok(exported);
|
||||
if (this.#inventory === undefined) {
|
||||
// The session owns the shared run; each caller only stops waiting.
|
||||
const run = this.#listInventory(installation);
|
||||
this.#inventory = run;
|
||||
void run.then((listed) => {
|
||||
// A failed quick listing is retried by the next request.
|
||||
// A failed listing is retried by the next request.
|
||||
if (!listed.ok && this.#inventory === run) this.#inventory = undefined;
|
||||
});
|
||||
}
|
||||
return waitFor(this.#inventory, options.signal, "list_procedures");
|
||||
}
|
||||
|
||||
/**
|
||||
* The export lists every callable entry before it decompiles, so listings
|
||||
* wait for that inventory. Only an export that ends without one, or none at
|
||||
* all, leads to a separate `kuna functions` run.
|
||||
*/
|
||||
async #listInventory(
|
||||
installation: KunaInstallation,
|
||||
): Promise<Result<readonly KunaInventoryFunction[], AnalysisError>> {
|
||||
const workspace = await this.#ensureWorkspace();
|
||||
if (!workspace.ok) return workspace;
|
||||
const run = await runKunaCommand({
|
||||
// A concurrent text request may still be choosing between the cache and a
|
||||
// build; its outcome decides whether an inventory already exists.
|
||||
await this.#starting?.catch(() => undefined);
|
||||
if (this.#reader !== undefined) return ok(this.#reader.project.inventory);
|
||||
const exported = await this.#exportInventory;
|
||||
if (exported !== undefined) return ok(exported);
|
||||
const run = await this.#commands.run({
|
||||
installation,
|
||||
arguments: ["functions", workspace.value.input, "--json"],
|
||||
workspace: workspace.value.root.path,
|
||||
arguments: (input) => ["functions", input, "--json"],
|
||||
operation: "list_procedures",
|
||||
timeoutMs: installation.configuration.functionTimeoutMs,
|
||||
reportedTimeoutMs: installation.configuration.functionTimeoutMs,
|
||||
outputBytes: KUNA_INVENTORY_OUTPUT_BYTES,
|
||||
signal: this.#session.signal,
|
||||
launcher: this.#launcher,
|
||||
});
|
||||
if (run.kind === "retained") {
|
||||
this.#retained.push({ error: run.error, owner: run.owner });
|
||||
return err(run.error);
|
||||
}
|
||||
if (run.kind === "failed") return err(run.error);
|
||||
const inventory = parseKunaInventory(run.snapshot.stdout.text);
|
||||
if (!run.ok) return run;
|
||||
const inventory = parseKunaInventory(run.value.stdout.text);
|
||||
return inventory.ok
|
||||
? ok(inventory.value)
|
||||
: err(
|
||||
@@ -1190,51 +1449,4 @@ export class KunaSessionClient implements AnalysisClient {
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
/** A private, digest-verified copy of the target for single-function runs. */
|
||||
async #ensureWorkspace(): Promise<
|
||||
Result<
|
||||
{ readonly root: PrivateRuntimeRoot; readonly input: string },
|
||||
AnalysisError
|
||||
>
|
||||
> {
|
||||
this.#workspace ??= (async () => {
|
||||
const root = await PrivateRuntimeRoot.create({ prefix: "rea-kuna-" });
|
||||
try {
|
||||
const input = join(root.path, basename(this.#target.path));
|
||||
const digest = await copyAndHash(
|
||||
this.#target.path,
|
||||
input,
|
||||
this.#session.signal,
|
||||
);
|
||||
if (digest !== this.#target.sha256) {
|
||||
await this.#release(root);
|
||||
return err(
|
||||
new AnalysisArtifactChangedError(
|
||||
"kuna:workspace",
|
||||
this.#target.path,
|
||||
"the target no longer matches the digest selected for analysis",
|
||||
),
|
||||
);
|
||||
}
|
||||
await prepareProcessOwnershipInspection(this.#session.signal);
|
||||
return ok({ root, input });
|
||||
} catch (cause: unknown) {
|
||||
await this.#release(root);
|
||||
throw cause;
|
||||
}
|
||||
})().catch((cause: unknown) => err(failure("kuna:workspace", cause)));
|
||||
const workspace = await this.#workspace;
|
||||
if (!workspace.ok) this.#workspace = undefined;
|
||||
return workspace;
|
||||
}
|
||||
|
||||
/** Remove a private root, keeping it for close to retry when removal fails. */
|
||||
async #release(root: PrivateRuntimeRoot): Promise<void> {
|
||||
try {
|
||||
await root.close();
|
||||
} catch {
|
||||
this.#leakedRoots.push(root);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import type { analysisStringSchema } from "../domain/hopperValues.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { compareKunaAddresses } from "./KunaAddress.js";
|
||||
import { parseKunaJson } from "./KunaJson.js";
|
||||
|
||||
import {
|
||||
KUNA_STRING_OUTPUT_BYTES,
|
||||
KUNA_STRING_MIN_LENGTH,
|
||||
} from "./KunaLimits.js";
|
||||
|
||||
const count = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const address = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
|
||||
const addressKeys = new Set(["address"]);
|
||||
const header = z.object({
|
||||
binary: z.string(),
|
||||
encoding: z.literal("all"),
|
||||
min_length: z.literal(KUNA_STRING_MIN_LENGTH),
|
||||
filter: z.null(),
|
||||
section: z.null(),
|
||||
termination: z.literal("any"),
|
||||
scanned: z.enum(["sections", "segments"]),
|
||||
xrefs: z.literal(false),
|
||||
count,
|
||||
strings: z.array(z.unknown()),
|
||||
});
|
||||
const row = z.object({
|
||||
address,
|
||||
address_hex: address,
|
||||
text: z.string(),
|
||||
length: count,
|
||||
byte_length: count,
|
||||
nul_terminated: z.boolean(),
|
||||
encoding: z.enum(["ascii", "utf8", "utf16"]),
|
||||
section: z.string().nullable(),
|
||||
xrefs_count: z.literal(0),
|
||||
functions: z.tuple([]),
|
||||
});
|
||||
|
||||
/** JSON text omits terminators; native lengths describe the encoded source. */
|
||||
const validateStringFacts = (entry: z.infer<typeof row>): void => {
|
||||
let characters = 0;
|
||||
for (const character of entry.text) {
|
||||
const point = character.codePointAt(0)!;
|
||||
if (
|
||||
point === 0 ||
|
||||
(point >= 0xd800 && point <= 0xdfff) ||
|
||||
(entry.encoding === "ascii" && point > 0x7f) ||
|
||||
(entry.encoding === "utf16" && point > 0xff)
|
||||
)
|
||||
throw new Error("string text contradicts its reported encoding");
|
||||
characters += 1;
|
||||
}
|
||||
const textBytes =
|
||||
entry.encoding === "utf16"
|
||||
? entry.text.length * 2
|
||||
: entry.encoding === "ascii"
|
||||
? entry.text.length
|
||||
: Buffer.byteLength(entry.text);
|
||||
const terminatorBytes = entry.nul_terminated
|
||||
? entry.encoding === "utf16"
|
||||
? 2
|
||||
: 1
|
||||
: 0;
|
||||
if (
|
||||
characters < KUNA_STRING_MIN_LENGTH ||
|
||||
entry.length !== characters ||
|
||||
entry.byte_length !== textBytes + terminatorBytes
|
||||
)
|
||||
throw new Error(
|
||||
"string length or termination contradicts its encoded text",
|
||||
);
|
||||
};
|
||||
|
||||
export type KunaStringItem = z.infer<typeof analysisStringSchema>;
|
||||
|
||||
export interface KunaStringInventory {
|
||||
readonly items: readonly KunaStringItem[];
|
||||
readonly sha256: string;
|
||||
readonly bytes: number;
|
||||
readonly scanned: "sections" | "segments";
|
||||
}
|
||||
|
||||
/** Validate the producer's complete scan before publishing any selected row. */
|
||||
export const parseKunaStringInventory = (
|
||||
text: string,
|
||||
input: string,
|
||||
): Result<KunaStringInventory, string> => {
|
||||
try {
|
||||
const document = header.parse(
|
||||
parseKunaJson(text, KUNA_STRING_OUTPUT_BYTES, addressKeys),
|
||||
);
|
||||
if (document.binary !== input)
|
||||
throw new Error("string inventory names a different input");
|
||||
if (document.count !== document.strings.length)
|
||||
throw new Error("string inventory count disagrees with examined rows");
|
||||
const seen = new Set<string>();
|
||||
const items = document.strings.map((value): KunaStringItem => {
|
||||
const entry = row.parse(value);
|
||||
validateStringFacts(entry);
|
||||
if (entry.address !== entry.address_hex)
|
||||
throw new Error("string address disagrees with its hex field");
|
||||
const identity = `${entry.address}:${entry.encoding}`;
|
||||
if (seen.has(identity))
|
||||
throw new Error("string inventory repeats an identity");
|
||||
seen.add(identity);
|
||||
return {
|
||||
address: entry.address,
|
||||
value: entry.text,
|
||||
string: {
|
||||
encoding: entry.encoding,
|
||||
encoding_status: "observed",
|
||||
termination: entry.nul_terminated
|
||||
? "present_or_not_required"
|
||||
: "missing",
|
||||
byte_length: entry.byte_length,
|
||||
},
|
||||
};
|
||||
});
|
||||
items.sort(
|
||||
(left, right) =>
|
||||
compareKunaAddresses(left.address, right.address) ||
|
||||
(left.string?.encoding ?? "").localeCompare(
|
||||
right.string?.encoding ?? "",
|
||||
"en",
|
||||
),
|
||||
);
|
||||
return ok({
|
||||
items,
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
bytes: Buffer.byteLength(text),
|
||||
scanned: document.scanned,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(cause instanceof Error ? cause.message : String(cause));
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,195 @@
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
analysisSearchPredicate,
|
||||
analysisSearchQuerySchema,
|
||||
} from "../domain/analysisSearch.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { normalizeKunaAddress } from "./KunaAddress.js";
|
||||
import type { KunaInstallation } from "./KunaInstallation.js";
|
||||
import type { KunaPendingQuery, KunaQueryWork } from "./KunaQueryWork.js";
|
||||
import { kunaQueryFailureForCaller } from "./KunaQueryErrors.js";
|
||||
import {
|
||||
KUNA_STRING_MIN_LENGTH,
|
||||
KUNA_STRING_OUTPUT_BYTES,
|
||||
} from "./KunaLimits.js";
|
||||
import {
|
||||
parseKunaStringInventory,
|
||||
type KunaStringInventory,
|
||||
type KunaStringItem,
|
||||
} from "./KunaStringInventory.js";
|
||||
|
||||
/** One validated scan shared by literal, regex and exact-address queries. */
|
||||
export class KunaStringQueries {
|
||||
#inventory: KunaStringInventory | undefined;
|
||||
#failure: AnalysisError | undefined;
|
||||
#pending: KunaPendingQuery<KunaStringInventory> | undefined;
|
||||
|
||||
constructor(private readonly work: KunaQueryWork) {}
|
||||
|
||||
async execute(
|
||||
operation: "list_strings" | "search_strings",
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
installation: KunaInstallation,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<
|
||||
Result<
|
||||
{ readonly result: JsonValue; readonly source: JsonValue },
|
||||
AnalysisError
|
||||
>
|
||||
> {
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
const query =
|
||||
operation === "search_strings"
|
||||
? analysisSearchQuerySchema.safeParse(parameters)
|
||||
: undefined;
|
||||
if (query !== undefined && !query.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(operation, query.error.issues, parameters),
|
||||
);
|
||||
const address =
|
||||
operation === "list_strings" && parameters.address !== undefined
|
||||
? normalizeKunaAddress(parameters.address)
|
||||
: undefined;
|
||||
if (
|
||||
operation === "list_strings" &&
|
||||
parameters.address !== undefined &&
|
||||
address === undefined
|
||||
)
|
||||
return err(
|
||||
new AnalysisInputError(operation, {
|
||||
userMessage: "Use an explicit hexadecimal u64 string address.",
|
||||
}),
|
||||
);
|
||||
const inventory = await this.#load(installation, operation, signal);
|
||||
if (!inventory.ok)
|
||||
return err(kunaQueryFailureForCaller(inventory.error, operation));
|
||||
if (this.#cancelled(signal))
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
const available = inventory.value.items;
|
||||
let items: KunaStringItem[] | undefined;
|
||||
if (address !== undefined)
|
||||
items = available.filter((item) => item.address === address);
|
||||
if (query?.success === true) {
|
||||
if (query.data.mode === "regex") {
|
||||
const matched = await this.work.match({
|
||||
operation,
|
||||
pattern: query.data.pattern,
|
||||
caseSensitive: query.data.case_sensitive,
|
||||
values: () => available.map((item) => item.value),
|
||||
signal,
|
||||
});
|
||||
if (!matched.ok) return matched;
|
||||
items = matched.value.map((index) => available[index]!);
|
||||
} else {
|
||||
const predicate = analysisSearchPredicate(query.data);
|
||||
items = available.filter((item) => predicate(item.value));
|
||||
}
|
||||
}
|
||||
const selected = items ?? [...available];
|
||||
return ok({
|
||||
result: selected,
|
||||
source: {
|
||||
kind: "kuna-string-inventory",
|
||||
sha256: inventory.value.sha256,
|
||||
bytes: inventory.value.bytes,
|
||||
scanned: inventory.value.scanned,
|
||||
encoding: "all",
|
||||
min_length: KUNA_STRING_MIN_LENGTH,
|
||||
termination: "any",
|
||||
xrefs: false,
|
||||
inventory_count: inventory.value.items.length,
|
||||
selected_count: selected.length,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async #load(
|
||||
installation: KunaInstallation,
|
||||
operation: string,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<KunaStringInventory, AnalysisError>> {
|
||||
if (this.#inventory !== undefined) return ok(this.#inventory);
|
||||
if (this.#failure !== undefined) return err(this.#failure);
|
||||
if (this.#pending?.controller.signal.aborted === true) {
|
||||
const stopped = await this.work.wait(this.#pending, signal, operation);
|
||||
if (!stopped.ok && !(stopped.error instanceof AnalysisCancelledError))
|
||||
return stopped;
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
return this.#load(installation, operation, signal);
|
||||
}
|
||||
if (this.#pending === undefined) {
|
||||
const pending = this.work.start<KunaStringInventory>(
|
||||
async (producerSignal) => {
|
||||
const run = await this.work.run({
|
||||
installation,
|
||||
operation: "kuna:strings",
|
||||
arguments: (input) => [
|
||||
"strings",
|
||||
input,
|
||||
"--json",
|
||||
"--encoding",
|
||||
"all",
|
||||
"--min-length",
|
||||
String(KUNA_STRING_MIN_LENGTH),
|
||||
"--termination",
|
||||
"any",
|
||||
"--no-xrefs",
|
||||
],
|
||||
timeoutMs: installation.configuration.projectTimeoutMs,
|
||||
outputBytes: KUNA_STRING_OUTPUT_BYTES,
|
||||
signal: producerSignal,
|
||||
});
|
||||
if (!run.ok) {
|
||||
if (
|
||||
run.error instanceof AnalysisResourceConstraintError &&
|
||||
run.error.resource === "memory"
|
||||
)
|
||||
this.#failure = run.error;
|
||||
return run;
|
||||
}
|
||||
const parsed = parseKunaStringInventory(
|
||||
run.value.stdout.text,
|
||||
run.value.input,
|
||||
);
|
||||
return parsed.ok
|
||||
? parsed
|
||||
: err(
|
||||
new AnalysisOutputError("kuna:strings", parsed.error, {
|
||||
userMessage: `Kuna's string inventory cannot be used: ${parsed.error}`,
|
||||
}),
|
||||
);
|
||||
},
|
||||
(value) => {
|
||||
this.#inventory = value;
|
||||
},
|
||||
);
|
||||
this.#pending = pending;
|
||||
const release = () => {
|
||||
if (this.#pending === pending) this.#pending = undefined;
|
||||
};
|
||||
void pending.promise.then(release, release);
|
||||
}
|
||||
return this.work.wait(this.#pending, signal, operation);
|
||||
}
|
||||
|
||||
#cancelled(signal: AbortSignal | undefined): boolean {
|
||||
return signal?.aborted === true;
|
||||
}
|
||||
|
||||
/** Called only after session-owned work has drained. */
|
||||
clear(): void {
|
||||
this.#inventory = undefined;
|
||||
this.#failure = undefined;
|
||||
this.#pending = undefined;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,229 @@
|
||||
import { basename, join } from "node:path";
|
||||
import { realpath } from "node:fs/promises";
|
||||
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisArtifactChangedError,
|
||||
AnalysisCancelledError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js";
|
||||
import type {
|
||||
ProviderProcessSnapshot,
|
||||
ProviderProcessSupervisor,
|
||||
} from "../process/ProviderProcess.js";
|
||||
import {
|
||||
copyAndHash,
|
||||
runKunaCommand,
|
||||
type KunaProcessLauncher,
|
||||
} from "./KunaCommand.js";
|
||||
import type { KunaInstallation } from "./KunaInstallation.js";
|
||||
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
|
||||
import type { KunaRunRelease } from "./KunaRunSlots.js";
|
||||
|
||||
interface Workspace {
|
||||
readonly root: PrivateRuntimeRoot;
|
||||
readonly input: string;
|
||||
}
|
||||
|
||||
export interface KunaTargetCommandSnapshot extends ProviderProcessSnapshot {
|
||||
/** Canonical, digest-checked private input used by this command. */
|
||||
readonly input: string;
|
||||
}
|
||||
|
||||
export interface KunaTargetCommandRequest {
|
||||
readonly installation: KunaInstallation;
|
||||
readonly operation: string;
|
||||
readonly arguments: (input: string) => readonly string[];
|
||||
readonly outputBytes: number;
|
||||
readonly timeoutMs: number;
|
||||
readonly signal: AbortSignal;
|
||||
/** Admission remains owned until the native process is proven stopped. */
|
||||
readonly releaseRun?: KunaRunRelease;
|
||||
}
|
||||
|
||||
/** Own the digest-checked target copy and every command that can still use it. */
|
||||
export class KunaTargetCommands {
|
||||
readonly #active = new Set<Promise<unknown>>();
|
||||
readonly #retained: {
|
||||
readonly error: ProviderCleanupError;
|
||||
readonly owner: ProviderProcessSupervisor | undefined;
|
||||
readonly releaseRun: KunaRunRelease | undefined;
|
||||
}[] = [];
|
||||
readonly #roots: PrivateRuntimeRoot[] = [];
|
||||
#workspace: Promise<Result<Workspace, AnalysisError>> | undefined;
|
||||
#closing = false;
|
||||
|
||||
constructor(
|
||||
private readonly target: BinaryTarget,
|
||||
private readonly lifetime: {
|
||||
readonly signal: AbortSignal;
|
||||
readonly launcher?: KunaProcessLauncher;
|
||||
},
|
||||
) {}
|
||||
|
||||
run(
|
||||
request: KunaTargetCommandRequest,
|
||||
): Promise<Result<KunaTargetCommandSnapshot, AnalysisError>> {
|
||||
if (this.#closing || request.signal.aborted) {
|
||||
request.releaseRun?.();
|
||||
return Promise.resolve(
|
||||
err(new AnalysisCancelledError(request.operation)),
|
||||
);
|
||||
}
|
||||
return this.#own(this.#run(request));
|
||||
}
|
||||
|
||||
#own<T>(work: Promise<T>): Promise<T> {
|
||||
this.#active.add(work);
|
||||
const release = () => this.#active.delete(work);
|
||||
void work.then(release, release);
|
||||
return work;
|
||||
}
|
||||
|
||||
/** Establish the immutable input before waiting for a foreground run slot. */
|
||||
prepare(): Promise<Result<null, AnalysisError>> {
|
||||
if (this.#closing || this.lifetime.signal.aborted)
|
||||
return Promise.resolve(err(new AnalysisCancelledError("kuna:workspace")));
|
||||
return this.#own(
|
||||
this.#ensureWorkspace().then((workspace) =>
|
||||
workspace.ok ? ok(null) : workspace,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async #run(
|
||||
request: KunaTargetCommandRequest,
|
||||
): Promise<Result<KunaTargetCommandSnapshot, AnalysisError>> {
|
||||
const { releaseRun, ...options } = request;
|
||||
let retained = false;
|
||||
try {
|
||||
const workspace = await this.#ensureWorkspace();
|
||||
if (!workspace.ok) return workspace;
|
||||
const outcome = await runKunaCommand({
|
||||
...options,
|
||||
arguments: request.arguments(workspace.value.input),
|
||||
workspace: workspace.value.root.path,
|
||||
reportedTimeoutMs: request.timeoutMs,
|
||||
launcher: this.lifetime.launcher,
|
||||
});
|
||||
if (outcome.kind === "retained") {
|
||||
this.#retained.push({
|
||||
error: outcome.error,
|
||||
owner: outcome.owner,
|
||||
releaseRun,
|
||||
});
|
||||
retained = true;
|
||||
return err(outcome.error);
|
||||
}
|
||||
return outcome.kind === "ok"
|
||||
? ok({ ...outcome.snapshot, input: workspace.value.input })
|
||||
: err(outcome.error);
|
||||
} finally {
|
||||
if (!retained) releaseRun?.();
|
||||
}
|
||||
}
|
||||
|
||||
/** Called after the session signal aborts; failures keep their retry owners. */
|
||||
async close(): Promise<Result<null, ProviderCleanupError>> {
|
||||
this.#closing = true;
|
||||
while (this.#active.size > 0) await Promise.allSettled(this.#active);
|
||||
for (const retained of [...this.#retained]) {
|
||||
const stopped = await retained.owner?.stop();
|
||||
if (stopped !== undefined && stopped.status !== "incomplete") {
|
||||
retained.releaseRun?.();
|
||||
this.#retained.splice(this.#retained.indexOf(retained), 1);
|
||||
}
|
||||
}
|
||||
const failures = this.#retained.map(({ error }) => error);
|
||||
const workspace = await this.#workspace;
|
||||
if (this.#retained.length === 0) {
|
||||
if (workspace?.ok === true) {
|
||||
this.#roots.push(workspace.value.root);
|
||||
this.#workspace = undefined;
|
||||
}
|
||||
for (const root of [...this.#roots])
|
||||
try {
|
||||
await root.close();
|
||||
this.#roots.splice(this.#roots.indexOf(root), 1);
|
||||
} catch (cause: unknown) {
|
||||
failures.push(
|
||||
new ProviderCleanupError(
|
||||
KUNA_PROVIDER_ID,
|
||||
[root.path],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
return failures.length === 0
|
||||
? ok(null)
|
||||
: err(
|
||||
new ProviderCleanupError(
|
||||
KUNA_PROVIDER_ID,
|
||||
failures.flatMap((failure) => failure.cleanupResources),
|
||||
{
|
||||
reason: failures
|
||||
.map((failure) =>
|
||||
String(failure.diagnostics?.["reason"] ?? failure.message),
|
||||
)
|
||||
.join("; "),
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async #ensureWorkspace(): Promise<Result<Workspace, AnalysisError>> {
|
||||
this.#workspace ??= (async () => {
|
||||
const root = await PrivateRuntimeRoot.create({ prefix: "rea-kuna-" });
|
||||
try {
|
||||
const input = join(root.path, basename(this.target.path));
|
||||
const digest = await copyAndHash(
|
||||
this.target.path,
|
||||
input,
|
||||
this.lifetime.signal,
|
||||
);
|
||||
if (digest !== this.target.sha256) {
|
||||
await this.#release(root);
|
||||
return err(
|
||||
new AnalysisArtifactChangedError(
|
||||
"kuna:workspace",
|
||||
this.target.path,
|
||||
"the target no longer matches the digest selected for analysis",
|
||||
),
|
||||
);
|
||||
}
|
||||
await prepareProcessOwnershipInspection(this.lifetime.signal);
|
||||
return ok({ root, input: await realpath(input) });
|
||||
} catch (cause: unknown) {
|
||||
await this.#release(root);
|
||||
throw cause;
|
||||
}
|
||||
})().catch((cause: unknown) =>
|
||||
err(
|
||||
new ProviderAdapterError(KUNA_PROVIDER_ID, "kuna:workspace", {
|
||||
cause,
|
||||
userMessage: `Kuna workspace failed: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
}),
|
||||
),
|
||||
);
|
||||
const result = await this.#workspace;
|
||||
if (!result.ok) this.#workspace = undefined;
|
||||
return result;
|
||||
}
|
||||
|
||||
async #release(root: PrivateRuntimeRoot): Promise<void> {
|
||||
try {
|
||||
await root.close();
|
||||
} catch {
|
||||
this.#roots.push(root);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,7 +5,7 @@ import { decodeAppleDispatchMetadata } from "./AppleDispatchMetadata.js";
|
||||
const base = 0x100000000n;
|
||||
const fixture = (
|
||||
relative = false,
|
||||
category?: "partial-method" | "unmapped-pointer",
|
||||
category?: "partial-method" | "unmapped-pointer" | "unmapped-class",
|
||||
) => {
|
||||
const bytes = Buffer.alloc(4096);
|
||||
const u32 = (offset: number, value: number) =>
|
||||
@@ -103,13 +103,15 @@ const fixture = (
|
||||
} else {
|
||||
ptr(0xb00, 0xb40);
|
||||
ptr(0xb40, 0xbc0);
|
||||
ptr(0xb48, 0x200);
|
||||
if (category === "unmapped-class") ptr(0xb48, 0x5000);
|
||||
else ptr(0xb48, 0x200);
|
||||
ptr(0xb50, 0xb80);
|
||||
u32(0xb80, 24);
|
||||
u32(0xb84, 1);
|
||||
ptr(0xb88, 0x460);
|
||||
ptr(0xb90, 0x5c0);
|
||||
bytes.writeBigUInt64LE(0xffffffffffffffffn, 0xb98);
|
||||
if (category === "unmapped-class") ptr(0xb98, 0x700);
|
||||
else bytes.writeBigUInt64LE(0xffffffffffffffffn, 0xb98);
|
||||
bytes.write("FixtureCategory\0", 0xbc0);
|
||||
}
|
||||
}
|
||||
@@ -398,6 +400,39 @@ describe("Apple dispatch coverage ownership", () => {
|
||||
reason: expect.stringContaining("category method FixtureCategory"),
|
||||
});
|
||||
});
|
||||
it("keeps a category whose class reference is unmapped", () => {
|
||||
const result = decodeAppleDispatchMetadata(
|
||||
fixture(false, "unmapped-class"),
|
||||
100,
|
||||
provenance,
|
||||
);
|
||||
expect(result.objc_categories).toMatchObject([
|
||||
{
|
||||
name: "FixtureCategory",
|
||||
class_name: null,
|
||||
class_source: "unresolved",
|
||||
decode: { status: "partial", reason: "category_class_unresolved" },
|
||||
},
|
||||
]);
|
||||
expect(result.objc_categories[0]?.instance_methods).toHaveLength(1);
|
||||
expect(result.objc_dispatch_implementations).toContainEqual(
|
||||
expect.objectContaining({
|
||||
category: "FixtureCategory",
|
||||
class_name: "(FixtureCategory)",
|
||||
implementation_address: "0x100000700",
|
||||
}),
|
||||
);
|
||||
expect(
|
||||
result.coverage.find(
|
||||
({ facet }) => facet === "objc_properties_categories",
|
||||
),
|
||||
).toMatchObject({
|
||||
status: "partial",
|
||||
reason: expect.stringContaining(
|
||||
"category_class_unresolved at 0x100000b48: Category FixtureCategory class reference 0x100005000",
|
||||
),
|
||||
});
|
||||
});
|
||||
it("keeps malformed category pointers out of class failure coverage", () => {
|
||||
const baseline = decodeAppleDispatchMetadata(fixture(), 100, provenance);
|
||||
const result = decodeAppleDispatchMetadata(
|
||||
@@ -851,4 +886,50 @@ describe("Apple dispatch list scan budget", () => {
|
||||
);
|
||||
expect(coverage?.reason).not.toContain("max_records_reached");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["RetainedProtocol", ["RetainedProtocol"], "complete"],
|
||||
["", [], "partial"],
|
||||
])(
|
||||
"keeps the class when the protocol name is %j",
|
||||
(protocolName, protocols, status) => {
|
||||
const bytes = unmappedProtocolImage("neither");
|
||||
bytes.writeBigUInt64LE(unmappedProtocolSectionBase + 0x700n, 0x300);
|
||||
bytes.writeBigUInt64LE(unmappedProtocolSectionBase + 0x800n, 0x708);
|
||||
bytes.writeUInt32LE(72, 0x740);
|
||||
bytes.write(`${protocolName}\0`, 0x800);
|
||||
const result = decodeAppleDispatchMetadata(bytes, 100, provenance);
|
||||
expect(result.objc_classes).toMatchObject([{ name: "RetainedClass" }]);
|
||||
expect(result.objc_protocols.map(({ name }) => name)).toEqual(protocols);
|
||||
const coverage = unmappedProtocolClassCoverage(result);
|
||||
expect(coverage).toMatchObject({ status });
|
||||
if (status === "partial")
|
||||
expect(coverage?.reason).toBe(
|
||||
"Protocol 0x100000700: Objective-C protocol has an empty name",
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps a protocol adopted only by an empty-named protocol", () => {
|
||||
const bytes = unmappedProtocolImage("neither");
|
||||
const at = (offset: bigint) => unmappedProtocolSectionBase + offset;
|
||||
bytes.writeBigUInt64LE(at(0x700n), 0x300);
|
||||
bytes.writeBigUInt64LE(at(0x800n), 0x708); // empty parent name
|
||||
bytes.writeBigUInt64LE(at(0x900n), 0x710); // adopted protocol list
|
||||
bytes.writeUInt32LE(72, 0x740);
|
||||
bytes.writeBigUInt64LE(1n, 0x900);
|
||||
bytes.writeBigUInt64LE(at(0xa00n), 0x908);
|
||||
bytes.writeBigUInt64LE(at(0xb00n), 0xa08);
|
||||
bytes.writeUInt32LE(72, 0xa40);
|
||||
bytes.write("ChildProtocol\0", 0xb00);
|
||||
const result = decodeAppleDispatchMetadata(bytes, 100, provenance);
|
||||
expect(result.objc_classes).toMatchObject([{ name: "RetainedClass" }]);
|
||||
expect(result.objc_protocols.map(({ name }) => name)).toEqual([
|
||||
"ChildProtocol",
|
||||
]);
|
||||
expect(unmappedProtocolClassCoverage(result)).toMatchObject({
|
||||
status: "partial",
|
||||
reason: "Protocol 0x100000700: Objective-C protocol has an empty name",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -52,8 +52,22 @@ export const decodeObjcCategories = (input: {
|
||||
const name = read.string(read.pointer(category));
|
||||
const external = boundClassName(read, category + 8n);
|
||||
const local = read.pointer(category + 8n);
|
||||
const className =
|
||||
external ?? (local === 0n ? null : input.localClassName(local));
|
||||
let className = external ?? null;
|
||||
if (external === undefined && local !== 0n)
|
||||
try {
|
||||
className = input.localClassName(local);
|
||||
} catch (cause: unknown) {
|
||||
// The category's own name and methods do not depend on its owner;
|
||||
// keep them and report the unreadable class reference.
|
||||
input.issues.push(
|
||||
issue(
|
||||
"objc_properties_categories",
|
||||
"category_class_unresolved",
|
||||
`Category ${name} class reference 0x${local.toString(16)}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
`0x${(category + 8n).toString(16)}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
const owner = className ?? `(${name})`;
|
||||
result.objc_categories.push({
|
||||
name,
|
||||
|
||||
Loaded 100 of 138 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user