Compare commits

...
Author SHA1 Message Date
gensokyo-dev a54464548e fix(environment)!: keep unsupported HAR body encodings opaque
Preserve declared encodings in capture Evidence and distinguish unsupported
response decoding from an absent body in shared replay and state extraction.
Use the production HAR decoder in replay regressions instead of synthesizing
its binary sidecars.

BREAKING CHANGE: replay metadata body_encoding can now be "unsupported".
Clients must treat its null body as unavailable decoded content while
body_recorded remains true; the cited Evidence retains the encoded source.
2026-10-23 19:03:32 +00:00
morluto 34563fcc9b fix(process): own bounded worker control framing
Share incremental replacement UTF-8 decoding across web and JavaScript workers, bound content before complete-line allocation and handle split CRLF consistently. Keep named schemas, request versus checkpoint failures, worker reuse and stream cleanup authority in their servers. Cover real server recovery and deterministic byte boundaries.
2026-10-24 02:35:59 +08:00
morluto f91b7f8009 fix(javascript): distinguish resolver discovery from module loading
Preserve load and resolve operations through semantic facts, worker transfer and graph Evidence. Only loads propagate executable ownership. Project ambient member writes and escapes through the shared path- and site-aware mutation owner; retain source literals, bases and uncertainty. Cover ownership and mutation boundaries through CLI and MCP.
2026-10-24 02:35:59 +08:00
morluto 635f673a8c fix(gpu): own CCOB boundaries and payload representations
Use completed codec input to establish legacy CCOB extents, preserving declaration/integrity failures independently. Reuse bounded input windows, charge actual reads and decoding attempts, and qualify recovery after incomplete decoding. Publish only established or qualified terminal extents so interruption retains completed declarations. Recognize bitcode representation in HIP bundles while leaving inner facts unsupported.

Fixes #6887
Fixes #6880
2026-10-24 02:34:35 +08:00
morluto e737e6c1c6 fix(erlang): distinguish empty optional BEAM chunks
Select legacy Atom when AtU8 is empty and skip zero-byte optional FunT. Preserve explicitly empty counted arrays, malformed nonempty failures, last-occurrence selection and original chunk provenance. Share structural fixtures across producer and public workflow regressions.

Fixes #6816
Fixes #6874
2026-10-24 02:34:35 +08:00
morluto 75a757d8ef fix(elf): distinguish compressed note storage from decoded notes
Refuse compressed note sections with typed unsupported-target semantics before framing or decoding, preserving stored-range checks and physical descriptor-offset provenance. Flutter retains independent build IDs with partial coverage when compressed notes are skipped. Verify the existing shared alignment fix with prefixed notes across classes and table kinds.

Fixes #6869
Closes #6563
2026-10-24 02:34:35 +08:00
morluto 25aaf89287 fix(sqlite): budget schema names by returned JSON text
Charge the whole-catalog admission guard by escaped decoded names, preserving raw-byte lookup identity and final worker reply limits. Keep UTF-16 inventory and detailed-schema results whose returned facts fit the documented allowance.

Fixes #6888
2026-10-24 02:34:35 +08:00
gensokyo-dev 3787ec5f83 test(ida): align native search fixtures with Unicode folding (#6898) 2026-10-23 18:31:32 +00:00
kiranandgensokyo-dev 5d2c8c2b78 feat(cli)!: export row-oriented results as JSON Lines (#5629)
* feat(cli): add JSON Lines output for row-oriented results (#5487)

- Support opt-in JSON Lines format (--format jsonl) for row-oriented results and collection views with clear record boundaries
- Preserve stable source identities and stream records deterministically line-by-line
- Emit final coverage and limitation metadata as a distinct line with https://rea.dev/schemas/jsonl-coverage-v1.json to prevent confusion with data rows
- Distinguish error outputs with https://rea.dev/schemas/jsonl-error-v1.json
- Reject non-row-oriented results that lack record boundaries with an unreadable_output error
- Document JSON Lines streaming conventions in docs/cli.md
- Add boundary tests covering empty results, multiline strings, partial coverage, large fields, and non-row rejection

* fix(cli): preserve diagnostics, provenance, and nested collections in JSONL

* fix(cli): frame classified failures and reject unrepresented JSONL facets

---------

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-24 02:13:28 +08:00
morluto 1de720e02e docs: stop and profile unexpectedly costly test runs 2026-10-24 02:06:18 +08:00
morluto 878da39caf chore: reconcile current main boundary fixes 2026-10-24 02:06:18 +08:00
morluto 80d5c8dc80 fix(comparison): preserve call procedure endpoint identity
Share endpoint identity and repeated-name ambiguity across call and reference dimensions. Compare same-artifact procedure entries by canonical coordinates while retaining raw reference sites and source Evidence. Keep observed comparison status separate from reconstruction proof.

Fixes #6853
2026-10-24 02:00:11 +08:00
morluto 6911a609f1 fix(model): preserve uncertainty across released GGUF layouts
Separate common row constraints from established storage sizes. Preserve Q8_1 and K-layout uncertainty, plus numeric IDs and ambiguous textual identities for historically shifted IDs16–25. Keep zero-byte facts and identity limitations independently, and propagate producer facts through totals, cancellation and retained views. Pin released source evidence and document the compatibility cost.

Fixes #6848
2026-10-24 02:00:10 +08:00
morluto 907f4d99f5 fix(flutter): derive dynamic symbol tables from their source
Decode only consumed dynamic-entry payloads. Discover PT_DYNAMIC symbols when retained sections omit SHT_DYNSYM, with table provenance governing section-index applicability. Preserve malformed present-table failures and shared source fixtures.

Fixes #6851
Fixes #6884
2026-10-24 02:00:10 +08:00
morluto 6fbe67d66e fix(apple)!: use archive UIDs for keyed nib identity
Use original object-table UIDs for inventory, hierarchy and connector references. Retain authored identity fields as data, refuse forged reference aliases and select UIKit top-level collections without substituting unrelated roots.

BREAKING CHANGE: Keyed archives that previously used authored objectID or object-id graph identities now use decimal table UIDs. Authored values remain attributes; ibtool and native NIBArchive identities retain their existing source semantics.

Fixes #6849
Fixes #6850
2026-10-24 02:00:09 +08:00
morluto 5e4004c413 fix(artifacts): preserve traversal coverage through bundle workflows
Retain producer traversal uncertainty through inventory, extraction, Interface Builder decoding and executable-root discovery. Keep known copied files and decoded facts with honest coverage; share omission normalization and bound omitted archive paths with their source reasons.

Fixes #6852
Fixes #6825
Fixes #6582
2026-10-24 02:00:09 +08:00
morluto d1fea89e1e style: format integrated boundary fixes 2026-10-24 01:58:35 +08:00
morluto 162b9e5863 test: share Frida SDK connection setup within fixtures 2026-10-24 01:57:28 +08:00
morluto 8ad3e0ea30 test(managed): split edge comparison scenarios by semantics 2026-10-24 01:57:28 +08:00
morluto 7665bf0831 test(apple): separate catalog traversal coverage cases 2026-10-24 01:57:28 +08:00
morluto a8758da09b fix(apple): account for roots skipped by ordered reads 2026-10-24 01:53:17 +08:00
morluto 5feac72c79 test: reuse Frida SDK setup across status scenarios 2026-10-24 01:49:39 +08:00
morluto 4f3d486d8f fix(setup): retain TOML comments from replaced registration code 2026-10-24 01:49:13 +08:00
morluto 1e53933786 test(session): acquire provider before mixed cleanup recovery 2026-10-24 01:49:13 +08:00
morluto 88bb8d5fe3 test(frida): pass the signal to finalization listeners 2026-10-24 01:47:31 +08:00
morluto 2732d538e8 fix(session): own target admission descriptors through finalization 2026-10-24 01:47:31 +08:00
morluto a8276d1289 docs: clarify capture comparison coverage and identity 2026-10-24 01:47:31 +08:00
morluto 6e324b2244 fix(javascript): share accessor selection and invocation effects 2026-10-24 01:47:31 +08:00
morluto 48f700ee35 fix!: retain Frida observations through shared finalization
Reuse the shared one-shot finalizer for discovery and instrumentation, preserve finalization cancellation with completed observations, and bind Frida cleanup errors to exact owners so confirmed retry recovers the primary error.

BREAKING CHANGE: instrument-with-frida JSON now returns the canonical Evidence record directly. Read normalized_result.cleanup_error instead of the earlier top-level cleanup_error wrapper field.
2026-10-24 01:47:31 +08:00
morluto 37122745b6 fix: retain status observations when session cleanup fails 2026-10-24 01:47:31 +08:00
morluto aedd394471 perf(tests): share native Interface Builder encoding production 2026-10-24 01:47:31 +08:00
morluto 78f8eca531 fix(apple): preserve bundle discovery coverage and decode failures 2026-10-24 01:47:31 +08:00
morluto 8f29c7ab66 fix(artifacts): preserve extraction coverage and provenance 2026-10-24 01:47:31 +08:00
morluto 693a3851ec test(managed): bound capacity fixtures with reserved budgets 2026-10-24 01:47:31 +08:00
morluto 357e354ac7 fix(managed): preserve member scope and nominal edge identity 2026-10-24 01:47:31 +08:00
morluto f5d791f2a1 fix(setup): preserve client-owned MCP policy on refresh 2026-10-24 01:47:31 +08:00
morluto a0c22ff012 fix(browser): preserve correspondence and uncertainty in capture comparisons 2026-10-24 01:47:31 +08:00
morluto edf7b3d32c fix(native): preserve endpoint context in function comparisons 2026-10-24 01:47:31 +08:00
morluto 211e3533b3 fix(setup): retain skill publication ownership through rollback 2026-10-24 01:47:31 +08:00
morluto b3c912ab79 fix(filesystem): retain read cleanup capabilities across retries 2026-10-24 01:47:31 +08:00
morluto 0c6fed90b7 perf: batch modern IDA callee function lookups 2026-10-24 01:47:30 +08:00
morluto 5e4ec0fa1b perf: retain call path ancestry once per visited node 2026-10-24 01:47:30 +08:00
morluto 8aa40e9a61 fix(cache): scope Evidence display refresh to matching snapshot bindings 2026-10-24 01:47:30 +08:00
morluto f8b65e7872 fix(reconstruction)!: bind proof to owner and obligation revision
Parse a named proof contract once and qualify exact obligation, owner, revision, fixture/case, and verifier/claim tuples. Preserve unavailable proof separately from applicable failed outcomes.

BREAKING CHANGE: ID-array reconstruction proof records no longer qualify for verified or ready ledgers. Producers must supply implementation and revision bindings with individual claim outcomes; ledger diagnostics now distinguish inapplicable, unavailable, and failed proof.
2026-10-24 01:47:30 +08:00
morluto 592c46b001 test(managed): preserve token identity in graph assertions
Replace ambiguous display-label sorting with exact relation assertions keyed by metadata tokens. Distinguish same-name methods and verify directed ownership, edge counts, properties and Evidence without depending on graph order.
2026-10-24 01:37:47 +08:00
morluto a496131e5a refactor(qlog): reuse capture redaction policy
Reuse credential header classification and lossless numeric redaction while preserving qlog event scopes, source literals and provenance pointers. Cover HTTP and extension fields plus explicit numeric sensitivity.
2026-10-24 01:33:41 +08:00
morluto cee14e1fda refactor(javascript): reuse semantic member-chain facts
Resolve member structure through the existing AST owner while preserving binding, shadowing and global process identity checks in their consumers.
2026-10-24 01:33:41 +08:00
morluto 92f28b2f56 refactor(evidence): share comparison payload interpretation
Decode named comparison payloads and validate nested citations at one domain boundary. Preserve workflow-specific authentication, source closure policy and validation order; cover malformed payloads and citation boundaries.
2026-10-24 01:33:41 +08:00
morluto ef475fcf8b fix(sqlite): distinguish query capacity and unavailable row sources
Use one bound JSON selection set for inventory metadata, keeping raw-name byte
comparison and filtering inside SQLite while preserving caller projection order
and repetition. Large requests reach native result-column capacity attribution.

Separate unavailable row sources from generated-expression failures. Preserve
SQLite's reason and selected table/path, offer schema-only recovery, and avoid
claiming corruption or recommending unreadable columns.

Validated UTF-8/UTF-16 identity, native inspection/worker tests and CLI/MCP failure
and recovery workflows. The large CLI selection uses the existing framed CLI
fixture to avoid host argv limits. Source bytes and identity remain unchanged.

Fixes #6855
Fixes #6604
2026-10-24 01:24:43 +08:00
morluto 39a91d4db9 fix(gpu): preserve declaration grammar and partial resource coverage
Share AMDGPU triple grammar between bundle and note owners so modern target
spellings participate in processor/feature conflict detection. Propagate CUBIN
register conflicts into object, target and report coverage.

Normalize adjacent PTX dot keywords in the lexer, removing the downstream pointer
splitter. Parse post-type alignment while distinguishing parameter and pointee
layout, preserving names and byte provenance when layout remains unknown.

Validated production-boundary matrices and public CLI/MCP coverage workflows with
inert fixtures. Correct the HIP-v3 control's ELF processor declaration. Real GPU,
assembler and compiler-produced modern-note behavior remains unverified.

Fixes #6844
Fixes #6823
Fixes #6858
Fixes #6868
Fixes #6861
2026-10-24 01:24:43 +08:00
morluto d137a4cdde fix(android): preserve metadata dialect and root manifest identity
Read Apktool 3's producer string dialect before YAML can discard hashes or
reinterpret values. Preserve older YAML behavior and decode producer escapes once.

Select the exact root AndroidManifest.xml resource in REA's JADX bridge and
derive summaries once with namespace-aware XML parsing. Preserve decoded text,
summarize before its display cap, invalidate cached facts with the loaded session,
and remove superseded upstream manifest handlers. Keep the engine pinned to 0.7.1.

Validated Apktool parser/provider and public adapter controls; compiled the bridge
against the verified pinned JAR and ran its real Java regressions plus public CLI
and stdio MCP manifest parity checks. Real Apktool execution remains unverified;
the new manifest fixtures use plain XML. No Android application was executed.

Fixes #6804
Fixes #6866
Fixes #6863
Fixes #6819
2026-10-24 01:24:43 +08:00
morluto f8f52f205a fix(python): preserve marshal names and packaged bytecode
Use one marshal surrogatepass decoder for PYZ directories and Python bytecode.
Validate ordinary UTF-8 spans with the native fatal decoder, preserving BOMs,
lone surrogates, raw bytes and marshal reference slots.

Distinguish legacy full-pyc CArchive modules from modern marshal-only modules
before reconstructing headers. Share typed source/codec failures between stream
output and completion while retaining native-read drain and cleanup ownership.

Validated focused producer, reader, lifecycle, CLI and MCP regressions, plus
public inventory/extraction of CPython 3.14-generated marshal tables and code.
No packaged application was executed. Document original versus derived bytes.

Fixes #6800
Fixes #6789
2026-10-24 01:24:43 +08:00
morluto 2443b949a4 docs: clarify evidence applicability and boundary limitations 2026-10-24 01:18:46 +08:00
morluto d0432d63c9 refactor(managed): share decoded type definition facts 2026-10-24 01:11:58 +08:00
morluto f276bcbf6c refactor(javascript): own paired graph evidence resolution 2026-10-24 01:11:58 +08:00
morluto 6cd3fa09b1 refactor(process): centralize post-run artifact verification 2026-10-24 01:11:58 +08:00
morluto 3e7e102a51 fix(rust): separate section storage from compressed stream admission
Keep validated PE padding in the selected Evidence range while consumed zlib bytes own compressed and inflate-ratio limits. Preserve typed resource failures and bounded allocation. Validated accepted padding, over-ratio and compressed-size controls through source, real CLI and stdio MCP workflows.
2026-10-24 01:00:14 +08:00
morluto 42e785925c fix(adb): accept Android Java float spellings unchanged
Use Float.valueOf grammar at shared admission and retain selected spelling through am --ef. Validated scientific notation, malformed controls and exact MCP forwarding with a source-owned ADB fixture; no device execution.

Fixes #6796
2026-10-24 01:00:14 +08:00
morluto d4415444c4 fix(native): retain processor, hidden-slice and property type facts
Separate AMDGPU processors from feature-qualified note identities. Read Apple hidden ARM64 table rows once and preserve visibility through artifact, dylib and fuse consumers. Keep Objective-C++ template commas within type attributes and incomplete metadata unknown.

Validated provider/parser and production property regressions, Electron CLI/MCP consumers, and native Apple lipo -hideARM64 output through the production slice reader and public artifact inspection. LLVM execution remains unverified.

Fixes #6806

Fixes #6799

Fixes #6801
2026-10-24 01:00:14 +08:00
morluto 6fdcac6cad fix(sqlite): preserve catalog byte identity and worker context
Match table metadata by stored name bytes and fold only ASCII code units for relationships, reusing parsed facts without per-trigger queries. Preserve caller paths on structured worker limits and V8 heap failures.

Validated invalid UTF-8 collisions, distinct UTF-16 names and real worker transport refusal through CLI and stdio MCP. Controlled stderr covers OOM classification; actual memory exhaustion was not induced.

Fixes #6795

Fixes #6607
2026-10-24 01:00:14 +08:00
morluto ccba355fa9 fix(javascript): preserve map boundaries and loader provenance
Share embedded-map version admission and derive indexed overlap from represented mappings. Carry createRequire filename bases through worker facts and graph identity. Select factory export bindings at each bundler owner.

Validated accepted and malformed source maps through real CLI and stdio MCP codec workflows; verified loader and esbuild graph output against Node runtime controls. Rebased consumer checks passed.

Fixes #6803

Fixes #6614

Fixes #6798

Fixes #6797
2026-10-24 01:00:14 +08:00
morluto d78892eb4b refactor: remove superseded Hopper and Babel compatibility paths 2026-10-24 00:54:44 +08:00
morluto 33f90f95b6 test: validate evidence and select fixtures structurally 2026-10-24 00:54:36 +08:00
morluto a32d731ed7 fix(providers): share selected-file error classification 2026-10-24 00:54:28 +08:00
morluto 8d8d85ba22 refactor(cli): share parsed output controls across consumers 2026-10-24 00:32:05 +08:00
morluto 4526862728 fix(apple): decode valid nonstandard plist table widths
Reuse preflight-parsed container references for bounded decoder-copy normalization. Preserve original artifact bytes, scalar provenance and reference order. Verify three-byte offset and reference encodings through CLI and stdio MCP.

Fixes #6805
2026-10-24 00:30:34 +08:00
morluto 6c6e2ad62b fix(wasm): preserve core inspection with optional metadata warnings
Retain warnings from validator, objdump and decoder through one diagnostic owner. Keep uninterpretable name bytes and explicit name-metadata coverage in complete results and summaries.

Fixes #6812
2026-10-24 00:30:34 +08:00
morluto 929a0a76c8 fix(rust): preserve JSON field identity and PE stream extents
Reject duplicate decoded JSON names before object conversion. Use one checksummed zlib stream parse and retained PE FileAlignment to distinguish exact zero padding from trailing payload bytes. Exercise public CLI and stdio MCP.

Fixes #6814

Fixes #6793
2026-10-24 00:27:57 +08:00
morluto 4c5fae5179 fix(il2cpp): decode valid shared-suffix string offsets
Preserve bounds, termination and UTF-8 checks; verify CLI and stdio MCP preserve names and selected artifact identity.

Fixes #6808
2026-10-24 00:27:57 +08:00
morluto fc63fe8682 fix(gpu): distinguish host descriptor failures from invalid targets
Share selected-file errno ownership with model inspection and codec result accounting with Rust. Verify actual descriptor exhaustion and unchanged-input recovery in an isolated child.

Fixes #6666
2026-10-24 00:27:57 +08:00
morluto 96ea32b7e4 perf(environment): decode only selected replay query values 2026-10-24 00:14:15 +08:00
morluto b784912811 fix(cli): share Markdown formatting with standalone reports
Route version and doctor report tables through the same formatter used for command output, preserving escaped cells and structured JSON facts.
2026-10-24 00:11:41 +08:00
morluto 87fa1b613d fix(managed): retain nested names and ReadyToRun observations
Resolve attribute display names from validated nesting and parsed member facts, admit composed names before allocation, and preserve unknown malformed chains. Keep ReadyToRun classification ahead of metadata hints while retaining each observed hint.

Fixes #6815
Fixes #6810
2026-10-24 00:11:41 +08:00
morluto ddd7ac7b84 fix(javascript): preserve Next filesystem route spelling
Apply Next-specific uppercase %5F normalization while retaining ordinary percent sequences, encoded separators, and literal percent characters as distinct source segments.

Fixes #6641
2026-10-24 00:11:41 +08:00
morluto f719dc46c5 fix(views): select every retained module path without losing identity
Use one retained-path owner for module references and items. Preserve selected paths and genuine ambiguity across module kinds; remove the superseded singular helper.

Fixes #6790
2026-10-24 00:11:41 +08:00
morluto 9a3780229b test(search): pin native Unicode substring matching
The shared literal owner already applies Unicode simple folding. Verify the reported production-helper cases across insensitive literal, sensitive literal and regex selection.

Fixes #6634
2026-10-24 00:11:41 +08:00
morlutoandEzra 678c4cfa68 fix(flutter): preserve short GNU build IDs through public workflows
Reuse the reader correction from PR #6705, and share the byte projection budget with output validation. Verify actual APK parsing through compiled CLI and stdio MCP at 0, 1, 2, 3, 64 and 65 descriptor bytes.

Co-authored-by: Ezra <64943463+ezrabrilliant@users.noreply.github.com>

Fixes #6697
2026-10-24 00:11:41 +08:00
morluto 866c72c907 docs(desktop): list all supported IL2CPP metadata versions
Match the declared probe versions and retain the version-24 layout limitation.

Fixes #6672
2026-10-24 00:11:41 +08:00
morluto 7ba05210fe fix(reference): preserve raw symlink target identity
Read link targets as bytes before strict UTF-8 admission. Retain malformed targets as unreadable with source hex; preserve literal replacement characters and valid BOM-prefixed targets.

Fixes #6765
2026-10-24 00:11:41 +08:00
morluto 51e5938776 fix(pcap): retain UTF-8 decoding failures across tool output
Validate supervised text once before version or capture interpretation. Exercise POSIX client probes through compiled CLI and stdio MCP; capture-summary native bounds remain host-specific.

Fixes #6775
2026-10-24 00:11:41 +08:00
+12 169512a35b feat(cli): report version and runtime with an explicit output format (#6530)
* feat(cli): report version and runtime with an explicit output format

`rea --version --json` and `--version --format <format>` ignored the format
and printed only the bare version, so support reports gathered the runtime
and platform by hand.

A root --version request that selects a format now renders a documented
report: package name and version, JavaScript runtime and version, platform,
architecture, and whether the runtime satisfies the supported range. It
contains no paths, environment values, user or host names, and starts no
provider. Plain --version is unchanged, --help keeps precedence, and other
output controls keep Incur's plain version.

The runtime verdict uses a new shared supportsJavaScriptRuntime rule that
rea doctor now uses too, so both judge Bun by its Node.js compatibility
version. mcp doctor's private JSON/format renderer moves to cliOutput as
renderCliDocument and is shared with the version report.

Closes #5483

* refactor(errors): centralize input failure construction

Carry caller messages through the existing schema issue owner instead of constructing and discarding an intermediate input error. Remove forwarding schema helpers and migrate equivalent constructors across investigation, firmware, dylib, process, and screenshot workflows while preserving diagnostic paths and causes.

* docs(readme): add Roman Urdu edition (#6499)

* docs(readme): add Roman Urdu edition

Register ur-Latn with a pending website catalog and preserve English command examples.

Script-tag handling follows morluto's shared helper proposed in PR #2043 (c46f606c89).

* test(website): cover completed script-tag page metadata

* fix(process): parse native teardown relation sentinels

* fix(cli): make version report privacy checks environment-independent

The CLI test rejected any report containing the OS account name, so an
account named node, linux or rea failed a correct report. Check unique
caller-controlled HOME, cwd and environment markers instead; the strict
schema still rejects any additional field.

The CLI guide said --version with other output controls prints the plain
version, but invalid combinations such as --json with --token-limit report
their existing output-flag error first. Describe both outcomes and test them.

* refactor(nativeaot): share report facts across budget stages

Centralize parse-wide identity and coverage facts while keeping conservative admission values separate from measured rows and counters. Reuse the derived truncation reason and retain the final serialization bound.

Tighten the report-budget regression so unexpected parser finalization errors cannot be swallowed by its budget search.

* fix(adb): preserve empty properties and verbatim version names

* fix(artifacts): recognize empty Zip64 containers

Fixes #6540

* fix(erlang): accept uninterpreted BEAM alignment bytes

Fixes #6517

* fix(gpu): accept PTX entries without parameter lists

Fixes #6493

* fix(python): preserve signed code-object line numbers

Fixes #6490

* refactor(ghidra): centralize load image check recording

Share expected and observed check construction across MZ, COM, and raw-binary attestations. Preserve format-specific rules and source coordinates, and derive mapping callback types from the recording owner.

* fix(inspector): preserve script identity and parse-failure facts

Fixes #6535

Fixes #6546

* fix(artifacts): recover malformed PYZ sizes without losing siblings

Fixes #6120

* fix(erlang): accept extended BEAM atom lengths

Fixes #6503

* refactor(erlang): reject overflowing BEAM atom lengths

* fix(gpu): preserve PTX and AMD target facts

Fixes #6469

Fixes #6483

* refactor(targets): preserve ambiguous MZ header evidence

* fix(adb): retain refused activity launch reports

Parse complete nonzero Activity Manager reports while preserving other command failures and their captured output. Admit dollar signs only in the activity component name for nested Android classes.

Fixes #6528

Fixes #6527

* fix(mcp): retain partial evidence through shared delivery

* fix(python): honor version-specific code equality

Fixes #6547

* fix(pyinstaller): preserve namespace declarations

Project PYZ namespace packages as metadata so their query paths do not claim output directories. This lets independent CArchive files retain the same logical path and remain inspectable and extractable.

Fixes #6539

* docs: clarify evidence ownership and verification guidance

* docs(adb): clarify emulator observation field

* docs(artifacts): clarify active-reader extraction scope

* fix(erlang): select the last repeated BEAM chunk

Fixes #6518

* fix(il2cpp): distinguish rank-one array types

Fixes #6534

* fix(python): align pyc and PYZ marshal decoding

Fixes #6560
Fixes #6559

* fix(browser): preserve frame identity across captures (#4040)

* fix(browser): preserve capture evidence fidelity

* fix(browser): align frame identity comparisons

* fix(browser): retain historical frame evidence

* fix(electron): retain same-document script owners

* fix(electron): bind frame proof to tree reads

* fix(browser): retain malformed frame scope

* fix(browser): retain WebMCP event scope

* fix(browser): retain malformed timeline identity

* fix(browser): unify frame identity and scope

* fix(electron): prove enclosing frame ancestry

* fix(electron): separate ancestry from file policy

* fix(browser): preserve transient fragment semantics

* fix(browser): retry uncertain initial frame scope

* fix(browser): preserve runtime frame identity

* refactor(browser): separate independent capture fixes

* fix(electron): detect tree-first document replacement

* fix(electron): retain same-document tree authority

---------

Co-authored-by: eco-null <eco-null@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(elf): resolve symbol versions from requirements

Fixes #6533

* fix(elf): follow loader version index precedence

VERDEF entries replace VERNEED names for shared indices, matching glibc lookup behavior.

* test(firmware): bind owned producers and exercise real limits

* fix(adb): preserve push target semantics

Fixes #6566

Fixes #6567

* fix(cli): merge sibling output filter slices

Fixes #6542

* fix(browser): preserve Link context semantics

Fixes #6504

Fixes #6505

Fixes #6506

* fix(python): classify historical headers and signed flags

Fixes #6568, #6487

* fix(python): normalize legacy code flags for equality

Preserve CPython 3.7–3.10 constructor flag semantics in code keys.

* fix(environment): compare captured bodies by representation

Keep JSON, text, and retained bytes distinct through fidelity comparison. Compare binary bodies by bytes, including valid UTF-8 bytes against equivalent text, and keep empty-body compatibility explicit.

Fixes #6564

* docs: clarify derived values and comparison controls

* fix(environment): scope fidelity ID normalization

Reserve equal global ID pairs and stop normalizing contested pairings. Keep numeric measurements literal, scope keyed collection identities through learning and comparison, and normalize string references only where learning established the same field as an ID.

Fixes #6565
Fixes #6550
Fixes #6557

* test(process): print the progress frames when the sparse-child bound fails (#6427)

The sparse-child capture test asserted the frame bound with a bare boolean,
so a failure only printed "expected false to be true". The assertion now
carries the running progress details (elapsed time, frames, samples,
interactions) as its failure message. This only adds diagnostics to a
failure; it does not change what passes and does not fix the flake.

Refs #5889.

🤖 Generated with Claude Code (Sonnet 5.5) 🦊

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(il2cpp): name the metadata file when linking finds malformed metadata (#6425)

* fix(flutter): honor ELF note alignment when reading build IDs (#6423)

* fix(flutter): honor ELF note alignment when reading build IDs

* test(flutter): key shared note ranges by effective alignment

Cover a program and section declaration that share one note range: differing effective alignments (8 and 4) must both be read, while declarations that both normalize to four bytes (1 and 4) still share one read. 24-byte build IDs pad identically at both alignments, so only the range key decides the read count.

* refactor(firmware): centralize audited tool requirements

* fix(javascript): associate requested bindings only with resolved native addons (#6402)

* fix(javascript): associate requested bindings only with resolved native addons

* test(javascript): cover the exact native addon association

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(python): reject code keys and set members with unhashable consts (#6424)

* fix(python): reject code keys and set members with unhashable consts

* fix(python): refuse to hash a tuple that is still being read

* fix(python): hash a code object's co_consts tuple, not its item list

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(model): recognize named chat templates in summaries

Count retained template content rather than only the default key; keep names-only inventories distinct.

Fixes #6569

* fix(model): retain resolved GGUF alignment under cancellation

Publish alignment from the parser when explicit metadata validates it, or when metadata completion establishes the default. Preserve that fact through later parsing and hashing failures.

Fixes #6579

* fix(environment): reject out-of-range clock offsets

Reject invalid ISO offsets while preserving valid boundary values. Validate stored clock-format offsets through the same parser so malformed imported formats cannot reach rendering.

Fixes #6524

* fix(environment): preserve submillisecond insert ordering

Compare retained submillisecond components after milliseconds, so ordered collections recognize and place distinct timestamps correctly.

Fixes #6525

* fix(environment): restrict epoch normalization to clock fields

Require a time-field token boundary before treating numeric values as epoch clocks, so identifiers such as media format codes remain literal. Clarify recognized field forms and clock offset and precision semantics in the fidelity guide.

Fixes #6551

* fix(process): preserve sibling path identities

Keep aliases on selected paths at clear text delimiters so a longer sibling filename is not rewritten as the executable. Version the normalization contract and cover source and actual PTY behavior.

Fixes #6586

* fix(sigrok): retain recording suffix for autodetection

Copy recordings under a private filename that retains the caller-selected final suffix, and pass that filename to sigrok. This restores filename-based autodetection without exposing the source directory.

Fixes #6580

* test(go): assert unresolved MZ candidate boundary

* fix(pe): ignore unused export ordinal slots

Fixes #6577

* fix(macho): recognize arm64_32 binaries

Fixes #6581

* fix(go): honor amd64p32 pointer width

Fixes #6590

* fix(pe): accept BER indefinite CMS values

Fixes #6574

* fix(browser): reject overlapping indexed source maps

Fixes #6562

* fix(browser): resolve prefixed XHTML form actions

Fixes #6561

* fix(browser): preserve well-known path case

Fixes #6571

* docs(browser): clarify parser evidence contracts

* fix(gpu): accept percent-prefixed PTX entry names

Fixes #6592

* fix(gpu): accept PTX compute target aliases

Preserve the source spelling of NVIDIA's documented compute_xx target synonym.

Fixes #6494

* fix(gpu): parse legacy PTX body parameters

PTX 1.0 through 1.3 declares kernel parameters inside the entry body. Keep later-version parameter-list parsing unchanged.

Fixes #6594

* fix(gpu): keep unknown targets out of exact filters

Fixes #6599

* fix(gpu): resolve extended CUBIN symbol sections

Fixes #6600

* docs(firmware): document the Windows Unblob submodule checkout (#6464)

On Windows, the pinned Unblob commit cannot be checked out because 11
upstream fixture paths under tests/integration/ are invalid on NTFS.
Document the verified sparse-checkout workaround and point Windows
contributors to it from CONTRIBUTING.


Claude-Session: https://claude.ai/code/session_01LjhoPGpqAwHwcCgectzRsp

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* fix(recovery): retain file outcomes and cleanup ownership

Use shared owned-file and directory primitives throughout Wakaru discovery,
snapshots, report parsing and publication. Preserve primary read failures and
completed fingerprints when final close fails or receives cancellation.

Retain exact cleanup capabilities before retiring workspaces, block new work
while cleanup is unresolved, and bind error status to the aggregate owner so
partial cleanup and later retry keep resource attribution current.

* fix(pe): preserve biased export ordinals

Accept the format-derived ordinal range for PE export slots.

* docs: clarify parser authority and compatibility evidence

* fix(python): preserve code-name identity

Fixes #6593

* fix(managed): validate metadata operands and locals declarations

Classify MemberRef operands from parsed signatures, validate signature table references and LocalVarSig blobs, and compare observed locals declaration presence. Keep partial CIL facts and transient-budget ownership at the method reader.

Fixes #6578
Fixes #6572
Fixes #6558
Fixes #6513

* fix(python): preserve historical signed code fields

Keep signed co_stacksize and co_flags for historical magic entries whose constructors accepted them. Preserve magic 3410 with a constructor-revision limitation; reject negative values from 3411 onward.

Fixes #6487

* fix(python): compare code objects without inline caches

Fixes #6597

* fix(tests): stabilize process fixtures and malformed input coverage (#6570)

Preserve renamed-executable identity, complete-line readiness and bounded child lifetimes. Use genuinely malformed Inspector input and explicit ADB file-probe/push/hash facts while preserving cancellation, cleanup, Evidence and completeness assertions. Adopt current main Go fixtures and retain malformed COFF coverage.

Fixes #6556
Refs #6480

* test(apktool): own lifecycle fixtures and process observations

* refactor(mcp): reuse parsed Evidence tool parameters

* fix(runtime): align remaining Node security pins and setup guidance (#6316)

Derive exported Docker image versions from .nvmrc through generated package metadata. Align active setup translations with patched Node release floors and verify actual CLI exports, real Docker/agentenv, package and website workflows.

Eight built-test shards pass; three unrelated managed source failures reproduce on clean main and remain tracked in #6650.

Refs #6480

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>

* fix(model): retain exact safetensors shape dimensions

Keep unsigned integer declarations exact until their owning field is interpreted. Bound actual offsets at conversion and validate shape products in declaration order, preserving large empty tensor shapes without hiding an earlier overflow.

Fixes #6627

* fix(python): read legacy TYPE_INT64 marshal values

Fixes #6609

* fix(apk): read constructed BER signer signatures

Share the bounded BER tree reader between PE certificates and APK v1 CMS, then accept constructed OCTET STRING signature values while retaining the declared signer and certificate facts. Embedded X.509 certificate decoding keeps its DER requirement.

Fixes #6603

* fix(adb): count Unicode requested permissions

Count requested-permission rows containing Unicode names, and keep counting subsequent rows in the same dumpsys section.

Fixes #6584

* fix(signals): reject missing sigrok channels

Recognize the exact selected-channel diagnostic even when sigrok-cli exits successfully, while preserving partial producer output in the invalid-input details. Cover both public JSON adapters and add the missing-channel case to the Linux real-tool verifier.

Fixes #6631

* fix(elf): reuse recognized machine architecture facts

Preserve the shared EM_HEXAGON classification in Go and Rust image reports.

Fixes #6629

* fix(asn1): decode four-byte BER lengths unsigned

Accumulate length octets arithmetically so a four-octet length with bit 31 set remains a positive unsigned value and is rejected by the containing-range check rather than becoming a negative length.

* fix(gpu): preserve parameter and target declarations

Decode observed V2 CUDA parameters, retain qualified PTX parameter names and entry resource counts, and preserve family and feature target declarations through selection and consistency checks.

Fixes #6624
Fixes #6625
Fixes #6626
Fixes #6618
Fixes #6613
Fixes #6606

* fix(captures): preserve canonical TShark alias fields

TShark resolves selected field aliases before writing packet JSON. Keep the requested selector in Evidence while accepting the canonical field key produced for it.

Fixes #6595

* fix(qlog): mark records with unknown STREAM FIN incomplete

Keep known STREAM ranges and the unknown FIN value, while listing the source event among incomplete fact records so summary and retained views agree.

Fixes #6598

* fix(pcap): retain absolute HTTP request targets

Reuse parsed URI facts and preserve uncertainty when mixed request records cannot all be resolved from the producer output.

Fixes #6537

* fix(rust): preserve historical cargo-auditable feature facts

Accept observed feature arrays, preserve omitted values as unknown across producer revisions, and keep older retained reports compatible. Reuse bounded JSON sizing before graph expansion.

Fixes #6622

* fix(ptx): normalize parameter declarators once

Fixes #6606

* fix(managed): compare observed call targets

Compare aligned managed call edges by their reported declaring type and method name, keeping missing target facts unknown without comparing build-local tokens. Document the static comparison limits.

Fixes #6630

* fix(artifacts): compare PyInstaller declaration facts

Compare stable producer-tagged declarations independently of materialized payloads. Preserve missing facts as unknown without applying PyInstaller rules to foreign metadata or nested container entries.

Fixes #6484
Fixes #6485

* test(managed): retain derived facts in cloned method fixtures

Clone caller-scoped call and field facts with the fixture methods. Keep known CIL changes alongside unavailable target metadata in the missing-row controls.

* refactor(apk): consume shared BER node facts

Remove the APK reader's pass-through children and content projections so format consumers read the shared BER owner's nodes directly. Keep only the encoded-span helper needed for exact issuer and certificate bytes, and describe CMS BER as this parser's structural owner.

* fix(adb): delimit permission section by indentation

* test(managed): preserve field-access owner identities

* fix(managed): share reference instruction semantics

Use the same parsed method/field operand predicate when producing member edges and comparing call and field shapes. Exclude ldtoken and operands without a value from both interpretations.

* test(managed): align source goldens with metadata validation

Fixes #6650

* fix(native): consolidate observed address validation

* fix(gdb): separate queued work from MI synchronization

Queued console followers have not dispatched MI records. Use active protocol requests when deciding whether the admitted close can verify and detach its inferior.

Fixes #6661

* fix(gdb): wait for the outer console command to finish

After a running result, queue a tokenized read-only MI query to establish that GDB has returned to its command loop. Keep collecting through intermediate inferior stops and retain the query reply as protocol Evidence.

Fixes #6660

* fix(erlang): retain admission cancellation reason

Preserve the caller's AbortSignal reason when BEAM inspection is cancelled before provider admission. Keep the production filesystem boundary and late-completion contracts covered.

Fixes #6658

* fix(desktop): probe GameMaker data filenames

Select conventional Linux and macOS game data files for the existing bounded FORM/GEN8 probe. Add service-boundary regressions and document the selected names.

Fixes #6659

* fix(harmony): exclude raw resources from manifests

Classify known resource paths once and keep a raw resource named module.json from changing the package model or appearing as a manifest. Add a filesystem-boundary regression and clarify the documented hint.

Fixes #6638

* fix(gpu): bound embedded section reads

Fixes #6657

* fix(gpu): retain resources across entry pragmas

Fixes #6651

* fix(reference-source): preserve raw POSIX filename identity

Mark directories with non-UTF-8 names unsupported without following replacement-character paths, while retaining readable sibling observations.

Fixes #6653

* fix(reference-imports): retain exact dynamic import targets

Use the shared exact-literal reader for dynamic imports, preserving expression-free templates and transparent TypeScript wrappers while leaving computed targets unknown.

Fixes #6654

* fix(reference-imports): retain exported import-equals dependencies

Route exported external import-equals declarations through the same relationship owner as plain declarations so their required target is not dropped.

Fixes #6655

* fix(managed): respect native module case semantics

Preserve the observed native format when comparing module identities so ELF case-only differences remain mismatches while unknown platform behavior stays unresolved.

Fixes #6652

* fix(il2cpp): reject duplicate method tokens per image

Reject duplicate MethodDef row identities within one owning image before publishing native addresses or Ghidra seeds, while retaining cross-image reuse and folded code addresses.

Fixes #6610

* refactor(reference-imports): share exact literal parsing

Route legacy nested import-expression handling through the same exact-literal owner as dynamic imports, removing a duplicate string-only check.

Fixes #6654

* fix(model): preserve optional metadata and tensor alignment

Fixes #6656
Fixes #6637
Fixes #6636

* fix(macho): infer legacy Intel simulator platforms

Fixes #6649

* fix(macho)!: preserve ambiguous chained import names

Keep the exact producer expression and report unknown symbol identity when chained-import text cannot distinguish a binding addend from a literal name suffix.

Fixes #6648

BREAKING CHANGE: inspect_macho import names can be null when identity is ambiguous. Use raw_name for the observed expression and name_status to distinguish it from an exact symbol name.

* fix(elf): retain raw export name bytes

Fixes #6647

* refactor(macho): separate dyld import and export parsing

Keep mode-specific inputs and parsed row types at their respective parser boundaries.

* fix(macho): preserve import identity in retained views

Use distinct row contracts for imports and exports, retaining ambiguous names and historical import defaults without adding import-only fields to export pages. Verify the SDK-advertised schemas and both facets through MCP.

* style(desktop): format GameMaker probe coverage

* chore(deps): update Vite security patch and dependency inventory (#6416)

Update Vite/PostCSS and refresh the lock-derived inventory while retaining TSRS platform selection and runtime dependency versions.

Source/static checks, installed-platform signatures, full/production audits, real CLI/MCP workflows and all four package platforms pass. Portable CI retains the independently reproduced main PTX fixture failure tracked in #6675; use the owner-authorized existing PR-only baseline exception. Refs #6480.

* fix(release): isolate npm publication from dependency execution (#6681)

Keep dependency execution in read-only release jobs and validate the original qualified archive with Node built-ins before publication. Refresh registry state on every publisher attempt; retain exact bytes/channel verification, v1 read-only recovery, and installed CLI/MCP checks before MCP publication.

Static/docs checks, artifact/registry/recovery tests and real package/rehearsal verification pass. All four hosted package platforms pass. Portable CI retains the independently reproduced main PTX failure in #6675 and fail-fast missing coverage; use the owner-authorized existing PR-only exception, with limitations documented in the PR. Refs #6480.

* refactor(json): consolidate representation size accounting

Share bounded UTF-8 and UTF-16 JSON measurements across state serving, write retention and source-map admission. Remove duplicate counters and distinguish retained state estimates from encoded bytes while preserving limits and exact refusal diagnostics.

* refactor(android): own search summaries at the producer boundary

Reuse validated class pages for inline pagination audits while retaining complete provider replies. Move retention into the shared Android workflow and remove the MCP-only service wrapper and application text reparser. Preserve unknown JSON metadata, late cancellation and retention failures.

* test: align boundary fixtures with producer contracts

Model GDB console completion, use real directory reads and cleanup for late-enumeration failures, and distinguish supported PTX pointer qualifiers from unknown declarations. Preserve original assertion and cleanup failures through scoped test finalizers.

* fix(rust): retain parsed defaults in cached Evidence views

Borrow sealed reports only when they match the parsed facts; otherwise cache the owned normalized snapshot. Keep historical feature omissions unknown through saved bundles and repeated retained pages without changing the source Evidence.

Fixes #6689

* fix(model): retain parsed headers when hashing is interrupted

Share partial observation ownership across GGUF and Safetensors. Preserve completed header facts through cancellation and MCP error delivery without claiming finalized digests or file identity.

Fixes #6684

* fix(gpu): preserve PTX parameter and target facts

Fixes #6664
Fixes #6675
Fixes #6680
Fixes #6695

* fix(go): read inline build strings from ELF mappings

Fixes #6716

* fix(rust): scan read-only ELF load data

Fixes #6663

* fix(il2cpp): deduplicate consistent ELF registration aliases

Fixes #6715

* fix(il2cpp): track unresolved ELF relocation widths

Fixes #6701

* fix(artifacts): preserve raw directory name identity

Fixes #6665

* fix(desktop): retain unsupported filename coverage

Fixes #6665

* fix: preserve unknown historical module resolution

Fixes #6679

Fixes #6678

Fixes #6612

* fix: retain proven createRequire URL bases

Fixes #6612

Fixes #6678

* fix(apple): preserve keyed archive table identity

Fixes #6662

* fix(plist): reject XML dictionary keys without values

Fixes #6682

* fix(apple): preserve NIBArchive special real values

Fixes #6583

* fix(signals): reject missing targets and invalid UTF-8

Fixes #6671

Fixes #6693

* fix(beam): accept bytes after declared form

Fixes #6717

* fix(artifacts): compare source map target identity

Fixes #6706

* fix(pyinstaller): retain executable member metadata

Fixes #6702

* fix(pyinstaller): decode interned unicode names

Fixes #6676

* fix(pyinstaller): retain colliding archive observations

Keep source-distinct CArchive rows and decoded PYZ children in inventory. Refuse only when extraction reaches a destination conflict.

Fixes #6674

* fix(dotnet): preserve managed metadata comparison facts

Decode VARARG sentinels and allow omitted empty blob heaps while retaining failures for referenced missing data. Compare field and catch targets using owning parsed facts instead of build-local tokens.

Fixes #6720
Fixes #6686
Fixes #6713
Fixes #6712

* fix(dotnet): keep TypeRef catch identity unknown

Fixes #6712

* test(apple): assert projected interface builder facts

* fix(dotnet): correct metadata comparison regressions

* fix(il2cpp): deduplicate aliased registration fields

Fixes #6715

* test(pyinstaller): match interned name producer encoding

Use Python 3.13 PYZ magic with the matching archive version and exercise FLAG_REF with TYPE_INTERNED.

Fixes #6676

* test(dotnet): split managed comparison blocks

* test(apple): isolate NIBArchive real projection group

* fix(parsers): correct Rust import and PTX alignment control

* fix: recognize aliased createRequire bindings

Fixes #6612

Fixes #6678

* test(apple): retain NIBArchive limitation path

* fix(il2cpp): deduplicate aliased metadata registration

Fixes #6715

* test(managed): retain observed target changes across member remaps

* fix: track require.resolve references

Fixes #6678

* test(pyinstaller): encode PYZ list length correctly

Write the marshal list length as its raw little-endian count and surface inventory limitations when completeness regresses.

Fixes #6676

* fix(managed): retain unknown catch identity beside observed name changes

* fix: retain require.resolve source facts

Fixes #6678

* refactor(javascript): name retained require call facts explicitly

* fix: resolve selected require call members

Fixes #6678

* fix(javascript): keep unresolved created-loader bases unknown

* refactor(artifacts): centralize completed file outcomes

Project completed values and cancellation from the descriptor owner once across artifact consumers. Preserve completed void outcomes and format-specific validation and cleanup policies.

Keep later aborts from replacing captured APK and SQLite close failures, and retain the exact APK cancellation reason. Exercise real descriptors, public workflows and cleanup retries.

* fix(javascript): isolate package facts from host prototypes

Build frozen package field snapshots with a null prototype and restrict reference-source main lookup to own properties. Preserve declared null and empty values, index fallback, source identity and unavailable manifest evidence.

Compare inherited main and exports lookup with the native Node resolver, retain package observation coverage and verify CLI/MCP package workflows.

* fix(captures): advertise offline capture inspection without a browser provider (#6725)

* fix(parsers): preserve format boundaries and query occurrence identity

Normalize parsed facts at their owning boundaries across PE exports and certificates, ELF section names, GPU bundles, IL2CPP generic types, dyld slice selection, source maps, environment replay, and JavaScript mapped arguments. Preserve partial facts and explicit unsupported cases; clarify the corresponding documentation.

Validation: 831 affected tests passed with one existing skip across 23 files; build, typecheck/lint, documentation, formatting, and 2133-file test discovery passed. Format controls are synthetic; real Unity, GPU, and ARM-device execution is unverified.

Fixes #6709
Fixes #6688
Fixes #6719
Fixes #6718
Fixes #6704
Fixes #6703
Fixes #6692
Fixes #6723
Fixes #6687
Fixes #6726
Fixes #6696
Fixes #6691
Fixes #6727

* fix(process): require observed environment ownership tokens

Centralize run-token extraction at the environment observation boundary and require an own field. Build decoded OS environment records without host prototypes while retaining source names, empty values and last-assignment precedence.

Reject inherited matching tokens before cleanup signaling, preserve normal owned-group behavior, and verify live Linux process environments alongside native CLI/MCP capture workflows.

* fix(parsers): retain independent facts and boundary identity

Share bounded ELF section-name decoding across native and note readers; retain read-only ranges and coverage reasons when names are unknown. Preserve ELF symbol-version bytes, dyld-bound method uncertainty, partial gRPC-Web trailers, and per-parent model identity checks. Compare GPU signatures as bytes and require a PTX module header. Share selected-path errno classification across EVM and BEAM.

Validation: 397 shared-parser tests passed (one existing skip), 5 MCP schema tests passed, 14 EVM error-mapping tests passed, and 23 protobuf CLI/MCP matrix cases passed across focused runs. Build, typecheck/lint, docs, formatting, and 2134-file discovery passed. Full EVM provider workflow remains locally unverified on Darwin arm64; format controls are synthetic.

Fixes #6738
Fixes #6737
Fixes #6745
Fixes #6740
Fixes #6749
Fixes #6747
Fixes #6739

* fix(setup): preserve source keys across configuration boundaries

* fix(setup): keep the CLI-only skill choice offered under a client ID (#6401)

* fix(setup): keep the CLI-only skill choice offered under a client ID

* test(setup): cover the accept-all CLI-only skill selection

* fix(apple): report a read-denied selected archive as host access denial (#6479)

* fix(apple): report a read-denied selected archive as host access denial

inspect-keyed-archive reported a host read refusal on the caller-selected bundle entry as artifact_operation_failed/execution_failure with reason io, leaving EACCES or EPERM only inside message text. The caller selected that entry, so the refusal is a selection outcome with its own typed classification, system code and read-access recovery.

DirectoryArtifactReader now marks a refused entry open with DirectoryArtifactEntryAccessDenied, which retains the host system code and the selected entry path. KeyedArchiveReader, which owns the caller's selection, maps it to the shared AnalysisAccessDeniedError, and artifact failure translation keeps that typed error instead of replacing it with a generic io failure. Other readers and other reader failures are unchanged; every existing reason keeps its current projection.

Adds the access_denied projection coverage plus a real host refusal through the production reader and a non-permission control.

* fix(apple): carry cleanup ownership and cover raw refusals for a denied selection

Addresses the two review findings on the denied-selection classification:

- The typed host access denial now carries the reader's cleanup observation, so a denied open that left a resource owned does not project as fully cleaned.
- Adds coverage that starts from raw host EACCES/EPERM/EIO refusals with the real DirectoryArtifactReader open path, so the marker itself is exercised rather than constructed by the test. EIO stays an artifact failure.

* fix(native): report plist symlink loops and overlong paths as selection failures (#6371)

Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>

* docs(cli): document opt-in Bash completion setup (#6250)

* fix(doctor): check only the current platform's Hopper launcher candidates (#6253)

* fix(doctor): check only the current platform's Hopper launcher candidates

Doctor built one launcher list for every platform, so on macOS it could
report a Linux-only path as a healthy Hopper ahead of a renamed bundle,
and setup then registered that path. Give per-platform automatic
candidates one owner that runtime configuration and doctor both read,
scan renamed macOS bundles and probe the Homebrew cask only on macOS,
and share runtime's renamed-bundle scan with the doctor host.

Fixes #6228

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(doctor): keep the renamed Hopper bundle scan asynchronous

Sharing runtime's synchronous scan made the doctor host read
~/Applications with readdirSync, which blocks the event loop on a slow
or network-mounted home. Give MacHopperPaths an asynchronous scan for
doctor beside runtime's lazy synchronous one, sharing the directory
filter and candidate builder, and check that both return the same
candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(doctor): model renamed launchers with runtime's POSIX joins

Since #5235, runtime builds macOS launcher paths with posix.join, so on
Windows the discovery table's native-join renamed launcher no longer
matched runtime's candidate. Model launchers with posix.join and keep
filesystem setup native, as MacHopperPaths.test.ts does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* fix(entropy): name offset when the byte window starts past EOF (#6495)

Fixes #6482

Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>

* fix(sqlite): report result-column capacity refusals as capacity (#6327)

A projection wider than SQLite's native result-column limit failed with the
generic "too many columns in result set" SQL error. The inspector could not
attribute that error to a generated column, so the worker reported "format"
and the parent blamed the caller's path as invalid_format even though the
selected database was readable.

Classify the refusal from SQLite's own column limit for the private
connection, carry the selected and engine counts through the worker reply,
and project it as a capacity resource constraint. The database stays
readable, no partial rows are returned, and the recovery names the column
counts and splitting the projection across calls.

* fix(web): refuse an existing script-export destination as invalid input (#6313)

* fix(web): refuse an existing script-export destination as invalid input

export_web_scripts refused an existing output directory or link correctly,
but reported it as artifact_operation_failed / execution_failure with no
field issue or recovery. SafeOutputTree.create raises that refusal as a
path failure before any output tree exists, and the export mapper wrapped
it as an artifact failure.

Attribute pre-admission path refusals to output_directory as an
AnalysisInputError, as environment export already does for its directory.
Both now share one helper owned beside SafeOutputTree; failures after the
tree is admitted keep their artifact semantics.

Fixes #6029

* fix(artifacts): type SafeOutputTree destination refusals

Admission conversion treated any cleanup-free path failure raised before
admission as a refusal of the selected directory, so an acquired-tree setup
failure, such as being unable to restrict a created root to 0700, blamed
valid output_directory or directory input.

SafeOutputTree.create now raises SafeOutputTreeDestinationRefusal for the
refusals of the selection itself: a relative or reserved root, an existing
root, and an accepted existing root that is not an empty directory. The shared
conversion accepts only that type, directly or wrapped by
SafeOutputTreeCreationFailure, so script and environment export no longer
track admission separately.

* fix(evm): reject repeated container keys in Solidity metadata (#6230)

* fix(evm): reject repeated container keys in Solidity metadata

* test(evm): check the repeat-key diagnostic for deeply nested keys

* docs(ci): add a repeatable static analysis example (#6496)

* docs(ci): add a repeatable static analysis example

* fix(ci): isolate example reports and repair the guide link

---------

Co-authored-by: gongfpp <22683417+gongfpp@users.noreply.github.com>

* fix(setup): retain unresolved environment facts when deriving budgets

* fix(parsers): preserve format evidence and caller input identity

Retain bounded FAT and offset-tagged PE observations through archive hashing,
classification, and content nodes. Preserve PyInstaller member paths according
to the observed carrier dialect and share APK v1 signing-name semantics.

Read SQLite's implicit sqlite_schema table without inventing a stored schema
declaration. Distinguish unsupported ELF numbering and decoded-name admission
from malformed data and fixed scan allowances. Preserve Sigrok basenames and
classify matching explicit input-format refusals at their owning boundary.

Validation: 554 distinct tests passed, 3 skipped across serial focused runs;
build, typecheck/lint, docs, formatting, and discovery (2141 files) passed.
Sigrok command-boundary simulators do not establish real-tool decoding.
Small repeated public PyInstaller CLI runs had equivalent inventory facts;
warm median elapsed 0.5431s before / 0.5426s after, with no speed claim.

Fixes #6733
Fixes #6741
Fixes #6722
Fixes #6732
Fixes #6736
Fixes #6734
Fixes #6764
Fixes #6766
Refs #6739

* fix(parsers): respect container facts and real decoding boundaries

Use zip.js's parsed symlink fact for inventory and open refusal. Treat an
unmatched Godot trailer as an unconfirmed candidate while preserving failures
for confirmed malformed packs. Report entropy path-resolution failures at the
selected-input boundary.

Separate Go header discovery from inline string extents in ELF and Mach-O,
and apply the aggregate content allowance after validating real source ranges.
Normalize binary plist UID widths through the existing scalar-copy owner,
budget exact clone growth, preserve source identity, and refuse inexact UIDs.

Validation: 389 distinct tests passed, one Linux /proc test skipped on Darwin;
build, typecheck/lint, docs, single-thread formatting, and discovery (2142 files)
passed. CLI/MCP Go fixtures and public ZIP/Godot/keyed-archive workflows ran.
Real producer prevalence and external provider execution were not established.

Repeated ordinary keyed-archive CLI inventory facts and estimated reply tokens
were equal before/after. Warm median elapsed was 0.5166s / 0.5177s; no speed claim.

Fixes #6751
Fixes #6763
Fixes #6754
Fixes #6767
Fixes #6762
Fixes #6779

* refactor(mcp): consolidate Evidence recording and operation progress

* fix(setup): reuse URL component admission for Hopper releases

* perf(cli): avoid loading analysis commands for version reports

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Hifza Zafar <152292315+hifzabuildsai@users.noreply.github.com>
Co-authored-by: Can <159528514+eco-null@users.noreply.github.com>
Co-authored-by: eco-null <eco-null@users.noreply.github.com>
Co-authored-by: Cassie_Fox <249835913+CassieFox23@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Wade <160517347+Sriram-PR@users.noreply.github.com>
Co-authored-by: RRFZ <247457381+Plastic-time@users.noreply.github.com>
Co-authored-by: pablodiazjorge <102741971+pablodiazjorge@users.noreply.github.com>
Co-authored-by: Ezra Brilliant <64943463+ezrabrilliant@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: GamerDubz <105694977+GamerDubz@users.noreply.github.com>
Co-authored-by: raidnouredden-ux <raidnouredden@gmail.com>
Co-authored-by: 吴佳浩Alben <35291372+XiaoBinGan@users.noreply.github.com>
Co-authored-by: Abel Yagubyan <abelyagubyan@berkeley.edu>
Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
Co-authored-by: ShuaixinHuang <1836656514@qq.com>
Co-authored-by: Paritosh Lahre <155938586+SandstromPL@users.noreply.github.com>
Co-authored-by: Dery Ferdika <114888155+DeryFerd@users.noreply.github.com>
Co-authored-by: gongfpp <1360187968@qq.com>
Co-authored-by: gongfpp <22683417+gongfpp@users.noreply.github.com>
2026-10-24 00:06:03 +08:00
morluto 8bf882f401 test(environment): derive capacity fixtures from current contracts 2026-10-24 00:05:06 +08:00
morluto 9066444f6b fix(source-maps): reuse admitted source URL identities 2026-10-24 00:01:19 +08:00
morluto 7fbc2b5652 refactor(cli): remove redundant runtime provider contexts 2026-10-23 23:54:04 +08:00
morluto ecd997e512 fix(managed): compare catch types by scoped nominal identity
Preserve boundary facts and uncertainty through their consumers. Add focused production-boundary regressions and update affected documentation.

Fixes #6760
2026-10-23 23:49:26 +08:00
morluto c409a8bbe8 fix(managed): preserve native module names under platform loader rules
Preserve boundary facts and uncertainty through their consumers. Add focused production-boundary regressions and update affected documentation.

Fixes #6758
2026-10-23 23:49:26 +08:00
morluto 17cd07342d fix(cli): preserve selected JSON members and Markdown table text
Preserve boundary facts and uncertainty through their consumers. Add focused production-boundary regressions and update affected documentation.

Fixes #6756
Fixes #6752
Fixes #6755
2026-10-23 23:49:26 +08:00
morluto 52b0a54afd fix(evm): reject nested duplicate CBOR map keys
Preserve boundary facts and uncertainty through their consumers. Add focused production-boundary regressions and update affected documentation.

Fixes #6780
2026-10-23 23:49:26 +08:00
morluto 7fd722e15b fix(native): retain Objective-C implementation alternatives 2026-10-23 23:42:39 +08:00
morluto 567b559474 fix(call-path): retain unresolved outgoing-call caveats
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6788
2026-10-23 23:36:21 +08:00
morluto 607f509505 fix(flutter): ignore unused ELF DT_NULL values
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6768
2026-10-23 23:36:21 +08:00
morluto 40c235c642 fix(comparison): preserve decoding uncertainty and known string facts
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6774
2026-10-23 23:36:21 +08:00
morluto a06dbd509d fix(browser): retain unknown resource sizes and accessibility roles
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6773
Fixes #6786
2026-10-23 23:36:21 +08:00
morluto 3d24683b06 fix(search): share Unicode literal matching and preserve field boundaries
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6785
Fixes #6714
Fixes #6735
Fixes #6750
2026-10-23 23:36:21 +08:00
morluto b3995a2ae5 fix(apple): preserve archived className fields and descriptor identity
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6776
2026-10-23 23:36:21 +08:00
morluto 5af154fc4b fix(gpu): accept numeric percent-prefixed PTX identifiers
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6771
2026-10-23 23:36:21 +08:00
morluto 7f613149be fix(model): accept released GGUF writer metadata and vocab-only output
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6772
Fixes #6787
2026-10-23 23:36:21 +08:00
morluto 50af8de5e8 fix(reconstruction): validate all retained case Evidence references
Preserve format and Evidence semantics at their owning boundaries, with focused regression coverage and documentation where applicable.

Fixes #6770
2026-10-23 23:36:21 +08:00
morluto e2efd10b70 fix(comparison): reuse parsed native address coordinates 2026-10-23 23:26:29 +08:00
morluto 6a34be416b refactor(adapters): remove redundant error and provider facades 2026-10-23 23:15:46 +08:00
morluto b33f68deb7 refactor(providers): separate identities from workflow profiles 2026-10-23 23:14:14 +08:00
morluto 60accad2de fix(setup): reuse URL component admission for Hopper releases 2026-10-23 23:04:32 +08:00
morluto c343b787f0 refactor(mcp): consolidate Evidence recording and operation progress 2026-10-23 23:04:32 +08:00
morluto 6ba6aea5eb fix(parsers): respect container facts and real decoding boundaries
Use zip.js's parsed symlink fact for inventory and open refusal. Treat an
unmatched Godot trailer as an unconfirmed candidate while preserving failures
for confirmed malformed packs. Report entropy path-resolution failures at the
selected-input boundary.

Separate Go header discovery from inline string extents in ELF and Mach-O,
and apply the aggregate content allowance after validating real source ranges.
Normalize binary plist UID widths through the existing scalar-copy owner,
budget exact clone growth, preserve source identity, and refuse inexact UIDs.

Validation: 389 distinct tests passed, one Linux /proc test skipped on Darwin;
build, typecheck/lint, docs, single-thread formatting, and discovery (2142 files)
passed. CLI/MCP Go fixtures and public ZIP/Godot/keyed-archive workflows ran.
Real producer prevalence and external provider execution were not established.

Repeated ordinary keyed-archive CLI inventory facts and estimated reply tokens
were equal before/after. Warm median elapsed was 0.5166s / 0.5177s; no speed claim.

Fixes #6751
Fixes #6763
Fixes #6754
Fixes #6767
Fixes #6762
Fixes #6779
2026-10-23 23:02:07 +08:00
morluto bd4f5e9c19 fix(parsers): preserve format evidence and caller input identity
Retain bounded FAT and offset-tagged PE observations through archive hashing,
classification, and content nodes. Preserve PyInstaller member paths according
to the observed carrier dialect and share APK v1 signing-name semantics.

Read SQLite's implicit sqlite_schema table without inventing a stored schema
declaration. Distinguish unsupported ELF numbering and decoded-name admission
from malformed data and fixed scan allowances. Preserve Sigrok basenames and
classify matching explicit input-format refusals at their owning boundary.

Validation: 554 distinct tests passed, 3 skipped across serial focused runs;
build, typecheck/lint, docs, formatting, and discovery (2141 files) passed.
Sigrok command-boundary simulators do not establish real-tool decoding.
Small repeated public PyInstaller CLI runs had equivalent inventory facts;
warm median elapsed 0.5431s before / 0.5426s after, with no speed claim.

Fixes #6733
Fixes #6741
Fixes #6722
Fixes #6732
Fixes #6736
Fixes #6734
Fixes #6764
Fixes #6766
Refs #6739
2026-10-23 22:54:28 +08:00
morluto a718037033 fix(setup): retain unresolved environment facts when deriving budgets 2026-10-23 22:45:41 +08:00
gongfppandgongfpp 1b3d57ca8a docs(ci): add a repeatable static analysis example (#6496)
* docs(ci): add a repeatable static analysis example

* fix(ci): isolate example reports and repair the guide link

---------

Co-authored-by: gongfpp <22683417+gongfpp@users.noreply.github.com>
2026-10-23 23:42:58 +09:00
Wade fd782cf2b1 fix(evm): reject repeated container keys in Solidity metadata (#6230)
* fix(evm): reject repeated container keys in Solidity metadata

* test(evm): check the repeat-key diagnostic for deeply nested keys
2026-10-23 23:42:45 +09:00
Muhammad Hamza b5dba08dba fix(web): refuse an existing script-export destination as invalid input (#6313)
* fix(web): refuse an existing script-export destination as invalid input

export_web_scripts refused an existing output directory or link correctly,
but reported it as artifact_operation_failed / execution_failure with no
field issue or recovery. SafeOutputTree.create raises that refusal as a
path failure before any output tree exists, and the export mapper wrapped
it as an artifact failure.

Attribute pre-admission path refusals to output_directory as an
AnalysisInputError, as environment export already does for its directory.
Both now share one helper owned beside SafeOutputTree; failures after the
tree is admitted keep their artifact semantics.

Fixes #6029

* fix(artifacts): type SafeOutputTree destination refusals

Admission conversion treated any cleanup-free path failure raised before
admission as a refusal of the selected directory, so an acquired-tree setup
failure, such as being unable to restrict a created root to 0700, blamed
valid output_directory or directory input.

SafeOutputTree.create now raises SafeOutputTreeDestinationRefusal for the
refusals of the selection itself: a relative or reserved root, an existing
root, and an accepted existing root that is not an empty directory. The shared
conversion accepts only that type, directly or wrapped by
SafeOutputTreeCreationFailure, so script and environment export no longer
track admission separately.
2026-10-23 23:42:33 +09:00
Dery Ferdika 89fea8130a fix(sqlite): report result-column capacity refusals as capacity (#6327)
A projection wider than SQLite's native result-column limit failed with the
generic "too many columns in result set" SQL error. The inspector could not
attribute that error to a generated column, so the worker reported "format"
and the parent blamed the caller's path as invalid_format even though the
selected database was readable.

Classify the refusal from SQLite's own column limit for the private
connection, carry the selected and engine counts through the worker reply,
and project it as a capacity resource constraint. The database stays
readable, no partial rows are returned, and the recovery names the column
counts and splitting the projection across calls.
2026-10-23 23:42:20 +09:00
Abel YagubyanandAbelo9996 e8dad898da fix(entropy): name offset when the byte window starts past EOF (#6495)
Fixes #6482

Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
2026-10-23 23:42:07 +09:00
Paritosh LahreandClaude Opus 5.5 8f91e60558 fix(doctor): check only the current platform's Hopper launcher candidates (#6253)
* fix(doctor): check only the current platform's Hopper launcher candidates

Doctor built one launcher list for every platform, so on macOS it could
report a Linux-only path as a healthy Hopper ahead of a renamed bundle,
and setup then registered that path. Give per-platform automatic
candidates one owner that runtime configuration and doctor both read,
scan renamed macOS bundles and probe the Homebrew cask only on macOS,
and share runtime's renamed-bundle scan with the doctor host.

Fixes #6228

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(doctor): keep the renamed Hopper bundle scan asynchronous

Sharing runtime's synchronous scan made the doctor host read
~/Applications with readdirSync, which blocks the event loop on a slow
or network-mounted home. Give MacHopperPaths an asynchronous scan for
doctor beside runtime's lazy synchronous one, sharing the directory
filter and candidate builder, and check that both return the same
candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(doctor): model renamed launchers with runtime's POSIX joins

Since #5235, runtime builds macOS launcher paths with posix.join, so on
Windows the discovery table's native-join renamed launcher no longer
matched runtime's candidate. Model launchers with posix.join and keep
filesystem setup native, as MacHopperPaths.test.ts does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:41:54 +09:00
ShuaixinHuang c9204b96d4 docs(cli): document opt-in Bash completion setup (#6250) 2026-10-23 23:41:42 +09:00
Abel YagubyanandAbelo9996 f290b27299 fix(native): report plist symlink loops and overlong paths as selection failures (#6371)
Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
2026-10-23 23:41:29 +09:00
吴佳浩Alben eab61d02a1 fix(apple): report a read-denied selected archive as host access denial (#6479)
* fix(apple): report a read-denied selected archive as host access denial

inspect-keyed-archive reported a host read refusal on the caller-selected bundle entry as artifact_operation_failed/execution_failure with reason io, leaving EACCES or EPERM only inside message text. The caller selected that entry, so the refusal is a selection outcome with its own typed classification, system code and read-access recovery.

DirectoryArtifactReader now marks a refused entry open with DirectoryArtifactEntryAccessDenied, which retains the host system code and the selected entry path. KeyedArchiveReader, which owns the caller's selection, maps it to the shared AnalysisAccessDeniedError, and artifact failure translation keeps that typed error instead of replacing it with a generic io failure. Other readers and other reader failures are unchanged; every existing reason keeps its current projection.

Adds the access_denied projection coverage plus a real host refusal through the production reader and a non-permission control.

* fix(apple): carry cleanup ownership and cover raw refusals for a denied selection

Addresses the two review findings on the denied-selection classification:

- The typed host access denial now carries the reader's cleanup observation, so a denied open that left a resource owned does not project as fully cleaned.
- Adds coverage that starts from raw host EACCES/EPERM/EIO refusals with the real DirectoryArtifactReader open path, so the marker itself is exercised rather than constructed by the test. EIO stays an artifact failure.
2026-10-23 23:41:16 +09:00
pablodiazjorge 6c1b7f86d1 fix(setup): keep the CLI-only skill choice offered under a client ID (#6401)
* fix(setup): keep the CLI-only skill choice offered under a client ID

* test(setup): cover the accept-all CLI-only skill selection
2026-10-23 23:41:04 +09:00
morluto 98ca6c8270 fix(setup): preserve source keys across configuration boundaries 2026-10-23 22:32:06 +08:00
morluto 99d2d7ad22 fix(parsers): retain independent facts and boundary identity
Share bounded ELF section-name decoding across native and note readers; retain read-only ranges and coverage reasons when names are unknown. Preserve ELF symbol-version bytes, dyld-bound method uncertainty, partial gRPC-Web trailers, and per-parent model identity checks. Compare GPU signatures as bytes and require a PTX module header. Share selected-path errno classification across EVM and BEAM.

Validation: 397 shared-parser tests passed (one existing skip), 5 MCP schema tests passed, 14 EVM error-mapping tests passed, and 23 protobuf CLI/MCP matrix cases passed across focused runs. Build, typecheck/lint, docs, formatting, and 2134-file discovery passed. Full EVM provider workflow remains locally unverified on Darwin arm64; format controls are synthetic.

Fixes #6738
Fixes #6737
Fixes #6745
Fixes #6740
Fixes #6749
Fixes #6747
Fixes #6739
2026-10-23 22:18:43 +08:00
morluto a68cfd7db0 fix(process): require observed environment ownership tokens
Centralize run-token extraction at the environment observation boundary and require an own field. Build decoded OS environment records without host prototypes while retaining source names, empty values and last-assignment precedence.

Reject inherited matching tokens before cleanup signaling, preserve normal owned-group behavior, and verify live Linux process environments alongside native CLI/MCP capture workflows.
2026-10-23 22:13:12 +08:00
morluto 5f702dd6cc fix(parsers): preserve format boundaries and query occurrence identity
Normalize parsed facts at their owning boundaries across PE exports and certificates, ELF section names, GPU bundles, IL2CPP generic types, dyld slice selection, source maps, environment replay, and JavaScript mapped arguments. Preserve partial facts and explicit unsupported cases; clarify the corresponding documentation.

Validation: 831 affected tests passed with one existing skip across 23 files; build, typecheck/lint, documentation, formatting, and 2133-file test discovery passed. Format controls are synthetic; real Unity, GPU, and ARM-device execution is unverified.

Fixes #6709
Fixes #6688
Fixes #6719
Fixes #6718
Fixes #6704
Fixes #6703
Fixes #6692
Fixes #6723
Fixes #6687
Fixes #6726
Fixes #6696
Fixes #6691
Fixes #6727
2026-10-23 22:01:16 +08:00
Wade 4193bc5bf8 fix(captures): advertise offline capture inspection without a browser provider (#6725) 2026-10-23 22:00:42 +08:00
morluto 610966b836 fix(javascript): isolate package facts from host prototypes
Build frozen package field snapshots with a null prototype and restrict reference-source main lookup to own properties. Preserve declared null and empty values, index fallback, source identity and unavailable manifest evidence.

Compare inherited main and exports lookup with the native Node resolver, retain package observation coverage and verify CLI/MCP package workflows.
2026-10-23 21:57:14 +08:00
morluto 2bcd7e6d13 refactor(artifacts): centralize completed file outcomes
Project completed values and cancellation from the descriptor owner once across artifact consumers. Preserve completed void outcomes and format-specific validation and cleanup policies.

Keep later aborts from replacing captured APK and SQLite close failures, and retain the exact APK cancellation reason. Exercise real descriptors, public workflows and cleanup retries.
2026-10-23 21:42:40 +08:00
morluto 7fd6e486ae fix(javascript): keep unresolved created-loader bases unknown 2026-10-23 21:36:43 +08:00
morluto 9b598076e3 fix: resolve selected require call members
Fixes #6678
2026-10-23 21:35:30 +08:00
morluto e966b9f9a5 refactor(javascript): name retained require call facts explicitly 2026-10-23 21:34:45 +08:00
morluto 1a430f055d fix: retain require.resolve source facts
Fixes #6678
2026-10-23 21:29:20 +08:00
morluto 529038f557 fix(managed): retain unknown catch identity beside observed name changes 2026-10-23 21:28:11 +08:00
morluto 66e25e7912 test(pyinstaller): encode PYZ list length correctly
Write the marshal list length as its raw little-endian count and surface inventory limitations when completeness regresses.

Fixes #6676
2026-10-23 21:28:11 +08:00
morluto ae013ad23c fix: track require.resolve references
Fixes #6678
2026-10-23 21:28:11 +08:00
morluto 0f04043a5a test(managed): retain observed target changes across member remaps 2026-10-23 21:28:11 +08:00
morluto 8cd10d4727 fix(il2cpp): deduplicate aliased metadata registration
Fixes #6715
2026-10-23 21:28:11 +08:00
morluto 7a93cfd118 test(apple): retain NIBArchive limitation path 2026-10-23 21:28:11 +08:00
morluto 6cb0684372 fix: recognize aliased createRequire bindings
Fixes #6612

Fixes #6678
2026-10-23 21:28:11 +08:00
morluto f09dde0cc9 fix(parsers): correct Rust import and PTX alignment control 2026-10-23 21:28:11 +08:00
morluto fabb68bb9b test(apple): isolate NIBArchive real projection group 2026-10-23 21:28:11 +08:00
morluto 97f6678578 test(dotnet): split managed comparison blocks 2026-10-23 21:28:10 +08:00
morluto 72270ac275 test(pyinstaller): match interned name producer encoding
Use Python 3.13 PYZ magic with the matching archive version and exercise FLAG_REF with TYPE_INTERNED.

Fixes #6676
2026-10-23 21:28:10 +08:00
morluto f40bd0c4fb fix(il2cpp): deduplicate aliased registration fields
Fixes #6715
2026-10-23 21:28:10 +08:00
morluto 4f0c33e8c4 fix(dotnet): correct metadata comparison regressions 2026-10-23 21:28:10 +08:00
morluto 5e16e30c6b test(apple): assert projected interface builder facts 2026-10-23 21:28:10 +08:00
morluto b305104e4a fix(dotnet): keep TypeRef catch identity unknown
Fixes #6712
2026-10-23 21:28:10 +08:00
morluto a8b8ff2acf fix(dotnet): preserve managed metadata comparison facts
Decode VARARG sentinels and allow omitted empty blob heaps while retaining failures for referenced missing data. Compare field and catch targets using owning parsed facts instead of build-local tokens.

Fixes #6720
Fixes #6686
Fixes #6713
Fixes #6712
2026-10-23 21:28:10 +08:00
morluto 5bcf71c660 fix(pyinstaller): retain colliding archive observations
Keep source-distinct CArchive rows and decoded PYZ children in inventory. Refuse only when extraction reaches a destination conflict.

Fixes #6674
2026-10-23 21:28:10 +08:00
morluto c4a59101ac fix(pyinstaller): decode interned unicode names
Fixes #6676
2026-10-23 21:28:10 +08:00
morluto 045c7dd63a fix(pyinstaller): retain executable member metadata
Fixes #6702
2026-10-23 21:28:10 +08:00
morluto 131c1f1219 fix(artifacts): compare source map target identity
Fixes #6706
2026-10-23 21:28:10 +08:00
morluto 8520d425d0 fix(beam): accept bytes after declared form
Fixes #6717
2026-10-23 21:28:10 +08:00
morluto f6230e2f9d fix(signals): reject missing targets and invalid UTF-8
Fixes #6671

Fixes #6693
2026-10-23 21:28:10 +08:00
morluto 43e7917ec2 fix(apple): preserve NIBArchive special real values
Fixes #6583
2026-10-23 21:28:10 +08:00
morluto 869d2bc2ec fix(plist): reject XML dictionary keys without values
Fixes #6682
2026-10-23 21:28:10 +08:00
morluto 78ce91f1c2 fix(apple): preserve keyed archive table identity
Fixes #6662
2026-10-23 21:28:10 +08:00
morluto 4bad48d873 fix: retain proven createRequire URL bases
Fixes #6612

Fixes #6678
2026-10-23 21:28:10 +08:00
morluto 857a75baac fix: preserve unknown historical module resolution
Fixes #6679

Fixes #6678

Fixes #6612
2026-10-23 21:28:10 +08:00
morluto 992e930b86 fix(desktop): retain unsupported filename coverage
Fixes #6665
2026-10-23 21:28:10 +08:00
morluto 47900f99c5 fix(artifacts): preserve raw directory name identity
Fixes #6665
2026-10-23 21:28:10 +08:00
morluto bdf13a63c4 fix(il2cpp): track unresolved ELF relocation widths
Fixes #6701
2026-10-23 21:28:10 +08:00
morluto d31e42699f fix(il2cpp): deduplicate consistent ELF registration aliases
Fixes #6715
2026-10-23 21:28:10 +08:00
morluto d2739960af fix(rust): scan read-only ELF load data
Fixes #6663
2026-10-23 21:28:10 +08:00
morluto 386359060b fix(go): read inline build strings from ELF mappings
Fixes #6716
2026-10-23 21:28:10 +08:00
morluto aa52ff6f4b fix(gpu): preserve PTX parameter and target facts
Fixes #6664
Fixes #6675
Fixes #6680
Fixes #6695
2026-10-23 21:28:10 +08:00
morluto eed99a588e fix(model): retain parsed headers when hashing is interrupted
Share partial observation ownership across GGUF and Safetensors. Preserve completed header facts through cancellation and MCP error delivery without claiming finalized digests or file identity.

Fixes #6684
2026-10-23 21:28:10 +08:00
morluto 02940796f1 fix(rust): retain parsed defaults in cached Evidence views
Borrow sealed reports only when they match the parsed facts; otherwise cache the owned normalized snapshot. Keep historical feature omissions unknown through saved bundles and repeated retained pages without changing the source Evidence.

Fixes #6689
2026-10-23 21:28:10 +08:00
725 changed files with 38755 additions and 6360 deletions

No files matched your search

+2
View File
@@ -72,6 +72,8 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
+107 -11
View File
@@ -10,6 +10,7 @@ import jadx.api.plugins.input.insns.Opcode;
import jadx.core.dex.info.MethodInfo;
import jadx.core.dex.nodes.*;
import java.io.PrintStream;
import java.io.StringReader;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.concurrent.CountDownLatch;
@@ -23,6 +24,10 @@ import kotlin.coroutines.intrinsics.IntrinsicsKt;
import kotlinx.coroutines.BuildersKt;
import kotlinx.io.CoreKt;
import kotlinx.io.JvmCoreKt;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.xml.sax.InputSource;
/** REA-owned metadata boundary for the unmodified jadx-headless-mcp 0.7.1 JAR. */
class ReaJadxBridge {
@@ -30,10 +35,13 @@ class ReaJadxBridge {
private final SessionHolder holder;
private JadxSession indexedSession;
private Map<String, List<ClassSnapshot>> classes = Map.of();
private JadxSession manifestSession;
private ManifestSnapshot manifestSnapshot;
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
Map<String, Object> summary) {}
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
private record ManifestSnapshot(String text, Map<String, Object> summary) {}
private static final class InputFailure extends IllegalArgumentException {
final String field;
@@ -93,6 +101,7 @@ class ReaJadxBridge {
args.has("threads") ? integer(args, "threads", 1) : null,
args.has("code_cache_size") ? integer(args, "code_cache_size", 64) : null,
optionalString(args, "resources"));
clearManifest();
return loadCall(next -> holder.load(path, options, next), continuation);
}
@@ -153,6 +162,77 @@ class ReaJadxBridge {
return current;
}
private void clearManifest() {
manifestSession = null;
manifestSnapshot = null;
}
/** Original archive identity, never a deobfuscated basename or name suffix. */
private ManifestSnapshot manifest(JadxSession current) {
if (manifestSession != current || manifestSnapshot == null) {
ResourceFile selected = null;
for (ResourceFile resource : current.getResources()) {
if (resource.getType() != ResourceType.MANIFEST
|| !resource.getOriginalName().equals("AndroidManifest.xml")) continue;
if (selected != null)
throw new IllegalArgumentException("Root AndroidManifest.xml resource identity is ambiguous");
selected = resource;
}
if (selected == null) throw new IllegalArgumentException("Root AndroidManifest.xml not found");
var content = selected.loadContent();
if (content == null || content.getDataType() != jadx.core.xmlgen.ResContainer.DataType.TEXT)
throw new IllegalArgumentException("Root AndroidManifest.xml did not decode as text");
String xml = content.getText().getCodeStr();
ManifestSnapshot snapshot = new ManifestSnapshot(xml, manifestSummary(xml));
manifestSnapshot = snapshot;
manifestSession = current;
}
return manifestSnapshot;
}
/** One parsed XML owner supplies namespace-aware values alongside the exact decoded text. */
static Map<String, Object> manifestSummary(String xml) {
Element root;
try {
// Match REA's existing JDK XML setup: document types are unsupported.
// Parse an in-memory string; XInclude and external DTD/schema I/O stay disabled.
DocumentBuilderFactory factory = DocumentBuilderFactory.newDefaultNSInstance();
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
factory.setXIncludeAware(false);
factory.setAttribute(javax.xml.XMLConstants.ACCESS_EXTERNAL_DTD, "");
factory.setAttribute(javax.xml.XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");
root = factory.newDocumentBuilder().parse(new InputSource(new StringReader(xml)))
.getDocumentElement();
} catch (javax.xml.parsers.ParserConfigurationException | org.xml.sax.SAXException
| java.io.IOException failure) {
throw new IllegalArgumentException("Root AndroidManifest.xml is not supported well-formed XML", failure);
}
if (!root.getTagName().equals("manifest") || root.getNamespaceURI() != null)
throw new IllegalArgumentException("AndroidManifest.xml root must be an unnamespaced <manifest>");
String android = "http://schemas.android.com/apk/res/android";
String minSdk = "";
String targetSdk = "";
List<String> permissions = new ArrayList<>();
for (Node child = root.getFirstChild(); child != null; child = child.getNextSibling()) {
if (!(child instanceof Element element) || element.getNamespaceURI() != null) continue;
if (element.getTagName().equals("uses-sdk")) {
minSdk = element.getAttributeNS(android, "minSdkVersion");
targetSdk = element.getAttributeNS(android, "targetSdkVersion");
} else if (element.getTagName().equals("uses-permission")) {
String permission = element.getAttributeNS(android, "name");
if (!permission.isEmpty()) permissions.add(permission);
}
}
Map<String, Object> summary = new LinkedHashMap<>();
summary.put("package", root.getAttribute("package"));
summary.put("version_name", root.getAttributeNS(android, "versionName"));
summary.put("version_code", root.getAttributeNS(android, "versionCode"));
summary.put("min_sdk", minSdk);
summary.put("target_sdk", targetSdk);
summary.put("permissions", List.copyOf(permissions));
return Collections.unmodifiableMap(summary);
}
private ClassSnapshot selected(JsonObject args) {
JadxSession current = session();
if (indexedSession != current) {
@@ -333,6 +413,9 @@ class ReaJadxBridge {
"java_version", System.getProperty("java.version"),
"metadata_scope", "parsed_members"));
JadxSession current = session();
if (name.equals("get_app_info")) return json(manifest(current).summary());
if (name.equals("get_android_manifest"))
return text(TextUtilKt.truncateToBytes(manifest(current).text(), current.getMaxSourceBytes()), false);
if (name.equals("list_classes")) {
List<String> names = current.getClassFqns();
String query = args.has("query") ? string(args, "query") : "";
@@ -420,6 +503,17 @@ class ReaJadxBridge {
return json(result);
}
private static void registerBridgeTool(Server server, ReaJadxBridge bridge, String name,
ToolSchema schema, String description) {
Tool tool = new Tool(name, schema, description, null, null, null, null, null, null);
server.addTool(tool, (connection, request, continuation) -> {
try { return bridge.handle(name, request.getArguments()); }
catch (InputFailure error) { return text(JSON.toJson(Map.of(
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
});
}
public static void main(String[] args) throws Exception {
PrintStream protocolOut = System.out;
System.setOut(System.err);
@@ -428,10 +522,9 @@ class ReaJadxBridge {
SessionHolder holder = new SessionHolder(defaults);
ServerCapabilities capabilities = new ServerCapabilities(
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
Server server = new Server(new Implementation("rea-jadx-bridge", "4", null, null, null),
Server server = new Server(new Implementation("rea-jadx-bridge", "5", null, null, null),
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
SessionToolsKt.registerSessionTools(server, holder);
ManifestToolsKt.registerManifestTools(server, holder);
ClassToolsKt.registerClassTools(server, holder);
MethodToolsKt.registerMethodTools(server, holder);
XrefToolsKt.registerXrefTools(server, holder);
@@ -440,7 +533,7 @@ class ReaJadxBridge {
server.removeTool("load_apk");
server.addTool(loadTool, (connection, request, continuation) ->
bridge.load(request.getArguments(), defaults, continuation));
Set<String> consumed = Set.of("load_apk", "get_app_info", "get_android_manifest",
Set<String> consumed = Set.of("load_apk",
"list_classes", "get_class_summary", "get_method_body",
"get_xrefs_to_class", "get_xrefs_to_method");
for (String name : new ArrayList<>(server.getTools().keySet()))
@@ -449,6 +542,13 @@ class ReaJadxBridge {
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
for (String name : List.of("get_app_info", "get_android_manifest")) {
String description = name.equals("get_app_info")
? "Namespace-aware summary of the exact root AndroidManifest.xml resource."
: "Exact decoded root AndroidManifest.xml text, bounded by the session source byte limit.";
registerBridgeTool(server, bridge, name,
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null), description);
}
for (String name : List.of("list_classes", "get_class_summary",
"get_method_body", "get_xrefs_to_class", "get_xrefs_to_method")) {
Tool tool = server.getTools().get(name).getTool();
@@ -470,14 +570,9 @@ class ReaJadxBridge {
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
};
Tool narrowed = new Tool(name, new ToolSchema(null, new kotlinx.serialization.json.JsonObject(properties),
tool.getInputSchema().getRequired(), null), description, null, null, null, null, null, null);
server.addTool(narrowed, (connection, request, continuation) -> {
try { return bridge.handle(name, request.getArguments()); }
catch (InputFailure error) { return text(JSON.toJson(Map.of(
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
});
registerBridgeTool(server, bridge, name,
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(properties),
tool.getInputSchema().getRequired(), null), description);
}
CountDownLatch done = new CountDownLatch(1);
server.onClose(() -> { done.countDown(); return Unit.INSTANCE; });
@@ -503,6 +598,7 @@ class ReaJadxBridge {
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
(scope, continuation) -> server.close(continuation));
} finally {
bridge.clearManifest();
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
(scope, continuation) -> holder.unload(continuation));
}
+27 -7
View File
@@ -165,7 +165,10 @@ Each such MCP tool call selects one of the records its manifest tool cites and
returns that record's recorded response. Selection compares the supplied path
parameters against the record's path segments at the parameter indexes, then
the supplied query values against the record's query string, and keeps the
record with the most matches; ties keep manifest order. The result reports
record with the most matches; ties keep manifest order. A query argument accepts
one string or an array of strings for repeated values; the array is matched
against every occurrence in captured order. The raw recorded query remains the
replay evidence. The result reports
where the answer came from:
```json
@@ -187,9 +190,14 @@ where the answer came from:
}
```
`exact_match` is true when every supplied path parameter and query value
equals the recorded one; it says nothing about parameters the caller omitted.
Recorded query parsing runs only when supplied query arguments affect selection.
`exact_match` is true when every supplied path parameter and query occurrence
matches the recorded one; it says nothing about parameters the caller omitted.
Comparison probes retain the raw query spelling and its decoded parameter pairs
in request order, including the interleaving of repeated values across names.
Standalone tool calls express repeated values as per-name arrays; when matching
records differ only in cross-name order, `exact_match` is false because those arguments cannot
identify that order. Recorded query parsing runs only when supplied query
arguments affect selection.
A path value matches a recorded segment as written or after percent-decoding;
when both kinds of match are available, the record whose segment equals the
value as written is chosen. `recorded_query` keeps the `?` of an explicit empty
@@ -207,7 +215,11 @@ text with `truncated: true`. `body_recorded` is false when the capture kept the
response's status and content type but no body; `body` is then `null`, which
is not a recorded JSON `null`. `body_encoding` is `base64` when the capture
retained response bytes that are not valid UTF-8; `body` is then exactly those
bytes in base64, never a lossy decoding. A retained base64 body is decoded only as far as the 256 KiB a reply can carry, and a leading byte-order mark is kept. For a `base64` body the limit applies to the recorded bytes, not their longer base64 spelling, and a cut keeps whole base64 groups. Request bodies passed to `POST`, `PUT` and `PATCH`
bytes in base64, never a lossy decoding. An unsupported declared HAR encoding
returns `body: null`, `body_recorded: true` and `body_encoding: "unsupported"`;
the original encoding and encoded text remain in the cited capture Evidence.
Such bodies supply no decoded JSON facts to state extraction.
A retained base64 body is decoded only as far as the 256 KiB a reply can carry, and a leading byte-order mark is kept. For a `base64` body the limit applies to the recorded bytes, not their longer base64 spelling, and a cut keeps whole base64 groups. Request bodies passed to `POST`, `PUT` and `PATCH`
tools are accepted and ignored: nothing is written, and no call changes what a
later call returns. Arguments the tool's input schema does not name are
refused by the MCP server as invalid arguments, so a misspelled parameter never
@@ -365,7 +377,8 @@ shape and carries no annotations.
Path parameters are required strings whose
description names the segment index, the templating rule and the observed
values (sampled within the list budget above; the manifest keeps them all);
query parameter names are optional strings; write methods accept an
query parameter names accept an optional string or array of strings (for
repeated values in captured order); write methods accept an
optional `body` object described by the recorded top-level key types. Each
input keeps its own argument: a query name equal to a path parameter's name is
advertised as `query_<name>`, a query name longer than 64 characters as
@@ -429,6 +442,10 @@ Each read tool answers according to its view in the state:
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
| `recorded` | The recording, as replay answers it. |
A read with repeated query values uses replay, because the state query model
does not infer repeated-parameter semantics. A repeated-query write is refused
before it can change state for the same reason.
An item or list view that always spells a clock field otherwise than the
entity's stored value names that spelling in `clock_formats`, and a value it
never recorded, such as a created entity's clock, is rendered in it: a list that
@@ -573,7 +590,10 @@ id are unique; `view_fields` is optional and names item or list views of its
type. Views render their values over the entity's fields, so a view's values
must not set the type's identity field to a value whose text differs from the
entity's `id`. Spell a collection as requests spell it, as `data/get` reports it: list
calls find entities under the checked line's recorded path. Each document
calls find entities under the checked line's recorded path. State collection
identity preserves repeated separators inside captured paths: `/api/books`
and `/api//books` are distinct. A single trailing slash continues to name the
same collection. Each document
names a `document` view once, at its initial document's `path`. The `version`,
and the `state_evidence_id` and `types` that `data/get` reports, must match
the served state, so a `data/get` result loads back unchanged. A number without
+12 -2
View File
@@ -171,8 +171,14 @@ the same source byte limit and explicit truncation reporting.
## Interpret the result
- Package inspection retains decoded manifest XML, summary declarations,
permissions and class/resource counts. Summary fields are extracted by the
upstream engine and should be checked against the retained XML when needed.
permissions and class/resource counts. REA's bridge selects the exact root
`AndroidManifest.xml` resource by original archive name and manifest type,
independently of asset order. One namespace-aware JDK XML parse supplies
summary attributes and direct permission/SDK declarations; namespace prefixes
and XML entity spelling do not change their values. Raw decoded XML is retained
exactly and only its displayed text is subject to the existing source byte cap.
Summaries use the complete selected XML before that cap. Document types are
unsupported; parsing does not load external DTDs, schemas, or XInclude files.
Signature verification is explicitly `not_performed`.
- Class search filters the engine's cached top-level names before transferring
pages, then follows contiguous inventory positions without rescanning earlier
@@ -256,6 +262,10 @@ such as
`libreactnativejni.so`, `libreactnative.so`, `libhermes.so`,
`libhermes_executor.so`, `libflutter.so`, and `libunity.so`; unrelated names
such as `libcommunity.so` and `libreactivity.so` do not identify those families.
The signing list retains the `META-INF/MANIFEST.MF`, `.SF`, `.RSA`, `.DSA`, and
`.EC` paths selected by Android's v1 verifier, including nested paths. These are
inventory metadata candidates, not proof that a v1 signature verifies or that
the APK is signed; use `inspect-apk-signature` for signature analysis.
The bridge basis is inferred from the native path and repeated for each managed
component, so it does not identify a specific JNI binding. Bridge hypotheses form a DEX/JVM-class by native-library
Cartesian product. The projection computes the product and exact serialized
+8
View File
@@ -96,6 +96,14 @@ still suppresses all string projection, including in this mode.
- **Metadata comes from apktool.yml as apktool printed it.** Version codes
keep their own spelling (`'7'` versus `7`); `package_name` is read from
the decoded manifest, not inferred.
Apktool 3's emitted `version: 3.x.y` header selects its single-line string
dialect: literal hashes, single quotes, empty strings, and the text `null`
stay part of the value; producer backslashes and Unicode escapes decode
once. Older/tagged documents retain YAML scalar semantics; double-quoted
values also retain YAML string semantics.
This distinction follows the released
[3.0.3 metadata writer](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.apktool/apktool-lib/src/main/java/brut/androlib/meta/VersionInfo.java#L49-L55)
and [string escape owner](https://github.com/iBotPeaches/Apktool/blob/18b5e99cb56ff9451e8aa55b065dcf5bbd616975/brut.j.yaml/src/main/java/brut/yaml/YamlStringEscapeUtils.java#L55-L116).
- **The manifest is the decoded text**, bounded at 512 KiB and reported as
truncated when larger.
- **Strings are projected verbatim.** Escaped entities stay escaped, and
+10 -3
View File
@@ -220,9 +220,16 @@ satisfy a dylib dependency. `platform-mismatch` means a matching-architecture
dylib was found but its observed platform is incompatible with the process.
These outcomes reject that candidate; dyld search may continue and resolve a
later candidate. Candidate outcomes describe individual search paths, while
`resolution.status` describes the final edge. Slices follow dyld's graded
architectures, so an `x86_64h` process also loads `x86_64`. The edge's
`resolution` is one of:
`resolution.status` describes the final edge. For each selected process-root
slice, candidate slices are matched by CPU type and dyld's subtype preference
order, independently of FAT table order. An ARMv7s root slice prefers ARMv7s,
then ARMv7; other known ARM32 subtypes use their compatibility rows from
[Apple's dyld source](https://github.com/apple-oss-distributions/dyld/blob/dyld-852.2/src/dyld2.cpp#L2811-L3105).
An `x86_64h` process also loads generic `x86_64`. Unknown ARM32 subtype policy
remains undetermined when ARM32 candidate slices exist. This is a static model
keyed to the selected root slice; it does not observe the target device's host
CPU or prove which slice the device will execute. The edge's `resolution` is
one of:
| Status | Meaning |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+9 -1
View File
@@ -341,13 +341,21 @@ count and SHA-256.
report null for the newly added encoding fields. `version` is null for an
unversioned symbol. A versioned
symbol reports its GNU index, a name resolved from `.gnu.version_d` or
`.gnu.version_r`, and the `hidden` flag from `.gnu.version`. `"unknown"`
`.gnu.version_r`, the corresponding `name_bytes_base64` and
`name_utf8_decodable` facts, and the `hidden` flag from `.gnu.version`.
Version names use the same UTF-8-when-valid, otherwise byte-preserving
Latin-1 display mapping as symbol names; ELF does not declare either
string's text encoding. Older retained results report null for the added
version-name encoding fields. `"unknown"`
means the symbol-version table could not be read. A hidden version is not
the default version that a by-name `dlsym` lookup selects.
Definition names take precedence when both tables name the same index.
Unreadable names, out-of-range name ordinals and unreadable forwarders are
listed in `issues`, their entries are not reported, and coverage is `partial`.
PE export names are bounded null-terminated byte strings and may continue
across adjacent file-backed section mappings when the virtual bytes are
unambiguous. Gaps, zero-fill and conflicting mappings remain unreadable names.
Unresolved symbol versions keep their rows: the version name is null when no
definition or requirement names the index, and `version` is
`"unknown"` when the version table cannot be read; coverage is likewise
+37 -7
View File
@@ -54,6 +54,26 @@ from a protocol omission. Rejected DOM facts also mark derived URL and agent-hin
metadata incomplete, while valid facts from other nodes and documents remain
available in the capture.
Passive comparison keeps lookup identity, cross-capture correspondence, content
evidence, and inventory coverage separate:
| Observation | Comparison outcome |
| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
| Renamed WebMCP tool keys/frame IDs with one matching name, owner origin, and frame URL | Compare the registration content; local IDs do not prove change |
| Duplicate WebMCP owners with the same name, origin, and frame URL | Unknown correspondence; complete group count differences still prove change |
| Matching script inventory keys with retained source digests | Compare the retained SHA-256 values even without a producer hash |
| One corresponding script source is unavailable, or neither source nor producer hash is available | Source equality remains unknown; inventory changes and other proven differences survive |
| A partial HTTP response group is a subset of the complete peer | Unknown; omitted repeated responses can explain the difference |
| An observed HTTP response fingerprint is absent from the complete peer group | Changed; omissions cannot explain that retained response |
| A WebSocket connection or frame inventory is incomplete | Do not align retained payloads by position |
Storage fingerprints decode supported CDP primitive/JSON values before hashing.
Debugger descriptions, previews, and remote references do not represent storage
content. Actual JSON properties with those names remain content. Reference-only
objects, unsupported values, and opaque cookie partition identities remain
incomplete. Previously exported fingerprints are not recomputed; recapture both
sides when comparing captures produced across this decoder change.
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
@@ -349,15 +369,15 @@ REA retains local observation data and removes only URL userinfo credentials, ex
- URL observations preserve the complete local URL, including query values, fragments, duplicate parameters, and their order. Only username/password userinfo is removed; malformed original URL strings remain available with `origin: null`.
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
- Supported DOM URL attributes are projected separately as resolved destinations. Empty `action` on HTML or XHTML forms and empty `formaction` on HTML or XHTML submit controls resolve to the document URL; other relative URL attributes resolve against the captured document base.
- Accessibility structure and roles are retained by default, while names and descriptions require `include_accessibility_text: true`.
- Accessibility structure and reported roles are retained by default, while names and descriptions require `include_accessibility_text: true`. The browser protocol may omit an individual role; that role remains unknown for comparison. Other observed accessibility differences can still establish a change.
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
For normalized passive inputs with complete request inventories on both sides,
request additions and removals remain observable even when selected body shapes,
response metadata, or WebSocket coverage are unavailable. HTTP request and
WebSocket inventory coverage are evaluated separately; complete HTTP coverage
cannot prove a missing WebSocket connection. Limited request inventory still
prevents request-absence claims. This comparison rule does not make live CDP's
cannot prove a missing WebSocket connection. A missing request in a limited inventory cannot prove absence; a retained
request absent from a complete peer can prove a change. This comparison rule does not make live CDP's
attach-limited inventory complete or establish body-shape equality.
An unavailable HTTP resource type also leaves request identity incomplete:
observations with the same method and URL remain possible matches across types,
@@ -372,11 +392,15 @@ REA retains local observation data and removes only URL userinfo credentials, ex
uses `websocket_connections` and `websocket_frames`. Drops in other event kinds
and the aggregate total do not by themselves limit HTTP inventory. Proven
modifications remain reportable despite incomplete inventory.
Repeated identical HTTP content samples establish multiplicity changes only
with complete request inventories; limited inventories retain known content
comparisons without interpreting duplicate sample counts as changes.
Repeated HTTP samples retain multiplicity. A partial group that is contained
in a complete peer cannot prove modification, even when the omitted response
had different content. A retained sample absent from the complete peer can
prove modification. Two partial groups cannot establish response
correspondence from their different retained samples alone.
WebSocket stream inventories and connection counts likewise require complete
WebSocket coverage. Redirect resource types use only shared known metadata:
WebSocket coverage. Retained frame positions are compared only with complete
connection/frame inventories; matching direction/opcode sequences alone do
not prove alignment after drops. Redirect resource types use only shared known metadata:
an omitted type leaves equality unknown rather than proving a modification.
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. Argument types are listed by CDP argument position. An absent or non-array argument inventory is `null`, and a malformed argument keeps its position as `unknown`; either makes `console_events` incomplete, while an observed empty inventory remains `[]`. A malformed stack or top stack frame is not replaced by its caller: the console event is excluded, and a request initiator location or WebMCP `registration_source` is `null`; each is reported as `invalid_protocol_value`. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim with their original `argument_index`; objects, getters, and remote properties are never expanded. Text capture is `unavailable` when the argument inventory is unknown.
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
@@ -464,6 +488,12 @@ An explicitly empty MIME type is valid. The optional content size may be absent;
an invalid supplied size remains unknown with incomplete resource coverage while
the resource's other facts are retained. Zero and nonnegative fractional sizes
are preserved.
`resource_key` remains the exact observation identity used by script
reconciliation. Capture comparison groups resources by URL, origin, type and
MIME type, then compares known sizes within a group. If a matched group contains
an unknown size, comparison reports unknown rather than treating the observation
as a resource addition or removal; complete inventory coverage still establishes
when an entire group is absent.
CDP discovery has a 10-second response deadline. HTTP discovery documents
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
+90
View File
@@ -0,0 +1,90 @@
# Static analysis in GitHub Actions
The [minimal workflow](https://github.com/morluto/rea/blob/main/examples/github-actions/rea-analysis.yml) builds a small
JavaScript application, analyzes its files without executing them, and uploads
machine-readable results. It uses the existing CLI, not a separate GitHub Action
or an MCP client. It does not promise support for every REA provider.
## Run the example
1. Copy the example to `.github/workflows/rea-analysis.yml` in your repository and
commit it to the default branch. No existing application or package manifest is
needed for the self-contained demonstration.
2. Enable GitHub Actions if necessary. Select **REA static analysis example** in
the Actions tab and choose **Run workflow**.
3. Download the `rea-analysis` artifact after the run. Its files are:
| File | Meaning |
| ----------------- | -------------------------------------------------------------------------------------------------- |
| `analysis.json` | Original CLI JSON, including complete Evidence on success or the error report when one is emitted. |
| `analysis.stderr` | Original diagnostics/progress; an empty file is normal when there are none. |
| `exit-code.txt` | Original analysis process exit code. |
The example pins Node.js 24.22.1, REA 6.7.0, and action commit revisions. It installs
REA into the runner's temporary directory rather than changing the application's
dependencies. This pins the tool version, not every transitive npm dependency;
use a reviewed, committed package lock and `npm ci` if your integration requires
that stronger dependency-reproducibility guarantee.
## Prerequisites and trust
The runner is GitHub-hosted Ubuntu 24.04 with Bash, npm, and `ps` from `procps`.
Other Linux images must supply `procps`; REA's supervised process controls use it.
This static JavaScript operation needs no Hopper, Ghidra, IDA, Java, browser,
Docker, GPU, external API key, or user-supplied GitHub secret. The workflow only
requests `contents: read`; artifact upload uses the runner's normal Actions
credentials.
Replace **Build a known artifact** with your reviewed build, and set
`ARTIFACT_PATH` to its application directory (relative to the checkout) or an ASAR.
The demonstration creates its own tiny application; it deliberately throws if
executed. REA reads it statically. Your replacement build itself can execute code:
review it separately and do not add untrusted pull-request builds to a privileged
`pull_request_target` workflow. The example is manually triggered, not an
untrusted-PR execution policy.
Reports can contain paths, URLs, source-derived facts, and caller-selected data.
Only analyze artifacts and publish reports you are authorized to share. Do not
rely on REA inferring secrets from their names. Review artifact access and
retention for your repository; the example retains reports for seven days.
## Failure, partial results, and unsupported capabilities
The analysis step captures stdout and stderr separately, records `$?`, and exits
with that same status. There is no `continue-on-error`, `|| true`, output filter,
or pipeline that could turn a failed analysis green. Before checkout or other
fallible steps, the workflow creates a fresh report directory under `RUNNER_TEMP`.
Analysis and upload both use that directory, never a checked-in `reports/rea` or
other workspace files. The upload step uses `always()` when that directory was
prepared, so available diagnostics survive an analysis failure. If installation
or the build failed before analysis began, it is empty and upload warns rather
than publishing stale results or inventing a successful result. Runner loss or
cancellation can still prevent upload.
`--json --full-output` selects a machine-readable envelope; it does **not** mean
that analysis was complete. A successful operation can retain partial analysis,
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
coverage, or a policy gate for those findings. Inspect the saved Evidence and
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
host/capability, or another operation failure. Keep the original JSON error (when
emitted), stderr, and exit code, and use their category/message to diagnose it.
An early startup failure may leave `analysis.json` empty; do not parse that as
successful analysis. Select a supported operation or explicitly provision its
[documented prerequisites](installation.md); do not silently switch providers,
execute the artifact, or ignore the failure to obtain a green job.
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
`build/missing-artifact` without changing the demo build. Analysis must fail,
`exit-code.txt` must be nonzero, and the available reports must still upload.
To exercise unsupported input, set `ARTIFACT_PATH` to `build/rea-ci-demo/index.js`:
this operation accepts an application directory or ASAR, not a single source file.
The retained JSON error has category `unsupported_target`; the step exits nonzero.
Restore the correct path before using the example normally.
To exercise partial analysis, append `export const broken = ;` to the demo's
JavaScript. REA 6.7.0 retains `data.normalized_result.statistics.parse_failures`
and an incomplete-coverage limitation while the operation can still exit zero.
Inspect those facts instead of treating process success as full coverage. The
report JSON and diagnostics remain unfiltered in all three cases.
+190 -2
View File
@@ -3,6 +3,8 @@
Use the CLI for a direct inspection or a script. It uses the same application
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
separate process; save results when you want to use them in a later command.
For a pinned, noninteractive workflow with retained reports, see
[static analysis in GitHub Actions](ci-analysis.md).
## Run a command
@@ -44,6 +46,119 @@ module page. See
[JavaScript application workflows](javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
## JSON Lines record output
Use `--format jsonl` to consume row-oriented results one record per line:
```bash
cat > observations-view.json <<'JSON'
{"source":{"kind":"evidence-file","path":"app-evidence.json"},"view":{"kind":"javascript-facts","property_keys":[],"offset":0,"limit":32}}
JSON
rea inspect-analysis-view observations-view.json --format jsonl > observations.jsonl
```
This request selects up to 32 observation identities and their provenance,
without observation property values.
This changes JSONL object-collection framing. Scripts that need one complete
report or a scalar field should use `--format json`. For a full application
graph, first project a supported analysis view rather than selecting an
arbitrary nested graph array.
A top-level array emits its elements in source order. An object must have one
clear `rows`, `items`, `records`, or `occurrences` array, directly or under
`page`. Evidence uses the same boundaries inside `normalized_result`.
An arbitrary report or a scalar selection is refused with an actionable error;
use `--format json` for a complete document. If several collections exist,
select one with `--filter-output`, for example `normalized_result.rows`.
Object collections finish with a control record:
```json
{
"$rea": "jsonl/v1",
"type": "metadata",
"records": {
"path": ["normalized_result", "rows"],
"emitted": 2,
"source": 10,
"filter": "normalized_result.rows[0,2]"
},
"context": {
"evidence_id": "ev_…",
"normalized_result": {
"coverage": {
"status": "page",
"examined": 10,
"total": 100,
"next_offset": 10,
"exhausted": false
}
}
}
}
```
This abbreviated example illustrates framing. The actual `context` preserves
all non-row fields, including Evidence subject, provider, authority, confidence,
locations, links, limitations, parent/view digests, pagination and coverage.
With `--full-output`, it also retains the command envelope and its metadata.
A data filter selects row fields/slices while this context remains available;
excluded peer record collections and unselected Evidence `raw_result` are not
restored. An explicitly selected `raw_result` remains selected. Retain the
original Evidence for complete observations.
`records.emitted` and `records.source` count the selected output and its source
array; they do not assert analysis completeness. Source coverage and unknowns
retain their original meaning. Existing row identities and indices remain
unchanged unless your field selection excludes them. Empty object collections
still emit metadata; an empty top-level array emits no lines. Bare top-level
arrays have no synthetic coverage record.
Command and output-validation failures emit `{"$rea":"jsonl/v1","type":"error","ok":false,"result":…}`
and retain the original diagnostic payload. SDK validation/sentinel errors use
the SDK's exit status rather than guessing from data fields. `$rea` is reserved
at the top level of a data record, so control records are unambiguous; collisions
are refused before rows are written. Nested `$rea` values are ordinary data.
A missing final metadata record or nonzero exit can indicate an interrupted
export; do not treat already emitted rows as a complete result.
`--token-count` estimates this exact representation, including metadata, one
line at a time. Errors remain error records even with token counting enabled.
Token windows cannot preserve JSON Lines and are rejected. JSON encoding and
stdout writes are incremental, including large escaped strings; this is output
streaming over a completed result, not incremental analysis. The analysis still
needs its normal resources. A single record must fit the runtime string limit
for token estimation; remove `--token-count` to stream a larger record.
Standalone version and MCP doctor diagnostics remain single JSON documents
in JSON Lines mode. Incur's streaming generator events and successful sentinel
responses retain their SDK protocol. MCP application results and Evidence
contracts are unchanged.
## Shell completion
The CLI includes optional shell completion. Run `rea completions --help` to
see the available shells and their setup commands. For Bash, load the hook
into the current shell:
```bash
eval "$(rea completions bash)"
```
The current Bash hook requires support for `complete -o nosort`; macOS's
system Bash 3.2 cannot load it. Use a newer Bash for this hook.
For future interactive non-login Bash sessions, add the command to
`~/.bashrc`. Interactive login Bash sessions read the first available file in
`~/.bash_profile`, `~/.bash_login`, and `~/.profile`; add the command to the
file your shell reads, or have that file source `~/.bashrc`. Remove the command
from the startup file to stop loading completion, and run `complete -r rea` to
remove the registration from the current shell. These are manual, opt-in
steps.
Completion is a discovery aid; use `rea COMMAND --help` to confirm the
available options and subcommands.
## Managed assemblies (.NET)
For a .NET PE/CLI assembly, inspect its identity and deployment form before
@@ -284,7 +399,11 @@ an exact result when the target bytes, operation, parameters, provider and
settings match. Mutations and cursor-dependent calls are excluded from the
cache. Snapshot files are local and use owner-only permissions.
Dispatching a metadata write invalidates the active session's snapshot cache,
Importing an Evidence bundle refreshes retained subject display names and paths.
It preserves cached analysis results, including observations for other targets.
Snapshot bindings that use a replaced record retain its updated display path.
Dispatching a provider metadata write invalidates the active session's snapshot cache,
even if its reply fails: the provider may already have applied the write.
Close and reopen the target before importing or saving another snapshot.
@@ -401,7 +520,10 @@ top-level limitation. That partial inventory does not establish absence among
unexamined children. POSIX directory-entry bytes that are not valid UTF-8 cannot
be represented as graph paths; the containing directory (or root) is marked
unsupported, and REA does not access the entry through a replacement-character
name. Readable siblings remain inventoried.
name. Symlink targets are read as bytes too: a target that is not valid UTF-8
is retained as an unreadable symlink with its bytes reported in the limitation,
and is never resolved through a replacement-character path. Readable siblings
remain inventoried.
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
bytes retain their original hashes and sizes with a decoding diagnostic; REA
@@ -410,6 +532,19 @@ strings, expression-free templates and runtime-transparent TypeScript wrappers
retain their targets; computed imports remain unknown. TypeScript import-equals
dependencies remain `requires` relationships whether or not they are exported.
REA derives historical CommonJS paths only when the selected loader and its
base are proven: this includes the exact
`createRequire(new URL("./file.cjs", import.meta.url))` form and
`require.resolve` with `{ paths: [__dirname] }`. Other custom bases and search
options remain partial with unknown resolution and the literal requested
specifier. Overwritten resolver methods retain that uncertainty; they are not
attributed to Node. `require.main` is a module object, so calling it does not
establish a loader reference. These relationships do not establish execution;
Node's [`require.resolve`](https://nodejs.org/docs/v24.22.1/api/modules.html#requireresolverequest-options)
performs lookup without loading the target. An excluded `package.json`
also leaves package-directory selection unknown rather than treating the
manifest as absent and applying an `index` fallback.
Historical-source import requires safe no-follow file opens on Linux or
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
or another supported host. See
@@ -536,6 +671,52 @@ without response bodies, or without the request bodies its writes need. See
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
```
## Report version and runtime
`rea --version` prints the package version. Add an output format to get the
version with the runtime and platform context that a support request needs:
Version reports do not initialize analysis providers. Invalid global settings
or output options still report their usual errors.
```bash
rea --version --json
```
```json
{
"name": "rea-agents",
"version": "6.7.0",
"runtime": { "id": "node", "version": "24.22.1" },
"platform": "linux",
"architecture": "x64",
"supported_runtime": {
"range": "^22.23.4 || ^24.22.1 || >=26.12.1",
"satisfied": true
}
}
```
| Field | Meaning |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| `name`, `version` | Published package name and version; `version` matches plain `--version`. |
| `runtime.id` | `node` or `bun`. |
| `runtime.version` | Version of that runtime. |
| `runtime.node_compatibility_version` | Bun only: the Node.js version it reports compatibility with. |
| `platform`, `architecture` | Node.js `process.platform` and `process.arch`, such as `linux` and `x64`. |
| `supported_runtime.range` | Supported Node.js range. |
| `supported_runtime.satisfied` | Whether this runtime is supported, judged as `rea doctor` judges it; Bun by its Node.js compatibility version. |
The field names are stable. The report contains no paths, environment values,
user or host names, and it starts no provider and reads no target. `--format`
selects any CLI output format, such as `--format yaml`. JSON follows the usual
rule: indented on a terminal, compact when piped or redirected. `--help` still
takes precedence. With other output controls, `--version` keeps its existing
behavior: valid controls such as `--filter-output` or `--full-output` print the
plain version, and invalid combinations report the usual output-flag error,
such as `UNSUPPORTED_OUTPUT_COMBINATION` for `--json` with `--token-limit`. Use
`rea doctor` for provider and host readiness.
## Output and exit status
`rea mcp doctor --json` checks production MCP startup and reports child logs in
@@ -554,10 +735,17 @@ redirected, which keeps large results smaller for agents and scripts; pipe
through `jq .` for indented files. Output selection and formatting do not change
operation status.
`--format md` escapes backslashes and pipes in table headers and cells, and
renders cell line breaks as `<br>`. These are presentation changes; retained
Evidence and JSON output keep the original values. Table widths are measured
iteratively, so an admitted row page is not limited by the JavaScript function
argument count.
`--filter-output` selects fields from successful command data. Classified failures
retain their complete typed diagnostic projection even when the selector matches
a field such as `code` or `message`, or names a missing field. `--full-output`
controls whether the outer `{ok, error, meta}` envelope is included.
Selected own JSON members, including `__proto__`, remain ordinary output data.
Sibling array slices that select different fields are combined by original row
index and emitted in source order, regardless of selector order. Selecting a
+32
View File
@@ -14,6 +14,12 @@ preserving your REA client entry. See the
## GitHub Copilot CLI
Setup preserves an existing `tools` selection in the REA registration, including
an empty list. New registrations default to `["*"]`. A restricted selection is
aligned configuration; doctor does not widen it to expose every REA tool. See
[Copilot's MCP configuration](https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers)
for tool-selection settings.
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
chat with REA's full input-schema profile before sending a model request
(`compaction_static_context_blocked`; [#1554](https://github.com/morluto/rea/issues/1554)).
@@ -248,6 +254,32 @@ standalone CLI check.
## Codex and Devin
REA manages its launcher and selected provider settings in the client
registration. Codex's server and per-tool approval policies are user choices.
Setup patches REA's connection fields in the selected file and retains other
server settings, including `default_tools_approval_mode`, nested
`[mcp_servers.rea.tools.<name>]` tables, tool allow/deny lists, and unrecognized
client settings. Doctor accepts Codex's per-tool policy tables. Known malformed
policy values block setup before it writes the configuration or a backup, and
the diagnostic identifies the setting to repair. See the
[Codex configuration reference](https://developers.openai.com/codex/config-reference/)
for the supported policy values.
| Settings | Setup ownership |
| ----------------------------------------- | --------------------------------------------------------------------- |
| Launcher, transport and activation | Refreshes the REA stdio connection and enables it. |
| Explicit server environment | Preserves configured overrides and merges selected provider settings. |
| Codex positive timeouts and client policy | Retains caller choices; adds missing recommended REA timeouts. |
| Unrelated or future server settings | Retains their parsed values. |
Dry-run applies the same policy validation and writes no files. A successful
refresh keeps an exact recovery snapshot of the original configuration;
repeating setup with the same settings performs no write. Uninstall removes
the owned REA entry, including its policy, while retaining other servers and
the recovery snapshot. A green registration check establishes known
configuration alignment; it does not prove that a particular client or fork
successfully exposes every REA tool.
Codex treats an unset or empty `CODEX_HOME` as `~/.codex`. Its MCP server
registration supports `startup_timeout_sec` for connection initialization and
`tool_timeout_sec` for each tool call. New REA registrations use a 30-second
+6 -5
View File
@@ -103,11 +103,12 @@ same bundle. Export writes only the caller-selected output file.
When the four-byte metadata magic is present but version bytes are incomplete,
the fact retains `raw_version: null`; it does not offer an IL2CPP metadata
inspection probe. A complete magic/version prefix with supported raw version 29,
31 or 39 offers an
`inspect_il2cpp_metadata` follow-up at its admitted file path, including when no
native library was found. This probe does not establish full metadata validity
or correspondence to a native binary. See [IL2CPP metadata](il2cpp-metadata.md).
inspection probe. A complete magic/version prefix with supported raw version
24, 27, 29, 31 or 39 offers an `inspect_il2cpp_metadata` follow-up at its
admitted file path, including when no native library was found. This probe does
not establish full metadata validity or correspondence to a native binary;
version 24 inspection still applies only to supported header layouts. See
[IL2CPP metadata](il2cpp-metadata.md).
Candidates are derived or inferred from these facts and do not prove loading or
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
+22 -5
View File
@@ -24,8 +24,11 @@ requires an absolute filesystem path and preserves that exact selection:
The inline Evidence result contains the artifact path, SHA-256, and byte length.
`tables` lists each declared note program header or section, including the
section name when a string table is present. Each `entries[]` item reports owner,
type, descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
section name when a string table is present. Zlib-compressed section-name
tables are decompressed within the selected file-size budget. An unsupported
compression type leaves section names unknown and adds a limitation; note bytes
remain independently examined. Each `entries[]` item reports owner, type,
descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
filled when the descriptor matches that type. Other owners keep `gnu` null.
@@ -40,14 +43,28 @@ the image never carried notes in another container or overlay.
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
32 note entries are examined by default across every table. Retained note
names, section names, and GNU descriptor text share that file's byte budget,
and repeated table ranges count again. Each descriptor range is hashed once,
names, section names, and GNU descriptor text share a scan budget based on
the source bytes plus any decoded section-name table, capped at
`max_file_bytes`. The raw source and decoded table must also fit together
within that ceiling. If admission fails, the error reports the required
`max_file_bytes`; raising it can admit the decoded table up to the operation's
512 MiB maximum. Once admitted, raising that setting does not increase the
scan budget for the same source and decoded names. Repeated table ranges count again. Each descriptor range is hashed once,
and each GNU version's trailing NUL run is scanned once; neither walk may
exceed that byte ceiling. Malformed ELF headers, truncated header
tables, declared note tables that extend past the file, and truncated
note entries fail without partial success. ELF extended
numbering (`PN_XNUM`, including a section count stored when `e_shnum` is 0) is
unsupported.
unsupported and returns an unsupported-target outcome rather than invalid-input
recovery.
`SHF_COMPRESSED` note sections return `unsupported_target` before their payload
is interpreted or decompressed. Their compression header is storage metadata,
not a note header. The current result requires physical file offsets for each
note and descriptor, which decoded compressed notes cannot provide. This refusal
does not label compressed note storage as malformed or allocate its declared
decoded size. Compressed section-name tables remain supported within the limits
above.
This workflow does not execute the image or verify GNU build-id uniqueness
against package catalogs.
+17 -1
View File
@@ -27,8 +27,11 @@ The inline Evidence result contains the artifact path, SHA-256, and byte length.
Repeated names retain separate chunk occurrences. Decoded tables and the Code
header use the last occurrence of each name, matching OTP's loader; the chunk
inventory still exposes earlier payloads and their source ranges.
The BEAM form size bounds the parsed chunks. Bytes after the declared form are
ignored by the parser but remain part of the selected artifact's byte length
and SHA-256.
When `AtU8` or `Atom` is present, `atoms` is the 1-based name table and
When a selected `AtU8` or `Atom` table is decoded, `atoms` is the 1-based name table and
`module` is atom 1. Both the pre-OTP 28 table and the OTP 28+ compact table
(negated count, compact-encoded name lengths) are read. `exports`, `imports`, and `lambdas` resolve atom indices to
those names when the table is present; unresolved indices keep `function` or
@@ -38,6 +41,19 @@ those names when the table is present; unresolved indices keep `function` or
function count, and the instruction stream's file range. Missing optional
tables are null, not empty-everywhere.
A zero-byte `AtU8` falls back to an available legacy `Atom` table, following
[OTP 20.3's atom selection](https://github.com/erlang/otp/blob/a113f6117fd696ea6f84ed754055b4ec97a7ccb2/erts/emulator/beam/beam_load.c).
A nonempty `AtU8` retains UTF-8 precedence, including an explicit zero atom
count; malformed nonempty data fails instead of falling back. An empty atom
chunk without an alternate table still fails its required count header.
Absent and zero-byte optional `FunT` chunks produce `lambdas: null`, while a
four-byte zero row count produces `lambdas: []`. This follows
[OTP 29's positive-size lambda parsing rule](https://github.com/erlang/otp/blob/550d7b7898706c7822362c42e7b93120c1d1f29a/erts/emulator/beam/beam_file.c).
Empty payloads retain their ranges and SHA-256 in the chunk inventory. These
rules apply to the last occurrence of each chunk name; an empty final occurrence
does not revive an earlier payload of that name. Nonempty truncated tables remain
format failures.
## Limits and failures
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
+6 -3
View File
@@ -19,8 +19,10 @@ rea inspect-evm-interface ./runtime.bin raw --json
Initial real verification covers flat legacy bytecode on Linux x64 and unchanged
bundled EVMole 0.9.3. EOF-style EF00 containers return `unsupported_target`,
with the selected path and format reason. Provider/configuration failures retain
their distinct unavailable-provider or capability classification.
with the selected path and format reason. Selected-path resolution errors
`ENOENT`, `ENOTDIR`, `ELOOP`, and `ENAMETOOLONG` are `invalid_request` path
issues. Permission, resource, cancellation, provider, and cleanup failures retain
their distinct classifications.
The host must already provide util-linux `prlimit`, normally `/usr/bin/prlimit`;
`REA_EVM_PRLIMIT_COMMAND` can select another absolute executable. REA installs
no system tools and changes no user configuration. [Upstream provenance and
@@ -43,7 +45,8 @@ license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md)
map from the selected bytes. Integers use decimal strings; byte strings use hex,
and scalar keys retain their types. The recorded offset/length addresses the
original bytecode, so summary delivery does not copy an opaque CBOR payload.
Nested containers have partial scalar coverage; unsupported or malformed
Nested containers have partial scalar coverage; map keys are still checked for
duplicates throughout nested maps and arrays. Unsupported or malformed
candidates report `not-decoded` with a diagnostic. Decoding is bounded to 64
container levels and the terminal payload's encoded size.
- An unavailable trailer does not prove metadata absence or Solidity origin.
+13 -1
View File
@@ -108,7 +108,12 @@ server; older servers keep the complete request after schema discovery.
`build_id_candidates` preserves distinct IDs. `build_id` is populated only
when all declared note ranges were examined and one ID was found;
conflicting IDs or malformed/bounded note metadata make coverage partial.
Nonempty GNU build-ID descriptors up to 64 bytes are retained as lowercase
hexadecimal, including IDs shorter than four bytes.
Matching bytes outside declared note ranges cannot supply a build ID.
Compressed `SHT_NOTE` sections are skipped with an unsupported-compression
limitation and partial coverage; IDs from other note ranges remain candidates.
Encoded note storage is never interpreted as note headers or inflated.
- Labeled strings must fit their complete grammar. `dart_version_capped`
and `toolchain_capped` independently report labels omitted by grammar or
projection bounds; either makes identification coverage partial. Values
@@ -152,9 +157,16 @@ server; older servers keep the complete request after schema discovery.
- **Selected ABI.** AOT inspection applies the library budget to the selected
ABI after selection; an oversized unselected library does not block it.
Undefined dynamic-symbol imports are not treated as snapshot exports.
A retained section directory without `SHT_DYNSYM` uses the same bounded
`PT_DYNAMIC` symbol description as a section-stripped image. Malformed
present symbol-table sections remain failures.
Section-stripped images require a bounded symbol count from `DT_HASH` or
`DT_GNU_HASH`; layouts without either are refused. Symbol names and their
terminators must fit their linked string table.
terminators must fit their linked string table. Only dynamic entries used for
symbol discovery are interpreted as numeric bounds. ABI-ignored payloads,
including [`DT_BIND_NOW`](https://gabi.xinuos.com/elf/08-dynamic.html#dynamic-section),
do not change discovery; their stored bytes still contribute to the selected
artifact's digest.
- REA performs **no hash-to-SDK lookup**: mapping the snapshot hash to a
Dart SDK release requires an external dataset and is deliberately out of
scope for this family.
+2
View File
@@ -6,6 +6,8 @@ For remote devices, pass a host with an optional port (`host[:port]`; bracket IP
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
A summary's selected fields look like this (excerpt):
+13 -5
View File
@@ -130,7 +130,13 @@ For ELF images without a named build-info section, discovery checks each
nonempty writable, non-executable load segment in program-header order. This
includes sectionless PIE images with metadata in a later data segment. Named
sections retain priority, and alignment, file mapping and record validation
remain required.
remain required. When ELF section names are unreadable, limitations report that
coverage gap; the fallback search policy is unchanged.
A named ELF or Mach-O build-info section selects the record's discovery range.
For inline encoding, the following version and module strings may continue past
that section into its file-backed load mapping; their bytes are still checked
against the mapping and the existing aggregate string limit.
Historical moduleless Go binaries can store their empty module-string header in
declared zero-initialized virtual memory rather than file-backed bytes. The reader
@@ -157,10 +163,12 @@ and `reported_limits.maximum_input_bytes`. Select a suitable input within that
fixed allowance; changing an output-write or transport budget does not admit it.
Resource guards bound the selected file and cumulative header searches to 256 MiB
each, aggregate embedded strings
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
report to 8 MiB. Identical search spans with the same virtual alignment are
searched once; partially overlapping spans consume the cumulative search budget
each, aggregate embedded string contents to 1 MiB (varint length prefixes are
framing and do not count toward this allowance), structural table decoding to
16 MiB and the complete normalized metadata report to 8 MiB. Inline strings must
also fit within their actual mapped source range. Identical search spans with
the same virtual alignment are searched once; partially overlapping spans
consume the cumulative search budget
and fail with a CPU resource constraint when it is exhausted. Oversized results
fail without partial success. MCP also applies
its [response transport budget](mcp-contracts.md), which counts both text and
+5
View File
@@ -34,6 +34,11 @@ The same CLI commands locate a PCK at the beginning of a file or through its
payloads or discover packs behind additional trailing signatures. The root
content SHA-256 covers the whole carrier; its content classification remains
native, while the inventory records the PCK role and exact pack bounds.
An EOF marker is only a candidate: an invalid size or a pointed-to location
without a `GDPC` header leaves native classification in place. Once the
trailer points to a pack header, malformed or unsupported PCK content remains
an error. This recognition boundary follows the Godot loader at pinned source
revision [06d1e7a](https://github.com/godotengine/godot/blob/06d1e7a34ccde4029eed53d018cfde77dfe771d5/core/io/file_access_pack.cpp#L2099-L2146).
Ordinary native `open_binary` detection remains unchanged.
## MCP and selected facts
+48 -3
View File
@@ -32,6 +32,33 @@ unexamined payloads remain `unexamined`. An exact AMDGPU target is normalized at
its note owner for comparison with a Clang bundle declaration; raw values remain
intact. Unresolved feature comparisons stay `unknown`.
A HIP target ID identifies a toolchain target, not its payload representation.
Recognized raw or wrapped LLVM bitcode stays `unsupported` for kernel metadata
inspection, with the bundle ID, range and selected payload digest retained.
Bitcode validity and inner targets remain unexamined; malformed recognized ELF
still reports `malformed`. Standalone bitcode is outside GPU container coverage.
HIP bundle and metadata note recognition accept both the legacy `amdgcn` triple
and LLVM's `amdgpu` subarchitecture triples, including declared generic processors
such as `gfx11-generic` and `gfx9-4-generic`. Notes with the empty or explicit
`unknown` environment use the same processor and feature comparison while
retaining their raw triple spelling. CUDA's legacy fatbinary compression flag is
retained as an unsupported codec with its encoded and decoded extents;
compressed bytes are not treated as PTX text. CCOB is recognized by the exact
four signature bytes `CCOB`; high-bit-corrupted bytes are not decoded as ASCII
and accepted as a bundle. CCOB v1 has no stored total size: the completed zlib
or Zstd stream determines its encoded extent. Embedded `CCOB` text is ordinary
compressed data; zero-byte alignment after the stream belongs to neither
payload. If decoding fails or its codec is unsupported, a later `CCOB` marker
is only an uncertain recovery candidate. The affected row retains that
limitation and discovery remains partial.
The declared decoded length bounds decompression. An understated length can
stop decoding before codec completion, so any subsequent recovery extent stays
unknown; the decoder does not expand its allowance to prove the bad declaration.
Interrupted observations omit unfinished v1 rows. A terminal capacity row
qualifies its retained range as the containing tail and does not infer siblings.
`cuda_target_header` retains the OS ABI, normalized raw `e_flags` and exact
header location. CUDA OS ABI `0x33` and the observed `0x41` flags layout use
different architecture fields; they are decoded at one ABI-aware owner.
@@ -90,9 +117,18 @@ Kernel representations have different meanings:
parameters are excluded from entry virtual-register counts. Parameter names
remain available when pointer qualifiers are understood or an unsupported
qualifier prevents layout decoding. A pointer's `.align` describes the
pointed-to memory and does not change parameter layout alignment.
pointed-to memory and does not change parameter layout alignment. Spaced,
joined and partly joined pointer qualifiers have the same meaning. Parameter
alignment before or after the scalar type retains its layout and name.
Parameter names and following declarations are retained when an array
extent uses a parenthesized expression or when dimensions are
multidimensional, even though those forms do not currently yield a layout.
PTX target selection uses the highest-numbered module-scope `.target`
directive, including directives after entry declarations; unsupported or
conflicting target profiles leave the effective target unknown.
Scalar register declarations count each bare or percent-prefixed name and
each parameterized range. Unsupported declaration forms or overflowing totals
each parameterized range, including compact keywords such as `.reg.b32`.
Unsupported declaration forms or overflowing totals
retain an unknown count and partial decoding coverage. Unsupported parameter
layouts and unknown shared-memory declarations likewise retain their known
facts, explicit limitations and partial coverage through retained pages.
@@ -100,7 +136,14 @@ Kernel representations have different meanings:
including V2 kernel-parameter sizes, and native section declarations. Global
register declarations also cover compiler
versions that leave the older text-section register field empty.
- AMDGPU reads v3+ MessagePack notes. Argument counts include hidden ABI arguments
Conflicting register declarations keep the physical count unknown, preserve the
kernel's other facts, and mark object/report/target decoding coverage partial.
- AMDGPU reads v3+ MessagePack notes. V3 processor identity comes from the
`EF_AMDGPU_MACH` field in ELF `e_flags`; known processor values are selectable
even when the note omits `amdhsa.target`. This inventory reports the processor
only; it does not interpret version-specific feature flags. Unknown processor values remain unknown. A known
processor conflict between ELF flags and note metadata leaves the processor
unresolved. Argument counts include hidden ABI arguments
when present; SGPR/VGPR counts and fixed group/private segment sizes retain
their declared meanings. Legacy YAML metadata remains unsupported.
@@ -177,3 +220,5 @@ failures preserve earlier successfully parsed declarations and the original
failure reason. Standalone PTX recognition tries a small header first, then
continues within the existing 16 MiB text budget when leading comments or
whitespace obscure its directives; admitted full-file probe bytes are reused.
After those comments and whitespace, `.version` must be the first PTX module
token; later PTX-looking text does not identify an arbitrary binary as PTX.
+17 -9
View File
@@ -212,6 +212,11 @@ linked method table or seed output is published. Row ids may repeat across
different images, and distinct rows may still resolve to the same address when
the binary uses identical-code folding.
For ELF, a repeated load mapping that resolves a complete module record to the
same virtual address and file bytes contributes one record. Distinct virtual
record addresses remain distinct candidates, and conflicting file mappings
remain malformed.
Module records establish names through their declared NUL-terminated string pointers. NUL/0xFF padding, spaces, Unicode and suffix-shared storage are supported. A pointer to a longer string does not match just its suffix; a pointer directly into a shared suffix may name it. Exact reversed UTF-8 matching avoids rescanning the file for every name and retains duplicate-name owners until module records resolve them. A contributor investigated a Linux
`GameAssembly.so` from Unity 2023.2.13f1 whose static registration was not located. When no static pair is resolved on x86-64, the reader searches executable ranges for bytes consistent with RIP-relative LEA/MOV references to module-array candidates. Exactly one referenced array candidate must remain. `code_gen_modules_field` is then `null`, and Evidence is **inferred**: instruction boundaries, the absence of a static registration elsewhere, and runtime use were not verified. The module records and slots themselves remain file observations. Static-table associations are **derived**. Otherwise the result is `code-registration-not-found`.
@@ -243,7 +248,9 @@ The result adds:
`rva` (address minus `image_base`), both hexadecimal strings. `state` is
`code` when the module slot holds an unambiguously mapped executable address, `no-body` when an observed slot is zero,
as for abstract, extern and most generic methods, and `unknown` when no type
range in any image holds the method or a symbol relocation leaves its slot unresolved.
range in any image holds the method, a symbol relocation leaves its slot
unresolved, or dyld will bind a Mach-O slot. The raw word in a dyld-bound
slot is not treated as the resolved address.
`address` and `rva` are `null` unless `state` is `code`. The summary counts
each state in `counts.method_address_states`. Ghidra seeds are written only
for `code` rows.
@@ -255,8 +262,10 @@ The result adds:
definitions, `T[]` for vectors, `T[*]` for rank-one multidimensional arrays,
`T[,]` for rank-two arrays and `T*`, generic instances as
`System.Collections.Generic.List<string>` and generic parameters by their
metadata name (`TKey`). A trailing `&` marks a by-reference type. `name` is
`null` only for type index `-1`.
metadata name (`TKey`). Nested generic arguments stay with the component
that declares them, for example `Game.Outer<int>+Inner<string>` or
`Game.Outer<int>+Leaf` when the nested type adds no parameters. A trailing
`&` marks a by-reference type. `name` is `null` only for type index `-1`.
`binary` also records `types_field` and `types_count`: the address of
`Il2CppMetadataRegistration.typesCount` and its value. That record is found as
@@ -293,7 +302,7 @@ Format and selection failures name `binary_path`; capacity refusals remain typed
Scans yield cooperatively in bounded batches. Name-address retention and reverse-trie construction are each limited to65,536 entries/nodes and fallback module-array candidates to64 (`native-index-capacity`); exceeding capacity does not prove absence. Rendered type text has a64MiB UTF-8 cache-workspace guard, separate from the64MiB encoded-report admission, and64 nested rendering frames (`native-type-text-capacity` / `type-nesting-capacity`). Genuine cycles are malformed; unsupported type kinds are explicit coverage failures. Native row/text admission runs before allocation or retention. Unused generic parameter names are read on demand from the same stable metadata snapshot. There is no cross-inspection file cache; validated immutable Evidence is reused for session views.
ELF RELATIVE writes into zero-fill are kept as a private virtual-pointer overlay. Unresolved supported-width symbol relocations, including RELA PLT entries, remain unknown; stored zeros never establish no-body. COPY, TLS descriptor, REL, packed APS2 and other unsupported relocation encodings fail closed. Input bytes are unchanged.
ELF RELATIVE writes into zero-fill are kept as a private virtual-pointer overlay. Unresolved supported-width symbol relocations, including RELA PLT entries, remain unknown only for the bytes their ELF ABI relocation type covers; an adjacent 32-bit relocation does not make a separate 64-bit method slot unknown. Stored zeros never establish no-body. COPY, TLS descriptor, REL, packed APS2 and other unsupported relocation encodings fail closed. Input bytes are unchanged.
Native address queries batch exact64-bit hexadecimal `addresses` and `states` (`code`, `no-body`, `unknown`), intersecting indices and pagination. Numerically equal address spellings match; original row indices and every shared-body alternative survive:
@@ -359,16 +368,15 @@ message names one reason:
| `nested-type-conflict` | A type is listed as nested in two types, in itself, or in a cycle of declaring types. |
| `declaring-type-mismatch` | A method inside a type's method range names a different type, or a method lies outside its type's range. |
| `image-range-overlap` | Two images claim the same type definitions. |
| `invalid-string` | A name index does not start a string, or the name is unterminated or not valid UTF-8. |
| `invalid-string` | A name index is outside the strings section, its string is unterminated, or it is not valid UTF-8. |
A version field that disagrees with the file's layout, for example a version
39 file relabeled as 29, usually surfaces as `unexpected-header-size` or
`record-size-mismatch`.
Two checks are stricter than the IL2CPP runtime: a name index must start a
string, and an empty range's start must lie between -1 and the collection size.
Every Unity-written file checked so far satisfies both; metadata rewritten by
other tools may be rejected.
An empty range's start must lie between -1 and the collection size, which is
stricter than some runtime paths. String indices are byte offsets; a valid
offset may select a suffix of another NUL-terminated name.
An incomplete inventory is never reported as complete. Failures after a verified
snapshot retain partial source Evidence with its SHA-256, byte length and any
+32 -2
View File
@@ -60,7 +60,10 @@ Before applying changes, REA checks your current configuration. The plan lists:
Malformed or unsafe existing configuration blocks the whole transaction before
Hopper installation or any file write. Declining or pressing Ctrl-C makes no
changes. Agent configuration writes preserve unrelated entries and comments,
changes. Client configuration files and bundled skill files have separate
publication and recovery paths; the guarantees in this paragraph apply to
client configuration. See [skill destinations](#skill-destinations) for skill
rollback limits. Agent configuration writes preserve unrelated entries and comments,
create backups, stage changes in a private directory, and verify their result.
Before publication, they refuse detected changes to the selected configuration.
Replacement uses an explicit destination under the selected canonical parent;
@@ -85,6 +88,11 @@ Setup also retains valid environment overrides in REA's existing server entry,
such as `REA_MCP_INPUT_SCHEMA_PROFILE`, `REA_MCP_OUTPUT_SCHEMA_PROFILE`, and
`REA_MCP_MAX_RESPONSE_BYTES`.
Detected provider settings replace only matching environment keys.
Setup manages REA's launcher, selected provider settings, and supported timeout
defaults. Client approval policies remain user choices, but the current Codex
refresh drops those policies inside the REA entry; see the
[Codex preservation limitation](client-configuration.md#codex-and-devin) before
refreshing a customized registration.
Setup and doctor report a malformed REA environment map or invalid runtime
setting before setup writes files. Repair the named setting in the client
configuration; setup does not silently discard the rest of the map.
@@ -162,6 +170,26 @@ paths as `unmanaged`, names the affected paths, and provides those alternatives;
an unreadable skill identity remains `unknown`. Uninstall retains and reports
files it cannot safely remove, including their ownership manifest.
The skill installer preserves byte-exact backups of changed existing files,
including invalid UTF-8, BOMs, and permission modes, and verifies each published
file through its owned descriptor. Before publication it checks the selected
file identity; first installation uses an exclusive hard link and refuses a
concurrently created destination. Filesystems without hard-link support cannot
publish a first-install skill file.
Rollback restores an existing file only while its published identity, state,
and bytes still match that installation attempt. A recovery backup preserves
the original bytes; it does not authorize overwriting a later user edit or
replacement. Conflicts report the affected paths and recovery backups. If a
newly created skill file cannot be verified, setup retains it and reports manual
recovery rather than deleting by pathname. Staging cleanup failures remain
owned and retryable, and setup does not report installation success while
cleanup is unresolved.
These checks do not lock out another writer modifying the same inode or exclude
pathname changes between the last check and publication. Avoid editing managed
skill files or running another installer while setup applies them.
`rea setup --skill-client claude_code --yes --json` refreshes and verifies the
selected skill destination independently of MCP registration selection. The
final `doctor.scope.skill_clients` records which skill destinations were
@@ -736,7 +764,9 @@ and skips directories.
On Linux, it prefers executable `/opt/hopper/bin/Hopper`, then executable
`~/.local/share/rea/hopper/bin/Hopper`, followed by `/opt/hopper-v4/bin/hopper`
and `/usr/bin/hopper`; if none exists, the former remains the diagnostic
fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
fallback. `rea doctor` checks only the current platform's candidates, in the
same order; other platforms have no automatic candidates.
`HOPPER_LAUNCHER_PATH` overrides those choices:
```bash
export HOPPER_LAUNCHER_PATH=/absolute/path/to/Hopper
+23 -2
View File
@@ -40,6 +40,12 @@ combine its fields with the fact's location. Semantic trace results include the
canonical context subset referenced by their returned facts, so those facts
remain self-contained in the response.
Content-deduplicated module references match every retained observation path.
A path-filtered reference reports the caller-selected path while keeping its
shared node identity. Item selection also accepts any retained path when it
identifies one module; if several module kinds share that path, select the
reported `node_id`. Node-ID selection uses the stable representative path.
## Retained module facts
For canonical identities and retained source-range counts, query the existing
@@ -406,8 +412,23 @@ when the object is an array element, nested inside mixed object/array
containers, or assigned to a previously declared identifier. Saving the getter
result in an alias does not make its old field values immutable. Known array
indices include holes; an element after a spread has only a minimum possible
index. Unrelated properties and known array prefixes remain precise, and a
getter read without a write does not invalidate the receiver's fields.
index. For these returned-reference writes, unrelated properties and known
array prefixes remain precise. Accessor applicability uses the shared object
reference owner for literal definition order, own-property shadowing, known
array elements and explicit literal prototypes. Getter invocation effects
project write and call operands through the same local alias and `this` owner
used by returned references. Reads of another selected getter continue that
effect walk. A receiver reached by these effects becomes uncertain on an
ordinary or optional read, compound assignment or update. A simple assignment
or deletion does not invoke the getter. Selected setters invalidate
the receiver on assignment, including through known array and prototype paths.
Inert getters and ordinary data slots retain their supported precision.
These are conservative source-derived effects, not execution results. The
analysis does not establish general accessor purity, model every class or
reflectively installed descriptor, or cover getter reads introduced by object
spread/destructuring and all implicit coercion or disposal invocations. Absence of a later explicit write does not establish getter purity;
unsupported invocation forms can still retain stale receiver-field facts.
Nested callable returns are not assigned to their parent callable. Projected
graph observations carry source ranges but never source text.
+32 -1
View File
@@ -252,7 +252,8 @@ Selected bounded text is then parsed as inert data to recover:
- Electron preload paths and renderer files visible in static syntax;
- local HTML script entrypoints;
- Webpack and Rspack chunk registrations and module factories represented as
AST literals;
AST literals, plus esbuild `__commonJS` module tables; recovered esbuild
CommonJS exports use its factory's `exports`-then-`module` parameter order;
- static imports, dynamic imports, CommonJS `require` calls, workers, and
service workers (only `new Worker(new URL(path, import.meta.url))` resolves
relative to the calling module; string worker and service-worker paths use
@@ -513,6 +514,11 @@ inventory digest before parsing, and malformed ASAR operations return typed
format diagnostics that retain the local container path. Native add-ons are
represented by metadata only.
POSIX directory names are read as raw bytes and admitted as logical paths only
when they round-trip through UTF-8. Invalid names are left untouched, retained
siblings remain inventoried, and traversal coverage is marked unknown with the
first rejected basename bytes identified in hexadecimal.
## Bounds and coverage
The reconstruction keeps internal resource safeguards for artifact entries,
@@ -636,6 +642,13 @@ execution and memory limits continue to determine examined coverage. Raising onl
a node capacity does not raise the unknown capacity. Graphs without unknown
omissions omit this optional coverage field, preserving their existing identities.
These node and unknown capacities bound retained graph projection. Parsing,
static analysis and semantic fact construction precede that projection for an
admitted source. Lowering the capacities can reduce projection and retention
work, but does not bound upstream semantic-analysis time or memory. Select fewer
`source_paths` to reduce the sources analyzed; heap and deadline controls set
separate resource failure boundaries with the runtime-specific limits above.
While sources remain, the sequential worker reserves one next compact static
checkpoint at its 8 MiB bound with the same 6x retention expansion used at
admission. Earlier checkpoints are already charged to retained bytes; the
@@ -756,6 +769,24 @@ URL targets before selecting a later package-local file; an unsupported bare
package or recursive alias keeps its unresolved outcome rather than selecting
a later fallback.
For a proven `createRequire(new URL(relativeFile, import.meta.url))` call,
relative `require()` edges use the selected filename's directory and retain the
source-relative `resolution_base_path` on the graph edge. This follows Node's
`createRequire(filename)` contract; the path is a loader-resolution fact and
does not change the source file or call-site Evidence identity. If the filename
cannot be proven, REA keeps the require target unresolved rather than silently
resolving it from the caller's directory.
Module-reference graph edges, including native-addon `loads` edges, retain
`properties.operation`: `load` identifies
an import or loader invocation that may execute the target; `resolve` identifies
a proven `require.resolve()` lookup that discovers a target without executing it.
Both retain the exact specifier, resolution mode, selected resolution base, and
call-site Evidence. Resolve-only edges do not propagate application ownership or
create executable-import gaps when their targets are missing. A modified
`.resolve` member remains an uncertain `load`, because its replacement may execute
code. These static facts do not establish that a call runs at runtime.
ESM relative module paths, legacy package subpaths without exports, and selected
package exports targets use URL suffix handling and one percent-decoding pass.
Bare specifiers retain their complete query/fragment text for exports key matching.
+11
View File
@@ -78,6 +78,17 @@ can differ from the routing base. Every affected occurrence keeps
is explicitly a path-name heuristic; `page` uses page-file or structured/typed configuration support. Name-only
`route`/`addRoute` calls remain `unknown` because they can also register server APIs.
Next page-source segments keep their filename spelling when converted to route
patterns. Next's [page-path conversion](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/shared/lib/page-path/absolute-path-to-page.ts#L21-L39)
removes page extensions and the `index` tail without URI-decoding segments, so
ordinary percent sequences are not decoded: `%61` and `a` remain separate
candidates and a literal `%` is retained. Its [App pathname normalizer](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/server/normalizers/built/app/app-pathname-normalizer.ts#L5-L18)
specifically applies the [underscore normalizer](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/server/normalizers/underscore-normalizer.ts#L2-L9),
which maps `%5F` to `_`; private-folder checks apply to the source spelling
before that escape is mapped. When classifying a chunk URL, REA reads its
pathname, then applies these same Next-specific rules. This is static source-path
interpretation, not browser request-path decoding or reachability.
Known capture omissions remain inline in `summary.reported_capture_omissions`
and queryable as `unknowns` of kind `capture-omission`. Each keeps a stable gap ID,
its script-record/script-source unit, and the reported count (including `null`
+59 -7
View File
@@ -51,6 +51,11 @@ This keeps the complete source in the session and returns a distinct linked
evidence, metadata status, module/MVID, assembly, frameworks, returned-row counts,
coverage/issues and limitations. No assembly is loaded or executed.
ECMA-335 permits empty metadata streams to be omitted. An absent `#Blob`
stream therefore does not prevent module or assembly identity from being read
when all blob indexes are zero; a nonzero reference to that missing heap is
reported as a missing-stream issue.
Use `normalized_result.parent_evidence_id` to page original rows:
```json
@@ -177,6 +182,9 @@ declares API compatibility across multiple .NET implementations. REA preserves
that target in `target_frameworks` and leaves `runtime_family` as `unknown`
unless independent markers identify a runtime family, such as a ReadyToRun
signature.
When ReadyToRun is observed, `runtime_family` remains `modern-dotnet` even if
Unity or target-framework hints are also present; the separate hint evidence
is retained in the classification evidence list.
### Resource admission
@@ -242,7 +250,13 @@ remain an assembly-wide inventory, so output size is not guaranteed to be small.
Coverage describes the selected members. Comparisons leave members outside the
opposite observation's selection unknown, and application graphs report their
selected scope as partial. Neither makes an absence claim about the whole
assembly. Older member Evidence without `selection` remains readable.
assembly. Older member Evidence without `selection` remains readable. The
`method_rows` interval records the examined zero-based MethodDef offset, row
count and full table total before type selection. A complete observation can
cover only that interval. Comparison keeps omissions from a method page unknown:
build-local row positions cannot prove member absence across builds. Historical
Evidence without this interval also cannot establish method absence; observed
pairs and field inventory comparisons remain available.
For MCP callers, request `detail: "summary"` and page the `types` collection
using the returned parent Evidence ID. Select one or more returned full names
@@ -311,6 +325,19 @@ native/runtime questions retain their own authority and coverage.
## Member comparison
Call edges and field accesses retain `il_offset` as their instruction site within
the caller's build-local method token and artifact identity. Their
`target_identity_sha256` combines the existing enclosing-type nominal digest,
member name and resolved decoded signature. Comparison joins edges to their
instruction sites rather than array positions and compares these identities;
display labels alone cannot establish equal targets. Same-named nested types
under different enclosing types and different overload signatures stay distinct.
This is nominal metadata identity, not assembly resolution or runtime binding.
Historical edges without a site or identity, unresolved/cyclic type scopes,
TypeSpec owners and unresolved MethodSpec instantiations retain unknown target
equivalence. Valid MethodSpecs combine the base member identity with the
resolved instantiation signature, preserving generic argument distinctions.
Names and semantic labels do not establish durable member identity. Comparison
preserves observed differences, structural inferences, and unknowns separately.
@@ -386,12 +413,23 @@ constants, and bounded shape context. It reports all candidates at the winning
score and remains ambiguous when the evidence does not distinguish them. The
digest does not itself remap metadata tokens.
Matched methods also compare aligned call sites. `call-shape` reports opcode and
operand-kind differences; `call-target` reports differences in the observed
target declaring-type or method name from `call_edges`. Target tokens remain
build-local and are not compared across artifacts. Missing or unaligned call
edges and missing target names leave that facet unknown; equal names do not
establish overload identity or runtime dispatch.
Matched methods also compare aligned references. `call-shape` and
`field-shape` report opcode and operand-kind differences; `call-target` and
`field-target` report differences in the observed target declaring type or
name from `call_edges` and `field_accesses`. Target tokens remain build-local
and are not compared across artifacts. Missing or unaligned edges and missing
target names leave that facet unknown; equal names do not establish overload
identity or runtime dispatch. `exception-shape` compares clause flags and
ranges. `exception-target` compares the nested nominal catch-type name digest;
for resolved TypeDefs it also compares the enclosing assembly identity. A
different known name or assembly scope establishes a change. Equal TypeRef
names remain unknown because their resolution scope is not resolved here.
Exception tokens remain in the source facts but are not cross-build identity;
an unresolved catch type leaves that facet unknown. The method body retains the
display `class_type`, nested `class_type_name_sha256`, and whether its local
token identifies a TypeDef. A TypeDef digest alone does not prove identity
without matching assembly identity. Missing assembly identity and equal TypeRef
names leave the comparison unknown.
Tokens are always remapped through observed structure. A caller cannot carry
`0x06001234` into a new MVID and assume it names the same method.
@@ -447,6 +485,16 @@ only when both identities are available and their format-specific comparison
establishes a difference. ELF module names retain case, PE module names compare
case-insensitively, and a case-only match for Mach-O or an unknown format stays
unresolved because the available Evidence does not establish case behavior.
For PE imports, `.dll`, `.exe`, a trailing period, and absolute paths suppress
appending `.dll`; other names may also be tried with `.dll`. Unix loaders use
the platform suffix (`.so` or `.dylib`) and `lib` prefix variations. A suffix
already present at the end of the name or before a period still gets an
appended-suffix variation, so Linux can match `nativedep.so.6` with
`libnativedep.so.6`. The `lib` prefix is omitted when the name contains `/`, and
absolute paths are used as supplied. POSIX paths keep backslashes as literal
characters. These candidates follow [Microsoft's native library loading rules](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/native-library-loading).
These filename rules compare declared and observed identities; they do not
establish which library the runtime loaded.
Metadata tokens are not native addresses. Native body mapping for C++/CLI,
ReadyToRun, NativeAOT, and IL2CPP requires separately verified format and provider
@@ -516,6 +564,10 @@ preserving readable member and attribute facts.
The retained inverse indexes are admitted against the existing representation
budget before allocation.
Custom-attribute `type_name` uses `Outer+Nested` when its local TypeDef or
TypeRef enclosing chain is readable. It remains `null` when the constructor's
type or required enclosing facts cannot be resolved.
`exported_types` reports ExportedType rows as declared. Facade assemblies such
as .NET `System.Runtime.dll` provide their types this way: `forwarder: true`
marks a type forwarder, and `implementation_token` names the AssemblyRef, File,
+59 -10
View File
@@ -44,12 +44,18 @@ root deltas. Missing graph members yield `truncated`; unknown traversal coverage
or unavailable occurrence hashes retain uncertainty. Integrity mismatches remain
contradictions. Static artifact equality does not establish unchanged runtime
behavior.
Source-map relation deltas include the content identity of the mapped source,
so an unchanged map file is reported when its target source bytes change.
Exhausted traversals with deliberately omitted filesystem entries retain
`unknown` coverage and identify each omitted path/type in `limitations`; FIFO,
socket and device entries are not opened. Case-distinct archive members remain
separate observations. Their case-insensitive materialization diagnostic does
not reduce traversal coverage when both spellings were read.
PyInstaller inventory also preserves distinct table rows that share a logical
path, including decoded children beneath a colliding file/directory path.
Inventory coverage describes observed archive facts; extraction still refuses
destinations where those paths cannot coexist.
Current inventory Evidence requires scoped `coverage` with a `complete` or
`unknown` status. Consumers use this producer-owned fact and reject records
@@ -80,6 +86,12 @@ can observe backend recovery; healthy dossiers keep their existing cache policy.
`procedure_pseudo_code` still fails for that function. Hopper's unavailable reference-kind metadata and reported unresolved
calls still prevent equality conclusions for those dimensions. Analysis profiles
version these declarations so older snapshots are not reused as current results.
For `strings_names`, a string marked with `decoding.available: false` keeps its
provider display text in the source Evidence, but that text is not compared as
decoded content. If no independently known string, name, or literal fact differs,
the dimension remains `unknown` with the decoding reason. Known content and typed
string-object differences can establish a change; missing typed metadata remains
unknown, and `encoding_status` describes provenance rather than content.
IDA declares the facets its selected adapter cannot provide; an empty
unsupported array does not prove observed absence.
Evidence and bundle imports reject unsupported fields in their envelopes,
@@ -1000,6 +1012,12 @@ materialized; extraction fails as `unavailable` with that logical path.
Extraction materializes regular files exposed by the active archive reader.
When that reader exposes a nested archive as a file, extraction copies the
archive file; inspect and extract that file separately to unpack its contents.
Extraction retains inventory and materialization traversal limitations in its
result and outer Evidence. If either traversal has unknown coverage, the
observed regular files can still be copied, but the result does not establish
a complete source copy. For example, a POSIX filename that is not valid UTF-8
remains omitted with its raw-name explanation; it is not replacement-decoded
into an output filename.
Packaging tools commonly sign or strip `.asar.unpacked` native binaries after
writing the archive header, so their declared hashes no longer match. Electron
@@ -1152,20 +1170,43 @@ includes all dimension limitations. Equal CFGs establish only equal observed
block sizes and successor relationships. Text from different provider
identities remains unknown.
The `references` dimension preserves each edge's direction, endpoint identity
and available kind facts. Within one artifact it compares exact raw endpoint
coordinates, including the address space; hexadecimal case and zero padding
do not change identity. Resolving an endpoint to a containing procedure does
not replace the raw coordinate. Unparseable coordinates stay unknown.
The `calls` and `references` dimensions share endpoint identity and name
ambiguity rules. Calls retain procedure entry coordinates and direction;
references retain each edge's raw source and target coordinates, direction and
available kind facts. Resolving a reference endpoint to a containing procedure
does not replace its raw coordinate. Hexadecimal case and zero padding do not
change coordinates; address spaces remain distinct.
Coordinate equality requires equal artifact digests and formats, a known equal
architecture, and matching provider-owned native coordinate commitments. Those
commitments retain the exact selected Mach-O image where available and a digest
of the loader's mapping inputs. They exclude unrelated analysis settings, so
changing an analyzer option alone does not invalidate coordinate correspondence.
Equal coordinates in different slices or mappings cannot establish equality.
Hopper's default executable profile and Ghidra's supported deterministic imports
publish these commitments. Hopper loader overrides and attached databases, IDA
profiles, Ghidra universal-image imports without an exact slice commitment, and
older or imported Evidence without a commitment leave coordinate identity
unknown. Unparseable coordinates also remain unknown.
Across different artifacts, endpoints use nonempty stable procedure names.
Missing or address-generated names remain unknown. A name observed at multiple
procedure entries across either dossier's call and reference facets retains its
label constraint but cannot establish equal endpoints. Same-container records
without a common coordinate context likewise cannot establish equality from
matching labels. Procedure labels do not carry authoritative symbol-origin
facts, so recognition of generated names remains conservative and cannot prove
whether every provider label was generated or explicitly named.
Across artifacts, reference endpoints use stable procedure names. Missing or
address-generated names remain unknown. A name observed at multiple procedure
entries retains its label constraint but cannot establish equal endpoints.
Pairing preserves duplicate counts and considers alternative compatible
pairings; array order never identifies an edge. Compatible records with
unknown endpoint identity or kind facts remain `unknown`. Independent known
pairings; array order never identifies an edge. Compatible records with unknown
endpoint identity or reference kind facts remain `unknown`. Independent known
facts that contradict every possible pairing still establish `changed`.
Source Evidence retains the original addresses, names and unavailable facts.
Reconstruction verification uses the selected dimension's status: a same-name
call retargeting in a common context fails a calls claim; an unresolved slice or
mapping cannot pass it. Empty function labels leave `function_match` ambiguous
and the identity dimension unknown.
### Comparison input shapes
@@ -1186,6 +1227,14 @@ invoking the handler. Domain validation additionally checks
relationships such as event sequence references and retained counts; JSON
Schema does not express those cross-field invariants.
Comparison also preserves the capture's coverage. A partial HTTP inventory can
establish an observed difference against a complete peer, but an omitted request
does not establish removal. Partial WebSocket streams do not establish frame
alignment. WebMCP frame IDs and tool keys are local to a capture; semantic owner
facts and known counts determine correspondence. Retained script digests can
establish source changes. See [browser comparison semantics](browser-observation.md)
for value uncertainty and storage-fingerprint recapture requirements.
## Generated catalog
Custom clients upgrading from 6.1 should follow the
+103 -5
View File
@@ -30,9 +30,11 @@ result records the exact selected path, file size, header length, header SHA-256
bytes read and read scope. GGUF alignment padding is outside this prefix.
`artifact.sha256` and the Evidence subject are null: a header digest is not a
commitment to the weights. Different files can have identical headers and sizes.
The reported GGUF tensor-data offset is aligned after the tensor directory,
including when the directory contains no tensors; `header_length` still ends at
the directory, before that padding.
The reported GGUF tensor-data offset follows the aligned tensor directory when
padding is present. For zero-tensor metadata-only files that end at the
directory, it is the directory end; no tensor payload is implied. A partial
alignment gap is rejected. `header_length` still ends at the directory, before
any padding.
`hash: "full"` additionally streams the remaining file bytes through SHA-256.
It returns a full-file digest and Evidence subject, while retaining the same
@@ -42,6 +44,12 @@ check descriptor and selected-path identity and metadata before completion;
this detects observed replacement or modification, not adversarial modification
that defeats filesystem metadata checks.
If full-file hashing fails or is cancelled after either format's header has
been validated, partial Evidence retains the parsed metadata and tensor directory.
Header coverage is complete; digests and final file identity remain unknown.
Safetensors header parsing is atomic, so interruption before validation does not
expose unvalidated JSON rows.
## Facts and selected queries
GGUF reports version, byte order, alignment, typed metadata, and each tensor's
@@ -49,9 +57,21 @@ name, original type ID, dimensions, relative data offset, file offset and derive
byte length. Alignment accepts every positive UINT32 multiple of eight, including
non-powers of two. Partial Evidence retains explicit alignment once validated;
the default alignment becomes known after metadata completes. Cancellation
during tensor parsing or hashing retains that fact without claiming finalized
digests or file identity. Metadata keys use ASCII lower-snake-case hierarchy components;
retains completed declarations with unknown digests and unverified final file
identity. Metadata keys preserve their spelling and use lowercase
ASCII alphanumeric components separated by dots, underscores or hyphens;
standard numeric components such as `general.base_model.0.name` are preserved.
Hyphenated components are retained for compatibility with the published
llama.cpp v0.7.0 writer, which emits keys such as
`gpt-oss.context_length` and `wavtokenizer-dec.context_length`; the current
[GGUF specification](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md#standardized-key-value-pairs)
restricts `general.architecture` values to letters and digits.
This documents producer output that conflicts with that written restriction,
not a claim that the specification permits hyphens. See the pinned
[WAVTOKENIZER_DEC](https://github.com/ggml-org/llama.cpp/blob/620d2225cc436ab0b001e3200e590e3b286ef9ab/gguf-py/gguf/constants.py#L1447)
and [GPT_OSS](https://github.com/ggml-org/llama.cpp/blob/620d2225cc436ab0b001e3200e590e3b286ef9ab/gguf-py/gguf/constants.py#L1466)
architecture names, [context-length key template](https://github.com/ggml-org/llama.cpp/blob/620d2225cc436ab0b001e3200e590e3b286ef9ab/gguf-py/gguf/constants.py#L110),
and [writer method](https://github.com/ggml-org/llama.cpp/blob/620d2225cc436ab0b001e3200e590e3b286ef9ab/gguf-py/gguf/gguf_writer.py#L698-L699).
Scalar metadata is complete. Arrays have declared element types,
counts and at most eight preview elements per array, with an explicit
`preview_complete` flag. This includes tokenizer vocabularies: omitted values
@@ -61,6 +81,84 @@ Nonfinite floating metadata uses `NaN`, `+Infinity` or `-Infinity` with its
declared float type; negative zero is `"-0"`. Unknown tensor types retain their numeric IDs and null byte
lengths; affected totals and storage validation remain unknown.
Q8_1 (type ID 9) has two released layouts: 40 bytes per 32 elements in
[llama.cpp b1500](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/ggml-quants.h#L49-L55)
and 36 bytes in the pinned
[modern ggml layout](https://github.com/ggml-org/ggml/blob/da8e2a54b919557b816b1c2beb432ccb766f7ce0/src/ggml-common.h#L242-L253). Both use GGUF v3 and
quantization version 2, so those declarations do not select a storage layout.
REA retains the known type, shape, parameter count and starting offsets, with
`byte_length: null`, `storage_validation: "unknown"`, partial storage/range
coverage and an explicit limitation for nonempty Q8_1 tensors. Per-type and
overall byte totals remain unknown where affected. Padding, the next tensor's
offset and file size do not establish a layout; retained queries preserve the
same uncertainty. Empty Q8_1 tensors still have a derived zero-byte length,
and the known 32-element row-block constraint remains enforced.
K-quantization IDs 10–15 also have incompatible released layouts. The
[b1500 build option](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/CMakeLists.txt#L88)
`LLAMA_QKK_64` selects `GGML_QKK_64`; the same release's default and
[b3001](https://github.com/ggml-org/llama.cpp/blob/c429b33beb35f13934a4dfbe0c138d30b45e5d54/ggml-common.h)
use 256-element blocks. The
[b1500 structures](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/ggml-quants.h#L61-L169)
give these source-derived sizes:
| ID | Stable name | Bytes per 64 elements, K64 | Bytes per 256 elements, K256 | Padded extent for 256 elements, either layout |
| --- | ----------- | -------------------------- | ---------------------------- | --------------------------------------------- |
| 10 | Q2_K | 24 | 84 | 96 |
| 11 | Q3_K | 28 | 110 | 128 |
| 12 | Q4_K | 38 | 144 | 160 |
| 13 | Q5_K | 46 | 176 | 192 |
| 14 | Q6_K | 54 | 210 | 224 |
| 15 | Q8_K | 76 | 292 | 320 |
Both builds write GGUF v3 and quantization version 2. The
[quantizer](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/llama.cpp#L7430-L7492)
writes `general.file_type` directly from the selected `llama_ftype`, without
encoding the K64 build option. `GGML_QNT_VERSION_FACTOR` is defined in the
header but is not used by this GGUF serialization path. Global file type also
does not identify individual tensor types: the quantizer
[retains unselected tensors](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/llama.cpp#L7561-L7586).
The [writer pads each payload](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/ggml.c#L19549-L19565)
to the default 32-byte alignment; the final column shows why even the same file
extents need not distinguish these layouts.
REA preserves each K type's name and enforces the common 64-element row
multiple, which does not establish a storage block size. Nonempty K tensor
lengths and affected totals are now unknown, including common modern files.
Empty K tensors still have derived zero-byte storage.
Integer IDs also shifted each time IQ types were added. These released
`ggml.h` snapshots all declare GGUF v3 and quantization version 2:
| Released snapshot | I8 ID | I16 ID | I32 ID |
| --------------------------------------------------------------------------------------------------- | ----- | ------ | ------ |
| [b1500](https://github.com/ggml-org/llama.cpp/blob/57ad015dc3011b046ed5a23186c86ea55f987c54/ggml.h) | 16 | 17 | 18 |
| [b1803](https://github.com/ggml-org/llama.cpp/blob/36e5a08b203542dca53cca4eaf172c5dc4bbc991/ggml.h) | 17 | 18 | 19 |
| [b1871](https://github.com/ggml-org/llama.cpp/blob/bb0c1392479398f9aba86d9ec98db0b95ede6e6d/ggml.h) | 18 | 19 | 20 |
| [b2100](https://github.com/ggml-org/llama.cpp/blob/4aa43fab569215a13495a7f1a0f8afc541b16d03/ggml.h) | 19 | 20 | 21 |
| [b2172](https://github.com/ggml-org/llama.cpp/blob/bd2d4e393b2b7d2a1b2e201058e26017c9728ead/ggml.h) | 20 | 21 | 22 |
| [b2221](https://github.com/ggml-org/llama.cpp/blob/a14679cc30c785e75d38028bae6ec39c6209ddef/ggml.h) | 21 | 22 | 23 |
| [b2253](https://github.com/ggml-org/llama.cpp/blob/4c4cb30736582cacb1a164a9d4bc8e17b1014be7/ggml.h) | 22 | 23 | 24 |
| [b2275](https://github.com/ggml-org/llama.cpp/blob/a33e6a0d2a66104ea9a906bdbf8a94d050189d91/ggml.h) | 23 | 24 | 25 |
| [b2300](https://github.com/ggml-org/llama.cpp/blob/6ea0f010ff6967034528d9e0b8330b9b0f0b7c13/ggml.h) | 24 | 25 | 26 |
[Upstream PR 6050](https://github.com/ggml-org/llama.cpp/pull/6050) later pinned
integer values explicitly. Modern IDs 16–23 mean IQ2_XXS, IQ2_XS, IQ3_XXS,
IQ1_S, IQ4_NL, IQ3_S, IQ2_S and IQ4_XS, respectively. Thus every ID 16–25 has
multiple released meanings. REA keeps its numeric `type_id`, uses
`AMBIGUOUS_<id>` for `type`, and records the released candidate names in a
limitation. Nonempty byte lengths are unknown. Empty declarations still derive
zero-byte storage, while their textual identity remains ambiguous. Type filters
match the reported `AMBIGUOUS_<id>` label. ID 26 remains I32 across these
snapshots; later stable entries retain their existing interpretation. This audit
does not establish compatibility with every producer fork or future revision.
The inventory does not select a historical layout from model names, global
file types, padding or guessed producer provenance. A source revision together
with its build configuration can establish an external layout fact; these
headers do not carry that identification, and REA currently has no layout
selection input. Authored regression fixtures use the source-derived sizes
above; historical executables were not run.
Safetensors reports the header extent, exact string `__metadata__` values, dtype,
shape and declared data ranges. A null `__metadata__` is treated as omitted;
unknown fields on tensor entries are ignored within the reader's 32-level JSON
+69 -3
View File
@@ -352,6 +352,12 @@ Available CPU type, subtype/capability bits, byte range, and alignment remain
checked against the source bytes. Inventory records source ranges and hashes, and
extraction copies those ranges without running the target. This does not
expand any analysis provider's supported architectures.
Apple lipo's hidden ARM64 compatibility row is retained separately from the
raw `nfat_arch` count and admitted only from its extra FAT table slot when its
in-range Mach-O header matches the row. `list_architectures` preserves the
per-row `hidden` marker from lipo output. Mach-O image and Electron fuse
inspection include the row in architecture selection while retaining that
marker in their reported slices.
`inspect_plist.format` describes the selected source, not plutil's conversion
format. Sources explicitly identified as XML, binary or JSON retain those
@@ -427,9 +433,16 @@ complete observation.
its outcome. Paths outside the analyzed root, including shared-cache system
libraries, stay undetermined. See
[Apple application analysis](apple-application-analysis.md#dylib-load-resolution).
Executable-root discovery preserves traversal uncertainty, including
undecodable filenames and skipped symlinks, as partial coverage. Known
executable roots remain available; unexamined names and symlink targets
cannot establish an empty root set or the absence of other roots.
- `inspect_asset_catalog` / `rea inspect-asset-catalog <app>` reads compiled
`Assets.car` metadata through macOS `assetutil --info`. Catalog digests, raw
rendition fields, pagination and exact UI resource-name matches are returned.
`inventory_coverage` describes traversal of source entries and remains
`unknown` when the reader cannot represent a filename. Represented symlinks
are inventoried but their targets remain uninspected, with a limitation.
Image extraction and undocumented rendition interpretation are unsupported.
- `inspect_keyed_archive` / `rea inspect-keyed-archive <plist>` reads XML or
binary Foundation keyed archives without instantiating classes. For an active
@@ -447,6 +460,19 @@ complete observation.
emits no bytes. Character references retain their source spelling for data
decoding. Non-ASCII content, child markup and self-closing `<data/>` are format
failures; explicit `<data></data>` represents empty data.
Each XML dictionary key must have a following value element; a final key
without one is malformed. An explicit empty string remains a value.
Binary UID payloads may use nonminimal byte widths. REA normalizes them only
in a private decoder copy; archive digests still identify the original bytes.
Positive binary-plist offset and object-reference table widths are read from
the source; nonstandard widths are normalized only in that private copy for
the pinned decoder. [Foundation's table checks](https://github.com/swiftlang/swift-corelibs-foundation/blob/swift-6.2.3-RELEASE/Sources/CoreFoundation/CFBinaryPList.c#L759-L789)
accept positive widths subject to extent/reference bounds, and its
[sized-integer reader](https://github.com/swiftlang/swift-corelibs-foundation/blob/swift-6.2.3-RELEASE/Sources/CoreFoundation/CFBinaryPList.c#L707-L730)
handles non-power sizes. Widths over eight bytes follow its unsigned 64-bit
accumulator semantics.
UID values above JavaScript's exact integer range return an unsupported-target
outcome instead of rounded reference identities.
UID 0 cannot distinguish conditional nil from ordinary nil. Limits are
64 MiB of input, 200,000 objects/references and 128 nesting levels.
XML dictionary entries decoded as `__proto__` are omitted with an explicit
@@ -490,6 +516,13 @@ complete observation.
Archives beyond either aggregate budget are omitted with their paths and an
explicitly partial `archive_decode` facet; previously decoded documents and
their identities remain available.
The same facet is partial when source traversal cannot represent a filename
or an archive-shaped symlink was skipped. Unknown filename coverage has no
invented document count; known skipped nib and storyboard links are counted
as omitted. A readable empty bundle can report complete zero-document
coverage. Source I/O and identity failures retain their acquisition error
instead of being labeled invalid archive content. Rejected plist decodes
release their representation reservation for later documents.
- `inspect_native_dispatch_metadata` /
`rea inspect-native-dispatch-metadata <app-or-binary>` prefers a validated macOS Mach-O byte reader. It decodes
64-bit little-endian Objective-C class/metaclass records, superclass pointers,
@@ -551,7 +584,8 @@ complete observation.
So an external superclass resolves from its `_OBJC_CLASS_$_` bind (for
example `NSObject`). The `pointer_fixups` coverage facet names the mechanism.
Class properties preserve each encoded attribute as a raw `{name, value}`
pair, for example `{ name: "N", value: "" }`, and report atomicity on the
pair, including commas inside C++ template arguments in the `T` type, for
example `{ name: "N", value: "" }`, and report atomicity on the
property: `N` means `nonatomic`; a complete typed attribute string without
`N` means the Objective-C default `atomic`;
missing or incomplete attribute metadata leaves `atomicity` and any
@@ -760,6 +794,15 @@ unresolved runtime/value semantics remain visible in results.
## Interface Builder hierarchy coverage
Bundle archive selection preserves directory traversal uncertainty, including
filenames that cannot be decoded as UTF-8. These names are not guessed or
counted as known archive omissions. Recognized symlinked `.nib` entries are
not followed; they retain their paths and reader limitations as counted archive
omissions. Either gap makes `archive_decode` partial and the graph truncated
while preserving decoded documents. Its `examined` count remains actual decode
attempts, and asset-catalog UI joins retain their existing incomplete-graph
qualification.
A document's `hierarchy_complete` flag follows its own hierarchy coverage.
If graph limits omit a hierarchy source or destination, or prevent a hierarchy
edge or traversal from being retained, that document reports partial hierarchy
@@ -808,8 +851,31 @@ attributes; exact and ambiguous projection counts cover emitted attribute values
Integral reals stay numbers, and ambiguous or incompletely observed values retain
explicit precision limitations. Before plist decoding, the existing aggregate
decode budget reserves estimated room for number metadata and attribute
projection; these estimates are not a hard process-memory bound. Native
NIBArchive decoding is unchanged by this plist-specific representation.
projection; these estimates are not a hard process-memory bound. Keyed-archive
object identities always come from their original `$objects` table indices.
Authored `objectID`, `object-id`, `archiveUID` and `id` fields remain ordinary
object data and do not change inventory, reference, connector or hierarchy
identity. This changes keyed-archive graph IDs that previously used authored
`objectID` or `object-id` values: `interface_builder_object_id` and linked node
IDs now use the decimal table UID, while the authored values remain in attributes.
For example, an authored `objectID: "2"` on table entry 1 now identifies object
`"1"`; its graph node ends in `object:["1"]`. Nonkeyed ibtool dictionary IDs
and native NIBArchive IDs keep their existing source-specific meaning.
Connector endpoints require encoded UID references to archived objects; strings
and dictionaries that merely resemble resolved reference stubs do not establish
identity. Inline hierarchy dictionaries have no table identity and count as
omitted references, while independently referenced children remain available.
The UID identity rule follows the
[Foundation unarchiver's table lookup](https://github.com/swiftlang/swift-corelibs-foundation/blob/28249ced079d8e7ec03b821c039bf9868738c3fc/Sources/Foundation/NSKeyedUnarchiver.swift#L213-L223).
UIKit keyed nibs select their explicitly named
`UINibTopLevelObjectsKey` collection; generic `root` is used when that named key
is absent. Unsupported or dangling named roots remain incomplete rather than
falling back to another collection. The key spelling follows the independent
[WinObjC UINib reader](https://github.com/microsoft/WinObjC/blob/94f6b5bfdc09d945e30607b56124b6f94a5b9d7b/Frameworks/UIKit/UINib.mm#L104-L109).
NIBArchive float32 and float64 values retain non-finite values and negative zero
through projection as the same typed real wrappers used for plist values; per-document limitations
count those values. NIBArchive nil remains JSON null.
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`.
XML dates require a complete UTC spelling with fixed two-digit month, day and
+6 -3
View File
@@ -112,9 +112,12 @@ wall deadlines are 300 s for dissection, 60 s for metadata and 20 s for probes.
Native ownership preparation has its own bounded lifecycle.
Malformed/oversized output returns no partial success. Unvalidated partial JSON
is not returned as facts. Cancellation reaches probes, acquisition, commands and
projection. Cleanup failure returns a typed error, retains completed Evidence
when available, and retains owned resources for retry on close.
is not returned as facts. Invalid UTF-8 in either `capinfos` or TShark stdout or
stderr is an unreadable-output error; replacement-decoded text is not parsed as
producer evidence. A literal U+FFFD character in valid UTF-8 remains text.
Cancellation reaches probes, acquisition, commands and projection. Cleanup
failure returns a typed error, retains completed Evidence when available, and
retains owned resources for retry on close.
## Verification scope
+5 -2
View File
@@ -49,8 +49,11 @@ each `SignerInfo` must carry a non-empty signature value. Each embedded X.509
certificate must include `subjectPublicKeyInfo` and `signatureValue`. Decoded
certificate name text, serials, validity spellings, and subject key identifiers
share a 64 KiB budget per PKCS#7 payload. Unsupported certificate name
encodings, including TeletexString (T.61) and GeneralString, fail explicitly
rather than returning names decoded with an assumed character set.
encodings, including GeneralString, fail explicitly. TeletexString (T.61)
values are rendered with the ISO-8859-1 compatibility mapping used by OpenSSL;
this is not a full T.61 decoder, so the result includes a limitation whenever
that mapping is used. RFC 5280 retains TeletexString for backward compatibility
and describes its repertoire as ASCII-like Latin with additional characters.
The structural CMS reader accepts definite lengths and BER indefinite lengths
for constructed values, including the outer `ContentInfo` sequence. Primitive
+34 -8
View File
@@ -6,18 +6,38 @@ MCP. They read archive bytes without launching the application. Bytecode
source decompilation, encrypted PYZ recovery, zipapp and PEX analysis are outside
this capability.
`extract_artifact.coverage` preserves the inventory producer's source traversal
scope and certainty. Successful publication proves that the selected inventoried
regular files were written and verified; it does not prove that every source
entry was representable or every embedded payload was decoded. For example, a
directory containing an invalid UTF-8 filename can publish its representable
files successfully while coverage remains `unknown`. Extraction retains the
inventory's command provenance and limitations, plus the selected occurrences'
path-qualified derivation notices. A reconstructed `.pyc` header therefore stays
identified as derived bytes in the extraction result.
## Facts and focused queries
The root occurrence's `format_metadata` records the CArchive layout (2.0 or
2.1), archive and cookie offsets, table extent, entry count, raw Python version,
normalized major/minor when known, library name and encryption marker. Member
metadata retains the original name, table index, type, compression flag,
relative offset and known directory record extent. Logical paths use archive
path rules; the source name retains its spelling, including backslashes.
relative offset and known directory record extent. Logical paths follow the
carrier's path dialect: backslashes separate components for PE carriers, while
they remain literal filename characters for POSIX carriers. The source name
always retains its stored spelling. If the carrier dialect is unconfirmed, REA
preserves backslashes verbatim instead of inferring POSIX semantics.
`relative_offset` is relative to the CArchive start for CArchive rows, or to
the decoded PYZ image for PYZ rows. A CArchive directory record starts at
`archive_start + toc_offset + directory_record_offset`; its size is
`directory_record_size`. PYZ row directory extents remain unknown.
CArchive type `b` retains PyInstaller's executable-member declaration in the
generic `executable` fact; type `x` remains non-executable. This comes from the
archive type code, not the filename or member contents.
Distinct CArchive rows retain their table-index identities even when they
share a logical path. Inventory can preserve both a file and a decoded PYZ
directory at that path, including the PYZ children; extraction refuses a
destination conflict instead of silently omitting those children.
Namespace packages remain metadata declarations at their projected query path;
they do not claim a destination directory. Their `relative_offset` and
@@ -105,7 +125,9 @@ returns a resource failure.
Supported final Python bytecode magic values cover Python 3.7 through 3.14.
Supported marshal directory values are lists, tuples, strings, integers and
bytes. Directory values follow the identified CPython marshal semantics; for
bytes, including interned Unicode strings used for module names. Unicode strings
use CPython's UTF-8 `surrogatepass` encoding, preserving lone surrogate code
points in decoded module names. Directory values follow the identified CPython marshal semantics; for
example, a long integer with a zero most-significant digit is malformed and is
not normalized into a valid table value. An unsupported magic or malformed or
unsupported directory stays an opaque member with a limitation and
@@ -122,11 +144,15 @@ The directory declares compressed module lengths, so uncompressed declared
sizes and source spans inside decoded data remain unknown.
PYZ modules receive a derived 16-byte `.pyc` header from their own supported
magic. CArchive bytecode receives that header only when every sibling PYZ magic
and the cookie's Python version agree. Otherwise it retains its original bytes
and `bytecode_header: "unknown"`. `bytecode_magic` and graph derivation edges
identify synthesized headers; member digests cover the resulting derived files.
These headers do not establish successful execution or decompilation.
magic. CArchive scripts receive that header only when every sibling PYZ magic
and the cookie's Python version agree. Under the same agreement, CArchive
modules (`m`/`M`) receive a header only when their decompressed prefix identifies
headerless marshal code. Legacy modules that retain a `.pyc` header, and modules
whose prefix is unrecognized, preserve their original bytes and report
`bytecode_header: "unknown"`. A header prefix does not establish that the entire
file is valid. `bytecode_magic` and graph derivation edges identify synthesized
headers; member digests cover the resulting derived files. These headers do not
establish successful execution or decompilation.
The nonempty CArchive Python module `pyimod00_crypto_key` (`m`/`M`) is recorded
as an encryption marker; a same-named data file or script is not. Affected
+88 -6
View File
@@ -17,6 +17,10 @@ input Evidence bundle. Contradictions, duplicate definitions or owners,
residual unknowns, missing dependencies, and unavailable authority keep closure
open or failed.
Every fixture and verifier proof must bind its outcome to the exact owner
implementation digest and obligation revision being evaluated. See
[proof applicability](#proof-applicability) for the caller-produced contract.
## Complete CLI request
This empty request is useful for checking client integration. It is valid and
@@ -225,11 +229,18 @@ sample Evidence ID with an authenticated review record already present in
## Manifest and closure
A manifest binding names one owner and its exact implementation digest. Both
A manifest binding names one owner and a caller-supplied `owner_sha256`. Both
`original_cases` and the reconstruction fixture list must cover every
`required_case_kinds` value with authenticated, comparable Evidence.
`parser_type` is mandatory when `requires_parser_type` is true.
The ledger does not read `module_path`, execute `verifier.command`, or compute
the implementation digest. The owner fields do not specify whether the digest
covers module bytes, a symbol representation, or a multi-file implementation.
The proof producer must document the exact hashed inputs and any canonicalization
so another verifier can reproduce that commitment; a path or symbol name alone
does not define it.
```json
{
"obligation_id": "protocol.rpc.cancel",
@@ -315,10 +326,79 @@ A manifest binding names one owner and its exact implementation digest. Both
}
```
All referenced Evidence IDs must exist in the bundle. A green unit test cannot
close a `packaged-process` obligation. A static-only source cannot close a
runtime or process claim. Marking a claim `blocked` or `out-of-scope` records the
disposition but does not silently count it as verified.
A ready ledger requires every retained Evidence reference to resolve in the
bundle, including references on optional case kinds. Unresolved references stay
visible in the ledger and produce diagnostics; they cannot close the obligation.
A green unit test cannot close a `packaged-process` obligation. A static-only
source cannot close a runtime or process claim. Marking a claim `blocked` or
`out-of-scope` records the disposition but does not silently count it as verified.
### Proof applicability
Keep three checks separate:
- **Authenticity:** the retained Evidence passes its integrity checks.
- **Authority:** the observation has the kind and strength required by the claim.
- **Applicability:** the proof describes the implementation and obligation
revision currently being evaluated.
Proof producers supply observed Evidence with predicate type
`rea.reconstruction-proof`, operation `verify_reconstruction_obligations`, and
the following `normalized_result` shape, defined by
`reconstructionObligationProofSchema`. REA evaluates retained results; this
operation label does not expose a proof executor or run the manifest command.
```json
{
"bindings": [
{
"obligation_id": "protocol.rpc.cancel",
"obligation_version": 1,
"owner_sha256": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
"fixtures": [
{
"fixture_id": "fixture.cancel.positive",
"case_kind": "positive",
"passed": true
}
],
"verifiers": [
{
"verifier_id": "verifier.cancel",
"claim_id": "claim.cancel",
"passed": true
}
]
}
]
}
```
`owner_sha256` is the same lowercase 64-hex SHA-256 commitment supplied by the
implementation owner in the manifest. The proof producer computes it from the
same documented inputs and canonicalization described above; REA compares the
commitments and does not recompute them. The proof's `obligation_version` must
exactly equal the evaluated obligation version, including for generated
obligations. Increase that version when revising a reviewed requirement.
Within one matching obligation/owner/revision binding, fixture proof must name
the exact `(fixture_id, case_kind)` pair, and verifier proof the exact
`(verifier_id, claim_id)` pair. Every selected claim must report `passed: true`;
a matching failed claim cannot be hidden by a duplicate passing claim.
Claims may share a record, but identities or outcomes from different bindings
are never joined. Fixture and verifier Evidence authority must also satisfy the
existing authority requirements.
Changing the owner digest or obligation version makes retained proof
inapplicable. Missing identity, including legacy ID-array proof records, keeps
the ledger open with `inapplicable-fixture-proof` or
`inapplicable-verifier-proof`; details distinguish malformed or missing metadata
(`invalid-proof`), `owner-mismatch`, `revision-mismatch`, and `claim-mismatch`.
An applicable false outcome produces `fixture-failed` or `verifier-failed` and
a failed required ledger. An unknown, unsupported, or skipped verifier declaration
produces `verifier-unavailable` and keeps closure open. Replace fixture and
verifier Evidence after implementation or requirement changes. Old Evidence
remains valid provenance and cannot establish current applicability.
The `summary` counts obligations. `by_status`, `by_application_layer`, and
`by_source_state` place each obligation in exactly one bucket.
@@ -329,7 +409,9 @@ obligation's `source_state` do not affect its authority bucket.
The tool returns every obligation inline. Its `ledger_id` and `closure_digest`
identify the complete result, including summaries, reports, ownership and
dependency graphs, Evidence links, and limitations.
dependency graphs, Evidence links, and limitations. A new `closure_digest`
identifies a changed ledger; it does not establish that a verifier ran again.
Applicability is checked against the retained proof's explicit bindings.
For named boundaries with explicit verifier contracts and package-proof kinds,
see [reconstruction coverage](reconstruction-coverage.md).
+26 -9
View File
@@ -82,13 +82,21 @@ runtime dependency coverage. A missing section remains unknown rather than an
empty dependency set. Retained views describe the captured artifact digest;
they do not assert that the current file or running application is unchanged.
ELF section names are resolved through the section-header string table.
Zlib-compressed name tables are decoded within the native structural budget.
If a name table uses unsupported compression, the result reports that
limitation and leaves `.dep-v0` presence unknown rather than reporting the
section absent.
The inline Evidence result contains the artifact path, SHA-256, byte length,
container format, architecture, word size, and byte order. When a `.dep-v0`
section is present, `auditable_metadata` contains:
- `section_name`, `.dep-v0` for ELF and PE or `__DATA,.dep-v0` for Mach-O;
- `file_offset` and `section_bytes`, the exact file range of the compressed section;
- `compressed_bytes` and `uncompressed_bytes`;
- `file_offset` and `section_bytes`, the exact file range of the dependency
section in the selected artifact, including accepted PE file-alignment bytes;
- `compressed_bytes`, the zlib stream length used for the inflate-ratio limit,
and `uncompressed_bytes`;
- `format_version`, the embedded integer, or `null` when the field is omitted;
- `packages`, in the section's original order, with `name`, `version`, `source`,
`kind`, `dependencies`, `features`, and `root`;
@@ -109,19 +117,28 @@ removed the field entirely, as recorded in its
[changelog](https://github.com/rust-secure-code/cargo-auditable/blob/v0.5.0/CHANGELOG.md).
`rustc_commit` is present only when a `/rustc/<40-hex>/` path is found in
read-only data. Its `evidence_kind` is `inferred`. The hash is not a claim that
that compiler produced the binary. Several distinct hashes leave a single
commit unknown.
read-only data. For ELF, read-only non-executable `PT_LOAD` file ranges are
searched alongside read-only sections, so an optional missing section-header
table does not remove this search source. Its `evidence_kind` is `inferred`.
The hash is not a claim that that compiler produced the binary. Several
distinct hashes leave a single commit unknown.
## Limits and failures
The selected file is limited to 256 MiB. A dependency section is limited to 1
MiB compressed. Inflated JSON is limited to the smaller of 8 MiB and 64 times
the compressed size. The complete report is limited to 8 MiB. Empty sections,
truncated or trailing zlib bytes, invalid JSON, unknown fields, dependency
The selected file is limited to 256 MiB. A dependency section's zlib stream is
limited to 1 MiB compressed. Inflated JSON is limited to the smaller of 8 MiB
and 64 times the compressed stream length. The complete report is limited to
8 MiB. Empty sections,
truncated streams, nonpadding bytes after a zlib stream, invalid JSON, duplicate
object fields, unknown fields, dependency
indices outside the package list, more than one root package, and compressed
dependency sections are typed failures. A cycle is reported and is not walked.
For PE images with `VirtualSize` zero, the raw-size fallback can include file
alignment bytes. REA excludes a zero suffix only when the raw section size is
the zlib stream length rounded up to the declared `FileAlignment`; nonzero or
oversized suffixes and truncated streams remain failures.
A missing section returns success with `auditable_metadata: null` and a
limitation. Absence of `.dep-v0` does not mean the binary has no dependencies.
Unknown embedded format revisions are retained in `format_version` and
+35 -19
View File
@@ -31,17 +31,23 @@ The corresponding MCP request is:
`decoders` uses sigrok's decoder stack and option/channel syntax. Channel names
must match the recording; REA does not guess wiring, baud rate or protocol.
When sigrok reports that a selected channel name is missing, REA returns
`invalid_request` for `decoders` even if sigrok continues with fallback
annotations and exits successfully. The diagnostic and any partial stdout remain
in `captured_output`; unrelated producer warnings do not change request status.
Each selected decoder channel needs a target, for example `uart:rx=RX`; a bare
channel key such as `uart:rx` is missing that target. When sigrok reports a
missing or unknown selected channel target, REA returns `invalid_request` for
`decoders` even if sigrok continues with fallback annotations and exits
successfully. The diagnostic and any partial stdout remain in `captured_output`;
unrelated producer warnings do not change request status.
Omitting `input_format` requests sigrok's autodetection. REA uses an immutable
private copy whose filename retains the selected recording's final suffix, so
filename-based format detection receives that hint without exposing the
private copy whose filename retains the selected recording's final suffix,
including a dotfile basename such as `.csv`. Filename-based format detection
receives that hint without exposing the
caller's directory to sigrok. Omitting `annotations`
retains every annotation class. Both omitted selections are recorded as null.
See the [upstream manual](https://sigrok.org/wiki/Sigrok-cli) for formats,
decoder stacking and annotation selectors. No arbitrary command flags are accepted.
If sigrok reports an explicitly selected input format as unknown, REA returns
`invalid_request` for `input_format`, retains the producer diagnostic, and
advises correcting the format or omitting it for autodetection. See the
[upstream manual](https://sigrok.org/wiki/Sigrok-cli) for formats, decoder
stacking and annotation selectors. No arbitrary command flags are accepted.
Output defaults to 50 literal LF-separated lines. Use `line_offset` and
`line_limit` for later pages, or MCP `line_limit: null` / CLI `--all_lines`
@@ -63,12 +69,18 @@ The returned Evidence includes the input path/digest, exact selections, tool
configured and resolved paths/digests, full version banner (including library versions), annotation text
with sample-number ranges, and stderr. For example, `1104-1938 uart-1: 52`
is producer display text over a sample range, not a timestamp or an independently
verified protocol fact. Text is retained verbatim; it is not parsed as a lossless
binary payload. Empty output does not establish absence of traffic. Warnings
remain visible even when sigrok exits zero, including `tool.version_diagnostics`
from the separate version probe. Post-run identity failures name the tool, limiter
or private copy involved, rather than blaming the admitted source. Resource-signal
failures retain the signal and effective limits without inferring its exact cause.
verified protocol fact. Valid UTF-8 text is retained verbatim; it is not parsed as
a lossless binary payload. If any sigrok stdout or stderr stream is invalid UTF-8,
REA returns `unreadable_output` with the replacement-decoded text in
`captured_output` and the affected streams in
`details.diagnostics.invalid_utf8_streams`.
It does not report that text as successful Evidence. A valid literal replacement
character remains ordinary producer text. Empty output does not establish absence
of traffic. Warnings remain visible even when sigrok exits zero, including
`tool.version_diagnostics` from the separate version probe. Post-run identity
failures name the tool, limiter or private copy involved, rather than blaming the
admitted source. Resource-signal failures retain the signal and effective limits
without inferring its exact cause.
REA decodes a private, read-only copy of the recording, verifies that copy and
the selected executable and resource limiter after decoding, and removes the workspace after owned
@@ -146,8 +158,12 @@ is copied so a retained page does not keep the complete producer output alive.
Discovery shares decoding's owned process, admission slot, resource limits,
cleanup retries and Evidence delivery. It records the executable and limiter
identities, version banner and warnings. Failed version checks and malformed
producer output preserve diagnostics. Help is producer text, not an independently
validated parameter schema. Each request probes again; it does not cache or claim
stable pages across installation changes. Libraries/decoder modules remain
trusted and are not fully fingerprinted. Catalog output includes a SHA-256 of
its executable, not a digest of the complete installation.
producer output preserve diagnostics. Catalog names, help lines and version
output follow decoding's UTF-8 handling: malformed streams produce an
`unreadable_output` error with replacement-decoded captured text and the affected
streams identified in `details.diagnostics.invalid_utf8_streams`. Help is
producer text, not an independently validated parameter schema. Each request
probes again; it does not cache or claim stable pages across installation
changes. Libraries/decoder modules remain trusted and are not fully fingerprinted.
Catalog output includes a SHA-256 of its executable, not a digest of the complete
installation.
@@ -59,7 +59,9 @@ later increment.
`recorded_line_truncated`; an explicit empty query is `?`), `body_recorded` (false
when the capture kept no response body, so a `null` body is not mistaken for
a recorded one), `body_encoding` (`base64` when the retained bytes are not
UTF-8, so they are returned exactly rather than decoded lossily) and
UTF-8, so they are returned exactly rather than decoded lossily;
`unsupported` with a null body and `body_recorded: true` when a declared HAR
encoding cannot be decoded; the cited capture retains its original text and encoding) and
`truncated` (a large retained base64 body is decoded only up to that limit),
as both
structured content and text, with REA's existing target-content provenance
@@ -11,8 +11,9 @@
2. Add `environmentStateDocument.ts` to validate `rea-state/v1` documents
against the served types and views, and `environmentStateStore.ts` for the
live store with reset, atomic replace and the `data/get` snapshot.
3. Export `selectsRecordedLine`, the body-taking `replayAnswer` and the path
rule `pathValueMatches` from `environmentReplay.ts`, export one recorded-body
3. Export `selectsRecordedLine` and the body-taking `replayAnswer` from
`environmentReplay.ts`, reuse `pathSegmentMatches` from the state rules,
export one recorded-body
predicate `stateBodyJson` from the state accumulator and `changesState` from
the state rules, and widen `collectionPathOf` to any value with a path, so
state answers reuse selection, matching, body and write rules.
+27 -2
View File
@@ -54,6 +54,13 @@ reported. If a virtual table cannot be resolved, possible backing tables have
ordinary tables remain selectable. Schema inventory does not execute views,
triggers or arbitrary SQL.
The exact table name `sqlite_schema` is also a supported read-only row source.
SQLite's [`table_list`](https://www.sqlite.org/pragma.html#pragma_table_list)
and `table_xinfo` pragmas establish its row-source type and columns. The
[schema catalog](https://www.sqlite.org/schematab.html) has no self-entry for
`sqlite_schema`, so REA does not invent a declaration, SQL statement or root
page for it in `schema.tables`.
Rows are optional. `table` selects one exact ordinary-table name. `columns`
(CLI repeated `--column`) selects exact visible column names in the requested
order; omission selects all visible columns, including generated columns.
@@ -65,8 +72,22 @@ each failing selected column with SQLite's message, and recommends only columns
whose sampled values were read; stored and other generated cells stay readable. Malformed column-name bytes
cause a format refusal when all columns are requested. In inventory scope, an explicit valid subset can
still be sampled without decoding unrelated malformed column names.
An unavailable row source, including a `WITHOUT ROWID` primary key requiring an
application collation, is also refused as `capability_unavailable`. The error
retains the selected table and SQLite's reason without identifying unproven
dependencies or diagnosing file corruption. Such a source does not provide
readable-column recommendations. Omit `table` and select `schema_scope: "database"`
to read stored declarations without requesting rows; the application's supported
consistent export may be needed to obtain those values.
Column selection requires a
table and at least one name; empty-string identifiers are valid. `row_limit`
table and at least one name; empty-string identifiers are valid. SQLite counts
every projected name, including a repeated name, against its native
result-column capacity; a wider projection is refused as `resource_constraint`
with `resource: "capacity"`, reporting the selected count and this connection's
column limit. The selected database is readable and is not reported as
malformed; request fewer columns per call instead. `row_limit`
requires `table`, defaults to 20 and accepts positive safe integers. Samples
remain subject to the byte and worker resource limits below, regardless of row count. The result reports
the selected limit, returned count and whether more rows were observed. Row
@@ -107,6 +128,9 @@ bounds simultaneous input buffering and private snapshot/worker counts.
The database and WAL together are limited to 256 MiB. SQLite values/records,
SQL statements and the complete worker JSON reply each have a 16 MiB limit.
Schema names count by their returned JSON text, including escaping, for UTF-8,
UTF-16le and UTF-16be databases. Internal raw-byte lookup keys do not consume
the reply allowance; exact name identity still comes from those stored bytes.
The owned worker has a 256 MiB JavaScript heap limit and defaults to a 30,000 ms
deadline. Select optional MCP `worker_timeout_ms`, or CLI
`--worker-timeout-ms <milliseconds>`, to change the budget for that call. Values
@@ -161,7 +185,8 @@ evidence ID for later retrieval or export.
Changed input identities, special files, malformed databases, unsupported
formats and resource failures are reported as errors with the selected path and
meaningful reason. Encrypted databases, database repair, arbitrary SQL,
meaningful reason. Worker resource refusals expose the caller-selected file in
`details.diagnostics.selected_path`. Encrypted databases, database repair, arbitrary SQL,
cross-database queries and live capture are outside this inspection's coverage.
## Compatibility verification
+25 -2
View File
@@ -168,6 +168,20 @@ Vitest worker limits do not bound subprocesses launched within a test. Bound
those batches separately so repeated CLI validation cannot oversubscribe the
runner. Preserve simultaneous launches where concurrency is itself under test.
Coordinate expensive checks across agents sharing a host. If a test unexpectedly
consumes heavy CPU, stop its owned process family, retain its log and revision,
and profile a bounded representative case before retrying. Separate production
work from fixture generation, schema compilation, subprocess startup and cleanup.
Choose elapsed, CPU and process-family memory limits before the run; a worker
count or test timeout alone does not enforce those limits.
Remove demonstrated repeated work while preserving distinct assertions and
resource ownership. Share immutable fixture preparation where safe; keep fresh
sessions when lifecycle or concurrency is the behavior under test. Compare the
same completed workload before and after, report cache state and measurement
limits, and record interrupted runs separately from passes. Increasing a timeout
does not explain a CPU regression.
Build/documentation writers hold checkout-local output locks. Tests do not
hold a broad command lock; `check:pr` finishes tests before document validation.
Turbo caches deterministic preparation and static checks, but `test:run` always
@@ -277,6 +291,13 @@ distinct pair that would collapse if representation tags or scope were lost.
For learned identifiers, cover conflicting pairs and reused values in different
collections; mappings must remain one-to-one within their declared scope.
For coverage propagation, compare the same returned rows under complete and
incomplete examination. If a row exists on the left but is missing from a fully
examined compatible right inventory, removal can be established. If the right
side examined only eight of ten applicable records, or returns only one page,
the same missing row remains unknown unless separate evidence proves absence.
Assert the structured coverage and comparison status as well as the explanation.
For format compatibility, identify the producer revision that establishes each
rule. Include an accepted boundary case beside the rejected case so a stricter
parser cannot pass by refusing both. Keep source-derived behavior, synthetic
@@ -1752,8 +1773,10 @@ the running Node engine's single-string limit. Shared input leaves keep the
fixture's graph small; the verifier writes one temporary output file, checks its
complete byte count and an independent digest, then removes it. It requires only
Node and the built REA runtime, with space for the output plus a 1 GiB reserve.
Run `npm run verify:javascript:output -- jsonl` for compact JSONL coverage. This
opt-in lane is separate from routine tests and the canonical hash check
Run `npm run verify:javascript:output -- jsonl` for compact JSONL coverage. Its
row collection must match independent row and final-metadata byte/digest goldens;
its token count is checked against the actual file one line at a time, even when
the complete export exceeds the engine string limit. This opt-in lane is separate from routine tests and the canonical hash check
`npm run verify:javascript:digests`. It verifies serialization rather than an
arbitrary third-party application's parsing cost or MCP client capacity.
+27
View File
@@ -80,6 +80,24 @@ satisfy. A plan without execution does not establish runtime behavior.
### Interpret boundary facts once
Comparison needs separate proofs for lookup, correspondence, content, and
coverage. A hash cannot supply the proof that two retained values belong to
corresponding objects.
| Comparison fact | What it establishes | What remains unknown |
| ----------------------------------------------------------------------------- | ----------------------------------------------- | --------------------------------------------- |
| Same capture-local frame or request ID | Lookup within that capture | Cross-capture identity |
| Same HTTP method, URL, and resource type | A repeated-request group | Which individual responses correspond |
| A partial HTTP response multiset is contained in a complete peer | Omissions can explain every retained sample | Equality or modification |
| A retained HTTP sample is absent from a complete peer group | An observed difference independent of omissions | Other unobserved differences |
| Complete WebSocket connection/frame inventories and matching metadata streams | Ordered payload positions can be compared | Equality of unavailable payload facts |
| Unique WebMCP name, owner origin, and frame URL | Tool correspondence without transient frame IDs | Correspondence between duplicate owners |
| A complete source digest or decoded storage value | Comparable content | Missing source bytes or opaque remote objects |
Complete traversal or inventory does not make unavailable values equal. A
partial aggregate fingerprint difference cannot establish modification when
omissions alone explain it; independently proven differences still survive.
The format or protocol reader owns interpretation. Pass its parsed facts and
explicit uncertainty to consumers; do not make adapters reparse display text,
reconstruct omitted values, or infer success from missing diagnostics.
@@ -203,6 +221,15 @@ completeness from examined records and exhaustiveness, rather than diagnostic
prose or an empty failure list. Preserve observations independently of cleanup
success.
For example, if ten applicable records are discovered, eight are examined, and
five are returned, the five returned records do not establish complete coverage
of the ten. Keep discovered, applicable, examined, and retained scope distinct.
A filtered page can be complete for its selection while the parent inventory
remains partial. Carry the upstream coverage and unresolved reasons through
filtering and paging; an item missing from that page is not proved absent from
the target. Structured completeness and absence fields must agree with the
stated limitations.
Update canonical contracts, output schemas, examples, generated artifacts, and
relevant docs together. Verify converted advertised schemas and success,
malformed, unavailable, cancellation, and partial outcomes through affected
+5 -5
View File
@@ -32,7 +32,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
24.9,src/server/registerInvestigationTools.ts:call path unknown,Call-path workflow remains incomplete,Call path START_ADDRESS → GOAL_ADDRESS is STATUS,Persists binary addresses and internal status wording,Can the requested call path be established from complete analysis?,implemented,verified by focused unresolved call-path MCP test
24.91,src/server/registerInvestigationTools.ts:static runtime unknown,Static/runtime correlation remains incomplete,Static/runtime correlation result EVIDENCE_ID is STATUS,Persists internal evidence identifier,Does runtime behavior match the available static analysis?,implemented,verified by focused unresolved static/runtime MCP test
24.92,src/server/registerInvestigationTools.ts:reconstruction unknown,Reconstruction verification retains unknown claims,Reconstruction SPECIFICATION_HASH has unknown claims,Persists internal hash,Does the reconstruction satisfy every declared claim?,implemented,verified by focused unknown reconstruction MCP test
24.93,src/server/registerFunctionComparisonTool.ts:probe rationale,Function comparison is incomplete,Capture complete dossiers with equal providers and analysis limits.,Exposes provider architecture and gives ambiguous equality guidance,"Capture complete dossiers for both functions under the same target context and analysis limits.",implemented,verified by focused function-comparison MCP test
24.93,src/server/registerFunctionComparisonTool.ts:probe rationale,Function comparison is incomplete,Capture complete dossiers with equal providers and analysis limits.,Exposes provider architecture and gives ambiguous equality guidance,"Capture complete dossiers for both functions under the same target context.",implemented,Source and assertion inspected: tests/boundary/mcp/functionComparisonMcp.test.ts; test not rerun for this documentation change
24.95,incur:CLI argument validation,Caller omits or mistypes a command argument,VALIDATION_ERROR with raw Zod Details and fieldErrors including code/path/expected/received,Exposes validator representation and buries the useful correction,"REA could not read the command arguments. Check the named argument and its expected type, then try again.",implemented,verified by sanitizer unit tests and real dispatcher missing-argument test
25,src/application/ReferenceSourceImport.ts:cancelled,Reference-source import is cancelled,Reference source import cancelled,No recovery timing guidance,"Reference source import was cancelled. Start it again when ready.",implemented,focused assertions passed
26,src/application/ReferenceSourceImportPolicy.ts:invalid-limits,Configured import limits are not positive integers,Import limits must be positive integers,Fixed importer ceilings prevented full source imports,"Removed fixed byte, entry, depth, and path limits.",removed,verified by greater-than-10000 entry and greater-than-16-MiB regressions
@@ -71,10 +71,10 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
58,src/application/SetupHost.ts:unsupported Node,Runtime version cannot run setup,"Install Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+ and rerun setup.",Low risk; exact dependency and retry,Keep current copy,unchanged,verified by setup host tests
59,src/application/SetupPlan.ts:unsupported macOS,"macOS version is missing, unparseable, or older than 12",Upgrade to macOS 12 or newer.,Low risk; exact support floor,Keep current copy,unchanged,verified by setup unit test
60,src/application/SetupPlan.ts:unsupported Linux,Linux distribution is outside supported set,"Automated Hopper setup requires an x86_64 host running Ubuntu 24.04+, Fedora 41+, Nobara 44+, Arch Linux, CachyOS, or Omarchy; configure an existing supported provider instead.",Low risk; exact supported alternatives,Keep current copy,unchanged,verified by setup unit test
61,src/application/process/ProcessCli.ts:readJson read,Process scenario or evidence file cannot be read,Raw filesystem exception through CLI framework,Can expose local paths/codes and no recovery,"Process input file could not be read. Check that the path exists and is readable.",implemented,verified by focused CLI test
61,src/application/process/ProcessCli.ts:readJson read,Process scenario or evidence path is missing or is not a regular file,Raw filesystem exception through CLI framework,Lacks actionable recovery; preserve caller-selected paths and filesystem failure details,"Process input file could not be read: {path} ({detail}). Check the reported filesystem constraint and retry.",implemented,Source and missing-path assertion inspected: tests/boundary/filesystem/processCliFileInput.test.ts; test not rerun for this documentation change
62,src/application/JsonInputFile.ts:memory admission,Projected process JSON value exceeds available heap headroom,Insufficient heap headroom to materialize the complete JSON value,Heap admission depends on live runtime capacity rather than a fixed 64 MiB file limit,"Provide a smaller valid JSON value. For Evidence workflows, re-analyze a smaller selection of the original target and use its complete Evidence; splitting JSON text or trimming Evidence fields does not produce valid workflow input.",implemented,Source verified against shared JSON reader and recovery constant; former fixed-size refusal is superseded
63,src/application/process/ProcessCli.ts:readJson parse,Process scenario or evidence file is malformed JSON,Raw JSON parser exception through CLI framework,Can expose parser implementation and lacks recovery,"Process input file is not valid JSON. Repair the file, then try again.",implemented,verified by focused CLI test
64,src/application/process/ProcessCli.ts:capture scenario parse,Scenario fails Process Capture schema,Raw Zod validation exception through CLI framework,Exposes schema internals and field paths without stable recovery,"Process scenario is invalid. Check its required fields and limits, then try again.",implemented,verified by focused CLI test
63,src/application/process/ProcessCli.ts:readJson parse,Process scenario or evidence file is malformed JSON or invalid UTF-8,Raw JSON parser exception through CLI framework,Lacks actionable recovery; preserve the selected path and failure detail,"Process input file is not valid UTF-8 JSON: {path} ({detail}). Repair the file, then try again.",implemented,Source and malformed-JSON/UTF-8 assertions inspected: tests/boundary/filesystem/processCliFileInput.test.ts; test not rerun for this documentation change
64,src/application/process/ProcessCli.ts:capture scenario parse,Scenario fails Process Capture schema,Raw Zod validation exception through CLI framework,Unstructured schema details obscure actionable field errors,"Analysis input is invalid. Check the arguments and try again.",implemented,Source and scenario-validation assertion inspected: tests/boundary/filesystem/processCliFileInput.test.ts; structured issues retain field paths; test not rerun for this documentation change
66,src/application/process/ProcessCli.ts:wrong evidence,Comparison receives valid Evidence from another workflow,Expected REA process capture Evidence,Uses internal schema name and does not explain recovery,"Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again.",implemented,verified by focused CLI test
67,src/application/process/ProcessCli.ts:unexpected,Unexpected parser/comparison/canonicalization failure escapes expected classifications,Raw thrown exception through CLI framework,Can expose stack or internal invariant,"Message keeps the sanitized cause. Remediation: Do not repeat this command. Tell the user the process command could not complete and ask them to check the input files, then run `rea doctor`.",implemented,verified by direct unexpected-cause projection test
68,src/server/registerEvidenceTools.ts:object guard,Validated provider-tool input is unexpectedly not an object,Provider tool input was not an object,Exposes provider architecture and implies caller can fix impossible state,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and MCP integration tests
@@ -116,5 +116,5 @@ Then rerun this installer.",implemented,verified by shell branch inspection
90,install.sh:version mismatch,Installed rea reports a different version,installed version ACTUAL does not match EXPECTED.,Low risk; gives exact integrity mismatch,Keep current copy,unchanged,verified by exact installer test
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Canonical invalid_request with the SDK diagnostic plus recovery.kind=inspect_input_contract and a bounded instruction to read the advertised schema or call tools/list",Keeps field/type correction while telling the agent how to recover; diagnostics over 4 KiB stay on the existing retained-error path,Normalize ordinary validation failures and retain oversized diagnostics,verified by real in-memory and stdio MCP client/server calls with wrong argument types
90.6,incur:unknown CLI command,Caller enters an unknown REA command,"'COMMAND' is not a command for 'rea'. Suggested command: rea --help",Low risk; identifies input and exact discovery action,Keep framework-generated copy,unchanged,verified by real packaged dispatcher invocation
91,src/application/binaryDiagnostics/ByteEntropyService.ts,Invalid file or window,"Expected a regular file.; Selected file could not be found.; Selected byte window extends beyond the end of the file.; File size exceeds exact numeric offset representation.",Invalid selection cannot produce complete entropy Evidence,Choose a readable regular file and a window within its byte size,implemented,entropy service and MCP boundary tests
91,src/application/binaryDiagnostics/ByteEntropyService.ts,Invalid file or window,"Expected a regular file.; Selected file could not be found.; Selected byte window starts beyond the end of the file.; Selected byte window extends beyond the end of the file.; File size exceeds exact numeric offset representation.",Invalid selection cannot produce complete entropy Evidence,Choose a readable regular file and a window within its byte size,implemented,entropy service and MCP boundary tests
92,src/domain/analysisView/byteEntropyView.ts,Non-summary entropy view,Byte entropy Evidence supports the summary view; retrieve parent Evidence for the complete block series.,Unsupported view selection,Use the summary view or retrieve complete parent Evidence,implemented,typed input rejection
1 harm_rank location trigger current_copy user_risk proposed_replacement implementation_status verification_result
32 24.9 src/server/registerInvestigationTools.ts:call path unknown Call-path workflow remains incomplete Call path START_ADDRESS → GOAL_ADDRESS is STATUS Persists binary addresses and internal status wording Can the requested call path be established from complete analysis? implemented verified by focused unresolved call-path MCP test
33 24.91 src/server/registerInvestigationTools.ts:static runtime unknown Static/runtime correlation remains incomplete Static/runtime correlation result EVIDENCE_ID is STATUS Persists internal evidence identifier Does runtime behavior match the available static analysis? implemented verified by focused unresolved static/runtime MCP test
34 24.92 src/server/registerInvestigationTools.ts:reconstruction unknown Reconstruction verification retains unknown claims Reconstruction SPECIFICATION_HASH has unknown claims Persists internal hash Does the reconstruction satisfy every declared claim? implemented verified by focused unknown reconstruction MCP test
35 24.93 src/server/registerFunctionComparisonTool.ts:probe rationale Function comparison is incomplete Capture complete dossiers with equal providers and analysis limits. Exposes provider architecture and gives ambiguous equality guidance Capture complete dossiers for both functions under the same target context and analysis limits. Capture complete dossiers for both functions under the same target context. implemented verified by focused function-comparison MCP test Source and assertion inspected: tests/boundary/mcp/functionComparisonMcp.test.ts; test not rerun for this documentation change
36 24.95 incur:CLI argument validation Caller omits or mistypes a command argument VALIDATION_ERROR with raw Zod Details and fieldErrors including code/path/expected/received Exposes validator representation and buries the useful correction REA could not read the command arguments. Check the named argument and its expected type, then try again. implemented verified by sanitizer unit tests and real dispatcher missing-argument test
37 25 src/application/ReferenceSourceImport.ts:cancelled Reference-source import is cancelled Reference source import cancelled No recovery timing guidance Reference source import was cancelled. Start it again when ready. implemented focused assertions passed
38 26 src/application/ReferenceSourceImportPolicy.ts:invalid-limits Configured import limits are not positive integers Import limits must be positive integers Fixed importer ceilings prevented full source imports Removed fixed byte, entry, depth, and path limits. removed verified by greater-than-10000 entry and greater-than-16-MiB regressions
71 58 src/application/SetupHost.ts:unsupported Node Runtime version cannot run setup Install Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+ and rerun setup. Low risk; exact dependency and retry Keep current copy unchanged verified by setup host tests
72 59 src/application/SetupPlan.ts:unsupported macOS macOS version is missing, unparseable, or older than 12 Upgrade to macOS 12 or newer. Low risk; exact support floor Keep current copy unchanged verified by setup unit test
73 60 src/application/SetupPlan.ts:unsupported Linux Linux distribution is outside supported set Automated Hopper setup requires an x86_64 host running Ubuntu 24.04+, Fedora 41+, Nobara 44+, Arch Linux, CachyOS, or Omarchy; configure an existing supported provider instead. Low risk; exact supported alternatives Keep current copy unchanged verified by setup unit test
74 61 src/application/process/ProcessCli.ts:readJson read Process scenario or evidence file cannot be read Process scenario or evidence path is missing or is not a regular file Raw filesystem exception through CLI framework Can expose local paths/codes and no recovery Lacks actionable recovery; preserve caller-selected paths and filesystem failure details Process input file could not be read. Check that the path exists and is readable. Process input file could not be read: {path} ({detail}). Check the reported filesystem constraint and retry. implemented verified by focused CLI test Source and missing-path assertion inspected: tests/boundary/filesystem/processCliFileInput.test.ts; test not rerun for this documentation change
75 62 src/application/JsonInputFile.ts:memory admission Projected process JSON value exceeds available heap headroom Insufficient heap headroom to materialize the complete JSON value Heap admission depends on live runtime capacity rather than a fixed 64 MiB file limit Provide a smaller valid JSON value. For Evidence workflows, re-analyze a smaller selection of the original target and use its complete Evidence; splitting JSON text or trimming Evidence fields does not produce valid workflow input. implemented Source verified against shared JSON reader and recovery constant; former fixed-size refusal is superseded
76 63 src/application/process/ProcessCli.ts:readJson parse Process scenario or evidence file is malformed JSON Process scenario or evidence file is malformed JSON or invalid UTF-8 Raw JSON parser exception through CLI framework Can expose parser implementation and lacks recovery Lacks actionable recovery; preserve the selected path and failure detail Process input file is not valid JSON. Repair the file, then try again. Process input file is not valid UTF-8 JSON: {path} ({detail}). Repair the file, then try again. implemented verified by focused CLI test Source and malformed-JSON/UTF-8 assertions inspected: tests/boundary/filesystem/processCliFileInput.test.ts; test not rerun for this documentation change
77 64 src/application/process/ProcessCli.ts:capture scenario parse Scenario fails Process Capture schema Raw Zod validation exception through CLI framework Exposes schema internals and field paths without stable recovery Unstructured schema details obscure actionable field errors Process scenario is invalid. Check its required fields and limits, then try again. Analysis input is invalid. Check the arguments and try again. implemented verified by focused CLI test Source and scenario-validation assertion inspected: tests/boundary/filesystem/processCliFileInput.test.ts; structured issues retain field paths; test not rerun for this documentation change
78 66 src/application/process/ProcessCli.ts:wrong evidence Comparison receives valid Evidence from another workflow Expected REA process capture Evidence Uses internal schema name and does not explain recovery Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again. implemented verified by focused CLI test
79 67 src/application/process/ProcessCli.ts:unexpected Unexpected parser/comparison/canonicalization failure escapes expected classifications Raw thrown exception through CLI framework Can expose stack or internal invariant Message keeps the sanitized cause. Remediation: Do not repeat this command. Tell the user the process command could not complete and ask them to check the input files, then run `rea doctor`. implemented verified by direct unexpected-cause projection test
80 68 src/server/registerEvidenceTools.ts:object guard Validated provider-tool input is unexpectedly not an object Provider tool input was not an object Exposes provider architecture and implies caller can fix impossible state Remove the impossible duplicate guard and parse through the declared JSON-object schema. removed verified by typecheck and MCP integration tests
116
117
118
119
120
@@ -0,0 +1,134 @@
# CLI JSON Lines verification — 2026-10-23
The analyst task was to export retained JavaScript observations as independent
records while keeping their Evidence provenance, coverage and unknowns. JSONL
framing stays in the CLI adapter; analysis and view projection continue to use
the same application workflow as MCP.
## Real artifact and completed workflow
The installed, locked `semver@7.8.5` npm package was analyzed through the public
`analyze-javascript-application` command, without executing its target code.
It parsed 50 selected files and produced a 12,185,829-byte Evidence document.
Preparation took 12.295 s elapsed, 12.648 s child CPU and 678.5 MiB sampled peak
process-family RSS; preparation is separate from the export measurements.
The artifact digest was
`65c9cce0fbebc34824c1b9657a514e7c5525ab079b232ce9da095697271b7953`.
The saved Evidence file SHA-256 was
`b34627bc12894d4c749e5b24c60731dd5a7eaaf90c98954f0e585eb442d484c3`.
Preparation ran on main `169512a35`; the exact retained parent was reused by
both refreshed export variants:
```json
{
"source": { "kind": "evidence-file", "path": "/absolute/path/parent.json" },
"view": {
"kind": "javascript-facts",
"property_keys": [],
"include_available_property_keys": true,
"offset": 0,
"limit": 256
}
}
```
Each completed measurement executed one public CLI call:
```bash
node scripts/rea.mjs inspect-analysis-view request.json --format jsonl
```
The view contained 202 observation rows. Every before/after pair reconstructed
exactly equal complete Evidence JSON, including identities, subject/provider,
authority/confidence, locations, parent/view digests, links, original coverage,
17 view limitations and 20 unknowns. These source limitations remain limitations;
exhausting this selected view does not establish complete application analysis.
Before emitted one document; after emitted 202 rows and one metadata record.
## Performance
Baseline: refreshed main `2443b949a48c3b2eba687e7381682e49e909d5b9`.
Both variants used Node 24.22.1 and the same locked dependency installation.
Three alternating pairs ran sequentially, with fresh processes and warm
filesystem/build artifacts. Node compile caching was not configured. Cold host
filesystem caches and repeated analysis were not measured.
| Completed export, median of three | Before | After |
| --------------------------------- | --------: | --------: |
| Elapsed time | 4.510 s | 4.576 s |
| Wait-accounted child CPU | 5.876 s | 5.987 s |
| Sampled peak process-family RSS | 496.3 MiB | 495.1 MiB |
| Emitted UTF-8 bytes | 365,494 | 365,608 |
| `tokenx` token estimate | 99,004 | 99,041 |
| CLI calls | 1 | 1 |
The small observed overhead does not establish an analysis speedup.
The 114-byte / 37-estimated-token increase is framing overhead, with the same
facts and no additional tool/schema round trips. The identical request file
contributes another 74 estimated tokens to each case. These are heuristic `tokenx` estimates, not model-specific BPE counts;
orchestration prompts, agent reasoning and tool-definition tokens were not
measured. Actual agent context consumption also depends on which rows the
consumer reads. No token reduction is claimed.
Rows and metadata share the existing bounded JSON encoder and awaited writes.
Token counting reuses that exact record plan, including context, and estimates
one complete line at a time rather than materializing the complete export.
A single line still must fit the estimator's runtime string limit. Token-count
performance, much larger applications and time to first consumed row were not
benchmarked.
The harness pinned the process family to two CPUs, used a 2 GiB V8 heap limit,
`GOMAXPROCS=2`, `GOMEMLIMIT=2GiB` and two libuv threads. It sampled descendant
RSS every 50 ms and would terminate the task at 6 GiB or 30 minutes. No resource
stop occurred. RSS sums can double-count shared pages and miss short peaks;
the V8 limit is per process and the sampler is not a kernel memory limit.
## Validation
`npm run check` passed on the refreshed integration, including type checking,
lint, formatting, dependency/metadata checks and architecture guards. The parsed
output-control contract from current main is retained; handlers, launcher and
reusable test drivers consume those same facts.
- 241 tests passed across 14 CLI/MCP/acceptance files: actual production MCP
catalogs, authenticated Evidence reconstruction, invalid SDK values, ordinary
and sentinel failures, generator streaming, formatter recovery, empty/partial
views, large output, pipe closure and asynchronous final writes after cleanup
success/failure. The run took 171.097 s elapsed, 203.968 s child CPU and
1589.3 MiB sampled peak family RSS.
- Another 60 tests passed across four consumer files, including ordinary graph
reports and scalar selections refusing JSONL, malformed JSON input retaining
diagnostics, managed CLI output and the direct SDK update-alias contract.
These took 223.152 s elapsed, 262.538 s child CPU and 1079.2 MiB peak family RSS.
- A new negative control showed that calling a collection encoder for a single
array-valued row undercounted tokens (15 instead of 21, 57 instead of 63).
Emission and counting now share `jsonLineParts`; both cases and all 29 JSONL
tests pass after the fix. The 241-test run includes these regressions.
- A real `rea mcp doctor --format jsonl` run reported healthy in one JSON line.
Startup/validation failures, full-output framing and advertised help are
exercised separately.
The opt-in verifier used shared 64 KiB leaves and independently checked complete
output bytes and SHA-256, without retaining the complete encoded document:
| Synthetic serialization fixture | JSON | JSONL |
| ----------------------------------------------- | ---------------: | ----------------: |
| Output bytes (engine string limit: 536,870,888) | 536,961,655 | 536,961,779 |
| Elapsed / child CPU | 9.811 / 10.197 s | 13.940 / 14.451 s |
| Sampled peak family RSS | 171.6 MiB | 186.0 MiB |
The JSONL run also checked its token count against the actual file line by line:
89,492,357 heuristic tokens matched. JSONL timing includes this extra counting
verification, so the two fixture timings are not equivalent-workflow performance
comparisons. Shared leaves keep input memory small; this verifies serialization
expansion and counting capacity, not a 512 MiB application's analysis footprint.
Both temporary output files were removed by the verifier's cleanup.
No resource stop occurred. Native provider behavior is outside this CLI change.
`npm run docs:check` passed, including nine README consistency tests.
The refreshed measurements are in the `refreshed/` subdirectory; earlier
measurements against `169512a35` remain separate. Raw stdout/stderr, commands,
per-run resource samples, equivalence assertions and summaries are retained under
`/tmp/rea-maintenance-20261023/5629/`; they are not committed artifacts.
@@ -0,0 +1,120 @@
# CLI version report verification — 2026-10-23
`rea --version --json` reports package/runtime/platform facts without loading
analysis command implementations. The shared CLI preflight retains configuration
and output-option error precedence. No cache was added: the unnecessary work was
loading the full command graph, and runtime facts must remain current.
## Scope and identities
- Real application: REA 6.7.0's built public CLI, on Linux x64; Node 24.22.1 and
npm 11.16.0 (repository pins). No target or analysis provider is required.
- Host CPU: AMD EPYC 7B12; affinity is restricted to two logical CPUs.
Node executable SHA-256: `e32e2d16081cf644f88392934288086e7662529d0d43277c34f7aad2207cfafb`.
- Baseline: `5655fbf17`, combining main `60accad2d` with original PR #6530 head
`044bd861b2df082eca5d4ad91d7cb79876c13430`. It already has the formatted report;
the comparison measures delivery of equivalent facts, not plain versus JSON.
- Original author commits are preserved. The optimized source is this PR's
launcher/preflight change; source hashes below bind the measured code.
- Both lanes use the same locked dependencies, executable and environment
controls. Package manifest SHA-256: `2218e615dc1e68ca31c02ee5f6f2dee1e2cf7aa6a78c1c22ac090e27fe49d2eb`.
- Built tree digests: baseline `a4bc8722b69046c5ace1d3e764bba81812427766f088418c27c6876812796722`
(2147 files); optimized `9919d1b6177536f75fd1acb9240525e82cb5af5a55f4db07cbcc4b243a5f8eb4`
(2153 files). Each digest hashes sorted relative
paths, a NUL separator, and each file's binary SHA-256.
```text
47eae72df85b722dcabab6860980a7285886a854daaec74b47490b41f9652154 scripts/rea.mjs
73e5f50400b11012ccf6c6c312e1f91e027e92c4a5616f112bf97ea135e3253d src/cliOutput.ts
64d82457c9f9f276379f9734caff181117afc3fd17fa50d90140a92853395f43 src/cliConfiguration.ts
fa221460cf337bb2463bd47cd919066a612f369ab866cc1dfbf27ae49259e182 src/domain/versionReport.ts
```
## Completed public tasks and resource measurements
Commands, each in a fresh process from each checkout:
```sh
node scripts/rea.mjs --version --json
node scripts/rea.mjs --version
node scripts/rea.mjs providers --json --filter-output 'capabilities[0,1].provider.id,open'
```
Three alternating pairs per task, reversing order in pair two. All 18 completed
with exit 0 and byte-identical stdout within each task. These are warm build/file
cache measurements; no OS-cold run or in-process warm session was measured.
`NODE_DISABLE_COMPILE_CACHE=1` disables Node's compile cache in both lanes.
CPU affinity is 0–1; Node heap limit is 2048 MiB; `GOMAXPROCS=2`,
`GOMEMLIMIT=2GiB`, and `UV_THREADPOOL_SIZE=2`. A sampled 6 GiB early-stop guard
was never reached; it is not a kernel memory limit.
Elapsed includes process startup through exit. CPU is wait-accounted child CPU.
Peak RSS is the maximum sampled sum across the payload process family, every
50 ms; shared pages can be counted more than once and brief peaks can be missed.
The external sampler is excluded. All measured commands here have no analysis
engine child. Other host work is uncontrolled, so small control differences do
not establish an improvement.
Medians; paired values are **before / after**:
| Task | Elapsed (s) | CPU (s) | Peak family RSS (MiB) | Stdout bytes |
| -------------------------- | ------------: | ------------: | --------------------: | -----------: |
| Formatted version | 1.716 / 0.203 | 1.968 / 0.198 | 335.6 / 72.1 | 207 |
| Plain version control | 1.876 / 1.783 | 2.221 / 1.990 | 346.8 / 338.9 | 6 |
| Filtered providers control | 2.467 / 2.182 | 2.624 / 2.461 | 384.7 / 378.2 | 67 |
First measured formatted-version process, separated from the repeat medians:
| Lane | Elapsed (s) | CPU (s) | Peak family RSS (MiB) |
| ------ | ----------: | ------: | --------------------: |
| before | 1.716 | 1.968 | 335.6 |
| after | 0.203 | 0.198 | 72.1 |
The JSON payload remains 207 bytes with all documented fields. No token
reduction is claimed. Token counts, host prompt handling, and actual model usage
were not measured; these CLI tasks have no MCP discovery or follow-up calls.
Earlier development probes, including probes overlapping focused tests, are
excluded from this final comparison. No general analysis/provider speedup is
claimed.
## Correctness and compatibility
- 231 tests passed across the 12 exact selected files below, serially.
This covers runtime/domain contracts, actual CLI output and error handling,
dispatcher cleanup, runtime bundling, real MCP doctor catalogs, doctor
composition, and both compiled CLI test drivers.
- The new dispatcher regression uses the real built preflight and CLI modules
that throw on import. JSON succeeds without those imports; invalid output and
configuration retain their errors. Restoring the original launcher makes this
regression fail specifically on the blocked analysis import (negative control).
- A real PTY invocation returned indented JSON matching piped facts. Plain
version, YAML and help precedence remain covered through the public CLI.
- A real Node 20.20.2 invocation returned its version with
`supported_runtime.satisfied: false`; this does not establish analysis support
for Node 20. Bun, Windows and macOS end-to-end behavior remain unverified.
- An earlier parallel 210-test selection had 209 passes and one existing
`cliOutput` case exceeding its 3000 ms process deadline. That failed attempt is
retained; the subsequent serial 231-test selection passed without relaxing any
deadline. Its cause is not conclusively attributed. The filtered-providers
benchmark above exercises that same public command as a compatibility control.
- `npm run check` passed after declaring the existing `cliOutput` runtime
entrypoint to Knip. This keeps the actual dynamically consumed entry visible
rather than ignoring the new preflight export.
- `npm run docs:check` passed, including 9 README consistency tests.
The serial selection used already built prerequisites:
```sh
node scripts/run-exact-tests.mjs run --maxWorkers=1 --no-file-parallelism \
src/domain/runtimeVersion.test.ts src/domain/versionReport.test.ts \
tests/boundary/cli/cliOutput.test.ts tests/boundary/cli/versionReportCli.test.ts \
tests/boundary/cli/dispatcher.test.ts tests/boundary/process/runtimeBundle.test.ts \
tests/boundary/mcp/mcpDoctor.test.ts tests/boundary/cli/mcpDoctorHelpCli.test.ts \
src/doctorRuntime.test.ts src/application/Doctor.test.ts \
tests/boundary/cli/compiledCliSession.test.ts \
tests/boundary/cli/sharedJavaScriptCliSession.test.ts
```
This workflow reports runtime metadata, not target-analysis Evidence. Existing
MCP server identity remains the MCP version surface; no duplicate tool or schema
was introduced.
+17 -8
View File
@@ -62,14 +62,23 @@ returns `resource_constraint` with resource `file-size` and its fixed byte
allowance in `reported_limits.maximum_input_bytes`; retrying the unchanged file
does not raise that allowance. WASM bytes are copied
into a private workspace as read-only `module.wasm`; its digest is rechecked
before and after each tool. WABT validates with `--enable-all`,
reports sections through `wasm-objdump -h -x`, and decodes through
`wasm2wat --enable-all`. The result retains selected byte SHA-256 and size,
section payload ranges, full header/detail output (including module-name
preambles), exact import/export WAT
forms, decoded WAT and its UTF-8 SHA-256. Producer format drift and invalid or
truncated modules are failures, with validation diagnostics and partial Evidence
retaining selected byte identity and tool provenance.
before and after each tool. WABT validates with
`wasm-validate --enable-all --ignore-custom-section-errors`, reports sections
through `wasm-objdump -h -x`, and decodes through
`wasm2wat --enable-all --ignore-custom-section-errors`. The result retains
selected byte SHA-256 and size, section payload ranges, full header/detail output
(including module-name preambles), exact import/export WAT forms, decoded WAT
and its UTF-8 SHA-256. Producer format drift and invalid or truncated modules
are failures, with validation diagnostics and partial Evidence retaining
selected byte identity and tool provenance.
Core validation remains authoritative when an optional custom section cannot
be interpreted. WABT warning output is preserved in `custom_section_warnings`;
`debug_name_metadata` reports whether the optional name section was absent,
fully interpreted, or uninterpretable. Older portable reports retain their
original strict command profile; missing metadata coverage is `unknown` and
missing warning history is `null`. The raw custom section remains in the
decoded WAT. An uninterpretable name section leaves debug module names unknown.
The tool profile preserves upstream release/source identity, resolved executable
paths, executable digests, exact banners and arguments. Executable digests are
+4
View File
@@ -137,6 +137,10 @@ chunks. Its offsets refer to **base64-decoded body bytes**; other offsets refer
to retained body bytes. Every field has a body-scoped `field_id` derived from
its byte offset, a `frame_index`, and a `parent_field_id` for schema-driven
nested messages or groups. Repeated field numbers remain separate occurrences.
Trailer blocks are parsed as HTTP field bytes. Valid UTF-8 values are retained;
when a field contains opaque non-UTF-8 bytes, that field is omitted and the
trailer frame is partial, while other valid trailer fields remain available.
The retained body and frame byte range preserve the original opaque value.
Without a schema, varints and fixed-width values retain exact unsigned decimal
representations. Length-delimited UTF-8, nested-message and packed readings
+14 -4
View File
@@ -29,9 +29,12 @@ zero-based UTF-16 column. The MCP arguments are:
}
```
The URL supplies the context for resolving `sources` and `sourceRoot`; it is not
an instruction to fetch. Absolute virtual or file contexts can be selected as
well. The map/script association is caller-selected: a declaration or local map
The URL supplies the context for resolving `sources` and `sourceRoot` using the
ECMA-426 URL parsing rule; it is not an instruction to fetch. Repeated path
separators remain part of the resolved URL identity. A non-null `sourceRoot`
prefixes each source, adding `/` only when the root does not already end with
one. An empty root therefore prefixes `/`; a null or omitted root adds no
prefix. Absolute virtual or file contexts can be selected as well. The map/script association is caller-selected: a declaration or local map
does not prove deployment authenticity or execution. Use
[import tracing](website-module-trace.md) to inspect outgoing native module edges.
@@ -64,7 +67,14 @@ The adapter uses the locked upstream
without modifying its code. It validates map/VLQ structure before allocation,
normalizes nested indexed offsets before upstream flattening, checks section
boundaries and indexes, and retains raw declaration identity. It supports regular
maps and inline indexed sections. External section URLs return an actionable
maps and inline indexed sections. Numeric `version: 3` is required on the root
and every embedded map, including indexed ancestors. Invalid embedded headers
report their section path and cannot produce descendant mapping facts. This
uses the rejection option for version errors allowed by
[ECMA-426](https://tc39.es/ecma426/#sec-decodeindexsourcemap).
Equal section starts are allowed when earlier sections contribute no mappings;
actual decoded mappings must stay before the next section boundary.
External section URLs return an actionable
unsupported-profile error. This is a defined decoder profile, not a claim to
implement every current source-map extension.
+60
View File
@@ -0,0 +1,60 @@
# Copy to .github/workflows/rea-analysis.yml on your default branch.
name: REA static analysis example
on:
workflow_dispatch:
permissions:
contents: read
jobs:
analyze:
runs-on: ubuntu-24.04
timeout-minutes: 10
env:
ARTIFACT_PATH: build/rea-ci-demo
steps:
- name: Prepare isolated report directory
id: reports
shell: bash
run: |
directory=$(mktemp -d "${RUNNER_TEMP:?}/rea-analysis.XXXXXX")
printf 'directory=%s\n' "${directory}" >> "${GITHUB_OUTPUT:?}"
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: "24.22.1"
- name: Install pinned REA
run: |
npm install --prefix "${RUNNER_TEMP}/rea-tool" --no-audit --no-fund --ignore-scripts --save-exact rea-agents@6.7.0
echo "${RUNNER_TEMP}/rea-tool/node_modules/.bin" >> "${GITHUB_PATH}"
- name: Build a known artifact
id: build
run: |
# Replace this step with your reviewed build, keeping ARTIFACT_PATH in sync.
mkdir -p build/rea-ci-demo
printf '%s\n' '{"name":"rea-ci-demo","version":"1.0.0","type":"module","main":"index.js"}' > build/rea-ci-demo/package.json
printf '%s\n' 'export const message = "hello";' 'throw new Error("This artifact must not be executed.");' > build/rea-ci-demo/index.js
- name: Analyze and retain the original exit status
id: analyze
shell: bash
env:
REPORT_DIR: ${{ steps.reports.outputs.directory }}
run: |
set +e
rea analyze-javascript-application "${GITHUB_WORKSPACE}/${ARTIFACT_PATH}" --json --full-output \
> "${REPORT_DIR}/analysis.json" 2> "${REPORT_DIR}/analysis.stderr"
status=$?
set -e
printf '%s\n' "${status}" > "${REPORT_DIR}/exit-code.txt"
exit "${status}"
- name: Upload available reports, including after analysis failure
if: ${{ always() && steps.reports.outputs.directory != '' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: rea-analysis
path: ${{ steps.reports.outputs.directory }}
if-no-files-found: warn
retention-days: 7
+1
View File
@@ -3,6 +3,7 @@
"entry": [
"src/main.ts",
"src/cli.ts",
"src/cliOutput.ts",
"src/composition/javascriptAnalysisWorkerProcess.ts",
"src/composition/webBundleAnalysisWorkerProcess.ts",
"src/contracts/errorSchemas.ts",
+1 -1
View File
@@ -125,7 +125,7 @@ const SOURCE_PATHS = {
idaProvider: "dist/ida/IdaProviderCapabilities.js",
kunaProvider: "dist/kuna/KunaProviderCapabilities.js",
nativeProvider: "dist/native/NativeMacOSProviderMetadata.js",
artifactProviders: "dist/application/InvestigationProviders.js",
artifactProviders: "dist/domain/investigationProviders.js",
browserIdentities: "dist/browser/providerIdentities.js",
v8InspectorProvider: "dist/inspector/providerIdentity.js",
evidence: "dist/domain/evidence.js",
+54 -29
View File
@@ -81,21 +81,34 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
writeStdout(result.output);
process.exitCode = result.exitCode;
} else {
const { cliConfigurationError, closeCliJavaScriptResources, createCli } =
await import("../dist/cli-runtime/cli.js");
const { createCliResultOutput } =
await import("../dist/cli-runtime/cliResultOutput.js");
const {
cliConfigurationError,
normalizeCliArguments,
renderCliOutputArgumentError,
renderEmptyFilteredCliOutput,
validateCliOutputArguments,
renderVersionReport,
parseCliOutputArguments,
} = await import("../dist/cli-runtime/cliOutput.js");
guardClosedStdout();
const cliArgs = normalizeCliArguments(args);
const outputArguments = validateCliOutputArguments(cliArgs);
const outputOptions = parseCliOutputArguments(cliArgs);
const outputArguments = outputOptions.validation;
// `--version` with an explicit format reports the runtime as a document;
// plain `--version` stays Incur's bare version string.
const versionOutput = renderVersionReport(
outputOptions,
{
packageName: packageJson.name,
packageVersion: packageJson.version,
versions: process.versions,
platform: process.platform,
architecture: process.arch,
},
process.stdout.isTTY === true,
);
const configurationError = cliConfigurationError(process.env);
let dispatchFailure;
let closeCliJavaScriptResources;
// Incur can exit from the callback below before this function returns.
// That callback owns the status, including a closed-pipe exit of 141.
let exiting = false;
@@ -108,9 +121,16 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
} else if (!outputArguments.ok) {
writeStdout(renderCliOutputArgumentError(outputArguments));
process.exitCode = 1;
} else if (versionOutput !== undefined) {
writeStdout(versionOutput);
} else {
const cliRuntime = await import("../dist/cli-runtime/cli.js");
closeCliJavaScriptResources = cliRuntime.closeCliJavaScriptResources;
const { createCli } = cliRuntime;
const { createCliResultOutput } =
await import("../dist/cli-runtime/cliResultOutput.js");
let wroteOutput = false;
const resultOutput = createCliResultOutput(cliArgs, process.stdout);
const resultOutput = createCliResultOutput(outputOptions, process.stdout);
try {
await createCli(process.env, resultOutput).serve(cliArgs, {
stdout: (output) => {
@@ -120,6 +140,7 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
return;
}
if (output.length > 0) wroteOutput = true;
if (resultOutput?.captureSdkOutput(output)) return;
writeStdout(output);
},
// A command failure caused by the closed pipe keeps the pipe's status.
@@ -127,26 +148,25 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
// shutdown is reported here because this callback is what exits.
exit: (code) => {
exiting = true;
const status = stdoutClosed ? SIGPIPE_EXIT_STATUS : code;
void Promise.resolve()
.then(() => closeCliJavaScriptResources())
.then(
() => {
process.exit(status);
},
(cleanup) => {
process.stderr.write(
`Failed to close the JavaScript analysis worker: ${cleanupFailureText(cleanup)}\n`,
);
process.exit(
stdoutClosed
? SIGPIPE_EXIT_STATUS
: status === 0
? 1
: status,
);
},
);
void (async () => {
let status = code;
try {
await closeCliJavaScriptResources();
} catch (cleanup) {
process.stderr.write(
`Failed to close the JavaScript analysis worker: ${cleanupFailureText(cleanup)}\n`,
);
if (status === 0) status = 1;
}
try {
await resultOutput?.finishSdkOutput(code);
} catch (cause) {
if (!stdoutClosed)
process.stderr.write(`${cleanupFailureText(cause)}\n`);
if (status === 0) status = 1;
}
process.exit(stdoutClosed ? SIGPIPE_EXIT_STATUS : status);
})();
},
});
} catch (cause) {
@@ -163,7 +183,12 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
process.exitCode = stdoutClosed ? SIGPIPE_EXIT_STATUS : 1;
}
if (!wroteOutput && !resultOutput?.handled)
writeStdout(renderEmptyFilteredCliOutput(cliArgs) ?? "");
writeStdout(renderEmptyFilteredCliOutput(outputOptions) ?? "");
// An SDK failure exits through the callback, which owns its pending write.
if (!exiting)
await resultOutput?.finishSdkOutput(
typeof process.exitCode === "number" ? process.exitCode : 0,
);
}
} catch (cause) {
dispatchFailure = cause;
@@ -171,7 +196,7 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
// Serve and output rendering can throw after a worker is parked. Stop it
// on that path too, and keep the dispatch failure when both fail.
try {
await closeCliJavaScriptResources();
await closeCliJavaScriptResources?.();
} catch (cleanup) {
// The exit callback reports this same shared failure and then exits.
if (!exiting) {
+1 -1
View File
@@ -13,7 +13,7 @@ import {
} from "../dist/application/RealHopperAssertions.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { REA_WORKFLOW_PROVIDER } from "../dist/application/InvestigationProviders.js";
import { REA_WORKFLOW_PROVIDER } from "../dist/domain/investigationProviders.js";
import { loadRealHopperFixtureTargets } from "./lib/real-hopper-fixture.mjs";
import { verifyHopperStringObjects } from "./lib/real-hopper-search.mjs";
import {
+7
View File
@@ -93,6 +93,10 @@ export const verifyAndroidMetadata = async ({
result.stdout,
/DIRECT_INVOKE_REGRESSION_PASS nodes=1 offsets=0,3 cross_dex=true/u,
);
assert.match(
result.stdout,
/MANIFEST_REGRESSION_PASS root=true namespaces=true entities=true reload=true missing=true bounded=true/u,
);
console.log(
"PASS real cold metadata, full generic types and stable overloaded method selection",
);
@@ -108,6 +112,9 @@ export const verifyAndroidMetadata = async ({
console.log(
"PASS real raw DEX direct-invoke offsets and cross-DEX source identity",
);
console.log(
"PASS root manifest selection, namespace/entity values and session cache identity",
);
await verifyAndroidMethodBodies({
apk: resolve(classes, "initializers.apk"),
repository,
+83 -16
View File
@@ -2,6 +2,8 @@ import assert from "node:assert/strict";
import { constants } from "node:buffer";
import { createHash } from "node:crypto";
import { once } from "node:events";
import { createInterface } from "node:readline";
import { Writable } from "node:stream";
import { createReadStream, createWriteStream } from "node:fs";
import { mkdtemp, rm, stat, statfs } from "node:fs/promises";
import { tmpdir } from "node:os";
@@ -9,7 +11,9 @@ import { join } from "node:path";
import { finished } from "node:stream/promises";
import { Cli } from "incur";
import { estimateTokenCount } from "tokenx";
import { parseCliOutputArguments } from "../../../dist/cliOutput.js";
import { createCliResultOutput } from "../../../dist/cli/cliResultOutput.js";
import { createEvidence } from "../../../dist/domain/evidence.js";
@@ -43,26 +47,47 @@ try {
{
operation: "inspect_fixture",
parameters: {},
result: { payload: Array.from({ length: count }, () => leaf) },
result: { rows: Array.from({ length: count }, () => leaf) },
},
);
sampleRss();
const metadata = { command: "inspect-output-fixture", duration: "0ms" };
const marker = "__REA_OUTPUT_ENCODING_MARKER__";
const template = {
ok: true,
data: { ...evidence, normalized_result: { payload: [marker] } },
meta: metadata,
};
// The destination is a redirected file, so both formats are compact.
const skeleton = JSON.stringify(template);
const encodedMarker = JSON.stringify(marker);
const markerIndex = skeleton.indexOf(encodedMarker);
assert.ok(markerIndex >= 0);
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
const prefix = skeleton.slice(0, markerIndex);
const suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
const separator = ",";
let prefix;
let suffix;
if (format === "json") {
const template = {
ok: true,
data: { ...evidence, normalized_result: { rows: [marker] } },
meta: metadata,
};
const skeleton = JSON.stringify(template);
const encodedMarker = JSON.stringify(marker);
const markerIndex = skeleton.indexOf(encodedMarker);
assert.ok(markerIndex >= 0);
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
prefix = skeleton.slice(0, markerIndex);
suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
} else {
// Independent protocol golden: rows are lines; the final record retains
// every non-row fact and the caller's full-output envelope.
prefix = "";
suffix = `\n${JSON.stringify({
$rea: "jsonl/v1",
type: "metadata",
records: {
path: ["data", "normalized_result", "rows"],
emitted: count,
source: count,
},
context: {
ok: true,
data: { ...evidence, normalized_result: {} },
meta: metadata,
},
})}\n`;
}
const separator = format === "jsonl" ? "\n" : ",";
const expectedHash = createHash("sha256");
expectedHash.update(prefix);
for (let index = 0; index < count; index += 1) {
@@ -79,7 +104,10 @@ try {
destination = createWriteStream(path, { flags: "wx", mode: 0o600 });
const argv = [metadata.command, "--format", format, "--full-output"];
const output = createCliResultOutput(argv, destination);
const output = createCliResultOutput(
parseCliOutputArguments(argv),
destination,
);
assert.ok(output);
let unexpectedOutputBytes = 0;
let exitCode = 0;
@@ -105,6 +133,7 @@ try {
assert.equal(exitCode, 0);
assert.equal(output.handled, true);
assert.equal(output.failed, false);
assert.equal(output.error, undefined);
assert.equal(unexpectedOutputBytes, 0);
const actualHash = createHash("sha256");
@@ -112,12 +141,50 @@ try {
const digest = actualHash.digest("hex");
assert.equal(digest, expectedHash.digest("hex"));
assert.equal((await stat(path)).size, expectedBytes);
let verifiedTokenCount;
if (format === "jsonl") {
let expectedTokens = 0;
for await (const line of createInterface({
input: createReadStream(path),
crlfDelay: Infinity,
}))
expectedTokens += estimateTokenCount(line);
let tokenText = "";
const counterDestination = new Writable({
write(chunk, _encoding, callback) {
// The numeric count is tiny; never retain a mistakenly emitted report.
if (tokenText.length + chunk.length > 64) {
callback(
new Error("Token counter emitted more than a numeric count"),
);
return;
}
tokenText += chunk.toString("utf8");
callback();
},
});
const counter = createCliResultOutput(
parseCliOutputArguments([...argv, "--token-count"]),
counterDestination,
);
assert.ok(counter);
assert.equal(await counter.write(evidence, { ...metadata, format }), true);
assert.equal(counter.error, undefined);
assert.equal(counter.failed, false);
counterDestination.end();
await finished(counterDestination);
assert.equal(tokenText, `${expectedTokens}\n`);
verifiedTokenCount = expectedTokens;
}
sampleRss();
finalReport = {
json_output_verified: true,
format,
output_bytes: expectedBytes,
engine_string_limit: constants.MAX_STRING_LENGTH,
...(verifiedTokenCount === undefined
? {}
: { verified_token_count: verifiedTokenCount }),
output_sha256: digest,
evidence_id: evidence.evidence_id,
rss: { method: "node-memoryUsage-rss-sampled", peak_bytes: peakRss },
+1 -1
View File
@@ -11,7 +11,7 @@ import { verifyManagedNativeBoundariesEvidence } from "../../../dist/application
import { importManagedReconstructionEvidence } from "../../../dist/application/managed/ManagedReconstructionService.js";
import { projectManagedApplicationGraphEvidence } from "../../../dist/application/managed/ManagedApplicationGraphService.js";
import { traceApplicationFeatureEvidence } from "../../../dist/application/javascript/JavaScriptApplicationWorkflowService.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/application/InvestigationProviders.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/domain/investigationProviders.js";
import { createEvidence } from "../../../dist/domain/evidence.js";
import { functionDossier } from "../../fixtures/managed/function-dossier.mjs";
import {
+1 -1
View File
@@ -7,7 +7,7 @@ import { inspectManagedMembersBytes } from "../../../dist/dotnet/ManagedMemberIn
import { projectManagedApplicationGraphEvidence } from "../../../dist/application/managed/ManagedApplicationGraphService.js";
import { traceApplicationFeatureEvidence } from "../../../dist/application/javascript/JavaScriptApplicationWorkflowService.js";
import { createEvidence } from "../../../dist/domain/evidence.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/application/InvestigationProviders.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/domain/investigationProviders.js";
import {
ensure,
methodTokenRow,
+1 -1
View File
@@ -9,7 +9,7 @@ import { inspectManagedArtifactBytes } from "../../../dist/dotnet/ManagedArtifac
import { inspectManagedMembersBytes } from "../../../dist/dotnet/ManagedMemberInspector.js";
import { importManagedReconstructionEvidence } from "../../../dist/application/managed/ManagedReconstructionService.js";
import { createEvidence } from "../../../dist/domain/evidence.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/application/InvestigationProviders.js";
import { MANAGED_STATIC_PROVIDER } from "../../../dist/domain/investigationProviders.js";
import { createManagedManifestVerifier } from "./manifest.mjs";
const execFileAsync = promisify(execFile);
+2 -2
View File
@@ -9,7 +9,7 @@ export const JADX_RELEASE = Object.freeze({
/** Identity of REA's metadata bridge, distinct from the producing engine. */
export const JADX_BRIDGE_IDENTITY = Object.freeze({
name: "rea-jadx-bridge",
version: "4",
version: "5",
});
/** Remediation when `REA_JADX_MCP_JAR` is missing or not an absolute path. */
@@ -62,5 +62,5 @@ export const JADX_LIMITATIONS = [
"Method signatures use provider display types; DEX prototypes come from frozen JADX metadata. Display names may be aliases, and overload indices are local to this engine and artifact.",
"Class inventories use parsed metadata before code generation, including synthetic members that may be omitted from decompiled source. Source positions remain unknown.",
"Incoming reference totals count JADX using nodes. Direct invoke entries within a using method report raw DEX code-unit offsets and method indexes, but coverage is limited to those nodes and directly matched invokes; it does not establish every possible dynamic or reflective call site.",
"Manifest summary fields are extracted by the upstream engine; decoded XML is retained for verification. APK signatures, split APKs, native libraries and Android runtime capture are not verified by this provider.",
"Manifest summaries are parsed once by REA's bridge from the exact root AndroidManifest.xml resource, using namespace-aware XML attributes; decoded XML is retained. APK signatures, split APKs, native libraries and Android runtime capture are not verified by this provider.",
] as const;
+8 -31
View File
@@ -24,37 +24,11 @@ import {
zipWith,
} from "../../tests/fixtures/apkSignature/v1SignatureBlock.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { apkSignatureResultSchema } from "../domain/apkSignature/apkSignatureReport.js";
import { ApkSignatureProvider } from "./ApkSignatureProvider.js";
import { apkSignatureFailure } from "./apkSignatureFailures.js";
import { apkSchemeValue } from "../../tests/fixtures/apkSignature/signingScheme.js";
type V1Signer = {
signature_file: string;
block_file: string;
certificate: { sha256_fingerprint: string; subject: string | null } | null;
certificates: { sha256_fingerprint: string; subject: string | null }[];
signer_infos: Record<string, unknown>[];
decode_error: string | null;
};
type Report = {
schemes: { v1: { present: boolean; signers: V1Signer[] } } & Record<
string,
{
present: boolean;
signers?: {
certificate: unknown;
digests: { algorithm: string; matches_apk: boolean | null }[];
}[];
}
>;
posture: Record<string, unknown>;
source_stamp: {
present: boolean;
certificate: { sha256_fingerprint: string } | null;
decode_error: string | null;
};
};
const zipOf = async (names: readonly string[]): Promise<Uint8Array> => {
const writer = new ZipWriter(new Uint8ArrayWriter());
for (const name of names) await writer.add(name, new TextReader("x"));
@@ -168,10 +142,10 @@ const write = async (
await writeFile(path, bytes);
return path;
};
const reportOf = async (path: string): Promise<Report> => {
const reportOf = async (path: string) => {
const result = await provider.inspect({ path });
if (!result.ok) throw new Error(`expected ok: ${result.error.message}`);
return result.value.result as Report;
return apkSignatureResultSchema.parse(result.value.result);
};
describe("ApkSignatureProvider without the Android SDK", () => {
@@ -389,7 +363,7 @@ describe("ApkSignatureProvider v1 signers", () => {
),
});
if (!result.ok) throw new Error(result.error.message);
const report = result.value.result as Report;
const report = apkSignatureResultSchema.parse(result.value.result);
expect(report.posture.v1_signers_match_block_signers).toBeNull();
expect(result.value.limitations).toContain(
"v1_signers_match_block_signers is null: the v3 signer's declared rotation lineage could not be decoded, so the original certificate v1 is expected to carry is unknown.",
@@ -599,7 +573,10 @@ describe("ApkSignatureProvider failure surfaces", () => {
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect((result.value.result as Report).posture.self_consistent).toBeNull();
expect(
apkSignatureResultSchema.parse(result.value.result).posture
.self_consistent,
).toBeNull();
expect(result.value.limitations.at(-1)).toContain(
"not immediately followed by the end-of-central-directory",
);
+1 -1
View File
@@ -397,7 +397,7 @@ export class ApkSignatureProvider implements ApkSignaturePort {
}
});
} catch (cause: unknown) {
return err(apkSignatureFailure(cause, input.path, options?.signal));
return err(apkSignatureFailure(cause, input.path));
}
}
+6 -10
View File
@@ -1,6 +1,7 @@
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import {
ArtifactFileCleanupFailure,
artifactFilePrimaryFailure,
artifactFileCancellationCause,
bindArtifactFileCleanup,
} from "../artifacts/ArtifactFileOperation.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
@@ -124,15 +125,8 @@ export const apkOutputBudgetError = (projectedBytes: number): AnalysisError =>
export const apkSignatureFailure = (
cause: unknown,
path: string,
signal?: AbortSignal,
): AnalysisError => {
const primary =
cause instanceof ArtifactFileCleanupFailure &&
cause.fileCleanupFailure.outcome.kind === "failed"
? primaryFailure(cause.fileCleanupFailure.outcome.cause, path)
: cause instanceof ArtifactFileCleanupFailure && signal?.aborted
? new AnalysisCancelledError(OPERATION, { cause: signal.reason })
: primaryFailure(cause, path);
const primary = primaryFailure(artifactFilePrimaryFailure(cause), path);
if (!(cause instanceof ArtifactReaderFailure) || cause.cleanup === undefined)
return primary;
const cleanup = new ProviderCleanupError(
@@ -187,7 +181,9 @@ const artifactReaderFailure = (
cause,
});
case "cancelled":
return new AnalysisCancelledError(OPERATION, { cause: cause.cause });
return new AnalysisCancelledError(OPERATION, {
cause: artifactFileCancellationCause(cause),
});
case "limit":
return new AnalysisResourceConstraintError(
OPERATION,
+81
View File
@@ -107,6 +107,87 @@ describe("parseApktoolYml", () => {
targetSdkVersion: null,
});
});
// Source-derived fixtures from Apktool 3.0.3 VersionInfo.write and
// YamlStringEscapeUtils at 18b5e99cb56ff9451e8aa55b065dcf5bbd616975.
it.each([
["1.0 #beta", "1.0 #beta"],
[String.raw`1.0\\beta`, String.raw`1.0\beta`],
[String.raw`1.0 \uD83D\uDE80`, "1.0 🚀"],
[String.raw`literal \\uD83D\\uDE80`, String.raw`literal \uD83D\uDE80`],
[String.raw`release \"quoted\"`, 'release "quoted"'],
["'release'", "'release'"],
["1.0: beta", "1.0: beta"],
["[beta]", "[beta]"],
[" release ", " release "],
["release\u2028beta", "release\u2028beta"],
["null", "null"],
["", ""],
['"1.0 #beta"', "1.0 #beta"],
])(
"reads Apktool 3 version string %j without YAML scalar loss",
(source, expected) => {
expect(
parseApktoolYml(
`version: 3.0.3\nversionInfo:\n versionCode: 7\n versionName: ${source}\nsdkInfo:\n minSdkVersion: 24\n targetSdkVersion: 35\n`,
),
).toEqual({
versionName: expected,
versionCode: "7",
minSdkVersion: "24",
targetSdkVersion: "35",
});
},
);
it("keeps Apktool 2 quoted and plain YAML semantics", () => {
expect(
parseApktoolYml(
String.raw`version: 2.7.0
versionInfo:
versionName: 'literal \n #beta'
versionCode: 0045
sdkInfo:
minSdkVersion: null
targetSdkVersion: 35 # comment
`,
),
).toEqual({
versionName: String.raw`literal \n #beta`,
versionCode: "0045",
minSdkVersion: null,
targetSdkVersion: "35",
});
});
it("keeps Apktool 3 field ownership across nested decoys and CRLF", () => {
expect(
parseApktoolYml(
[
"version: 3.0.3",
"versionName: root decoy",
"versionInfo:",
" custom:",
" versionName: nested decoy",
" versionName: 1.0 #beta",
"other:",
" versionName: unrelated decoy",
"",
].join("\r\n"),
),
).toMatchObject({ versionName: "1.0 #beta" });
});
it.each([
"versionInfo:\n versionName: one\n versionName: two\n",
"versionInfo:\n versionName: one\nversionInfo:\n versionName: two\n",
"version: 2.7.0\nversionInfo:\n versionName: ambiguous\n",
String.raw`versionInfo:
versionName: invalid\uZZZZ
`,
])("rejects ambiguous or malformed Apktool 3 fields", (fields) => {
expect(() => parseApktoolYml(`version: 3.0.3\n${fields}`)).toThrow();
});
});
describe("parseManifestPackage", () => {
+81 -3
View File
@@ -23,11 +23,15 @@ const metaInfoTag = {
};
/**
* Read values from Apktool's tagged metadata document. YAML's core schema
* distinguishes null from quoted strings; numeric values use their source
* range so version codes retain the spelling Apktool emitted.
* Read the producer's metadata dialect once. Apktool 3 emits an untagged,
* line-oriented document with Java-escaped strings; older tagged documents
* use YAML core semantics. Numeric values retain their source spelling.
*/
export const parseApktoolYml = (yml: string): ApktoolMetadata => {
const source = yml.replace(/^\uFEFF/u, "");
const header = source.split(/\r\n?|\n/u, 1)[0] ?? "";
if (/^version: 3\.\d+\.\d+(?:[-+][0-9A-Za-z.+-]+)?$/u.test(header))
return parseApktool3Metadata(source);
const document = parseDocument(yml, {
schema: "core",
customTags: [metaInfoTag],
@@ -52,6 +56,80 @@ export const parseApktoolYml = (yml: string): ApktoolMetadata => {
};
};
/** Fields written by VersionInfo.write and SdkInfo.write in Apktool 3. */
const apktool3Fields = new Map<string, keyof ApktoolMetadata>([
["versionInfo.versionName", "versionName"],
["versionInfo.versionCode", "versionCode"],
["sdkInfo.minSdkVersion", "minSdkVersion"],
["sdkInfo.targetSdkVersion", "targetSdkVersion"],
]);
/**
* Apktool 3.0.3 YamlWriter writes strings without YAML quoting or comment
* syntax. Read its direct mapping records before a YAML parser can discard
* hashes, consume quotes, or reinterpret an empty string or literal `null`.
*/
const parseApktool3Metadata = (source: string): ApktoolMetadata => {
const values = new Map<keyof ApktoolMetadata, string>();
const sections = new Set<string>();
let section: string | undefined;
for (const line of source.split(/\r\n?|\n/u)) {
if (line.trim() === "" || /^[#!]/u.test(line)) continue;
if (!line.startsWith(" ")) {
const separator = line.indexOf(":");
section = separator === -1 ? undefined : line.slice(0, separator);
if (section === "version") {
if (sections.has(section))
throw new TypeError("apktool.yml has a duplicate producer version");
sections.add(section);
continue;
}
if (section !== "versionInfo" && section !== "sdkInfo") continue;
if (line.slice(separator + 1).trim() !== "" || sections.has(section))
throw new TypeError(`apktool.yml has an invalid ${section} mapping`);
sections.add(section);
continue;
}
const record = /^ ([^ :]+):(?: (.*))?$/su.exec(line);
if (record === null) continue;
const path = `${section ?? ""}.${record[1] ?? ""}`;
const field = apktool3Fields.get(path);
if (field === undefined) continue;
if (values.has(field))
throw new TypeError(`apktool.yml has a duplicate ${path} field`);
const value = record[2] ?? "";
values.set(field, field === "versionCode" ? value : apktool3String(value));
}
return {
versionName: values.get("versionName") ?? null,
versionCode: values.get("versionCode") ?? null,
minSdkVersion: values.get("minSdkVersion") ?? null,
targetSdkVersion: values.get("targetSdkVersion") ?? null,
};
};
const apktool3String = (source: string): string => {
// The producer escapes every double quote. An unescaped opening double
// quote therefore belongs to a standard quoted-YAML compatibility input.
if (source.startsWith('"')) {
const document = parseDocument(source, { schema: "core", strict: true });
if (
document.errors.length === 0 &&
isScalar(document.contents) &&
typeof document.contents.value === "string"
)
return document.contents.value;
throw new TypeError("apktool.yml has an invalid quoted string");
}
// Every escape emitted by YamlStringEscapeUtils is also a JSON string
// escape, including UTF-16 surrogate pairs. Decode once, preserving a
// doubled backslash followed by literal `u` or `n` as source text.
const decoded: unknown = JSON.parse(`"${source}"`);
if (typeof decoded !== "string")
throw new TypeError("apktool.yml has an invalid Apktool 3 string");
return decoded;
};
/** Read the package attribute from the decoded Android manifest root. */
export const parseManifestPackage = (manifest: string): string | null => {
const parsed = parseXmlDocument(manifest, {
+53 -1
View File
@@ -108,6 +108,7 @@ const decodeStub = (
readonly manifestFile?: string;
readonly stringsFile?: string;
readonly stringsXml?: string;
readonly version?: string;
readonly extraValuesDirectories?: readonly string[];
} = {},
): ((root: string) => string) => {
@@ -133,7 +134,11 @@ const decodeStub = (
`printf '%s\\n' '<resources><string name="wrong">Wrong</string></resources>' > "$5/res/values-${name}/strings.xml"`,
)
.join("\n ");
return () => `if [ "$1" = "--version" ]; then ${VERSION_BLOCK}; exit 0; fi
const versionBlock =
options.version === undefined
? VERSION_BLOCK
: `printf '%s\\n' '${options.version.replace(/'/gu, "'\\''")}'`;
return () => `if [ "$1" = "--version" ]; then ${versionBlock}; exit 0; fi
if [ "$1" = "d" ]; then
${
options.failDecode === true
@@ -201,6 +206,53 @@ it("returns table-only facts for a single selected primary table", async () => {
});
});
it("publishes Apktool 3 producer-escaped metadata through the shared evidence workflow", async () => {
const { command, root } = await writeApktoolStub((path) =>
decodeStub({
ymlFile: join(path, "producer.yml"),
version: "3.0.3",
})(path),
);
// A synthetic decoded workspace using the released 3.0.3 writer's format;
// this exercises the production provider, not a real Apktool decoder.
await writeFile(
join(root, "producer.yml"),
String.raw`version: 3.0.3
versionInfo:
versionCode: 7
versionName: 1.0 #beta \\ \uD83D\uDE80 \\u0041
sdkInfo:
minSdkVersion: 24
targetSdkVersion: 35
`,
);
const apk = await apkFixture(root);
const original = await readFile(apk.path);
const provider = providerOver(command);
onTestFinished(() => provider.close());
const outcome = await new ApktoolResourceAnalysisService(provider).execute(
"decode_android_resources",
{ path: apk.path },
);
if (!outcome.ok) throw outcome.error;
expect(outcome.value).toMatchObject({
parameters: { path: apk.path },
subject: { digest: { sha256: apk.sha256 } },
normalized_result: {
client: { apktool_version: "3.0.3" },
target: { path: apk.path, sha256: apk.sha256, bytes: apk.bytes },
metadata: {
version_name: String.raw`1.0 #beta \ 🚀 \u0041`,
version_code: "7",
min_sdk_version: "24",
target_sdk_version: "35",
},
coverage: "complete",
},
});
expect(await readFile(apk.path)).toEqual(original);
});
it.each([
[false, false],
[true, false],
@@ -5,8 +5,36 @@ import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { binaryPlistObjectTable } from "../../tests/fixtures/binaryPlistObjectTable.js";
import { resolveAppBundleExecutable } from "./AppBundleExecutable.js";
const binaryPlistExecutableWithThreeByteUid = (): Buffer => {
const ascii = (value: string): Buffer => {
const bytes = Buffer.from(value, "ascii");
return bytes.length < 15
? Buffer.concat([Buffer.from([0x50 | bytes.length]), bytes])
: Buffer.concat([Buffer.from([0x5f, 0x10, bytes.length]), bytes]);
};
const objects = [
Buffer.from([0xd2, 1, 3, 2, 4]),
ascii("CFBundleExecutable"),
ascii("Fixture"),
ascii("ignored"),
Buffer.from([0x82, 1, 2, 3]),
];
const tableOffset =
8 + objects.reduce((sum, object) => sum + object.length, 0);
const bytes = Buffer.alloc(tableOffset + objects.length + 32);
let cursor = 8;
const offsets = objects.map((object) => {
const offset = cursor;
object.copy(bytes, cursor);
cursor += object.length;
return offset;
});
return binaryPlistObjectTable(bytes, offsets, tableOffset);
};
it.each([
["macOS", ["Contents"], ["Contents", "MacOS"]],
["iOS", [], []],
@@ -34,6 +62,21 @@ it.each([
);
describe("app bundle Info.plist magic", () => {
it("decodes an executable when an unrelated UID uses a three-byte width", async () => {
const root = await createTestTempDirectory("rea-app-three-byte-uid-");
const bundle = join(root, "Fixture.app");
const plistPath = join(bundle, "Contents", "Info.plist");
const executable = join(bundle, "Contents", "MacOS", "Fixture");
await mkdir(join(bundle, "Contents", "MacOS"), { recursive: true });
await writeFile(plistPath, binaryPlistExecutableWithThreeByteUid());
await writeFile(executable, "inert fixture; never executed");
expect(await resolveAppBundleExecutable(bundle)).toEqual({
ok: true,
value: { executable, infoPlist: plistPath },
});
});
it("does not decode a high-bit fold of bplist as a binary Info.plist", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-bplist-fold-"));
try {
+7 -2
View File
@@ -5,6 +5,7 @@ import { parseBinary } from "plist";
import { z } from "zod";
import { preflightPropertyListDecode } from "../artifacts/apple/PropertyListDecodeBudget.js";
import { decodeBinaryPlistUids } from "../artifacts/apple/PlistScalars.js";
import { BinaryTargetError } from "../domain/configurationErrors.js";
import { isPathWithinRoot } from "../domain/localPath.js";
import {
@@ -335,10 +336,14 @@ const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() });
/** Decode a binary plist in-process, so no host plist tool is required. */
const parseBinaryPlistExecutable = (plist: Buffer): string => {
preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
const plan = preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
// `plist` assigns a `__proto__` key as the prototype; copying drops it, as
// the XML path does.
return executableName(omitPrototypeKeys(parseBinary(plist)).value);
return executableName(
omitPrototypeKeys(
parseBinary(decodeBinaryPlistUids(plist, plan.uidOffsetsToNormalize)),
).value,
);
};
const isSafeExecutableName = (name: string): boolean =>
+372 -176
View File
@@ -1,4 +1,19 @@
import { regularFileCancellation } from "../filesystem/OwnedRegularFile.js";
import {
regularFileCancellation,
RegularFileCleanupFailure,
RegularFileCancelled,
withOwnedRegularFile,
OwnedRegularFileOperationFailure,
} from "../filesystem/OwnedRegularFile.js";
import {
bindRegularFileCleanup,
RetainedRegularFileCleanupFailure,
retryRegularFileCleanup,
} from "./RegularFileRead.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { constants, type Stats } from "node:fs";
import { createHash } from "node:crypto";
import {
@@ -11,10 +26,7 @@ import {
import { extname } from "node:path";
import { BinaryTargetError } from "../domain/configurationErrors.js";
import {
openRegularFile,
sameRegularFileState,
} from "../filesystem/RegularFile.js";
import { sameRegularFileState } from "../filesystem/RegularFile.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -81,6 +93,7 @@ export interface BinaryTargetResolverOptions {
readonly architecture?: BinaryArchitecture;
readonly entryPoint?: string;
readonly signal?: AbortSignal;
readonly resources?: ArtifactResourceScope;
}
/**
@@ -96,13 +109,12 @@ export const parseBinaryTarget = async (
Result<BinaryTarget, BinaryTargetError | AnalysisCancelledError>
> => {
const cwd = options.cwd ?? process.cwd();
const hostArchitecture = options.hostArchitecture ?? process.arch;
const { targetKind, formatHint, signal, architecture, entryPoint } = options;
const { signal } = options;
const candidate = absoluteCallerPath(input, { cwd });
// Failures before canonicalization concern the selected path itself; later
// failures concern the canonical path opened with symbolic links rejected.
let stage: TargetFilesystemStage = "resolve";
try {
const resolve = async (): Promise<TargetResolution> => {
throwIfTargetResolutionCancelled(signal);
await access(candidate, constants.R_OK);
throwIfTargetResolutionCancelled(signal);
@@ -117,165 +129,344 @@ export const parseBinaryTarget = async (
const { executable: path, infoPlist } = resolved.value;
const bundle =
infoPlist === undefined ? {} : { bundleInfoPlist: infoPlist };
const handle = await openRegularFile(path, {
symlinks: "reject",
signal,
});
try {
throwIfTargetResolutionCancelled(signal);
const metadata = await handle.stat();
throwIfTargetResolutionCancelled(signal);
if (formatHint === "raw-binary") {
if (targetKind !== undefined && targetKind !== "executable")
return err(
new BinaryTargetError(
path,
"raw-binary format requires an executable target kind",
{ constraint: "load_selection_rejected" },
),
return withOwnedRegularFile<TargetResolution>(
path,
{
symlinks: "reject",
...(signal === undefined ? {} : { signal }),
},
async (handle) => {
try {
return await readOpenedBinaryTarget(
handle,
{ path, sourcePath, ...bundle },
options,
);
const admitted = admitRawBinary({
architecture,
entryPoint,
fileBytes: metadata.size,
});
if (!admitted.ok)
return err(
new BinaryTargetError(path, admitted.error.reason, {
constraint: admitted.error.constraint,
}),
);
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
format: "raw-binary",
architecture: admitted.value.architecture,
availableArchitectures: [admitted.value.architecture],
entryPoint: admitted.value.entryPoint,
entryPointSource: admitted.value.entryPointSource,
});
}
const pairing = rawBinarySelectionError({
...(formatHint === undefined ? {} : { format: formatHint }),
architecture,
entryPoint,
});
if (pairing !== undefined)
} catch (cause: unknown) {
throw new OwnedRegularFileOperationFailure(cause);
}
},
);
};
try {
const resolved = await (options.resources === undefined
? resolve()
: options.resources.run(resolve));
return finalizedTargetResolution(resolved, signal);
} catch (cause: unknown) {
// A caller-selected abort value never transfers file ownership.
if (signal?.aborted === true && cause === signal.reason)
return err(
new AnalysisCancelledError("resolve_binary_target", { cause }),
);
if (cause instanceof RegularFileCleanupFailure) {
const fileFailure = cause as RegularFileCleanupFailure<TargetResolution>;
if (options.resources === undefined)
return err(
new BinaryTargetError(path, pairing, {
constraint: "load_selection_rejected",
}),
targetFileCleanupError(
new RetainedRegularFileCleanupFailure(fileFailure),
candidate,
stage,
),
);
if (formatHint === "dos-com") {
if (targetKind !== undefined && targetKind !== "executable")
return err(
new BinaryTargetError(
path,
"DOS COM format requires an executable target kind",
{ constraint: "load_selection_rejected" },
),
);
const length = validateDosComLength(metadata.size);
if (!length.ok)
return err(
new BinaryTargetError(path, length.error, {
constraint: "file_format_rejected",
}),
);
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
format: "dos-com",
architecture: "x86",
availableArchitectures: ["x86"],
});
}
// Probe WASM content before suffix-only database routing; an explicit
// database selection keeps the caller's chosen interpretation.
const artifactFormat =
targetKind === "database"
? undefined
: await detectArtifactFormat(path, handle, signal);
if (
targetKind === "database" ||
(targetKind === undefined && path.toLowerCase().endsWith(".hop"))
)
return ok({
path,
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "database",
format: "analysis-database",
});
if (artifactFormat !== undefined) {
const identity = {
path,
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
};
return isArchiveFormat(artifactFormat)
? ok({ ...identity, kind: "archive", format: artifactFormat })
: ok({ ...identity, kind: "artifact", format: artifactFormat });
}
const detected = await readExecutableMetadata(
handle,
hostArchitecture,
const retried = await retryRegularFileCleanup(
fileFailure,
options.resources,
signal,
);
if (!detected.ok) {
// Only bytes no other format claims may lead a located ZIP, so an
// executable with an appended archive keeps its executable route.
if (
!(await claimsLeadingExecutableHeader(
handle,
metadata.size,
signal,
)) &&
(await locateTrailingZipArchive(metadata.size, (position, length) =>
readHandleRange(handle, position, length, signal),
)) !== undefined
)
return ok({
path,
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "archive",
format: zipRootPackageFormat(path),
});
const reason =
typeof detected.error === "string"
? detected.error
: detected.error.reason;
const diagnostics =
typeof detected.error === "string"
? undefined
: detected.error.diagnostics;
return err(
new BinaryTargetError(path, reason, {
constraint: "file_format_rejected",
...(diagnostics === undefined ? {} : { diagnostics }),
}),
if (retried.failure !== undefined)
return err(targetFileCleanupError(retried.failure, candidate, stage));
if (retried.outcome.kind === "failed")
return targetResolutionFailure(
fileFailure.operationFailure ?? retried.outcome.cause,
candidate,
stage,
signal,
);
}
return finalizedTargetResolution(retried.outcome.value, signal);
}
if (cause instanceof RegularFileCancelled)
return finalizedTargetResolution(
cause.outcome.value as TargetResolution,
signal,
);
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) {
const error = new BinaryTargetError(
candidate,
"A previous target file could not be closed",
{
cause,
cleanup: cause.cleanup,
partialObservation: cause.partialObservation,
},
);
bindRegularFileCleanup(cause, error);
return err(error);
}
return targetResolutionFailure(cause, candidate, stage, signal);
}
};
type TargetResolution = Result<
BinaryTarget,
BinaryTargetError | AnalysisCancelledError
>;
/** Classify and hash the one descriptor admitted by the lifetime owner. */
const readOpenedBinaryTarget = async (
handle: FileHandle,
selected: {
readonly path: string;
readonly sourcePath: string;
readonly bundleInfoPlist?: string;
},
options: BinaryTargetResolverOptions,
): Promise<TargetResolution> => {
const { path, sourcePath, bundleInfoPlist } = selected;
const bundle = bundleInfoPlist === undefined ? {} : { bundleInfoPlist };
const hostArchitecture = options.hostArchitecture ?? process.arch;
const { targetKind, formatHint, signal, architecture, entryPoint } = options;
throwIfTargetResolutionCancelled(signal);
const metadata = await handle.stat();
throwIfTargetResolutionCancelled(signal);
if (formatHint === "raw-binary") {
if (targetKind !== undefined && targetKind !== "executable")
return err(
new BinaryTargetError(
path,
"raw-binary format requires an executable target kind",
{ constraint: "load_selection_rejected" },
),
);
const admitted = admitRawBinary({
architecture,
entryPoint,
fileBytes: metadata.size,
});
if (!admitted.ok)
return err(
new BinaryTargetError(path, admitted.error.reason, {
constraint: admitted.error.constraint,
}),
);
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
format: "raw-binary",
architecture: admitted.value.architecture,
availableArchitectures: [admitted.value.architecture],
entryPoint: admitted.value.entryPoint,
entryPointSource: admitted.value.entryPointSource,
});
}
const pairing = rawBinarySelectionError({
...(formatHint === undefined ? {} : { format: formatHint }),
architecture,
entryPoint,
});
if (pairing !== undefined)
return err(
new BinaryTargetError(path, pairing, {
constraint: "load_selection_rejected",
}),
);
if (formatHint === "dos-com") {
if (targetKind !== undefined && targetKind !== "executable")
return err(
new BinaryTargetError(
path,
"DOS COM format requires an executable target kind",
{ constraint: "load_selection_rejected" },
),
);
const length = validateDosComLength(metadata.size);
if (!length.ok)
return err(
new BinaryTargetError(path, length.error, {
constraint: "file_format_rejected",
}),
);
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
format: "dos-com",
architecture: "x86",
availableArchitectures: ["x86"],
});
}
// Probe WASM content before suffix-only database routing; an explicit
// database selection keeps the caller's chosen interpretation.
const artifactFormat =
targetKind === "database"
? undefined
: await detectArtifactFormat(path, handle, signal);
if (
targetKind === "database" ||
(targetKind === undefined && path.toLowerCase().endsWith(".hop"))
)
return ok({
path,
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "database",
format: "analysis-database",
});
if (artifactFormat !== undefined) {
const identity = {
path,
sourcePath,
sha256: await sha256Handle(handle, path, metadata, signal),
};
return isArchiveFormat(artifactFormat)
? ok({ ...identity, kind: "archive", format: artifactFormat })
: ok({ ...identity, kind: "artifact", format: artifactFormat });
}
const detected = await readExecutableMetadata(
handle,
hostArchitecture,
signal,
);
if (!detected.ok) {
// Only bytes no other format claims may lead a located ZIP, so an
// executable with an appended archive keeps its executable route.
if (
!(await claimsLeadingExecutableHeader(handle, metadata.size, signal)) &&
(await locateTrailingZipArchive(metadata.size, (position, length) =>
readHandleRange(handle, position, length, signal),
)) !== undefined
)
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
...detected.value,
kind: "archive",
format: zipRootPackageFormat(path),
});
} finally {
await handle.close();
}
} catch (cause: unknown) {
if (cause instanceof AnalysisCancelledError) return err(cause);
const reason =
typeof detected.error === "string"
? detected.error
: detected.error.reason;
const diagnostics =
typeof detected.error === "string"
? undefined
: detected.error.diagnostics;
return err(
new BinaryTargetError(path, reason, {
constraint: "file_format_rejected",
...(diagnostics === undefined ? {} : { diagnostics }),
}),
);
}
return ok({
path,
sourcePath,
...bundle,
sha256: await sha256Handle(handle, path, metadata, signal),
kind: "executable",
...detected.value,
});
};
/** Preserve an earlier admission refusal over an interruption during final close. */
const finalizedTargetResolution = (
result: TargetResolution,
signal?: AbortSignal,
): TargetResolution => {
if (!result.ok || signal?.aborted !== true) return result;
return err(
new AnalysisCancelledError("resolve_binary_target", {
cause: signal.reason,
diagnostics: { completed_target: jsonValueSchema.parse(result.value) },
}),
);
};
/** Descriptor cleanup retains the physical result and binds its exact release lease. */
const targetFileCleanupError = (
failure: RetainedRegularFileCleanupFailure<TargetResolution>,
candidate: string,
stage: TargetFilesystemStage,
): BinaryTargetError => {
const cancellation = regularFileCancellation(failure);
const completed =
failure.outcome.kind === "completed" ? failure.outcome.value : undefined;
const completedTarget =
completed?.ok === true ? jsonValueSchema.parse(completed.value) : undefined;
let primary: BinaryTargetError | AnalysisCancelledError | undefined;
if (failure.outcome.kind === "failed") {
const physical = failure.outcome.cause;
primary =
physical instanceof BinaryTargetError ||
physical instanceof AnalysisCancelledError
? physical
: cancellation !== undefined
? new AnalysisCancelledError("resolve_binary_target", {
cause: cancellation.reason,
})
: (classifyTargetFilesystemFailure(candidate, physical, stage) ??
new BinaryTargetError(
failure.path,
"Target read failed before file cleanup",
{ cause: physical },
));
} else if (completed?.ok === false) primary = completed.error;
else if (cancellation !== undefined)
primary = new AnalysisCancelledError("resolve_binary_target", {
cause: cancellation.reason,
...(completedTarget === undefined
? {}
: { diagnostics: { completed_target: completedTarget } }),
});
const error = new BinaryTargetError(
failure.path,
"The target file handle could not be closed",
{
cause: failure,
cleanupPrimary: primary,
cleanup: {
reason:
failure.cleanupCause instanceof Error
? failure.cleanupCause.message
: String(failure.cleanupCause),
resources: [failure.path],
},
diagnostics: {
...(primary === undefined
? {}
: { primary_error: projectAnalysisError(primary) }),
...(completedTarget === undefined
? {}
: { completed_target: completedTarget }),
},
},
);
bindRegularFileCleanup(failure, error);
return error;
};
const targetResolutionFailure = (
cause: unknown,
candidate: string,
stage: TargetFilesystemStage,
signal?: AbortSignal,
): TargetResolution => {
// Physical rejection values do not grant ownership of foreign descriptors.
const physical =
cause instanceof OwnedRegularFileOperationFailure
? cause.operationCause
: cause;
if (
physical instanceof AnalysisCancelledError ||
physical instanceof BinaryTargetError
)
return err(physical);
if (!(cause instanceof OwnedRegularFileOperationFailure)) {
const cancellation = regularFileCancellation(cause);
if (cancellation !== undefined)
return err(
@@ -283,16 +474,14 @@ export const parseBinaryTarget = async (
cause: cancellation.reason,
}),
);
if (signal?.aborted === true && cause === signal.reason)
return err(
new AnalysisCancelledError("resolve_binary_target", { cause }),
);
if (cause instanceof BinaryTargetError) return err(cause);
if (!(cause instanceof Error)) throw cause;
const failure = classifyTargetFilesystemFailure(candidate, cause, stage);
if (failure === undefined) throw cause;
return err(failure);
}
if (signal?.aborted === true && physical === signal.reason)
return err(
new AnalysisCancelledError("resolve_binary_target", { cause: physical }),
);
const failure = classifyTargetFilesystemFailure(candidate, physical, stage);
if (failure === undefined) throw physical;
return err(failure);
};
/**
@@ -359,7 +548,9 @@ const targetFailureReason = (
const throwIfTargetResolutionCancelled = (signal?: AbortSignal): void => {
if (signal?.aborted === true)
throw new AnalysisCancelledError("resolve_binary_target");
throw new AnalysisCancelledError("resolve_binary_target", {
cause: signal.reason,
});
};
const detectArtifactFormat = async (
@@ -574,7 +765,9 @@ const readExecutableMetadata = async (
throwIfTargetResolutionCancelled(signal);
if (bytes.length >= 2 && bytes[0] === 0x4d && bytes[1] === 0x5a) {
let evidence = parseMzHeaderEvidence(bytes, fileSize);
let signatureBytes: Buffer | undefined;
let signatureEvidence:
| { readonly offset: number; readonly bytes: Buffer }
| undefined;
if (evidence.windowsHeader.state === "signature-needed") {
const { offset } = evidence.windowsHeader;
const signature = Buffer.alloc(Math.min(4, fileSize - offset));
@@ -585,8 +778,11 @@ const readExecutableMetadata = async (
offset,
);
throwIfTargetResolutionCancelled(signal);
signatureBytes = signature.subarray(0, observedSignature.bytesRead);
evidence = parseMzHeaderEvidence(bytes, fileSize, signatureBytes);
signatureEvidence = {
offset,
bytes: signature.subarray(0, observedSignature.bytesRead),
};
evidence = parseMzHeaderEvidence(bytes, fileSize, signatureEvidence);
}
const { windowsHeader } = evidence;
if (windowsHeader.state === "declared") {
@@ -618,7 +814,7 @@ const readExecutableMetadata = async (
evidence = parseMzHeaderEvidence(
table.subarray(0, observed.bytesRead),
fileSize,
signatureBytes,
signatureEvidence,
);
if (evidence.windowsHeader.state === "candidate")
return err(
@@ -628,6 +824,7 @@ const readExecutableMetadata = async (
table.subarray(0, observed.bytesRead),
hostArchitecture,
fileSize,
signatureEvidence,
);
}
if (windowsHeader.state === "candidate")
@@ -636,7 +833,7 @@ const readExecutableMetadata = async (
bytes,
hostArchitecture,
fileSize,
signatureBytes,
signatureEvidence,
);
}
const fatEvidence = await readFatSliceHeaderEvidence(
@@ -797,11 +994,10 @@ const readPeMetadata = async (
throwIfTargetResolutionCancelled(signal);
if (fileHeaderRead.bytesRead !== fileHeader.length)
return err("invalid or truncated PE header");
const windowsHeader = observeMzWindowsHeader(
mzPrefix,
fileSize,
fileHeader.subarray(0, 4),
);
const windowsHeader = observeMzWindowsHeader(mzPrefix, fileSize, {
offset,
bytes: fileHeader.subarray(0, 4),
});
if (windowsHeader.state !== "declared")
return err("invalid Windows new-header signature in MZ image");
if (["NE", "LE", "LX"].includes(windowsHeader.signature))
+26 -21
View File
@@ -35,11 +35,14 @@ export interface CallPathTraceInput {
readonly direction: CallRelationshipDirection;
}
interface TraceNode {
readonly address: string;
readonly depth: number;
readonly parent: TraceNode | undefined;
}
interface TraceState {
readonly visited: Map<
string,
{ readonly depth: number; readonly path: string[] }
>;
readonly visited: Map<string, TraceNode>;
readonly queue: string[];
readonly edges: Map<
string,
@@ -64,7 +67,7 @@ interface AddressExpansion {
readonly input: CallPathTraceInput;
readonly state: TraceState;
readonly address: string;
readonly current: { readonly depth: number; readonly path: string[] };
readonly current: TraceNode;
readonly signal?: AbortSignal;
}
@@ -123,7 +126,9 @@ export const traceCallPath = async (
};
const createTraceState = (input: CallPathTraceInput): TraceState => ({
visited: new Map([[input.start, { depth: 0, path: [input.start] }]]),
visited: new Map([
[input.start, { address: input.start, depth: 0, parent: undefined }],
]),
queue: [input.start],
edges: new Map(),
failures: [],
@@ -195,23 +200,15 @@ const expandAddress = async (
relatedAddress,
discoveryDepth: current.depth + 1,
});
const path = state.visited.get(relatedAddress)?.path ?? [
...current.path,
relatedAddress,
];
if (state.visited.has(relatedAddress)) {
if (relatedAddress === input.goal) {
state.traversalPath = path;
return ok(true);
}
continue;
}
state.visited.set(relatedAddress, {
if (state.visited.has(relatedAddress)) continue;
const discovered: TraceNode = {
address: relatedAddress,
depth: current.depth + 1,
path,
});
parent: current,
};
state.visited.set(relatedAddress, discovered);
if (relatedAddress === input.goal) {
state.traversalPath = path;
state.traversalPath = reconstructPath(discovered);
return ok(true);
}
state.queue.push(relatedAddress);
@@ -219,6 +216,14 @@ const expandAddress = async (
return ok(false);
};
/** Only the selected goal needs a path; each discovered node owns one parent link. */
const reconstructPath = (goal: TraceNode): string[] => {
const path: string[] = [];
for (let node: TraceNode | undefined = goal; node; node = node.parent)
path.push(node.address);
return path.reverse();
};
const recordEdge = (
input: CallPathTraceInput,
state: TraceState,
@@ -0,0 +1,102 @@
import { expect, it } from "vitest";
import {
parseClientConfiguration,
serializeClientConfiguration,
withClientServers,
} from "./ClientConfigurationDocument.js";
const original = [
'root = "retained # root string"',
"[mcp_servers.rea] # server header note",
'command = "old # command token" # command note',
"args = [",
' "argument # value token", # array note',
"]",
'description = """',
"# multiline value token",
'""" # multiline note',
"literal = 'literal # value token' # literal note",
"[mcp_servers.rea.tools.binary_session] # tool header note",
'approval_mode = "approve" # caller approval',
"[mcp_servers.other]",
'command = "other" # peer note',
"",
].join("\n");
for (const format of ["toml", "grok"] as const)
for (const lineBreak of ["\n", "\r\n"] as const)
it.each(["replace", "remove"] as const)(
`retains real comments through %s in ${format} with ${JSON.stringify(lineBreak)}`,
(operation) => {
const source = original.replaceAll("\n", lineBreak);
const parsed = parseClientConfiguration(source, format);
const { rea: removed, ...peers } = parsed.servers;
expect(removed).toBeDefined();
const servers =
operation === "replace"
? { ...peers, rea: { command: "new", args: [] } }
: peers;
const result = serializeClientConfiguration(
withClientServers(parsed, servers),
format,
source,
);
expect(result.ok).toBe(true);
if (!result.ok) throw new Error("TOML edit was refused");
for (const comment of [
"# server header note",
"# command note",
"# array note",
"# multiline note",
"# literal note",
"# tool header note",
"# caller approval",
"# peer note",
])
expect(result.value.split(comment)).toHaveLength(2);
for (const stringContent of [
"# command token",
"# value token",
"# multiline value token",
])
expect(result.value).not.toContain(stringContent);
expect(parseClientConfiguration(result.value, format).document).toEqual(
{
root: "retained # root string",
mcp_servers: servers,
},
);
if (lineBreak === "\r\n")
expect(result.value.replaceAll("\r\n", "")).not.toContain("\n");
},
);
it("retains comments from dotted REA assignments and nested inline values", () => {
for (const source of [
'mcp_servers.rea.command = "old # string token" # dotted note\n',
'mcp_servers = { rea = { args = ["old # string token", # inline value note\n] }, other = { command = "other" } }\n',
]) {
const parsed = parseClientConfiguration(source, "toml");
const { rea: removed, ...servers } = parsed.servers;
expect(removed).toBeDefined();
const result = serializeClientConfiguration(
withClientServers(parsed, servers),
"toml",
source,
);
expect(result.ok).toBe(true);
if (!result.ok) throw new Error("TOML removal was refused");
expect(result.value).toContain(
source.includes("# dotted note")
? "# dotted note"
: "# inline value note",
);
expect(result.value).not.toContain("# string token");
if (source.includes("# inline value note"))
expect(result.value.split("\n")[0]).not.toContain("# inline value note");
expect(parseClientConfiguration(result.value, "toml").servers).toEqual(
servers,
);
}
});
+79 -21
View File
@@ -17,7 +17,6 @@ import {
createScanner,
format as formatJson,
getNodeValue,
parse as parseJsonc,
parseTree,
printParseErrorCode,
type Edit,
@@ -58,7 +57,14 @@ export interface ClientConfigurationDocument {
readonly legacyServers: Record<string, unknown>;
}
const objectSchema = z.record(z.string(), z.unknown());
/** Admit source mappings without Zod's record clone dropping caller keys. */
export const clientConfigurationRecordSchema = z
.custom<Record<string, unknown>>((value) => {
if (typeof value !== "object" || value === null) return false;
const prototype: unknown = Object.getPrototypeOf(value);
return prototype === null || prototype === Object.prototype;
})
.transform((value) => Object.fromEntries(Object.entries(value)));
/** Compare parsed configuration values without depending on parser prototypes. */
export const clientConfigurationValuesEqual = (
@@ -171,24 +177,27 @@ const parseDocument = (
const errors: ParseError[] = [];
// Accept a UTF-8 BOM without shifting diagnostics or editing the original text.
const jsonText = text.startsWith("\uFEFF") ? ` ${text.slice(1)}` : text;
const document = parseJsonc(jsonText, errors, { allowTrailingComma: true });
const root = parseTree(jsonText, errors, { allowTrailingComma: true });
const firstError = errors[0];
if (firstError !== undefined)
throw new SyntaxError(
`Invalid JSON/JSONC at offset ${firstError.offset}: ${printParseErrorCode(firstError.error)}`,
);
return requireObject(document, "The configuration root");
return requireObject(
root === undefined ? undefined : getNodeValue(root),
"The configuration root",
);
};
/** A parsed configuration whose structure, not its syntax, cannot hold MCP servers. */
export class ClientConfigurationShapeError extends TypeError {}
/** Validate and copy own properties, excluding non-record YAML values and parser-injected prototypes. */
/** Validate source mappings and copy their own properties without rewriting values. */
const requireObject = (
value: unknown,
description: string,
): Record<string, unknown> => {
const parsed = objectSchema.safeParse(value);
const parsed = clientConfigurationRecordSchema.safeParse(value);
if (parsed.success) return parsed.data;
throw new ClientConfigurationShapeError(`${description} must be an object`, {
cause: parsed.error,
@@ -283,7 +292,9 @@ export const effectiveClientServer = (
): unknown =>
Object.hasOwn(parsed.servers, name)
? parsed.servers[name]
: parsed.legacyServers[name];
: Object.hasOwn(parsed.legacyServers, name)
? parsed.legacyServers[name]
: undefined;
/**
* Return `parsed.document` with `servers` at its server table. A legacy OpenCode
@@ -299,7 +310,9 @@ export const withClientServers = (
return Object.hasOwn(servers, PRODUCT_IDENTITY.mcpServerKey)
? editDshDocument(
parsed,
objectSchema.parse(servers[PRODUCT_IDENTITY.mcpServerKey]),
clientConfigurationRecordSchema.parse(
servers[PRODUCT_IDENTITY.mcpServerKey],
),
)
: editDshDocument(parsed);
const [key = "", nested] = parsed.serversPath;
@@ -308,7 +321,9 @@ export const withClientServers = (
// ZCode `config.json` without an `mcp` object), so treat absence as empty.
const outerValue = parsed.document[key];
const outer =
outerValue === undefined ? {} : { ...objectSchema.parse(outerValue) };
outerValue === undefined
? {}
: clientConfigurationRecordSchema.parse(outerValue);
if (
parsed.dialect === "opencode_v2" &&
removeLegacy !== undefined &&
@@ -325,7 +340,7 @@ const valueAt = (
): unknown => {
let current: unknown = document;
for (const key of path) {
const object = objectSchema.safeParse(current);
const object = clientConfigurationRecordSchema.safeParse(current);
if (!object.success || !Object.hasOwn(object.data, key)) return undefined;
current = object.data[key];
}
@@ -756,11 +771,14 @@ const editInlineServerTable = (
table: string,
serverKey: string,
entry: unknown,
): string | undefined => {
):
| { readonly text: string; readonly comments: readonly string[] }
| undefined => {
if (!table.startsWith("{")) return undefined;
const end = endOfContainer(table, 0);
if (table[end - 1] !== "}") return undefined;
const pairs: InlinePair[] = [];
const comments: string[] = [];
let index = 1;
let trailing = "";
while (index < end - 1) {
@@ -775,10 +793,14 @@ const editInlineServerTable = (
if (table[equals] !== "=") return undefined;
const valueEnd = endOfTomlValue(table, equals + 1);
if (valueEnd > end - 1) return undefined;
const pairText = table.slice(trivia.next, valueEnd);
const drop = key.segments[0] === serverKey;
if (drop)
for (const comment of tomlCommentLines(pairText)) comments.push(comment);
pairs.push({
leading: trivia.text,
text: table.slice(trivia.next, valueEnd),
drop: key.segments[0] === serverKey,
text: pairText,
drop,
});
index = valueEnd;
}
@@ -787,8 +809,8 @@ const editInlineServerTable = (
? undefined
: `${tomlInlineKey(serverKey)} = ${tomlInlineValue(entry)}`;
if (insertion === undefined && pairs.every((pair) => !pair.drop))
return table;
return rebuildInlineTable(pairs, trailing, insertion);
return { text: table, comments };
return { text: rebuildInlineTable(pairs, trailing, insertion), comments };
};
const rebuildInlineTable = (
@@ -977,10 +999,32 @@ const withoutTrailingLineFeeds = (text: string): string => {
return text.slice(0, end);
};
/** Retain comment trivia from removed code without promoting string contents. */
const tomlCommentLines = (text: string): readonly string[] => {
const comments: string[] = [];
let index = 0;
while (index < text.length) {
if (text.startsWith('"""', index) || text.startsWith("'''", index)) {
const basic = text[index] === '"';
index =
closeMultiline(text, index + 3, basic ? '"""' : "'''", basic) ??
text.length;
} else if (text[index] === '"' || text[index] === "'") {
index = parseQuotedKey(text, index)?.next ?? text.length;
} else if (text[index] === "#") {
const end = text.indexOf("\n", index);
comments.push(text.slice(index, end < 0 ? text.length : end));
index = end < 0 ? text.length : end + 1;
} else index += 1;
}
return comments;
};
/**
* Replace REA's TOML server tables and assignments. Every comment line stays
* where it was, including comments before, inside, and after REA's own table;
* only blank lines that separated a removed table go with it. Text inside
* inline comments on replaced code become standalone comment lines.
* Only blank lines that separated a removed table go with it. Text inside
* strings is not treated as a header. Only Grok Build reads the root
* `disabled_mcp_servers` list that registration also edits.
*/
@@ -1040,6 +1084,7 @@ const upsertTomlServerSection = (
tablePath = header.path;
if (isTomlServerTable(header.path, serverKey)) {
discardPending();
appendLines(tomlCommentLines(physical));
skipping = true;
lineIndex += 1;
continue;
@@ -1068,6 +1113,7 @@ const upsertTomlServerSection = (
endLine += 1;
if (skipping) {
discardPending();
appendLines(tomlCommentLines(text.slice(lineStart, statementEnd)));
lineIndex = endLine + 1;
mode = "none";
continue;
@@ -1117,10 +1163,16 @@ const upsertTomlServerSection = (
const replacement =
edited === undefined
? text.slice(lineStart, statementEnd)
: `${prefix}${edited}${suffix}`;
: `${prefix}${edited.text}${suffix}`;
appendLines(replacement.split("\n"));
// TOML 1.0 inline tables cannot gain standalone comments between
// their fields. Retain removed value comments after the assignment.
if (edited !== undefined) appendLines(edited.comments);
inlineServerTable = true;
} else discardPending();
} else {
discardPending();
appendLines(tomlCommentLines(text.slice(lineStart, statementEnd)));
}
} else {
flushPending();
for (let index = lineIndex; index <= endLine; index += 1)
@@ -1181,7 +1233,9 @@ const tomlServerEntry = (
document: Record<string, unknown>,
serverKey: string,
): unknown => {
const servers = objectSchema.safeParse(document.mcp_servers);
const servers = clientConfigurationRecordSchema.safeParse(
document.mcp_servers,
);
if (!servers.success || !Object.hasOwn(servers.data, serverKey))
return undefined;
return servers.data[serverKey];
@@ -1255,7 +1309,9 @@ const serializeGrokConfiguration = (
const withoutEmptyServerTable = (
document: Record<string, unknown>,
): Record<string, unknown> => {
const servers = objectSchema.safeParse(document.mcp_servers);
const servers = clientConfigurationRecordSchema.safeParse(
document.mcp_servers,
);
if (!servers.success || Object.keys(servers.data).length > 0) return document;
const { mcp_servers: empty, ...rest } = document;
void empty;
@@ -1291,7 +1347,9 @@ const tomlTextHolds = (
): boolean => {
try {
return clientConfigurationValuesEqual(
withoutEmptyServerTable(objectSchema.parse(parseClientToml(text))),
withoutEmptyServerTable(
clientConfigurationRecordSchema.parse(parseClientToml(text)),
),
withoutEmptyServerTable(document),
);
} catch (cause: unknown) {
+1 -1
View File
@@ -164,7 +164,7 @@ export const effectiveClientConfiguration = (
const last = documents.at(-1);
if (last === undefined) return undefined;
if (documents.length === 1) return last;
const servers: Record<string, unknown> = {};
const servers: Record<string, unknown> = Object.create(null);
for (const document of documents) {
const names = new Set([
...Object.keys(document.legacyServers),
+218
View File
@@ -0,0 +1,218 @@
import { z } from "zod";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import {
MCP_STARTUP_POLICY,
recommendedMcpToolTimeoutMs,
recommendedMcpToolTimeoutSeconds,
type McpToolTimeoutInputs,
} from "../mcpStartupPolicy.js";
import {
clientConfigurationRecordSchema,
clientRegistrationEntry,
type ClientRegistrationDialect,
} from "./ClientConfigurationDocument.js";
import type { SetupClient } from "./SupportedClients.js";
const tomlTimeoutSchema = z.union([
z.number().positive(),
z.bigint().positive(),
]);
/** Common launcher facts, without assigning a shared meaning to client policy. */
const registrationSchema = z
.object({
command: z.string().min(1),
args: z.array(z.string()).default([]),
startup_timeout_sec: tomlTimeoutSchema.optional(),
tool_timeout_sec: tomlTimeoutSchema.optional(),
startupTimeoutMs: z.number().positive().optional(),
toolTimeoutMs: z.number().positive().optional(),
type: z.string().optional(),
transport: z.string().optional(),
serverName: z.string().optional(),
disabled: z.boolean().optional(),
enabled: z.boolean().optional(),
})
.passthrough();
export type ClientRegistration = z.output<typeof registrationSchema>;
// https://developers.openai.com/codex/config-reference/#mcp_servers
const codexApprovalModeSchema = z.enum(["auto", "prompt", "writes", "approve"]);
const codexToolPolicySchema = z
.object({
approval_mode: codexApprovalModeSchema.optional(),
output_token_limit: z
.union([z.number().int().positive(), z.bigint().positive()])
.optional(),
})
.passthrough();
const codexPolicySchema = z
.object({
default_tools_approval_mode: codexApprovalModeSchema.optional(),
enabled_tools: z.array(z.string()).optional(),
disabled_tools: z.array(z.string()).optional(),
tools: clientConfigurationRecordSchema
.superRefine((tools, context) => {
for (const [name, value] of Object.entries(tools)) {
const policy = codexToolPolicySchema.safeParse(value);
if (!policy.success)
for (const issue of policy.error.issues)
context.addIssue({ ...issue, path: [name, ...issue.path] });
}
})
.optional(),
})
.passthrough();
const registrationError = (error: z.ZodError): ConfigurationError =>
new ConfigurationError("Invalid client registration settings", {
settings: error.issues.map((issue) => ({
setting: issue.path.join("."),
constraint: issue.message,
})),
});
/** Policy belongs to the client dialect, and is validated before refresh writes. */
const validateClientPolicy = (
value: unknown,
client: SetupClient,
): Result<void, ConfigurationError> => {
const policy =
client.name === "codex"
? codexPolicySchema.safeParse(value)
: client.format === "copilot_cli"
? z.object({ tools: z.array(z.string()).optional() }).safeParse(value)
: undefined;
return policy === undefined || policy.success
? ok(undefined)
: err(registrationError(policy.error));
};
/** Normalize launcher facts once; policy fields keep their dialect-specific shape. */
export const parseClientRegistration = (
value: unknown,
client: SetupClient,
): Result<ClientRegistration, ConfigurationError> => {
const policy = validateClientPolicy(value, client);
if (!policy.ok) return policy;
if (client.format === "dsh") {
const parsed = registrationSchema
.pick({ command: true, args: true, serverName: true, transport: true })
.strip()
.safeParse(value);
return parsed.success
? ok(parsed.data)
: err(registrationError(parsed.error));
}
if (client.format === "opencode") {
const entry = z
.object({
type: z.literal("local"),
command: z.array(z.string()).min(1),
environment: z.record(z.string(), z.string()).optional(),
})
.passthrough()
.safeParse(value);
if (!entry.success) return err(registrationError(entry.error));
const [command = "", ...args] = entry.data.command;
value = { ...entry.data, command, args };
}
const parsed = registrationSchema.safeParse(value);
if (!parsed.success) return err(registrationError(parsed.error));
const registration = parsed.data;
const transportInvalid =
((client.format === "kimi" || client.format === "commandcode") &&
registration.transport !== undefined &&
registration.transport !== "stdio") ||
(client.format === "commandcode" && registration.transport === undefined);
const typeInvalid =
((client.format === "vscode" || client.format === "copilot_cli") &&
registration.type !== "stdio") ||
((client.format === "kimi" ||
client.format === "omp" ||
client.format === "pi") &&
registration.type !== undefined &&
registration.type !== "stdio");
if (transportInvalid || typeInvalid)
return err(
new ConfigurationError("Expected an stdio registration", {
settings: [
{
setting: transportInvalid ? "transport" : "type",
constraint: "Expected stdio",
},
],
}),
);
return ok(registration);
};
// These fields select REA's connection and activation. Everything else belongs
// to the caller/client, including future policy keys. Client-specific timeout
// defaults and preservation rules are applied below.
const managedConnectionKeys = new Set([
"command",
"args",
"env",
"environment",
"type",
"transport",
"enabled",
"disabled",
"enable",
"serverName",
"url",
"httpUrl",
]);
/** Patch a source fragment, retaining client-owned settings rather than rebuilding it. */
export const refreshClientRegistration = (
client: SetupClient,
dialect: ClientRegistrationDialect,
command: readonly string[],
environment: Readonly<Record<string, string>>,
previous: unknown,
timeoutInputs: McpToolTimeoutInputs,
): Result<Record<string, unknown>, ConfigurationError> => {
const source = clientConfigurationRecordSchema.safeParse(previous);
const existing = source.success ? source.data : {};
const policy = validateClientPolicy(existing, client);
if (!policy.ok) return policy;
const registration = {
...clientRegistrationEntry(dialect, command, environment),
...Object.fromEntries(
Object.entries(existing).filter(
([key]) => !managedConnectionKeys.has(key),
),
),
};
if (client.name === "codex" || client.name === "grok_build")
registration.startup_timeout_sec =
(client.name === "codex"
? positiveTimeout(existing.startup_timeout_sec)
: undefined) ?? MCP_STARTUP_POLICY.codexStartupTimeoutSeconds;
if (client.name === "codex")
registration.tool_timeout_sec =
positiveTimeout(existing.tool_timeout_sec) ??
recommendedMcpToolTimeoutSeconds(timeoutInputs);
if (client.name === "kimi") {
registration.startupTimeoutMs =
positiveTimeout(existing.startupTimeoutMs) ??
MCP_STARTUP_POLICY.kimiStartupTimeoutMs;
registration.toolTimeoutMs =
positiveTimeout(existing.toolTimeoutMs) ??
recommendedMcpToolTimeoutMs(timeoutInputs);
}
return ok(registration);
};
const positiveTimeout = (value: unknown): number | bigint | undefined => {
// TOML keeps integers beyond the safe Number range exact as bigint.
if (typeof value === "bigint") return value > 0n ? value : undefined;
return typeof value === "number" && Number.isFinite(value) && value > 0
? value
: undefined;
};
@@ -1,13 +1,13 @@
import { z } from "zod";
import {
parseEnvironment,
parseEnvironmentValues,
type Environment,
} from "../config/environment.js";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import {
effectiveClientServer,
clientConfigurationRecordSchema,
type ClientConfigurationDocument,
} from "./ClientConfigurationDocument.js";
import { PRODUCT_IDENTITY } from "../identity.js";
@@ -27,9 +27,37 @@ export type ProviderRequestBudgetResolver = (
export interface ClientRegistrationEnvironmentDetails {
readonly values: Readonly<Record<string, string>>;
readonly unresolvedSettings: ReadonlySet<string>;
readonly timeoutInputs: McpToolTimeoutInputs | undefined;
}
interface ResolvedClientRegistrationEnvironmentDetails extends ClientRegistrationEnvironmentDetails {
readonly timeoutInputs: McpToolTimeoutInputs;
}
const TIMEOUT_INPUT_SETTINGS: ReadonlySet<string> = new Set([
"REA_ANALYSIS_PROVIDER",
"REA_GHIDRA_STARTUP_TIMEOUT_MS",
"REA_HOPPER_STARTUP_TIMEOUT_MS",
"REA_JEB_MCP_TIMEOUT_MS",
]);
const environmentSchema = clientConfigurationRecordSchema.transform(
(values, context) => {
const entries: [string, string][] = [];
for (const [key, value] of Object.entries(values)) {
if (typeof value === "string") entries.push([key, value]);
else
context.addIssue({
code: "custom",
path: [key],
message: "Environment values must be strings",
});
}
return Object.fromEntries(entries);
},
);
/** Read explicit REA overrides without dropping peers when one value is invalid. */
export const clientRegistrationEnvironment = (
parsed: ClientConfigurationDocument | undefined,
@@ -37,12 +65,14 @@ export const clientRegistrationEnvironment = (
validation: "fragment" | "effective" = "effective",
platform: NodeJS.Platform = process.platform,
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
if (parsed?.dialect === "dsh")
return dshDocument(parsed.document).environment(
if (parsed?.dialect === "dsh") {
const environment = dshDocument(parsed.document).environment(
overrides,
validation,
platform,
);
return environment.ok ? ok(environment.value.values) : environment;
}
const details = clientRegistrationEnvironmentDetails(
parsed,
overrides,
@@ -66,7 +96,13 @@ export const clientRegistrationEnvironmentDetails = (
platform,
);
if (!environment.ok) return environment;
return validatedDetails(environment.value, validation, platform);
return ok(
environmentDetails(
environment.value.values,
environment.value.configuration,
environment.value.unresolvedSettings,
),
);
}
const key =
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
@@ -80,9 +116,7 @@ export const clientRegistrationEnvironmentDetails = (
typeof registration === "object" && registration !== null
? Reflect.get(registration, key)
: undefined;
const existing = z
.record(z.string(), z.string())
.safeParse(raw === undefined ? {} : raw);
const existing = environmentSchema.safeParse(raw === undefined ? {} : raw);
if (!existing.success)
return err(
new ConfigurationError("Invalid REA server environment", {
@@ -111,7 +145,7 @@ export const clientRegistrationEnvironmentDetailsWithBudget = async (
resolveProviderRequestBudget: ProviderRequestBudgetResolver | undefined,
platform: NodeJS.Platform = process.platform,
): Promise<
Result<ClientRegistrationEnvironmentDetails, ConfigurationError>
Result<ResolvedClientRegistrationEnvironmentDetails, ConfigurationError>
> => {
const details = clientRegistrationEnvironmentDetails(
parsed,
@@ -119,8 +153,31 @@ export const clientRegistrationEnvironmentDetailsWithBudget = async (
validation,
platform,
);
if (!details.ok || resolveProviderRequestBudget === undefined) return details;
return validatedBudget(details.value, resolveProviderRequestBudget, platform);
if (!details.ok) return details;
const unresolved = [...details.value.unresolvedSettings].filter(
(setting) =>
resolveProviderRequestBudget !== undefined ||
TIMEOUT_INPUT_SETTINGS.has(setting),
);
if (details.value.timeoutInputs === undefined || unresolved.length > 0)
return err(
new ConfigurationError(
"Cannot derive a provider request budget from expression-backed settings",
{
settings: unresolved.map((setting) => ({
setting,
constraint:
"Resolve this setting to a literal before deriving a request budget",
})),
},
),
);
const resolved = {
...details.value,
timeoutInputs: details.value.timeoutInputs,
};
if (resolveProviderRequestBudget === undefined) return ok(resolved);
return validatedBudget(resolved, resolveProviderRequestBudget, platform);
};
const validatedDetails = (
@@ -134,29 +191,40 @@ const validatedDetails = (
? parseEnvironmentValues(runtimeEnvironment)
: parseEnvironment(runtimeEnvironment);
if (!validated.ok) return validated;
return ok({
values: Object.fromEntries(
Object.entries(environment).sort(([left], [right]) =>
compareUnicodeCodePoints(left, right),
),
),
timeoutInputs: {
analysisProvider: validated.value.REA_ANALYSIS_PROVIDER,
ghidraStartupTimeoutMs: validated.value.REA_GHIDRA_STARTUP_TIMEOUT_MS,
hopperStartupTimeoutMs: validated.value.REA_HOPPER_STARTUP_TIMEOUT_MS,
...(validated.value.REA_JEB_MCP_TIMEOUT_MS === undefined
? {}
: { jebMcpTimeoutMs: validated.value.REA_JEB_MCP_TIMEOUT_MS }),
},
});
return ok(environmentDetails(environment, validated.value));
};
const environmentDetails = (
environment: Readonly<Record<string, string>>,
configuration: Environment,
unresolvedSettings: ReadonlySet<string> = new Set(),
): ClientRegistrationEnvironmentDetails => ({
values: Object.fromEntries(
Object.entries(environment).sort(([left], [right]) =>
compareUnicodeCodePoints(left, right),
),
),
unresolvedSettings,
timeoutInputs: [...unresolvedSettings].some((setting) =>
TIMEOUT_INPUT_SETTINGS.has(setting),
)
? undefined
: {
analysisProvider: configuration.REA_ANALYSIS_PROVIDER,
ghidraStartupTimeoutMs: configuration.REA_GHIDRA_STARTUP_TIMEOUT_MS,
hopperStartupTimeoutMs: configuration.REA_HOPPER_STARTUP_TIMEOUT_MS,
...(configuration.REA_JEB_MCP_TIMEOUT_MS === undefined
? {}
: { jebMcpTimeoutMs: configuration.REA_JEB_MCP_TIMEOUT_MS }),
},
});
const validatedBudget = async (
details: ClientRegistrationEnvironmentDetails,
details: ResolvedClientRegistrationEnvironmentDetails,
resolveProviderRequestBudget: ProviderRequestBudgetResolver,
platform: NodeJS.Platform,
): Promise<
Result<ClientRegistrationEnvironmentDetails, ConfigurationError>
Result<ResolvedClientRegistrationEnvironmentDetails, ConfigurationError>
> => {
const providerBudget = await resolveProviderRequestBudget(
details.timeoutInputs.analysisProvider,
+20 -96
View File
@@ -11,7 +11,6 @@ import { lstat } from "node:fs/promises";
import { err, ok, type Result } from "../domain/result.js";
import { isAbsolute } from "node:path";
import { z } from "zod";
import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
@@ -41,6 +40,11 @@ import { clientRegistrationEnvironmentDetailsWithBudget } from "./ClientRegistra
import type { ProviderRequestBudgetResolver } from "./ClientRegistrationEnvironment.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import {
parseClientRegistration,
type ClientRegistration,
} from "./ClientRegistration.js";
interface ClientRegistrationStatusBase {
readonly client: string;
readonly config_path: string;
@@ -81,29 +85,6 @@ export type UnhealthyClientRegistrationStatus = Exclude<
{ readonly state: "aligned" }
>;
const tomlTimeoutSchema = z.union([
z.number().positive(),
z.bigint().positive(),
]);
const registrationSchema = z
.object({
command: z.string().min(1),
args: z.array(z.string()).default([]),
// TOML keeps integers beyond the safe Number range exact as bigint.
startup_timeout_sec: tomlTimeoutSchema.optional(),
tool_timeout_sec: tomlTimeoutSchema.optional(),
startupTimeoutMs: z.number().positive().optional(),
toolTimeoutMs: z.number().positive().optional(),
type: z.string().optional(),
transport: z.string().optional(),
serverName: z.string().optional(),
tools: z.array(z.string()).optional(),
disabled: z.boolean().optional(),
enabled: z.boolean().optional(),
})
.passthrough();
interface ClientRegistrationReadOptions {
readonly platform?: NodeJS.Platform;
readonly environment?: NodeJS.ProcessEnv;
@@ -259,7 +240,18 @@ export const readClientRegistrationInspections = async (
format: "zcode_fallback",
}
: client;
const registration = parseRegistration(raw, effectiveClient);
const parsedRegistration = parseClientRegistration(raw, effectiveClient);
if (!parsedRegistration.ok) {
record({
client: client.name,
config_path: configPath,
command: [],
state: "invalid",
remediation: `${parsedRegistration.error.message}: ${parsedRegistration.error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}`,
});
continue;
}
const registration = parsedRegistration.value;
const registrationEnvironment =
client.name === "kimi" || client.name === "codex"
? await clientRegistrationEnvironmentDetailsWithBudget(
@@ -362,7 +354,7 @@ const ignoresDisabledKey = new Set(["codex", "kimi"]);
const ignoresEnabledKey = new Set(["devin"]);
const registrationActivation = (
registration: z.output<typeof registrationSchema>,
registration: ClientRegistration,
client: SetupClient,
parsed: ClientConfigurationDocument,
): "enabled" | "disabled" =>
@@ -385,7 +377,7 @@ const registrationAligned = ({
platform,
kimiInputSchemaProfileSelected,
}: {
readonly registration: z.output<typeof registrationSchema>;
readonly registration: ClientRegistration;
readonly client: SetupClient;
readonly currentCommandPath: string | undefined;
readonly platform: NodeJS.Platform;
@@ -417,16 +409,6 @@ const registrationAligned = ({
)
return "stale";
if (client.name === "kimi" && !kimiInputSchemaProfileSelected) return "stale";
if (
(client.format === "kimi" ||
client.format === "omp" ||
client.format === "pi") &&
registration.type !== undefined &&
registration.type !== "stdio"
)
return "stale";
if (client.format === "vscode" && registration.type !== "stdio")
return "stale";
// A Harness registration only connects as the rea stdio server; another
// server name or transport leaves the session without REA tools.
if (
@@ -434,14 +416,6 @@ const registrationAligned = ({
(registration.serverName !== "rea" || registration.transport !== "stdio")
)
return "stale";
if (
client.format === "copilot_cli" &&
(registration.type !== "stdio" ||
JSON.stringify(registration.tools) !== JSON.stringify(["*"]))
)
return "stale";
if (client.format === "commandcode" && registration.transport !== "stdio")
return "stale";
if (
(["npx", "bunx"] as const).some((runner) => {
const expected = packageRunnerRegistrationCommand(runner, platform);
@@ -459,56 +433,6 @@ const registrationAligned = ({
return isAbsolute(command[0] ?? "") ? "aligned" : "stale";
};
const parseRegistration = (
value: unknown,
client: SetupClient,
): z.output<typeof registrationSchema> => {
if (client.format === "dsh")
return registrationSchema
.pick({ command: true, args: true, serverName: true, transport: true })
.strip()
.parse(value);
if (client.format === "opencode") {
const entry = z
.object({
type: z.literal("local"),
command: z.array(z.string()).min(1),
environment: z.record(z.string(), z.string()).optional(),
})
.passthrough()
.parse(value);
const [command = "", ...args] = entry.command;
return registrationSchema.parse({
...entry,
command,
args,
});
}
const registration = registrationSchema.parse(value);
if (
client.format === "kimi" &&
registration.transport !== undefined &&
registration.transport !== "stdio"
)
throw new TypeError("Expected an stdio registration");
if (
(client.format === "vscode" || client.format === "copilot_cli") &&
registration.type !== "stdio"
)
throw new TypeError("Expected an stdio registration");
if (client.format === "commandcode" && registration.transport !== "stdio")
throw new TypeError("Expected an stdio registration");
if (
(client.format === "kimi" ||
client.format === "omp" ||
client.format === "pi") &&
registration.type !== undefined &&
registration.type !== "stdio"
)
throw new TypeError("Expected an stdio registration");
return registration;
};
/** A client that rewrites its file from memory must be quit before setup. */
const remediation = (client: SetupClient): string => {
if (client.quitBeforeSetup !== true)
@@ -553,7 +477,7 @@ const configuredStatus = (
const timeoutAdvisory = (
client: SetupClient,
registration: z.output<typeof registrationSchema>,
registration: ClientRegistration,
timeoutInputs: McpToolTimeoutInputs,
): string | undefined => {
const toolTimeout =
+15 -13
View File
@@ -28,12 +28,8 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { access, stat } from "node:fs/promises";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { readAnalysisSnapshot } from "./binary/AnalysisSnapshotFiles.js";
import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js";
import {
classifyBinaryTargetFilesystemFailure,
parseBinaryTarget,
} from "./BinaryTargetResolver.js";
import { classifyBinaryTargetFilesystemFailure } from "./BinaryTargetResolver.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
snapshotEvidenceForQuery,
@@ -48,10 +44,8 @@ import type {
ProviderIdentity,
} from "./AnalysisProvider.js";
import { createAnalysisExecutionEvidence } from "../domain/analysisObservation.js";
import {
REA_WORKFLOW_PROVIDER,
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import { REA_WORKFLOW_PROVIDER } from "../domain/investigationProviders.js";
import { workflowAnalysisProfile } from "./WorkflowAnalysisProfile.js";
import {
executeComposedWorkflow,
createWorkflowEvidence,
@@ -343,6 +337,7 @@ const runAnalysis = async (
);
}
const prepared = await prepareSnapshot({
session,
path,
snapshotPath,
signal,
@@ -484,7 +479,7 @@ const runAnalysis = async (
const recovered =
written.error instanceof EvidenceFileError &&
written.error.published !== undefined &&
(await session.closeWriteResources()).ok;
(await session.closeFileResources()).ok;
if (!recovered)
return err(written.error.retainPartialObservation(evidence));
}
@@ -655,6 +650,10 @@ const withProcessCancellation = async <Value>(
};
const prepareSnapshot = async (options: {
readonly session: Pick<
BinarySession,
"readAnalysisSnapshot" | "resolveTarget"
>;
readonly path: string;
readonly formatHint?: ExecutableFormatHint;
readonly architecture?: BinaryArchitecture;
@@ -667,12 +666,15 @@ const prepareSnapshot = async (options: {
AnalysisError
>
> => {
const { path, snapshotPath } = options;
const { path, snapshotPath, session } = options;
if (snapshotPath === undefined || !(await fileExists(snapshotPath)))
return ok({});
const loaded = await readAnalysisSnapshot(snapshotPath, options.signal);
const loaded = await session.readAnalysisSnapshot(
snapshotPath,
options.signal,
);
if (!loaded.ok) return loaded;
const target = await parseBinaryTarget(path, {
const target = await session.resolveTarget(path, {
signal: options.signal,
...(options.formatHint === undefined
? {}
+1 -2
View File
@@ -1,4 +1,3 @@
import type { Evidence } from "../domain/evidence.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
@@ -33,7 +32,7 @@ export const withSessionCleanup = async <Value>(
/** Project typed one-shot outcomes only after execution and cleanup settle. */
export const directAnalysisCleanupFailure = (
result: Result<Evidence, AnalysisError>,
result: Result<JsonValue, AnalysisError>,
error: AnalysisError,
): JsonValue => {
const cleanup = projectAnalysisError(error);
+30 -23
View File
@@ -10,6 +10,11 @@ import { sqliteDatabaseCapability } from "./sqlite/SqliteDatabaseCapability.js";
import { CURRENT_ENTRYPOINT_PATH } from "../process/CurrentEntrypointPath.js";
import { createCatalogIdentity } from "../catalogIdentity.js";
import { compactPresentationForMcpInputSchemaProfile } from "../config/mcpInputSchemaProfile.js";
import {
directAnalysisCleanupFailure,
withSessionCleanup,
} from "./DirectAnalysisCleanup.js";
import { ok } from "../domain/result.js";
/** Read target-free provider status and operation availability. */
export const runSessionStatus = async (
@@ -22,28 +27,30 @@ export const runSessionStatus = async (
const config = dependencies.readConfiguration();
if (!config.ok) return { error: projectAnalysisError(config.error) };
const session = await dependencies.createBinarySession(config.value, logger);
try {
const status = jsonObjectSchema.parse(await session.status());
return {
...status,
capabilities: [
...providerCapabilitySet.array().parse(status.capabilities),
...providerCapabilitySets([
sqliteDatabaseCapability(process.versions.node),
]),
],
server_identity: createServerIdentity({
startedAt: new Date().toISOString(),
commandPath: CURRENT_ENTRYPOINT_PATH,
catalog: createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(
config.value.mcpInputSchemaProfile,
return withSessionCleanup<JsonValue>(
session,
async () => {
const status = jsonObjectSchema.parse(await session.status());
return {
...status,
capabilities: [
...providerCapabilitySet.array().parse(status.capabilities),
...providerCapabilitySets([
sqliteDatabaseCapability(process.versions.node),
]),
],
server_identity: createServerIdentity({
startedAt: new Date().toISOString(),
commandPath: CURRENT_ENTRYPOINT_PATH,
catalog: createCatalogIdentity(
compactPresentationForMcpInputSchemaProfile(
config.value.mcpInputSchemaProfile,
),
config.value.mcpOutputSchemaProfile ?? "full",
),
config.value.mcpOutputSchemaProfile ?? "full",
),
}),
};
} finally {
await session.close();
}
}),
};
},
(status, cleanup) => directAnalysisCleanupFailure(ok(status), cleanup),
);
};
+84 -10
View File
@@ -1,11 +1,16 @@
import { mkdir } from "node:fs/promises";
import { join, posix } from "node:path";
import { describe, expect, it } from "vitest";
import { defaultHopperLauncherPath } from "../config/parseConfig.js";
import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
import type { ClientRegistrationStatus } from "./ClientRegistrationStatus.js";
import { createDoctorHostFixture as host } from "./Doctor.fixture.js";
import { runDoctor, systemDoctorHost, type DoctorHost } from "./Doctor.js";
import { err } from "../domain/result.js";
import { HopperReleaseError } from "./HopperReleaseFeed.js";
import { MAC_HOPPER_SYSTEM_LAUNCHER } from "./MacHopperPaths.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
describe("doctor Hopper startup timeout", () => {
const timeoutHost = (
@@ -1042,18 +1047,87 @@ describe("doctor installed Hopper discovery", () => {
);
expect(result.hopperPath).toBe(path);
});
});
it("detects a manually installed Hopper application", async () => {
const path = "/Applications/Hopper v6.app/Contents/MacOS/hopper";
const result = await runDoctor(
undefined,
host({
executable: (candidate) => Promise.resolve(candidate === path),
manualHopperPaths: () => Promise.resolve([path]),
}),
);
expect(result.hopperPath).toBe(path);
describe("doctor Hopper discovery matches runtime selection", () => {
// Model launchers with runtime's POSIX joins; filesystem work stays native.
const launchers = (home: string) => ({
macSystem: MAC_HOPPER_SYSTEM_LAUNCHER,
renamed: posix.join(
home,
"Applications/Hopper 5.app/Contents/MacOS/hopper",
),
linuxSystem: "/opt/hopper-v4/bin/hopper",
});
type Launcher = keyof ReturnType<typeof launchers>;
it.each<[NodeJS.Platform, readonly Launcher[], Launcher | undefined]>([
["darwin", ["linuxSystem"], undefined],
["darwin", ["linuxSystem", "renamed"], "renamed"],
["darwin", ["linuxSystem", "macSystem"], "macSystem"],
["darwin", ["renamed", "macSystem"], "macSystem"],
["linux", ["macSystem", "renamed"], undefined],
["linux", ["macSystem", "linuxSystem"], "linuxSystem"],
["win32", ["macSystem", "linuxSystem", "renamed"], undefined],
])(
"on %s with %j executable, selects %s",
async (platform, executable, expected) => {
// Runtime enumerates renamed bundles from disk; doctor reads its host scan.
const home = await createTestTempDirectory("rea-doctor-hopper-");
await mkdir(join(home, "Applications/Hopper 5.app"), { recursive: true });
const paths = launchers(home);
const executables = new Set(executable.map((name) => paths[name]));
const probed: string[] = [];
let renamedScans = 0;
let brewProbes = 0;
const result = await runDoctor(
undefined,
host({
platform,
homeDirectory: home,
executable: (path) => {
probed.push(path);
return Promise.resolve(executables.has(path));
},
manualHopperPaths: () => {
renamedScans += 1;
return Promise.resolve([paths.renamed]);
},
brewHopperPath: () => {
brewProbes += 1;
return Promise.resolve(undefined);
},
}),
);
const runtime = defaultHopperLauncherPath(platform, home, (path) =>
executables.has(path),
);
expect(result.hopperPath).toBe(
expected === undefined ? undefined : paths[expected],
);
// Hopper has no automatic launcher on other platforms; runtime's fixed
// fallback there is never probed.
if (platform === "win32") expect(probed).toEqual([]);
else
expect(result.hopperPath).toBe(
executables.has(runtime) ? runtime : undefined,
);
expect(renamedScans).toBe(
platform === "darwin" && expected !== "macSystem" ? 1 : 0,
);
expect(brewProbes).toBe(
platform === "darwin" && expected === undefined ? 1 : 0,
);
if (expected === undefined)
expect(
result.checks.find(({ name }) => name === "hopper"),
).toMatchObject({
ok: false,
classification: "missing_analysis_engine",
});
},
);
});
describe("doctor target admission on Linux", () => {
+6 -25
View File
@@ -8,7 +8,7 @@ import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js";
import { STARTUP_OWNER_CLOSED } from "../process/SharedStartupAttempt.js";
import { constants } from "node:fs";
import { access, readdir, stat } from "node:fs/promises";
import { access, stat } from "node:fs/promises";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import type { RetainedResourceScope } from "./AnalysisProvider.js";
import { nativeAnalysisConfigurationError } from "../config/parseConfig.js";
@@ -27,10 +27,7 @@ import {
type JavaScriptRuntime,
} from "../domain/runtimeVersion.js";
import { installedCaskAppdir, probeHomebrew } from "./homebrew.js";
import {
macHopperApplicationRoots,
macHopperRenamedLauncherCandidates,
} from "./MacHopperPaths.js";
import { readMacHopperRenamedLauncherCandidates } from "./MacHopperPaths.js";
import {
inspectLinuxHopperLauncher,
linuxSharedLibrariesAvailable,
@@ -128,7 +125,9 @@ export interface DoctorHost {
path: string,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>>;
linuxDemoRuntimeCheck(): Promise<DoctorCheck>;
/** Homebrew cask launcher; consulted only where macOS bundles are probed. */
brewHopperPath(): Promise<string | undefined>;
/** Renamed `Hopper*.app` launchers; consulted only where macOS bundles are probed. */
manualHopperPaths(): Promise<readonly string[]>;
providerInspections?(): Promise<readonly DoctorProviderInspection[]>;
/** Resolve a parsed upstream request budget from the effective client environment. */
@@ -645,7 +644,8 @@ export const systemDoctorHost = (
: join(appdir, "Hopper Disassembler.app/Contents/MacOS/hopper");
});
},
manualHopperPaths: () => manualHopperPaths(homeDirectory),
manualHopperPaths: () =>
readMacHopperRenamedLauncherCandidates(homeDirectory),
...(options.providerInspections === undefined
? {}
: {
@@ -777,25 +777,6 @@ const uncomposedLinuxDemoRuntimeCheck = (): Promise<DoctorCheck> =>
remediation: "Run rea doctor through the production CLI adapter.",
});
const manualHopperPaths = async (home: string): Promise<readonly string[]> => {
const roots = macHopperApplicationRoots(home);
const bundleNamesByRoot = new Map<string, readonly string[]>();
for (const root of roots) {
try {
bundleNamesByRoot.set(
root,
(await readdir(root, { withFileTypes: true }))
.filter((entry) => entry.isDirectory())
.map(({ name }) => name),
);
} catch (cause: unknown) {
// A missing or unreadable optional application directory is not fatal.
void cause;
}
}
return macHopperRenamedLauncherCandidates(home, bundleNamesByRoot);
};
const uniqueLines = (value: string): string[] => [
...new Set(
value
+14 -34
View File
@@ -1,18 +1,10 @@
import {
SUPPORTED_NODE_VERSION_PROSE,
supportsNodeVersion,
supportsJavaScriptRuntime,
} from "../domain/runtimeVersion.js";
import { providerStartupTimeoutSchema } from "../config/providerStartupTimeout.js";
import {
LINUX_HOPPER_SETUP_HOSTS,
linuxHopperLauncherCandidates,
systemLinuxHopperLauncherCandidates,
supportsReaHost,
} from "./LinuxHopper.js";
import {
MAC_HOPPER_SYSTEM_LAUNCHER,
macHopperCanonicalLauncherCandidates,
} from "./MacHopperPaths.js";
import { hopperLauncherDiscovery } from "./HopperLauncherDiscovery.js";
import { LINUX_HOPPER_SETUP_HOSTS, supportsReaHost } from "./LinuxHopper.js";
import type {
DoctorCheck,
DoctorHost,
@@ -102,7 +94,7 @@ export const doctorHealthy = (
);
};
/** Bun passes when its reported Node.js compatibility version is supported. */
/** Runtime support follows the shared rule, which judges Bun by its Node.js compatibility. */
const nodeCheck = ({ runtime }: DoctorHost): DoctorCheck => {
const failure = {
remediation: `Install ${SUPPORTED_NODE_VERSION_PROSE}.`,
@@ -112,7 +104,7 @@ const nodeCheck = ({ runtime }: DoctorHost): DoctorCheck => {
return {
...check(
"node",
supportsNodeVersion(runtime.version),
supportsJavaScriptRuntime(runtime),
runtime.version,
failure,
),
@@ -121,7 +113,7 @@ const nodeCheck = ({ runtime }: DoctorHost): DoctorCheck => {
return {
...check(
"node",
supportsNodeVersion(runtime.nodeCompatibilityVersion),
supportsJavaScriptRuntime(runtime),
`Bun ${runtime.version} (Node.js ${runtime.nodeCompatibilityVersion} compatibility)`,
failure,
),
@@ -154,26 +146,14 @@ const hostCheck = async (host: DoctorHost): Promise<DoctorCheck> => {
};
const findHopper = async (host: DoctorHost): Promise<string | undefined> => {
const candidates =
host.configuredHopperPath === undefined
? [
...(host.platform === "darwin"
? macHopperCanonicalLauncherCandidates(host.homeDirectory)
: [MAC_HOPPER_SYSTEM_LAUNCHER]),
...(host.platform === "linux"
? linuxHopperLauncherCandidates(host.homeDirectory)
: [...systemLinuxHopperLauncherCandidates]),
...(host.platform === "darwin"
? []
: macHopperCanonicalLauncherCandidates(host.homeDirectory).slice(
1,
)),
...(await host.manualHopperPaths()),
]
: [host.configuredHopperPath];
const hopperPath = await firstExecutable(host, candidates);
if (hopperPath !== undefined || host.configuredHopperPath !== undefined)
return hopperPath;
if (host.configuredHopperPath !== undefined)
return firstExecutable(host, [host.configuredHopperPath]);
const discovery = hopperLauncherDiscovery(host.platform, host.homeDirectory);
const fixed = await firstExecutable(host, discovery.candidates);
if (fixed !== undefined || !discovery.probesMacApplicationBundles)
return fixed;
const renamed = await firstExecutable(host, await host.manualHopperPaths());
if (renamed !== undefined) return renamed;
const brewCandidate = await host.brewHopperPath();
return firstExecutable(
host,
+18 -5
View File
@@ -2,6 +2,7 @@ import { isMap, isSeq, type Node, type YAMLMap } from "yaml";
import {
parseEnvironment,
parseEnvironmentValues,
type Environment,
} from "../config/environment.js";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -190,7 +191,14 @@ export class DshCordisDocument {
overrides: Readonly<Record<string, string>> = {},
validation: "fragment" | "effective" = "effective",
platform: NodeJS.Platform = process.platform,
): Result<Readonly<Record<string, string>>, ConfigurationError> {
): Result<
{
readonly values: Readonly<Record<string, string>>;
readonly configuration: Environment;
readonly unresolvedSettings: ReadonlySet<string>;
},
ConfigurationError
> {
const config = this.#field("config");
if (config instanceof CordisExpression)
return err(
@@ -223,7 +231,7 @@ export class DshCordisDocument {
return err(
new ConfigurationError("Cordis server environment must be a mapping"),
);
const literals: Record<string, string> = {};
const literals: Record<string, string> = Object.create(null);
const unresolvedSettings = new Set<string>();
for (const [key, value] of Object.entries(
mergeEnvironmentOverrides<unknown>(raw ?? {}, overrides, platform),
@@ -246,8 +254,7 @@ export class DshCordisDocument {
);
literals[key] = value;
}
const environment = literals;
const runtimeEnvironment = snapshotEnvironment(environment, platform);
const runtimeEnvironment = snapshotEnvironment(literals, platform);
const unresolved = new Set(
Object.keys(
snapshotEnvironment(
@@ -260,7 +267,13 @@ export class DshCordisDocument {
validation === "fragment"
? parseEnvironmentValues(runtimeEnvironment, unresolved)
: parseEnvironment(runtimeEnvironment, unresolved);
return validated.ok ? ok(environment) : validated;
return validated.ok
? ok({
values: literals,
configuration: validated.value,
unresolvedSettings: unresolved,
})
: validated;
}
desired(
@@ -0,0 +1,49 @@
import {
linuxHopperLauncherCandidates,
REA_MANAGED_LINUX_HOPPER_LAUNCHER,
} from "./LinuxHopper.js";
import {
MAC_HOPPER_SYSTEM_LAUNCHER,
macHopperCanonicalLauncherCandidates,
} from "./MacHopperPaths.js";
/**
* Where automatic Hopper discovery looks on one platform. Runtime configuration
* and doctor both read it, so doctor checks the same candidates, in the same
* order, as CLI/MCP startup.
*/
export interface HopperLauncherDiscovery {
/** Fixed launcher paths, probed in order. */
readonly candidates: readonly string[];
/**
* Whether renamed `Hopper*.app` bundles are probed after the fixed paths.
* Doctor also reports the Homebrew cask application only when this is set.
*/
readonly probesMacApplicationBundles: boolean;
/** Launcher runtime configures when no candidate is executable. */
readonly fallback: string;
}
/** Select the automatic Hopper launcher candidates for one platform. */
export const hopperLauncherDiscovery = (
platform: NodeJS.Platform,
homeDirectory: string,
): HopperLauncherDiscovery => {
if (platform === "darwin")
return {
candidates: macHopperCanonicalLauncherCandidates(homeDirectory),
probesMacApplicationBundles: true,
fallback: MAC_HOPPER_SYSTEM_LAUNCHER,
};
if (platform === "linux")
return {
candidates: linuxHopperLauncherCandidates(homeDirectory),
probesMacApplicationBundles: false,
fallback: REA_MANAGED_LINUX_HOPPER_LAUNCHER,
};
return {
candidates: [],
probesMacApplicationBundles: false,
fallback: MAC_HOPPER_SYSTEM_LAUNCHER,
};
};
@@ -46,6 +46,10 @@ describe("Linux Hopper release feed boundary", () => {
"https://other.example/downloader/public/Hopper-99.2.1-Linux-demo.deb",
},
{ filename: release.filename + "?token=caller" },
{ filename: release.filename + "?" },
{ filename: release.filename + "#" },
{ filename: release.filename + "?#" },
{ filename: release.filename + "#selection" },
{
filename: release.filename.replace("https://", "https://user:password@"),
},
+2 -2
View File
@@ -2,6 +2,7 @@ import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
import { HOPPER_DOWNLOAD_BYTES } from "./HopperDownload.js";
import { urlDeclaresQueryOrFragment } from "../domain/urlComponents.js";
/** Vendor endpoints, shared by installation and existing-build inspection. */
export const HOPPER_RELEASES_URL =
@@ -92,8 +93,7 @@ export const parseLinuxHopperRelease = (
url.origin !== "https://www.hopperapp.com" ||
url.username !== "" ||
url.password !== "" ||
url.search !== "" ||
url.hash !== "" ||
urlDeclaresQueryOrFragment(url) ||
!url.pathname.startsWith("/downloader/public/Hopper-") ||
!url.pathname.endsWith(`-Linux-demo${suffixes[family]}`)
)
+146 -63
View File
@@ -15,8 +15,11 @@ import {
} from "../domain/evidenceErrors.js";
import {
AnalysisCancelledError,
type AnalysisInputError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { err, ok, type Result } from "../domain/result.js";
import {
JsonInputPhysicalReadFailure,
@@ -26,7 +29,11 @@ import {
type JsonObjectMemberReader,
} from "./JsonInputFile.js";
import { NonRegularFileReadError } from "../filesystem/RegularFile.js";
import { RegularFileCleanupFailure } from "../filesystem/OwnedRegularFile.js";
import {
RegularFileCleanupFailure,
regularFileCancellation,
} from "../filesystem/OwnedRegularFile.js";
import { bindRegularFileCleanup } from "./RegularFileRead.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { bindArtifactFileCleanup } from "../artifacts/ArtifactFileOperation.js";
@@ -115,80 +122,156 @@ export const readJsonObjectMembersFile = (
signal,
);
type JsonFileReadFailure =
| EvidenceFileError
| AnalysisResourceConstraintError
| AnalysisCancelledError;
type DecodedJsonInput<Value> = Result<
Value,
AnalysisInputError | AnalysisResourceConstraintError
>;
const readJsonFileInput = async <Value>(
path: string,
read: (
selected: string,
) => Promise<
Result<
Value,
| import("../domain/analysisErrorCore.js").AnalysisInputError
| AnalysisResourceConstraintError
>
>,
read: (selected: string) => Promise<DecodedJsonInput<Value>>,
signal?: AbortSignal,
operation = "read_evidence_file",
): Promise<
Result<
Value,
EvidenceFileError | AnalysisResourceConstraintError | AnalysisCancelledError
>
> => {
): Promise<Result<Value, JsonFileReadFailure>> => {
const requestedPath = absoluteCallerPath(path);
try {
const decoded = await read(requestedPath);
if (!decoded.ok) {
if (decoded.error instanceof AnalysisResourceConstraintError)
return err(decoded.error);
return err(
new EvidenceFileError("read", "invalid-json", {
cause:
decoded.error.cause ??
new SyntaxError(
decoded.error.issues[0]?.message ?? decoded.error.message,
),
path: requestedPath,
}),
);
}
return ok(decoded.value);
return decodedJsonFileResult(await read(requestedPath), requestedPath);
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (cause instanceof AnalysisResourceConstraintError) return err(cause);
if (cause instanceof JsonInputCancellationFailure)
// Caller reasons are opaque, including instances of lifecycle failure types.
if (signal?.aborted === true && cause === signal.reason)
return err(new AnalysisCancelledError(operation, { cause }));
if (cause instanceof RegularFileCleanupFailure)
return err(
new AnalysisCancelledError(operation, {
cause: cause.reason,
}),
jsonFileCleanupError(
cause as RegularFileCleanupFailure<DecodedJsonInput<Value>>,
requestedPath,
operation,
signal,
),
);
if (cause instanceof JsonInputPhysicalReadFailure)
return err(
new EvidenceFileError("read", "io", {
cause: cause.physicalCause,
path: requestedPath,
}),
);
if (isJsonReadCancellation(cause, signal))
return err(
new AnalysisCancelledError(operation, {
cause: signal?.reason,
}),
);
return err(
new EvidenceFileError(
"read",
cause instanceof NonRegularFileReadError
? "not-file"
: missingOrIo(cause),
{
cause,
path: requestedPath,
},
),
);
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) {
const primary =
cause.reason === "cancelled"
? new AnalysisCancelledError(operation, {
cause:
cause.cancellation === undefined
? cause.cause
: cause.cancellation.reason,
partialObservation: cause.partialObservation,
})
: undefined;
const error = new EvidenceFileError("read", "io", {
cause,
path: requestedPath,
cleanup: cause.cleanup,
partialObservation: cause.partialObservation,
cleanupPrimary: primary,
...(primary === undefined
? {}
: { diagnostics: { primary_error: projectAnalysisError(primary) } }),
});
bindRegularFileCleanup(cause, error);
return err(error);
}
return err(jsonFileReadFailure(cause, requestedPath, operation, signal));
}
};
/** Parsing owns its typed refusal even when final descriptor cleanup fails. */
const decodedJsonFileResult = <Value>(
decoded: DecodedJsonInput<Value>,
path: string,
): Result<Value, JsonFileReadFailure> => {
if (decoded.ok) return decoded;
if (decoded.error instanceof AnalysisResourceConstraintError)
return err(decoded.error);
return err(
new EvidenceFileError("read", "invalid-json", {
cause:
decoded.error.cause ??
new SyntaxError(
decoded.error.issues[0]?.message ?? decoded.error.message,
),
path,
}),
);
};
const jsonFileReadFailure = (
cause: unknown,
path: string,
operation: string,
signal?: AbortSignal,
): JsonFileReadFailure => {
if (cause instanceof AnalysisResourceConstraintError) return cause;
if (cause instanceof JsonInputCancellationFailure)
return new AnalysisCancelledError(operation, { cause: cause.reason });
if (cause instanceof JsonInputPhysicalReadFailure)
return new EvidenceFileError("read", "io", {
cause: cause.physicalCause,
path,
});
if (isJsonReadCancellation(cause, signal))
return new AnalysisCancelledError(operation, { cause: signal?.reason });
return new EvidenceFileError(
"read",
cause instanceof NonRegularFileReadError ? "not-file" : missingOrIo(cause),
{ cause, path },
);
};
/** Translate the physical outcome once, binding the resulting Result to its owner. */
const jsonFileCleanupError = <Value>(
failure: RegularFileCleanupFailure<DecodedJsonInput<Value>>,
path: string,
operation: string,
signal?: AbortSignal,
): EvidenceFileError => {
const cleanup = new EvidenceFileError("read", "io", {
cause: failure,
path,
cleanup: {
reason:
failure.cleanupCause instanceof Error
? failure.cleanupCause.message
: String(failure.cleanupCause),
resources: [failure.path],
},
});
bindRegularFileCleanup(failure, cleanup);
const cancellation = regularFileCancellation(failure);
let primary: AnalysisError | undefined;
if (failure.outcome.kind === "failed") {
primary =
cancellation === undefined
? jsonFileReadFailure(failure.outcome.cause, path, operation, signal)
: new AnalysisCancelledError(operation, { cause: cancellation.reason });
} else {
const decoded = decodedJsonFileResult(failure.outcome.value, path);
if (!decoded.ok) primary = decoded.error;
else if (cancellation !== undefined)
primary = new AnalysisCancelledError(operation, {
cause: cancellation.reason,
});
}
if (primary === undefined) return cleanup;
return new EvidenceFileError("read", "io", {
path,
cause: new AggregateError(
[primary, cleanup],
"JSON read and cleanup failures",
{ cause: primary },
),
cleanupPrimary: primary,
cleanup: cleanup.cleanup,
diagnostics: { primary_error: projectAnalysisError(primary) },
}).bindCleanupDependencies([primary, cleanup]);
};
const isJsonReadCancellation = (
cause: unknown,
signal?: AbortSignal,
+1 -1
View File
@@ -463,7 +463,7 @@ export const linuxHopperLauncherPath = (home: string): string =>
export const REA_MANAGED_LINUX_HOPPER_LAUNCHER = "/opt/hopper/bin/Hopper";
/** Other system-wide Hopper paths used by package installations. */
export const systemLinuxHopperLauncherCandidates: readonly string[] = [
const systemLinuxHopperLauncherCandidates: readonly string[] = [
"/opt/hopper-v4/bin/hopper",
"/usr/bin/hopper",
];
+15 -12
View File
@@ -1,12 +1,13 @@
import { mkdtemp, mkdir, rm } from "node:fs/promises";
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, posix, relative } from "node:path";
import { describe, expect, it } from "vitest";
import {
localMacHopperRenamedLauncherCandidates,
macHopperCanonicalLauncherCandidates,
macHopperLauncherCandidates,
macHopperRenamedLauncherCandidates,
readMacHopperRenamedLauncherCandidates,
} from "./MacHopperPaths.js";
describe("macOS Hopper launcher paths", () => {
@@ -35,13 +36,7 @@ describe("macOS Hopper launcher paths", () => {
]);
});
it("always includes the installer's canonical user destination", () => {
expect(macHopperLauncherCandidates(home)).toContain(
"/fixture/home/Applications/Hopper Disassembler.app/Contents/MacOS/hopper",
);
});
it("finds renamed user bundles through the runtime filesystem scan", async () => {
it("finds the same renamed user bundles through runtime and doctor scans", async () => {
// Keep native ownership for all I/O, including UNC temporary directories.
// On Windows, a checkout-relative home models POSIX separators without
// discarding the drive/share or normalizing a UNC prefix through posix.join.
@@ -53,14 +48,22 @@ describe("macOS Hopper launcher paths", () => {
process.platform === "win32"
? relative(process.cwd(), temporary).replaceAll("\\", "/")
: temporary;
const appDirectory = join(temporary, "Applications", "Hopper Preview.app");
const applications = join(temporary, "Applications");
try {
await mkdir(appDirectory, { recursive: true });
expect([...macHopperLauncherCandidates(home)]).toContain(
await mkdir(join(applications, "Hopper Preview.app"), {
recursive: true,
});
await mkdir(join(applications, "Hopper Disassembler.app"));
await writeFile(join(applications, "Hopper File.app"), "");
const runtime = [...localMacHopperRenamedLauncherCandidates(home)];
expect(runtime.filter((path) => path.startsWith(home))).toEqual([
posix.join(
home,
"Applications/Hopper Preview.app/Contents/MacOS/hopper",
),
]);
expect(await readMacHopperRenamedLauncherCandidates(home)).toEqual(
runtime,
);
} finally {
await rm(temporary, { recursive: true, force: true });
+27 -9
View File
@@ -1,4 +1,5 @@
import { readdirSync } from "node:fs";
import { readdirSync, type Dirent } from "node:fs";
import { readdir } from "node:fs/promises";
import { posix } from "node:path";
export const MAC_HOPPER_APPLICATION_DIRECTORY = "/Applications";
@@ -52,13 +53,12 @@ export const macHopperRenamedLauncherCandidates = (
.map((name) => posix.join(root, name, "Contents/MacOS/hopper")),
);
const directoryNames = (entries: readonly Dirent[]): readonly string[] =>
entries.filter((entry) => entry.isDirectory()).map(({ name }) => name);
const localBundleNames = (root: string): readonly string[] => {
try {
return readdirSync(root, { withFileTypes: true })
.filter(
(entry) => entry.isDirectory() && /^Hopper.*\.app$/iu.test(entry.name),
)
.map(({ name }) => name);
return directoryNames(readdirSync(root, { withFileTypes: true }));
} catch (cause: unknown) {
// Missing or unreadable optional app directories contain no candidates.
void cause;
@@ -66,14 +66,32 @@ const localBundleNames = (root: string): readonly string[] => {
}
};
/** Yield canonical paths first; enumerate renamed bundles only if needed. */
export function* macHopperLauncherCandidates(
/** Yield renamed bundles from the local filesystem, reading each root only if needed. */
export function* localMacHopperRenamedLauncherCandidates(
homeDirectory: string,
): Generator<string> {
yield* macHopperCanonicalLauncherCandidates(homeDirectory);
for (const root of macHopperApplicationRoots(homeDirectory))
yield* macHopperRenamedLauncherCandidates(
homeDirectory,
new Map([[root, localBundleNames(root)]]),
);
}
/** Read the same renamed bundles without blocking the event loop, for doctor. */
export const readMacHopperRenamedLauncherCandidates = async (
homeDirectory: string,
): Promise<readonly string[]> => {
const bundleNamesByRoot = new Map<string, readonly string[]>();
for (const root of macHopperApplicationRoots(homeDirectory)) {
try {
bundleNamesByRoot.set(
root,
directoryNames(await readdir(root, { withFileTypes: true })),
);
} catch (cause: unknown) {
// Missing or unreadable optional app directories contain no candidates.
void cause;
}
}
return macHopperRenamedLauncherCandidates(homeDirectory, bundleNamesByRoot);
};
@@ -8,7 +8,7 @@ import {
type ReconstructionClosureResult,
} from "../domain/reconstructionCoverage.js";
import { err, type Result } from "../domain/result.js";
import { requestInputError } from "./workflowInputError.js";
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
const OPERATION = "evaluate_reconstruction_coverage";
const requestEnvelopeSchema =
@@ -23,7 +23,11 @@ export const evaluateReconstructionCoverage = (
): Result<ReconstructionClosureResult, AnalysisError> => {
const parsed = requestEnvelopeSchema.safeParse(rawInput);
if (!parsed.success)
return err(requestInputError(OPERATION, parsed.error, rawInput));
return err(
analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput, {
cause: parsed.error,
}),
);
const coverage = readReconstructionCoverageData(parsed.data.coverage, [
"coverage",
]);
@@ -1,16 +1,49 @@
import { err, ok, type Result } from "../domain/result.js";
import {
reconstructionObligationProofSchema,
type ReconstructionObligationProof,
} from "../domain/reconstructionObligationProof.js";
import type { Evidence } from "../domain/evidence.js";
import type { ReconstructionObligation } from "../domain/reconstructionObligationLedgerSchemas.js";
type ProofAuthority = ReconstructionObligation["required_fixture_authority"];
type OriginalAuthority =
ReconstructionObligation["required_original_authority"];
interface ExpectedProof {
type ExpectedProof = {
readonly obligationId: string;
readonly fixtureId?: string;
readonly caseKind?: string;
readonly verifierId?: string;
readonly claimId?: string;
}
readonly obligationVersion: number;
readonly ownerSha256: string;
} & (
| {
readonly kind: "fixture";
readonly fixtureId: string;
readonly caseKind: ReconstructionObligation["required_case_kinds"][number];
}
| {
readonly kind: "verifier";
readonly verifierId: string;
readonly claimId: string;
}
);
type ProofRejection =
| "unobserved-proof"
| "unrecognized-proof"
| "invalid-proof"
| "insufficient-authority"
| "obligation-mismatch"
| "owner-mismatch"
| "revision-mismatch"
| "claim-mismatch"
| "failed-proof";
type ParsedProof = Result<
{
readonly authority: Evidence["authority"];
readonly result: ReconstructionObligationProof;
},
ProofRejection
>;
const proofEvidenceAuthorities: Readonly<
Record<ProofAuthority, readonly Evidence["authority"][]>
@@ -60,33 +93,64 @@ export const proofAuthoritySatisfies = (
return accepted[required].includes(actual);
};
/** Check that Evidence can authenticate the declared reconstruction authority. */
export const evidenceAuthoritySupportsProof = (
/** Parse retained proof once; authenticity is checked by bundle admission. */
export const parseReconstructionProofEvidence = (
evidence: Evidence,
authority: ProofAuthority,
expected: ExpectedProof,
): boolean => {
): ParsedProof => {
if (evidence.confidence !== "observed") return err("unobserved-proof");
if (
evidence.confidence !== "observed" ||
!proofEvidenceAuthorities[authority].includes(evidence.authority) ||
evidence.predicate_type !== "rea.reconstruction-proof" ||
evidence.operation !== "verify_reconstruction_obligations"
)
return false;
const result = evidence.normalized_result;
if (typeof result !== "object" || result === null || Array.isArray(result))
return false;
const includes = (key: string, value: string | undefined): boolean =>
value === undefined ||
(Array.isArray(result[key]) && result[key].includes(value));
return (
result.passed === true &&
includes("obligation_ids", expected.obligationId) &&
includes("fixture_ids", expected.fixtureId) &&
includes("case_kinds", expected.caseKind) &&
includes("verifier_ids", expected.verifierId) &&
includes("claim_ids", expected.claimId)
return err("unrecognized-proof");
const parsed = reconstructionObligationProofSchema.safeParse(
evidence.normalized_result,
);
return parsed.success
? ok({ authority: evidence.authority, result: parsed.data })
: err("invalid-proof");
};
/** Qualify one exact claim, without joining unrelated identities or results. */
export const qualifyReconstructionProof = (
proof: ParsedProof,
authority: ProofAuthority,
expected: ExpectedProof,
): Result<null, ProofRejection> => {
if (!proof.ok) return proof;
if (!proofEvidenceAuthorities[authority].includes(proof.value.authority))
return err("insufficient-authority");
const obligations = proof.value.result.bindings.filter(
(binding) => binding.obligation_id === expected.obligationId,
);
if (obligations.length === 0) return err("obligation-mismatch");
const owners = obligations.filter(
(binding) => binding.owner_sha256 === expected.ownerSha256,
);
if (owners.length === 0) return err("owner-mismatch");
const revisions = owners.filter(
(binding) => binding.obligation_version === expected.obligationVersion,
);
if (revisions.length === 0) return err("revision-mismatch");
const outcomes = revisions.flatMap((binding) =>
expected.kind === "fixture"
? binding.fixtures
.filter(
(fixture) =>
fixture.fixture_id === expected.fixtureId &&
fixture.case_kind === expected.caseKind,
)
.map(({ passed }) => passed)
: binding.verifiers
.filter(
(verifier) =>
verifier.verifier_id === expected.verifierId &&
verifier.claim_id === expected.claimId,
)
.map(({ passed }) => passed),
);
if (outcomes.length === 0) return err("claim-mismatch");
return outcomes.every((passed) => passed) ? ok(null) : err("failed-proof");
};
/** Check that Evidence observes the original at the required boundary. */
@@ -34,7 +34,7 @@ import {
createJavaScriptApplicationNode,
} from "../domain/javascript/javascriptApplicationGraph.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { MANAGED_WORKFLOW_PROVIDER } from "./InvestigationProviders.js";
import { MANAGED_WORKFLOW_PROVIDER } from "../domain/investigationProviders.js";
import { createProcessCaptureEvidence } from "./process/ProcessEvidence.js";
const applicationWithUnknowns = (
@@ -61,8 +61,8 @@ describe("reconstruction obligation ledger", () => {
expect(result.obligations[0]?.diagnostics.map(({ code }) => code)).toEqual(
expect.arrayContaining([
"weak-fixture-authority",
"weak-verifier-authority",
"inapplicable-fixture-proof",
"inapplicable-verifier-proof",
]),
);
});
@@ -72,11 +72,25 @@ describe("reconstruction obligation ledger", () => {
const evidence = boundProofEvidence(
"fixture-chain",
{
obligation_ids: ids,
fixture_ids: ids.map((id) => `fixture.${id}`),
case_kinds: ["positive"],
verifier_ids: ids.map((id) => `verifier.${id}`),
claim_ids: ids.map((id) => `claim.${id}`),
bindings: ids.map((id) => ({
obligation_id: id,
obligation_version: 1,
owner_sha256: "6".repeat(64),
fixtures: [
{
fixture_id: `fixture.${id}`,
case_kind: "positive" as const,
passed: true,
},
],
verifiers: [
{
verifier_id: `verifier.${id}`,
claim_id: `claim.${id}`,
passed: true,
},
],
})),
},
"shipped-artifact",
);
@@ -264,10 +264,12 @@ describe("reconstruction obligation manifest references", () => {
if (binding === undefined) throw new Error("Expected binding");
const variants: {
readonly code: string;
readonly kind: "positive" | "malformed";
readonly binding: typeof binding;
}[] = [
{
code: "missing-original-case",
kind: "positive",
binding: {
...binding,
original_cases: [
@@ -278,6 +280,7 @@ describe("reconstruction obligation manifest references", () => {
},
{
code: "missing-case",
kind: "positive",
binding: {
...binding,
fixtures: [
@@ -291,17 +294,88 @@ describe("reconstruction obligation manifest references", () => {
],
},
},
{
code: "missing-original-case",
kind: "malformed",
binding: {
...binding,
original_cases: [
...binding.original_cases,
{ kind: "malformed", evidence_id: absent, location: "/old" },
],
},
},
{
code: "missing-case",
kind: "malformed",
binding: {
...binding,
fixtures: [
...binding.fixtures,
{
fixture_id: "packaged.old",
case_kind: "malformed",
authority: "packaged-process",
evidence_ids: [absent],
},
],
},
},
];
for (const { code, binding: dangling } of variants) {
for (const { code, kind, binding: dangling } of variants) {
const open = build({
...ledgerRequest,
manifest: { ...ledgerRequest.manifest, bindings: [dangling] },
});
expect(open.status).not.toBe("ready");
expect(open.obligations[0]?.status).toBe("implemented");
expect(open.obligations[0]?.diagnostics).toEqual([
{ code, detail: expect.stringContaining(absent) },
]);
expect(open.obligations[0]?.diagnostics).toContainEqual({
code,
detail: expect.stringMatching(new RegExp(`${kind}.*${absent}`)),
});
expect(open.evidence_links).toContain(absent);
}
const surplusEvidence = proofEvidence("surplus-case");
const validSurplus = build(
request([...ledgerRequest.evidence_bundle.records, surplusEvidence], {
manifest: {
...ledgerRequest.manifest,
bindings: [
{
...binding,
original_cases: [
...binding.original_cases,
{
kind: "malformed",
evidence_id: surplusEvidence.evidence_id,
location: "/retained",
},
],
fixtures: [
...binding.fixtures,
{
fixture_id: "packaged.malformed",
case_kind: "malformed",
authority: "packaged-process",
evidence_ids: [surplusEvidence.evidence_id],
},
],
},
],
},
}),
);
expect(validSurplus.status).toBe("ready");
expect(validSurplus.obligations[0]?.observed_cases).toContainEqual(
expect.objectContaining({ kind: "malformed", location: "/retained" }),
);
expect(validSurplus.obligations[0]?.binding?.fixtures).toContainEqual(
expect.objectContaining({
case_kind: "malformed",
fixture_id: "packaged.malformed",
}),
);
expect(validSurplus.evidence_links).toContain(surplusEvidence.evidence_id);
});
});
@@ -1,3 +1,7 @@
import {
reconstructionObligationProofSchema,
type ReconstructionObligationProof,
} from "../domain/reconstructionObligationProof.js";
import { PROCESS_PROVIDER } from "../domain/process/processEvidenceProvider.js";
import { buildReconstructionObligationLedgerEvidence } from "./ReconstructionObligationLedgerService.js";
import { resolveReconstructionObligationLedgerRequest } from "./ReconstructionObligationLedgerRequest.js";
@@ -56,13 +60,7 @@ export const processEvidence = (
/** Build verifier Evidence bound to enumerated fixture and obligation identities. */
export const boundProofEvidence = (
id: string,
result: {
readonly obligation_ids: readonly string[];
readonly fixture_ids: readonly string[];
readonly case_kinds: readonly string[];
readonly verifier_ids: readonly string[];
readonly claim_ids: readonly string[];
},
result: ReconstructionObligationProof,
authority: "shipped-artifact" | "controlled-replay" = "controlled-replay",
) =>
createEvidence(
@@ -72,14 +70,9 @@ export const boundProofEvidence = (
predicateType: "rea.reconstruction-proof",
operation: "verify_reconstruction_obligations",
parameters: {},
result: {
passed: true,
obligation_ids: [...result.obligation_ids],
fixture_ids: [...result.fixture_ids],
case_kinds: [...result.case_kinds],
verifier_ids: [...result.verifier_ids],
claim_ids: [...result.claim_ids],
},
result: jsonValueSchema.parse(
reconstructionObligationProofSchema.parse(result),
),
confidence: "observed",
authority,
},
@@ -130,11 +123,25 @@ export const readyProcessLedgerRequest =
(caseKind) => `packaged.${caseKind}`,
);
const proof = boundProofEvidence("packaged", {
obligation_ids: [obligation.obligation_id],
fixture_ids: fixtureIds,
case_kinds: obligation.required_case_kinds,
verifier_ids: ["verifier.packaged"],
claim_ids: ["claim.process"],
bindings: [
{
obligation_id: obligation.obligation_id,
obligation_version: obligation.obligation_version,
owner_sha256: "2".repeat(64),
fixtures: obligation.required_case_kinds.map((caseKind, index) => ({
fixture_id: fixtureIds[index]!,
case_kind: caseKind,
passed: true,
})),
verifiers: [
{
verifier_id: "verifier.packaged",
claim_id: "claim.process",
passed: true,
},
],
},
],
});
return request([capture, proof], {
manifest: {
@@ -284,3 +291,81 @@ export const deriveCandidates = (
manifest: { bindings: [], contradictions: [] },
}),
);
/** Exact reviewed proof control shared by public CLI/MCP applicability tests. */
export const readyReviewedLedgerRequest =
(): ReconstructionObligationLedgerInput => {
const original = authorityEvidence("reviewed-original", "shipped-artifact");
const obligation = reviewedObligation(
"review.identity",
original.evidence_id,
);
const proof = boundProofEvidence("reviewed", {
bindings: [
{
obligation_id: obligation.obligation_id,
obligation_version: obligation.obligation_version,
owner_sha256: "1".repeat(64),
fixtures: [
{
fixture_id: "fixture.positive",
case_kind: "positive",
passed: true,
},
],
verifiers: [
{
verifier_id: "verifier.reviewed",
claim_id: "claim.reviewed",
passed: true,
},
],
},
],
});
return request([original, proof], {
reviewed_obligations: [obligation],
manifest: {
bindings: [
{
obligation_id: obligation.obligation_id,
owner: {
module_path: "src/owner.ts",
symbol: "run",
owner_sha256: "1".repeat(64),
},
parser_type: null,
original_cases: [
{
kind: "positive",
evidence_id: original.evidence_id,
location: "/normalized_result",
},
],
fixtures: [
{
fixture_id: "fixture.positive",
case_kind: "positive",
authority: "unit",
evidence_ids: [proof.evidence_id],
},
],
verifier: {
verifier_id: "verifier.reviewed",
claim_id: "claim.reviewed",
command: "synthetic proof (not executed)",
authority: "unit",
status: "pass",
result_evidence_id: proof.evidence_id,
enumerated_obligation_ids: [obligation.obligation_id],
nondeterminism: {
mode: "total-order",
specification: "Single result",
},
},
},
],
contradictions: [],
},
});
};
@@ -19,7 +19,8 @@ import {
} from "../domain/reconstructionObligationLedgerIntegrity.js";
import type { ReconstructionObligationCandidate } from "./ReconstructionObligationCandidates.js";
import {
evidenceAuthoritySupportsProof,
parseReconstructionProofEvidence,
qualifyReconstructionProof,
evidenceSupportsOriginalAuthority,
proofAuthoritySatisfies,
} from "./ReconstructionObligationAuthority.js";
@@ -43,6 +44,10 @@ type Binding = NonNullable<ReconstructionObligation["binding"]>;
interface EvaluationContext {
readonly candidates: ReadonlyMap<string, ReconstructionObligationCandidate>;
readonly evidence: ReadonlyMap<string, Evidence>;
readonly proofs: ReadonlyMap<
string,
ReturnType<typeof parseReconstructionProofEvidence>
>;
readonly bindings: ReadonlyMap<string, readonly Binding[]>;
readonly contradictions: ReturnType<typeof groupManifestContradictions>;
}
@@ -118,6 +123,12 @@ const createEvaluationContext = (
candidates.map((candidate) => [candidate.obligation_id, candidate]),
),
evidence,
proofs: new Map(
[...evidence].map(([id, record]) => [
id,
parseReconstructionProofEvidence(record),
]),
),
bindings: groupManifestBindings(manifest),
contradictions: groupManifestContradictions(manifest),
});
@@ -250,6 +261,16 @@ const evaluateBinding = (
"missing-parser-type",
"This boundary requires an explicit parser, schema, and domain type.",
);
// Optional fixture kinds are retained and linked too, so their Evidence
// references must resolve even though only required kinds close the claim.
for (const fixture of binding.fixtures)
for (const evidenceId of fixture.evidence_ids)
if (!context.evidence.has(evidenceId))
addDiagnostic(
diagnostics,
"missing-case",
`${fixture.case_kind} fixture ${fixture.fixture_id} references Evidence ${evidenceId}, which is absent from the bundle.`,
);
for (const caseKind of candidate.required_case_kinds) {
const fixtures = binding.fixtures.filter(
({ case_kind: fixtureCase }) => fixtureCase === caseKind,
@@ -262,38 +283,51 @@ const evaluateBinding = (
);
continue;
}
for (const fixture of fixtures)
for (const evidenceId of fixture.evidence_ids)
if (!context.evidence.has(evidenceId))
addDiagnostic(
diagnostics,
"missing-case",
`${caseKind} fixture ${fixture.fixture_id} references Evidence ${evidenceId}, which is absent from the bundle.`,
);
const qualifications = fixtures.map((fixture) => ({
authoritySatisfied: proofAuthoritySatisfies(
fixture.authority,
candidate.required_fixture_authority,
),
proofs: fixture.evidence_ids.map((id) => {
const proof = context.proofs.get(id);
return proof === undefined
? ({ ok: false, error: "missing-evidence" } as const)
: qualifyReconstructionProof(proof, fixture.authority, {
kind: "fixture",
obligationId: candidate.obligation_id,
obligationVersion: candidate.obligation_version,
ownerSha256: binding.owner.owner_sha256,
fixtureId: fixture.fixture_id,
caseKind: fixture.case_kind,
});
}),
}));
if (
!fixtures.some(
(fixture) =>
proofAuthoritySatisfies(
fixture.authority,
candidate.required_fixture_authority,
) &&
fixture.evidence_ids.every((id) => {
const evidence = context.evidence.get(id);
return (
evidence !== undefined &&
evidenceAuthoritySupportsProof(evidence, fixture.authority, {
obligationId: candidate.obligation_id,
fixtureId: fixture.fixture_id,
caseKind: fixture.case_kind,
})
);
}),
qualifications.some(
({ authoritySatisfied, proofs }) =>
authoritySatisfied && proofs.every((proof) => proof.ok),
)
)
continue;
const failures = qualifications.flatMap(({ proofs }) =>
proofs.flatMap((proof) => (proof.ok ? [] : [proof.error])),
);
if (
qualifications.some(({ authoritySatisfied }) => !authoritySatisfied) ||
failures.includes("insufficient-authority")
)
addDiagnostic(
diagnostics,
"weak-fixture-authority",
`${caseKind} fixture lacks ${candidate.required_fixture_authority} authority or authenticated Evidence.`,
`${caseKind} fixture lacks ${candidate.required_fixture_authority} authority.`,
);
if (failures.length > 0)
addDiagnostic(
diagnostics,
failures.includes("failed-proof")
? "fixture-failed"
: "inapplicable-fixture-proof",
`${caseKind} fixture proof qualification: ${[...new Set(failures)].sort().join(", ")}.`,
);
}
evaluateVerifier(candidate, binding, context, diagnostics);
@@ -331,23 +365,42 @@ const evaluateVerifier = (
!proofAuthoritySatisfies(
verifier.authority,
candidate.required_verifier_authority,
) ||
(resultEvidence !== undefined &&
!evidenceAuthoritySupportsProof(resultEvidence, verifier.authority, {
obligationId: candidate.obligation_id,
verifierId: verifier.verifier_id,
claimId: verifier.claim_id,
}))
)
)
addDiagnostic(
diagnostics,
"weak-verifier-authority",
`Verifier authority does not satisfy ${candidate.required_verifier_authority}.`,
);
const proof = context.proofs.get(verifier.result_evidence_id);
if (proof !== undefined) {
const qualification = qualifyReconstructionProof(
proof,
verifier.authority,
{
kind: "verifier",
obligationId: candidate.obligation_id,
obligationVersion: candidate.obligation_version,
ownerSha256: binding.owner.owner_sha256,
verifierId: verifier.verifier_id,
claimId: verifier.claim_id,
},
);
if (!qualification.ok)
addDiagnostic(
diagnostics,
qualification.error === "failed-proof"
? "verifier-failed"
: qualification.error === "insufficient-authority"
? "weak-verifier-authority"
: "inapplicable-verifier-proof",
`Verifier proof qualification: ${qualification.error}.`,
);
}
if (verifier.status !== "pass")
addDiagnostic(
diagnostics,
"verifier-failed",
verifier.status === "fail" ? "verifier-failed" : "verifier-unavailable",
`Latest verifier status is ${verifier.status}.`,
);
};
@@ -13,7 +13,7 @@ import {
type ReconstructionObligationManifest,
} from "../domain/reconstructionObligationLedgerSchemas.js";
import { err, ok, type Result } from "../domain/result.js";
import { requestInputError } from "./workflowInputError.js";
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
const OPERATION = "build_reconstruction_obligation_ledger";
const requestSchema = reconstructionObligationLedgerInputSchema.extend({
@@ -38,7 +38,11 @@ export const resolveReconstructionObligationLedgerRequest = (
): Result<ResolvedReconstructionObligationRequest, AnalysisError> => {
const frame = requestSchema.safeParse(input);
if (!frame.success)
return err(requestInputError(OPERATION, frame.error, input));
return err(
analysisInputErrorFromIssues(OPERATION, frame.error.issues, input, {
cause: frame.error,
}),
);
const bundle = readEvidenceBundle(frame.data.evidence_bundle);
if (!bundle.ok) return bundle;
const records = new Map(
@@ -2,7 +2,7 @@ import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import type { ReconstructionObligationLedger } from "../domain/reconstructionObligationLedgerSchemas.js";
import { ok, type Result } from "../domain/result.js";
import { JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER } from "./InvestigationProviders.js";
import { JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER } from "../domain/investigationProviders.js";
import { deriveReconstructionObligationCandidates } from "./ReconstructionObligationCandidates.js";
import { evaluateReconstructionObligationLedger } from "./ReconstructionObligationLedgerEvaluation.js";
import {
@@ -36,6 +36,14 @@ export const originalCaseDiagnostics = (
): Diagnostic[] => {
const diagnostics: Diagnostic[] = [];
const originalCases = originalCasesFor(candidate, binding);
// Retain optional case rows, but never leave one of their Evidence links
// unresolved merely because its kind is not required for closure.
for (const { kind, evidence_id: evidenceId } of originalCases)
if (!evidenceById.has(evidenceId))
diagnostics.push({
code: "missing-original-case",
detail: `Original ${kind} case references Evidence ${evidenceId}, which is absent from the bundle.`,
});
for (const caseKind of candidate.required_case_kinds) {
const cases = originalCases.filter(({ kind }) => kind === caseKind);
if (cases.length === 0) {
@@ -45,14 +53,6 @@ export const originalCaseDiagnostics = (
});
continue;
}
// A valid entry for this case kind does not resolve another entry's
// reference; every referenced Evidence ID must be in the bundle.
for (const { evidence_id: evidenceId } of cases)
if (!evidenceById.has(evidenceId))
diagnostics.push({
code: "missing-original-case",
detail: `Original ${caseKind} case references Evidence ${evidenceId}, which is absent from the bundle.`,
});
if (
!cases.some(({ evidence_id: evidenceId }) => {
const evidence = evidenceById.get(evidenceId);
+1 -1
View File
@@ -210,7 +210,7 @@ export const importReferenceSource = async (
);
const { entries, relationships, parseFailures, limitations } =
parseReferenceSourceEntries(read, filePaths, options.signal);
parseReferenceSourceEntries(read, filePaths, exclusions, options.signal);
if (isAborted(options.signal)) return err(cancelled(read));
@@ -22,7 +22,7 @@ it("projects an unsupported non-regular entry as unknown kind instead of file",
bytesRead: 0,
limitations: [],
};
const parsed = parseReferenceSourceEntries(read, new Set());
const parsed = parseReferenceSourceEntries(read, new Set(), []);
expect(parsed.entries).toContainEqual({
path: "src/pipe",
kind: "unknown",
@@ -49,7 +49,7 @@ it("keeps a failed regular file as file kind", () => {
bytesRead: 0,
limitations: [],
};
const parsed = parseReferenceSourceEntries(read, new Set());
const parsed = parseReferenceSourceEntries(read, new Set(), []);
expect(parsed.entries).toContainEqual({
path: "src/broken.ts",
kind: "file",
@@ -88,6 +88,7 @@ it("keeps source entries when a module has an empty specifier", () => {
const parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
[],
);
expect(parsed.entries.map(({ path }) => path)).toEqual([
@@ -136,6 +137,7 @@ it("retains Kotlin source identity and language across the inventory projection"
const parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
[],
);
for (const [path, content, classifications, language] of files)
expect(parsed.entries).toContainEqual({
@@ -153,9 +155,12 @@ it("retains Kotlin source identity and language across the inventory projection"
expect(parsed.relationships).toEqual([]);
});
it("resolves runtime require package mains without an index and before typed indexes", () => {
it("resolves declared package mains and index fallback from source-owned fields", () => {
const files = [
["main.cjs", 'require("./pkg");\nrequire("./dep");\n'],
[
"main.cjs",
'require("./pkg");\nrequire("./dep");\nrequire("./fallback");\n',
],
[
"main.ts",
'import dep = require("./typed");\n' +
@@ -165,6 +170,9 @@ it("resolves runtime require package mains without an index and before typed ind
],
["pkg/package.json", '{"main":"actual.js"}\n'],
["pkg/actual.js", 'module.exports = "package main";\n'],
["fallback/package.json", "{}\n"],
["fallback/index.js", 'module.exports = "index";\n'],
["fallback/phantom.js", 'module.exports = "host decoy";\n'],
["dep.js", 'module.exports = "file";\n'],
["dep/index.ts", 'export const value = "index";\n'],
["typed.js", 'module.exports = "runtime file";\n'],
@@ -190,11 +198,32 @@ it("resolves runtime require package mains without an index and before typed ind
limitations: [],
};
const parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
);
const previous = Object.getOwnPropertyDescriptor(Object.prototype, "main");
let parsed: ReturnType<typeof parseReferenceSourceEntries>;
try {
Object.defineProperty(Object.prototype, "main", {
configurable: true,
writable: true,
value: "phantom.js",
});
parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
[],
);
} finally {
if (previous === undefined)
Reflect.deleteProperty(Object.prototype, "main");
else Object.defineProperty(Object.prototype, "main", previous);
}
expect(parsed.relationships).toContainEqual({
from_path: "main.cjs",
to: "fallback/index.js",
kind: "requires",
resolution: "internal",
parse_state: "parsed",
});
expect(parsed.relationships).toContainEqual({
from_path: "main.cjs",
to: "pkg/actual.js",
@@ -281,6 +310,7 @@ it("matches package manifests by exact or unique canonical directory identity",
const parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
[],
);
expect(parsed.relationships.map(({ to }) => to)).toEqual([
@@ -319,6 +349,7 @@ it("resolves extensionless imports and requires to .mts and .cts files and direc
const parsed = parseReferenceSourceEntries(
read,
new Set(files.map(([path]) => path)),
[],
);
expect(parsed.relationships).toEqual(
expect.arrayContaining([
@@ -107,12 +107,18 @@ type PackageMain =
*/
const readPackageMains = (
read: ReferenceSourceRead,
exclusions: HistoricalSourceGraphInput["exclusions"],
): {
readonly mains: ReadonlyMap<string, PackageMain>;
readonly pathLookup: ReferenceSourcePathLookup;
} => {
const mains = new Map<string, PackageMain>();
const manifestPaths: string[] = [];
for (const exclusion of exclusions) {
if (posix.basename(exclusion.path) !== "package.json") continue;
manifestPaths.push(exclusion.path);
mains.set(posix.dirname(exclusion.path), { state: "unknown" });
}
for (const entry of read.entries) {
if (posix.basename(entry.path) !== "package.json") continue;
manifestPaths.push(entry.path);
@@ -130,7 +136,9 @@ const readPackageMains = (
new TextDecoder("utf-8", { fatal: true }).decode(entry.bytes),
);
const main =
typeof manifest === "object" && manifest !== null
typeof manifest === "object" &&
manifest !== null &&
Object.hasOwn(manifest, "main")
? Reflect.get(manifest, "main")
: undefined;
mains.set(
@@ -345,7 +353,8 @@ const appendFile = (
if (parsed === undefined) return;
output.limitations.push(...parsed.limitations);
for (const relationship of parsed.relationships) {
const { resolutionStrategy, ...graphRelationship } = relationship;
const { resolutionStrategy, resolutionBasePath, ...graphRelationship } =
relationship;
if (graphRelationship.resolution !== "internal") {
output.relationships.push(graphRelationship);
continue;
@@ -357,7 +366,7 @@ const appendFile = (
? "source-require"
: "source-import";
const resolved = resolveInternalSpecifier(
graphRelationship.from_path,
resolutionBasePath ?? graphRelationship.from_path,
graphRelationship.to,
context,
mode,
@@ -412,6 +421,7 @@ const appendReadEntry = (
export const parseReferenceSourceEntries = (
read: ReferenceSourceRead,
filePaths: ReadonlySet<string>,
exclusions: HistoricalSourceGraphInput["exclusions"],
signal?: AbortSignal,
): ParsedReferenceSourceEntries => {
const output: ParsedReferenceSourceEntries = {
@@ -424,7 +434,7 @@ export const parseReferenceSourceEntries = (
output.limitations.push(
`Reference root enumeration is incomplete: ${read.rootFailure.message}`,
);
const packageMainInventory = readPackageMains(read);
const packageMainInventory = readPackageMains(read, exclusions);
const context: SpecifierContext = {
lookupPath: createReferenceSourcePathLookup(filePaths),
packageMains: packageMainInventory.mains,
Loaded 100 of 725 files, more files were not shown because too many files have changed in this diff. Show more