Compare commits

...
Author SHA1 Message Date
N0zoM1z0 2b72592f89 perf(ida): batch native lookups and narrow literal search
Bind each lookup reply to its query, batch unique native callees, and fetch conservative literal string candidates while preserving Unicode, case, pagination, and raw source Evidence. Preserve explicit native search failures and describe unknown string representations.

Verified with Windows IDA Pro 9.3 and the native MCP plugin built from c9d15d98e8a143fc74f6e14037c48f63889b6d6c with a loopback listener. Production CLI/MCP parity and two-function switching passed on the public fixture; equivalent tasks passed on Windows whoami.exe. Windows IDA Free was not used. No personal paths or license registration information are included.

Validation: 182 focused tests, static checks, architecture guards, test discovery, and docs checks passed. Same-main measurements show warm task time 675 to 317 ms and estimated response tokens 16618 to 11509; cold-start, overall CPU, and memory improvement were not established. Detailed scope and measurements are in experiments/analysis-efficiency/ida-native-windows.md.
2026-10-13 02:14:25 +08:00
N0zoM1z0 fb34ccbec7 chore: sync main for native IDA verification
The native IDA follow-up is being verified on Windows IDA Pro 9.3. IDA Free is not used in this follow-up.
2026-10-13 00:47:56 +08:00
Ha1baraA1andmorluto 59be866719 fix(ghidra): encode Java probe output as UTF-8 (#2206)
* fix(ghidra): encode Java probe output as UTF-8

* fix(ghidra): preserve Unicode Java homes across supported JDKs

* test(ghidra): verify snapshot rejection without matching prose

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 00:45:23 +08:00
Ysopwnableandfactory-droid[bot] 8e004522f2 docs: define performance priorities for analysis workflows (#2245)
Set agent tokens and context first, analysis time second, and CPU and memory
third while preserving correctness, Evidence and existing contracts.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-10-13 00:42:06 +08:00
Lucienandmorluto d80b55e11b fix(adb): preserve existing bugreport archives (#2221)
* fix(adb): preserve existing bugreport archives

* fix(adb): retain bugreports after publication failure

Record acquired archive facts before publishing, preserve the staging archive when the exclusive copy fails, and keep retained evidence out of automatic cleanup. Reuse the same observation through publication and digest failures.

Add a real-filesystem ENOSPC regression covering error projection, existing evidence, subsequent collection and provider shutdown. Verified 96 focused tests, static checks and docs on macOS with simulated ADB; no real-device verification.

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 00:25:17 +08:00
Luis Araya 85c5477930 docs(readiness): align required stage identifier and checks with evaluator (#2200) 2026-10-13 00:15:54 +08:00
Saim Shafique 7724edbdd3 fix(process): report single empty settlement observation as unverifiable (#2232)
observeSettlement classified a group as alive_at_deadline when the settlement
budget expired after one empty observation, though no live member was ever
seen. Track observed liveness and return unverifiable when the deadline ends
observation with only empty rounds. The two-consecutive-empty rule for
quiesced is unchanged, and the cleanup_outcome projection for non-quiesced
settlements stays cleaned.

Fixes #2170.
2026-10-13 00:15:12 +08:00
The one -Niuandmorluto 0baad58e60 feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane (#2151)
* feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane

Refs #2063.

Recognize `EM_RISCV` (243) as a provider-neutral family that carries its own ELF
facts, and bound the first Ghidra lane to little-endian `ET_EXEC` images that
declare the double-float ABI, pinned to `RISCV:LE:32:default` or
`RISCV:LE:64:default` with the `gcc` compiler specification.

Ghidra 12.1.4 is not reliable without the pin: its `riscv.opinion` pins
`variant="default"` for the 64-bit constraint and leaves the 32-bit constraint
with the comment that the loader "must choose between 'default' and 'AndeStar'
variants". Measured on the installed release tree, a source-owned generic RV32
ELF with no Andes declaration imports as `RISCV:LE:32:AndeStar_v5:gcc`, while
RV64 resolves to `RISCV:LE:64:default:gcc`. The lane therefore states the
language per width and refuses any caller-selected processor outside it instead
of inheriting the header guess.

Refusals name the declaration rather than falling back to a guessed profile:
big-endian images (Ghidra 12.1.4 ships no big-endian RISC-V language),
non-`ET_EXEC` images, `EF_RISCV_RVE`, `EF_RISCV_TSO`, and every float ABI other
than double, which the pinned generic compiler specifications model. The raw
e_flags bits stay in Evidence as declarations, not as instruction semantics.

Family recognition now has one owner for the ELF-metadata families:
`ELF_METADATA_ARCHITECTURES`, `isElfMetadataArchitecture`, and
`HeaderArchitecture` replace ten `Exclude<BinaryArchitecture, "mips">` sites that
would each have needed a second hand-edited member. `ghidraPinnedLanguage` is
that owner for the import commitment: the session readback previously re-derived
only the MIPS pin, so a profile committing any second pinned language failed with
a language-mismatch error before the bridge started.

Hopper and IDA refuse RISC-V by name, matching the MIPS wording. Caller-supplied
raw-binary entry keeps its existing four-family list; this lane is ELF-header
admission only.

Verification on macOS 15.7.7 arm64, Node 24.18.0, Ghidra 12.1.4 PUBLIC and JDK
21.0.12.1, with the host decompiler built from the release sources for
`os/mac_arm_64`: `npm run check` passes; `npm run test:fast` passes 396 test
files; `npm run verify:test-discovery` discovers all 917 test files exactly once.
Source-owned RV32 and RV64 double-float `ET_EXEC` fixtures complete
`rea analyze --provider ghidra` and `rea decompile rea_riscv_entry`, committing
`RISCV:LE:32:default` and `RISCV:LE:64:default` with `language_selection:
provider-pinned-v1`; the soft-float RV64 shape is refused with the named
float-ABI reason, and the full refusal matrix including the RV32 and
width-mismatch shapes is covered by unit tests. No `verify:ghidra:riscv` lane
script is included in this change; the runs above were performed manually against
the installed engine. Fork pull requests receive no CI check runs, so the
real-provider scope above is the evidence available for review.

* fix(ghidra): bound the RISC-V lane by declared flags and document it

Address review feedback on #2151.

`e_flags` bits outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI` are now refused with
the offending value named, so a reserved or newer producer declaration cannot
enter the verified generic lane by falling through the RVE, TSO and float-ABI
checks.

Admission reads `e_type` only, so the refusal text no longer calls `ET_EXEC`
static: a fixed-address declaration does not establish static linkage, and
`PT_INTERP` and dynamic dependencies are not inspected. The session records that
boundary as a limitation rather than inferring a load model.

Add `docs/ghidra-riscv.md` in the shape of the MIPS lane guide (admission
boundary, refusal table, why the language is pinned per class, Evidence fields,
and what the fixtures do not establish), and correct
`docs/firmware-analysis.md`, which still told users that MIPS and RISC-V
decompilation had not been added.

Verification after the change: `npm run check` and `npm run docs:check` pass; the
RISC-V module tests pass (17 lane cases plus 9 parser cases); on Ghidra 12.1.4
with JDK 21.0.12.1 the RV32 and RV64 double-float fixtures still analyze and
decompile with `RISCV:LE:32:default` / `RISCV:LE:64:default` and
`language_selection: provider-pinned-v1`, the soft-float fixture is still refused
by name, and the new linkage limitation appears in the emitted Evidence.

* fix(ghidra): align RISC-V pinning with admission

Reuse the complete lane admission boundary when selecting a language, retain unsigned unsupported flag bits in diagnostics, and derive float ABI metadata from the source declaration.

* test(ghidra): verify RISC-V CLI and MCP workflows

Build source-owned RV32/RV64 fixtures with Clang and LLD, establish independent ELF and symbol facts with LLVM readelf, and verify real provider evidence, instruction bytes, calls, decompilation, schema projections, snapshot replay, refusal diagnostics, source immutability and cleanup.

Verified on macOS ARM64 with Ghidra 12.1.4, JDK 21 and LLVM/LLD 23.1.2. Dynamic linkage, big-endian, Andes and RVE/TSO workflows remain unverified; ISA attributes are not inspected.

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 00:11:35 +08:00
Ysopwnableandfactory-droid[bot] 1770f38cfc test(native): verify real PE and DOS entry views
Record real Ghidra/MCP and Codex CLI verification of shipped ShareX native PE and FreeDOS DOS MZ entry views. Preserve original Evidence provenance, separate run-specific references from facts, and reject stale/colliding producer identities or unsupported model claims.

Sixteen focused experiment regression cases, original public-reply/transcript replays, and CI run 38273269072 pass for head 0667f7cb7c. No production API/default change or general native speedup claim; final resource counters remain unknown. Continues #2016.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-10-13 00:04:00 +08:00
N0zoM1z0 c9c40f6f9e fix(ida): reconcile native bridge with main and verify function switching
Merge the PR 1544 native IDA bridge onto current main and validate two-function pseudocode identity with a unique second-body marker. Preserve current input-error and GUI-effect semantics, identify the adapter without implying an IDA edition, and document the upstream listener and navigation limits.

Verified on Windows IDA Pro 9.3 through the attached legacy MCP workflow: production CLI/MCP analysis, distinct-function pseudocode, return to the original function, target identity, and cleanup. Native Windows plugin and IDA Free runtime workflows were not verified in this follow-up. No private target paths or license details are included.
2026-10-12 23:53:01 +08:00
Ysopwnable 428eee00fd ci: keep validation fresh and preserve failed-shard reports (#2210)
Run the existing README consistency and Python producer guards from their
CI owners. Execute full local validation fresh while preserving cached
preparation, and permit scoped reuse of earlier portable build entries.

Collect failed-shard diagnostics before enforcing success and publish
compact, portable timing history only after a complete successful merge.
Preserve installer truncation failure across supported Bash versions.

Keep cancellation tests focused on the called MCP tool, observe server
progress before timing target readiness, and retain delayed-startup,
cancellation, cleanup, and live follow-up request assertions.

Validation: all 32 CI jobs and the separate Go lane passed; 935 files,
9,630 cases, 9,544 passed, 86 skipped, zero failed. Controlled regressions
and fault/restoration probes are recorded in PR #2210.

Refs #2190.

Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
2026-10-12 23:42:47 +08:00
morlutoandJax367 94153f1c75 fix(uninstall): label YAML configuration syntax failures correctly
Report malformed Hermes and DeepSeek Harness documents as YAML. Carry the
compiled setup/uninstall diagnostics coverage from #2153 onto the existing
validation from #2157, including unchanged files and no backup writes.
Add malformed YAML scenarios for both clients.

Validation: 174 focused tests, npm run check, test discovery, package
verification, and npm pack --dry-run.

Co-authored-by: Jax367 <239124526+Jax367@users.noreply.github.com>
2026-10-12 23:38:39 +08:00
Ysopwnable ac6ef95e12 docs: define contribution scope and simplify proposal forms
Define REA contribution scope and fail closed on unresolved integration boundaries.

Make proposal forms easier to complete with structured choices, explicit unknowns, and optional technical details; welcome exploratory drafts while retaining maintainer review before integration.
2026-10-12 23:36:56 +08:00
b1bf4a70d4 fix(setup): name invalid client config shape errors clearly (#2157)
* fix(setup): name invalid client config shape errors clearly

Setup was dumping raw Zod issues for non-object roots or server tables,
and uninstall always claimed the file was not valid JSON. Report the
structural problem instead, and reserve parse failures for real syntax errors.

Fixes #2152

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(setup): preserve client configuration record validation

Keep descriptive shape errors while retaining Zod record validation and its own-property copy. Reject tagged YAML objects before setup or uninstall writes and prevent inherited JSONC properties from fabricating client registrations.

Add production-boundary regressions for tagged YAML roots and server tables, plus inherited Cursor and OpenCode settings. All 91 focused tests, npm run check, verify:test-discovery, verify:package, and npm pack --dry-run passed.

---------

Co-authored-by: Beqa Ksovreli <beqa@Beqas-Macbook-Pro-2.local>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-12 23:24:24 +08:00
7711b00005 fix(apple): preflight XML plist nesting before keyed archive decoding (#2161)
* fix(apple): preflight XML plist nesting before keyed archive decoding

inspect_keyed_archive documented a 128-level nesting limit, but the XML
plist path reached recursive decoding before any depth check: plist and
the prototype-key traversal in parseXmlPropertyList recurse once per
nesting level, so a ~75 KiB, 3000-level XML plist exhausted the call
stack and reported Maximum call stack size exceeded instead of the
bounded-resource limit that shallower over-limit documents (129, 200)
already produced. The CLI classified both as truncated, so no crash or
classification bypass was involved; the diagnostic was the defect.

Reject excessive nesting in parseXmlPropertyList before any recursive
pass, using the projection's own limit (MAX_PLIST_DEPTH) so the error is
identical to the one projectPlistValue raises: callers keep the
limit/truncated classification with no error-handler translation. Depth
counts values the way the projection does - container members nest one
level deeper, dictionary keys name members rather than nesting, and a
self-closing element is the value at its own level - so the guard
rejects exactly the documents the projection rejected, at the same
boundary, and every in-limit document decodes unchanged. The guard sits
at the shared decoder boundary, so Interface Builder, app Info.plist
resolution and native inspect_plist gain the same deterministic limit;
the binary plist path keeps its existing expansion preflight.

Binary plist behavior, ordinary XML decode, __proto__ omission counting,
and archive digest/Evidence provenance are unchanged. OpenStep
autodetection is untouched by this guard, as the existing Interface
Builder preflight also scopes its depth check to XML.

Regression coverage: decoder unit tests at and just past the limit for
dictionary and array nesting with self-closing and opaque markup,
module tests through decodeKeyedArchiveBytes including a valid
NSKeyedArchiver XML envelope at the limit, a production-boundary test
through inspectBundleKeyedArchive, and the compiled public CLI
reporting code truncated / reason limit for a 3000-level plist.

* fix(apple): check XML nesting against parsed element structure

Reuse the iterative XML decoder document for depth preflight and prototype-key lookup. Bound recursive plist wrappers alongside dictionaries and arrays without interpreting markup inside attributes or DOCTYPE declarations as values.

Validated with 104 focused tests including compiled CLI, MCP prototype-key, and Interface Builder coverage; npm run check and verify:test-discovery passed. No real-provider verification was run.

---------

Co-authored-by: Zburgers <Zburgers@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-12 23:12:10 +08:00
Kaoru0822andClaude Opus 5.5 c9aecde580 fix(frida): reject an unreadable script file before starting a session (#2211)
* fix(frida): reject an unreadable script file before starting a session

The shared instrument_with_frida workflow started the Frida session, which
attaches or spawns the target, before the provider read a file script
source. A missing script file then failed loadScript with a bare
invalid_request, and session cleanup resumed the paused spawn, so the
target ran without instrumentation. Check that the selected file is a
readable regular file first and report the path, or access_denied for
EACCES/EPERM, before any device effect.

Fixes #2209

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(frida): let cancellation win over the script file check

An already-aborted request with an unreadable file source returned the
input or access-denied error instead of AnalysisCancelledError. Check the
signal before the script file preflight and again after it, so file
sources keep the existing cancellation semantics.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(frida): keep the pre-start checks out of instrument

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 23:10:39 +08:00
Ysopwnableandfactory-droid[bot] ae412654d5 docs: preserve the native guide title anchor (#2224)
Keep the former native-ui-and-dispatch-investigation fragment after the
page title was changed in #2207. The old and current fragments resolve to
the same heading position in the generated page.

Validation: VitePress production build, generated-document checks, format
and link checks, and real Chrome navigation to both fragments.
Refs #2156.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-10-12 23:09:11 +08:00
Kaoru0822andClaude Opus 5.5 86de439ba4 fix(frida): name the argument each Frida input error rejects (#2217)
Frida session lookups, script lookups, unreadable script files and the
instrument-with-frida PID and script checks returned invalid_request with
no issues, while the remediation told callers to correct the listed
arguments. Attach the session_id, script_id, path, target or script issue
with the reason so CLI and MCP callers can see what to fix.

Fixes #2213

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 23:00:59 +08:00
Iven Kooandmorluto 109769319a feat(clients): register Qoder CLI during setup (#1572)
* feat(clients): register Qoder CLI during setup

- SupportedClients: qoder definition (~/.qoder/settings.json, format
  json, QODER_CONFIG_DIR override honored)
- Qoder CLI documents mcpServers as the user-scope server table with
  command/args/env and stdio as the default transport; no personal
  skills directory is documented, so setup skips the skill copy
- tests: qoderClientConfiguration.test.ts (path resolution and
  QODER_CONFIG_DIR override, upsert preserving unrelated settings,
  fresh/empty configs, uninstall)

Verified: tsc --noEmit clean; vitest run
tests/boundary/filesystem/qoderClientConfiguration.test.ts 6/6.

* fix(clients): install Qoder CLI's personal REA skill

Review follow-up (#1572): the Qoder CLI Skills documentation lists
~/.qoder/skills/{skill-name}/SKILL.md with automatic discovery, so the
definition now carries skillPath following Qoder's configuration
location (QODER_CONFIG_DIR honored) and setup installs the bundled
skill copy there. The installation docs no longer claim Qoder has no
personal skills directory, and the bundled REA skill's supported
client list includes qoder.

Verified: tsc --noEmit clean; vitest run
tests/boundary/filesystem/qoderClientConfiguration.test.ts 7/7.

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-12 22:55:55 +08:00
Fernando 0b60a2ce90 feat(ida): support optional native IDA Free MCP in attached mode 2026-10-10 12:21:17 +01:00
111 changed files with 6170 additions and 617 deletions

No files matched your search

@@ -46,7 +46,7 @@ rea setup --client codex --dry-run --json
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
`hermes`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
`hermes`, `qoder`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
For unsupported clients, use version-pinned manual stdio registration or CLI.
+3
View File
@@ -1,5 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Contribution scope and integration requirements
url: https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope
about: Check supported capabilities, excluded workflows, and the fail-closed integration rule before proposing a feature.
- name: REA Documentation
url: https://github.com/morluto/rea#readme
about: Read the README and AGENTS.md for usage guides and configuration details.
+55 -13
View File
@@ -1,5 +1,5 @@
name: Design discussion
description: Resolve an REA architecture, provider, or contract question before implementation
description: Discuss an REA architecture, provider, contract, or scope decision
title: "[Design] "
labels:
- enhancement
@@ -9,7 +9,12 @@ body:
attributes:
value: |
Use this for a substantial provider, public/persisted contract, or
authority-boundary decision. Keep one decision per discussion.
authority-boundary decision. One question and a short explanation are
enough to start. Choose "Not sure yet" where needed; maintainers can help
with technical details, alternatives, and verification.
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
Unresolved boundaries block integration, not opening a discussion.
- type: checkboxes
id: before-submitting
@@ -18,14 +23,16 @@ body:
options:
- label: I searched the open and closed issues and pull requests for a related discussion.
required: true
- label: I removed credentials, tokens, private paths, and sensitive target data.
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
required: true
- label: This discussion does not include credentials, private samples, or sensitive target data.
required: true
- type: textarea
id: decision
attributes:
label: Decision needed
description: State the exact question maintainers need to resolve.
description: What would you like maintainers to help decide? A short question is enough.
placeholder: Should REA ... or ...?
validations:
required: true
@@ -34,23 +41,57 @@ body:
id: context-evidence
attributes:
label: Context and evidence
description: Explain why the decision is needed now. Separate observed behavior, derived evidence, and inference.
description: Explain the problem or motivation. If you have evidence, distinguish observed behavior from ideas or guesses.
validations:
required: true
- type: dropdown
id: target-interaction
attributes:
label: Expected interaction with targets
description: >-
Select all known effects, including upstream tool behavior. Add "Not sure
yet" if anything is unclear. Use "No target interaction" only on its own.
These selections help discussion and do not establish scope approval.
multiple: true
options:
- Read existing files or captured data
- Launch or attach to a process
- Inject instrumentation or run scripts/code
- Change process memory or registers
- Write or change files/configuration
- Contact network services
- Recover protected code or work around protections
- No target interaction (documentation or usability only)
- Not sure yet
validations:
required: true
- type: textarea
id: boundary-invariants
attributes:
label: Boundary and invariants
description: Identify the owning layer, provider/bridge boundary, artifact or evidence invariants, and any process or authorization constraints.
validations:
required: true
label: Boundary details (optional)
description: >-
If known, clarify target selection, selected effects, provider boundaries,
non-goals, or evidence invariants. Leave blank if you need help identifying
them. Do not include private samples or proof-of-authorization documents.
- type: textarea
id: options
attributes:
label: Options and tradeoffs
description: Compare a small number of viable approaches, including platform, compatibility, safety, evidence, and maintenance costs.
description: Optional approaches or tradeoffs. One idea is enough; maintainers can help compare alternatives.
- type: dropdown
id: verification-readiness
attributes:
label: Verification available so far
description: Pick the closest match. An idea without samples or test results is welcome.
options:
- Idea only; I need help defining checks
- I can suggest a sample or manual check
- I have a prototype or observed results
- Not sure yet
validations:
required: true
@@ -58,9 +99,10 @@ body:
id: success-criteria
attributes:
label: Success criteria and open questions
description: State what outcome would settle the decision and list only unresolved questions that could change the choice.
validations:
required: true
description: >-
Optional expected output, sample, possible check, or unresolved question.
A complete test plan is not required to start; maintainers and implementers
will resolve material verification gaps before integration.
- type: textarea
id: references
+64 -11
View File
@@ -8,9 +8,13 @@ body:
- type: markdown
attributes:
value: |
Keep one primary capability or workflow outcome per request. For a
substantial provider, public contract, or architecture change, explain
the decision maintainers need to make before implementation.
Describe one problem and the outcome you would like. Short answers are
welcome. Choose "Not sure yet" wherever needed; you do not need code, a
finished design, or a complete test plan to propose an idea.
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
Maintainers can help resolve scope and technical questions. Unresolved
boundaries block integration, not opening a request for discussion.
- type: checkboxes
id: before-submitting
@@ -19,7 +23,9 @@ body:
options:
- label: I searched the open and closed issues and pull requests for a related proposal.
required: true
- label: I removed credentials, tokens, private paths, and sensitive target data.
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
required: true
- label: This request does not include credentials, private samples, or sensitive target data.
required: true
- type: dropdown
@@ -36,6 +42,7 @@ body:
- MCP or agent integration
- Documentation
- Other
- Not sure yet
validations:
required: true
@@ -43,7 +50,7 @@ body:
id: current-limitation
attributes:
label: Current limitation
description: What user or agent workflow is blocked, ambiguous, or unnecessarily difficult today?
description: What are you trying to understand or do, and what is difficult today? A short example is enough.
validations:
required: true
@@ -51,23 +58,69 @@ body:
id: use-cases
attributes:
label: Use cases
description: Give concrete applications, targets, callers, or investigation workflows. Explain why current tools are insufficient.
validations:
required: true
description: Optional additional examples of applications, targets, or callers. Leave blank if covered above.
- type: textarea
id: desired-outcome
attributes:
label: Desired outcome and success criteria
description: Describe observable behavior, inputs, outputs, evidence, and compatibility criteria that would make this useful.
description: What would a useful result look like? Describe the output or behavior you want; implementation details are optional.
placeholder: For a selected APK, show whether it is packed and include the detection evidence.
validations:
required: true
- type: dropdown
id: target-interaction
attributes:
label: Expected interaction with targets
description: >-
Select all known effects, including upstream tool behavior. Add "Not sure
yet" if anything is unclear. Use "No target interaction" only on its own.
These selections help discussion and do not establish scope approval.
multiple: true
options:
- Read existing files or captured data
- Launch or attach to a process
- Inject instrumentation or run scripts/code
- Change process memory or registers
- Write or change files/configuration
- Contact network services
- Recover protected code or work around protections
- No target interaction (documentation or usability only)
- Not sure yet
validations:
required: true
- type: textarea
id: boundaries
attributes:
label: Authority boundaries and non-goals
description: Call out process execution, target mutation, provider licensing, evidence provenance, supported platforms, and work that should remain out of scope.
label: Target and scope details (optional)
description: >-
If known, name the target/platform, clarify selected effects, or list a
non-goal or scope question. You can leave this blank; maintainers can help.
Do not include private samples or proof-of-authorization documents.
- type: dropdown
id: verification-readiness
attributes:
label: Verification available so far
description: Pick the closest match. An idea without samples or test results is welcome.
options:
- Idea only; I need help defining checks
- I can suggest a sample or manual check
- I have a prototype or observed results
- Not sure yet
validations:
required: true
- type: textarea
id: verification
attributes:
label: Verification details (optional)
description: >-
Share an expected output, a possible check, or existing results if you have
them. Self-owned or appropriately licensed samples can help, but are not
required to open a request. Do not attach private or sensitive data.
- type: textarea
id: alternatives
@@ -11,7 +11,7 @@ runs:
id: plan
shell: bash
env:
BUILD_CACHE_PREFIX: rea-portable-build-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc') }}
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'package-lock.json') }}
run: |
npm run --silent test:prepare -- --dry=json |
node --input-type=module -e '
@@ -25,6 +25,7 @@ runs:
}).sort();
const hash = createHash("sha256").update(hashes.join("\n")).digest("hex");
console.log(`key=${process.env.BUILD_CACHE_PREFIX}-${hash}`);
console.log(`prefix=${process.env.BUILD_CACHE_PREFIX}-`);
' >> "$GITHUB_OUTPUT"
- name: Restore portable build outputs
if: inputs.save != 'true'
@@ -32,9 +33,11 @@ runs:
with:
path: .turbo/cache/
key: ${{ steps.plan.outputs.key }}
restore-keys: ${{ steps.plan.outputs.prefix }}
- name: Restore and save portable build outputs
if: inputs.save == 'true'
uses: actions/cache@v4
with:
path: .turbo/cache/
key: ${{ steps.plan.outputs.key }}
restore-keys: ${{ steps.plan.outputs.prefix }}
+29 -3
View File
@@ -19,17 +19,39 @@ describe the use case and observable outcome. -->
<!-- Explain the chosen approach, why it fits REA's architecture, and what is
intentionally not included. -->
Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
Unresolved scope, authority, side effects, or required verification block
integration. Link the required maintainer scope decision for new providers or
capabilities involving target execution, injection, mutation, or protection
recovery; ordinary fixes within existing boundaries may state "not applicable".
Draft PRs may leave the scope decision pending and the final review checklist
unchecked. Maintainers can help resolve open questions.
## Contract and boundary impact
<!-- Complete the applicable lines; use "none" or "not applicable" explicitly. -->
<!-- Short answers are welcome. Complete applicable lines; mark unresolved facts as "unknown".
Use "none" or "not applicable" only when established. -->
- Semantic owner and earliest changed stage:
- Contribution scope fit and linked maintainer decision:
- CLI and MCP/tool-catalog contract:
- Provider, bridge, target-format, or platform compatibility:
- Evidence, artifact, provenance, or reconstruction contract:
- Process execution, authorization, cleanup, or containment impact:
- Target selection, authority, upstream behavior, and cleanup details (if applicable):
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs):
Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.
- [ ] Read existing files or captured data.
- [ ] Launch or attach to a process.
- [ ] Inject instrumentation or run scripts/code.
- [ ] Change process memory, registers, files, or configuration.
- [ ] Contact network services.
- [ ] Recover protected code or work around protections.
- [ ] Documentation or usability only; no target interaction.
- [ ] Not sure yet; investigation or maintainer guidance needed.
## Evidence and regression coverage
<!-- State whether evidence is executed, source-derived, or proposed. For fixes,
@@ -68,9 +90,13 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
- Package or release metadata impact:
- Security, privacy, process, or containment review:
## Review checklist
## Before merge
<!-- Draft PRs may leave pending items unchecked. Resolve material scope and
authority questions before integration. -->
- [ ] This PR addresses a concrete problem or an agreed enhancement.
- [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
- [ ] The PR has one focused outcome and the title follows `type(scope): outcome`.
- [ ] Related issue is linked, or the reason for not linking one is stated above.
- [ ] Tests cover changed observable behavior and meaningful failure paths.
+29 -27
View File
@@ -94,6 +94,9 @@ jobs:
run: npm run check:ci
- if: needs.changes.outputs.code == 'true'
run: npm run verify:test-discovery
- name: Check LLDB producer retention and launch contracts
if: needs.changes.outputs.code == 'true'
run: npm run test:python
- if: needs.changes.outputs.code != 'true'
run: npm run format:check
@@ -168,7 +171,7 @@ jobs:
continue-on-error: true
uses: actions/download-artifact@v4
with:
name: test-results
name: test-timings
run-id: ${{ steps.history.outputs.run-id }}
github-token: ${{ github.token }}
path: .cache/ci-test-history
@@ -203,14 +206,10 @@ jobs:
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Require successful test shards
- name: Require successful change classification
env:
CHANGES_RESULT: ${{ needs.changes.result }}
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
SHARD_RESULT: ${{ needs.test-shard.result }}
run: |
test "${CHANGES_RESULT}" = "success"
test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
run: test "${CHANGES_RESULT}" = "success"
- uses: actions/checkout@v4
if: needs.changes.outputs.code == 'true' && success()
- uses: actions/setup-node@v4
@@ -227,6 +226,7 @@ jobs:
pattern: vitest-report-*
merge-multiple: true
- name: Merge coverage, JUnit, and timing reports
id: merge
if: needs.changes.outputs.code == 'true' && success()
run: >-
npm run test:ci:merge --
@@ -240,27 +240,13 @@ jobs:
--outputFile.junit=test-results.xml
--outputFile.json=test-results.json
- name: Summarize slowest test files
if: needs.changes.outputs.code == 'true' && success()
shell: bash
run: |
node <<'NODE' >> "${GITHUB_STEP_SUMMARY}"
const report = JSON.parse(require("node:fs").readFileSync("test-results.json", "utf8"));
const files = (report.testResults ?? [])
.map((result) => ({
name: result.name,
duration: Math.max(0, (result.endTime ?? result.startTime) - result.startTime),
}))
.sort((left, right) => right.duration - left.duration)
.slice(0, 20);
console.log("## Slowest deterministic test files\n");
console.log("| File | Duration (ms) |");
console.log("| --- | ---: |");
for (const file of files) {
console.log(`| \`${file.name}\` | ${file.duration} |`);
}
NODE
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != ''
env:
SHARD_RESULT: ${{ needs.test-shard.result }}
MERGE_RESULT: ${{ steps.merge.outcome }}
run: node scripts/ci/test-report.mjs
- uses: actions/upload-artifact@v4
if: needs.changes.outputs.code == 'true' && always()
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != ''
with:
name: test-results
path: |
@@ -269,6 +255,21 @@ jobs:
coverage/
if-no-files-found: error
retention-days: 30
- name: Publish successful timing history
if: needs.changes.outputs.code == 'true' && success() && needs.test-shard.result == 'success'
uses: actions/upload-artifact@v4
with:
name: test-timings
path: .cache/ci-test-history/test-results.json
include-hidden-files: true
if-no-files-found: error
retention-days: 30
- name: Require successful test shards
if: ${{ !cancelled() }}
env:
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
SHARD_RESULT: ${{ needs.test-shard.result }}
run: test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
conformance-fixtures:
name: Source-built conformance fixtures
@@ -503,6 +504,7 @@ jobs:
tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts
tests/conformance/providers/ghidra/windowsCapabilities.test.ts
src/ghidra/GhidraInstallation.test.ts
tests/boundary/filesystem/ghidraJavaProbe.test.ts
tests/boundary/providers/ghidra/ghidraTransport.test.ts
tests/boundary/providers/ghidra/ghidraTargetSnapshot.test.ts
tests/boundary/providers/ghidra/ghidraLauncher.test.ts
+27
View File
@@ -4,6 +4,8 @@
REA is a local-only reverse-engineering tool with shared CLI/MCP workflows. Distinguish observations, derivations, inferences, and unknowns; preserve Evidence provenance and session semantics through both adapters. Provider support and setup live in [README.md](README.md#choosing-a-deep-analysis-provider) and its linked guides.
Follow the [contribution scope and fail-closed integration rule](CONTRIBUTING.md#contribution-scope). Do not integrate capabilities while material scope, authority, side-effect, or required verification questions remain unresolved. For new providers or capabilities involving target execution, injection, mutation, or protection recovery, seek the required maintainer scope decision before substantial implementation for integration. Welcome early proposals, small exploratory prototypes, and draft PRs that help resolve questions; they do not authorize integration.
Ghidra uses ephemeral databases without modifying executable bytes or controlling a GUI; Windows Ghidra P0 has no mutation authority. IDA analysis is read-only; never save or close an attached GUI database.
Configuration changes must be additive and idempotent, with backups. Installers must not install or upgrade unrelated software, including Homebrew, Node.js, npm, Java, or Ghidra. Ghidra and IDA are bring-your-own. `rea setup` must show its planned changes and require approval before writing files or installing Hopper.
@@ -26,6 +28,31 @@ Parse unknown boundary values and model expected failures with the tagged error
Validate advertised JSON Schemas after SDK conversion against their declared dialect and actual clients. Bind handlers to named contracts with exact types; catalog ordering must never select an operation or schema.
## Performance
Design analysis workflows and agent interfaces for efficiency from the first
implementation. Correctness, useful answers, Evidence provenance, caller choices,
compatibility and safe resource bounds remain required.
Optimize in this order:
1. Agent tokens and context. Return the smallest sufficient answer with
necessary provenance, coverage and limitations. Use task selection,
summaries, paging and retained Evidence where supported; honor requested
detail and keep complete Evidence accessible. Count the whole task,
including schemas, extra calls and retries.
2. Analysis time. Reuse validated results and shared work. Cache keys must
cover result-affecting inputs; define invalidation and lifetime. Validate
freshness before reporting cached live state as current.
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
retained state and serialization growth.
For changes affecting these costs, measure representative completed tasks and
compare equivalent workflows before and after where available. Report token
usage, elapsed time, CPU time and peak process-family memory. Separate cold
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
do not establish fewer tokens or faster analysis.
## Development
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
+104
View File
@@ -17,8 +17,108 @@ targets; optional adapter failures must preserve successful peers. Run
`npm run verify:test-discovery` after adding or moving tests. Resolved import
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
For analysis workflows and agent interfaces, follow the
[performance priorities](AGENTS.md#performance). Explain the selected output
scope, reuse/cache decisions, and relevant before/after measurements when these
costs change.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
## Contribution scope
REA focuses on software structure, runtime behavior, and evidence through
general-purpose reverse-engineering workflows. Evaluate a proposal by its actual
capabilities, side effects, defaults, and complete workflow. A tool name or a
reference to vulnerability research, malware analysis, or CTFs does not determine
whether it belongs in REA.
Supported contributions include format inspection, ordinary archive and standard
compression extraction, disassembly, decompilation, cross-references, general
static deobfuscation, vulnerability analysis, and evidence reporting. Static
protection detection is distinct from defeating that protection. Security fixes
and isolated regression tests remain in scope.
Runtime investigation needs explicit boundaries. Launching or attaching to a
process, injecting instrumentation, running scripts, and changing memory or
registers require review of target selection, execution and mutation authority,
provenance, resource limits, cancellation, and cleanup. General Frida or debugger
support is not inherently read-only, and existing support does not authorize
every workflow built on it. General-purpose interfaces are not classified as
dedicated bypass workflows solely because caller-provided code could be misused;
their execution and mutation authority must still be reviewed.
REA does not integrate or officially distribute:
- Purpose-built workflows to defeat commercial packing or code-virtualization
protections and automatically recover protected code.
- Anti-debugging or anti-instrumentation evasion, or DRM, license, anti-cheat, or
access-control bypass workflows.
- Exploit or attack-payload automation, credential theft, covert persistence,
unauthorized access, or bulk attack workflows.
These boundaries apply to first-party providers, adapters, plugins, bundled
scripts, defaults, examples, and documentation. Moving a capability into an
optional dependency or plugin does not exempt it. Review what the integration
exposes, including upstream defaults and raw-command or script execution paths.
Ordinary extraction and general static deobfuscation are not excluded merely
because the input is packed or obfuscated; assess added bypass or execution
behavior separately.
This is a product and maintenance boundary, not a judgment about a contributor's
intentions or a declaration that all excluded research is unlawful. Contributors
must have the rights and authorization needed for their work. A local-only design
or an "authorized use only" disclaimer does not establish containment or settle
legal and scope questions.
### Fail-closed integration
If scope fit, target or authorization boundaries, side effects, or the verification
needed to establish those boundaries remain materially unresolved, **do not merge,
enable, or include the capability in official packages or distributions**. Keep
the proposal pending, narrow it, or decline it. Record unknowns explicitly; do not
turn them into "none" or assumed safety. Passing CI, completed implementation,
upstream popularity, or silence from maintainers is not scope approval.
For a new provider or a capability involving target execution, injection,
mutation, or protection recovery, seek an explicit maintainer scope decision in
the linked issue or design discussion before substantial implementation intended
for integration. Small exploratory prototypes and draft PRs may help inform that
decision; they do not authorize integration. The decision must cover the final
proposed behavior; a scope expansion needs a new decision. Routine fixes and
documentation changes within existing boundaries do not need a separate advance
decision, but must not silently expand those boundaries.
### Starting a proposal
To open a feature request or design discussion, describe the problem and desired
outcome or decision. Short answers and the forms' "Not sure yet" options are
welcome. You do not need code, a complete design or test plan, legal analysis, or
private proof-of-authorization documents to start a discussion. Share what you
know and leave optional details blank when you need help.
Form selections guide triage; they are not proof of authorization or scope
approval. Unselected effects must not be treated as established absence.
Maintainers and implementers work together to resolve the following before
integration:
- The concrete analyst question, requested inputs and outputs, and why existing
REA operations are insufficient.
- Why the workflow fits this scope, its non-goals, and any protection recovery,
bypass, or attack automation it would expose.
- The selected targets and every execution, injection, mutation, network, or
persistence effect, including behavior inherited from upstream tools.
- How the boundaries and outcome will be verified with self-owned or appropriately
licensed samples; identify real CLI/MCP and provider coverage, cleanup and
resource checks, and remaining unknowns. Do not publish private binaries,
credentials, or sensitive target data; follow [SECURITY.md](SECURITY.md) for
private vulnerability reports.
Mixed proposals should separate acceptable analysis capabilities from excluded
workflows so each can be decided independently. An issue may discuss unresolved
questions; opening an issue or PR does not authorize integration. This is a
maintainer integration gate, not a per-call confirmation requirement or a sandbox
guarantee.
## Development setup
Use the development toolchain pinned in `.nvmrc` and
@@ -87,6 +187,10 @@ CI diagnosis. Packaging/setup/distribution changes additionally require
`npm run verify:package` and `npm pack --dry-run`; provider changes require the
matching real-provider lane. State which host/provider workflows ran.
`check:pr` reuses build and static-check outputs while executing the test suite
fresh, including the fast Python producer checks. `docs:check` also runs the
onboarding example consistency suite for README-only changes.
Pre-commit formats and lints staged files; pre-push runs `check:fast`.
### Generated files and command ownership
+1 -1
View File
@@ -179,7 +179,7 @@ using your user permissions; each runtime guide describes its effects.
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md), and
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), and
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
+5
View File
@@ -148,6 +148,11 @@ existing analysis families directly, for example
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Bugreports collect into isolated staging and publish with exclusive creation;
filename collisions select the next numeric suffix. If publication fails, the
completed archive remains in staging. The error's partial observation gives
its recoverable `local_path`, the original `adb_reported_path`, and an unknown
digest status. Later operations and shutdown preserve this retained evidence.
- Failed package acquisition rollback retains its directory identity and retries
cleanup before later work and during shutdown. Cleanup refuses a replacement
directory. Terminal shutdown stops admission and drains active operations.
+9
View File
@@ -145,6 +145,15 @@ REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
and home expansion before resolving that profile. Setup previews the resolved
configuration and skill destinations; doctor and uninstall use those same paths.
## Qoder CLI
For [Qoder](https://docs.qoder.com/cli/mcp-reference), setup writes
`mcpServers.rea` into the user-level `~/.qoder/settings.json` (the
`QODER_CONFIG_DIR` override is honored). Entries carry `command`, `args`, and
`env`; stdio is the default transport. Qoder CLI discovers personal skills
from `~/.qoder/skills` (following `QODER_CONFIG_DIR` when set), so setup
installs its skill copy there.
## Trae
For [Trae](https://docs.trae.ai/ide/add-mcp-servers), setup writes
+4 -2
View File
@@ -123,8 +123,10 @@ Select a verified native child for the existing tools:
```
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. This milestone does
not add MIPS/PPC/RISC-V decompilation, raw flash load profiles, bootloader/device
Native architecture and executable validation still apply. Generic
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
documents with their own admission boundaries. This milestone does
not add PPC decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
## Resource and lifecycle policy
+105
View File
@@ -0,0 +1,105 @@
# RISC-V ELF analysis through Ghidra
This lane extends native ELF analysis for `EM_RISCV` targets. It does not
execute RISC-V code, emulate a boot sequence, or rehost firmware. It addresses
[#2063](https://github.com/morluto/rea/issues/2063) and follows the same
lane shape as [generic MIPS support](ghidra-mips.md).
## Admission boundary
ELF `EM_RISCV` (243) is retained as the provider-neutral `riscv` family. The
target also retains the raw ELF declarations that decide a lane: class, byte
order, `e_type`, and `e_flags`. Recognition does not mean every provider or
RISC-V variant is supported. Existing PE, managed and Mach-O architecture sets
are unchanged, and caller-supplied headerless raw-binary entry still accepts only
its existing four families.
The Ghidra lane admits little-endian ELF32 and ELF64 `ET_EXEC` images that
declare `EF_RISCV_FLOAT_ABI_DOUBLE` in `e_flags`, with no `EF_RISCV_RVE`,
`EF_RISCV_TSO`, or any bit outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI`. These are
header declarations, not proof that every instruction conforms to that ISA.
Provider selection returns the specific unsupported constraint rather than
guessing a compatible profile:
| Declaration | Outcome |
| ----------------------------------------------- | --------------------------------------------------------------------------- |
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` |
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` |
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
| `EF_RISCV_RVE` | refused; the embedded 15-register ABI needs its own processor specification |
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float |
| Reserved or newer `e_flags` bits | refused with the offending bits named |
`e_type` is the only load-shape declaration this lane reads. A fixed-address
`ET_EXEC` declaration does **not** establish static linkage: `PT_INTERP` and
dynamic dependencies are not inspected, and the session records that boundary as
a limitation rather than inferring a load model.
## Why the language is pinned
Ghidra's header-based selection is not reliable for RV32. The 12.1.4
`riscv.opinion` pins `variant="default"` for the 64-bit constraint and leaves the
32-bit constraint unpinned with the comment that the loader "must choose between
'default' and 'AndeStar' variants". Measured against the installed release tree:
| Fixture | Ghidra auto-selection |
| ----------------------------------------------- | ----------------------------- |
| RV64 ELF, `e_flags 0x5` (RVC, double-float ABI) | `RISCV:LE:64:default:gcc` |
| RV32 ELF, `e_flags 0x5`, no Andes declaration | `RISCV:LE:32:AndeStar_v5:gcc` |
So an ordinary RV32 image lands on an Andes processor model unless the language
is stated. REA therefore pins `RISCV:LE:32:default` or `RISCV:LE:64:default`
with the `gcc` compiler specification per class, records
`language_selection: provider-pinned-v1`, and refuses a caller-selected
`REA_GHIDRA_LANGUAGE_ID` or `REA_GHIDRA_COMPILER_SPEC_ID` that disagrees with the
pin without changing the requested override. An override that matches the pin is
accepted and recorded as `configured-v1`.
## Evidence
The lane identity is `elf32-exec-le-double-float-generic-v1` or
`elf64-exec-le-double-float-generic-v1`, and `riscv_elf` retains `elf_class`,
`byte_order`, `type`, and the raw `flags` alongside the read-only booleans
`float_abi_double`, `rvc_declared`, `rve_declared`, and `tso_declared`. Those
fields report declarations the header makes; none of them is evidence of the
original compiler, of instruction conformance, or of a floating-point calling
convention beyond the ABI the pinned language models.
## Verification and limits
Ghidra 12.1.4 PUBLIC with a 64-bit JDK 21+ is required, and on macOS the release
tree ships no host decompiler binary, so `Decompiler/os/<platform>/decompile` and
`sleigh` must come from your own build; see
[installation](installation.md#ghidra).
Run `npm run verify:ghidra:riscv` with an existing Ghidra 12.1.4 installation,
JDK 21+, and a RISC-V-capable Clang, LLD and LLVM readelf. The verifier acquires
no tools and never executes target code. Select installed tools explicitly when
they are not on `PATH`:
```sh
GHIDRA_INSTALL_DIR=/path/to/ghidra_12.1.4_PUBLIC \
JAVA_HOME=/path/to/jdk-21 \
REA_RISCV_CLANG=/path/to/llvm/bin/clang \
REA_RISCV_LLD=/path/to/lld/bin/ld.lld \
REA_RISCV_READELF=/path/to/llvm/bin/llvm-readelf \
npm run verify:ghidra:riscv
```
The lane builds the committed freestanding C and assembly sources into RV32 and
RV64 `ET_EXEC` fixtures. Independent LLVM readelf observations establish ELF
declarations and symbol addresses. Public CLI and stdio MCP workflows verify
function enumeration, decompilation, pinned language/compiler identity, decoded
probe bytes, global data, and direct calls. It validates advertised MCP output
schemas and both JSON representations, saves and reopens snapshots, rejects
header-mutated soft-float and unsupported high-bit declarations, and checks
unchanged source bytes and owned cleanup. Failures retain the diagnostic
workspace; successful runs remove it and report the observed process lineage.
Module tests cover admission and malformed headers. Call-graph completeness,
string inventories, dynamic linkage, Andes or RVE/TSO images, and big-endian
targets are **not** established by this lane. `.riscv.attributes` ISA extensions
are not inspected; the generic language pin remains an analysis convention.
+35 -5
View File
@@ -1,6 +1,6 @@
# IDA Pro provider
# IDA provider
REA adapts an existing [mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp) server into its native analysis tools. Install and configure IDA, Hex-Rays, Python, and upstream MCP using that repository's instructions. REA does not install, upgrade, activate, or license them, and does not bundle another IDA server.
REA adapts an existing IDA MCP server into its native analysis tools. Existing IDA Pro profiles use [mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); an optional native bridge for IDA Pro and IDA Free is described below. Install and configure your selected engine and upstream MCP using their instructions. REA does not install, upgrade, activate, or license them, and does not bundle another IDA server.
Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP_CONFIG` to that file, then select `ida` once through `REA_ANALYSIS_PROVIDER`, the CLI's `--provider ida`, or MCP's `open_binary.provider_id`. No provider startup occurs during discovery.
@@ -32,7 +32,33 @@ Open the **original input binary** in REA, not an `.idb` or `.i64` database. Its
REA's close operation releases its MCP connection or proxy. It never saves or closes the existing GUI database. External edits remain visible because results are live and are not replayed from snapshots. These checks do not lock the GUI or prove that the database is unpatched.
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. `binary_session` reports these operations with `effects.may_show_ui: true` in attached mode; headless operations and the other attached operations report `false`.
## Optional native IDA MCP bridge
IDA Pro and IDA Free can use the separate [ida-free-mcp native plugin](https://github.com/0xshlomil/ida-free-mcp) through an explicit attached registration. Install and start that plugin using its upstream instructions, open the original binary in the IDA GUI, and wait for analysis. The native MCP 1.1.2 release binds `0.0.0.0` without a listen-address setting. Restrict inbound access with the host firewall or use an upstream build that binds only to loopback before starting it. REA's loopback URL restriction does not change the plugin's listener. REA does not install IDA, build the plugin, activate a license, or launch the GUI.
```json
{
"mode": "attached",
"protocol": "native",
"url": "http://127.0.0.1:13337/mcp"
}
```
Set `REA_IDA_MCP_CONFIG` to this file and select `ida` using the same CLI/MCP commands above. Omitting `protocol` preserves existing IDA Pro registrations. Native mode requires `resources/read` for `ida://idb/metadata` and its recorded input SHA-256; it never substitutes a digest of an unrelated local file. The native resource's `path` describes the IDB in the upstream protocol, not the original input path. REA retains it and checks database identity and image base before and after requests.
This experimental profile provides function/string inventory and search, function lookup, assembly, GUI pseudocode, callees, address xrefs, instruction windows, and function dossiers. Direct callers remain unavailable: native code/data xref labels do not prove call edges. It requires neither IDAPython nor idalib. Native headless registrations are rejected.
Native string observations use upstream C-string rendering; other representations remain unknown. Literal searches request conservative candidates and apply REA's original case and Unicode matching rules. Regex searches retain the full scan because native byte-oriented regex semantics cannot replace REA's Unicode regex semantics. All candidate pages are examined, and upstream search errors remain failures.
Native callee resolution batches unique queries and checks each reply's query identity. Results remain live: a recorded input hash does not establish database revision, so cross-request analysis caches cannot prove freshness. For focused questions, use pseudocode, callees, or instruction-window tools; a function dossier requests additional facets and returns more Evidence.
IDA Free's cloud decompiler needs network access and may request consent. Follow its licensing and supported architecture limits; see [IDA Free](https://hex-rays.com/ida-free). Decompilation moves GUI focus and the upstream plugin closes existing pseudocode tabs before requesting fresh output. REA advertises UI/network effects and closes only its MCP connection, preserving the open database. Rendered pseudocode has upstream traversal limits and does not prove complete source recovery. Cloud/widget failures remain explicit failures; retry after the GUI finishes or dismiss its prompt.
The native MCP 1.1.2 release can navigate in the previously focused pseudocode tab when switching functions. Its producing navigation order must focus the disassembly view **before** jumping to the requested entry. Use a build including [upstream #7](https://github.com/0xshlomil/ida-free-mcp/pull/7), merged in `6adf2c615a2a03661059673e9c136ef28d0dc778`, or a release containing that correction. REA does not repair stale GUI output. Verify two different functions before claiming reliable switching. The initial Linux verification used a local version of this upstream correction.
Native tool observations retain actual tool names and complete JSON text. Upstream `structuredContent` can be a shortened preview even while text contains the full result. REA reads the full native representation and rejects missing/malformed text instead of treating previews as complete observations.
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. Native GUI decompilation also changes the active view. `binary_session` reports these operations with `effects.may_show_ui: true`; headless operations and the other attached operations report `false`.
## Give REA a target for headless analysis
@@ -98,9 +124,9 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <cl
## Coverage and evidence
Both profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern profile does not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern and native profiles do not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. `binary_overview`, whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
@@ -110,4 +136,8 @@ Complete body ranges, typed reference edges, CFG, referenced data, source locals
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](testing.md) for the distinction between adapter tests and real-provider evidence.
To verify GUI function switching, add `--second-procedure other_function --second-marker unique_second_body_text`. The marker must occur in the second function's pseudocode and be absent from the first; the verifier also checks that returning to the first restores its text.
The initial real workflows use stdio registrations for the legacy Windows GUI profile and a Windows x64 headless supervisor with IDA 9.3. The headless workflow has also passed on macOS arm64 with IDA Professional 9.1 (build 9.1.250226) against the same upstream commit `c133c3853faa111a9b00ee615c013b720d0c4acd`, using a source-built arm64 Mach-O fixture; Mach-O C symbols carry a leading underscore, so the procedure name is `_rea_fixture_add`. Streamable HTTP is tested against a loopback MCP fixture with the pinned client SDK; a real IDA HTTP-supervisor workflow remains unverified. Linux headless engine operation, other IDA releases, architectures, and modern attached GUI tools require their corresponding real workflows before support is claimed.
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
+1
View File
@@ -280,6 +280,7 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| OMP | `omp` |
| Pi | `pi` |
| Hermes | `hermes` |
| Qoder | `qoder` |
| Trae | `trae` |
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
+2
View File
@@ -1,3 +1,5 @@
<div id="native-ui-and-dispatch-investigation"></div>
# Native binary investigation
Use a native provider to discover functions, inspect instructions and pseudocode,
+1 -1
View File
@@ -36,7 +36,7 @@ check project-owned conformance fixtures and fail-closed report behavior.
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `discover-classify` | target-free startup, artifact inventory, compatible workflow selection, and exact visible limits |
| `diagnose-environment` | broken runtime separated from analysis failure, version skew detected, and revision changed after recovery |
| `declare-request` | exact target, operation, and lifecycle are carried by the request before launch |
| `acquire-authority` | missing scope reported, narrow grant resumed, expansion reprompted, and denial launched nothing |
| `static-analysis` | native and JavaScript routing plus preserved ambiguity, truncation, unsupported syntax, and residual unknowns |
| `reactive-scenarios` | predicate-driven interaction, descendant settlement, Electron correlation, and cancellation cleanup Evidence |
| `compare-authority-candidate` | explicit partial-order/finite-trace semantics, primary divergence localization, and unknown for incomplete comparison |
+35 -12
View File
@@ -19,11 +19,12 @@ behavior, and complete CI evidence before merging.
| `npm run test:boundary` | Boundary, CLI boundary, MCP boundary, process boundary, and process-global projects |
| `npm run test:mcp` | MCP boundary project |
| `npm run test:acceptance` | Complete compiled CLI and MCP acceptance workflows |
| `npm run test:python` | LLDB producer retention and launch contracts with a fake LLDB API; no native debugger required |
| `npm run test:watch` | Dirty source tests in watch mode, without build |
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks |
| `npm run docs:check` | Generated-document validation from current source and build outputs |
| `npm run docs:check` | Generated-document validation and fresh onboarding example consistency checks |
For example:
@@ -97,6 +98,12 @@ runner. Preserve simultaneous launches where concurrency is itself under test.
Build/documentation writers hold checkout-local output locks. Tests do not
hold a broad command lock; `check:pr` finishes tests before document validation.
Turbo caches deterministic preparation and static checks, but `test:run` always
executes. Repository files do not fully describe installed tool contents or host
state. A previous test pass therefore cannot replace current validation, even
when the source tree is unchanged. `check:pr` also runs the fast Python producer
suite; CI runs it in the static job for code changes. These producer tests do not
establish real LLDB engine coverage.
Vitest's persistent compile cache remains disabled. The `cliTest` fixture shares
a temporary Node compile cache among its fresh CLI subprocesses. The worker
@@ -247,17 +254,25 @@ Coverage does not replace named boundary or real-provider scenarios.
CI runs six native Vitest shards without retries. Each shard emits a blob
report; the merge job produces aggregate coverage plus JUnit and JSON timing
reports, uploads them together, and writes the slowest files to the workflow
summary. Static checks, documentation, build, package verification, and
reports even when a shard fails, uploads available diagnostics, and writes the
slowest files to the workflow summary. Partial reports are marked diagnostic
only. Failed, cancelled, or missing selected shards still fail the required gate,
and aggregate coverage thresholds remain enforced by the merge command.
Only successful complete merges publish the small `test-timings` artifact, with
checkout-relative paths; the build job downloads that artifact for the sequencer
instead of the coverage/JUnit bundle. Missing history falls back to native
sharding. Static checks, documentation, build, package verification, and
real-system lanes remain separate jobs so one kind of evidence cannot stand in
for another.
The build job saves portable Turbo outputs across runs. Eight Linux real-system
lanes restore this cache without waiting for the build job. The shared action
derives its key from `test:prepare --dry=json` task hashes, runner OS and
architecture, and `.npmrc`; test-only changes can reuse compiled outputs. Every
architecture, `.npmrc`, `.nvmrc`, and the lockfile; test-only changes can reuse compiled outputs. Every
build and verification command still runs, so Turbo checks task inputs and
rebuilds misses. There is no prefix fallback to accumulate older build outputs.
rebuilds misses. A restore prefix with the same platform and toolchain permits
partial task reuse when one generator's inputs change; Turbo still invalidates
that generator's entry. Native outputs are not stored in this cache.
Native package jobs, the Windows curated lane, and the Apple artifact lane reuse
the same workflow's Turbo cache for portable compiled JavaScript, skills, and
@@ -266,9 +281,10 @@ commands, which restore matching outputs or rebuild on a cache miss. Native
artifacts, dependency installs, package archives, and runtime verification remain
host-local. Keep platform-dependent outputs out of this shared cache.
The PR acceptance target is a median `npm run check:pr` wall time below three
minutes across three warm-build runs on the benchmark host. Keep Vitest caches
cold unless separately identified. A PR that touches packaging or real-system
Record cold and warm preparation separately from fresh test execution when
benchmarking the optional `npm run check:pr` gate. Compare multiple runs on the
same host, including subprocess and queue costs; a cached pass is not a test
runtime measurement. A PR that touches packaging or real-system
behavior requires the applicable `verify:*` lanes; packaging and installation
changes also require `npm run verify:package`. The full-gate benchmark measures
that explicit lane, not the routine iteration requirement.
@@ -292,6 +308,7 @@ prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:seeds` | POSIX; raw x86 fixture; seed ordering, BSS labels, analyzer readback and unknown-name rejection | Ghidra 12.1.4 and full JDK 21 |
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:riscv` | Linux/macOS x64/arm64 host; source-owned RV32/RV64 ELF through CLI and MCP | RISC-V Clang, LLD, LLVM readelf, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
@@ -600,11 +617,12 @@ or upgrade Java, Ghidra, .NET or native toolchains. See
`npm run verify:ida -- --target /absolute/path/to/program --procedure main`
uses the existing `REA_IDA_MCP_CONFIG` registration. It installs no engine,
Python package, or compiler. The target must already be open in the GUI for
the legacy attached profile; the database-supervisor headless profile opens
the legacy or native attached profile; the database-supervisor headless profile opens
a digest-verified private copy. A caller-supplied fixture keeps prerequisites
limited to the selected engine and host. `tests/conformance/ida/inventory.c`
provides an optional small native fixture source with an exported
`rea_fixture_add` function.
provides an optional small native fixture source with exported
`rea_fixture_add` and `rea_fixture_subtract` functions. For their switching check,
use `--second-procedure rea_fixture_subtract --second-marker='- 13'`.
The lane invokes the production CLI dispatcher and connects the pinned MCP
client SDK to the production REA server. It verifies function Evidence and
@@ -632,6 +650,10 @@ failed intermittently on that host. Linux headless, modern attached GUI tools,
other engine versions and architectures remain unverified; see the
[provider guide](ida-provider.md).
Native IDA MCP verification uses `"protocol": "native"` with an already-running native plugin. Add `--second-procedure other_function --second-marker unique_second_body_text` to exercise GUI function switching and a return to the first function. The literal marker must appear in the second function's pseudocode and be absent from the first. Native cloud/widget failures are reported without hidden retries.
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
## DOS Ghidra analysis
`npm run verify:ghidra:dos` requires the supported Ghidra and JDK installation
@@ -682,7 +704,8 @@ not make native host acceptance unavailable.
Pull requests that change only root `README*.md` files, `docs/`, `AGENTS.md`,
`CONTRIBUTING.md`, `CHANGELOG.md`, `SECURITY.md`, `native/windows/README.md`,
or the GitHub issue and pull-request templates run formatting and
generated-document validation without the source-test shards or native package
generated-document validation, including the small onboarding example consistency
suite, without the source-test shards or native package
lanes. Classification compares the PR head with its merge base, so later
base-branch changes do not expand that scope. Static and coverage aggregate jobs
remain present and fail if classification fails. Every code change retains the
+6 -4
View File
@@ -48,10 +48,12 @@ satisfy. A plan without execution does not establish runtime behavior.
- Use strict inputs with explicit target identity and required fields. Preserve
meaningful target, action, capture, and output choices; remove ignored options
and repeated permission declarations.
- Return complete evidence inline, with artifact/provider identity, source
locations, Evidence references, and relevant confidence and limitations.
Missing coverage is unknown, not absence. Avoid extra lookups solely to
understand the answer or its provenance.
- Return the requested facts inline with artifact/provider identity, source
locations, Evidence references, coverage, and relevant confidence and
limitations. Use supported selection, summaries or views when they answer
the question; preserve access to complete source Evidence and honor existing
contracts and requested detail. Missing coverage is unknown, not absence.
Avoid extra lookups solely to understand the answer or its provenance.
- Keep observed, derived, and inferred relationships distinct. Preserve
unresolved edges and partial provider facts instead of requiring every
provider to supply the same metadata.
@@ -7,6 +7,10 @@ JSON measurements belong outside the checkout. This document keeps the measured
findings, verification scope and revision references; the scripts and `corpus.json`
keep the reproduction inputs. Generated `results/` is ignored.
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
records the post-merge native continuation, including a small-function case where
four selected replies cost more bytes than the complete dossier.
The 12 detailed checkpoint receipts were archived byte-for-byte before removal
from the final PR tree. The archive `review-receipts-00d51e0fc-20261012` is in the
private experiment lab outside the checkout; its `archive-index.json` records
+15 -1
View File
@@ -30,6 +30,7 @@ const sameSettings = [
"provider_search_path_sha256",
"provider_environment_policy_sha256",
"measure_source_sha256",
"native_workflow_source_sha256",
"package_lock_sha256",
"node_version",
"sdk_client_version",
@@ -40,7 +41,9 @@ const sameSettings = [
const settingDifferences = sameSettings.filter(
(key) =>
!isDeepStrictEqual(left[key], right[key]) ||
(key === "provider_search_path_sha256" &&
((key === "provider_search_path_sha256" ||
(key === "native_workflow_source_sha256" &&
typeof left.native_question === "string")) &&
!/^[a-f0-9]{64}$/.test(left[key] ?? "")),
);
// Producer/view IDs identify different delivery observations; a named recovery
@@ -54,6 +57,17 @@ const facts = (answer) =>
const a = facts(leftAnswer);
const b = facts(rightAnswer);
const runSpecific = [];
for (const [side, answer] of [
["left", leftAnswer],
["right", rightAnswer],
]) {
if (Object.hasOwn(answer, "evidence_ids"))
runSpecific.push({
side,
path: ["evidence_ids"],
value: answer.evidence_ids,
});
}
// Binwalk assigns a new opaque region UUID in each invocation. Preserve both
// original lookup values in this report; compare the observed region facts
// separately. The product and original replies retain those IDs unchanged.
@@ -116,6 +116,36 @@
"derivation": "selected regular ZIP member ShareX.HelpersLib.dll; no target execution"
}
},
{
"id": "sharex.exe",
"file": "ShareX.exe",
"bytes": 198656,
"sha256": "46af1ccc514b395cb703633abb5a505a771942b7db9b1915f31aeea541b3685f",
"provenance": {
"project": "ShareX native executable component",
"version": "19.0.2",
"upstreamFile": "ShareX-19.0.2-portable.zip",
"url": "https://github.com/ShareX/ShareX/releases/download/v19.0.2/ShareX-19.0.2-portable.zip",
"transport_sha256": "967b5d838df5ce25fb5fd7cbd16f1bd797427e2df16907b022b74a4bc0fb9d54",
"transport_bytes": 168796487,
"derivation": "selected regular ZIP member ShareX.exe; PE32+ x86-64 with no CLR directory; no target execution"
}
},
{
"id": "freedos.edlin.exe",
"file": "EDLIN.EXE",
"bytes": 31492,
"sha256": "96b2b6d543e2d8bdc83e313756444e103339eb5ee2c1229abe3607539b31ad6e",
"provenance": {
"project": "FreeDOS EDLIN",
"version": "2.24",
"upstreamFile": "edlin.zip",
"url": "https://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/repositories/1.4/base/edlin.zip",
"transport_sha256": "9db8a0b50bc4be5c2358cd0fa1fb6fc77342c6d5175a6b1084547c370e428f8a",
"transport_bytes": 377009,
"derivation": "selected regular ZIP member BIN/EDLIN.EXE; original UPX-compressed DOS MZ; APPINFO/EDLIN.LSM reports version; no execution or unpacking"
}
},
{
"id": "vscode.tree-sitter.wasm",
"file": "tree-sitter.wasm",
@@ -0,0 +1,97 @@
# Native IDA MCP verification on Windows
Verification date: 2026-10-13. These runs used **IDA Pro 9.3** (executable
product version `9.3.26.0213`), not Windows IDA Free. Personal paths, raw
observations, and license registration information are excluded.
The native plugin was built from upstream
[`c9d15d98e8a143fc74f6e14037c48f63889b6d6c`](https://github.com/0xshlomil/ida-free-mcp/commit/c9d15d98e8a143fc74f6e14037c48f63889b6d6c)
with MinGW GCC 13 and its pinned public SDK. The only local source change
bound the listener to `127.0.0.1`; the upstream function-navigation fix was
already present. This does not verify the stock 1.1.2 release listener or
its older navigation order.
## Correctness and authority
The public [IDA fixture](../../tests/conformance/ida/inventory.c) contains
two exported functions with distinct bodies. Production CLI/MCP verification
checks matching function Evidence, assembly, a unique second-function marker,
return to the first function, original-input identity, invalid requests, and
connection cleanup. It preserves the attached GUI database.
The real application was a digest-verified, unchanged copy of Windows
`whoami.exe`: 94,208 bytes, 243 recognized functions, and 314 upstream
C-string observations. The selected 1,741-byte function had 20 resolved
callees. Before and after, the task returned identical pseudocode, callee
addresses, and its one `whoami` string match. The executable was never run.
Other string representations remain unknown in the native profile.
A Windows C++ producer golden checked 24 literal/case combinations and
408 candidate comparisons, including metacharacters, Kelvin sign, dotted I,
sigma, supplementary characters, and null bytes. Adapter tests additionally
cover malformed UTF-16 fallback, candidate pagination, explicit search errors,
and reordered, missing, duplicate, unrelated, or failed lookup replies.
These checks do not establish IDA recognition of every Unicode string format.
## Equivalent completed tasks
Both builds used main `59be86671`. The baseline was `fb34ccbec`, before
the native performance edits; the comparison build added those edits.
Schemas were identical. Each run opened the same analyzed binary, read the
specified function's pseudocode and callees, searched literal `whoami`, and
closed the REA session. The task then repeated on the same MCP connection.
There were two runs per build on a shared host. Node 24.18.0 and IDA were
restricted to one logical CPU with below-normal priority; each Node process
had a 768 MiB heap ceiling. The attached database was already analyzed.
Node compile caching was disabled for the primary comparison.
| Measurement | Before | After |
| --------------------------------------------------- | -----------: | -----------: |
| Task response text | 49,139 bytes | 32,546 bytes |
| Task response token estimate | 16,618 | 11,509 |
| Callee lookup calls, including selector | 17 | 2 |
| String-search pages | 4 | 1 |
| String-search Evidence | 19,022 bytes | 3,344 bytes |
| Mean task time, warm MCP connection | 675 ms | 317 ms |
| Mean task time, fresh MCP process and catalog | 19.72 s | 20.86 s |
| Mean Node family CPU, fresh and warm tasks together | 17.72 s | 18.62 s |
| Largest Node family peak-RSS upper bound | 776.8 MiB | 780.0 MiB |
| Mean attached IDA CPU during both tasks | 226.6 ms | 171.9 ms |
The warm task was about 53% faster and its response estimate about 31%
smaller. Fresh-start, overall CPU, and memory improvements were not
established. Startup and full catalog conversion dominate the fresh task.
Two samples do not establish a general performance guarantee.
Token counts use local `cl100k_base` tokenization of complete serialized
MCP response text and request arguments; they are estimates, not model usage.
Requests added 99 estimated tokens per task. The unchanged full schema
catalog contained 2,422,229 bytes and 613,965 estimated tokens. Including it
once gives first-task estimates of 630,682 before and 625,573 after. A host
may select schemas lazily; actual host injection, prompts, reasoning tokens,
and model-side costs were not measured. No retries were needed.
Node CPU counters were captured at shutdown. The memory figure sums the
two processes' individual OS peak RSS values: an upper bound, not a
simultaneous family peak. Attached IDA is outside that process family; its
unchanged lifetime peak working set was 376.9 MiB, including initial analysis.
Measurement overhead and whole-machine resource use were not measured.
## Reuse and cleanup
Callee queries are deduplicated and batched within an operation. Literal
search fetches a conservative candidate set, including non-ASCII text when
needed, and preserves REA's final Unicode predicate. It examines all candidate
pages and retains full source observations. Regex searches keep the full scan.
Cross-request analysis results remain live because an input digest does not
establish database revision or freshness. An exploratory caller-selected
[Node compile cache](https://nodejs.org/download/release/v24.16.0/docs/api/module.html#module-compile-cache)
did not demonstrate startup improvement: empty-cache and cache-hit MCP starts
were 7.35 s and 7.56 s. It occupied 6.7 MiB and was limited to this experiment;
no new default cache or installation behavior was introduced.
Temporary source, SDK, build, portable runtime, cache, and raw measurement
outputs are test-owned cleanup targets. Attached GUI databases are retained;
REA never saves or closes them.
+32 -116
View File
@@ -1,16 +1,14 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import {
mkdir,
open,
readFile,
readdir,
stat,
writeFile,
} from "node:fs/promises";
import { mkdir, readFile, readdir, stat, writeFile } from "node:fs/promises";
import { dirname, isAbsolute, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import {
NATIVE_ARTIFACTS,
measureNativeEntry,
nativeEntrySeed,
} from "./native-entry.mjs";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { Ajv2020 } from "ajv/dist/2020.js";
@@ -38,10 +36,15 @@ if (
!isAbsolute(values.path) ||
!isAbsolute(values.output) ||
!["complete", "summary"].includes(values.delivery) ||
!["entry", "entry-and-main"].includes(values["native-question"]) ||
!["entry", "entry-and-main", "entry-views"].includes(
values["native-question"],
) ||
!["inventory", "selected"].includes(values["javascript-question"]) ||
!["default", "selected"].includes(values.schemas) ||
(values.artifact !== "jq" && values["native-question"] !== "entry") ||
(!NATIVE_ARTIFACTS.includes(values.artifact) &&
values["native-question"] !== "entry") ||
(values.artifact !== "jq" &&
values["native-question"] === "entry-and-main") ||
(values.artifact !== "obsidian.asar" &&
values["javascript-question"] !== "inventory")
)
@@ -118,6 +121,9 @@ await writeFile(
);
const fingerprints = {
measure_source_sha256: await fileSha(fileURLToPath(import.meta.url)),
native_workflow_source_sha256: await fileSha(
fileURLToPath(new URL("./native-entry.mjs", import.meta.url)),
),
runtime_files_sha256: sha(JSON.stringify(runtimeFiles)),
runtime_dispatcher_sha256: await fileSha(
resolve(values["runtime-root"], "scripts/rea.mjs"),
@@ -545,111 +551,19 @@ const workflows = {
evidence_ids: [info.evidence_id],
};
},
jq: async () => {
const file = await open(values.path, "r");
let header;
try {
header = (await file.read(Buffer.alloc(32), 0, 32, 0)).buffer;
} finally {
await file.close();
}
if (header.subarray(0, 6).toString("hex") !== "7f454c460201")
throw new Error("Expected ELF64 little endian entry seed");
const address = `0x${header.readBigUInt64LE(24).toString(16)}`;
await call("open_binary", { path: values.path, provider_id: "ghidra" });
try {
const containing = await call("resolve_containing_procedure", {
address,
});
const info = await call("analyze_function", { procedure: address });
const n = normalized(info);
// This exact application entry is materialized in the pinned jq entry's
// observed MOV RDI, 0x401f00 instruction, not an original-source claim.
let main;
const mainEvidenceIds = [];
if (values["native-question"] === "entry-and-main") {
let sourceId;
try {
const completed = await call("analyze_function", {
procedure: "0x401f00",
});
sourceId = completed.evidence_id;
} catch (error) {
const reference =
error instanceof ToolFailure &&
error.value?.error?.code === "resource_constraint"
? error.value.error.details?.reported_limits?.evidence_reference
: undefined;
if (reference?.kind !== "retained-evidence") throw error;
sourceId = reference.evidence_id;
}
mainEvidenceIds.push(sourceId);
main = {};
for (const facet of [
"procedure",
"pseudocode",
"referenced_strings",
"callees",
"callers",
"basic_blocks",
]) {
let offset = 0;
const items = [];
const pages = [];
do {
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: sourceId },
view: {
kind: "native",
facet,
offset,
limit:
facet === "pseudocode"
? 65536
: facet === "basic_blocks"
? 1
: 64,
},
});
mainEvidenceIds.push(selected.evidence_id);
const v = normalized(selected);
items.push(v.item);
pages.push({
coverage: v.coverage,
limitations: v.limitations,
unknowns: v.unknowns,
});
if (v.coverage.exhausted || facet === "basic_blocks") break;
if (
v.coverage.next_offset === null ||
v.coverage.next_offset <= offset
)
throw new Error("Native view did not advance");
offset = v.coverage.next_offset;
} while (true);
main[facet] = { items, pages };
}
}
return {
artifact_sha256: targetSha(info),
entry_address: address,
containing_procedure: normalized(containing),
procedure: n.procedure,
pseudocode_prefix: n.pseudocode.slice(0, 512),
caller_count: n.callers.length,
callee_count: n.callees.length,
...(main === undefined ? {} : { main }),
analysis_profile: info.analysis_profile,
evidence_ids: [
containing.evidence_id,
info.evidence_id,
...mainEvidenceIds,
],
};
} finally {
await call("close_binary", {});
}
},
...Object.fromEntries(
NATIVE_ARTIFACTS.map((artifact) => [
artifact,
async () =>
measureNativeEntry({
seed: await nativeEntrySeed(artifact, values.path),
values,
call,
inputSha,
ToolFailure,
}),
]),
),
};
try {
@@ -740,7 +654,9 @@ const report = {
artifact_sha256_after: finalInputSha,
delivery: values.delivery,
sdk_schema_mode: values.schemas,
native_question: values.artifact === "jq" ? values["native-question"] : null,
native_question: NATIVE_ARTIFACTS.includes(values.artifact)
? values["native-question"]
: null,
javascript_question:
values.artifact === "obsidian.asar" ? values["javascript-question"] : null,
provider_settings_sha256: sha(JSON.stringify(selectedProviders)),
@@ -0,0 +1,178 @@
# Native PE and DOS entry-view checkpoint
Continuation of [#2016](https://github.com/morluto/rea/issues/2016) after
#2122 and #2193 merged. Real Ghidra observations and real Codex CLI answers
support the selected native facets on these shipped inputs. They do not justify
an automatic native summary policy: the four selected replies are larger than
the complete dossier for the small PE entry, and smaller for the DOS entry.
The initial complete analysis still runs and is delivered in both cases.
This checkpoint changes experiment scripts, corpus inputs and grading only.
Existing production CLI/MCP contracts, defaults and session behavior are unchanged.
## Inputs and runtime
Production runtime: main `10400fdb57e4226c5ce15af17968a664f35498ce`,
including the merged WASM summary work and subsequent main changes. Node
24.18.0/npm 11.16.0, locked dependencies, actual Ghidra 12.1.4 on Linux.
Default auto-analysis remains enabled, with a 768 MiB JVM ceiling and one
active processor. The whole process family uses the existing one-CPU,
4 GiB hard/3 GiB early-stop, no-swap, bounded-task resource scope.
| Shipped artifact | Selected bytes | SHA-256 | Static scope |
| ---------------------------------- | -------------: | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- |
| ShareX 19.0.2 `ShareX.exe` | 198,656 | `46af1ccc514b395cb703633abb5a505a771942b7db9b1915f31aeea541b3685f` | Native PE32+ x86-64 component with no CLR directory; entry `0x1400125c0` derived from original image base/RVA |
| FreeDOS EDLIN 2.24 `BIN/EDLIN.EXE` | 31,492 | `96b2b6d543e2d8bdc83e313756444e103339eb5ee2c1229abe3607539b31ad6e` | Original UPX-compressed DOS MZ; admitted entry `0x10000`, independently verified static load image |
Acquisition URLs, archive sizes/digests and exact regular ZIP members are in
[corpus.json](corpus.json). EDLIN's package `APPINFO/EDLIN.LSM` reports its
version and UPX compression. No binary was executed, unpacked or committed.
The independent PE header does not establish Windows runtime behavior.
DOS load-image verification and unpacking-stub pseudocode do not recover the
editor's application logic or establish DOS execution. DOS COM is still untested.
The SDK and frozen model runtime share the exact `dist/` and `bridge/` file
table digest:
`e2b67eb0966072fd8b159edfab074cbaf1db9bfc0ac9c57d25ba1435cc0fdbcf`.
The model copy was independently checked against all 1,375 compiled files and
30 bridge files. Runtime copies and detailed receipts remain outside Git.
## Public SDK observations
The workflow opens the selected target with Ghidra, resolves the entry,
calls `analyze_function` once, reads retained procedure/pseudocode/callers/callees
facets, and closes the ephemeral session. DOS obtains its address from
`inspect_native_load_image` before resolving the procedure. Each facet is checked
against the complete producer facts, original parent ID, target digest and distinct
derived Evidence ID. The actual SDK validates named advertised output contracts.
| Entry | Complete dossier JSON bytes | Four facet replies combined | Workflow seconds | First provider query seconds | Last-observed family CPU seconds | Sampled family peak MiB |
| ------------ | --------------------------: | --------------------------: | ---------------: | ---------------------------: | -------------------------------: | ----------------------: |
| ShareX PE | 50,936 | 68,257 | 63.123 | 57.490 | 63.660 | 1,197.85 |
| EDLIN DOS MZ | 224,670 | 70,539 | 26.436 | 21.366 | 26.899 | 939.25 |
These are exploratory observations, not a paired implementation performance
cohort. JSON byte counts include the SDK tool-result representations, excluding
catalog discovery and wire framing; they are not model token counts. Four PE
facets cost about 34% more bytes than that dossier. The DOS facets cost about
69% fewer bytes **when reading already-retained Evidence**. Their first analysis
still returned the full dossier; summing both paths increases first-task traffic.
The first provider query includes the observable wait for the imported Program
and its analysis, not an independently measured engine-only stage.
The observed PE entry contains 18 analyzed bytes, zero recorded callers and two
resolved callees. Ghidra's terminal-call caveat remains explicit. The DOS entry
contains 53 analyzed bytes and zero recorded callers/callees; overlapping
instruction and abnormal stack-frame warnings remain in the pseudocode.
Neither zero count establishes absence of unresolved runtime calls.
An earlier PE probe completed production analysis and two facets, then failed
its harness assertion: a pseudocode page is `{text, unit}`, not a string. That
failed attempt remains archived and is excluded from successful workflow figures.
The corrected exploratory SDK driver fingerprints are
`4876c0ecaa84069ddc508e29649715c6c914bd3c4960a76fa9724b5992123699`
and native-helper `76023ad45cd1edab57dcd4a8ccf29a9b8a8378029ff2d9711ae35c73940e80d6`.
The published helper in `6d7f6885c` removes redundant scopes; replaying the actual
PE/DOS replies reproduces every answer field and the same 9/10 calls. Recorded
real jq entry and entry/main replies also preserve their selected facts, profile
and 4/12-call ordering. These replays are driver regression checks, not fresh
provider runs or timing measurements.
## Real Codex CLI observations
Three serial runs used Codex CLI 0.161.0, requested alias `gpt-5.6-terra`, high
reasoning, temporary isolated profiles and the frozen modified REA MCP runtime.
Resolved server-side model identity is not inferred. The same explicit seven-tool
subset was enabled: open, containing-procedure resolution, dossier, analysis view,
load image, retained Evidence discovery and close. This is not full-catalog tool
selection verification. Questions are [PE](questions/native-pe.txt) and
[DOS](questions/native-dos.txt).
| Trial | Total input | Cached input | Uncached input | Output | Seconds | Calls / failed |
| ----- | ----------: | -----------: | -------------: | -----: | ------: | -------------: |
| PE 1 | 451,337 | 399,616 | 51,721 | 3,820 | 130.073 | 12 / 2 |
| DOS 1 | 730,578 | 658,944 | 71,634 | 3,310 | 134.435 | 9 / 0 |
| PE 2 | 498,322 | 429,056 | 69,266 | 3,038 | 192.373 | 9 / 1 |
All three answers match the selected original digest, address, procedure
classification/signature/locals, inclusive body ranges, body byte count,
pseudocode prefix and relationship counts. Actual Evidence references were
observed in the responses; target bytes remained unchanged, close completed and
the copied profiles were removed. DOS additionally matches the verified load
status and every supplied load-image metadata field. Manual review preserves
static/packed-target and terminal-flow qualifications.
The first grader expected only particular nested JSON shapes. It rejected PE's
equivalent text-page object/inclusive-range label and could not grade DOS's
explicit `entry @ 0x10000` display. The updated grader compares exact source
values for those representations without parsing displays into lookup identities.
It still rejects wrong text units, returned lengths, body endpoints and missing
facts. Original answers and first grading attempts remain archived; regrading
does not repeat model calls or change their tokens/times.
The PE agent hit the advertised-but-unimplemented native summary recipe: both
trials attempted `view: {kind: "summary"}`; the first also tried the invalid
native facet `summary`. Those failed calls and all subsequent recovery/repeated
reads remain included. The public SDK independently reproduced `invalid_request`
for native summary on both inputs. This pre-existing contract-description mismatch
is recorded in `/tmp/rea-analysis-efficiency-bugs-20261012.md` for team repair,
as requested; this branch does not repair it.
Model sampled peaks were 1,318.74 / 1,017.70 / 1,234.92 MiB, with last-observed
family CPU 72.956 / 71.534 / 74.089 seconds. SDK/model scopes did not abort and
their last-observed limit-event counters were zero. Final cgroup counters remained
unavailable after scope teardown: the strict resource-qualified overall grades
remain unverified. The selected fact/lifecycle checks pass; no universal latency,
OOM absence or CPU/memory saving is inferred from these samples.
## Reproduction and next decision
Use the pinned toolchain and build the selected production revision. Keep its
runtime copy, matching dependencies, provider settings, binaries and receipts
outside the checkout, as described in [README.md](README.md).
```sh
python3 experiments/analysis-efficiency/resource-run.py -- \
node experiments/analysis-efficiency/measure.mjs \
--runtime-root /absolute/frozen-runtime --revision EXACT_PRODUCTION_SHA \
--artifact sharex.exe --path /absolute/ShareX.exe \
--providers /absolute/explicit-providers.json \
--native-question entry-views --schemas selected \
--output /absolute/new-native-measurement
```
For DOS select `--artifact freedos.edlin.exe` and the original extracted
`EDLIN.EXE`. `--native-question entry` retains the existing compact experimental
answer; `entry-views` performs additional verification reads and the explicit
summary refusal probe. It does not select summary delivery from `analyze_function`.
The helper fingerprint participates in comparison, and missing native helper
fingerprints are not treated as a comparable current driver.
The reviewed `entry-views` answer separates selected facet facts from run-specific
references: `evidence_ids.retained_views` retains each facet's original view and
parent ID, while `retained_views` holds its item, coverage, limitations and unknowns.
DOS load-image facts retain their source ID and must match the selected carrier
digest before their entry address can be used; a stale result closes the session
without querying a procedure.
Derived view IDs are checked against every prior producer reference as well as
other derived views. DOS model grades require the actual completed load-image
response and its reported source ID; status, digest, entry and supplied metadata
are checked against that observation. Missing, failed or unreferenced responses
cannot establish load-image verification.
The comparator preserves both runs' reference maps and compares the facts. Existing
`entry`/`entry-and-main` answers keep their ID arrays. Historical SDK receipts above
retain their original pre-review representation; no replies or measurements were
rewritten, and no provider/model runs were repeated for this representation fix.
Relevant validation includes the real workflows above, recorded-reply regression,
grader negative probes, JavaScript/Python syntax, formatting/diff checks and
`npm run check:fast`. The cold type/lint gate passed in 112.602 seconds with
sampled peak 3,220,914,176 bytes and five last-observed `memory.high` events;
this gate is not a performance-qualified experiment. Unrelated domain suites,
packaging and earlier model cohorts were not rerun for this experiment-only change.
The next useful comparison is question-specific native primitives versus a
complete dossier, with identical requested facts and actual model trials.
Larger PE functions and another unpacked DOS/COM application can extend the
coverage progressively. Preserve existing complete defaults and solve the
summary-description bug separately before recommending a native summary route.
@@ -0,0 +1,272 @@
import { open, readFile } from "node:fs/promises";
import { isDeepStrictEqual } from "node:util";
export const NATIVE_ARTIFACTS = ["jq", "sharex.exe", "freedos.edlin.exe"];
const normalized = (value) => value.normalized_result;
const retainedEntryViews = async (
call,
info,
inputSha,
ToolFailure,
sourceEvidenceIds,
) => {
const n = normalized(info);
const retainedViews = [];
const retainedViewEvidence = [];
const observedEvidenceIds = new Set([info.evidence_id, ...sourceEvidenceIds]);
let nativeSummaryOutcome, summaryEvidenceId;
for (const facet of ["procedure", "pseudocode", "callers", "callees"]) {
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: info.evidence_id },
view: {
kind: "native",
facet,
offset: 0,
limit: facet === "pseudocode" ? 512 : 64,
},
});
const view = normalized(selected);
if (observedEvidenceIds.has(selected.evidence_id))
throw new Error("Retained native view reused an Evidence identity");
const expected =
facet === "procedure"
? n.procedure
: facet === "pseudocode"
? { text: n.pseudocode.slice(0, 512), unit: "utf16-code-units" }
: n[facet].slice(0, 64);
if (
!isDeepStrictEqual(view.item, expected) ||
view.parent_evidence_id !== info.evidence_id ||
view.artifact.sha256 !== inputSha
)
throw new Error(
"Retained native view changed the selected source facts or identity",
);
observedEvidenceIds.add(selected.evidence_id);
retainedViews.push({
facet,
item: view.item,
coverage: view.coverage,
limitations: view.limitations,
unknowns: view.unknowns,
});
retainedViewEvidence.push({
facet,
evidence_id: selected.evidence_id,
parent_evidence_id: view.parent_evidence_id,
});
}
// Exercise the advertised summary recipe separately. A known refusal
// is recorded as such; it does not validate a nonexistent capability.
try {
const summary = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: info.evidence_id },
view: { kind: "summary" },
});
if (observedEvidenceIds.has(summary.evidence_id))
throw new Error("Native summary reused an Evidence identity");
nativeSummaryOutcome = { status: "supported" };
summaryEvidenceId = summary.evidence_id;
} catch (error) {
if (
!(error instanceof ToolFailure) ||
error.value?.error?.code !== "invalid_request"
)
throw error;
nativeSummaryOutcome = {
status: "unsupported",
error: error.value.error,
};
}
return {
retainedViews,
retainedViewEvidence,
nativeSummaryOutcome,
summaryEvidenceId,
};
};
const retainedJqMain = async (call, ToolFailure) => {
// This exact application entry is materialized in the pinned jq entry's
// observed MOV RDI, 0x401f00 instruction, not an original-source claim.
let main;
const mainEvidenceIds = [];
let sourceId;
try {
const completed = await call("analyze_function", {
procedure: "0x401f00",
});
sourceId = completed.evidence_id;
} catch (error) {
const reference =
error instanceof ToolFailure &&
error.value?.error?.code === "resource_constraint"
? error.value.error.details?.reported_limits?.evidence_reference
: undefined;
if (reference?.kind !== "retained-evidence") throw error;
sourceId = reference.evidence_id;
}
mainEvidenceIds.push(sourceId);
main = {};
for (const facet of [
"procedure",
"pseudocode",
"referenced_strings",
"callees",
"callers",
"basic_blocks",
]) {
let offset = 0;
const items = [];
const pages = [];
do {
const selected = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: sourceId },
view: {
kind: "native",
facet,
offset,
limit:
facet === "pseudocode" ? 65536 : facet === "basic_blocks" ? 1 : 64,
},
});
mainEvidenceIds.push(selected.evidence_id);
const v = normalized(selected);
items.push(v.item);
pages.push({
coverage: v.coverage,
limitations: v.limitations,
unknowns: v.unknowns,
});
if (v.coverage.exhausted || facet === "basic_blocks") break;
if (v.coverage.next_offset === null || v.coverage.next_offset <= offset)
throw new Error("Native view did not advance");
offset = v.coverage.next_offset;
} while (true);
main[facet] = { items, pages };
}
return { main, mainEvidenceIds };
};
export const measureNativeEntry = async ({
seed,
values,
call,
inputSha,
ToolFailure,
}) => {
await call("open_binary", { path: values.path, provider_id: "ghidra" });
try {
// DOS addresses follow the actual admitted load image. A file header
// offset is not a provider address or proof of runtime execution.
let loadImage, loadImageEvidenceId;
if (seed === undefined) {
const imageEvidence = await call("inspect_native_load_image", {});
loadImage = normalized(imageEvidence);
if (loadImage.status !== "verified")
throw new Error(
"The shipped DOS load image was not independently verified",
);
if (loadImage.target_sha256 !== inputSha)
throw new Error("The DOS load image belongs to a different target");
loadImageEvidenceId = imageEvidence.evidence_id;
seed = loadImage.entry.linear_address;
}
const address = seed;
const containing = await call("resolve_containing_procedure", {
address,
});
const info = await call("analyze_function", { procedure: address });
const n = normalized(info);
const selected =
values["native-question"] === "entry-views"
? await retainedEntryViews(call, info, inputSha, ToolFailure, [
containing.evidence_id,
...(loadImageEvidenceId === undefined ? [] : [loadImageEvidenceId]),
])
: {};
const { main, mainEvidenceIds = [] } =
values["native-question"] === "entry-and-main"
? await retainedJqMain(call, ToolFailure)
: {};
return {
artifact_sha256: info.subject?.digest.sha256,
entry_address: address,
containing_procedure: normalized(containing),
procedure: n.procedure,
pseudocode_prefix: n.pseudocode.slice(0, 512),
caller_count: n.callers.length,
callee_count: n.callees.length,
...(main === undefined ? {} : { main }),
analysis_profile: info.analysis_profile,
...(loadImage === undefined ? {} : { load_image: loadImage }),
...(selected.retainedViews === undefined
? {}
: {
retained_views: selected.retainedViews,
native_summary_outcome: selected.nativeSummaryOutcome,
}),
evidence_ids:
selected.retainedViews === undefined
? [
containing.evidence_id,
info.evidence_id,
...mainEvidenceIds,
...(loadImageEvidenceId === undefined
? []
: [loadImageEvidenceId]),
]
: {
containing_procedure: containing.evidence_id,
function_dossier: info.evidence_id,
...(loadImageEvidenceId === undefined
? {}
: { native_load_image: loadImageEvidenceId }),
retained_views: selected.retainedViewEvidence,
...(selected.summaryEvidenceId === undefined
? {}
: { native_summary: selected.summaryEvidenceId }),
},
};
} finally {
await call("close_binary", {});
}
};
export const nativeEntrySeed = async (artifact, path) => {
if (artifact === "jq") {
const file = await open(path, "r");
let header;
try {
header = (await file.read(Buffer.alloc(32), 0, 32, 0)).buffer;
} finally {
await file.close();
}
if (header.subarray(0, 6).toString("hex") !== "7f454c460201")
throw new Error("Expected ELF64 little endian entry seed");
return `0x${header.readBigUInt64LE(24).toString(16)}`;
}
if (artifact === "sharex.exe") {
const bytes = await readFile(path);
const pe = bytes.readUInt32LE(0x3c);
const optional = pe + 24;
if (
bytes.subarray(pe, pe + 4).toString("hex") !== "50450000" ||
bytes.readUInt16LE(pe + 4) !== 0x8664 ||
bytes.readUInt16LE(optional) !== 0x20b ||
bytes.readUInt32LE(optional + 108) <= 14 ||
bytes.readUInt32LE(optional + 112 + 14 * 8) !== 0 ||
bytes.readUInt32LE(optional + 116 + 14 * 8) !== 0
)
throw new Error(
"Expected the pinned native PE32+ entry seed without CLR metadata",
);
const address =
bytes.readBigUInt64LE(optional + 24) +
BigInt(bytes.readUInt32LE(optional + 16));
return `0x${address.toString(16)}`;
}
if (artifact === "freedos.edlin.exe") return undefined;
throw new Error("Select a pinned native artifact");
};
@@ -0,0 +1 @@
Statically inspect {{artifact_path}} using only the registered REA tools and Ghidra. This is the original UPX-compressed FreeDOS EDLIN 2.24 DOS MZ executable; inspect its admitted load image and entry stub, without running or unpacking it. Keep default auto-analysis enabled. Open the target with provider_id ghidra, call inspect_native_load_image, and use its verified entry.linear_address as the seed. Resolve the containing procedure, analyze that procedure once, and close the ephemeral session after obtaining the facts. If a result is clipped, use retained native facets without reanalysis. Report exact target SHA-256, entry_address, containing_procedure, complete procedure metadata including classification, signature, locals and inclusive body ranges, the first 512 UTF-16 units of pseudocode, caller_count, callee_count, real Evidence IDs and limitations. Preserve missing/partial facts and terminal-flow caveats. Verified static import and unpacking-stub pseudocode do not prove DOS execution or recovery of the editor's application logic. Do not execute targets, launch a GUI, or use shell/file tools/network or other agents. Final JSON fields: artifact_sha256, load_image_status, entry_address, containing_procedure, procedure, pseudocode_prefix, caller_count, callee_count, evidence_ids, limitations. Stop when these facts and cleanup are supported; report unsupported outcomes explicitly.
@@ -0,0 +1 @@
Statically inspect {{artifact_path}} using only the registered REA tools and Ghidra. The independently read, pinned PE32+ header reports image base 0x140000000 and entry RVA 0x125c0, so use address 0x1400125c0 as the analyst seed. This is a native x86-64 Windows PE component with no CLR directory, inspected on Linux; do not claim Windows runtime behavior. Keep default auto-analysis enabled. Open the target with provider_id ghidra, resolve the containing procedure, analyze that procedure once, and close the ephemeral session after obtaining the facts. If a result is clipped, use retained native facets without reanalysis. Report exact target SHA-256, entry_address, containing_procedure, complete procedure metadata including classification, signature, locals and inclusive body ranges, the first 512 UTF-16 units of pseudocode, caller_count, callee_count, real Evidence IDs and limitations. Preserve terminal-flow caveats and unresolved static edges; recovered pseudocode is not original source. Do not execute targets, launch a GUI, or use shell/file tools/network or other agents. Final JSON fields: artifact_sha256, entry_address, containing_procedure, procedure, pseudocode_prefix, caller_count, callee_count, evidence_ids, limitations. The seed applies only to the pinned ShareX.exe in corpus.json. Stop when these facts and cleanup are supported.
@@ -182,18 +182,61 @@ def main():
} for row in rows], key=lambda row: row['node_id'])
check('all_selected_module_identities', select(answer.get('modules', [])), select(reference['modules']))
elif domain == 'native':
for key in ['entry_address', 'pseudocode_prefix', 'caller_count', 'callee_count']:
for key in ['entry_address', 'caller_count', 'callee_count']:
check(key, answer.get(key), reference[key])
containing = answer['containing_procedure']
prefix = answer.get('pseudocode_prefix')
if isinstance(prefix, dict) and prefix.get('unit') == 'utf16-code-units':
text = prefix.get('text')
if 'requested' in prefix:
check('pseudocode_prefix.requested', prefix['requested'], 512)
if 'returned' in prefix:
check('pseudocode_prefix.returned', prefix['returned'], len(reference['pseudocode_prefix'].encode('utf-16-le')) // 2)
prefix = text
check('pseudocode_prefix', prefix, reference['pseudocode_prefix'])
containing = answer.get('containing_procedure')
expected_containing = reference['containing_procedure']['procedure']
for key in ['address', 'name']:
check('containing_procedure.' + key, containing.get(key, containing.get('procedure', {}).get(key)), expected_containing[key])
procedure = answer['procedure']
if isinstance(containing, str):
# Compare the explicitly reported display to the known source;
# never parse a display name into a canonical lookup identity.
check('containing_procedure.display', containing,
expected_containing['name'] + ' @ ' + expected_containing['address'])
else:
containing = containing if isinstance(containing, dict) else {}
nested = containing.get('procedure')
nested = nested if isinstance(nested, dict) else {}
for key in ['address', 'name']:
check('containing_procedure.' + key, containing.get(key, nested.get(key)), expected_containing[key])
procedure = answer.get('procedure')
procedure = procedure if isinstance(procedure, dict) else {}
expected = reference['procedure']
for key in ['address', 'name', 'classification', 'signature', 'locals']:
check('procedure.' + key, procedure.get(key), expected[key])
check('procedure.body_ranges', procedure.get('body_ranges', procedure.get('body', {}).get('ranges')), expected['body']['ranges'])
check('procedure.total_bytes', procedure.get('total_bytes', procedure.get('body', {}).get('total_bytes')), expected['body']['total_bytes'])
body = procedure.get('body')
body = body if isinstance(body, dict) else {}
check('procedure.body_ranges', procedure.get('body_ranges', body.get('ranges', body.get('ranges_inclusive'))), expected['body']['ranges'])
check('procedure.total_bytes', procedure.get('total_bytes', body.get('total_bytes')), expected['body']['total_bytes'])
if 'load_image' in reference:
sources = [source for source in producer_results.get('inspect_native_load_image', [])
if isinstance(source, dict)
and source.get('operation') == 'inspect_native_load_image'
and isinstance(source.get('evidence_id'), str)
and source.get('evidence_id') in reported_ids
and isinstance(source.get('normalized_result'), dict)]
checks['load_image_source_observed_and_referenced'] = bool(sources)
load = sources[-1]['normalized_result'] if sources else {}
entry = load.get('entry')
entry = entry if isinstance(entry, dict) else {}
check('load_image.target_sha256', load.get('target_sha256'), reference['artifact_sha256'])
check('load_image.source_status', load.get('status'), reference['load_image']['status'])
check('load_image.source_entry', entry.get('linear_address'), reference['entry_address'])
status = answer.get('load_image_status')
check('load_image_status', status.get('status') if isinstance(status, dict) else status, load.get('status'))
if isinstance(status, dict):
for key in ['format', 'load_segment', 'header_bytes', 'module_bytes', 'overlay_bytes']:
if key in status:
check('load_image.' + key, status[key], load.get(key))
if 'entry_linear_address' in status:
check('load_image.entry_linear_address', status['entry_linear_address'], entry.get('linear_address'))
calls = summary['completed_mcp_calls']
checks['default_analysis_not_disabled'] = all(not call['arguments'].get('skip_auto_analysis', False) for call in calls if call['tool'] == 'open_binary')
checks['session_close_completed'] = any(call['tool'] == 'close_binary' and call['status'] == 'completed' for call in calls)
@@ -248,6 +291,7 @@ def main():
and events['memory.events'].get('high') == 0
)
report = {
'review_source_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
'domain': domain, 'checks': checks, 'all_selected_checks_pass': all(checks.values()),
'differences': differences, 'tool_errors': tool_errors, 'summary': summary,
'fingerprints': {key: value for key, value in settings.items() if 'sha256' in key or key in ['runtime_revision', 'enabled_tools', 'authority']},
+22 -14
View File
@@ -25,6 +25,14 @@ fail() {
exit 1
}
# Bash can return status 0 while parsing an incomplete [[ condition. Keep that
# grammar in a constant quoted expression so truncated downloads fail while
# parsing the outer brace group. eval parses only this literal; the arguments
# are expanded as data by the conditional, not interpolated into shell code.
matches() {
eval '[[ "$1" =~ $2 ]]'
}
# Resolve the current host's Node.js tarball label pair; a label we cannot map
# is kept verbatim so the operator can still read it. Falls back on the raw
# `uname` outputs rather than inventing a label the Node.js release index does
@@ -95,41 +103,41 @@ main() {
esac
done
if [[ "$verbose" == true ]]; then
if [ "$verbose" = true ]; then
set -x
fi
platform="$(uname -s)"
[[ "$platform" == "Darwin" || "$platform" == "Linux" ]] || fail "REA supports macOS and selected 64-bit Linux distributions."
[ "$platform" = "Darwin" ] || [ "$platform" = "Linux" ] || fail "REA supports macOS and selected 64-bit Linux distributions."
command -v curl >/dev/null 2>&1 || fail "curl is required. Install curl, then rerun this installer."
command -v node >/dev/null 2>&1 || fail "Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+ is required; install it from https://nodejs.org and retry."
command -v npm >/dev/null 2>&1 || fail "npm is required; install it with Node.js and retry."
node_version="$(node -p 'process.versions.node' 2>/dev/null || true)"
IFS=. read -r node_major node_minor _node_patch <<<"$node_version"
[[ "$node_major" =~ ^[0-9]+$ && "$node_minor" =~ ^[0-9]+$ ]] || fail "the active Node.js version could not be read. Check that node works and is on PATH, then retry."
if ! [[ "$_node_patch" =~ ^[0-9]+(\+[0-9A-Za-z.-]+)?$ ]] ||
matches "$node_major" '^[0-9]+$' && matches "$node_minor" '^[0-9]+$' || fail "the active Node.js version could not be read. Check that node works and is on PATH, then retry."
if ! matches "$_node_patch" '^[0-9]+(\+[0-9A-Za-z.-]+)?$' ||
! ((node_major == 22 && node_minor >= 19 || node_major == 24 && node_minor >= 11 || node_major >= 26)); then
fail_node_version "$node_version"
fi
if [[ -n "$version" ]]; then
[[ "$version" =~ ^${version_pattern}$ ]] || fail "version must be an exact semantic version."
if [ -n "$version" ]; then
matches "$version" "^${version_pattern}$" || fail "version must be an exact semantic version."
else
release_json="$(curl -fsSL --proto '=https' --tlsv1.2 --retry 3 -H 'Accept: application/vnd.github+json' "https://api.github.com/repos/$REPOSITORY/releases/latest")" || fail "the latest REA release could not be resolved. Check network access or pass --version VERSION, then retry."
tag="$(printf '%s' "$release_json" | node -e 'let s="";process.stdin.on("data",c=>s+=c).on("end",()=>{const v=JSON.parse(s).tag_name;if(typeof v!=="string")process.exit(1);process.stdout.write(v)})' 2>/dev/null)" || fail "the release response was invalid. Retry later or pass --version VERSION."
[[ "$tag" =~ ^rea-agents-(${version_pattern})$ ]] || fail "the latest release tag was invalid. Retry later or pass --version VERSION."
matches "$tag" "^rea-agents-(${version_pattern})$" || fail "the latest release tag was invalid. Retry later or pass --version VERSION."
version="${BASH_REMATCH[1]}"
fi
# npm's exact-version resolver cleans build metadata from the requested version,
# so it may install the base release and make the post-install identity check fail.
if [[ "$version" =~ \+ ]]; then
if matches "$version" '\+'; then
fail "npm cannot install an exact version with build metadata. Pass a version without build metadata, then retry."
fi
install_args=(install --global)
if [[ "$platform" == "Linux" ]]; then
if [ "$platform" = "Linux" ]; then
install_args+=(--prefix "$HOME/.local")
install_bin="$HOME/.local/bin/rea"
else
@@ -145,21 +153,21 @@ main() {
printf ' %q' npm "${install_args[@]}"
printf '\n'
printf ' Binary: %s\n' "$install_bin"
printf ' Setup: %s\n' "$([[ "$start_setup" == true ]] && printf 'start when a terminal is available' || printf 'skipped')"
printf ' Setup: %s\n' "$([ "$start_setup" = true ] && printf 'start when a terminal is available' || printf 'skipped')"
if [[ "$dry_run" == true ]]; then
if [ "$dry_run" = true ]; then
printf 'Dry run complete; no changes made.\n'
exit 0
fi
printf 'Installing %s@%s...\n' "$PACKAGE" "$version"
npm "${install_args[@]}" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
[[ -x "$install_bin" ]] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
[ -x "$install_bin" ] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
[[ "$installed_version" == "$version" ]] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
[ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
printf 'REA %s is installed.\n' "$version"
if [[ "$start_setup" == true && -r /dev/tty && -w /dev/tty && -t 1 ]]; then
if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ]; then
printf 'Starting REA setup...\n'
"$install_bin" setup </dev/tty >/dev/tty
else
+4 -2
View File
@@ -110,7 +110,7 @@
"precheck:test": "npm run deps:check",
"check:test": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check test:run",
"precheck:pr": "npm run deps:check",
"check:pr": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check test:run && npm run docs:check && npm run jscpd && npm run scan:todos",
"check:pr": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check test:run && npm run test:python && npm run docs:check && npm run jscpd && npm run scan:todos",
"precheck:ci": "npm run deps:check",
"check:ci": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check jscpd scan:todos && npm run verify:architecture-guards",
"build:cached": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:skills",
@@ -128,6 +128,7 @@
"test": "npm run test:with-build",
"test:with-build": "npm run test:prepare && vitest run",
"test:run": "vitest run",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v",
"test:local": "node scripts/run-development-tests.mjs local",
"test:fast": "vitest run --project domain --project services --project adapters --project composition --project conformance --project evaluation",
"pretest:boundary": "npm run test:prepare",
@@ -169,6 +170,7 @@
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
"verify:ghidra:seeds": "npm run build:cached && node scripts/verify/ghidra/seeds.mjs",
"verify:ghidra:mips": "npm run build:cached && node scripts/verify-real-ghidra-mips.mjs",
"verify:ghidra:riscv": "npm run build:cached && node scripts/verify-real-ghidra-riscv.mjs",
"verify:ghidra:mips-shared": "npm run build:cached && node scripts/verify-real-ghidra-mips-shared.mjs",
"verify:ghidra:mips-relocatable": "npm run build:cached && node scripts/verify-real-ghidra-mips-relocatable.mjs",
"verify:ghidra:swift": "npm run build:cached && node scripts/verify-real-ghidra-swift.mjs",
@@ -231,7 +233,7 @@
"docs:build": "vitepress build docs",
"docs:preview": "vitepress preview docs",
"docs:generate": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts npm run docs:prepare",
"docs:check": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run docs:check:built",
"docs:check": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run docs:check:built && npm run test:local -- tests/conformance/readmeConsistency.test.ts",
"docs:check:built": "node scripts/generate-package-metadata.mjs --check && node scripts/generate-skill-metadata.mjs --check && node scripts/generate-product-catalog.mjs --check && node scripts/generate-completion-ledger.mjs --check && node scripts/check-doc-facts.mjs && node scripts/generate-error-schema.mjs --check",
"docs:error-schema": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:error-schema",
"prepack": "node scripts/prepack.mjs",
+1
View File
@@ -64,6 +64,7 @@ export const SCOPES = {
"tests/boundary/filesystem/javascript-applications/javascriptArtifactReconstruction.test.ts",
"tests/conformance/providers/ghidra/windowsCapabilities.test.ts",
"src/ghidra/GhidraInstallation.test.ts",
"tests/boundary/filesystem/ghidraJavaProbe.test.ts",
"tests/boundary/providers/ghidra/ghidraTransport.test.ts",
"tests/boundary/providers/ghidra/ghidraTargetSnapshot.test.ts",
"tests/boundary/providers/ghidra/ghidraLauncher.test.ts",
+73
View File
@@ -0,0 +1,73 @@
import { appendFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { isAbsolute, join, relative, resolve, sep } from "node:path";
import { z } from "zod";
const root = process.cwd();
const historyDirectory = join(root, ".cache/ci-test-history");
const historyPath = join(historyDirectory, "test-results.json");
// Never let a failed or partial rerun leave a previous successful history file.
await rm(historyPath, { force: true });
const report = z
.object({
success: z.boolean(),
numFailedTests: z.number().int().nonnegative(),
testResults: z.array(
z.object({
name: z.string().min(1),
startTime: z.number().finite(),
endTime: z.number().finite(),
}),
),
})
.parse(JSON.parse(await readFile("test-results.json", "utf8")));
const files = report.testResults.map(({ name, startTime, endTime }) => {
const path = relative(root, resolve(root, name));
if (
path === "" ||
path === ".." ||
path.startsWith(".." + sep) ||
isAbsolute(path)
)
throw new Error(`Test report path is outside the checkout: ${name}`);
const duration = endTime - startTime;
if (!Number.isFinite(duration) || duration < 0)
throw new Error(`Test report has an invalid duration: ${name}`);
return { name: path.split(sep).join("/"), startTime: 0, endTime: duration };
});
const complete = process.env.SHARD_RESULT === "success";
const passed =
complete &&
process.env.MERGE_RESULT === "success" &&
report.success &&
report.numFailedTests === 0;
const summary = [
complete ? "## Merged test diagnostics" : "## Partial test diagnostics",
"",
`Shard result: ${process.env.SHARD_RESULT ?? "unknown"}; merge result: ${process.env.MERGE_RESULT ?? "unknown"}.`,
passed
? "Successful timing history is available."
: "This report is diagnostic only and is not a successful timing baseline.",
"",
"| Slowest file | Duration (ms) |",
"| --- | ---: |",
...files
.toSorted((left, right) => right.endTime - left.endTime)
.slice(0, 20)
.map(
({ name, endTime }) => `| ${name.replaceAll("|", "\\|")} | ${endTime} |`,
),
"",
].join("\n");
if (process.env.GITHUB_STEP_SUMMARY)
await appendFile(process.env.GITHUB_STEP_SUMMARY, summary);
else console.log(summary);
if (passed) {
await mkdir(historyDirectory, { recursive: true });
await writeFile(
historyPath,
JSON.stringify({ success: true, numFailedTests: 0, testResults: files }),
);
}
@@ -36,16 +36,11 @@ export async function verifyGhidraSnapshotLifecycle(
);
const error = requireMcpToolError(reply);
assert.equal(
error?.code,
error.code,
"evidence_integrity_mismatch",
JSON.stringify(reply),
);
assert.match(error.message, /after analysis metadata mutations/u);
assert.match(
error.remediation.action,
/close.*without saving a snapshot/iu,
);
assert.match(error.remediation.action, /export_evidence_bundle/u);
assert.equal(error.retryable, false);
rejectedCalls++;
};
try {
+391
View File
@@ -0,0 +1,391 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import {
mkdtemp,
mkdir,
readFile,
readdir,
rm,
writeFile,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import Ajv from "ajv/dist/2020.js";
import addFormats from "ajv-formats";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import { functionDossierSchema } from "../dist/domain/hopperValues.js";
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
import { buildRiscvFixture } from "../tests/conformance/ghidra/riscv-fixture.mjs";
import {
mcpTextValue,
requireMcpOperationResult,
requireMcpToolError,
} from "./lib/mcp-verifier-results.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
if (process.argv.length !== 2)
throw new Error("Usage: node scripts/verify-real-ghidra-riscv.mjs");
if (!["linux", "darwin"].includes(process.platform))
throw new Error("This optional lane requires a Linux/macOS Ghidra host.");
if (process.env.GHIDRA_INSTALL_DIR === undefined)
throw new Error("Missing prerequisite: caller-supplied GHIDRA_INSTALL_DIR");
for (const key of ["REA_GHIDRA_LANGUAGE_ID", "REA_GHIDRA_COMPILER_SPEC_ID"])
if (process.env[key] !== undefined)
throw new Error(
`Unset ${key}: this lane verifies provider-pinned selection.`,
);
const installation = inspectGhidraInstallation({
environment: process.env,
installDir: process.env.GHIDRA_INSTALL_DIR,
...(process.env.JAVA_HOME === undefined
? {}
: { javaHome: process.env.JAVA_HOME }),
});
assert.equal(installation.status, "available", JSON.stringify(installation));
assert.equal(installation.providerVersion, "12.1.4");
const execute = promisify(execFile);
const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url));
const run = createVerifierRun();
const workspace = await mkdtemp(join(tmpdir(), "rea-riscv-proof-"));
const runtime = join(workspace, "runtime");
await mkdir(runtime);
const env = {
PATH: process.env.PATH ?? "/usr/bin:/bin",
TMPDIR: runtime,
REA_LOG_LEVEL: "silent",
REA_ANALYSIS_PROVIDER: "ghidra",
REA_PROCESS_RUN_ID: run.run_id,
GHIDRA_INSTALL_DIR: installation.installDir,
JAVA_HOME: installation.javaHome,
};
for (const key of ["GHIDRA_HEADLESS_MAXMEM", "GHIDRA_MAXMEM"])
if (process.env[key] !== undefined) env[key] = process.env[key];
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env,
stderr: "pipe",
});
const client = new Client({ name: "rea-real-riscv-proof", version: "1" });
let stderr = "";
transport.stderr?.on("data", (chunk) => {
stderr = (stderr + chunk.toString()).slice(-65536);
});
const reports = [];
let failure;
try {
await client.connect(transport);
const catalog = await client.listTools();
const ajv = new Ajv({ strict: false });
addFormats(ajv);
const validators = new Map();
for (const tool of catalog.tools) {
assert.ok(
ajv.validateSchema(tool.outputSchema),
`${tool.name}: invalid output schema`,
);
const existing =
typeof tool.outputSchema.$id === "string"
? ajv.getSchema(tool.outputSchema.$id)
: undefined;
validators.set(tool.name, existing ?? ajv.compile(tool.outputSchema));
}
for (const bits of [32, 64]) {
const fixture = await buildRiscvFixture(workspace, bits);
const resolved = await parseBinaryTarget(fixture.path);
if (!resolved.ok) throw resolved.error;
assert.equal(resolved.value.architecture, "riscv");
assert.equal(resolved.value.sha256, fixture.sha256);
assert.deepEqual(resolved.value.riscv, fixture.riscv);
const expectedLanguage = `RISCV:LE:${bits}:default`;
const assertProfile = (profile) => {
assert.equal(profile.provider.id, "ghidra");
assert.equal(profile.provider.version, installation.providerVersion);
const parameters = profile.parameters;
assert.equal(parameters.language_id, expectedLanguage);
assert.equal(parameters.compiler_spec_id, "gcc");
assert.equal(parameters.language_selection, "provider-pinned-v1");
assert.equal(
parameters.riscv_support_lane,
`elf${bits}-exec-le-double-float-generic-v1`,
);
assert.deepEqual(parameters.riscv_elf, {
elf_class: bits,
byte_order: "little",
type: 2,
flags: 5,
float_abi_double: true,
rvc_declared: true,
rve_declared: false,
tso_declared: false,
});
};
const assertEvidence = (evidence) => {
assert.equal(evidence.provider.id, "ghidra");
assert.equal(evidence.subject.architecture, "riscv");
assert.equal(evidence.subject.digest.sha256, fixture.sha256);
assertProfile(evidence.analysis_profile);
};
const call = async (name, args = {}) => {
const reply = await client.callTool(
{ name, arguments: args },
{ timeout: 240000 },
);
const projected = requireMcpOperationResult(reply, name);
const textual = JSON.parse(mcpTextValue(reply));
assert.deepEqual(reply.structuredContent, textual);
const validate = validators.get(name);
assert.ok(validate, `${name} missing from advertised catalog`);
assert.ok(
validate(textual),
`${name}: ${JSON.stringify(validate.errors)}`,
);
if (!["open_binary", "close_binary", "binary_session"].includes(name))
assertEvidence(parseEvidence(textual));
return projected;
};
const opened = await call("open_binary", {
path: fixture.path,
provider_id: "ghidra",
});
assert.equal(opened.architecture, "riscv");
assert.equal(opened.sha256, fixture.sha256);
const image = await call("inspect_native_load_image");
assert.equal(image.observations.language_id, expectedLanguage);
assert.equal(image.observations.compiler_spec_id, "gcc");
assert.ok(image.observations.source_files.length > 0);
for (const source of image.observations.source_files) {
assert.equal(source.original_sha256, fixture.sha256);
assert.equal(source.modified_sha256, fixture.sha256);
}
const procedures = await call("list_procedures");
for (const name of [
"rea_riscv_entry",
"rea_riscv_leaf",
"rea_riscv_float",
"rea_riscv_probe",
]) {
const procedure = procedures.find((value) => value.value === name);
assert.ok(procedure, `${name} missing from Ghidra inventory`);
assert.equal(BigInt(procedure.address), BigInt(fixture.symbols[name]));
}
const probe = await call("inspect_native_instruction", {
address: fixture.symbols.rea_riscv_probe,
});
assert.equal(probe.status, "decoded");
assert.equal(probe.architecture, expectedLanguage);
assert.equal(probe.length, 4);
assert.equal(probe.bytes, "13051501"); // addi a0, a0, 17
const mapping = await call("address_to_file_offset", {
address: fixture.symbols.rea_riscv_probe,
});
const sourceBytes = await readFile(fixture.path);
assert.equal(
sourceBytes
.subarray(mapping.file_offset, mapping.file_offset + 4)
.toString("hex"),
probe.bytes,
);
const global = await call("read_bytes", {
address: fixture.symbols.rea_riscv_global,
length: 4,
});
assert.equal(global.bytes_hex, "07000000");
const callees = await call("procedure_callees", {
procedure: fixture.symbols.rea_riscv_entry,
});
for (const name of ["rea_riscv_leaf", "rea_riscv_probe"])
assert.ok(
callees.some(
(address) => BigInt(address) === BigInt(fixture.symbols[name]),
),
);
const dossier = functionDossierSchema.parse(
await call("analyze_function", {
procedure: fixture.symbols.rea_riscv_entry,
}),
);
assert.ok(dossier.pseudocode.trim().length > 0);
const floatCode = await call("procedure_pseudo_code", {
procedure: fixture.symbols.rea_riscv_float,
});
assert.ok(typeof floatCode === "string" && floatCode.trim().length > 0);
assert.match(floatCode, /\bdouble\b/u);
const refusals = [];
for (const [flags, reason] of [
[1, "EF_RISCV_FLOAT_ABI_DOUBLE"],
[0x80000005, "(0x80000000)"],
]) {
// Header-mutated copies test admission; they are not new compiler claims.
const rejectedPath = join(
workspace,
`rv${bits}-flags-${flags.toString(16)}.elf`,
);
const rejectedBytes = Buffer.from(sourceBytes);
rejectedBytes.writeUInt32LE(flags, bits === 32 ? 36 : 48);
await writeFile(rejectedPath, rejectedBytes);
const reply = await client.callTool(
{
name: "open_binary",
arguments: { path: rejectedPath, provider_id: "ghidra" },
},
{ timeout: 60000 },
);
const error = requireMcpToolError(reply);
assert.ok(JSON.stringify(error).includes(reason), JSON.stringify(error));
await assert.rejects(
execute(
process.execPath,
[
entrypoint,
"function",
rejectedPath,
"rea_riscv_entry",
"--provider",
"ghidra",
"--json",
],
{
env,
timeout: 60000,
maxBuffer: 1024 * 1024,
},
),
(cause) =>
Number.isInteger(cause.code) &&
cause.code !== 0 &&
`${cause.stdout}\n${cause.stderr}`.includes(reason),
);
refusals.push({ flags, reason });
}
const snapshotPath = join(workspace, `rv${bits}.snapshot.json`);
const saved = await call("close_binary", { snapshot_path: snapshotPath });
assert.ok(saved.evidence_records > 0);
const snapshot = parseAnalysisSnapshot(
JSON.parse(await readFile(snapshotPath, "utf8")),
);
assert.equal(snapshot.target.architecture, "riscv");
assert.equal(snapshot.target.sha256, fixture.sha256);
assertProfile(snapshot.binding.analysis_profile);
for (const record of snapshot.evidence_bundle.records)
assertEvidence(parseEvidence(record));
await call("open_binary", {
path: fixture.path,
provider_id: "ghidra",
snapshot_path: snapshotPath,
});
assert.deepEqual(await call("list_procedures"), procedures);
await call("close_binary");
for (const [command, arguments_] of [
["search", ["rea_riscv_", "--kind", "procedures"]],
["function", ["rea_riscv_entry"]],
]) {
const cli = await execute(
process.execPath,
[
entrypoint,
command,
fixture.path,
...arguments_,
"--provider",
"ghidra",
"--json",
],
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
);
const evidence = parseEvidence(JSON.parse(cli.stdout));
assertEvidence(evidence);
if (command === "search")
for (const name of [
"rea_riscv_entry",
"rea_riscv_leaf",
"rea_riscv_float",
"rea_riscv_probe",
])
assert.ok(
evidence.normalized_result.some((value) => value.value === name),
);
else {
const cliDossier = functionDossierSchema.parse(
evidence.normalized_result,
);
assert.equal(cliDossier.procedure.address, dossier.procedure.address);
assert.ok(cliDossier.pseudocode.trim().length > 0);
}
}
assert.equal(
createHash("sha256")
.update(await readFile(fixture.path))
.digest("hex"),
fixture.sha256,
);
reports.push({
...fixture,
language: expectedLanguage,
procedures: procedures.length,
refusals,
cli: true,
stdio_mcp: true,
snapshot_replay: true,
original_unchanged: true,
});
}
} catch (cause) {
failure = cause;
}
for (const close of [() => client.close(), () => transport.close()]) {
try {
await close();
} catch (cause) {
failure = new AggregateError(
failure === undefined ? [cause] : [failure, cause],
"Verification or cleanup failed",
);
}
}
const completed = await completeVerifierRun(run);
if (
completed.process_lineage.status !== "verified" ||
completed.process_lineage.descendants.length !== 0
)
throw new Error(`Cleanup unverified; retained workspace: ${workspace}`, {
cause: failure,
});
if (failure !== undefined)
throw new Error(
`RISC-V verification failed; retained workspace: ${workspace}; MCP stderr: ${stderr}`,
{ cause: failure },
);
assert.deepEqual(
await readdir(runtime),
[],
`Runtime resources survived server shutdown; retained workspace: ${workspace}`,
);
await rm(workspace, { recursive: true, force: true });
console.log(
JSON.stringify(
{
verifier_run: completed,
lane: "ghidra-riscv",
mocked: false,
host: { platform: process.platform, architecture: process.arch },
provider: { id: "ghidra", version: installation.providerVersion },
fixtures: reports,
cleanup: "complete",
},
null,
2,
),
);
+49 -2
View File
@@ -16,6 +16,8 @@ const { values } = parseArgs({
options: {
target: { type: "string" },
procedure: { type: "string", default: "rea_fixture_add" },
"second-procedure": { type: "string" },
"second-marker": { type: "string" },
"package-root": { type: "string" },
report: { type: "string" },
},
@@ -24,6 +26,14 @@ if (values.target === undefined || process.env.REA_IDA_MCP_CONFIG === undefined)
throw new Error(
"IDA verification requires --target and REA_IDA_MCP_CONFIG; see docs/ida-provider.md. No toolchain is installed by this lane.",
);
if (
(values["second-procedure"] === undefined) !==
(values["second-marker"] === undefined) ||
values["second-marker"] === ""
)
throw new Error(
"IDA verification requires --second-procedure and a nonempty --second-marker together; choose a marker unique to the second function's pseudocode.",
);
const packageRoot = resolve(
values["package-root"] ?? fileURLToPath(new URL("..", import.meta.url)),
);
@@ -57,6 +67,8 @@ const environment = Object.fromEntries(
const report = {
passed: false,
mode: config.mode,
protocol:
config.protocol ?? (config.mode === "attached" ? "legacy" : "modern"),
checks: [],
observations: [],
};
@@ -156,7 +168,7 @@ const mcp = async (cliDossier) => {
["procedure_address", { procedure: dossier.procedure.name }],
["procedure_assembly", { procedure: dossier.procedure.address }],
["procedure_callees", { procedure: dossier.procedure.address }],
...(config.mode === "attached"
...(config.mode === "attached" && config.protocol !== "native"
? [["procedure_callers", { procedure: dossier.procedure.address }]]
: []),
["search_procedures", { pattern: dossier.procedure.name }],
@@ -168,6 +180,38 @@ const mcp = async (cliDossier) => {
const result = await call(name, args);
observe(result);
}
if (values["second-procedure"] !== undefined) {
const secondAddress = observe(
await call("procedure_address", {
procedure: values["second-procedure"],
}),
);
assert.notEqual(secondAddress, dossier.procedure.address);
const secondCode = observe(
await call("procedure_pseudo_code", {
procedure: values["second-procedure"],
}),
);
assert.equal(typeof secondCode, "string");
assert.ok(secondCode.length > 0);
assert.ok(
!dossier.pseudocode.includes(values["second-marker"]),
"The second marker also appears in the first function's pseudocode.",
);
assert.ok(
secondCode.includes(values["second-marker"]),
"The second function's pseudocode omitted its expected marker.",
);
const restored = observe(
await call("procedure_pseudo_code", {
procedure: dossier.procedure.address,
}),
);
assert.equal(restored, dossier.pseudocode);
report.checks.push(
"second-function pseudocode identity and return to the original function",
);
}
const invalid = await client.callTool({
name: "xrefs",
arguments: { address: "not-an-address" },
@@ -195,7 +239,10 @@ const cleanup = async () => {
try {
await upstream.connect();
if (config.mode === "attached") {
const metadata = await upstream.call("get_metadata", {});
const metadata =
config.protocol === "native"
? await upstream.readResource("ida://idb/metadata")
: await upstream.call("get_metadata", {});
assert.equal(metadata.sha256.toLowerCase(), await digest());
report.checks.push(
"existing GUI target remains open with the original input identity",
+4 -4
View File
@@ -1086,10 +1086,10 @@ it("rejects mistyped intent extras before touching the device", async () => {
it("collects a device bugreport archive with its digest", async () => {
const { binary, root } = await writeAdbStub(
(root) => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "-s" ] && [ "$3" = "bugreport" ]; then
printf 'bugreport-bytes' > "${join(root, "bugreport test.zip")}"
printf 'Bug report copied to ${join(root, "bugreport test.zip")}\\n'
printf 'bugreport-bytes' > "$4/bugreport test.zip"
printf 'Bug report copied to %s/bugreport test.zip\\n' "$4"
exit 0
fi
exit 1`,
@@ -1103,7 +1103,7 @@ exit 1`,
local_path: join(root, "bugreport test.zip"),
bytes: 15,
sha256: digest,
adb_reported_path: join(root, "bugreport test.zip"),
adb_reported_path: expect.stringContaining(".rea-bugreport-"),
});
});
+224 -88
View File
@@ -2,14 +2,16 @@ import { createHash } from "node:crypto";
import { constants as fsConstants } from "node:fs";
import {
access,
copyFile,
lstat,
mkdtemp,
mkdir,
rmdir,
rm,
stat,
writeFile,
} from "node:fs/promises";
import { join, resolve } from "node:path";
import { basename, dirname, extname, join, resolve } from "node:path";
import type {
ExecutionOptions,
@@ -20,7 +22,10 @@ import {
createAnalysisExecution,
type AnalysisExecution,
} from "../application/AnalysisProvider.js";
import type { AdbRequest } from "../domain/adb/adbDeviceAnalysis.js";
import type {
AdbAcquiredFilePartialObservation,
AdbRequest,
} from "../domain/adb/adbDeviceAnalysis.js";
import {
AnalysisArtifactChangedError,
AnalysisCancelledError,
@@ -305,30 +310,56 @@ const refuseExistingTarget = async (
return undefined;
};
/** Publish an ADB-produced archive without replacing an existing caller file. */
const publishBugreport = async (
source: string,
directory: string,
): Promise<string> => {
const fileName = basename(source);
const extension = extname(fileName);
const stem =
extension === "" ? fileName : fileName.slice(0, -extension.length);
for (let suffix = 0; ; suffix += 1) {
const destination = join(
directory,
suffix === 0 ? fileName : `${stem} (${String(suffix)})${extension}`,
);
try {
await copyFile(source, destination, fsConstants.COPYFILE_EXCL);
return destination;
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "EEXIST")
continue;
throw cause;
}
}
};
export interface AdbProviderOptions {
readonly environment?: Readonly<Record<string, string | undefined>>;
}
interface PendingPackageDirectoryCleanup {
interface PendingOutputDirectoryCleanup {
readonly operation: "pull_adb_package" | "collect_adb_bugreport";
readonly path: string;
readonly identity: PackageDirectoryIdentity | undefined;
readonly identity: OutputDirectoryIdentity | undefined;
readonly primary: AnalysisError;
readonly remove: () => Promise<void>;
}
interface PackageDirectoryIdentity {
interface OutputDirectoryIdentity {
readonly dev: number;
readonly ino: number;
readonly mode: number;
}
const removeOwnedPackageDirectory = async (
const removeOwnedOutputDirectory = async (
path: string,
identity: PackageDirectoryIdentity | undefined,
identity: OutputDirectoryIdentity | undefined,
): Promise<void> => {
if (identity === undefined)
throw new Error(
`Cannot verify the created package directory identity; refusing to remove ${path}`,
`Cannot verify the created output directory identity; refusing to remove ${path}`,
);
let current: Awaited<ReturnType<typeof lstat>>;
try {
@@ -346,7 +377,7 @@ const removeOwnedPackageDirectory = async (
current.mode !== identity.mode
)
throw new Error(
`The created package directory was replaced or changed; refusing to remove ${path}`,
`The created output directory was replaced or changed; refusing to remove ${path}`,
);
await rm(path, { recursive: true, force: true });
};
@@ -363,7 +394,7 @@ const reservePackageDestinations = async (
): Promise<AnalysisOutputError | undefined> => {
const reservations: {
readonly path: string;
readonly identity: PackageDirectoryIdentity;
readonly identity: OutputDirectoryIdentity;
}[] = [];
for (const entry of entries) {
const path = join(packageDirectory, entry.file_name);
@@ -424,12 +455,12 @@ const reservePackageDestinations = async (
*/
export class AdbProvider {
private readonly environment: Readonly<Record<string, string | undefined>>;
readonly #pendingPackageDirectoryCleanup =
new Set<PendingPackageDirectoryCleanup>();
readonly #pendingOutputDirectoryCleanup =
new Set<PendingOutputDirectoryCleanup>();
readonly #activeOperations = new Set<Promise<void>>();
#closing = false;
#closePromise: Promise<void> | undefined;
#packageCleanupPromise: Promise<void> | undefined;
#outputCleanupPromise: Promise<void> | undefined;
constructor(options: AdbProviderOptions = {}) {
this.environment = options.environment ?? process.env;
@@ -462,13 +493,13 @@ export class AdbProvider {
}
}
/** Retry removal of package directories retained after failed rollback. */
/** Retry removal of owned output directories retained after failed cleanup. */
close(): Promise<void> {
this.#closing = true;
if (this.#closePromise !== undefined) return this.#closePromise;
const closing = (async () => {
await Promise.all(this.#activeOperations);
await this.#retryPackageDirectoryCleanup();
await this.#retryOutputDirectoryCleanup();
})().finally(() => {
if (this.#closePromise === closing) this.#closePromise = undefined;
});
@@ -476,23 +507,23 @@ export class AdbProvider {
return this.#closePromise;
}
#retryPackageDirectoryCleanup(): Promise<void> {
if (this.#packageCleanupPromise !== undefined)
return this.#packageCleanupPromise;
const cleanup = this.#performPackageDirectoryCleanup().finally(() => {
if (this.#packageCleanupPromise === cleanup)
this.#packageCleanupPromise = undefined;
#retryOutputDirectoryCleanup(): Promise<void> {
if (this.#outputCleanupPromise !== undefined)
return this.#outputCleanupPromise;
const cleanup = this.#performOutputDirectoryCleanup().finally(() => {
if (this.#outputCleanupPromise === cleanup)
this.#outputCleanupPromise = undefined;
});
this.#packageCleanupPromise = cleanup;
this.#outputCleanupPromise = cleanup;
return cleanup;
}
async #performPackageDirectoryCleanup(): Promise<void> {
async #performOutputDirectoryCleanup(): Promise<void> {
const failures: AnalysisError[] = [];
for (const owner of this.#pendingPackageDirectoryCleanup) {
for (const owner of this.#pendingOutputDirectoryCleanup) {
try {
await owner.remove();
this.#pendingPackageDirectoryCleanup.delete(owner);
this.#pendingOutputDirectoryCleanup.delete(owner);
} catch (cause: unknown) {
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
@@ -500,13 +531,13 @@ export class AdbProvider {
{
reason: cause instanceof Error ? cause.message : String(cause),
},
{ operation: "pull_adb_package", cause },
{ operation: owner.operation, cause },
);
failures.push(
analysisErrorWithCleanupFailure(
owner.primary,
cleanup,
"pull_adb_package",
owner.operation,
),
);
}
@@ -515,20 +546,22 @@ export class AdbProvider {
if (failures.length > 1)
throw new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[...this.#pendingPackageDirectoryCleanup].map(({ path }) => path),
[...this.#pendingOutputDirectoryCleanup].map(({ path }) => path),
{
reason:
"Failed package output directories remain after cleanup retry",
reason: "Failed ADB output directories remain after cleanup retry",
operations: [...this.#pendingOutputDirectoryCleanup].map(
({ operation }) => operation,
),
failures: failures.map(({ message, cleanupResources }) => ({
message,
resources: [...cleanupResources],
})),
},
{
operation: "pull_adb_package",
operation: "close_binary",
cause: new AggregateError(
failures,
"Multiple ADB package rollback cleanups failed",
"Multiple ADB output directory cleanups failed",
),
},
);
@@ -546,16 +579,16 @@ export class AdbProvider {
});
this.#activeOperations.add(active);
try {
if (this.#pendingPackageDirectoryCleanup.size > 0) {
if (this.#pendingOutputDirectoryCleanup.size > 0) {
try {
await this.#retryPackageDirectoryCleanup();
await this.#retryOutputDirectoryCleanup();
} catch (cause: unknown) {
return err(
cause instanceof AnalysisError
? cause
: new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[...this.#pendingPackageDirectoryCleanup].map(
[...this.#pendingOutputDirectoryCleanup].map(
({ path }) => path,
),
{
@@ -1245,7 +1278,7 @@ export class AdbProvider {
if (!directory.ok) return directory;
const packageDirectory = join(directory.value, packageName);
let packageDirectoryCreated = false;
let packageDirectoryIdentity: PackageDirectoryIdentity | undefined;
let packageDirectoryIdentity: OutputDirectoryIdentity | undefined;
try {
if (
await lstat(packageDirectory).then(
@@ -1275,17 +1308,18 @@ export class AdbProvider {
{ cause },
);
if (!packageDirectoryCreated) return err(primary);
const owner: PendingPackageDirectoryCleanup = {
const owner: PendingOutputDirectoryCleanup = {
operation,
path: packageDirectory,
identity: packageDirectoryIdentity,
primary,
remove: () =>
removeOwnedPackageDirectory(
removeOwnedOutputDirectory(
packageDirectory,
packageDirectoryIdentity,
),
};
this.#pendingPackageDirectoryCleanup.add(owner);
this.#pendingOutputDirectoryCleanup.add(owner);
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[packageDirectory],
@@ -1299,12 +1333,13 @@ export class AdbProvider {
const failOwnedPackage = async (
primary: AnalysisError,
): Promise<Result<AnalysisExecution, AnalysisError>> => {
const owner: PendingPackageDirectoryCleanup = {
const owner: PendingOutputDirectoryCleanup = {
operation,
path: packageDirectory,
identity: packageDirectoryIdentity,
primary,
remove: () =>
removeOwnedPackageDirectory(
removeOwnedOutputDirectory(
packageDirectory,
packageDirectoryIdentity,
),
@@ -1312,7 +1347,7 @@ export class AdbProvider {
try {
await owner.remove();
} catch (cause: unknown) {
this.#pendingPackageDirectoryCleanup.add(owner);
this.#pendingOutputDirectoryCleanup.add(owner);
const cleanup = new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[packageDirectory],
@@ -2084,48 +2119,71 @@ export class AdbProvider {
input.output_directory,
);
if (!directory.ok) return directory;
const collected = await runAdb(
operation,
input.serial,
selection.binary,
adbArguments(input.serial, ["bugreport", directory.value]),
{
timeoutMs: BUGREPORT_TIMEOUT_MS,
maxOutputBytes: BUGREPORT_MAX_OUTPUT_BYTES,
},
signal,
);
if (!collected.ok) return collected;
const reportedPath = parseBugreportCopiedPath(collected.value.stdout);
if (reportedPath === null)
return err(
new AnalysisProtocolError(
"adb bugreport did not report an archive path; the build may use an unsupported format",
),
);
let stagingDirectory: string;
let stagingIdentity: OutputDirectoryIdentity | undefined;
let retainedObservation: AdbAcquiredFilePartialObservation | undefined;
let publishedPath: string | undefined;
try {
const digest = await hashAdbFile(reportedPath, operation, signal);
return ok(
createAnalysisExecution(
{
client,
serial: input.serial,
local_path: reportedPath,
bytes: digest.bytes,
sha256: digest.sha256,
adb_reported_path: reportedPath,
},
provider,
{
limitations: [
"The archive is device-generated: dumpstate, dumpsys, and logcat content is the device's own report",
],
},
),
stagingDirectory = await mkdtemp(
join(directory.value, ".rea-bugreport-"),
);
} catch (cause) {
if (cause instanceof AnalysisCancelledError) {
cause.retainPartialObservation({
return err(
new AnalysisOutputError(
operation,
"The bugreport staging directory could not be created",
{ cause },
),
);
}
const collectedResult = await (async (): Promise<
Result<AnalysisExecution, AnalysisError>
> => {
let reportedPath: string | null = null;
try {
const created = await lstat(stagingDirectory);
if (!created.isDirectory() || created.isSymbolicLink())
throw new Error(
`The created bugreport staging output is not a regular directory: ${stagingDirectory}`,
);
stagingIdentity = {
dev: created.dev,
ino: created.ino,
mode: created.mode,
};
const collected = await runAdb(
operation,
input.serial,
selection.binary,
adbArguments(input.serial, ["bugreport", stagingDirectory]),
{
timeoutMs: BUGREPORT_TIMEOUT_MS,
maxOutputBytes: BUGREPORT_MAX_OUTPUT_BYTES,
},
signal,
);
if (!collected.ok) return collected;
reportedPath = parseBugreportCopiedPath(collected.value.stdout);
if (reportedPath === null)
return err(
new AnalysisProtocolError(
"adb bugreport did not report an archive path; the build may use an unsupported format",
),
);
if (dirname(resolve(reportedPath)) !== stagingDirectory)
return err(
new AnalysisProtocolError(
`adb bugreport reported an archive outside its staging directory: ${reportedPath}`,
),
);
const archive = await lstat(reportedPath);
if (!archive.isFile() || archive.isSymbolicLink())
return err(
new AnalysisProtocolError(
`adb bugreport did not report a regular archive file: ${reportedPath}`,
),
);
retainedObservation = {
provider_id: "adb",
operation,
result: {
@@ -2135,16 +2193,94 @@ export class AdbProvider {
adb_reported_path: reportedPath,
digest_status: "unknown",
},
});
return err(cause);
}
return err(
new AnalysisOutputError(
};
try {
publishedPath = await publishBugreport(reportedPath, directory.value);
} catch (cause) {
return err(
new AnalysisOutputError(
operation,
`The reported bugreport archive could not be published: ${reportedPath}`,
{ cause, partialObservation: retainedObservation },
),
);
}
retainedObservation = {
...retainedObservation,
result: { ...retainedObservation.result, local_path: publishedPath },
};
const digest = await hashAdbFile(publishedPath, operation, signal);
const report = {
client,
serial: input.serial,
local_path: publishedPath,
bytes: digest.bytes,
sha256: digest.sha256,
adb_reported_path: reportedPath,
};
retainedObservation = {
provider_id: "adb",
operation,
result: report,
};
return ok(
createAnalysisExecution(report, provider, {
limitations: [
"The archive is device-generated: dumpstate, dumpsys, and logcat content is the device's own report",
],
}),
);
} catch (cause) {
if (cause instanceof AnalysisCancelledError) {
if (retainedObservation !== undefined)
cause.retainPartialObservation(retainedObservation);
return err(cause);
}
return err(
new AnalysisOutputError(
operation,
`The reported bugreport archive could not be read back: ${reportedPath ?? stagingDirectory}`,
{ cause, partialObservation: retainedObservation },
),
);
}
})();
// A completed acquisition becomes caller evidence even if publication fails.
// Its staging directory must not enter automatic cleanup or cleanup retries.
if (retainedObservation !== undefined && publishedPath === undefined)
return collectedResult;
try {
await removeOwnedOutputDirectory(stagingDirectory, stagingIdentity);
} catch (cause) {
const primary = collectedResult.ok
? new AnalysisOutputError(
operation,
"The archive was published, but its staging directory could not be removed",
)
: collectedResult.error;
if (retainedObservation !== undefined)
primary.retainPartialObservation(retainedObservation);
this.#pendingOutputDirectoryCleanup.add({
operation,
path: stagingDirectory,
identity: stagingIdentity,
primary,
remove: () =>
removeOwnedOutputDirectory(stagingDirectory, stagingIdentity),
});
return err(
analysisErrorWithCleanupFailure(
primary,
new ProviderCleanupError(
ADB_PROVIDER_IDENTITY.id,
[stagingDirectory],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation, cause },
),
operation,
`The reported bugreport archive could not be read back: ${reportedPath}`,
{ cause },
),
);
}
return collectedResult;
}
}
@@ -0,0 +1,34 @@
import { describe, expect, it } from "vitest";
import { parseClientConfiguration } from "./ClientConfigurationDocument.js";
describe("parseClientConfiguration shape diagnostics", () => {
it.each([
["null", "null"],
["array", "[]"],
["string", '"x"'],
] as const)("names a non-object JSON root (%s)", (_label, text) => {
expect(() => parseClientConfiguration(text, "json")).toThrow(
new TypeError("The configuration root must be an object"),
);
});
it.each([
["null", "null"],
["array", "[]"],
["string", '"x"'],
] as const)(
'names a non-object "mcpServers" value (%s)',
(_label, servers) => {
expect(() =>
parseClientConfiguration(`{"mcpServers":${servers},"keep":1}`, "json"),
).toThrow(new TypeError('"mcpServers" must be an object'));
},
);
it("still rejects unparsable JSON with SyntaxError", () => {
expect(() => parseClientConfiguration("not-json", "json")).toThrow(
SyntaxError,
);
});
});
+10 -16
View File
@@ -107,18 +107,21 @@ const parseDocument = (
format: ClientConfigurationFormat,
): Record<string, unknown> => {
if (format === "toml" || format === "grok")
return objectSchema.parse(parseToml(text));
return requireObject(parseToml(text), "The configuration root");
// Pi loads `mcp.json` with a strict `JSON.parse` and no BOM tolerance, so
// comments, trailing commas, a BOM, or an empty existing file must be
// rejected instead of registering into a document Pi cannot load.
if (format === "pi") return objectSchema.parse(JSON.parse(text));
if (format === "pi")
return requireObject(JSON.parse(text), "The configuration root");
// An empty file, which some clients create before any server is added,
// holds no settings to preserve.
if (isEmptyClientConfigurationText(text)) return {};
// Hermes stores MCP servers in `mcp_servers` inside a YAML document.
if (format === "hermes") return objectSchema.parse(parseYaml(text));
if (format === "hermes")
return requireObject(parseYaml(text), "The configuration root");
// ZCode loads nonempty native files with strict JSON.parse.
if (format === "zcode") return objectSchema.parse(JSON.parse(text));
if (format === "zcode")
return requireObject(JSON.parse(text), "The configuration root");
const errors: ParseError[] = [];
// Accept a UTF-8 BOM without shifting diagnostics or editing the original text.
const jsonText = text.startsWith("\uFEFF") ? ` ${text.slice(1)}` : text;
@@ -128,12 +131,13 @@ const parseDocument = (
throw new SyntaxError(
`Invalid JSON/JSONC at offset ${firstError.offset}: ${printParseErrorCode(firstError.error)}`,
);
return objectSchema.parse(document);
return requireObject(document, "The configuration root");
};
/** A parsed configuration whose structure, not its syntax, cannot hold MCP servers. */
export class ClientConfigurationShapeError extends TypeError {}
/** Validate and copy own properties, excluding non-record YAML values and parser-injected prototypes. */
const requireObject = (
value: unknown,
description: string,
@@ -174,17 +178,7 @@ export const parseClientConfiguration = (
throw new TypeError("client does not have a supported MCP config format");
// DeepSeek Harness stores a Cordis patch list, not a server table.
if (format === "dsh") return parseDshCordisDocument(text);
let document: Record<string, unknown>;
try {
document = parseDocument(text, format);
} catch (cause: unknown) {
if (cause instanceof z.ZodError)
throw new ClientConfigurationShapeError(
"The configuration root must be an object",
{ cause },
);
throw cause;
}
const document = parseDocument(text, format);
const serversKey = clientConfigurationServersKey(format);
const value = document[serversKey];
const servers =
+13
View File
@@ -128,6 +128,9 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string =>
const codexDirectory = ({ home, env }: ClientPathContext): string =>
env.CODEX_HOME || join(home, ".codex");
const qoderDirectory = ({ home, env }: ClientPathContext): string =>
env.QODER_CONFIG_DIR || join(home, ".qoder");
/** Match Qwen Code's home override, including tilde and cwd-relative paths. */
const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => {
const override = env.QWEN_HOME;
@@ -528,6 +531,16 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: hermesDirectory,
format: "hermes",
},
{
name: "qoder",
displayName: "Qoder",
skillPath: (context: ClientPathContext) =>
join(qoderDirectory(context), "skills"),
configPath: (context: ClientPathContext) =>
join(qoderDirectory(context), "settings.json"),
markerPath: qoderDirectory,
format: "json",
},
{
name: "zcode",
zcodeFallbackPath: ({ home }: ClientPathContext) =>
+1 -1
View File
@@ -289,7 +289,7 @@ const readClientConfiguration = async (
"failed",
cause instanceof ClientConfigurationShapeError
? `Configuration was not changed: ${cause.message}. Repair it, then rerun uninstall.`
: `Configuration is not valid ${client.format === "toml" || client.format === "grok" ? "TOML" : client.format === "opencode" ? "JSONC" : "JSON"} and was not changed. Repair it, then rerun uninstall.`,
: `Configuration is not valid ${client.format === "toml" || client.format === "grok" ? "TOML" : client.format === "opencode" ? "JSONC" : client.format === "hermes" || client.format === "dsh" ? "YAML" : "JSON"} and was not changed. Repair it, then rerun uninstall.`,
),
);
}
@@ -986,3 +986,39 @@ it("serializes script loading against concurrent close and unloads the loaded sc
expect(bindings.device.session.scripts[0]?.isDestroyed).toBe(true);
expect(bindings.device.session.isDetached()).toBe(true);
});
it("names the unknown session or script ID it rejects", async () => {
const { manager, started } = await startLocalAttachedSession();
if (!started.ok) throw started.error;
const missing = "00000000-0000-4000-8000-000000000009";
const sessionIssue = {
path: ["session_id"],
reason: "invalid_value",
message: `No open Frida session ${missing}; start one with start_frida_session.`,
};
for (const result of [
await manager.closeSession(missing),
await manager.resumeSession(missing),
await manager.loadScript(missing, { sourceKind: "inline", source: "" }),
await manager.unloadScript(missing, missing),
])
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError", issues: [sessionIssue] },
});
expect(
await manager.unloadScript(started.value.sessionId, missing),
).toMatchObject({
ok: false,
error: {
issues: [
{
path: ["script_id"],
reason: "invalid_value",
message: `No loaded Frida script ${missing} in this session.`,
},
],
},
});
await manager.closeAll();
});
@@ -1,3 +1,6 @@
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, it } from "vitest";
import type { FridaInstrumentationPort } from "./FridaInstrumentationPort.js";
@@ -102,6 +105,66 @@ it.each(["startSession", "instrument"] as const)(
},
);
it.each([
[join(tmpdir(), "rea-missing-frida-script.js"), "no file exists there"],
[tmpdir(), "it is not a regular file"],
])(
"rejects the unreadable script file %s before starting Frida",
async (path, reason) => {
let starts = 0;
const provider = createProvider({
async startSession() {
starts += 1;
return { ok: true, value: createSession() };
},
});
const service = new FridaInstrumentationService(provider);
const result = await service.instrument({
mode: "spawn",
deviceId: "local",
program: "/tmp/fixture",
source: { sourceKind: "file", path },
durationMs: 1,
});
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error).toBeInstanceOf(AnalysisInputError);
expect((result.error as AnalysisInputError).issues).toEqual([
{
path: ["path"],
reason: "invalid_value",
message: `Cannot read Frida script file ${path}: ${reason}.`,
},
]);
expect(starts).toBe(0);
},
);
it("reports cancellation before checking a file script source", async () => {
const service = new FridaInstrumentationService(createProvider());
const controller = new AbortController();
controller.abort(new Error("cancelled before call"));
const result = await service.instrument(
{
mode: "spawn",
deviceId: "local",
program: "/tmp/fixture",
source: {
sourceKind: "file",
path: join(tmpdir(), "rea-missing-frida-script.js"),
},
durationMs: 1,
},
controller.signal,
);
expect(result.ok).toBe(false);
if (!result.ok) expect(result.error._tag).toBe("AnalysisCancelledError");
});
it("does not resume a spawned target when cancellation arrives during script loading", async () => {
const controller = new AbortController();
let resumes = 0;
@@ -1,11 +1,15 @@
import { constants } from "node:fs";
import { access, stat } from "node:fs/promises";
import { setTimeout as delay } from "node:timers/promises";
import { createEvidence, type Evidence } from "../../domain/evidence.js";
import { jsonObjectSchema, jsonValueSchema } from "../../domain/jsonValue.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisAccessDeniedError,
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../../domain/result.js";
import type {
@@ -19,6 +23,64 @@ import type {
StartFridaSessionInput,
} from "./FridaInstrumentationPort.js";
const unreadableScriptFile = async (
source: FridaScriptSource,
): Promise<AnalysisError | undefined> => {
if (source.sourceKind !== "file") return undefined;
let reason: string;
let cause: unknown;
try {
if (!(await stat(source.path)).isFile())
reason = "it is not a regular file";
else {
await access(source.path, constants.R_OK);
return undefined;
}
} catch (failure: unknown) {
cause = failure;
const code =
failure instanceof Error && "code" in failure ? failure.code : undefined;
if (code === "EACCES" || code === "EPERM")
return new AnalysisAccessDeniedError(
"instrument_with_frida",
source.path,
code,
{ cause },
);
reason =
code === "ENOENT" ? "no file exists there" : String(code ?? failure);
}
return new AnalysisInputError(
"instrument_with_frida",
cause === undefined ? undefined : { cause },
[
{
path: ["path"],
reason: "invalid_value",
message: `Cannot read Frida script file ${source.path}: ${reason}.`,
},
],
);
};
/**
* Reject an unreadable script file before any attach or spawn effect;
* cancellation before or during the check still wins.
*/
const rejectBeforeStart = async (
source: FridaScriptSource,
signal: AbortSignal | undefined,
): Promise<AnalysisError | undefined> => {
const cancelled = () =>
new AnalysisCancelledError(
"instrument_with_frida",
signal?.reason === undefined ? {} : { cause: signal.reason },
);
if (signal?.aborted) return cancelled();
const unreadable = await unreadableScriptFile(source);
return signal?.aborted ? cancelled() : unreadable;
};
/** Shared CLI/MCP workflows for Frida's device and instrumentation operations. */
export class FridaInstrumentationService {
readonly #provider: FridaInstrumentationPort;
@@ -163,6 +225,8 @@ export class FridaInstrumentationService {
AnalysisError
>
> {
const rejected = await rejectBeforeStart(input.source, signal);
if (rejected !== undefined) return err(rejected);
const started = await this.#startSession(
input,
signal,
+27
View File
@@ -0,0 +1,27 @@
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
/** A session ID that names no open Frida session owned by this process. */
export const unknownFridaSession = (
operation: string,
sessionId: string,
): AnalysisInputError =>
new AnalysisInputError(operation, undefined, [
{
path: ["session_id"],
reason: "invalid_value",
message: `No open Frida session ${sessionId}; start one with start_frida_session.`,
},
]);
/** A script ID that names no loaded script in the selected session. */
export const unknownFridaScript = (
operation: string,
scriptId: string,
): AnalysisInputError =>
new AnalysisInputError(operation, undefined, [
{
path: ["script_id"],
reason: "invalid_value",
message: `No loaded Frida script ${scriptId} in this session.`,
},
]);
+65 -1
View File
@@ -1,6 +1,7 @@
import { describe, expect, it } from "vitest";
import { build, buildBinary } from "plist";
import { build, buildBinary, type PlistValue } from "plist";
import { decodeKeyedArchiveBytes } from "./KeyedArchiveReader.js";
import { deeplyNestedXmlPlist } from "../../../tests/fixtures/keyedArchiveXmlNesting.js";
const archive = {
$archiver: "NSKeyedArchiver",
@@ -323,3 +324,66 @@ describe("binary plist reference preflight", () => {
);
});
});
describe("XML plist nesting preflight", () => {
const selection = { offset: 0, limit: 1 };
/** A valid XML NSKeyedArchiver envelope whose object 1 nests `depth` dictionaries. */
const deeplyNestedArchive = (depth: number) => {
let payload: PlistValue = "leaf";
for (let level = 0; level < depth; level += 1) payload = { k: payload };
return Buffer.from(
build({
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: ["$null", payload],
$top: { root: { UID: 1 } },
}),
);
};
/** Follow the single-member chain of a decoded object to its deepest value. */
const deepestValue = (value: unknown): unknown => {
let cursor = value;
while (
cursor !== null &&
typeof cursor === "object" &&
!Array.isArray(cursor) &&
"k" in cursor
) {
cursor = (cursor as Record<string, unknown>).k;
}
return cursor;
};
it.each([129, 200, 3000, 4000])(
"rejects %i-level XML nesting as the documented resource limit",
(depth) => {
// Recursive decoding exceeds the call stack before the projection limit
// can report it, so the decoder must reject the nesting up front with
// the same bounded-resource error an in-reach document would produce.
expect(() =>
decodeKeyedArchiveBytes(
Buffer.from(deeplyNestedXmlPlist(depth)),
selection,
),
).toThrow(new RangeError("Plist exceeds 128 nesting levels"));
},
);
it("decodes a valid XML keyed archive exactly at the nesting limit", () => {
const graph = decodeKeyedArchiveBytes(deeplyNestedArchive(126), {
root: "root",
offset: 1,
limit: 1,
});
expect(graph.objects).toHaveLength(1);
expect(deepestValue(graph.objects[0]?.value)).toBe("leaf");
});
it("rejects one nesting level past the limit before decoding", () => {
expect(() =>
decodeKeyedArchiveBytes(deeplyNestedArchive(127), selection),
).toThrow(new RangeError("Plist exceeds 128 nesting levels"));
});
});
+23 -2
View File
@@ -220,7 +220,18 @@ export const registerFridaCommands = (
(options.scriptFile === undefined)
)
return projectAnalysisError(
new AnalysisInputError(CLI_COMMANDS.instrumentWithFrida),
new AnalysisInputError(
CLI_COMMANDS.instrumentWithFrida,
undefined,
[
{
path: ["script"],
reason: "invalid_value",
message:
"Provide exactly one of --script or --script-file.",
},
],
),
);
const remote = remoteFrom(options);
const selected =
@@ -248,7 +259,17 @@ export const registerFridaCommands = (
};
if (targetInput === undefined)
return projectAnalysisError(
new AnalysisInputError(CLI_COMMANDS.instrumentWithFrida),
new AnalysisInputError(
CLI_COMMANDS.instrumentWithFrida,
undefined,
[
{
path: ["target"],
reason: "invalid_value",
message: "attach mode needs a positive integer process ID.",
},
],
),
);
const result = await withProviderCleanup(() =>
service.instrument(
+12 -10
View File
@@ -709,7 +709,7 @@ export type AdbPackagePullPartialObservation = {
readonly result: z.infer<typeof adbResultSchemas.pull_adb_package>;
};
/** A completed acquisition whose digest read was cancelled before verification. */
/** Acquired file facts retained when publication, digest verification or cleanup fails. */
export type AdbAcquiredFilePartialObservation =
| {
readonly provider_id: "adb";
@@ -727,13 +727,15 @@ export type AdbAcquiredFilePartialObservation =
| {
readonly provider_id: "adb";
readonly operation: "collect_adb_bugreport";
readonly result: {
readonly client: z.infer<
typeof adbResultSchemas.collect_adb_bugreport
>["client"];
readonly serial: string;
readonly local_path: string;
readonly adb_reported_path: string;
readonly digest_status: "unknown";
};
readonly result:
| z.infer<typeof adbResultSchemas.collect_adb_bugreport>
| {
readonly client: z.infer<
typeof adbResultSchemas.collect_adb_bugreport
>["client"];
readonly serial: string;
readonly local_path: string;
readonly adb_reported_path: string;
readonly digest_status: "unknown";
};
};
+6 -1
View File
@@ -2,7 +2,12 @@ import { z } from "zod";
import type { JsonValue } from "../jsonValue.js";
const MAX_PLIST_DEPTH = 128;
/**
* Maximum plist value nesting the JSON projection supports. The XML decode
* preflight shares this limit so oversized documents are rejected before any
* recursive pass rather than during one.
*/
export const MAX_PLIST_DEPTH = 128;
/** A decoded plist projected into JSON, with values JSON cannot express. */
export interface ProjectedPlistValue {
+98
View File
@@ -0,0 +1,98 @@
import { describe, expect, it } from "vitest";
import { parseExecutableHeader } from "./binaryTarget.js";
const header = (
little: boolean,
bits: 32 | 64 = 32,
flags = 0x5,
type = 2,
machine = 243,
): Buffer => {
const bytes = Buffer.alloc(bits === 32 ? 52 : 64);
bytes.set([0x7f, 0x45, 0x4c, 0x46, bits === 32 ? 1 : 2, little ? 1 : 2, 1]);
if (little) {
bytes.writeUInt16LE(type, 16);
bytes.writeUInt16LE(machine, 18);
bytes.writeUInt32LE(flags, bits === 32 ? 36 : 48);
} else {
bytes.writeUInt16BE(type, 16);
bytes.writeUInt16BE(machine, 18);
bytes.writeUInt32BE(flags, bits === 32 ? 36 : 48);
}
return bytes;
};
describe("RISC-V ELF header boundaries", () => {
it.each([
[true, 32],
[false, 32],
[true, 64],
[false, 64],
] as const)(
"retains class, endian, e_type and e_flags (little=%s, class=%s)",
(little, bits) => {
const parsed = parseExecutableHeader(header(little, bits), "x64");
if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value).toEqual({
format: "elf",
architecture: "riscv",
availableArchitectures: ["riscv"],
riscv: {
elfClass: bits,
byteOrder: little ? "little" : "big",
type: 2,
flags: 0x5,
},
});
},
);
it("reads ELF64 flags at their own offset", () => {
const bytes = header(true, 64);
bytes.writeUInt32LE(0xdeadbeef, 36);
bytes.writeUInt32LE(0x00000014, 48);
const parsed = parseExecutableHeader(bytes, "x64");
if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value).toMatchObject({
architecture: "riscv",
riscv: { elfClass: 64, flags: 0x14 },
});
});
it("does not attach ELF metadata to a family that declares none", () => {
const parsed = parseExecutableHeader(header(true, 64, 0x5, 2, 62), "x64");
if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value).toEqual({
format: "elf",
architecture: "x86_64",
availableArchitectures: ["x86_64"],
});
});
it("rejects a truncated header before reading flags", () => {
const parsed = parseExecutableHeader(header(true).subarray(0, 39), "x64");
expect(parsed).toEqual({
ok: false,
error:
"truncated ELF header: an ELF32 header needs 52 bytes; the file has 39",
});
});
it("does not turn invalid byte order into little-endian RISC-V", () => {
const bytes = header(true);
bytes[5] = 0;
expect(parseExecutableHeader(bytes, "x64")).toEqual({
ok: false,
error: "unsupported ELF endianness",
});
});
it("keeps an unknown e_machine outside the recognized family set", () => {
expect(parseExecutableHeader(header(true, 32, 0x5, 2, 244), "x64")).toEqual(
{
ok: false,
error: "unsupported ELF architecture",
},
);
});
});
+45 -34
View File
@@ -1,6 +1,9 @@
import type {
BinaryArchitecture,
ElfFamilyMetadata,
HeaderArchitecture,
MipsElfMetadata,
RiscvElfMetadata,
UniversalMachOSlice,
} from "./binaryTargetTypes.js";
import { err, ok, type Result } from "./result.js";
@@ -15,31 +18,28 @@ export type ExecutableMetadata =
readonly availableArchitectures: readonly "mips"[];
readonly mips: MipsElfMetadata;
}
| {
readonly format: "elf";
readonly architecture: "riscv";
readonly availableArchitectures: readonly "riscv"[];
readonly riscv: RiscvElfMetadata;
}
| {
readonly format: "mach-o" | "elf" | "dos-mz" | "dos-com";
readonly architecture: Exclude<BinaryArchitecture, "mips">;
readonly availableArchitectures: readonly Exclude<
BinaryArchitecture,
"mips"
>[];
readonly architecture: HeaderArchitecture;
readonly availableArchitectures: readonly HeaderArchitecture[];
readonly universalSlice?: never;
}
| {
readonly format: "mach-o-universal";
readonly architecture: Exclude<BinaryArchitecture, "mips">;
readonly availableArchitectures: readonly Exclude<
BinaryArchitecture,
"mips"
>[];
readonly architecture: HeaderArchitecture;
readonly availableArchitectures: readonly HeaderArchitecture[];
readonly universalSlice: UniversalMachOSlice;
}
| {
readonly format: "pe";
readonly architecture: Exclude<BinaryArchitecture, "mips">;
readonly availableArchitectures: readonly Exclude<
BinaryArchitecture,
"mips"
>[];
readonly architecture: HeaderArchitecture;
readonly availableArchitectures: readonly HeaderArchitecture[];
readonly executableRole:
| "application"
| "shared-library"
@@ -115,9 +115,9 @@ const parseFatMachO = (
const entrySize = is64 ? 32 : 20;
if (count === 0 || count > 128 || bytes.length < 8 + count * entrySize)
return err("truncated or invalid FAT architecture table");
const architectures: Exclude<BinaryArchitecture, "mips">[] = [];
const architectures: HeaderArchitecture[] = [];
const slices = new Map<
Exclude<BinaryArchitecture, "mips">,
HeaderArchitecture,
{ readonly offset: bigint; readonly size: bigint }
>();
for (let index = 0; index < count; index += 1) {
@@ -191,20 +191,31 @@ const parseElf = (
const machine = little ? bytes.readUInt16LE(18) : bytes.readUInt16BE(18);
const architecture = elfArchitecture(machine);
if (architecture === undefined) return err("unsupported ELF architecture");
if (architecture === "mips")
return ok({
format: "elf",
architecture,
availableArchitectures: [architecture],
mips: {
elfClass: bytes[4] === 1 ? 32 : 64,
byteOrder: little ? "little" : "big",
type: little ? bytes.readUInt16LE(16) : bytes.readUInt16BE(16),
flags: little
? bytes.readUInt32LE(bytes[4] === 1 ? 36 : 48)
: bytes.readUInt32BE(bytes[4] === 1 ? 36 : 48),
},
});
if (architecture === "mips" || architecture === "riscv") {
const elfClass = bytes[4] === 1 ? 32 : 64;
const flagsOffset = elfClass === 32 ? 36 : 48;
const metadata: ElfFamilyMetadata = {
elfClass,
byteOrder: little ? "little" : "big",
type: little ? bytes.readUInt16LE(16) : bytes.readUInt16BE(16),
flags: little
? bytes.readUInt32LE(flagsOffset)
: bytes.readUInt32BE(flagsOffset),
};
return architecture === "mips"
? ok({
format: "elf",
architecture,
availableArchitectures: [architecture],
mips: metadata,
})
: ok({
format: "elf",
architecture,
availableArchitectures: [architecture],
riscv: metadata,
});
}
return ok({
format: "elf",
architecture,
@@ -299,9 +310,7 @@ const peManagedStatus = (
);
};
const machArchitecture = (
cpu: number,
): Exclude<BinaryArchitecture, "mips"> | undefined => {
const machArchitecture = (cpu: number): HeaderArchitecture | undefined => {
switch (cpu) {
case 7:
return "x86";
@@ -327,6 +336,8 @@ const elfArchitecture = (machine: number): BinaryArchitecture | undefined => {
return "arm";
case 183:
return "arm64";
case 243: // EM_RISCV: XLEN, byte order and e_flags remain separate ELF facts.
return "riscv";
}
return undefined;
};
+48 -7
View File
@@ -5,15 +5,44 @@ export const BINARY_ARCHITECTURES = [
"arm",
"arm64",
"mips",
"riscv",
] as const;
export type BinaryArchitecture = (typeof BINARY_ARCHITECTURES)[number];
/** Raw ELF facts; family recognition does not establish ISA or ABI support. */
export interface MipsElfMetadata {
/**
* ELF header facts for a family whose ISA and ABI are not established by the
* family name alone. Consumers admit a lane from these declarations rather than
* re-reading the header.
*/
export interface ElfFamilyMetadata {
readonly elfClass: 32 | 64;
readonly byteOrder: "little" | "big";
readonly type: number;
readonly flags: number;
}
/** Families whose parsed target variant carries `ElfFamilyMetadata`. */
export const ELF_METADATA_ARCHITECTURES = [
"mips",
"riscv",
] as const satisfies readonly BinaryArchitecture[];
export type ElfMetadataArchitecture =
(typeof ELF_METADATA_ARCHITECTURES)[number];
/** Families whose ISA and ABI need a separate ELF declaration to be admitted. */
export const isElfMetadataArchitecture = (
architecture: BinaryArchitecture,
): architecture is ElfMetadataArchitecture =>
(ELF_METADATA_ARCHITECTURES as readonly string[]).includes(architecture);
/** Recognized family with no ELF metadata member; the header alone is the fact. */
export type HeaderArchitecture = Exclude<
BinaryArchitecture,
ElfMetadataArchitecture
>;
/** Raw ELF facts; family recognition does not establish ISA or ABI support. */
export interface MipsElfMetadata extends ElfFamilyMetadata {
/** Undefined until inspected; null means no ABI record was observed. */
readonly abiFlags?: MipsAbiFlags | null;
/** Unavailable declaration sources; missing records there remain unknown. */
@@ -44,6 +73,12 @@ export interface MipsAbiFlags {
readonly flags2: number;
}
/**
* ELF class and flags retain the width and ABI declarations used by this lane.
* ISA extensions in `.riscv.attributes` are not inspected by this header parser.
*/
export type RiscvElfMetadata = ElfFamilyMetadata;
/** Byte range of the selected slice inside its universal Mach-O container. */
export interface UniversalMachOSlice {
readonly offset: number;
@@ -67,11 +102,8 @@ type NonExecutableMetadata = {
type ExecutableTarget = BinaryTargetIdentity & {
readonly kind: "executable";
readonly architecture: Exclude<BinaryArchitecture, "mips">;
readonly availableArchitectures: readonly Exclude<
BinaryArchitecture,
"mips"
>[];
readonly architecture: HeaderArchitecture;
readonly availableArchitectures: readonly HeaderArchitecture[];
};
/**
@@ -88,6 +120,15 @@ export type BinaryTarget =
readonly executableRole?: never;
readonly managed?: never;
})
| (BinaryTargetIdentity & {
readonly kind: "executable";
readonly format: "elf";
readonly architecture: "riscv";
readonly availableArchitectures: readonly "riscv"[];
readonly riscv: RiscvElfMetadata;
readonly executableRole?: never;
readonly managed?: never;
})
| (ExecutableTarget & {
readonly format: "mach-o" | "elf" | "dos-mz" | "dos-com";
readonly universalSlice?: never;
+2 -2
View File
@@ -1,6 +1,6 @@
import type { BinaryArchitecture } from "./binaryTargetTypes.js";
import type { HeaderArchitecture } from "./binaryTargetTypes.js";
export type PeArchitecture = Exclude<BinaryArchitecture, "mips">;
export type PeArchitecture = HeaderArchitecture;
/** A decoded COFF machine and whether it carried a ReadyToRun OS override. */
export interface PeMachine {
+113
View File
@@ -98,6 +98,119 @@ it("preserves OpenStep dictionary and array autodetection", () => {
});
});
/** The single-member chain of a nested dictionary, down to its deepest value. */
const deepestValue = (value: unknown): unknown => {
let cursor = value;
while (
cursor !== null &&
typeof cursor === "object" &&
!Array.isArray(cursor) &&
"k" in cursor
) {
cursor = (cursor as Record<string, unknown>).k;
}
return cursor;
};
/** The single-member chain of a nested array, down to its deepest value. */
const deepestEntry = (value: unknown): unknown => {
let cursor = value;
while (Array.isArray(cursor) && cursor.length === 1) cursor = cursor[0];
return cursor;
};
const nestedDictionaries = (depth: number) =>
plist(
"<dict><key>k</key>".repeat(depth) +
"<string>x</string>" +
"</dict>".repeat(depth),
);
const nestedArrays = (depth: number) =>
plist(
"<array>".repeat(depth) + "<string>x</string>" + "</array>".repeat(depth),
);
it("rejects XML nesting beyond the projection limit with the limit diagnostic", () => {
// `plist` and the prototype-key traversal recurse once per level, so the
// depth check must report the same bounded-resource error as the projection
// instead of letting those recursive passes exhaust the call stack.
const limit = new RangeError("Plist exceeds 128 nesting levels");
expect(() => parseXmlPropertyList(nestedDictionaries(129))).toThrow(limit);
expect(() => parseXmlPropertyList(nestedArrays(129))).toThrow(limit);
expect(() => parseXmlPropertyList(nestedDictionaries(3000))).toThrow(limit);
expect(() => parseXmlPropertyList(nestedArrays(4000))).toThrow(limit);
});
it("decodes XML property lists exactly at the projection nesting limit", () => {
expect(
deepestValue(parseXmlPropertyList(nestedDictionaries(128)).value),
).toBe("x");
expect(deepestEntry(parseXmlPropertyList(nestedArrays(128)).value)).toBe("x");
expect(
parseXmlPropertyList(
plist("<array>".repeat(128) + "<array/>" + "</array>".repeat(128)),
).value,
).toBeDefined();
});
it("does not count markup inside comments, CDATA, or the doctype as nesting", () => {
const text = `<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd" [<!ENTITY deep "<dict/>">]>
<plist version="1.0"><dict>
<key>a</key><string><![CDATA[<dict><array><dict>]]></string>
<!-- <dict><key>k</key> -->
<key>b</key><array><!-- <array> --><string>x</string></array>
</dict></plist>`;
expect(parseXmlPropertyList(text)).toEqual({
value: { a: "<dict><array><dict>", b: ["x"] },
omittedPrototypeKeys: 0,
});
});
it.each(['"', "'"])(
"counts containers with quoted XML delimiters using %s attributes",
(quote) => {
const opening = `<dict note=${quote}/>${quote}><key>k</key>`;
const text = plist(
opening.repeat(3000) + "<string>x</string>" + "</dict>".repeat(3000),
);
expect(() => parseXmlPropertyList(text)).toThrow(
new RangeError("Plist exceeds 128 nesting levels"),
);
expect(
parseXmlPropertyList(
plist(
"<array>" + `<dict note=${quote}>${quote}/>`.repeat(130) + "</array>",
),
).value,
).toEqual(Array.from({ length: 130 }, () => ({})));
},
);
it("ignores complete DOCTYPE internal subsets when checking value depth", () => {
const text = `<!DOCTYPE plist [
<!ENTITY first "unused">
<!-- <dict><array> -->
<!ENTITY second "${"<dict>".repeat(130)}">
]><plist><string>x</string></plist>`;
expect(parseXmlPropertyList(text)).toEqual({
value: "x",
omittedPrototypeKeys: 0,
});
});
it("bounds nested plist wrappers before recursive decoding", () => {
const text =
"<plist>".repeat(3000) + "<string>x</string>" + "</plist>".repeat(3000);
expect(() => parseXmlPropertyList(text)).toThrow(
new RangeError("Plist exceeds 128 nesting levels"),
);
expect(
parseXmlPropertyList(plist("<plist><string>x</string></plist>")).value,
).toEqual(["x"]);
});
it.each(["\n", "\r\n", "\r", "\u0085", "\u2028", "\u2029"])(
"preserves unrelated source values around %j line endings",
(newline) => {
+60 -7
View File
@@ -1,6 +1,13 @@
import { DOMParser, normalizeLineEndings } from "@xmldom/xmldom";
import {
DOMParser,
normalizeLineEndings,
type Document,
type Node,
} from "@xmldom/xmldom";
import { parse } from "plist";
import { MAX_PLIST_DEPTH } from "./apple/plistValue.js";
/** The dictionary key REA results cannot carry: validation strips it. */
const PROTOTYPE_KEY = "__proto__";
@@ -88,18 +95,64 @@ export const parseXmlPropertyList = (
// by app Info.plist resolution. Only XML needs the key-element substitution.
const first = text.trimStart()[0];
if (first === "{" || first === "(") return omitPrototypeKeys(parse(text));
// Normalize only the literal line endings the pinned decoder normalizes.
// Preserve character references in the source: serializing a DOM would turn
// e.g. an encoded carriage return into a literal newline on the second parse.
const normalized = normalizeLineEndings(text);
// The pinned XML parser is iterative. Reuse its document for the depth
// preflight and key lookup so quoted attributes and declarations have the
// same meaning at both boundaries, before plist's recursive value decode.
const document = new DOMParser().parseFromString(normalized, "text/xml");
assertXmlPlistDepthWithinLimit(document);
const { substituted, placeholder } = substitutePrototypeKeys(
normalizeLineEndings(text),
normalized,
document,
);
return withoutKey(parse(substituted), placeholder);
};
/** Elements whose children the pinned plist value decoder visits recursively. */
const PLIST_CONTAINER_ELEMENTS = new Set(["plist", "dict", "array"]);
/**
* Reject XML plists nested deeper than the projection supports before any
* recursive value pass runs. `plist` and the prototype-key traversal recurse once
* per nesting level, so a deeply nested document exhausts the call stack
* before the projection limit can report the resource bound. The failure
* matches `projectPlistValue` so callers classify it as a limit failure.
*
* Depth counts values the way the projection does: the root value is level 0
* and every container member is one level deeper. A dictionary key names a
* member instead of nesting one, and a self-closing element is the value at
* its own level. The outer plist envelope is unwrapped by the decoder;
* nested plist elements decode as arrays and must count toward the bound.
*/
const assertXmlPlistDepthWithinLimit = (document: Document): void => {
const root = document.documentElement;
if (root === null) return;
const pending: Array<{ node: Node; depth: number }> = [
{ node: root, depth: -1 },
];
while (pending.length > 0) {
const entry = pending.pop();
if (entry === undefined) break;
const { node, depth } = entry;
// Schedule only the next sibling, keeping traversal storage proportional
// to depth rather than the number of members in a wide container.
if (node.nextSibling !== null)
pending.push({ node: node.nextSibling, depth });
if (node.nodeType !== node.ELEMENT_NODE) continue;
if (node.nodeName !== "key" && depth > MAX_PLIST_DEPTH)
throw new RangeError(
`Plist exceeds ${String(MAX_PLIST_DEPTH)} nesting levels`,
);
if (PLIST_CONTAINER_ELEMENTS.has(node.nodeName) && node.firstChild !== null)
pending.push({ node: node.firstChild, depth: depth + 1 });
}
};
/** Locate keys with the same XML parser and first-child semantics as plist. */
const substitutePrototypeKeys = (text: string) => {
// Normalize only the literal line endings the pinned decoder normalizes.
// Preserve character references in the source: serializing a DOM would turn
// e.g. an encoded carriage return into a literal newline on the second parse.
const document = new DOMParser().parseFromString(text, "text/xml");
const substitutePrototypeKeys = (text: string, document: Document) => {
const keys = Array.from(document.getElementsByTagName("key"));
const decodedKeys = new Set(keys.map((key) => key.firstChild?.nodeValue));
let placeholder = "__rea_prototype_key__";
+5 -2
View File
@@ -1,6 +1,9 @@
import { z } from "zod";
import type { BinaryArchitecture } from "./binaryTargetTypes.js";
import type {
BinaryArchitecture,
HeaderArchitecture,
} from "./binaryTargetTypes.js";
import type { ExecutableFormatHint } from "./dosCom.js";
import { err, ok, type Result } from "./result.js";
@@ -55,7 +58,7 @@ export const validateRawBinaryLength = (
/** Admitted headerless load identity used by target resolution and profiles. */
export interface RawBinaryAdmission {
readonly architecture: Exclude<BinaryArchitecture, "mips">;
readonly architecture: HeaderArchitecture;
readonly entryPoint: string;
readonly entryPointSource: string;
}
+18 -6
View File
@@ -31,6 +31,10 @@ import {
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import {
unknownFridaScript,
unknownFridaSession,
} from "../application/frida/fridaInputErrors.js";
const MAX_SESSION_CAPTURE_BYTES = 16 * 1024 * 1024;
const MAX_MESSAGE_METADATA_BYTES = 128 * 1024;
@@ -334,7 +338,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort {
): Promise<Result<FridaScriptObservation, AnalysisError>> {
const managed = this.#sessions.get(sessionId);
if (managed === undefined || managed.state === "detached")
return err(new AnalysisInputError("load_frida_script"));
return err(unknownFridaSession("load_frida_script", sessionId));
let scriptSource: string;
try {
@@ -354,7 +358,13 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort {
cause.code,
{ cause },
)
: new AnalysisInputError("load_frida_script", { cause }),
: new AnalysisInputError("load_frida_script", { cause }, [
{
path: ["path"],
reason: "invalid_value",
message: `Cannot read Frida script file ${sourcePath ?? "<inline>"}.`,
},
]),
);
}
@@ -419,7 +429,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort {
): Promise<Result<null, AnalysisError>> {
const managed = this.#sessions.get(sessionId);
if (managed === undefined || managed.state === "detached")
return err(new AnalysisInputError("resume_frida_session"));
return err(unknownFridaSession("resume_frida_session", sessionId));
if (managed.mode !== "spawn" || managed.state !== "paused")
return err(
new AnalysisCapabilityUnavailableError(
@@ -454,8 +464,10 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort {
): Promise<Result<null, AnalysisError>> {
const managed = this.#sessions.get(sessionId);
const entry = managed?.scripts.get(scriptId);
if (managed === undefined || entry === undefined)
return err(new AnalysisInputError("unload_frida_script"));
if (managed === undefined)
return err(unknownFridaSession("unload_frida_script", sessionId));
if (entry === undefined)
return err(unknownFridaScript("unload_frida_script", scriptId));
const cleanupReason = await this.#releaseScript(managed, entry);
return cleanupReason === undefined
? ok(null)
@@ -536,7 +548,7 @@ export class FridaInstrumentationManager implements FridaInstrumentationPort {
> {
const managed = this.#sessions.get(sessionId);
if (managed === undefined)
return err(new AnalysisInputError("close_frida_session"));
return err(unknownFridaSession("close_frida_session", sessionId));
const scriptFailures = new Map<string, string>();
const cleanupFailures: string[] = [];
let parentSessionReleased =
+27 -7
View File
@@ -17,6 +17,10 @@ import {
ghidraMipsProfileParameters,
ghidraMipsSharedLanguageOverride,
} from "./GhidraMipsProfile.js";
import {
ghidraRiscvLanguageOverride,
ghidraRiscvProfileParameters,
} from "./GhidraRiscvProfile.js";
import { ghidraRawBinaryLanguage } from "./GhidraRawBinaryLanguage.js";
import {
@@ -32,6 +36,17 @@ import {
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
} from "../domain/native/nativeAotPe.js";
/**
* The processor language REA pins for this target, or undefined when Ghidra may
* select from the header. The committed profile and the session readback both
* compare against this single owner rather than re-deriving a lane.
*/
export const ghidraPinnedLanguage = (
target: BinaryTarget,
): GhidraLanguageOverride | undefined =>
ghidraMipsSharedLanguageOverride(target) ??
ghidraRiscvLanguageOverride(target);
/** Resolve version-bound, deterministic semantics before Ghidra imports a target. */
export const resolveGhidraAnalysisProfile = async (
target: BinaryTarget,
@@ -136,22 +151,26 @@ export const resolveGhidraAnalysisProfile = async (
const dosMz = target.format === "dos-mz";
const dosCom = target.format === "dos-com";
const dos = dosMz || dosCom;
const mipsLanguage = ghidraMipsSharedLanguageOverride(target);
const pinnedLanguage = ghidraPinnedLanguage(target);
if (
mipsLanguage !== undefined &&
pinnedLanguage !== undefined &&
languageOverride !== undefined &&
(languageOverride.languageId !== mipsLanguage.languageId ||
(languageOverride.languageId !== pinnedLanguage.languageId ||
(languageOverride.compilerSpecId !== undefined &&
languageOverride.compilerSpecId !== mipsLanguage.compilerSpecId))
languageOverride.compilerSpecId !== pinnedLanguage.compilerSpecId))
)
return err(
new AnalysisUnsupportedTargetError(
"resolve_analysis_profile",
target.path,
"The caller-selected processor or compiler specification is outside the verified endian-matched MIPS32 shared-image profile. The requested override has not been changed.",
`The caller-selected processor or compiler specification is outside the verified ${
target.architecture === "riscv"
? "generic little-endian RISC-V"
: "endian-matched MIPS32 shared-image"
} profile. The requested override has not been changed.`,
),
);
const selectedLanguage = languageOverride ?? mipsLanguage;
const selectedLanguage = languageOverride ?? pinnedLanguage;
// DOS imports already force an admitted loader, language, and context.
if (dos && languageOverride !== undefined)
return Promise.resolve(
@@ -227,7 +246,7 @@ export const resolveGhidraAnalysisProfile = async (
? "auto-default"
: (selectedLanguage.compilerSpecId ?? "language-default"),
...(languageOverride === undefined
? mipsLanguage === undefined
? pinnedLanguage === undefined
? {}
: { language_selection: "provider-pinned-v1" }
: { language_selection: "configured-v1" }),
@@ -251,6 +270,7 @@ export const resolveGhidraAnalysisProfile = async (
: {}),
...analyzerParameters,
...ghidraMipsProfileParameters(target),
...ghidraRiscvProfileParameters(target),
...(pspExtension?.ok === true
? {
mips_support_lane: "psp-elf32-exec-eabi32-allegrex-v2",
+4 -1
View File
@@ -22,6 +22,7 @@ const WINDOWS_INSTALL = "C:\\tools\\ghidra_12.1.4_PUBLIC";
const WINDOWS_PROPERTIES = `${WINDOWS_INSTALL}\\Ghidra\\application.properties`;
const WINDOWS_HEADLESS = `${WINDOWS_INSTALL}\\support\\analyzeHeadless.bat`;
const WINDOWS_DECOMPILER = `${WINDOWS_INSTALL}\\Ghidra\\Features\\Decompiler\\os\\win_x86_64\\decompile.exe`;
const JAVA: GhidraJavaObservation = {
version: "21.0.11",
major: SUPPORTED_GHIDRA_JAVA_MAJOR,
@@ -43,7 +44,9 @@ const host = (
...overrides,
});
afterEach(() => vi.unstubAllEnvs());
afterEach(() => {
vi.unstubAllEnvs();
});
it("probes the selected Java environment without reading conflicting ambient settings", () => {
vi.stubEnv("REA_GHIDRA_CONTEXT", "ambient");
+10 -1
View File
@@ -734,7 +734,16 @@ const systemGhidraInstallationHost = (): GhidraInstallationHost => ({
probeJava(command, environment) {
const observed = spawnSync(
command,
["-XX:-UsePerfData", "-XshowSettings:properties", "-version"],
[
"-Dstdout.encoding=UTF-8",
"-Dstderr.encoding=UTF-8",
// JDK 17/18 use the legacy names; JDK 19+ use the names above.
"-Dsun.stdout.encoding=UTF-8",
"-Dsun.stderr.encoding=UTF-8",
"-XX:-UsePerfData",
"-XshowSettings:properties",
"-version",
],
{
encoding: "utf8",
env: environment,
+1
View File
@@ -66,6 +66,7 @@ const SUPPORTED_ARCHITECTURES = new Set([
"arm",
"arm64",
"mips",
"riscv",
]);
/** Ghidra candidate backed by an isolated ephemeral headless import. */
+4 -2
View File
@@ -14,6 +14,7 @@ import {
ghidraSeedLimitations,
} from "./GhidraAnalysisSeeds.js";
import {
ghidraPinnedLanguage,
ghidraProfileDisabledAnalyzers,
ghidraProfileLanguageOverride,
sameGhidraDisabledAnalyzers,
@@ -61,10 +62,10 @@ import {
ghidraMipsSoftFloatFailure,
ghidraMipsRelocatableLimitations,
ghidraMipsSharedLimitations,
ghidraMipsSharedLanguageOverride,
ghidraMipsSoftFloatLimitations,
isGhidraMipsSoftFloatTarget,
} from "./GhidraMipsProfile.js";
import { ghidraRiscvLimitations } from "./GhidraRiscvProfile.js";
import { unverifiedGhidraBuildLimitation } from "./GhidraInstallationPolicy.js";
import { GhidraHeadlessLauncher } from "./GhidraLauncher.js";
import { attestGhidraNativeLoadImage } from "./GhidraLoadImageAttest.js";
@@ -453,7 +454,7 @@ export const createGhidraProviderClient = (input: {
if (
!sameGhidraLanguageOverride(
languageOverride,
config.ghidraLanguageOverride ?? ghidraMipsSharedLanguageOverride(target),
config.ghidraLanguageOverride ?? ghidraPinnedLanguage(target),
)
)
return unavailableClient(
@@ -523,6 +524,7 @@ export const createGhidraProviderClient = (input: {
: []),
...ghidraMipsRelocatableLimitations(target),
...ghidraMipsSharedLimitations(target),
...ghidraRiscvLimitations(target),
];
const providerLimitations =
installation.platform === "win32"
+3
View File
@@ -1,5 +1,6 @@
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { ghidraMipsUnsupportedReason } from "./GhidraMipsProfile.js";
import { ghidraRiscvUnsupportedReason } from "./GhidraRiscvProfile.js";
/** Allegrex's ELF machine declaration, not a filename or inferred instruction. */
export const isGhidraPspTarget = (target: BinaryTarget): boolean =>
@@ -13,6 +14,8 @@ export const isGhidraPspTarget = (target: BinaryTarget): boolean =>
export const ghidraProcessorUnsupportedReason = (
target: BinaryTarget,
): string | null => {
if (target.kind === "executable" && target.architecture === "riscv")
return ghidraRiscvUnsupportedReason(target);
if (
target.kind !== "executable" ||
target.architecture !== "mips" ||
+2 -2
View File
@@ -1,4 +1,4 @@
import type { BinaryArchitecture } from "../domain/binaryTargetTypes.js";
import type { HeaderArchitecture } from "../domain/binaryTargetTypes.js";
/** Bound Ghidra language for a caller-supplied raw-binary architecture. */
export interface GhidraRawBinaryLanguage {
@@ -13,7 +13,7 @@ export interface GhidraRawBinaryLanguage {
* Big-endian, Thumb-only, and Cortex-M variants are out of scope.
*/
export const ghidraRawBinaryLanguage = (
architecture: Exclude<BinaryArchitecture, "mips">,
architecture: HeaderArchitecture,
): GhidraRawBinaryLanguage => {
switch (architecture) {
case "x86":
+256
View File
@@ -0,0 +1,256 @@
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { parseConfig } from "../config/parseConfig.js";
import type {
BinaryTarget,
RiscvElfMetadata,
} from "../domain/binaryTargetTypes.js";
import { silentLogger } from "../logger.js";
import { HopperProvider } from "../hopper/HopperProvider.js";
import { hopperLoaderArgsForTarget } from "../hopper/HopperAnalysisProfile.js";
import { IdaProvider } from "../ida/IdaProvider.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { ghidraRiscvLimitations } from "./GhidraRiscvProfile.js";
import type { GhidraInstallationHost } from "./GhidraInstallation.js";
import {
ghidraPinnedLanguage,
ghidraProfileLanguageOverride,
sameGhidraLanguageOverride,
} from "./GhidraAnalysisProfile.js";
// Abbreviated from Ghidra 12.1.4 Processors/RISCV/data/languages/riscv.ldefs.
// The release tree ships no big-endian or ELF-class-mismatched RISC-V language.
const RISCV_LDEFS = `<?xml version="1.0" encoding="UTF-8"?>
<language_definitions>
<language processor="RISCV"
endian="little"
size="64"
variant="default"
version="1.4"
slafile="riscv.lp64d.sla"
processorspec="RV64.pspec"
id="RISCV:LE:64:default">
<description>RISC-V 64 little default</description>
<compiler name="gcc" spec="riscv64-fp.cspec" id="gcc"/>
</language>
<language processor="RISCV"
endian="little"
size="32"
variant="default"
version="1.4"
slafile="riscv.ilp32d.sla"
processorspec="RV32.pspec"
id="RISCV:LE:32:default">
<description>RISC-V 32 little default</description>
<compiler name="gcc" spec="riscv32-fp.cspec" id="gcc"/>
</language>
</language_definitions>
`;
// Fake installation inspection tests admission/profile contracts, not Ghidra.
const host: GhidraInstallationHost = {
platform: "linux",
architecture: "x64",
readText: () => "application.version=12.1.4\n",
executable: () => true,
probeJava: () => ({
version: "21.0.11",
major: 21,
home: "/jdk-21",
bits: 64,
runtime: "jdk",
}),
};
const configOf = (
installDir: string,
environment: Record<string, string> = {},
) => {
const parsed = parseConfig({
GHIDRA_INSTALL_DIR: installDir,
...environment,
});
if (!parsed.ok) throw parsed.error;
return parsed.value;
};
const target = (change: Partial<RiscvElfMetadata> = {}): BinaryTarget => ({
path: "/fixture.elf",
sha256: "a".repeat(64),
kind: "executable",
format: "elf",
architecture: "riscv",
availableArchitectures: ["riscv"],
riscv: {
elfClass: 64,
byteOrder: "little",
type: 2,
flags: 0x5,
...change,
},
});
let installDir = "";
beforeAll(async () => {
installDir = await mkdtemp(join(tmpdir(), "rea-ghidra-riscv-languages-"));
const languages = join(installDir, "Ghidra/Processors/RISCV/data/languages");
await mkdir(languages, { recursive: true });
await writeFile(join(languages, "riscv.ldefs"), RISCV_LDEFS);
});
afterAll(async () => {
await rm(installDir, { recursive: true, force: true });
});
const ghidra = (environment: Record<string, string> = {}) =>
new GhidraProvider(configOf(installDir, environment), silentLogger, {}, host);
describe("bounded Ghidra RISC-V admission", () => {
it("pins the generic language per width instead of accepting the header guess", async () => {
const provider = ghidra();
const rv64 = target();
const rv32 = target({ elfClass: 32, flags: 0x4 });
expect(provider.inspectTargetSupport(rv64).status).toBe("supported");
expect(provider.inspectTargetSupport(rv32).status).toBe("supported");
expect(ghidraPinnedLanguage(rv64)).toEqual({
languageId: "RISCV:LE:64:default",
compilerSpecId: "gcc",
});
expect(ghidraPinnedLanguage(rv32)).toEqual({
languageId: "RISCV:LE:32:default",
compilerSpecId: "gcc",
});
const wide = await provider.resolveAnalysisProfile(rv64);
const narrow = await provider.resolveAnalysisProfile(rv32);
if (!wide.ok || !narrow.ok)
throw new Error("expected admitted RISC-V profiles");
expect(wide.value.profile?.parameters.riscv_elf).toEqual({
elf_class: 64,
byte_order: "little",
type: 2,
flags: 5,
float_abi_double: true,
rvc_declared: true,
rve_declared: false,
tso_declared: false,
});
expect(wide.value.profile?.parameters.riscv_support_lane).toBe(
"elf64-exec-le-double-float-generic-v1",
);
expect(narrow.value.profile?.parameters.riscv_support_lane).toBe(
"elf32-exec-le-double-float-generic-v1",
);
expect(narrow.value.profile?.parameters.riscv_elf).toMatchObject({
elf_class: 32,
rvc_declared: false,
});
expect(wide.value.profile?.digest).not.toBe(narrow.value.profile?.digest);
});
it("commits the pinned language the session readback expects", async () => {
// A pinned lane has one owner: the committed profile and the open_binary
// readback must agree without either re-deriving a different family's pin.
const provider = ghidra();
for (const value of [target(), target({ elfClass: 32 })]) {
const resolved = await provider.resolveAnalysisProfile(value);
if (!resolved.ok || resolved.value.profile === null)
throw new Error("expected a RISC-V profile");
expect(
sameGhidraLanguageOverride(
ghidraProfileLanguageOverride(resolved.value.profile.parameters),
ghidraPinnedLanguage(value),
),
).toBe(true);
}
});
it.each([
[{ byteOrder: "big" }, "little-endian"],
[{ type: 3 }, "fixed-address ET_EXEC only"],
[{ type: 1 }, "fixed-address ET_EXEC only"],
[{ type: 0xffa0 }, "e_type 65440"],
[{ flags: 0xc }, "EF_RISCV_RVE"],
[{ flags: 0x14 }, "EF_RISCV_TSO"],
[{ flags: 0x1 }, "soft-float convention"],
[{ flags: 0x3 }, "single-float"],
[{ flags: 0x7 }, "quad-float"],
[{ flags: 0x8000_0005 }, "(0x80000000)"],
[{ flags: 0x24 }, "bits outside the verified lane"],
] satisfies [Partial<RiscvElfMetadata>, string][])(
"refuses an unverified RISC-V interpretation %j",
async (change, reason) => {
const provider = ghidra();
const value = target(change);
expect(provider.inspectTargetSupport(value)).toMatchObject({
status: "unsupported",
code: "architecture_unsupported",
reason: expect.stringContaining(reason),
});
expect(await provider.resolveAnalysisProfile(value)).toEqual(
expect.objectContaining({ ok: false }),
);
expect(ghidraPinnedLanguage(value)).toBeUndefined();
},
);
it("refuses a caller-selected processor outside the verified generic lane", async () => {
const resolved = await ghidra({
REA_GHIDRA_LANGUAGE_ID: "RISCV:LE:32:AndeStar_v5",
}).resolveAnalysisProfile(target({ elfClass: 32 }));
expect(resolved.ok).toBe(false);
if (resolved.ok) throw new Error("expected a refusal");
expect(resolved.error.message).toContain("generic little-endian RISC-V");
});
it("keeps a caller-selected language that matches the pin as configured", async () => {
const resolved = await ghidra({
REA_GHIDRA_LANGUAGE_ID: "RISCV:LE:64:default",
REA_GHIDRA_COMPILER_SPEC_ID: "gcc",
}).resolveAnalysisProfile(target());
if (!resolved.ok || resolved.value.profile === null)
throw new Error("expected a configured profile");
expect(resolved.value.profile.parameters.language_selection).toBe(
"configured-v1",
);
expect(resolved.value.profile.parameters.language_id).toBe(
"RISCV:LE:64:default",
);
});
it("states the linkage claim boundary in the lane limitations", () => {
const notes = ghidraRiscvLimitations(target());
expect(notes).toHaveLength(3);
expect(notes.join(" ")).toContain("does not establish static linkage");
expect(notes[0]).toContain("RISCV:LE:64:default");
expect(
ghidraRiscvLimitations({
path: "/other.elf",
sha256: "b".repeat(64),
kind: "executable",
format: "elf",
architecture: "x86_64",
availableArchitectures: ["x86_64"],
}),
).toEqual([]);
});
it("keeps RISC-V out of unverified Hopper and IDA adapter lanes", () => {
const hopper = new HopperProvider(
configOf("/hopper"),
silentLogger,
{},
"linux",
);
expect(hopper.inspectTargetSupport(target()).status).toBe("unsupported");
expect(hopperLoaderArgsForTarget(target()).ok).toBe(false);
expect(hopperLoaderArgsForTarget(target())).toEqual(
expect.objectContaining({
ok: false,
error: expect.objectContaining({
message: expect.stringContaining("RISC-V"),
}),
}),
);
const ida = new IdaProvider(configOf("/ida"), {});
expect(ida.inspectTargetSupport(target()).status).toBe("unsupported");
});
});
+129
View File
@@ -0,0 +1,129 @@
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { GhidraLanguageOverride } from "../config/ghidraLanguageOverride.js";
/**
* Ghidra's generic RISC-V languages model the double-float ABI: the shipped
* `riscv.ldefs` maps `RISCV:LE:{32,64}:default` to the `riscv{32,64}-fp.cspec`
* compiler specification. Soft, single and quad ABI declarations therefore need
* their own verification rather than inheriting this lane.
*/
const EF_RISCV_RVC = 0x0001;
const EF_RISCV_FLOAT_ABI = 0x0006;
const EF_RISCV_FLOAT_ABI_DOUBLE = 0x0004;
const EF_RISCV_RVE = 0x0008;
const EF_RISCV_TSO = 0x0010;
const ET_REL = 1;
const ET_EXEC = 2;
const ET_DYN = 3;
/**
* Bits this lane reads. Anything else in `e_flags` — reserved today or defined by
* a newer psABI — is refused rather than assumed compatible.
*/
const EF_RISCV_DECLARED_BITS = EF_RISCV_RVC | EF_RISCV_FLOAT_ABI;
/** Header declarations outside the verified generic-language lane. */
const riscvUnsupportedFloatAbi = (abi: number): string =>
abi === 0
? "unspecified (soft-float convention)"
: abi === 0x0002
? "single-float"
: "quad-float";
/**
* Keep family recognition separate from the deliberately bounded Ghidra lane.
*
* Ghidra 12.1.4 selects `RISCV:LE:32:AndeStar_v5` for an ordinary RV32 ELF: its
* `riscv.opinion` pins `variant="default"` for the 64-bit constraint and leaves
* the 32-bit one with the comment that the loader "must choose between 'default'
* and 'AndeStar' variants". The lane therefore pins the language instead of
* accepting the header-based guess.
*/
export const ghidraRiscvUnsupportedReason = (
target: BinaryTarget,
): string | null => {
if (target.kind !== "executable" || target.architecture !== "riscv")
return null;
const metadata = target.riscv;
if (target.format !== "elf")
return "RISC-V analysis requires ELF header metadata; a family name alone does not select an ISA or ABI.";
if (metadata.byteOrder !== "little")
return "This Ghidra RISC-V lane supports little-endian images only; Ghidra 12.1.4 ships no big-endian RISC-V language.";
if (metadata.elfClass !== 32 && metadata.elfClass !== 64)
return "This Ghidra RISC-V lane requires ELF32 or ELF64; the class decides which generic language is pinned.";
if ((metadata.flags & EF_RISCV_RVE) !== 0)
return "EF_RISCV_RVE declares the embedded 15-register ABI, which needs its own processor specification and compiler verification.";
if ((metadata.flags & EF_RISCV_TSO) !== 0)
return "EF_RISCV_TSO declares the total-store-order memory model; this lane verifies the baseline model only.";
if (metadata.type !== ET_EXEC)
return `This Ghidra RISC-V lane supports fixed-address ET_EXEC only; ${
metadata.type === ET_REL
? "relocatable"
: metadata.type === ET_DYN
? "shared and position-independent"
: `e_type ${metadata.type}`
} images need separate load verification. ET_EXEC alone does not establish static linkage, which this lane does not inspect.`;
const abi = metadata.flags & EF_RISCV_FLOAT_ABI;
if (abi !== EF_RISCV_FLOAT_ABI_DOUBLE)
return `This Ghidra RISC-V lane requires the EF_RISCV_FLOAT_ABI_DOUBLE declaration (${riscvUnsupportedFloatAbi(
abi,
)} observed); the pinned generic language uses the double-float compiler specification.`;
const unsupportedFlags = (metadata.flags & ~EF_RISCV_DECLARED_BITS) >>> 0;
if (unsupportedFlags !== 0)
return `RISC-V e_flags declares bits outside the verified lane (0x${unsupportedFlags.toString(16)}); reserved or newer declarations are not treated as compatible with this profile.`;
return null;
};
/** Pin a language only for declarations admitted by the same lane boundary. */
export const ghidraRiscvLanguageOverride = (
target: BinaryTarget,
): GhidraLanguageOverride | undefined =>
target.kind === "executable" &&
target.architecture === "riscv" &&
ghidraRiscvUnsupportedReason(target) === null
? {
languageId: `RISCV:LE:${target.riscv.elfClass}:default`,
compilerSpecId: "gcc",
}
: undefined;
/** Commit source ELF interpretation without confusing target ISA with host CPU. */
export const ghidraRiscvProfileParameters = (
target: BinaryTarget,
): Readonly<Record<string, JsonValue>> =>
target.kind === "executable" &&
target.architecture === "riscv" &&
target.riscv !== undefined
? {
riscv_elf: {
elf_class: target.riscv.elfClass,
byte_order: target.riscv.byteOrder,
type: target.riscv.type,
flags: target.riscv.flags,
float_abi_double:
(target.riscv.flags & EF_RISCV_FLOAT_ABI) ===
EF_RISCV_FLOAT_ABI_DOUBLE,
rvc_declared: (target.riscv.flags & EF_RISCV_RVC) !== 0,
rve_declared: (target.riscv.flags & EF_RISCV_RVE) !== 0,
tso_declared: (target.riscv.flags & EF_RISCV_TSO) !== 0,
},
riscv_support_lane:
target.riscv.elfClass === 32
? "elf32-exec-le-double-float-generic-v1"
: "elf64-exec-le-double-float-generic-v1",
}
: {};
/** The pinned language is an analysis convention, not proof of the producer. */
export const ghidraRiscvLimitations = (
target: BinaryTarget,
): readonly string[] =>
target.kind === "executable" && target.architecture === "riscv"
? [
`RISC-V ELF: REA pins Ghidra's generic \`RISCV:LE:${target.riscv.elfClass}:default\` language with the \`gcc\` compiler specification instead of accepting the loader's header-based choice, which selects \`RISCV:LE:32:AndeStar_v5\` for ordinary RV32 images on Ghidra 12.1.4.`,
"The pinned generic language models the double-float ABI declared by `e_flags`; the declaration is recorded as an ELF fact and is not evidence of the original compiler or of verified instruction semantics.",
"Admission reads `e_type` only: a fixed-address ET_EXEC declaration does not establish static linkage, and program headers (`PT_INTERP`, dynamic dependencies) are not inspected by this lane.",
]
: [];
+15 -6
View File
@@ -8,8 +8,10 @@ import type {
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import {
isMachOFormat,
isElfMetadataArchitecture,
type BinaryTarget,
type BinaryArchitecture,
type ElfMetadataArchitecture,
type HeaderArchitecture,
} from "../domain/binaryTargetTypes.js";
import {
AnalysisCancelledError,
@@ -101,6 +103,15 @@ export const resolveHopperAnalysisProfile = async (
});
};
/** Ghidra owns these lanes; Hopper names the family it declines rather than guessing. */
const HOPPER_DECLINED_ARCHITECTURE_LABELS: Record<
ElfMetadataArchitecture,
string
> = {
mips: "MIPS",
riscv: "RISC-V",
};
/** Derive Hopper's complete non-interactive CLI loader selection. */
export const hopperLoaderArgsForTarget = (
target: BinaryTarget,
@@ -113,12 +124,12 @@ export const hopperLoaderArgsForTarget = (
if (target.kind !== "executable")
return err(new ProviderAdapterError("hopper", "resolve_analysis_profile"));
const architecture = target.architecture;
if (architecture === "mips")
if (isElfMetadataArchitecture(architecture))
return err(
new AnalysisCapabilityUnavailableError(
"hopper",
"resolve_analysis_profile",
"REA's Hopper adapter does not admit MIPS targets. Select a verified Ghidra MIPS profile instead.",
`REA's Hopper adapter does not admit ${HOPPER_DECLINED_ARCHITECTURE_LABELS[architecture]} targets. Select a verified Ghidra ${HOPPER_DECLINED_ARCHITECTURE_LABELS[architecture]} profile instead.`,
),
);
const flag = hopperArchitectureFlag(architecture);
@@ -151,9 +162,7 @@ export const hopperLoaderArgsForTarget = (
}
};
const hopperArchitectureFlag = (
architecture: Exclude<BinaryArchitecture, "mips">,
): string => {
const hopperArchitectureFlag = (architecture: HeaderArchitecture): string => {
switch (architecture) {
case "x86":
return "--intel-32";
+5 -1
View File
@@ -4,6 +4,7 @@ import { z } from "zod";
import {
isMachOFormat,
isElfMetadataArchitecture,
type BinaryTarget,
} from "../domain/binaryTargetTypes.js";
import { BinaryTargetError } from "../domain/configurationErrors.js";
@@ -47,7 +48,10 @@ export const resolveHopperMachOImage = async (
),
);
if (signal?.aborted) return err(new AnalysisCancelledError("open_binary"));
if (!isMachOFormat(target.format) || target.architecture === "mips")
if (
!isMachOFormat(target.format) ||
isElfMetadataArchitecture(target.architecture)
)
return invalid("Hopper FAT64 image preparation requires a Mach-O target");
let file: FileHandle;
try {
+8 -3
View File
@@ -16,7 +16,10 @@ import type {
} from "../application/AnalysisProvider.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import type { AppConfig } from "../config/types.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
isElfMetadataArchitecture,
type BinaryTarget,
} from "../domain/binaryTargetTypes.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { Logger } from "pino";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
@@ -118,11 +121,13 @@ export class HopperProvider implements AnalysisProviderCandidate {
reason: null,
diagnostics,
};
if (target.architecture === "mips")
if (isElfMetadataArchitecture(target.architecture))
return {
status: "unsupported",
code: "architecture_unsupported",
reason: "REA's Hopper adapter does not admit MIPS targets.",
reason: `REA's Hopper adapter does not admit ${
target.architecture === "mips" ? "MIPS" : "RISC-V"
} targets.`,
diagnostics,
};
if (target.format === "raw-binary")
+37 -27
View File
@@ -6,43 +6,53 @@ import { err, ok, type Result } from "../domain/result.js";
const common = {
mode: z.enum(["attached", "headless"]).default("attached"),
protocol: z.literal("native").optional(),
timeoutMs: z.number().int().positive().max(2_147_483_647).default(300_000),
workspaceRoot: z
.string()
.refine(isAbsolute, "workspaceRoot must be absolute on the REA host")
.optional(),
};
const registrationSchema = z.union([
z
.object({
...common,
type: z.literal("stdio").optional(),
command: z.string().min(1),
args: z.array(z.string()).default([]),
env: z.record(z.string(), z.string()).default({}),
})
.strict(),
z
.object({
...common,
type: z.literal("http").optional(),
url: z.url().refine((value) => {
const url = new URL(value);
return (
["http:", "https:"].includes(url.protocol) &&
["localhost", "127.0.0.1", "[::1]"].includes(url.hostname) &&
url.username === "" &&
url.password === ""
);
}, "IDA MCP must use a local HTTP endpoint without URL credentials"),
headers: z.record(z.string(), z.string()).default({}),
})
.strict(),
]);
const registrationSchema = z
.union([
z
.object({
...common,
type: z.literal("stdio").optional(),
command: z.string().min(1),
args: z.array(z.string()).default([]),
env: z.record(z.string(), z.string()).default({}),
})
.strict(),
z
.object({
...common,
type: z.literal("http").optional(),
url: z.url().refine((value) => {
const url = new URL(value);
return (
["http:", "https:"].includes(url.protocol) &&
["localhost", "127.0.0.1", "[::1]"].includes(url.hostname) &&
url.username === "" &&
url.password === ""
);
}, "IDA MCP must use a local HTTP endpoint without URL credentials"),
headers: z.record(z.string(), z.string()).default({}),
})
.strict(),
])
.refine(
(config) => config.protocol !== "native" || config.mode === "attached",
"The native IDA MCP protocol requires attached mode",
);
/** An upstream MCP registration plus REA's database lifecycle selection. */
export type IdaConfiguration = z.infer<typeof registrationSchema>;
/** Select a protocol without changing existing attached/headless registrations. */
export const idaProtocol = (config: IdaConfiguration) =>
config.protocol ?? (config.mode === "attached" ? "legacy" : "modern");
/** Read one explicit registration without starting or installing a provider. */
export const readIdaConfiguration = (
path: string,
+46 -1
View File
@@ -19,6 +19,7 @@ export interface IdaMcpConnection {
name: string,
args: Readonly<Record<string, JsonValue>>,
): Promise<JsonValue>;
readResource?(uri: string): Promise<JsonValue>;
serverInfo(): JsonValue;
close(): Promise<void>;
}
@@ -43,8 +44,23 @@ const toolResultSchema = z.object({
.optional(),
});
const parseIdaJsonText = (text: string): JsonValue => {
let value: unknown;
try {
value = JSON.parse(text);
} catch (cause: unknown) {
throw new AnalysisProtocolError("IDA MCP returned malformed JSON text.", {
cause,
});
}
return jsonValueSchema.parse(value);
};
/** Decode Python FastMCP's scalar wrapper without dropping structured dictionaries. */
export const decodeIdaToolResult = (input: unknown): JsonValue => {
export const decodeIdaToolResult = (
input: unknown,
protocol?: "native",
): JsonValue => {
const result = toolResultSchema.parse(input);
const blocks = result.content.flatMap((block) =>
block.type === "text" && block.text !== undefined ? [block.text] : [],
@@ -57,6 +73,15 @@ export const decodeIdaToolResult = (input: unknown): JsonValue => {
throw new AnalysisProtocolError(
`IDA MCP truncated its output; the preview cannot establish a complete observation. ${result._meta.ida_mcp.download_hint ?? "Use the upstream output download workflow to retrieve the full result."}`,
);
// Native MCP retains the complete JSON text while structuredContent may
// contain a shortened preview. Parse that producer-owned representation once.
if (protocol === "native") {
if (blocks.length !== 1)
throw new AnalysisProtocolError(
"Native IDA MCP omitted its complete JSON text result.",
);
return parseIdaJsonText(blocks[0] ?? "");
}
if (result.structuredContent !== undefined) {
const value = result.structuredContent;
return jsonValueSchema.parse(
@@ -156,8 +181,28 @@ export const createIdaMcpConnection = (
await client
.callTool({ name, arguments: args }, { timeout: config.timeoutMs })
.catch(transportFailure),
config.protocol,
);
},
async readResource(uri) {
const response = await client
.readResource({ uri }, { timeout: config.timeoutMs })
.catch(transportFailure);
const resource = z
.object({
contents: z
.array(
z.object({
uri: z.literal(uri),
mimeType: z.literal("application/json"),
text: z.string(),
}),
)
.length(1),
})
.parse(response);
return parseIdaJsonText(resource.contents[0]?.text ?? "");
},
serverInfo: () => jsonValueSchema.parse(client.getServerVersion() ?? null),
close: () => client.close().catch(transportFailure),
};
+629
View File
@@ -0,0 +1,629 @@
import { rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
createIdaTarget,
RecordingIdaMcp,
} from "../../tests/fixtures/idaMcp.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
AnalysisInputError,
AnalysisProtocolError,
} from "../domain/analysisErrorCore.js";
import { readIdaConfiguration } from "./IdaConfiguration.js";
import {
decodeIdaToolResult,
type IdaMcpConnection,
} from "./IdaMcpConnection.js";
import { parseConfig } from "../config/parseConfig.js";
import { IdaProvider } from "./IdaProvider.js";
import { idaCapabilities } from "./IdaProviderCapabilities.js";
import { IdaSessionClient } from "./IdaSessionClient.js";
const roots: string[] = [];
afterEach(async () => {
for (const root of roots.splice(0))
await rm(root, { recursive: true, force: true });
});
const fixture = async () => {
const { root, target } = await createIdaTarget();
roots.push(root);
const producer = new RecordingIdaMcp(target);
producer.overrideCall = (name) => {
if (name === "decompile")
return {
addr: "0x1000",
ok: true,
pseudocode: producer.pseudocode,
lines: 1,
};
if (name === "disasm")
return {
asm: {
name: "main",
start_ea: "0x1000",
lines: "main (.text @ 0x1000):\n1000 ret",
},
cursor: { done: true },
};
if (name === "callees")
return [
{
callees: [{ addr: "0x2000", name: "puts", type: "external" }],
more: false,
error: null,
},
];
if (name === "xrefs_to")
return [
{
xrefs: [{ addr: "0x1004", type: "code", fn: null }],
more: false,
error: null,
},
];
return undefined;
};
const resources: string[] = [];
const connection: IdaMcpConnection = {
connect: async () => [
"list_funcs",
"lookup_funcs",
"find_regex",
"decompile",
"disasm",
"callees",
"xrefs_to",
],
call: (name, args) => producer.call(name, args),
readResource: async (uri) => {
resources.push(uri);
const metadata = await producer.call("get_metadata", {});
if (
metadata === null ||
typeof metadata !== "object" ||
Array.isArray(metadata)
)
throw new Error("Invalid metadata fixture");
return { ...metadata, path: `${producer.inputPath}.i64` };
},
serverInfo: () => producer.serverInfo(),
close: () => producer.close(),
};
const client = new IdaSessionClient(
{
url: "http://127.0.0.1:13337/mcp",
headers: {},
mode: "attached",
protocol: "native",
timeoutMs: 1000,
},
target,
connection,
);
return { producer, client, target, resources, root };
};
describe("native IDA attached protocol", () => {
it("requires explicit opt-in and rejects native headless registrations", async () => {
const { root } = await fixture();
const path = join(root, "native.json");
await writeFile(
path,
JSON.stringify({ url: "http://127.0.0.1:13337/mcp", protocol: "native" }),
);
expect(readIdaConfiguration(path)).toMatchObject({
ok: true,
value: { mode: "attached", protocol: "native" },
});
await writeFile(
path,
JSON.stringify({
url: "http://127.0.0.1:13337/mcp",
protocol: "native",
mode: "headless",
}),
);
expect(readIdaConfiguration(path).ok).toBe(false);
await writeFile(path, JSON.stringify({ command: "python" }));
const old = readIdaConfiguration(path);
expect(old.ok && old.value.protocol).toBeUndefined();
});
it("commits native profile semantics and advertises GUI effects without starting a provider", async () => {
const { root, target, producer } = await fixture();
const path = join(root, "profile.json");
await writeFile(
path,
JSON.stringify({ url: "http://127.0.0.1:13337/mcp", protocol: "native" }),
);
const config = parseConfig({ REA_IDA_MCP_CONFIG: path });
if (!config.ok) throw config.error;
const provider = new IdaProvider(config.value, {}, () => {
throw new Error("Unexpected discovery startup");
});
expect(provider.inspectAvailability().status).toBe("available");
const profile = await provider.resolveAnalysisProfile(target);
if (!profile.ok) throw profile.error;
expect(profile.value.profile.provider.name).toBe("IDA MCP adapter");
expect(profile.value.profile.parameters).toMatchObject({
compatibility_profile: "native-ida-mcp",
mode: "attached",
cache_policy: "live",
engine_version: null,
});
expect(
provider
.capabilities()
.find(({ operation }) => operation === "procedure_pseudo_code")?.effects
.mayShowUi,
).toBe(true);
expect(
provider
.capabilities()
.find(({ operation }) => operation === "procedure_assembly")?.effects
.mayShowUi,
).toBe(false);
expect(producer.connects).toBe(0);
});
it("uses complete native JSON text instead of a shortened structured preview", () => {
const complete = [
{
data: [{ addr: "0x1000", name: "main", size: "0x20" }],
next_offset: null,
},
];
const response = {
content: [{ type: "text", text: JSON.stringify(complete) }],
structuredContent: { result: [], _output_truncated: true },
};
expect(decodeIdaToolResult(response, "native")).toEqual(complete);
expect(() =>
decodeIdaToolResult({ ...response, content: [] }, "native"),
).toThrow("complete JSON");
expect(() =>
decodeIdaToolResult(
{ ...response, content: [{ type: "text", text: "broken JSON" }] },
"native",
),
).toThrow();
expect(() =>
decodeIdaToolResult({ ...response, isError: true }, "native"),
).toThrow("tool failed");
});
});
describe("native IDA provider observations", () => {
it("retains native tool provenance, reads recorded metadata, and closes only its connection", async () => {
const { client, producer, resources } = await fixture();
for (const operation of [
"list_procedures",
"list_strings",
"procedure_assembly",
"procedure_pseudo_code",
"procedure_callees",
"xrefs",
"analyze_function",
] as const) {
const result = await client.execute(
operation,
operation === "list_procedures" || operation === "list_strings"
? {}
: operation === "xrefs"
? { address: "0x1000" }
: { procedure: "main" },
);
if (!result.ok) throw result.error;
expect(result.value.provider.name).toBe("IDA MCP adapter");
if (operation === "procedure_assembly")
expect(result.value.result).toBe("0x1000: ret");
if (operation === "procedure_pseudo_code") {
expect(result.value.result).toBe(producer.pseudocode);
expect(result.value.rawResult).toMatchObject({
observations: expect.arrayContaining([
expect.objectContaining({
tool: "decompile",
result: {
addr: "0x1000",
ok: true,
pseudocode: producer.pseudocode,
lines: 1,
},
}),
]),
});
}
}
expect(resources).toHaveLength(14);
expect(new Set(resources)).toEqual(new Set(["ida://idb/metadata"]));
expect(
producer.calls.find(({ name }) => name === "find_regex")?.args.pattern,
).toBe("[\\s\\S]*");
expect(
(await client.execute("procedure_callers", { procedure: "main" })).ok,
).toBe(false);
expect((await client.close()).ok).toBe(true);
expect(producer.closes).toBe(1);
expect(producer.calls.some(({ name }) => name.startsWith("idb_"))).toBe(
false,
);
const capabilities = idaCapabilities("attached", false, true);
expect(
capabilities.find(({ operation }) => operation === "procedure_callers")
?.available,
).toBe(false);
expect(
capabilities.find(
({ operation }) => operation === "procedure_pseudo_code",
)?.effects,
).toMatchObject({ mayShowUi: true, mayAccessNetwork: true });
expect(
idaCapabilities("attached", false).find(
({ operation }) => operation === "procedure_callers",
)?.available,
).toBe(true);
});
it("rejects unavailable or mismatched recorded input hashes and database switches", async () => {
const { client, producer, target } = await fixture();
for (const hash of ["unavailable", "a".repeat(64)]) {
producer.inputSha256 = hash;
expect((await client.execute("health", {})).ok).toBe(false);
}
producer.inputSha256 = target.sha256;
expect((await client.execute("health", {})).ok).toBe(true);
producer.inputPath += ".other.i64";
expect((await client.execute("list_procedures", {})).ok).toBe(false);
await client.close();
});
});
describe("native IDA callee lookup batches", () => {
it("batches unique callees and binds reordered replies to their source queries", async () => {
const { client, producer } = await fixture();
const previous = producer.overrideCall;
producer.overrideCall = (name, args) => {
if (name === "callees")
return [
{
callees: [
{ addr: "0x2004", name: "first", type: "internal" },
{ addr: "0x2004", name: "first", type: "internal" },
{ addr: "0x3000", name: "second", type: "internal" },
{ addr: "0x4000", name: "import", type: "external" },
],
more: false,
error: null,
},
];
if (
name === "lookup_funcs" &&
Array.isArray(args.queries) &&
args.queries.length > 1
)
return [
{
query: "0x3000",
fn: { addr: "0x3000", name: "second", size: "0x20" },
error: null,
},
{
query: "0x2004",
fn: { addr: "0x2000", name: "first", size: "0x20" },
error: null,
},
];
return previous?.(name, args);
};
try {
const result = await client.execute("procedure_callees", {
procedure: "main",
});
if (!result.ok) throw result.error;
expect(result.value.result).toEqual(["0x4000", "0x2000", "0x3000"]);
expect(
producer.calls
.filter(({ name }) => name === "lookup_funcs")
.map(({ args }) => args.queries),
).toEqual([["main"], ["0x2004", "0x3000"]]);
expect(result.value.rawResult).toMatchObject({
observations: expect.arrayContaining([
expect.objectContaining({
tool: "lookup_funcs",
arguments: { queries: ["0x2004", "0x3000"] },
result: [
expect.objectContaining({ query: "0x3000" }),
expect.objectContaining({ query: "0x2004" }),
],
}),
]),
});
} finally {
await client.close();
}
});
it("rejects missing, duplicate, unrelated, and failed callee lookup rows as provider faults", async () => {
const { client, producer } = await fixture();
const previous = producer.overrideCall;
const first = {
query: "0x2000",
fn: { addr: "0x2000", name: "first", size: "0x20" },
error: null,
};
const second = {
query: "0x3000",
fn: { addr: "0x3000", name: "second", size: "0x20" },
error: null,
};
let rows: JsonValue = [];
producer.overrideCall = (name, args) => {
if (name === "callees")
return [
{
callees: [
{ addr: "0x2000", name: "first", type: "internal" },
{ addr: "0x3000", name: "second", type: "internal" },
],
more: false,
error: null,
},
];
if (
name === "lookup_funcs" &&
Array.isArray(args.queries) &&
args.queries.length > 1
)
return rows;
return previous?.(name, args);
};
try {
for (const reply of [
[first],
[first, first],
[first, { ...second, query: "unrequested" }],
[first, { query: "0x3000", fn: null, error: "Not found" }],
]) {
rows = reply;
const result = await client.execute("procedure_callees", {
procedure: "main",
});
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toBeInstanceOf(AnalysisProtocolError);
}
} finally {
await client.close();
}
});
});
describe("native IDA literal matching", () => {
it("retains literal, Unicode, case, and regex semantics while narrowing native candidates", async () => {
const { client, producer } = await fixture();
const values = [
"A.B",
"axb",
"Kelvin",
"İstanbul",
"σ",
"ς",
"literal[bracket",
"a\\b",
"🧪",
];
producer.overrideCall = (name, args) => {
if (name !== "find_regex") return undefined;
if (typeof args.pattern !== "string")
throw new Error("Invalid fixture pattern");
// Exercise a byte-oriented producer seam, including Unicode candidates
// whose final matching semantics belong to REA, not the upstream regex.
const regex = new RegExp(
Buffer.from(args.pattern).toString("latin1"),
"i",
);
return {
matches: values.flatMap((value, i) =>
regex.test(Buffer.from(value).toString("latin1"))
? [{ addr: `0x${(0x3000 + i).toString(16)}`, string: value }]
: [],
),
cursor: { done: true },
};
};
const cases = [
{
pattern: "a.b",
mode: "literal",
case_sensitive: false,
expected: ["A.B"],
},
{ pattern: "a.b", mode: "literal", case_sensitive: true, expected: [] },
{
pattern: "kelvin",
mode: "literal",
case_sensitive: false,
expected: ["Kelvin"],
},
{
pattern: "İ",
mode: "literal",
case_sensitive: false,
expected: ["İstanbul"],
},
{ pattern: "σ", mode: "literal", case_sensitive: false, expected: ["σ"] },
{
pattern: "σ",
mode: "regex",
case_sensitive: false,
expected: ["σ", "ς"],
},
{
pattern: "[",
mode: "literal",
case_sensitive: false,
expected: ["literal[bracket"],
},
{
pattern: "\\",
mode: "literal",
case_sensitive: true,
expected: ["a\\b"],
},
{
pattern: "\ud800",
mode: "literal",
case_sensitive: false,
expected: [],
},
{
pattern: "🧪",
mode: "literal",
case_sensitive: true,
expected: ["🧪"],
},
];
try {
for (const { expected, ...query } of cases) {
const before = producer.calls.length;
const result = await client.execute("search_strings", query);
if (!result.ok) throw result.error;
expect(result.value.result).toEqual(
expected.map((value) => ({
address: `0x${(0x3000 + values.indexOf(value)).toString(16)}`,
value,
})),
);
const reads = producer.calls
.slice(before)
.filter(({ name }) => name === "find_regex");
expect(reads).toHaveLength(1);
if (query.mode === "regex" || query.pattern === "\ud800")
expect(reads[0]?.args.pattern).toBe("[\\s\\S]*");
else expect(reads[0]?.args.pattern).not.toBe("[\\s\\S]*");
}
const before = producer.calls.length;
const invalid = await client.execute("search_strings", {
pattern: "[",
mode: "regex",
});
expect(invalid.ok).toBe(false);
if (!invalid.ok) expect(invalid.error).toBeInstanceOf(AnalysisInputError);
expect(
producer.calls.slice(before).some(({ name }) => name === "find_regex"),
).toBe(false);
} finally {
await client.close();
}
});
});
describe("native IDA string pagination and failures", () => {
it("reads all candidate pages and reports native search failures explicitly", async () => {
const { client, producer } = await fixture();
producer.overrideCall = (name, args) =>
name === "find_regex"
? {
matches: [
{
addr: args.offset === 0 ? "0x3000" : "0x3001",
string: args.offset === 0 ? "σ" : "needle",
},
],
cursor: args.offset === 0 ? { next: 1 } : { done: true },
}
: undefined;
try {
const found = await client.execute("search_strings", {
pattern: "needle",
});
if (!found.ok) throw found.error;
expect(found.value.result).toEqual([
{ address: "0x3001", value: "needle" },
]);
const reads = producer.calls.filter(({ name }) => name === "find_regex");
expect(reads.map(({ args }) => args.offset)).toEqual([0, 1]);
expect(new Set(reads.map(({ args }) => args.pattern)).size).toBe(1);
producer.overrideCall = (name) =>
name === "find_regex"
? {
matches: [],
cursor: { done: true },
error: "Regex resource failure",
}
: undefined;
const failed = await client.execute("search_strings", {
pattern: "needle",
});
expect(failed.ok).toBe(false);
if (!failed.ok) {
expect(failed.error).toBeInstanceOf(AnalysisProtocolError);
expect(failed.error.message).toContain("Regex resource failure");
}
} finally {
await client.close();
}
});
});
describe("native IDA malformed observations", () => {
it("rejects wrong-function pseudocode, upstream cloud failures, and malformed assembly", async () => {
const { client, producer } = await fixture();
for (const value of [
{ addr: "0x2000", ok: true, pseudocode: "int wrong(void);" },
{
addr: "0x1000",
ok: false,
error: "No pseudocode widget for this function after decompilation",
},
{ addr: "0x1000", ok: true },
]) {
producer.overrideCall = (name) =>
name === "decompile" ? value : undefined;
const result = await client.execute("procedure_pseudo_code", {
procedure: "main",
});
expect(result.ok).toBe(false);
}
producer.overrideCall = (name) =>
name === "disasm"
? {
asm: {
name: "main",
start_ea: "0x1000",
lines: "main (.text @ 0x1000):\nunsupported",
},
cursor: { done: true },
}
: undefined;
expect(
(await client.execute("procedure_assembly", { procedure: "main" })).ok,
).toBe(false);
await client.close();
});
it("follows native assembly pages and rejects non-advancing cursors", async () => {
const { client, producer } = await fixture();
producer.overrideCall = (name, args): JsonValue | undefined =>
name === "disasm"
? {
asm: {
name: "main",
start_ea: "0x1000",
lines: `main (.text @ 0x1000):\n${args.offset === 0 ? "1000 nop" : "1001 ret"}`,
},
cursor: args.offset === 0 ? { next: 1 } : { done: true },
}
: undefined;
const result = await client.execute("procedure_assembly", {
procedure: "main",
});
expect(result.ok && result.value.result).toBe("0x1000: nop\n0x1001: ret");
producer.overrideCall = (name) =>
name === "find_regex" ? { matches: [], cursor: { next: 0 } } : undefined;
expect((await client.execute("list_strings", {})).ok).toBe(false);
await client.close();
});
});
+115
View File
@@ -0,0 +1,115 @@
import { z } from "zod";
import { AnalysisProtocolError } from "../domain/analysisErrorCore.js";
import {
cursorSchema,
disassemblySchema,
idaAddressSchema,
modernCalleesSchema,
modernStringsSchema,
modernXrefsSchema,
} from "./IdaProtocolValues.js";
/** Native MCP renders an assembly header followed by bare hexadecimal coordinates. */
const nativeAssemblySchema = disassemblySchema
.omit({ lines: true })
.extend({
lines: z.string(),
})
.transform((asm) => {
const [header, ...body] = asm.lines.split("\n");
const match = header?.match(/^(.+) \((.*) @ (0x[0-9a-f]+)\):$/iu);
if (
match?.[1] !== asm.name ||
idaAddressSchema.parse(match?.[3]) !== asm.start_ea
)
throw new AnalysisProtocolError(
"Native IDA assembly header contradicts its function identity.",
);
return {
...asm,
lines: body.map((line) => {
const item = line.match(/^([0-9a-f]+) (.*)$/iu);
if (item === null)
throw new AnalysisProtocolError(
"Native IDA returned an unsupported assembly line.",
);
return {
address: idaAddressSchema.parse(item[1]),
instruction: item[2] ?? "",
};
}),
};
});
/** Native assembly retains the same explicit pagination cursor as modern MCP. */
export const nativeDisassemblySchema = z.object({
asm: nativeAssemblySchema.nullable(),
error: z.string().optional(),
cursor: cursorSchema,
});
/** GUI decompilation reports the resolved entry and an explicit success/failure. */
export const nativeDecompileSchema = z.object({
addr: idaAddressSchema,
ok: z.boolean(),
pseudocode: z.string().min(1).optional(),
lines: z.number().int().nonnegative().optional(),
error: z.string().optional(),
});
/** Native relationship successes explicitly carry null errors. Normalize that omission here. */
const nativeErrorSchema = z
.string()
.nullish()
.transform((value) => value ?? undefined);
export const nativeCalleesSchema = z
.array(modernCalleesSchema.element.extend({ error: nativeErrorSchema }))
.length(1);
export const nativeXrefsSchema = z
.array(modernXrefsSchema.element.extend({ error: nativeErrorSchema }))
.length(1);
/** Native regex failures carry an empty page, which must not imply absence. */
export const nativeStringsSchema = modernStringsSchema
.extend({ error: z.string().optional() })
.transform((page) => {
if (page.error !== undefined)
throw new AnalysisProtocolError(
`Native IDA string search failed: ${page.error}`,
);
return page;
});
/**
* Native regexes fold bytes, while REA's insensitive literal search folds
* Unicode strings. Include every non-ASCII value as a candidate, then apply
* REA's exact predicate. Ill-formed UTF-16 cannot cross upstream JSON/UTF-8.
*/
export const nativeLiteralSearchPattern = (
pattern: string,
caseSensitive: boolean,
): string | undefined => {
if (/[\uD800-\uDFFF]/u.test(pattern)) return undefined;
let literal = (caseSensitive ? pattern : pattern.toLowerCase()).replace(
/[.*+?^${}()|[\]\\]/gu,
"\\$&",
);
// Spell ASCII variants explicitly rather than depending on an upstream locale.
if (!caseSensitive)
literal = literal.replace(
/[a-z]/gu,
(char) => `[${char}${char.toUpperCase()}]`,
);
return caseSensitive ? literal : `(?:${literal})|[\\x80-\\xff]`;
};
/** Read-only tools required from the native bridge; GUI lifecycle tools are excluded. */
export const nativeTools = [
"list_funcs",
"lookup_funcs",
"find_regex",
"decompile",
"disasm",
"callees",
"xrefs_to",
];
+141 -56
View File
@@ -1,4 +1,12 @@
import { z } from "zod";
import {
nativeDecompileSchema,
nativeDisassemblySchema,
nativeCalleesSchema,
nativeXrefsSchema,
nativeLiteralSearchPattern,
nativeStringsSchema,
} from "./IdaNativeProtocol.js";
import { analysisSearchInput } from "../contracts/analysisSearchContract.js";
import {
AnalysisInputError,
@@ -14,7 +22,10 @@ import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import { readCallGraphCallees, unavailableCallGraph } from "./IdaCallGraph.js";
import type { IdaMcpConnection } from "./IdaMcpConnection.js";
import type { IdaOperation } from "./IdaProviderCapabilities.js";
import { IDA_LIMITATIONS } from "./IdaProviderCapabilities.js";
import {
IDA_LIMITATIONS,
IDA_NATIVE_STRING_LIMITATION,
} from "./IdaProviderCapabilities.js";
import type { EvidenceLocation } from "../domain/evidence.js";
import {
disassemblySchema,
@@ -28,6 +39,7 @@ import {
modernDecompileSchema,
modernDisassemblySchema,
modernLookupSchema,
modernLookupBatchSchema,
modernPageSchema,
modernStringsSchema,
modernXrefsSchema,
@@ -60,6 +72,7 @@ const MISSING_PROCEDURE_ANSWERS: ReadonlySet<string> = new Set([
]);
const lines = (asm: IdaDisassembly) =>
asm.lines.map(({ address, instruction }) => `${address}: ${instruction}`);
const searchQuerySchema = z.object(analysisSearchInput);
/** Translate explicit analyst questions into a closed set of read-only upstream requests. */
export class IdaOperationRunner {
@@ -69,7 +82,7 @@ export class IdaOperationRunner {
relationshipCoverage: RelationshipCoverage | undefined;
constructor(
private readonly connection: IdaMcpConnection,
private readonly protocol: "legacy" | "modern",
private readonly protocol: "legacy" | "modern" | "native",
private readonly database?: string,
private readonly options: {
readonly signal?: AbortSignal | undefined;
@@ -131,37 +144,11 @@ export class IdaOperationRunner {
* addresses were reported by IDA itself.
*/
async resolve(query: string, requestedBy?: string): Promise<IdaFunction> {
if (this.protocol === "modern") {
if (this.protocol !== "legacy") {
const item = modernLookupSchema.parse(
await this.call("lookup_funcs", { queries: [query] }),
)[0];
if (
requestedBy !== undefined &&
item?.query === query &&
item.fn === null &&
item.error !== null &&
MISSING_PROCEDURE_ANSWERS.has(item.error)
)
throw new AnalysisInputError(
requestedBy,
{
cause: new AnalysisProtocolError(
`IDA function lookup failed for ${query}: ${item.error}`,
),
},
[
{
path: [],
reason: "invalid_value",
message: `IDA could not resolve procedure ${JSON.stringify(query)}: ${item.error}.`,
},
],
);
if (item?.fn == null || item.error !== null)
throw new AnalysisProtocolError(
`IDA function lookup failed for ${query}: ${item?.error ?? "missing function"}`,
);
return item.fn;
return this.lookupFunction(query, item, requestedBy);
}
const address = idaAddressSchema.safeParse(
/^0x/iu.test(query) ? query : undefined,
@@ -184,6 +171,43 @@ export class IdaOperationRunner {
return entry;
}
private lookupFunction(
query: string,
item: z.infer<typeof modernLookupBatchSchema>[number] | undefined,
requestedBy?: string,
): IdaFunction {
if (item?.query !== query)
throw new AnalysisProtocolError(
`IDA function lookup omitted or contradicted query ${JSON.stringify(query)}.`,
);
if (
requestedBy !== undefined &&
item.fn === null &&
item.error !== null &&
MISSING_PROCEDURE_ANSWERS.has(item.error)
)
throw new AnalysisInputError(
requestedBy,
{
cause: new AnalysisProtocolError(
`IDA function lookup failed for ${query}: ${item.error}`,
),
},
[
{
path: [],
reason: "invalid_value",
message: `IDA could not resolve procedure ${JSON.stringify(query)}: ${item.error}.`,
},
],
);
if (item.fn === null || item.error !== null)
throw new AnalysisProtocolError(
`IDA function lookup failed for ${query}: ${item.error ?? "missing function"}`,
);
return item.fn;
}
/** True only for modern callee reads on targets not proven x86-only; x86 reads never reach the graph code. */
private readsCallGraph(direction: "callers" | "callees"): boolean {
return (
@@ -212,7 +236,11 @@ export class IdaOperationRunner {
result.set(callee.address, identity({ ...callee, size: "unknown" }));
}
async strings(): Promise<{ address: string; value: string }[]> {
async strings(
nativePattern?: string,
): Promise<{ address: string; value: string }[]> {
if (this.protocol === "native")
this.limitations.push(IDA_NATIVE_STRING_LIMITATION);
const result: { address: string; value: string }[] = [];
let offset: number | null = 0;
while (offset !== null) {
@@ -229,9 +257,14 @@ export class IdaOperationRunner {
);
offset = page.next_offset;
} else {
const page = modernStringsSchema.parse(
const page = (
this.protocol === "native" ? nativeStringsSchema : modernStringsSchema
).parse(
await this.call("find_regex", {
pattern: "(?s).*",
pattern:
this.protocol === "native"
? (nativePattern ?? "[\\s\\S]*")
: "(?s).*",
offset,
limit: 100,
}),
@@ -260,7 +293,11 @@ export class IdaOperationRunner {
let result: IdaDisassembly | undefined;
let offset: number | null = 0;
while (offset !== null) {
const page = modernDisassemblySchema.parse(
const page = (
this.protocol === "native"
? nativeDisassemblySchema
: modernDisassemblySchema
).parse(
await this.call("disasm", {
addr: fn.address,
offset,
@@ -294,6 +331,23 @@ export class IdaOperationRunner {
.string()
.nullable()
.parse(await this.call("decompile_function", { address: fn.address }));
if (this.protocol === "native") {
const page = nativeDecompileSchema.parse(
await this.call("decompile", { addr: fn.address }),
);
if (page.addr !== fn.address)
throw new AnalysisProtocolError(
"Native IDA pseudocode identifies a different function than the resolved entry.",
);
if (!page.ok || page.error !== undefined || page.pseudocode === undefined)
throw new AnalysisProtocolError(
`Native IDA decompilation failed: ${page.error ?? "missing pseudocode"}`,
);
this.limitations.push(
"Native pseudocode is rendered GUI text; the upstream viewer traversal is bounded and does not establish complete source recovery.",
);
return page.pseudocode;
}
const result: string[] = [];
let offset: number | null = 0;
while (offset !== null) {
@@ -323,9 +377,9 @@ export class IdaOperationRunner {
return legacyXrefsSchema.parse(
await this.call("get_xrefs_to", { address }),
);
const page = modernXrefsSchema.parse(
await this.call("xrefs_to", { addrs: [address], limit: 1000 }),
)[0];
const page = (
this.protocol === "native" ? nativeXrefsSchema : modernXrefsSchema
).parse(await this.call("xrefs_to", { addrs: [address], limit: 1000 }))[0];
if (page?.xrefs == null || page.error !== undefined)
throw new AnalysisProtocolError(
`IDA xrefs failed: ${page?.error ?? "missing references"}`,
@@ -367,16 +421,17 @@ export class IdaOperationRunner {
}
} else if (direction === "callers") {
this.limitations.push(
"Modern upstream xrefs classify references as code or data, without proving call edges. Direct callers are unavailable in this profile.",
"Upstream xrefs classify references as code or data, without proving call edges. Direct callers are unavailable in this profile.",
);
this.relationshipCoverage = {
status: "unknown",
reason:
"Modern upstream xrefs do not establish direct caller relationships.",
reason: "Upstream xrefs do not establish direct caller relationships.",
};
return [];
} else {
const page = modernCalleesSchema.parse(
const page = (
this.protocol === "native" ? nativeCalleesSchema : modernCalleesSchema
).parse(
await this.call("callees", { addrs: [fn.address], limit: 500 }),
)[0];
if (page?.callees == null || page.error !== undefined)
@@ -405,9 +460,31 @@ export class IdaOperationRunner {
else queries.push(item.addr);
}
}
for (const query of new Set(queries)) {
const resolved = identity(await this.resolve(query));
result.set(resolved.address, resolved);
const querySet = new Set(queries);
if (this.protocol === "native" && querySet.size > 0) {
const rows = modernLookupBatchSchema.parse(
await this.call("lookup_funcs", { queries: [...querySet] }),
);
const byQuery = new Map(rows.map((row) => [row.query, row]));
if (
rows.length !== querySet.size ||
byQuery.size !== rows.length ||
rows.some((row) => !querySet.has(row.query))
)
throw new AnalysisProtocolError(
"Native IDA returned an inconsistent callee lookup batch.",
);
for (const query of querySet) {
const resolved = identity(
this.lookupFunction(query, byQuery.get(query)),
);
result.set(resolved.address, resolved);
}
} else {
for (const query of querySet) {
const resolved = identity(await this.resolve(query));
result.set(resolved.address, resolved);
}
}
if (this.relationshipCoverage === undefined)
this.relationshipCoverage = {
@@ -421,19 +498,29 @@ export class IdaOperationRunner {
operation: IdaOperation,
parameters: Readonly<Record<string, JsonValue>>,
): Promise<JsonValue> {
const query =
operation === "search_procedures" || operation === "search_strings"
? searchQuerySchema.parse(parameters)
: undefined;
const matches =
query === undefined ? undefined : this.searchPredicate(query);
if (operation === "list_procedures" || operation === "search_procedures") {
const items = (await this.functions()).map((fn) => ({
address: fn.address,
value: fn.name,
}));
return operation === "list_procedures"
return matches === undefined
? items
: this.search(items, parameters);
: items.filter(({ value }) => matches(value));
}
if (operation === "list_strings" || operation === "search_strings") {
const items = await this.strings();
return operation === "search_strings"
? this.search(items, parameters)
const nativePattern =
this.protocol === "native" && query?.mode === "literal"
? nativeLiteralSearchPattern(query.pattern, query.case_sensitive)
: undefined;
const items = await this.strings(nativePattern);
return matches !== undefined
? items.filter(({ value }) => matches(value))
: parameters.address === undefined
? items
: items.filter(
@@ -459,7 +546,7 @@ export class IdaOperationRunner {
case "procedure_assembly":
return lines(await this.assembly(fn)).join("\n");
case "procedure_callers":
if (this.protocol === "modern")
if (this.protocol !== "legacy")
throw new AnalysisProtocolError(
"This IDA MCP profile cannot distinguish direct callers from other code references.",
);
@@ -481,11 +568,9 @@ export class IdaOperationRunner {
}
}
private search(
items: { address: string; value: string }[],
input: Readonly<Record<string, JsonValue>>,
): JsonValue {
const query = z.object(analysisSearchInput).parse(input);
private searchPredicate(
query: z.infer<typeof searchQuerySchema>,
): (value: string) => boolean {
let matches: (value: string) => boolean;
if (query.mode === "regex") {
try {
@@ -504,7 +589,7 @@ export class IdaOperationRunner {
matches = (value) =>
(query.case_sensitive ? value : value.toLowerCase()).includes(needle);
}
return items.filter(({ value }) => matches(value));
return matches;
}
private async dossier(fn: IdaFunction): Promise<JsonValue> {
+17 -7
View File
@@ -1,7 +1,10 @@
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
isElfMetadataArchitecture,
type BinaryTarget,
} from "../domain/binaryTargetTypes.js";
import { err, ok } from "../domain/result.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import type {
@@ -55,6 +58,7 @@ export class IdaProvider implements AnalysisProviderCandidate {
return idaCapabilities(
config.ok ? config.value.mode : "attached",
config.ok && "command" in config.value,
config.ok && config.value.protocol === "native",
);
}
inspectAvailability(): ProviderAvailability {
@@ -101,12 +105,16 @@ export class IdaProvider implements AnalysisProviderCandidate {
target_kind: target.kind,
target_format: target.format,
};
if (target.kind === "executable" && target.architecture === "mips")
if (
target.kind === "executable" &&
isElfMetadataArchitecture(target.architecture)
)
return {
status: "unsupported",
code: "architecture_unsupported",
reason:
"REA's IDA adapter has no verified MIPS lane. Select a supported Ghidra profile instead.",
reason: `REA's IDA adapter has no verified ${
target.architecture === "mips" ? "MIPS" : "RISC-V"
} lane. Select a supported Ghidra profile instead.`,
diagnostics,
};
if (target.kind !== "executable")
@@ -142,9 +150,11 @@ export class IdaProvider implements AnalysisProviderCandidate {
upstream_distribution_version: null,
mode: config.value.mode,
compatibility_profile:
config.value.mode === "attached"
? "legacy-1.4"
: "database-supervisor",
config.value.protocol === "native"
? "native-ida-mcp"
: config.value.mode === "attached"
? "legacy-1.4"
: "database-supervisor",
database_revision: null,
cache_policy: "live",
registration_digest: createHash("sha256")
+19 -8
View File
@@ -3,14 +3,14 @@ import type {
CapabilityDescriptor,
} from "../application/AnalysisProvider.js";
/** This version identifies REA's adapter, not IDA or the upstream Python package. */
/** This version identifies REA's adapter, not IDA or the upstream MCP server. */
export const IDA_PROVIDER_IDENTITY = Object.freeze({
id: "ida",
name: "IDA Pro MCP adapter",
name: "IDA MCP adapter",
version: "1",
});
/** Read-only analyst outcomes supported by both upstream compatibility profiles. */
/** Read-only analyst outcomes supported by the upstream compatibility profiles. */
export const IDA_OPERATIONS = Object.freeze([
"list_procedures",
"search_procedures",
@@ -37,6 +37,9 @@ export const IDA_LIMITATIONS = Object.freeze([
"Complete function body ranges, typed references, unresolved indirect calls, referenced data, and basic blocks are unavailable in this adapter. Empty unsupported dossier facets are unknown, not observed absence.",
]);
export const IDA_NATIVE_STRING_LIMITATION =
"Native string observations use upstream C-string rendering; other string representations remain unknown.";
/**
* Attached mode requires the legacy 1.4 profile, whose `disassemble_function`
* jumps to the function and whose `decompile_function` opens or reuses a
@@ -53,15 +56,17 @@ const ATTACHED_GUI_NAVIGATION_OPERATIONS: ReadonlySet<IdaOperation> = new Set([
export const idaCapabilities = (
mode: "attached" | "headless",
stdio: boolean,
native = false,
): readonly CapabilityDescriptor[] =>
IDA_OPERATIONS.map((operation) => ({
provider: IDA_PROVIDER_IDENTITY,
operation,
...(mode === "headless" && operation === "procedure_callers"
...((mode === "headless" || native) && operation === "procedure_callers"
? ({
available: false,
reason:
"The modern upstream profile reports code references without proving direct call edges.",
reason: native
? "The native upstream profile reports code references without proving direct call edges."
: "The modern upstream profile reports code references without proving direct call edges.",
} as const)
: ({ available: true, reason: null } as const)),
cachePolicy: "live",
@@ -70,11 +75,17 @@ export const idaCapabilities = (
launchesProcess: stdio || mode === "headless",
mayShowUi:
mode === "attached" &&
ATTACHED_GUI_NAVIGATION_OPERATIONS.has(operation),
(native
? operation === "procedure_pseudo_code" ||
operation === "analyze_function"
: ATTACHED_GUI_NAVIGATION_OPERATIONS.has(operation)),
mayAccessNetwork: true,
mayWriteFilesystem: stdio || mode === "headless",
changesPermissions: false,
requiresRoot: false,
},
limitations: IDA_LIMITATIONS,
limitations:
native && (operation === "list_strings" || operation === "search_strings")
? [...IDA_LIMITATIONS, IDA_NATIVE_STRING_LIMITATION]
: IDA_LIMITATIONS,
}));
+27 -6
View File
@@ -23,7 +23,9 @@ import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import type { IdaConfiguration } from "./IdaConfiguration.js";
import { idaProtocol } from "./IdaConfiguration.js";
import type { IdaMcpConnection } from "./IdaMcpConnection.js";
import { nativeTools } from "./IdaNativeProtocol.js";
import { readSettledHealth } from "./IdaBusyHealth.js";
import { isX86OnlyTarget } from "./IdaCallGraph.js";
import { IdaOperationRunner } from "./IdaOperationRunner.js";
@@ -75,6 +77,7 @@ const modernTools = [
"xrefs_to",
];
const operationSet: ReadonlySet<string> = new Set(IDA_OPERATIONS);
const isIdaOperation = (operation: string): operation is IdaOperation =>
operationSet.has(operation);
const wasCancelled = (options: ExecutionOptions | undefined): boolean =>
@@ -191,12 +194,15 @@ export class IdaSessionClient implements AnalysisClient {
"This integration admits read-only analysis operations only.",
),
);
if (operation === "procedure_callers" && this.config.mode === "headless")
if (
operation === "procedure_callers" &&
idaProtocol(this.config) !== "legacy"
)
return err(
new AnalysisCapabilityUnavailableError(
"ida",
operation,
"The modern upstream profile does not prove direct callers.",
"This upstream profile does not prove direct callers.",
),
);
if (operation !== "health") {
@@ -280,7 +286,7 @@ export class IdaSessionClient implements AnalysisClient {
#runner(options: ExecutionOptions | undefined): IdaOperationRunner {
return new IdaOperationRunner(
this.connection,
this.config.mode === "attached" ? "legacy" : "modern",
idaProtocol(this.config),
this.#database,
{
signal: options?.signal,
@@ -418,11 +424,15 @@ export class IdaSessionClient implements AnalysisClient {
const tools = new Set(await this.connection.connect());
this.#callGraphAdvertised = tools.has("callgraph");
const expected =
this.config.mode === "attached" ? legacyTools : modernTools;
this.config.protocol === "native"
? nativeTools
: this.config.mode === "attached"
? legacyTools
: modernTools;
const missing = expected.filter((name) => !tools.has(name));
if (missing.length > 0)
throw new AnalysisProtocolError(
`IDA MCP ${this.config.mode} compatibility profile is missing tools: ${missing.join(", ")}. Attached mode requires the legacy 1.4 profile; headless mode requires the database supervisor profile. See docs/ida-provider.md.`,
`IDA MCP ${this.config.protocol === "native" ? "native attached" : this.config.mode} compatibility profile is missing tools: ${missing.join(", ")}. Native attached mode requires the native IDA MCP tool catalog; legacy attached mode requires the legacy 1.4 profile; headless mode requires the database supervisor profile. See docs/ida-provider.md.`,
);
if (this.config.mode === "headless") {
this.#workspace = await IdaWorkspace.allocate(
@@ -472,13 +482,24 @@ export class IdaSessionClient implements AnalysisClient {
}
}
async #nativeMetadata(): Promise<JsonValue> {
if (this.connection.readResource === undefined)
throw new AnalysisProtocolError(
"Native IDA MCP requires resources/read for recorded input metadata.",
);
return this.connection.readResource("ida://idb/metadata");
}
async #observe(
options?: ExecutionOptions,
onObserved?: (metadata: JsonValue) => void,
): Promise<JsonValue> {
let identity: string;
if (this.config.mode === "attached") {
const raw = await this.connection.call("get_metadata", {});
const raw =
this.config.protocol === "native"
? await this.#nativeMetadata()
: await this.connection.call("get_metadata", {});
const metadata = legacyMetadataSchema.parse(raw);
this.#metadata = raw;
onObserved?.(raw);
+10 -2
View File
@@ -662,6 +662,7 @@ export const observeSettlement = async (
}
const started = Date.now();
let consecutiveEmpty = 0;
let sawLive = false;
let deadlineReached = false;
while (!deadlineReached) {
onLiveProgress?.();
@@ -685,13 +686,20 @@ export const observeSettlement = async (
recordEvent("lifecycle", 1);
return { state: "quiesced", elapsed_ms: Date.now() - started };
}
} else consecutiveEmpty = 0;
} else {
consecutiveEmpty = 0;
sawLive = true;
}
deadlineReached = Date.now() - started >= settleMs;
if (deadlineReached) break;
await delay(50, undefined, { signal });
}
recordEvent("lifecycle", 1);
return { state: "alive_at_deadline", elapsed_ms: Date.now() - started };
// A lone empty observation proves neither liveness nor quiescence; only an
// observed live member justifies alive_at_deadline.
return sawLive
? { state: "alive_at_deadline", elapsed_ms: Date.now() - started }
: { state: "unverifiable", elapsed_ms: Date.now() - started };
};
/** Name why an attempt is unverified; a blank message never reaches the capture schema. */
+2 -1
View File
@@ -12,6 +12,7 @@ import { jsonValueSchema } from "../domain/jsonValue.js";
import { logToolExecution } from "./toolLogging.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import type { ToolResultDelivery } from "./toolResult.js";
import { unknownFridaSession } from "../application/frida/fridaInputErrors.js";
/** Register Frida device discovery and persistent instrumentation operations. */
export const registerFridaTools = (
@@ -199,7 +200,7 @@ export const registerFridaTools = (
return delivery.toCallToolResult(
{
ok: false as const,
error: new AnalysisInputError(status.name),
error: unknownFridaSession(status.name, input.session_id),
},
status,
);
@@ -30,7 +30,7 @@ type CaptureOutcome =
| { readonly kind: "partial"; readonly partial: unknown };
const captureScenario = (
mode: "cooperative" | "ignoring" | "descendant",
mode: "cooperative" | "ignoring" | "delayed-ignoring" | "descendant",
root: string,
additions: CaptureScenario = {},
finalExitCode?: number,
@@ -453,11 +453,13 @@ captureTest(
"rea-finalization-cli-cancel-",
);
const cliScenarioPath = join(cliRoot, "scenario.json");
// Match the other finalization cases' observation window. The old 300 ms
// deadline could signal the fixture before it installed its handlers.
await writeFile(
cliScenarioPath,
JSON.stringify(
captureScenario("ignoring", cliRoot, {
timeout_ms: 300,
captureScenario("delayed-ignoring", cliRoot, {
timeout_ms: 1_500,
finalization_ms: 20_000,
}),
),
@@ -527,18 +529,36 @@ captureTest(
const mcpRoot = await createTestTempDirectory(
"rea-finalization-mcp-cancel-",
);
const { client } = await connectLocalToolsMcp();
const { client, call: callTool } = await connectLocalToolsMcp();
const controller = new AbortController();
const call = client.callTool(
{
name: "capture_process_scenario",
arguments: captureScenario("ignoring", mcpRoot, {
timeout_ms: 300,
finalization_ms: 20_000,
}),
},
{ signal: controller.signal },
let reportStarted: (() => void) | undefined;
const captureStarted = new Promise<void>((resolve) => {
reportStarted = resolve;
});
const call = callTool(
"capture_process_scenario",
captureScenario("delayed-ignoring", mcpRoot, {
timeout_ms: 1_500,
finalization_ms: 20_000,
}),
{ signal: controller.signal, onprogress: () => reportStarted?.() },
);
// Observe rejection immediately: a failed readiness wait must not leave a
// pending request to reject unhandled when the fixture closes its client.
// The original promise remains available for the cancellation assertion.
const observedCall = call.catch(() => undefined);
onTestFinished(async () => {
controller.abort();
await observedCall;
});
// Start the fixture's readiness wait after the server begins this capture;
// SDK schema preparation before dispatch is not target startup time.
await Promise.race([
captureStarted,
call.then(() => {
throw new Error("MCP capture completed before reporting startup");
}),
]);
const mcpFixturePid = await waitForFinalization(mcpRoot);
const mcpStarted = Date.now();
controller.abort(new Error("cancel finalization acceptance fixture"));
@@ -548,7 +568,7 @@ captureTest(
).rejects.toThrow("cancel finalization acceptance fixture");
await waitForProcessExit(mcpFixturePid);
await expect(
client.listTools(),
client.listTools(undefined, { cacheMode: "refresh" }),
"the server accepts a follow-up request after cancelling the capture",
).resolves.toMatchObject({ tools: expect.any(Array) });
expect(
@@ -0,0 +1,154 @@
import { mkdir, readFile, readdir, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { expect } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
const cases = [
{
client: "cursor",
path: ".cursor/mcp.json",
text: "[]",
diagnostic: "The configuration root must be an object",
},
{
client: "cursor",
path: ".cursor/mcp.json",
text: "null",
diagnostic: "The configuration root must be an object",
},
{
client: "cursor",
path: ".cursor/mcp.json",
text: '{"mcpServers":null,"keep":1}',
diagnostic: '"mcpServers" must be an object',
},
{
client: "cursor",
path: ".cursor/mcp.json",
text: '{"mcpServers":"x","keep":1}',
diagnostic: '"mcpServers" must be an object',
},
{
client: "pi",
path: ".pi/agent/mcp.json",
text: '{"mcpServers":[]}',
diagnostic: '"mcpServers" must be an object',
},
{
client: "codex",
path: ".codex/config.toml",
text: 'mcp_servers = "x"\n',
diagnostic: '"mcp_servers" must be an object',
},
{
client: "opencode",
path: ".config/opencode/opencode.json",
text: '{"mcp":{"servers":null}}',
diagnostic: '"mcp.servers" must be an object',
},
{
client: "hermes",
path: ".hermes/config.yaml",
text: "- item\n",
diagnostic: "The configuration root must be an object",
},
{
client: "hermes",
path: ".hermes/config.yaml",
text: "mcp_servers: null\nkeep: 1\n",
diagnostic: '"mcp_servers" must be an object',
},
{
client: "cursor",
path: ".cursor/mcp.json",
text: "not-json",
diagnostic: "Configuration is not valid JSON",
syntax: "JSON",
},
{
client: "hermes",
path: ".hermes/config.yaml",
text: "mcp_servers: [\n",
diagnostic: "Configuration is not valid YAML",
syntax: "YAML",
},
{
client: "dsh",
path: ".dsh/cordis.patch.yml",
text: "- insert: [\n",
diagnostic: "Configuration is not valid YAML",
syntax: "YAML",
},
] as const;
for (const scenario of cases)
cliTest(
`setup and uninstall explain ${scenario.client} configuration ${scenario.text}`,
async ({ cli }) => {
const home = await createTestTempDirectory(
"rea-client-config-diagnostic-",
);
const config = join(home, scenario.path);
await mkdir(dirname(config), { recursive: true });
await writeFile(config, scenario.text);
const environment = {
HOME: home,
USERPROFILE: home,
XDG_CONFIG_HOME: join(home, ".config"),
HERMES_HOME: join(home, ".hermes"),
DSH_HOME: join(home, ".dsh"),
};
const setup = await cli.run({
arguments: [
"setup",
"--client",
scenario.client,
"--skill=false",
"--yes",
"--json",
],
cwd: home,
environment,
});
expect(setup.exitCode).toBe(1);
expect(setup.json).toMatchObject({
status: "needs_human",
plannedActions: [],
appliedActions: [],
});
if ("syntax" in scenario) {
expect(setup.json).toMatchObject({ remediation: expect.any(String) });
if (scenario.syntax === "JSON")
expect(setup.stdout).toContain("Invalid JSON/JSONC");
} else {
expect(setup.json).toMatchObject({
remediation: expect.stringContaining(scenario.diagnostic),
});
expect(setup.stdout).not.toContain("invalid_type");
}
const uninstall = await cli.run({
arguments: ["uninstall", "--json"],
cwd: home,
environment,
});
expect(uninstall.exitCode).toBe(1);
expect(uninstall.json).toMatchObject({
status: "failed",
items: expect.arrayContaining([
expect.objectContaining({
name: scenario.client,
status: "failed",
detail: expect.stringContaining(scenario.diagnostic),
}),
]),
});
if (!("syntax" in scenario))
expect(uninstall.stdout).not.toContain("not valid JSON");
expect(await readFile(config, "utf8")).toBe(scenario.text);
expect(await readdir(dirname(config))).toEqual([
scenario.path.split("/").at(-1),
]);
},
);
+40
View File
@@ -234,3 +234,43 @@ it("preserves observed devices when CLI provider cleanup fails", async () => {
details: { diagnostics: { partial_observation: { devices } } },
});
});
it.each([
[
["attach", "abc", "--script", "send(1)"],
"target",
"attach mode needs a positive integer process ID.",
],
[
["attach", "71"],
"script",
"Provide exactly one of --script or --script-file.",
],
])(
"names the rejected instrument-with-frida argument for %j",
async (argv, path, message) => {
let starts = 0;
const provider = {
async startSession() {
starts += 1;
throw new Error("Frida must not start for invalid input");
},
} as unknown as FridaInstrumentationPort;
const cli = createCli({}, undefined, provider);
let stdout = "";
await cli.serve(["instrument-with-frida", ...argv, "--json"], {
env: {},
exit: () => undefined,
stdout: (text) => {
stdout += text;
},
});
expect(JSON.parse(stdout)).toMatchObject({
code: "invalid_request",
details: {
issues: [{ path: [path], reason: "invalid_value", message }],
},
});
expect(starts).toBe(0);
},
);
@@ -0,0 +1,78 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
import { build } from "plist";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { deeplyNestedXmlPlist } from "../../fixtures/keyedArchiveXmlNesting.js";
import { cliTest } from "../../support/cli/cliFixture.js";
for (const [name, xml] of [
["dictionaries", deeplyNestedXmlPlist(3000)],
[
"quoted attributes",
deeplyNestedXmlPlist(3000).replaceAll("<dict>", '<dict note="/>">'),
],
[
"plist wrappers",
"<plist>".repeat(3000) + "<string>x</string>" + "</plist>".repeat(3000),
],
] as const) {
cliTest(
`compiled CLI reports deeply nested XML ${name} as the decode limit`,
async ({ cli }) => {
const root = await createTestTempDirectory("rea-keyed-cli-depth-");
const path = join(root, "Deep.plist");
await writeFile(path, xml);
const output = await cli.run({
arguments: ["inspect-keyed-archive", path, "--json"],
environment: { REA_LOG_LEVEL: "silent", REA_ANALYSIS_PROVIDER: "auto" },
});
expect(output.exitCode, output.stderr).toBe(1);
expect(output.json).toMatchObject({
code: "truncated",
category: "truncated",
details: {
reason: "limit",
truncated: true,
detail:
"Cannot decode selected Foundation keyed archive: Plist exceeds 128 nesting levels",
},
});
},
);
}
cliTest(
"compiled CLI decodes a shallow archive with opaque DOCTYPE markup",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-keyed-cli-doctype-");
const path = join(root, "Shallow.plist");
const archive = build({
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: ["$null", "leaf"],
$top: { root: { UID: 1 } },
});
await writeFile(
path,
archive.replace(
/<!DOCTYPE[^>]*>/u,
`<!DOCTYPE plist [<!ENTITY first "unused"><!ENTITY second "${"<dict>".repeat(130)}">]>`,
),
);
const output = await cli.run({
arguments: ["inspect-keyed-archive", path, "--json"],
environment: { REA_LOG_LEVEL: "silent", REA_ANALYSIS_PROVIDER: "auto" },
});
expect(output.exitCode, output.stderr).toBe(0);
expect(parseEvidence(output.json).normalized_result).toMatchObject({
archive_format: "xml-plist",
objects: expect.arrayContaining([
expect.objectContaining({ value: "leaf" }),
]),
});
},
);
@@ -0,0 +1,336 @@
import { execFile } from "node:child_process";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const execute = promisify(execFile);
// A recording port checks the experimental producer/consumer representation.
// Real provider verification remains in the separate native experiment receipts.
const generate = `
import {measureNativeEntry} from './experiments/analysis-efficiency/native-entry.mjs';
const sha='0'.repeat(64);
const duplicate=process.argv[1]==='duplicate';
const stale=process.argv[1]==='stale-dos';
const collision=process.argv[1];
const dos=process.argv[1]==='dos'||stale||collision==='load-collision';
const dossier={procedure:{address:'0x1000'},pseudocode:'return;',callers:[],callees:[]};
class ToolFailure extends Error {}
const build=async digit=>{
const parent='ev_'+digit.repeat(63)+'0';
let ordinal=0;
const call=async(name,args)=>{
if(stale)process.stdout.write(name+'\\n');
if(name==='open_binary'||name==='close_binary')return {};
if(name==='inspect_native_load_image')return {evidence_id:'ev_'+digit.repeat(63)+'6',normalized_result:{status:'verified',target_sha256:stale?'f'.repeat(64):sha,entry:{linear_address:'0x1000'}}};
if(name==='resolve_containing_procedure')return {evidence_id:'ev_'+digit.repeat(63)+'7',normalized_result:{found:true}};
if(name==='analyze_function')return {evidence_id:parent,subject:{digest:{sha256:sha}},normalized_result:dossier};
const id='ev_'+digit.repeat(63)+(collision==='containing-collision'?7:collision==='load-collision'?6:duplicate?1:++ordinal);
if(args.view.kind==='summary')return {evidence_id:id};
const facet=args.view.facet;
const item=facet==='pseudocode'?{text:dossier.pseudocode,unit:'utf16-code-units'}:dossier[facet];
return {evidence_id:id,normalized_result:{parent_evidence_id:parent,artifact:{sha256:sha},item,coverage:{exhausted:true},limitations:[],unknowns:[]}};
};
return measureNativeEntry({seed:dos?undefined:'0x1000',values:{path:'/selected/input','native-question':'entry-views'},call,inputSha:sha,ToolFailure});
};
process.stdout.write(JSON.stringify([await build('1'),await build('2')]));
`;
it.each([
"observed",
"missing",
"errored",
"wrong-target",
"wrong-entry",
"unreferenced",
])(
"binds the DOS model grade to its observed load image: %s",
async (scenario) => {
// Synthetic transcripts exercise grading; they are not real-provider or
// resource measurements. Real DOS transcripts are replayed separately.
const root = await createTestTempDirectory("rea-dos-grader-");
const trial = join(root, "trial");
const referenceDirectory = join(root, "reference");
await mkdir(trial);
await mkdir(referenceDirectory);
const sha = "0".repeat(64);
const imageId = "ev_" + "1".repeat(64);
const procedureId = "ev_" + "2".repeat(64);
const procedure = {
address: "0x1000",
name: "entry",
classification: "observed",
signature: null,
locals: [],
body: { ranges: [], total_bytes: 0 },
};
const loadImage = {
status: "verified",
target_sha256: sha,
entry: { linear_address: "0x1000" },
};
const reference = {
artifact_sha256: sha,
entry_address: "0x1000",
caller_count: 0,
callee_count: 0,
pseudocode_prefix: "return;",
containing_procedure: { procedure },
procedure,
load_image: loadImage,
};
const answer = {
...reference,
load_image_status: "verified",
evidence_ids: [
procedureId,
...(["missing", "errored", "unreferenced"].includes(scenario)
? []
: [imageId]),
],
};
const event = (tool: string, status: string, value: unknown) => ({
type: "item.completed",
item: {
type: "mcp_tool_call",
tool,
status,
arguments: {},
result: { structured_content: value },
},
});
const events = [
event("analyze_function", "completed", { evidence_id: procedureId }),
];
if (scenario !== "missing")
events.push(
event(
"inspect_native_load_image",
scenario === "errored" ? "failed" : "completed",
scenario === "errored"
? { error: { code: "unsupported_target" } }
: {
operation: "inspect_native_load_image",
evidence_id: imageId,
normalized_result: {
...loadImage,
target_sha256:
scenario === "wrong-target" ? "f".repeat(64) : sha,
entry: {
linear_address:
scenario === "wrong-entry" ? "0x2000" : "0x1000",
},
},
},
),
);
const resourceLog = join(root, "resource.log");
const output = join(root, "review.json");
for (const [path, value] of [
[
join(trial, "summary.json"),
{
completed_mcp_calls: [
{ tool: "close_binary", status: "completed", arguments: {} },
],
},
],
[join(trial, "trial-settings.json"), {}],
[join(trial, "cleanup.json"), { profile_removed: true }],
[join(trial, "answer.txt"), answer],
[join(referenceDirectory, "answer.json"), reference],
[
resourceLog,
{
resource_scope: "synthetic-grader-fixture",
abort: null,
resource_counters_verified: true,
resource_events_final: {
"memory.events": { high: 0, max: 0, oom: 0, oom_kill: 0 },
"pids.events": { max: 0 },
},
},
],
] as const)
await writeFile(path, JSON.stringify(value));
await writeFile(
join(trial, "events.jsonl"),
events.map((value) => JSON.stringify(value)).join("\n"),
);
const command = execute("python3", [
resolve("experiments/analysis-efficiency/review-model.py"),
"--domain",
"native",
"--trial",
trial,
"--reference",
referenceDirectory,
"--resource-log",
resourceLog,
"--output",
output,
]);
if (scenario === "observed") await command;
else await expect(command).rejects.toMatchObject({ code: 1 });
const report = JSON.parse(await readFile(output, "utf8"));
expect(report.all_selected_checks_pass).toBe(scenario === "observed");
if (["missing", "errored", "unreferenced"].includes(scenario))
expect(report.checks.load_image_source_observed_and_referenced).toBe(
false,
);
if (scenario === "wrong-target")
expect(report.checks["load_image.target_sha256"]).toBe(false);
if (scenario === "wrong-entry")
expect(report.checks["load_image.source_entry"]).toBe(false);
},
);
it("retains the DOS load-image source for its facts and derived entry seed", async () => {
const { stdout } = await execute(process.execPath, [
"--input-type=module",
"-e",
generate,
"dos",
]);
const [answer] = JSON.parse(stdout);
expect(answer).toMatchObject({
entry_address: "0x1000",
load_image: { status: "verified", entry: { linear_address: "0x1000" } },
evidence_ids: { native_load_image: "ev_" + "1".repeat(63) + "6" },
});
});
it("rejects a stale DOS load image before using its entry and closes the session", async () => {
await expect(
execute(process.execPath, [
"--input-type=module",
"-e",
generate,
"stale-dos",
]),
).rejects.toMatchObject({
stdout: "open_binary\ninspect_native_load_image\nclose_binary\n",
stderr: expect.stringContaining(
"The DOS load image belongs to a different target",
),
});
});
it.each(["containing-collision", "load-collision"])(
"rejects a retained view that reuses an earlier producer identity: %s",
async (collision) => {
await expect(
execute(process.execPath, [
"--input-type=module",
"-e",
generate,
collision,
]),
).rejects.toMatchObject({
stderr: expect.stringContaining(
"Retained native view reused an Evidence identity",
),
});
},
);
it("rejects a producer that reuses a view identity across facets", async () => {
await expect(
execute(process.execPath, [
"--input-type=module",
"-e",
generate,
"duplicate",
]),
).rejects.toMatchObject({
stderr: expect.stringContaining(
"Retained native view reused an Evidence identity",
),
});
});
it.each(["references", "item", "coverage", "limitations", "unknowns"])(
"separates native references while comparing %s exactly",
async (change) => {
const { stdout: generated } = await execute(process.execPath, [
"--input-type=module",
"-e",
generate,
]);
const [leftAnswer, rightAnswer] = JSON.parse(generated);
if (change === "item")
rightAnswer.retained_views[0].item.address = "0x1001";
if (change === "coverage")
rightAnswer.retained_views[0].coverage.exhausted = false;
if (change === "limitations")
rightAnswer.retained_views[0].limitations.push("partial");
if (change === "unknowns")
rightAnswer.retained_views[0].unknowns.push("unresolved");
const root = await createTestTempDirectory("rea-native-comparison-");
const left = join(root, "left");
const right = join(root, "right");
const summary = {
artifact: "sharex.exe",
native_question: "entry-views",
provider_search_path_sha256: "0".repeat(64),
native_workflow_source_sha256: "1".repeat(64),
calls: [],
failure: null,
};
for (const [directory, answer] of [
[left, leftAnswer],
[right, rightAnswer],
]) {
await mkdir(directory);
await writeFile(join(directory, "summary.json"), JSON.stringify(summary));
await writeFile(join(directory, "answer.json"), JSON.stringify(answer));
}
const output = join(root, "comparison.json");
const command = execute(process.execPath, [
resolve("experiments/analysis-efficiency/compare.mjs"),
"--left",
left,
"--right",
right,
"--output",
output,
]);
if (change !== "references") {
await expect(command).rejects.toMatchObject({
code: 1,
stdout: expect.stringContaining('"retained_views"'),
});
return;
}
const { stdout } = await command;
expect(JSON.parse(stdout)).toMatchObject({
settings_match: true,
facts_match: true,
preserved_run_specific_values: [
{
side: "left",
path: ["evidence_ids"],
value: leftAnswer.evidence_ids,
},
{
side: "right",
path: ["evidence_ids"],
value: rightAnswer.evidence_ids,
},
],
});
expect(leftAnswer.evidence_ids).not.toEqual(rightAnswer.evidence_ids);
expect(leftAnswer.evidence_ids.retained_views[0]).toMatchObject({
evidence_id: "ev_" + "1".repeat(64),
parent_evidence_id: "ev_" + "1".repeat(63) + "0",
});
expect(leftAnswer.evidence_ids.native_summary).toBe(
"ev_" + "1".repeat(63) + "5",
);
},
);
@@ -312,6 +312,14 @@ it.each([
"tests/boundary/process/darwinProcessRunTokenReader.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"tests/boundary/cli/nativePlistCli.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"tests/boundary/filesystem/windowsNpxRegistration.test.ts",
{ code: true, package: true, runtime: true, apple: false },
],
[
"src/process/ProcessOwnership.test.ts",
{ code: true, package: true, apple: false, evm: false, browser: false },
@@ -0,0 +1,174 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { parse } from "yaml";
import { z } from "zod";
import { parseTestDurations } from "../../ciTestSequencer.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const execute = promisify(execFile);
const script = new URL("../../../scripts/ci/test-report.mjs", import.meta.url);
const stepsSchema = z.array(
z.object({
name: z.string().optional(),
run: z.string().optional(),
uses: z.string().optional(),
if: z.string().optional(),
env: z.record(z.string(), z.string()).optional(),
with: z.record(z.string(), z.unknown()).optional(),
}),
);
const workflow = async () =>
z
.object({
jobs: z.object({
test: z.object({ steps: stepsSchema }),
build: z.object({ steps: stepsSchema }),
docs: z.object({ steps: stepsSchema }),
static: z.object({ steps: stepsSchema }),
}),
})
.parse(
parse(
await readFile(
new URL("../../../.github/workflows/ci.yml", import.meta.url),
"utf8",
),
),
).jobs;
it("collects failed-shard diagnostics before enforcing the shard result", async () => {
const steps = (await workflow()).test!.steps;
const merge = steps.findIndex((step) => step.run?.includes("test:ci:merge"));
const upload = steps.findIndex((step) => step.with?.name === "test-results");
const gate = steps.findIndex(
(step) => step.env?.SHARD_RESULT && step.run?.includes("test "),
);
expect(merge).toBeGreaterThan(0);
expect(upload).toBeGreaterThan(merge);
expect(gate).toBeGreaterThan(upload);
expect(steps[merge]?.if).not.toContain("needs.test-shard.result");
const command = steps[gate]?.run;
if (command === undefined) throw new Error("Missing shard result gate");
for (const result of ["failure", "cancelled", "skipped", ""]) {
await expect(
execute("bash", ["-c", command], {
env: { ...process.env, CODE_REQUIRED: "true", SHARD_RESULT: result },
}),
).rejects.toMatchObject({ code: 1 });
}
await expect(
execute("bash", ["-c", command], {
env: { ...process.env, CODE_REQUIRED: "false", SHARD_RESULT: "skipped" },
}),
).resolves.toMatchObject({ stderr: "" });
});
it("keeps diagnosis available after merge failure and publishes only complete successful history", async () => {
const steps = (await workflow()).test!.steps;
const diagnosis = steps.find((step) => step.with?.name === "test-results");
expect(diagnosis?.if).toContain("!cancelled()");
expect(diagnosis?.if).not.toContain("success()");
const history = steps.find((step) => step.with?.name === "test-timings");
expect(history?.if).toContain("success()");
expect(history?.if).toContain("needs.test-shard.result == 'success'");
expect(history?.with?.path).toBe(".cache/ci-test-history/test-results.json");
const download = (await workflow()).build!.steps.find(
(step) => step.name === "Download previous timing report",
);
expect(download?.with?.name).toBe(history?.with?.name);
});
it("runs the existing onboarding and Python producer guards from their selected CI owners", async () => {
const jobs = await workflow();
const scripts = z
.object({ scripts: z.record(z.string(), z.string()) })
.parse(
JSON.parse(
await readFile(
new URL("../../../package.json", import.meta.url),
"utf8",
),
),
).scripts;
expect(
jobs.docs.steps.some((step) => step.run === "npm run docs:check"),
).toBe(true);
expect(scripts["docs:check"]).toContain(
"tests/conformance/readmeConsistency.test.ts",
);
const producer = jobs.static.steps.find(
(step) => step.run === "npm run test:python",
);
expect(producer?.if).toBe("needs.changes.outputs.code == 'true'");
expect(scripts["test:python"]).toContain(
"bridge/native/test_rea_lldb_tracer.py",
);
});
it("exports portable timing history and removes it on a failed, partial, or invalid rerun", async () => {
const root = await createTestTempDirectory("rea-ci-report-");
const summary = join(root, "summary.md");
const history = join(root, ".cache/ci-test-history/test-results.json");
const report = {
success: true,
numFailedTests: 0,
testResults: [
{
name: join(root, "tests/slow.test.ts"),
startTime: 10,
endTime: 110,
assertionResults: [{ large: "not timing data" }],
},
{ name: join(root, "tests/fast.test.ts"), startTime: 20, endTime: 30 },
],
};
const run = async (value: unknown, shards = "success", merge = "success") => {
await writeFile(join(root, "test-results.json"), JSON.stringify(value));
await writeFile(summary, "");
return execute(process.execPath, [fileURLToPath(script)], {
cwd: root,
env: {
...process.env,
GITHUB_STEP_SUMMARY: summary,
SHARD_RESULT: shards,
MERGE_RESULT: merge,
},
});
};
await run(report);
const contents = await readFile(history, "utf8");
expect(contents).not.toContain("assertionResults");
expect(contents).not.toContain(root);
expect(
parseTestDurations(contents, resolve(root, "different-checkout")),
).toEqual(
new Map([
["tests/slow.test.ts", 100],
["tests/fast.test.ts", 10],
]),
);
for (const [value, shards, merge] of [
[{ ...report, success: false, numFailedTests: 1 }, "failure", "failure"],
[report, "cancelled", "success"],
[report, "success", "failure"],
] as const) {
await run(report);
await run(value, shards, merge);
await expect(readFile(history)).rejects.toMatchObject({ code: "ENOENT" });
expect(await readFile(summary, "utf8")).toContain("diagnostic only");
}
await run(report);
await expect(
run({
...report,
testResults: [{ name: "../outside.test.ts", startTime: 1, endTime: 3 }],
}),
).rejects.toMatchObject({ code: 1 });
await expect(readFile(history)).rejects.toMatchObject({ code: "ENOENT" });
});
@@ -70,6 +70,7 @@ describe("client configuration filesystem lifecycle", () => {
"omp",
"pi",
"hermes",
"qoder",
"zcode",
"dsh",
"grok_bot",
@@ -1660,8 +1661,9 @@ describe("uninstall configuration preflight", () => {
});
it.each([
['{"mcpServers": null}', '"mcpServers" must be an object'],
['{"mcpServers":null,"keep":1}', '"mcpServers" must be an object'],
["[]", "The configuration root must be an object"],
['{"mcpServers":"x"}', '"mcpServers" must be an object'],
])(
"names the structural problem in valid JSON %s",
async (content, problem) => {
@@ -1678,12 +1680,14 @@ describe("uninstall configuration preflight", () => {
},
});
const result = await runUninstall(false, systemUninstallHost(home));
expect(result.status).toBe("failed");
expect(result.items).toContainEqual({
name: "cursor",
status: "failed",
detail: `Configuration was not changed: ${problem}. Repair it, then rerun uninstall.`,
});
expect(await readFile(config, "utf8")).toBe(content);
await expect(readFile(`${config}.rea.backup`, "utf8")).rejects.toThrow();
},
);
@@ -0,0 +1,82 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, expect, it, vi } from "vitest";
import { inspectGhidraInstallation } from "../../../src/ghidra/GhidraInstallation.js";
import { SUPPORTED_GHIDRA_VERSION } from "../../../src/ghidra/GhidraInstallationPolicy.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const { mockSpawnSync } = vi.hoisted(() => ({ mockSpawnSync: vi.fn() }));
vi.mock("node:child_process", async (importOriginal) => ({
...(await importOriginal<typeof import("node:child_process")>()),
spawnSync: mockSpawnSync,
}));
afterEach(() => mockSpawnSync.mockReset());
it.each([
{ major: 17, fullJdk: true },
{ major: 21, fullJdk: true },
{ major: 21, fullJdk: false },
])(
"preserves the reported Unicode Java home and classifies JDK $major (full JDK: $fullJdk)",
async ({ major, fullJdk }) => {
const root = await createTestTempDirectory("rea-java-probe-");
const configuredHome = join(root, "selected-jdk");
const reportedHome = join(root, "Réa逆向", `JDK ${String(major)}`);
const installDir = join(root, "ghidra");
await mkdir(join(installDir, "Ghidra"), { recursive: true });
await writeFile(
join(installDir, "Ghidra", "application.properties"),
`application.version=${SUPPORTED_GHIDRA_VERSION}\napplication.java.min=17\n`,
);
if (fullJdk) {
await mkdir(join(reportedHome, "bin"), { recursive: true });
await writeFile(
join(
reportedHome,
"bin",
process.platform === "win32" ? "javac.exe" : "javac",
),
"compiler presence fixture",
{ mode: 0o755 },
);
}
const version = `${String(major)}.0.11`;
mockSpawnSync.mockReturnValue({
stdout: "",
stderr: ` java.version = ${version}\n java.home = ${reportedHome}\n sun.arch.data.model = 64\n`,
status: 0,
});
const inspection = inspectGhidraInstallation({
environment: {},
installDir,
javaHome: configuredHome,
});
expect(inspection.javaHome).toBe(reportedHome);
expect(inspection.javaVersion).toBe(version);
expect(inspection.checks.find(({ name }) => name === "java")).toMatchObject(
{
status: fullJdk ? "passed" : "failed",
detail: expect.stringContaining(fullJdk ? "; JDK;" : "; runtime only;"),
},
);
expect(mockSpawnSync).toHaveBeenCalledExactlyOnceWith(
join(
configuredHome,
"bin",
process.platform === "win32" ? "java.exe" : "java",
),
expect.arrayContaining([
"-Dstdout.encoding=UTF-8",
"-Dstderr.encoding=UTF-8",
"-Dsun.stdout.encoding=UTF-8",
"-Dsun.stderr.encoding=UTF-8",
]),
expect.objectContaining({ encoding: "utf8" }),
);
},
);
@@ -6,9 +6,15 @@ import { parse as parseYaml } from "yaml";
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
import {
configureClientConfiguration,
inspectClientConfiguration,
} from "../../../src/application/SetupClientConfiguration.js";
import { supportedClients } from "../../../src/application/SupportedClients.js";
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
import {
runUninstall,
systemUninstallHost,
} from "../../../src/application/Uninstall.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
@@ -90,18 +96,75 @@ describe("Hermes configuration paths", () => {
});
});
describe("Hermes YAML registration", () => {
const hermesClient = async () => {
const home = await createTestTempDirectory("rea-hermes-");
const client = supportedClients(home, process.platform, {}).find(
({ name }) => name === "hermes",
);
if (client?.markerPath === undefined)
throw new Error("missing Hermes client");
await mkdir(client.markerPath, { recursive: true });
return { home, client };
};
const hermesClient = async () => {
const home = await createTestTempDirectory("rea-hermes-");
const client = supportedClients(home, process.platform, {}).find(
({ name }) => name === "hermes",
);
if (client?.markerPath === undefined)
throw new Error("missing Hermes client");
await mkdir(client.markerPath, { recursive: true });
return { home, client };
};
describe("Hermes YAML shape diagnostics", () => {
it.each(
[
"!!set {rea: null}",
"!!omap [{rea: {command: rea, args: [mcp]}}]",
"!!timestamp 2001-12-15",
"!!binary cmVh",
].flatMap((value) => [
{
original: `${value}\n`,
problem: "The configuration root must be an object",
},
{
original: `mcp_servers: ${value}\n`,
problem: '"mcp_servers" must be an object',
},
]),
)(
"blocks setup and uninstall before changes for tagged non-record YAML: $original",
async ({ original, problem }) => {
const { home, client } = await hermesClient();
const cursor = join(home, ".cursor", "mcp.json");
const cursorOriginal =
'{"mcpServers":{"rea":{"command":"rea","args":["mcp"]}}}';
await mkdir(join(home, ".cursor"));
await writeFile(cursor, cursorOriginal);
await writeFile(client.configPath, original);
expect(
await inspectClientConfiguration(client, {}, command),
).toMatchObject({
status: "invalid",
remediation: `Configuration ${client.configPath} is malformed: ${problem}`,
});
expect(await configureClientConfiguration(client, {}, command)).toEqual({
status: "failed",
reason: "readback",
});
const result = await runUninstall(
false,
systemUninstallHost(home, undefined, {}),
);
expect(result.status).toBe("failed");
expect(result.items).toContainEqual({
name: "hermes",
status: "failed",
detail: `Configuration was not changed: ${problem}. Repair it, then rerun uninstall.`,
});
expect(await readFile(client.configPath, "utf8")).toBe(original);
expect(await readFile(cursor, "utf8")).toBe(cursorOriginal);
await expect(
readFile(`${client.configPath}.rea.backup`, "utf8"),
).rejects.toThrow();
await expect(readFile(`${cursor}.rea.backup`, "utf8")).rejects.toThrow();
},
);
});
describe("Hermes YAML registration", () => {
it("writes mcp_servers, preserves comments, reads back, and uninstalls", async () => {
const { home, client } = await hermesClient();
await writeFile(
@@ -334,13 +334,18 @@ describe("native macOS installer", { timeout: 20_000 }, () => {
});
describe("installer semantic version parsing", { timeout: 20_000 }, () => {
it.each(["1.2.3-01"])(
it.each([
"1.2.3-01",
"1.2.3-$(printf unexpected)",
'1.2.3-"; printf unexpected; #',
])(
"rejects malformed semantic versions before invoking npm: %s",
async (version) => {
const fixture = await createFixture();
await expect(
runInstaller(fixture, ["--version", version, "--dry-run"]),
).rejects.toMatchObject({
stdout: "",
stderr:
"REA installation failed: version must be an exact semantic version.\n",
});
@@ -0,0 +1,29 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { inspectBundleKeyedArchive } from "../../../src/artifacts/apple/KeyedArchiveReader.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { deeplyNestedXmlPlist } from "../../fixtures/keyedArchiveXmlNesting.js";
it.each([129, 3000])(
"reports XML nesting of %i levels as the keyed archive decode limit",
async (depth) => {
const root = await createTestTempDirectory("rea-keyed-depth-");
await writeFile(join(root, "Deep.plist"), deeplyNestedXmlPlist(depth));
const rejected = await inspectBundleKeyedArchive({
bundlePath: root,
targetSha256: "a".repeat(64),
parameters: { path: "Deep.plist" },
platform: "linux",
}).catch((cause: unknown) => cause);
// Recursive decoding exhausts the call stack before the projection limit
// can report it, so the limit diagnostic must come from the preflight.
expect(rejected).toMatchObject({
reason: "limit",
message:
"Cannot decode selected Foundation keyed archive: Plist exceeds 128 nesting levels",
});
},
);
@@ -0,0 +1,166 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
import { skillDestinations } from "../../../src/application/SetupSkill.js";
import { supportedClients } from "../../../src/application/SupportedClients.js";
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
/**
* The registration command setup writes on the host running the tests. Windows
* wraps it in `cmd.exe /d /c`, so the expected entry follows the host too.
*/
const command = npxRegistrationCommand(process.platform);
beforeEach(clearClientLocationEnvironment);
afterEach(() => vi.unstubAllEnvs());
const qoderPath = (
home: string,
platform: NodeJS.Platform = "linux",
env: Parameters<typeof supportedClients>[2] = {},
) => supportedClients(home, platform, env).find(({ name }) => name === "qoder");
describe("Qoder configuration paths", () => {
it("resolves the user settings file and marker directory", () => {
expect(qoderPath("/home/a")).toEqual({
name: "qoder",
displayName: "Qoder",
configPath: join("/home/a", ".qoder", "settings.json"),
markerPath: join("/home/a", ".qoder"),
format: "json",
});
expect(qoderPath("/Users/a", "darwin")?.configPath).toBe(
join("/Users/a", ".qoder", "settings.json"),
);
expect(qoderPath("C:\\Users\\a", "win32")?.configPath).toBe(
join("C:\\Users\\a", ".qoder", "settings.json"),
);
});
it("honors the QODER_CONFIG_DIR override", () => {
expect(
qoderPath("/home/a", "linux", { QODER_CONFIG_DIR: "/opt/qoder" })
?.configPath,
).toBe(join("/opt/qoder", "settings.json"));
expect(
qoderPath("/home/a", "linux", { QODER_CONFIG_DIR: "" })?.configPath,
).toBe(join("/home/a", ".qoder", "settings.json"));
});
it("resolves the personal skills directory beside the config root", () => {
expect(
skillDestinations("/home/a", ["qoder"]).map(({ path }) => path),
).toEqual([
join("/home/a", ".qoder", "skills", "reverse-engineer-anything"),
]);
expect(
skillDestinations("/home/a", ["qoder"], {
QODER_CONFIG_DIR: "/opt/qoder",
}).map(({ path }) => path),
).toEqual([join("/opt/qoder", "skills", "reverse-engineer-anything")]);
});
});
describe("Qoder JSON registration", () => {
const qoderClient = async () => {
const home = await createTestTempDirectory("rea-qoder-");
const client = qoderPath(home, process.platform);
if (client?.markerPath === undefined)
throw new Error("missing Qoder client");
await mkdir(client.markerPath, { recursive: true });
return { home, client };
};
it("upserts rea into mcpServers while preserving unrelated settings", async () => {
const { home, client } = await qoderClient();
await writeFile(
client.configPath,
`{
"mcpServers": {
"other": { "command": "other-server" }
},
"model": { "name": "qoder-model" }
}`,
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.model).toEqual({ name: "qoder-model" });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
expect(await configureClientConfiguration(client, {}, command)).toEqual({
status: "unchanged",
});
expect(
(
await readClientRegistrationStatuses(home, undefined, {
platform: process.platform,
environment: {},
})
).find(({ client }) => client === "qoder"),
).toMatchObject({ state: "aligned", command });
});
it("creates the mcpServers table in an existing settings file without servers", async () => {
const { client } = await qoderClient();
await writeFile(
client.configPath,
JSON.stringify({ general: { vimMode: false } }, null, 2),
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.general).toEqual({ vimMode: false });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("registers into an empty settings file", async () => {
const { client } = await qoderClient();
await writeFile(client.configPath, "");
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("removes only the rea entry on uninstall", async () => {
const { home, client } = await qoderClient();
await writeFile(
client.configPath,
JSON.stringify(
{ mcpServers: { other: { command: "other-server" } } },
null,
2,
),
);
await configureClientConfiguration(client, {}, command);
expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
"removed",
);
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.mcpServers.rea).toBeUndefined();
});
});
+89 -1
View File
@@ -5,7 +5,11 @@ import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { resolveClientConfigTransactionPath } from "../../../src/application/ClientConfigPath.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
import { readConfigurationFile } from "../../../src/application/ClientConfigurationFiles.js";
import {
configureClientConfiguration,
inspectClientConfiguration,
} from "../../../src/application/SetupClientConfiguration.js";
import { PRODUCT_IDENTITY } from "../../../src/identity.js";
const pinnedPackage = PRODUCT_IDENTITY.registrationPackageSpecifier;
@@ -330,3 +334,87 @@ describe("JSON client configuration migration and provider paths", () => {
).rejects.toThrow();
});
});
describe("JSON client configuration shape diagnostics", () => {
it.each([
{
format: "json" as const,
original:
'{"__proto__":{"mcpServers":{"rea":{"command":"rea","args":["mcp"]}}}}',
serversKey: "mcpServers",
},
{
format: "opencode" as const,
original:
'{"mcp":{"__proto__":{"servers":{"rea":{"type":"local","command":["rea","mcp"]}}}}}',
serversKey: "mcp",
},
])(
"registers an own server instead of trusting inherited $format settings",
async ({ format, original, serversKey }) => {
const directory = await createTestTempDirectory("rea-setup-prototype-");
const configPath = join(directory, "mcp.json");
const client = {
name: format === "json" ? "cursor" : "opencode",
configPath,
format,
};
await writeFile(configPath, original);
expect(
await inspectClientConfiguration(client, {}, ["rea", "mcp"]),
).toMatchObject({ status: "update" });
expect(
await configureClientConfiguration(client, {}, ["rea", "mcp"]),
).toMatchObject({ status: "configured" });
const document = JSON.parse(await readFile(configPath, "utf8"));
expect(Object.hasOwn(document, serversKey)).toBe(true);
expect(Object.hasOwn(document[serversKey], "rea")).toBe(true);
expect(document[serversKey].rea.command).toEqual(
format === "json" ? "rea" : ["rea", "mcp"],
);
const originalDocument = JSON.parse(original);
expect(
format === "json" ? document.__proto__ : document[serversKey].__proto__,
).toEqual(
format === "json"
? originalDocument.__proto__
: originalDocument[serversKey].__proto__,
);
expect(await readFile(`${configPath}.rea.backup`, "utf8")).toBe(original);
expect(
await inspectClientConfiguration(client, {}, ["rea", "mcp"]),
).toMatchObject({ status: "already_current" });
},
);
it.each([
['{"mcpServers":null,"keep":1}', '"mcpServers" must be an object'],
["[]", "The configuration root must be an object"],
['{"mcpServers":"x"}', '"mcpServers" must be an object'],
] as const)(
"names structural shape problems in setup remediation: %s",
async (original, problem) => {
const directory = await createTestTempDirectory("rea-setup-shape-");
const configPath = join(directory, "mcp.json");
const client = {
name: "cursor",
configPath,
format: "json" as const,
};
await writeFile(configPath, original);
const read = await readConfigurationFile(configPath, "json");
expect(read.ok).toBe(false);
if (read.ok) throw new Error("expected malformed configuration");
expect(read.error.detail).toBe(
`Configuration ${configPath} is malformed: ${problem}`,
);
expect(
await inspectClientConfiguration(client, {}, ["rea", "mcp"]),
).toEqual({
status: "invalid",
remediation: `Configuration ${configPath} is malformed: ${problem}`,
});
expect(await readFile(configPath, "utf8")).toBe(original);
},
);
});
@@ -0,0 +1,108 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { z } from "zod";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { workflowGit } from "../../support/workflowGit.js";
const execute = promisify(execFile);
it.skipIf(process.platform === "android")(
"reruns test validation after repository or external input changes while reusing preparation",
async () => {
const config = z
.object({
tasks: z.record(z.string(), z.record(z.string(), z.unknown())),
})
.parse(
JSON.parse(
await readFile(
new URL("../../../turbo.json", import.meta.url),
"utf8",
),
),
);
const testTask = config.tasks["test:run"];
const dependencies = z.array(z.string()).parse(testTask?.dependsOn);
const root = await createTestTempDirectory("rea-test-cache-");
const external = await createTestTempDirectory("rea-test-dependency-");
const dependency = join(external, "provider.json");
const scripts: Record<string, string> = { "test:run": "node check.mjs" };
const tasks: Record<string, unknown> = { "test:run": testTask };
for (const [index, task] of dependencies.entries()) {
scripts[task] = `node prepare.mjs ${index}`;
tasks[task] = {
inputs: ["artifact-input.txt"],
outputs: [`prepared-${index}.txt`],
};
}
await writeFile(
join(root, "package.json"),
JSON.stringify({
name: "rea-cache-fixture",
version: "1.0.0",
private: true,
type: "module",
packageManager: "npm@11.16.0",
scripts,
}),
);
await writeFile(
join(root, "package-lock.json"),
JSON.stringify({
name: "rea-cache-fixture",
version: "1.0.0",
lockfileVersion: 3,
packages: { "": { name: "rea-cache-fixture", version: "1.0.0" } },
}),
);
await writeFile(join(root, "turbo.json"), JSON.stringify({ tasks }));
await writeFile(
join(root, "artifact-input.txt"),
"unchanged preparation input",
);
await writeFile(join(root, "README_zh.md"), "aligned");
await writeFile(dependency, "healthy");
await writeFile(
join(root, "prepare.mjs"),
'import { appendFileSync } from "node:fs"; appendFileSync(`prepared-${process.argv[2]}.txt`, "prepared\\n");',
);
await writeFile(
join(root, "check.mjs"),
`import { readFileSync } from "node:fs"; if(readFileSync("README_zh.md","utf8") !== "aligned" || readFileSync(${JSON.stringify(dependency)},"utf8") !== "healthy") throw new Error("current input is invalid");`,
);
await workflowGit(root, ["init", "--initial-branch=main"]);
await workflowGit(root, ["add", "."]);
await workflowGit(root, ["commit", "-m", "test: seed cache fixture"]);
const run = () =>
execute(
process.execPath,
[
join(process.cwd(), "node_modules/turbo/bin/turbo"),
"run",
"test:run",
"--output-logs=errors-only",
],
{ cwd: root, env: { ...process.env, TURBO_DAEMON: "false" } },
);
await run();
await writeFile(join(root, "README_zh.md"), "wrong command");
await expect(run()).rejects.toMatchObject({ code: 1 });
await writeFile(join(root, "README_zh.md"), "aligned");
await run();
await writeFile(dependency, "faulty");
await expect(run()).rejects.toMatchObject({ code: 1 });
await expect(run()).rejects.toMatchObject({ code: 1 });
await writeFile(dependency, "healthy");
await run();
for (const index of dependencies.keys()) {
expect(await readFile(join(root, `prepared-${index}.txt`), "utf8")).toBe(
"prepared\n",
);
}
},
);
+42 -4
View File
@@ -19,7 +19,7 @@ const requestSchema = z.object({
.passthrough()
.optional(),
});
const fixture = async (redirectUrl?: string, cycle = false) => {
const fixture = async (redirectUrl?: string, cycle = false, native = false) => {
const methods: string[] = [];
const headers: (string | undefined)[] = [];
const server = createServer(async (request, response) => {
@@ -47,7 +47,7 @@ const fixture = async (redirectUrl?: string, cycle = false) => {
input.method === "initialize"
? {
protocolVersion: input.params?.protocolVersion,
capabilities: { tools: {} },
capabilities: { tools: {}, ...(native ? { resources: {} } : {}) },
serverInfo: {
name: "ida-transport-fixture",
version: "observed-server-version",
@@ -72,7 +72,32 @@ const fixture = async (redirectUrl?: string, cycle = false) => {
}
: {}),
}
: { content: [], structuredContent: { observed: true } };
: input.method === "resources/read"
? {
contents: [
{
uri: "ida://idb/metadata",
mimeType: "application/json",
text: JSON.stringify({
path: "/fixture/input.i64",
sha256: "a".repeat(64),
base: "0x1000",
module: "input",
}),
},
],
}
: native
? {
content: [
{
type: "text",
text: JSON.stringify([{ observed: true }]),
},
],
structuredContent: { result: [], _output_truncated: true },
}
: { content: [], structuredContent: { observed: true } };
response.writeHead(200, { "Content-Type": "application/json" });
response.end(JSON.stringify({ jsonrpc: "2.0", id: input.id, result }));
});
@@ -93,7 +118,8 @@ const fixture = async (redirectUrl?: string, cycle = false) => {
{
url: `http://127.0.0.1:${address.port}/mcp`,
headers: { Authorization: "Bearer transport-fixture-secret" },
mode: "headless",
mode: native ? "attached" : "headless",
...(native ? { protocol: "native" as const } : {}),
timeoutMs: 2000,
},
{},
@@ -138,3 +164,15 @@ it("rejects nonterminating SDK pagination instead of returning a partial tool ca
methods.filter((method) => method === "tools/list").length,
).toBeGreaterThan(2);
});
it("reads native metadata resources and complete JSON text through the real SDK transport", async () => {
const { connection, methods } = await fixture(undefined, false, true);
expect(await connection.connect()).toEqual(["first", "second"]);
expect(await connection.call("first", {})).toEqual([{ observed: true }]);
expect(await connection.readResource?.("ida://idb/metadata")).toMatchObject({
path: "/fixture/input.i64",
sha256: "a".repeat(64),
});
await expect(connection.readResource?.("ida://other")).rejects.toThrow();
expect(methods).toContain("resources/read");
});
@@ -0,0 +1,73 @@
import { spawn } from "node:child_process";
import { describe, expect, it } from "vitest";
import { observeSettlement } from "../../../src/process/capture/ProcessCaptureLifecycle.js";
import { observeOwnedProcessGroups } from "../../../src/process/ProcessOwnershipObservation.js";
/** Process group id that cannot exist on any supported host. */
const ABSENT_GROUP_ID = 4_294_967_295;
const observeGroup = async (groupId: number) =>
observeOwnedProcessGroups("settlement-deadline-probe", [groupId]);
describe("settlement deadline classification", () => {
it("does not report a single empty observation as alive_at_deadline", async () => {
expect(
(await observeGroup(ABSENT_GROUP_ID)).get(ABSENT_GROUP_ID)?.state,
).toBe("empty");
const settlement = await observeSettlement(
"settlement-single-empty",
[ABSENT_GROUP_ID],
0,
() => undefined,
"linux",
);
expect(settlement.state).toBe("unverifiable");
});
it("still reports observed liveness as alive_at_deadline", async () => {
const runId = "settlement-live-group";
// A detached child leads its own process group and carries the run token,
// so the group observes as a genuinely owned live member.
const child = spawn("sleep", ["30"], {
detached: true,
stdio: "ignore",
env: { ...process.env, REA_PROCESS_RUN_ID: runId },
});
child.unref();
try {
// Wait until the child is actually listed before asserting on it.
const started = Date.now();
while (
(await observeOwnedProcessGroups(runId, [child.pid!])).get(child.pid!)
?.state !== "alive"
) {
if (Date.now() - started > 5_000)
throw new Error("live fixture group never observed as alive");
await new Promise((resolve) => setTimeout(resolve, 50));
}
const settlement = await observeSettlement(
runId,
[child.pid!],
0,
() => undefined,
"linux",
);
expect(settlement.state).toBe("alive_at_deadline");
} finally {
child.kill("SIGKILL");
}
});
it("still reports two consecutive empty observations as quiesced", async () => {
const settlement = await observeSettlement(
"settlement-two-empty",
[ABSENT_GROUP_ID],
1_000,
() => undefined,
"linux",
);
expect(settlement.state).toBe("quiesced");
});
});
@@ -0,0 +1,266 @@
import { createHash } from "node:crypto";
import {
copyFile,
mkdtemp,
readFile,
readdir,
rm,
writeFile,
} from "node:fs/promises";
import { basename, dirname, join } from "node:path";
import { afterEach, expect, test, vi } from "vitest";
import { AdbProvider } from "../../../../src/adb/AdbProvider.js";
import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js";
import {
execFileBytes,
execFileOutput,
} from "../../../../src/process/ExecFileOutput.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
// Exercise real filesystem publication; the Android process reply is simulated.
vi.mock(
"../../../../src/process/ExecFileOutput.js",
async (importOriginal) => ({
...(await importOriginal<
typeof import("../../../../src/process/ExecFileOutput.js")
>()),
execFileBytes: vi.fn(),
execFileOutput: vi.fn(),
}),
);
vi.mock("node:fs/promises", async (importOriginal) => {
const actual = await importOriginal<typeof import("node:fs/promises")>();
return {
...actual,
copyFile: vi.fn(actual.copyFile),
mkdtemp: vi.fn(actual.mkdtemp),
rm: vi.fn(actual.rm),
};
});
afterEach(() => vi.restoreAllMocks());
const fixture = async (duringCollection?: (root: string) => Promise<void>) => {
const root = await createTestTempDirectory("rea-adb-bugreport-");
const binary = join(root, "adb");
let reportedPath = "";
vi.mocked(execFileOutput).mockImplementation(async (_binary, args) => {
expect(args).toEqual(["version"]);
return {
stdout: `Android Debug Bridge version 1.0.41\nVersion 34.0.5-stub\nInstalled as ${binary}\n`,
stderr: "",
};
});
vi.mocked(execFileBytes).mockImplementation(async (_binary, args) => {
expect(args.slice(0, 3)).toEqual(["-s", "device", "bugreport"]);
const staging = args[3]!;
reportedPath = join(staging, "bugreport test.zip");
await writeFile(reportedPath, "bugreport-bytes");
await duringCollection?.(root);
return {
stdout: Buffer.from(`Bug report copied to ${reportedPath}\n`),
stderr: Buffer.alloc(0),
};
});
const provider = new AdbProvider({ environment: { REA_ADB_PATH: binary } });
return {
root,
provider,
reportedPath: () => reportedPath,
request: {
operation: "collect_adb_bugreport" as const,
input: { serial: "device", output_directory: root },
},
};
};
test.each(["before", "during"] as const)(
"preserves a same-name caller archive created %s collection",
async (when) => {
const existing = async (root: string) => {
await writeFile(join(root, "bugreport test.zip"), "existing evidence");
await writeFile(join(root, "bugreport test (1).zip"), "other evidence");
};
const f = await fixture(when === "during" ? existing : undefined);
if (when === "before") await existing(f.root);
const result = await f.provider.execute(f.request);
if (!result.ok) throw result.error;
const localPath = join(f.root, "bugreport test (2).zip");
expect(result.value.result).toMatchObject({
local_path: localPath,
adb_reported_path: f.reportedPath(),
bytes: 15,
sha256: createHash("sha256").update("bugreport-bytes").digest("hex"),
});
expect(await readFile(join(f.root, "bugreport test.zip"), "utf8")).toBe(
"existing evidence",
);
expect(await readFile(join(f.root, "bugreport test (1).zip"), "utf8")).toBe(
"other evidence",
);
expect(await readFile(localPath, "utf8")).toBe("bugreport-bytes");
expect((await readdir(f.root)).sort()).toEqual([
"bugreport test (1).zip",
"bugreport test (2).zip",
"bugreport test.zip",
]);
await f.provider.close();
},
);
test("retains a completed archive after failed publication and later provider work", async () => {
const f = await fixture();
const existing = join(f.root, "bugreport test.zip");
await writeFile(existing, "existing evidence");
const publicationFailure = Object.assign(new Error("no space left"), {
code: "ENOSPC",
});
vi.mocked(copyFile).mockRejectedValueOnce(publicationFailure);
const failed = await f.provider.execute(f.request);
if (failed.ok) throw new Error("expected publication failure");
const retainedPath = f.reportedPath();
expect(failed.error._tag).toBe("AnalysisOutputError");
expect(failed.error.cause).toBe(publicationFailure);
expect(
projectAnalysisError(failed.error).details?.partial_observation,
).toMatchObject({
provider_id: "adb",
operation: f.request.operation,
result: {
serial: "device",
local_path: retainedPath,
adb_reported_path: retainedPath,
digest_status: "unknown",
},
});
expect(await readFile(retainedPath, "utf8")).toBe("bugreport-bytes");
expect(await readFile(existing, "utf8")).toBe("existing evidence");
// New work and terminal cleanup must not reclaim the retained acquisition.
const next = await f.provider.execute(f.request);
if (!next.ok) throw next.error;
expect(next.value.result).toMatchObject({
local_path: join(f.root, "bugreport test (1).zip"),
bytes: 15,
sha256: createHash("sha256").update("bugreport-bytes").digest("hex"),
});
await f.provider.close();
expect(await readFile(retainedPath, "utf8")).toBe("bugreport-bytes");
expect(await readFile(existing, "utf8")).toBe("existing evidence");
expect(await readFile(join(f.root, "bugreport test (1).zip"), "utf8")).toBe(
"bugreport-bytes",
);
expect((await readdir(f.root)).sort()).toEqual(
[
basename(dirname(retainedPath)),
"bugreport test.zip",
"bugreport test (1).zip",
].sort(),
);
});
test("retains final and producer paths when digest verification is cancelled", async () => {
const f = await fixture();
const controller = new AbortController();
const actualCopy =
await vi.importActual<typeof import("node:fs/promises")>(
"node:fs/promises",
);
vi.mocked(copyFile).mockImplementationOnce(async (...args) => {
await actualCopy.copyFile(...args);
controller.abort();
});
const result = await f.provider.execute(f.request, {
signal: controller.signal,
});
expect(result.ok).toBe(false);
if (result.ok) throw new Error("expected cancellation");
expect(result.error._tag).toBe("AnalysisCancelledError");
expect(result.error.partialObservation).toMatchObject({
operation: f.request.operation,
result: {
local_path: join(f.root, "bugreport test.zip"),
adb_reported_path: f.reportedPath(),
digest_status: "unknown",
},
});
expect(await readFile(join(f.root, "bugreport test.zip"), "utf8")).toBe(
"bugreport-bytes",
);
expect(await readdir(f.root)).toEqual(["bugreport test.zip"]);
await f.provider.close();
});
test("returns staging allocation failures without invoking device collection", async () => {
const f = await fixture();
vi.mocked(mkdtemp).mockRejectedValueOnce(new Error("staging denied"));
const result = await f.provider.execute(f.request);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("expected output failure");
expect(result.error._tag).toBe("AnalysisOutputError");
expect(execFileBytes).not.toHaveBeenCalled();
await f.provider.close();
});
test("retains published evidence and retries failed staging cleanup", async () => {
const f = await fixture();
vi.mocked(rm).mockRejectedValueOnce(new Error("cleanup denied"));
const result = await f.provider.execute(f.request);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("expected cleanup failure");
expect(result.error.cleanupResources).toContain(dirname(f.reportedPath()));
expect(result.error.partialObservation).toMatchObject({
operation: f.request.operation,
result: {
local_path: join(f.root, "bugreport test.zip"),
adb_reported_path: f.reportedPath(),
bytes: 15,
sha256: createHash("sha256").update("bugreport-bytes").digest("hex"),
},
});
await f.provider.close();
expect(await readdir(f.root)).toEqual(["bugreport test.zip"]);
});
test("refuses an archive reported outside the owned staging directory", async () => {
const f = await fixture();
const existing = join(f.root, "bugreport test.zip");
await writeFile(existing, "existing evidence");
vi.mocked(execFileBytes).mockResolvedValueOnce({
stdout: Buffer.from(`Bug report copied to ${existing}\n`),
stderr: Buffer.alloc(0),
});
const result = await f.provider.execute(f.request);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("expected protocol refusal");
expect(result.error._tag).toBe("AnalysisProtocolError");
expect(await readFile(existing, "utf8")).toBe("existing evidence");
expect(await readdir(f.root)).toEqual(["bugreport test.zip"]);
await f.provider.close();
});
test("preserves the primary protocol failure when staging cleanup also fails", async () => {
const f = await fixture();
vi.mocked(execFileBytes).mockResolvedValueOnce({
stdout: Buffer.from("unsupported completion format\n"),
stderr: Buffer.alloc(0),
});
vi.mocked(rm).mockRejectedValueOnce(new Error("cleanup denied"));
const result = await f.provider.execute(f.request);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("expected protocol and cleanup failure");
expect(result.error.cause).toMatchObject({
errors: [
{
_tag: "AnalysisProtocolError",
message: expect.stringContaining("did not report an archive path"),
},
expect.any(Error),
],
});
expect(result.error.cleanupIncomplete).toBe(true);
expect(result.error.cleanupResources).toHaveLength(1);
await f.provider.close();
expect(await readdir(f.root)).toEqual([]);
});
@@ -374,9 +374,7 @@ describe("evidence metadata imports and snapshot cache", () => {
expect(session.exportAnalysisSnapshot()).toMatchObject({
ok: false,
error: {
message: expect.stringContaining(
"Analysis snapshots are unavailable because analysis metadata may have changed",
),
_tag: "EvidenceIntegrityError",
},
});
expect((await session.open(target)).ok).toBe(true);
@@ -384,9 +382,7 @@ describe("evidence metadata imports and snapshot cache", () => {
expect(session.importAnalysisSnapshot(baseline.value)).toMatchObject({
ok: false,
error: {
userMessage: expect.stringContaining(
"Close the active target without saving a snapshot",
),
_tag: "EvidenceIntegrityError",
},
});
expect(session.exportEvidenceBundle().records).toContainEqual(relocated);
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff. Show more