diff --git a/.agents/skills/reverse-engineer-anything/SKILL.md b/.agents/skills/reverse-engineer-anything/SKILL.md
index 39f9ae20b2..426a05ae56 100644
--- a/.agents/skills/reverse-engineer-anything/SKILL.md
+++ b/.agents/skills/reverse-engineer-anything/SKILL.md
@@ -2,7 +2,7 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
- version: "41"
+ version: "42"
---
# REA
@@ -31,6 +31,49 @@ Diagnose the observed blocker, continue useful supported work, and resume the
original task after repair. Client approval, shell PATH, registration and engine
availability have different remedies; avoid repeating setup for all of them.
+1. Diagnose without changing REA or client configuration. `npx` may download
+ the package and write to npm's cache. For Codex, run
+ `npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
+ supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
+ `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
+ `commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
+ file registration: doctor reports the Grok Bot chat step, and setup does not
+ write `mcp.json`. If the client is unknown, use `doctor --json` and
+ inspect its registration results before choosing a setup scope.
+2. Distinguish the reason. Missing, malformed, or stale registration needs a
+ scoped configuration repair. An aligned registration with no tools in the
+ active session needs a restart/reconnection; doctor cannot prove that the
+ current agent has connected. A missing provider affects only tasks requiring
+ that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
+ and Android inspection has separate bring-your-own JADX/Java prerequisites.
+3. For a configuration repair, prepare the read-only plan:
+ `npx -y rea-agents@latest setup --client codex --dry-run --json`.
+ Use the current client's ID, show its exact proposed paths, backups, and
+ changes, and obtain approval before setup writes configuration or installs
+ Hopper. Setup normally installs the matching bundled skill too; include that
+ replacement in the reviewed plan. After approval, apply the same scope with
+ `npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
+ only if that separate installation was needed and explicitly approved.
+4. Restart/reconnect the affected agent. Verify that REA tools actually appear
+ in the session, then resume the original investigation. If they remain
+ absent, inspect the client's MCP launch error rather than repeating setup.
+
+REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
+JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
+software to repair
+MCP registration. For an unsupported client, use manual stdio registration
+with a version-pinned `rea-agents` package or continue through the CLI.
+
+A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
+
+```bash
+npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
+```
+
+No MCP registration or native engine is required. Read the returned Evidence,
+graph, limitations, and unknowns with the same care as an MCP result; the CLI
+returns the Evidence record directly. This fallback does not establish that
+MCP is configured. Native CLI tasks still require their selected engine.
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
@@ -61,6 +104,10 @@ explicit path or endpoint and do not need it.
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
optional metadata recovery and its supported host/layout boundary.
- User-owned browser page already open: `list_browser_targets`.
+- Browser action the agent must perform (click, type, navigate):
+ `capture_browser_scenario` with declared actions. See
+ [runtime observation](references/runtime-observation.md) for connect/launch
+ and the focused follow-up browser tools.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -75,7 +122,7 @@ explicit path or endpoint and do not need it.
pages; select one trace, stream ID and direction for contribution/range pages.
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
and PCAP correlation are unsupported.
-- User-owned Electron runtime already open: `list_electron_targets`.
+- User-owned Electron runtime already open: `list_electron_targets`. Before an electron-main Inspector attempt, check packaged fuses with `inspect_electron_fuses`; disabled `EnableNodeCliInspectArguments` means the app ignores `--inspect`.
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
@@ -280,7 +327,8 @@ as complete while required questions remain open.
[references/javascript-applications.md](references/javascript-applications.md)
- Android APK declarations, classes, methods, and static references:
[references/android-applications.md](references/android-applications.md)
-- Passive browser/Electron observation and static/runtime reconciliation:
+- Browser/Electron observation (passive and scenario) and static/runtime
+ reconciliation:
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)
diff --git a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md
index c651287f3e..539bfb6b2f 100644
--- a/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md
+++ b/.agents/skills/reverse-engineer-anything/references/evidence-workflows.md
@@ -65,3 +65,7 @@ Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
method, origin and path template. Format matches are inferences, not handler
identity or schemas. Group pages cite parent record ordinals; consult record
coverage and exclusions before claiming completeness. No requests are replayed.
+When the connected schema advertises `include_query_parameter_names`, select
+it in the view for distinct observed names with original record ordinals. Query
+values are omitted; the names establish neither required parameters nor a
+complete server schema. Keep the original parent Evidence for exact URLs.
diff --git a/.agents/skills/reverse-engineer-anything/references/runtime-observation.md b/.agents/skills/reverse-engineer-anything/references/runtime-observation.md
index 2d660d675d..df75bcaf3d 100644
--- a/.agents/skills/reverse-engineer-anything/references/runtime-observation.md
+++ b/.agents/skills/reverse-engineer-anything/references/runtime-observation.md
@@ -1,4 +1,4 @@
-# Passive browser and Electron observation
+# Browser and Electron runtime observation
Supply the existing browser or Electron instance's literal-port loopback CDP
endpoint in the request. No separate permission configuration or per-call
@@ -7,13 +7,16 @@ approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
selected page's current origin. Electron discovery lists eligible local
`file://` targets without a configured file-root allowlist.
-Start browser work with `list_browser_targets`; start Electron runtime work with
-`list_electron_targets`. Inspect the target selected for the user's task. Observation
-is passive: never claim REA clicked, navigated, evaluated page JavaScript,
-invoked IPC, captured prior activity, or contained the page's network.
+Start passive browser work with `list_browser_targets`; start Electron runtime work with
+`list_electron_targets`. Before an electron-main Inspector attempt, check
+packaged fuses with `inspect_electron_fuses`. Inspect the target selected for
+the user's task. Observation with these tools is passive: never claim REA
+clicked, navigated, evaluated page JavaScript, invoked IPC, or captured prior
+activity. REA does not contain the page's network.
-Credentials, cookies, authorization headers, storage values, and raw
-WebSocket/JSON bodies are deliberately absent. Local URLs retain ordinary query
+Passive captures omit raw header collections, credential header values, storage
+values, and raw WebSocket/JSON bodies. Selected response metadata, including
+content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
values and fragments; userinfo credentials are removed. Browser accessibility
text, console primitives, value-free shapes, script sources, source maps, and
storage key names have request-specific capture options. Electron script
@@ -21,6 +24,38 @@ sources are also optional. Use `capture_web_screenshot` when pixels are needed.
Treat scope filtering, truncation, and attach-window coverage as limitations.
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
+## Who performs the action
+
+- The user performs it: `list_browser_targets`, then `inspect_web_page` or
+ `observe_web_session` on the selected target.
+- The agent must perform it: one `capture_browser_scenario` with declared
+ actions. Use connect mode with the loopback CDP endpoint and `target_id` for
+ an existing browser, or launch mode with a caller-selected executable for a
+ temporary profile. Connect mode first navigates the selected page to
+ `start_url`, which can clear unsaved form values and other temporary state.
+ Recreate the state needed for the investigation in the declared actions.
+ Connect mode leaves the external browser open; launch mode owns a temporary
+ browser and profile. Check `browser.cleanup` for the outcome.
+ Request only the DOM, accessibility, or network details needed for the question.
+ The default final capture contains the URL; step outcomes are reported separately.
+ Add a wait action for the expected page state before capturing it. Report that
+ REA performed the declared actions.
+
+Scenario network records have no request initiator stacks. For that evidence,
+use passive capture while the user performs the action. Listener source locations
+identify candidate handlers; they do not establish which script sent a particular
+request. Avoid overlapping passive capture and scenario actions on the same target.
+
+Focused follow-up tools:
+
+- `observe_web_execution`: execution coverage during a window; it resets
+ counters and temporarily disables optimization.
+- `inspect_web_event_listeners`: source locations for listeners registered
+ directly on the first CSS match, without dispatching an event.
+- `compare_web_captures`: two normalized passive captures or recorded scenarios.
+- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
+ the discovered tools.
+
Use `reconcile_javascript_runtime` only after static application Evidence and
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
diff --git a/.github/actions/npm-cache/action.yml b/.github/actions/npm-cache/action.yml
new file mode 100644
index 0000000000..1af22da2b9
--- /dev/null
+++ b/.github/actions/npm-cache/action.yml
@@ -0,0 +1,42 @@
+name: Cache npm package downloads
+description: Restore npm's download cache and save it only from trusted default-branch events.
+outputs:
+ can-save:
+ description: Whether this run may update the shared npm download cache.
+ value: ${{ steps.context.outputs.can-save }}
+ path:
+ description: The npm download-cache directory.
+ value: ${{ steps.context.outputs.path }}
+runs:
+ using: composite
+ steps:
+ - id: context
+ shell: bash
+ env:
+ DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+ LOCKFILES_HASH: ${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}
+ run: |
+ cache_path="$(npm config get cache)"
+ runtime_arch="$(node -p 'process.arch')"
+ can_save=false
+ if [[ -n "${DEFAULT_BRANCH}" && "${GITHUB_REF}" == "refs/heads/${DEFAULT_BRANCH}" && ( "${GITHUB_EVENT_NAME}" == "push" || "${GITHUB_EVENT_NAME}" == "schedule" || "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ) ]]; then
+ can_save=true
+ fi
+ cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
+ {
+ printf 'can-save=%s\n' "${can_save}"
+ printf 'path=%s\n' "${cache_path}"
+ printf 'key=%s\n' "${cache_key}"
+ } >> "${GITHUB_OUTPUT}"
+ - name: Restore and save npm package downloads
+ if: steps.context.outputs.can-save == 'true'
+ uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+ with:
+ path: ${{ steps.context.outputs.path }}
+ key: ${{ steps.context.outputs.key }}
+ - name: Restore npm package downloads without saving
+ if: steps.context.outputs.can-save != 'true'
+ uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+ with:
+ path: ${{ steps.context.outputs.path }}
+ key: ${{ steps.context.outputs.key }}
diff --git a/.github/workflows/ci-timings.yml b/.github/workflows/ci-timings.yml
index 73a9ba4b97..74f27af32c 100644
--- a/.github/workflows/ci-timings.yml
+++ b/.github/workflows/ci-timings.yml
@@ -12,12 +12,13 @@ concurrency:
permissions:
contents: read
actions: read
+ pull-requests: read
jobs:
report:
name: Collect recent completed workflow attempts
runs-on: ubuntu-latest
- timeout-minutes: 5
+ timeout-minutes: 10
steps:
# Consume only API data and execute batch code from the trusted default branch.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
@@ -27,6 +28,18 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
+ - name: Cache npm package downloads
+ id: npm-cache
+ continue-on-error: true
+ uses: ./.github/actions/npm-cache
+ - name: Diagnose missing and startup-failed PR CI
+ continue-on-error: true
+ env:
+ GH_TOKEN: ${{ github.token }}
+ CI_REPOSITORY: ${{ github.repository }}
+ CI_REPOSITORY_ID: ${{ github.repository_id }}
+ CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+ run: node scripts/ci/pr-ci-health.mjs
- name: Record queue, execution, and matched step timing
continue-on-error: true
env:
@@ -44,3 +57,28 @@ jobs:
if-no-files-found: error
overwrite: true
retention-days: 14
+ - name: Install dependencies for static failure attribution
+ id: failure-deps
+ if: ${{ !cancelled() && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
+ continue-on-error: true
+ run: npm ci
+ - name: Attribute supported static failures against main
+ if: ${{ !cancelled() && steps.failure-deps.outcome == 'success' && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
+ continue-on-error: true
+ env:
+ GH_TOKEN: ${{ github.token }}
+ CI_REPOSITORY: ${{ github.repository }}
+ CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+ run: node scripts/ci/failure-report.mjs
+ - name: Upload CI health and failure diagnostics
+ if: ${{ !cancelled() }}
+ continue-on-error: true
+ uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+ with:
+ name: ci-health-diagnostics
+ path: |
+ .cache/ci-pr-health/
+ .cache/ci-failures/
+ include-hidden-files: true
+ if-no-files-found: warn
+ retention-days: 14
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 8b47fcf895..f55667dce7 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -42,6 +42,7 @@ jobs:
tier: ${{ steps.plan.outputs.tier }}
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
code: ${{ steps.plan.outputs.code }}
+ reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
sqlite: ${{ steps.plan.outputs.sqlite }}
runtime: ${{ steps.plan.outputs.runtime }}
package: ${{ steps.plan.outputs.package }}
@@ -122,11 +123,28 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
+ - name: Restore static task cache
+ if: needs.changes.outputs.code == 'true'
+ id: static-cache
+ uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+ with:
+ path: .turbo/cache/
+ key: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
+ # Turbo validates each task's inputs after restoring this snapshot.
+ restore-keys: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
+ - name: Record static task source and execution identity
+ continue-on-error: true
+ if: needs.changes.outputs.code == 'true'
+ env:
+ CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ run: npm run ci:static-context
- if: needs.changes.outputs.code == 'true'
run: npm run check:ci
- if: needs.changes.outputs.code == 'true'
@@ -136,6 +154,15 @@ jobs:
run: npm run test:python
- if: needs.changes.outputs.code != 'true'
run: npm run format:check
+ - name: Save validated static task cache
+ if: >-
+ success() && needs.changes.outputs.code == 'true' &&
+ steps.npm-cache.outputs.can-save == 'true' &&
+ steps.static-cache.outputs.cache-hit != 'true'
+ uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+ with:
+ path: .turbo/cache/
+ key: ${{ steps.static-cache.outputs.cache-primary-key }}
- name: Upload complete static diagnostics
if: always() && needs.changes.outputs.code == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -146,6 +173,34 @@ jobs:
if-no-files-found: ignore
retention-days: 7
+ test-source:
+ name: Source tests
+ needs: changes
+ if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast'
+ runs-on: ubuntu-latest
+ timeout-minutes: 10
+ steps:
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+ - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+ with:
+ node-version-file: .nvmrc
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
+ - run: npm ci
+ - name: Run source tests without waiting for build
+ run: npm run test:fast -- --reporter=default --reporter=blob
+ env:
+ VITEST_BLOB_LABEL: source
+ - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+ if: ${{ !cancelled() }}
+ with:
+ name: vitest-report-source
+ path: .vitest/blob/*
+ include-hidden-files: true
+ if-no-files-found: error
+ retention-days: 7
+
test-shard:
name: Test shard (${{ matrix.shard }})
needs: [changes, build]
@@ -153,15 +208,21 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
- fail-fast: false
+ # Failed PRs release the remaining matrix slots; qualification retains
+ # every shard's diagnostics and coverage. Passing PRs run the full suite.
+ fail-fast: ${{ github.event_name == 'pull_request' && needs.changes.outputs.tier == 'fast' }}
matrix:
- shard: [1/6, 2/6, 3/6, 4/6, 5/6, 6/6]
+ # Fast feedback spends the same six test jobs on one source group and
+ # five built groups; full qualification retains six complete shards.
+ shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/5","2/5","3/5","4/5","5/5"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -172,21 +233,42 @@ jobs:
run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
env:
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- - name: Run complete portable suite for fast feedback
+ - name: Run built-dependent tests for fast feedback
if: needs.changes.outputs.tier == 'fast'
- run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
+ run: npm run test:built -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
env:
+ VITEST_BLOB_LABEL: built
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ !cancelled() }}
with:
- name: vitest-report-${{ strategy.job-index }}
+ name: vitest-report-built-${{ strategy.job-index }}
path: .vitest/blob/*
include-hidden-files: true
if-no-files-found: error
overwrite: true
retention-days: 7
+ reverse-tools-quality:
+ name: Reverse tools coverage and mutation
+ needs: [changes, static]
+ if: needs.changes.outputs.reverse_tools == 'true'
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ steps:
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+ - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+ with:
+ node-version-file: .nvmrc
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
+ - run: npm ci
+ - name: Run reverse tools tests with coverage
+ run: npm run test:coverage:reverse-tools
+ - name: Report reverse tools mutation results
+ run: npm run test:mutation:reverse-tools
+
build:
name: Build package
needs: changes
@@ -212,7 +294,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- name: Reuse portable build outputs across runs
uses: ./.github/actions/portable-build-cache
@@ -271,7 +355,7 @@ jobs:
test:
name: Portable tests
- needs: [changes, test-shard]
+ needs: [changes, test-source, test-shard]
if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }}
runs-on: ubuntu-latest
timeout-minutes: 10
@@ -286,13 +370,18 @@ jobs:
if: needs.changes.outputs.code == 'true' && success()
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ if: needs.changes.outputs.code == 'true' && success()
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- if: needs.changes.outputs.code == 'true' && success()
run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
if: >-
!cancelled() && needs.changes.outputs.code == 'true' &&
- (needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
+ (needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
+ (needs.changes.outputs.tier == 'fast' &&
+ (needs.test-source.result == 'success' || needs.test-source.result == 'failure')))
with:
path: .vitest/blob
pattern: vitest-report-*
@@ -320,7 +409,8 @@ jobs:
if: >-
!cancelled() && needs.changes.outputs.code == 'true' &&
needs.changes.outputs.tier == 'fast' &&
- (needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
+ (needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
+ needs.test-source.result == 'success' || needs.test-source.result == 'failure')
run: >-
npm exec -- vitest --merge-reports
--reporter=default
@@ -333,6 +423,8 @@ jobs:
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-timings.json') != ''
env:
SHARD_RESULT: ${{ needs.test-shard.result }}
+ SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
+ SOURCE_RESULT: ${{ needs.test-source.result }}
MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }}
CI_VERIFY_TEST_INVENTORY: "true"
CI_TEST_SELECTION: ${{ needs.changes.outputs.test_selection }}
@@ -365,7 +457,13 @@ jobs:
env:
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
SHARD_RESULT: ${{ needs.test-shard.result }}
- run: test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
+ SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
+ SOURCE_RESULT: ${{ needs.test-source.result }}
+ run: |
+ if [ "${CODE_REQUIRED}" = "true" ]; then
+ test "${SHARD_RESULT}" = "success" || exit 1
+ test "${SOURCE_REQUIRED}" != "true" || test "${SOURCE_RESULT}" = "success"
+ fi
conformance-fixtures:
name: Source-built conformance fixtures
@@ -450,7 +548,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- name: Verify native runtime identity
env:
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
@@ -472,11 +572,6 @@ jobs:
- run: npm run verify:windows-native:artifact
- name: Verify NativeAOT diagnostic applicability on this host
run: npm run test:focused -- tests/evaluation/package/nativeAotDiagnostics.test.ts
- - name: Resolve npm cache before verifier HOME isolation
- id: npm-cache
- run: |
- cache_path="$(npm config get cache)"
- echo "path=${cache_path}" >> "${GITHUB_OUTPUT}"
- run: npm run verify:package
env:
npm_config_cache: ${{ steps.npm-cache.outputs.path }}
@@ -506,7 +601,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -537,6 +634,7 @@ jobs:
tests/boundary/cli/nativeSignatureCli.test.ts
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
src/process/capture/FilesystemSnapshot.test.ts
+ tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/process/darwinProcessRunTokenReader.test.ts
tests/boundary/process/directAnalysisSignals.test.ts
tests/boundary/process/processCaptureLifecycle.test.ts
@@ -554,7 +652,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- name: Run the public CLI/MCP reconstruction journey
env:
@@ -583,7 +683,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -613,6 +715,8 @@ jobs:
- name: Run curated Windows capability tests
run: >-
npx vitest run --coverage.enabled=false
+ src/process/capture/FilesystemSnapshot.test.ts
+ tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
tests/boundary/filesystem/safeOutputTree.test.ts
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
@@ -703,7 +807,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:check
- name: Generation leaves authored files unchanged
@@ -792,7 +898,9 @@ jobs:
changes,
static,
build,
+ reverse-tools-quality,
test-shard,
+ test-source,
sqlite-runtime,
test,
conformance-fixtures,
diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml
index 6b8a05b46b..baa6ba785e 100644
--- a/.github/workflows/pages.yml
+++ b/.github/workflows/pages.yml
@@ -23,6 +23,8 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:build
diff --git a/.github/workflows/real-binary-layout.yml b/.github/workflows/real-binary-layout.yml
index 0218543f05..06fcc290a6 100644
--- a/.github/workflows/real-binary-layout.yml
+++ b/.github/workflows/real-binary-layout.yml
@@ -32,7 +32,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/real-browser.yml b/.github/workflows/real-browser.yml
index 3c3fe955ab..aa7e6ebb22 100644
--- a/.github/workflows/real-browser.yml
+++ b/.github/workflows/real-browser.yml
@@ -25,7 +25,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/real-cutter.yml b/.github/workflows/real-cutter.yml
new file mode 100644
index 0000000000..91121f2073
--- /dev/null
+++ b/.github/workflows/real-cutter.yml
@@ -0,0 +1,76 @@
+name: Real upstream Cutter verification
+
+on:
+ workflow_dispatch:
+ repository_dispatch:
+ types: [real-cutter-upstream]
+
+permissions:
+ contents: read
+
+jobs:
+ windows-native-artifact:
+ name: Build Windows bridge security boundary
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+ - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+ with:
+ node-version-file: .nvmrc
+ - name: Install the cross compiler used by the Windows native lane
+ run: |
+ sudo apt-get update
+ sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
+ - run: npm run build:windows-native
+ - uses: actions/upload-artifact@v4
+ with:
+ name: cutter-windows-native
+ path: native/windows/build/
+ if-no-files-found: error
+
+ cutter-upstream:
+ name: Cutter upstream smoke (${{ matrix.platform }})
+ needs: windows-native-artifact
+ runs-on: ${{ matrix.runner }}
+ timeout-minutes: 20
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - platform: linux
+ runner: [self-hosted, cutter-upstream, linux]
+ - platform: macOS
+ runner: [self-hosted, cutter-upstream, macOS]
+ - platform: Windows
+ runner: [self-hosted, cutter-upstream, Windows]
+ steps:
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+ - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+ with:
+ node-version-file: .nvmrc
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
+ - run: npm ci
+ - if: matrix.platform == 'Windows'
+ uses: actions/download-artifact@v4
+ with:
+ name: cutter-windows-native
+ path: native/windows/build/
+ - if: matrix.platform == 'Windows'
+ run: npm run verify:windows-native:artifact
+ - if: matrix.platform == 'Windows'
+ run: npm run build:cached
+ - if: matrix.platform == 'Windows'
+ run: npm run verify:windows-native
+ - name: Verify host and real Cutter bridge
+ env:
+ EXPECTED_PLATFORM: ${{ matrix.platform }}
+ run: npm run verify:cutter
+ - uses: actions/upload-artifact@v4
+ if: always()
+ with:
+ name: cutter-upstream-${{ matrix.platform }}
+ path: cutter-smoke.json
+ if-no-files-found: ignore
+ retention-days: 30
diff --git a/.github/workflows/real-evm-interface.yml b/.github/workflows/real-evm-interface.yml
index 8588a269f7..3a47b8f907 100644
--- a/.github/workflows/real-evm-interface.yml
+++ b/.github/workflows/real-evm-interface.yml
@@ -26,7 +26,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/real-go-binary.yml b/.github/workflows/real-go-binary.yml
index c7d74b0014..d99622067d 100644
--- a/.github/workflows/real-go-binary.yml
+++ b/.github/workflows/real-go-binary.yml
@@ -48,7 +48,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
with:
go-version: "1.26.1"
diff --git a/.github/workflows/real-hopper-linux.yml b/.github/workflows/real-hopper-linux.yml
index d5e5f8c337..417af2ca5a 100644
--- a/.github/workflows/real-hopper-linux.yml
+++ b/.github/workflows/real-hopper-linux.yml
@@ -19,7 +19,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- name: Pack CLI for setup verification
shell: bash
@@ -40,7 +42,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- name: Install packed CLI in isolation
shell: bash
diff --git a/.github/workflows/real-hopper.yml b/.github/workflows/real-hopper.yml
index b3ceeb7d9c..da1c60c9f9 100644
--- a/.github/workflows/real-hopper.yml
+++ b/.github/workflows/real-hopper.yml
@@ -19,7 +19,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- name: Install packed CLI in isolation
shell: bash
diff --git a/.github/workflows/real-javascript-recovery.yml b/.github/workflows/real-javascript-recovery.yml
index a5043b62e3..786d4a2d4a 100644
--- a/.github/workflows/real-javascript-recovery.yml
+++ b/.github/workflows/real-javascript-recovery.yml
@@ -27,7 +27,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/real-packet-capture.yml b/.github/workflows/real-packet-capture.yml
index 8daebfd8aa..12cb3d23db 100644
--- a/.github/workflows/real-packet-capture.yml
+++ b/.github/workflows/real-packet-capture.yml
@@ -47,7 +47,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci
- run: npm run build:cached
diff --git a/.github/workflows/real-recorded-crash.yml b/.github/workflows/real-recorded-crash.yml
index 0463df27d6..58cf90aed3 100644
--- a/.github/workflows/real-recorded-crash.yml
+++ b/.github/workflows/real-recorded-crash.yml
@@ -29,7 +29,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/real-signal-recording.yml b/.github/workflows/real-signal-recording.yml
index 5c1380e2df..1ddc2d5e0b 100644
--- a/.github/workflows/real-signal-recording.yml
+++ b/.github/workflows/real-signal-recording.yml
@@ -40,7 +40,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci
- run: npm run build:cached
diff --git a/.github/workflows/real-web-network-captures.yml b/.github/workflows/real-web-network-captures.yml
index 3ee59d2c87..7901114f0b 100644
--- a/.github/workflows/real-web-network-captures.yml
+++ b/.github/workflows/real-web-network-captures.yml
@@ -26,7 +26,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- name: Use pinned package manager
run: npm install --global npm@11.16.0
- run: npm ci
diff --git a/.github/workflows/real-web-runtime.yml b/.github/workflows/real-web-runtime.yml
index f5b9f201a1..0ceb9b0573 100644
--- a/.github/workflows/real-web-runtime.yml
+++ b/.github/workflows/real-web-runtime.yml
@@ -21,12 +21,14 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
build_cache_artifact: ${{ inputs.build_cache_artifact }}
- - run: npm run verify:browser:runtime
+ - run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
diff --git a/.github/workflows/real-web-source-map.yml b/.github/workflows/real-web-source-map.yml
index 2a1e65dc0a..eb1ecde342 100644
--- a/.github/workflows/real-web-source-map.yml
+++ b/.github/workflows/real-web-source-map.yml
@@ -21,7 +21,9 @@ jobs:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: ./.github/actions/portable-build-cache
with:
diff --git a/.github/workflows/sqlite-runtime.yml b/.github/workflows/sqlite-runtime.yml
index fc379d1dc2..1ea6964b73 100644
--- a/.github/workflows/sqlite-runtime.yml
+++ b/.github/workflows/sqlite-runtime.yml
@@ -27,7 +27,9 @@ jobs:
with:
node-version: ${{ matrix.node }}
check-latest: true
- cache: npm
+ - name: Cache npm package downloads
+ id: npm-cache
+ uses: ./.github/actions/npm-cache
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
diff --git a/.gitignore b/.gitignore
index 0d006e13fd..e4dc408141 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,6 +33,8 @@ codedb.snapshot
htmlcov/
.coverage
coverage/
+.stryker-tmp/
+reports/mutation/
.vitest/
.vitest-reports/
test-results.xml
diff --git a/README.md b/README.md
index cc2235ee4b..7468dd181d 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,7 @@
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
+· [Ελληνικά](docs/readme/README_el.md)
# REA: Reverse Engineer Anything
@@ -23,6 +24,12 @@
npx rea-agents setup
+> [!IMPORTANT]
+> Get REA from [morluto/rea](https://github.com/morluto/rea),
+> [rea.tools](https://rea.tools/), or the [rea-agents npm package](https://www.npmjs.com/package/rea-agents).
+> Follow the installation and release links from these official sources. A similar
+> repository name or copied README does not establish that a download is official.
+
@@ -57,6 +64,11 @@ Setup registers REA with your agent and installs matching workflow instructions.
## Quick start
+On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
+analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
+checks. Recovery steps are in `docs/installation.md` under
+"Linux owned-process prerequisite".
+
### Set up your agent
With Node.js and npm installed, run:
@@ -152,7 +164,7 @@ Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
-| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
+| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
@@ -160,8 +172,8 @@ Additional tools and host support depend on the target:
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
-| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](docs/javascript-artifact-reconstruction.md) |
-| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
+| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
+| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
@@ -180,6 +192,11 @@ Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
using your user permissions; each runtime guide describes its effects.
+For offline native-binary inspection, REA also supports caller-supplied GNU
+objdump and Rizin. See the [reverse-engineering tools guide](docs/reverse-engineering-tools.md)
+for provider setup, CLI commands, and Evidence limitations. REA does not install
+or update either provider.
+
Native formats and host support vary by provider. See
@@ -323,7 +340,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
-🎉 **180,000 GitHub stars — thank you!**
+🎉 **200,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
diff --git a/bridge/cutter/rea_bridge.py b/bridge/cutter/rea_bridge.py
new file mode 100644
index 0000000000..473f1d5fe6
--- /dev/null
+++ b/bridge/cutter/rea_bridge.py
@@ -0,0 +1,816 @@
+"""REA's experimental local bridge plugin for Cutter.
+
+Install this file in Cutter's user plugin directory. The bridge is REA-owned
+IPC, not an upstream Cutter transport. It binds to loopback and requires a
+per-process random token stored in a user-only descriptor file.
+"""
+
+from __future__ import annotations
+
+import json
+import hashlib
+import hmac
+import os
+import secrets
+import socket
+import stat
+import tempfile
+import threading
+import time
+import uuid
+from pathlib import Path
+from typing import Any
+
+if os.name == "nt":
+ import ctypes
+ from ctypes import wintypes
+
+import cutter
+
+try:
+ from PySide6.QtCore import QObject, Signal
+except ImportError:
+ from PySide2.QtCore import QObject, Signal
+
+
+MAX_FRAME_BYTES = 16 * 1024 * 1024
+SOCKET_TIMEOUT_SECONDS = 20
+AUTH_FRAME_TIMEOUT_SECONDS = 1
+MAX_DESCRIPTOR_BYTES = 64 * 1024
+MAX_COMPLETION_RECEIPTS = 64
+
+
+def _windows_api() -> tuple[Any, Any]:
+ kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+ advapi32 = ctypes.WinDLL("advapi32", use_last_error=True)
+ kernel32.GetCurrentProcess.restype = wintypes.HANDLE
+ kernel32.LocalFree.argtypes = [wintypes.HLOCAL]
+ kernel32.LocalFree.restype = wintypes.HLOCAL
+ kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
+ kernel32.CloseHandle.restype = wintypes.BOOL
+ kernel32.GetFileAttributesW.argtypes = [wintypes.LPCWSTR]
+ kernel32.GetFileAttributesW.restype = wintypes.DWORD
+ kernel32.CreateDirectoryW.argtypes = [wintypes.LPCWSTR, wintypes.LPVOID]
+ kernel32.CreateDirectoryW.restype = wintypes.BOOL
+ kernel32.CreateFileW.argtypes = [wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD,
+ wintypes.LPVOID, wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE]
+ kernel32.CreateFileW.restype = wintypes.HANDLE
+ kernel32.WriteFile.argtypes = [wintypes.HANDLE, wintypes.LPVOID, wintypes.DWORD,
+ ctypes.POINTER(wintypes.DWORD), wintypes.LPVOID]
+ kernel32.WriteFile.restype = wintypes.BOOL
+ kernel32.FlushFileBuffers.argtypes = [wintypes.HANDLE]
+ kernel32.FlushFileBuffers.restype = wintypes.BOOL
+ kernel32.MoveFileExW.argtypes = [wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD]
+ kernel32.MoveFileExW.restype = wintypes.BOOL
+ kernel32.DeleteFileW.argtypes = [wintypes.LPCWSTR]
+ kernel32.DeleteFileW.restype = wintypes.BOOL
+ advapi32.OpenProcessToken.argtypes = [wintypes.HANDLE, wintypes.DWORD, ctypes.POINTER(wintypes.HANDLE)]
+ advapi32.OpenProcessToken.restype = wintypes.BOOL
+ advapi32.GetTokenInformation.argtypes = [wintypes.HANDLE, ctypes.c_int, wintypes.LPVOID,
+ wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
+ advapi32.GetTokenInformation.restype = wintypes.BOOL
+ advapi32.ConvertSidToStringSidW.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.LPWSTR)]
+ advapi32.ConvertSidToStringSidW.restype = wintypes.BOOL
+ advapi32.GetNamedSecurityInfoW.argtypes = [wintypes.LPWSTR, ctypes.c_int, wintypes.DWORD,
+ ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
+ ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
+ ctypes.POINTER(wintypes.LPVOID)]
+ advapi32.GetNamedSecurityInfoW.restype = wintypes.DWORD
+ advapi32.GetSecurityDescriptorControl.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.WORD),
+ ctypes.POINTER(wintypes.DWORD)]
+ advapi32.GetSecurityDescriptorControl.restype = wintypes.BOOL
+ advapi32.EqualSid.argtypes = [wintypes.LPVOID, wintypes.LPVOID]
+ advapi32.EqualSid.restype = wintypes.BOOL
+ advapi32.GetAclInformation.argtypes = [wintypes.LPVOID, wintypes.LPVOID, wintypes.DWORD, ctypes.c_int]
+ advapi32.GetAclInformation.restype = wintypes.BOOL
+ advapi32.GetAce.argtypes = [wintypes.LPVOID, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID)]
+ advapi32.GetAce.restype = wintypes.BOOL
+ advapi32.IsValidSid.argtypes = [wintypes.LPVOID]
+ advapi32.IsValidSid.restype = wintypes.BOOL
+ advapi32.GetLengthSid.argtypes = [wintypes.LPVOID]
+ advapi32.GetLengthSid.restype = wintypes.DWORD
+ advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.argtypes = [
+ wintypes.LPCWSTR, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.DWORD),
+ ]
+ advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.restype = wintypes.BOOL
+ return kernel32, advapi32
+
+
+def _windows_user_sid_text() -> str:
+ kernel32, advapi32 = _windows_api()
+ token = wintypes.HANDLE()
+ if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ required = wintypes.DWORD()
+ advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
+ if required.value == 0:
+ raise ctypes.WinError(ctypes.get_last_error())
+ buffer = ctypes.create_string_buffer(required.value)
+ if not advapi32.GetTokenInformation(token, 1, buffer, required, ctypes.byref(required)):
+ raise ctypes.WinError(ctypes.get_last_error())
+
+ class SID_AND_ATTRIBUTES(ctypes.Structure):
+ _fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
+
+ class TOKEN_USER(ctypes.Structure):
+ _fields_ = [("User", SID_AND_ATTRIBUTES)]
+
+ sid = ctypes.cast(buffer, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
+ text = wintypes.LPWSTR()
+ if not advapi32.ConvertSidToStringSidW(sid, ctypes.byref(text)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ return text.value
+ finally:
+ kernel32.LocalFree(text)
+ finally:
+ kernel32.CloseHandle(token)
+
+
+def _verify_windows_private_path(path: Path, directory: bool) -> None:
+ kernel32, advapi32 = _windows_api()
+ attributes = kernel32.GetFileAttributesW(str(path))
+ if attributes == 0xFFFFFFFF:
+ raise ctypes.WinError(ctypes.get_last_error())
+ if attributes & 0x400 or bool(attributes & 0x10) != directory:
+ raise RuntimeError("Cutter bridge path is a reparse point or has the wrong object type")
+
+ class ACL_SIZE_INFORMATION(ctypes.Structure):
+ _fields_ = [("AceCount", wintypes.DWORD), ("AclBytesInUse", wintypes.DWORD), ("AclBytesFree", wintypes.DWORD)]
+
+ class ACE_HEADER(ctypes.Structure):
+ _fields_ = [("AceType", wintypes.BYTE), ("AceFlags", wintypes.BYTE), ("AceSize", wintypes.WORD)]
+
+ class SID_HEADER(ctypes.Structure):
+ _fields_ = [("Revision", wintypes.BYTE), ("SubAuthorityCount", wintypes.BYTE),
+ ("IdentifierAuthority", wintypes.BYTE * 6)]
+
+ class ACCESS_ALLOWED_ACE(ctypes.Structure):
+ _fields_ = [("Header", ACE_HEADER), ("Mask", wintypes.DWORD), ("SidStart", wintypes.DWORD)]
+
+ owner = wintypes.LPVOID()
+ dacl = wintypes.LPVOID()
+ descriptor = wintypes.LPVOID()
+ result = advapi32.GetNamedSecurityInfoW(
+ str(path), 1, 0x1 | 0x4, ctypes.byref(owner), None, ctypes.byref(dacl), None, ctypes.byref(descriptor),
+ )
+ if result != 0:
+ raise ctypes.WinError(result)
+ try:
+ control = wintypes.WORD()
+ revision = wintypes.DWORD()
+ if not advapi32.GetSecurityDescriptorControl(descriptor, ctypes.byref(control), ctypes.byref(revision)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ if not control.value & 0x1000 or not dacl:
+ raise RuntimeError("Cutter bridge ACL must be present and protected from inheritance")
+
+ token = wintypes.HANDLE()
+ if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ required = wintypes.DWORD()
+ advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
+ token_user = ctypes.create_string_buffer(required.value)
+ if not advapi32.GetTokenInformation(token, 1, token_user, required, ctypes.byref(required)):
+ raise ctypes.WinError(ctypes.get_last_error())
+
+ class SID_AND_ATTRIBUTES(ctypes.Structure):
+ _fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
+
+ class TOKEN_USER(ctypes.Structure):
+ _fields_ = [("User", SID_AND_ATTRIBUTES)]
+
+ current_sid = ctypes.cast(token_user, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
+ if not advapi32.EqualSid(owner, current_sid):
+ raise RuntimeError("Cutter bridge path owner does not match the current user")
+ finally:
+ kernel32.CloseHandle(token)
+
+ info = ACL_SIZE_INFORMATION()
+ if not advapi32.GetAclInformation(dacl, ctypes.byref(info), ctypes.sizeof(info), 2) or info.AceCount != 2:
+ raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
+ expected_user = _windows_user_sid_text()
+ user_allowed = False
+ system_allowed = False
+ for index in range(info.AceCount):
+ ace_pointer = wintypes.LPVOID()
+ if not advapi32.GetAce(dacl, index, ctypes.byref(ace_pointer)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ header = ctypes.cast(ace_pointer, ctypes.POINTER(ACE_HEADER)).contents
+ if header.AceType != 0 or header.AceSize < ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD) + ctypes.sizeof(SID_HEADER):
+ raise RuntimeError("Cutter bridge ACL contains a malformed or unsupported ACE")
+ ace_address = ctypes.cast(ace_pointer, ctypes.c_void_p).value
+ if ace_address is None:
+ raise RuntimeError("Cutter bridge ACL returned a null ACE")
+ sid_pointer = ctypes.c_void_p(ace_address + ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD))
+ sid_header = ctypes.cast(sid_pointer, ctypes.POINTER(SID_HEADER)).contents
+ sid_size = ctypes.sizeof(SID_HEADER) + sid_header.SubAuthorityCount * ctypes.sizeof(wintypes.DWORD)
+ if sid_size > header.AceSize - ctypes.sizeof(ACE_HEADER) - ctypes.sizeof(wintypes.DWORD) or not advapi32.IsValidSid(sid_pointer) or advapi32.GetLengthSid(sid_pointer) != sid_size:
+ raise RuntimeError("Cutter bridge ACL contains an invalid SID")
+ ace = ctypes.cast(ace_pointer, ctypes.POINTER(ACCESS_ALLOWED_ACE)).contents
+ sid_text = wintypes.LPWSTR()
+ if ace.Mask != 0x001F01FF or ace.Header.AceFlags != 0x03:
+ raise RuntimeError("Cutter bridge ACL contains an unsupported ACE")
+ if not advapi32.ConvertSidToStringSidW(sid_pointer, ctypes.byref(sid_text)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ if sid_text.value == expected_user:
+ user_allowed = True
+ elif sid_text.value == "S-1-5-18":
+ system_allowed = True
+ else:
+ raise RuntimeError("Cutter bridge ACL allows an unrelated principal")
+ finally:
+ kernel32.LocalFree(sid_text)
+ if not user_allowed or not system_allowed:
+ raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
+ finally:
+ kernel32.LocalFree(descriptor)
+
+
+def _windows_security_attributes() -> tuple[Any, Any]:
+ kernel32, advapi32 = _windows_api()
+ sid = _windows_user_sid_text()
+ sddl = f"O:{sid}G:{sid}D:P(A;OICI;FA;;;{sid})(A;OICI;FA;;;SY)"
+ descriptor = wintypes.LPVOID()
+ if not advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW(sddl, 1, ctypes.byref(descriptor), None):
+ raise ctypes.WinError(ctypes.get_last_error())
+
+ class SECURITY_ATTRIBUTES(ctypes.Structure):
+ _fields_ = [("nLength", wintypes.DWORD), ("lpSecurityDescriptor", wintypes.LPVOID), ("bInheritHandle", wintypes.BOOL)]
+
+ attributes = SECURITY_ATTRIBUTES(ctypes.sizeof(SECURITY_ATTRIBUTES), descriptor, False)
+ return kernel32, (descriptor, attributes)
+
+
+def _ensure_windows_bridge_directory(directory: Path) -> None:
+ directory.parent.mkdir(parents=True, exist_ok=True)
+ if not directory.exists():
+ kernel32, (descriptor, attributes) = _windows_security_attributes()
+ try:
+ if not kernel32.CreateDirectoryW(str(directory), ctypes.byref(attributes)):
+ error = ctypes.get_last_error()
+ if error != 183:
+ raise ctypes.WinError(error)
+ finally:
+ kernel32.LocalFree(descriptor)
+ _verify_windows_private_path(directory, True)
+
+
+def _ensure_posix_bridge_directory(directory: Path) -> None:
+ directory.mkdir(mode=0o700, parents=True, exist_ok=True)
+ if not hasattr(os, "O_NOFOLLOW") or not hasattr(os, "O_DIRECTORY"):
+ raise RuntimeError("Cutter bridge directory requires no-follow directory handles")
+ descriptor = os.open(directory, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
+ try:
+ directory_info = os.fstat(descriptor)
+ if directory_info.st_uid != os.getuid():
+ raise RuntimeError("Cutter bridge directory must be owned by the current user")
+ os.fchmod(descriptor, 0o700)
+ directory_info = os.fstat(descriptor)
+ if stat.S_IMODE(directory_info.st_mode) != 0o700:
+ raise RuntimeError("Cutter bridge directory must have mode 0700")
+ finally:
+ os.close(descriptor)
+
+
+def _write_windows_descriptor(directory: Path, destination: Path, payload: bytes) -> None:
+ if len(payload) > MAX_DESCRIPTOR_BYTES:
+ raise RuntimeError("Cutter bridge descriptor exceeds its byte limit")
+ kernel32, (security_descriptor, attributes) = _windows_security_attributes()
+ temporary = directory / f".{destination.name}.{uuid.uuid4().hex}.tmp"
+ handle = wintypes.HANDLE()
+ try:
+ handle = kernel32.CreateFileW(str(temporary), 0x40000000 | 0x00020000 | 0x00040000,
+ 0x1 | 0x2 | 0x4, ctypes.byref(attributes), 1, 0x80, None)
+ if handle == wintypes.HANDLE(-1).value:
+ raise ctypes.WinError(ctypes.get_last_error())
+ _verify_windows_private_path(temporary, False)
+ if payload:
+ data = ctypes.create_string_buffer(payload)
+ written = wintypes.DWORD()
+ if not kernel32.WriteFile(handle, data, len(payload), ctypes.byref(written), None) or written.value != len(payload):
+ raise ctypes.WinError(ctypes.get_last_error())
+ if not kernel32.FlushFileBuffers(handle):
+ raise ctypes.WinError(ctypes.get_last_error())
+ kernel32.CloseHandle(handle)
+ handle = wintypes.HANDLE()
+ if not kernel32.MoveFileExW(str(temporary), str(destination), 0x1 | 0x8):
+ raise ctypes.WinError(ctypes.get_last_error())
+ finally:
+ if handle and handle != wintypes.HANDLE(-1).value:
+ kernel32.CloseHandle(handle)
+ kernel32.DeleteFileW(str(temporary))
+ kernel32.LocalFree(security_descriptor)
+
+
+def _bridge_directory() -> Path:
+ configured = os.environ.get("REA_CUTTER_BRIDGE_DIR")
+ if configured:
+ return Path(configured).expanduser()
+ local_app_data = os.environ.get("LOCALAPPDATA")
+ if os.name == "nt" and local_app_data:
+ return Path(local_app_data) / "REA" / "CutterBridge"
+ return Path.home() / ".cache" / "rea" / "cutter-bridge"
+
+
+class _RequestDispatcher(QObject):
+ dispatch = Signal(object, object)
+
+ def __init__(self) -> None:
+ super().__init__()
+ self.dispatch.connect(self._handle)
+ self.generation = 0
+ self.completed_commands: dict[str, str] = {}
+ self._receipt_lock = threading.Lock()
+ self.current_file = self._read_current_file()
+ try:
+ self.cutter_version = str(cutter.core().getVersionInformation())
+ except Exception:
+ self.cutter_version = None
+
+ @staticmethod
+ def _read_current_file() -> str | None:
+ try:
+ value = cutter.core().currentlyOpenFile()
+ return str(value) if value else None
+ except Exception:
+ return None
+
+ def _handle(self, request: dict[str, Any], reply: dict[str, Any]) -> None:
+ kind = request.get("kind")
+ request_id = request.get("request_id")
+ if kind in ("command", "ack"):
+ try:
+ valid_request_id = (
+ isinstance(request_id, str)
+ and str(uuid.UUID(request_id)) == request_id.lower()
+ )
+ except (ValueError, AttributeError):
+ valid_request_id = False
+ if not valid_request_id:
+ reply.update({
+ "ok": False,
+ "error": "invalid-command-request-id",
+ "execution_state": "not_started",
+ })
+ request["_event"].set()
+ return
+ if kind == "ack":
+ with self._receipt_lock:
+ acknowledged = self.completed_commands.pop(request_id, None) is not None
+ reply.update({"ok": True, "acknowledged": acknowledged})
+ request["_event"].set()
+ return
+ if request["_cancelled"].is_set():
+ reply.update({
+ "ok": False,
+ "error": "request-cancelled-before-execution",
+ "execution_state": "not_started",
+ })
+ if kind == "command":
+ self._record_command_completion(request_id, "not_started")
+ request["_event"].set()
+ return
+ self._refresh_document_identity()
+ if kind == "status":
+ with self._receipt_lock:
+ completed_commands = [
+ {"request_id": receipt_id, "execution_state": state}
+ for receipt_id, state in self.completed_commands.items()
+ ]
+ reply.update({
+ "ok": True,
+ "session_id": request.get("session_id"),
+ "current_file": self.current_file,
+ "document_generation": self.generation,
+ "cutter_version": self.cutter_version,
+ "identity_status": "partial",
+ "completed_commands": completed_commands,
+ })
+ request["_event"].set()
+ return
+ if kind != "command":
+ reply.update({"ok": False, "error": "unsupported-request-kind"})
+ request["_event"].set()
+ return
+ if request.get("expected_generation") != self.generation:
+ reply.update({
+ "ok": False,
+ "error": "active-document-generation-changed",
+ "execution_state": "not_started",
+ })
+ self._record_command_completion(request_id, "not_started")
+ request["_event"].set()
+ return
+ command = request.get("command")
+ if not isinstance(command, str) or not command.strip():
+ reply.update({
+ "ok": False,
+ "error": "invalid-command",
+ "execution_state": "not_started",
+ })
+ self._record_command_completion(request_id, "not_started")
+ request["_event"].set()
+ return
+ reply["execution_state"] = "running"
+ try:
+ output = cutter.cmdj(command) if request.get("json") is True else cutter.cmd(command)
+ command_error = None
+ except Exception as error:
+ output = None
+ command_error = str(error)
+ self._refresh_document_identity()
+ if command_error is None:
+ reply.update({
+ "ok": True,
+ "output": output,
+ "current_file": self.current_file,
+ "document_generation": self.generation,
+ "cutter_version": self.cutter_version,
+ "identity_status": "partial",
+ "execution_state": "complete",
+ })
+ self._record_command_completion(request_id, "complete")
+ else:
+ reply.update({
+ "ok": False,
+ "error": command_error,
+ "current_file": self.current_file,
+ "document_generation": self.generation,
+ "cutter_version": self.cutter_version,
+ "identity_status": "partial",
+ "execution_state": "unknown",
+ "message": "The command raised an exception; it may have produced partial effects. Do not retry automatically.",
+ })
+ self._record_command_completion(request_id, "failed")
+ request["_event"].set()
+
+ def _record_command_completion(
+ self, request_id: str, execution_state: str
+ ) -> None:
+ with self._receipt_lock:
+ if request_id in self.completed_commands or len(self.completed_commands) < MAX_COMPLETION_RECEIPTS:
+ self.completed_commands[request_id] = execution_state
+
+ def can_accept_command(self) -> bool:
+ with self._receipt_lock:
+ return len(self.completed_commands) < MAX_COMPLETION_RECEIPTS
+
+ def _refresh_document_identity(self) -> None:
+ current_file = self._read_current_file()
+ if current_file != self.current_file:
+ self.current_file = current_file
+ self.generation += 1
+
+
+class _BridgeServer(threading.Thread):
+ def __init__(self, dispatcher: _RequestDispatcher) -> None:
+ super().__init__(name="rea-cutter-bridge", daemon=True)
+ self.dispatcher = dispatcher
+ self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ self.server.bind(("127.0.0.1", 0))
+ self.server.listen(8)
+ self.server.settimeout(0.5)
+ self.port = self.server.getsockname()[1]
+ self.session_id = str(uuid.uuid4())
+ self.token = secrets.token_urlsafe(32)
+ self.poisoned = False
+ self.connection_ordinal = 0
+ self.directory = _bridge_directory()
+ self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
+
+ def publish(self) -> None:
+ if os.name == "nt":
+ _ensure_windows_bridge_directory(self.directory)
+ else:
+ _ensure_posix_bridge_directory(self.directory)
+ if os.name == "nt" and (self.directory.is_symlink() or not self.directory.is_dir()):
+ raise RuntimeError("Cutter bridge directory must be a real directory")
+ payload = {
+ "session_id": self.session_id,
+ "pid": os.getpid(),
+ "host": "127.0.0.1",
+ "port": self.port,
+ "token": self.token,
+ "document_generation": self.dispatcher.generation,
+ "current_file": self.dispatcher.current_file,
+ "cutter_version": self.dispatcher.cutter_version,
+ "identity_status": "partial",
+ }
+ if os.name == "nt":
+ _write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
+ return
+ descriptor, temporary_path = tempfile.mkstemp(prefix=f".{self.descriptor.name}.", dir=self.directory)
+ try:
+ os.fchmod(descriptor, 0o600)
+ descriptor_info = os.fstat(descriptor)
+ if descriptor_info.st_uid != os.getuid() or stat.S_IMODE(descriptor_info.st_mode) & 0o077:
+ raise RuntimeError("Cutter bridge descriptor must be owned by the current user with mode 0600")
+ with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
+ stream.write(json.dumps(payload))
+ stream.flush()
+ os.fsync(stream.fileno())
+ os.replace(temporary_path, self.descriptor)
+ except Exception:
+ try:
+ os.close(descriptor)
+ except OSError:
+ pass
+ try:
+ os.unlink(temporary_path)
+ except OSError:
+ pass
+ raise
+
+ def run(self) -> None:
+ try:
+ self.publish()
+ except Exception:
+ self.server.close()
+ return
+ while True:
+ try:
+ connection, _ = self.server.accept()
+ except socket.timeout:
+ if getattr(self, "stopping", False):
+ break
+ continue
+ except OSError:
+ break
+ self.connection_ordinal += 1
+ connection_ordinal = self.connection_ordinal
+ with connection:
+ connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
+ authenticated = False
+ try:
+ hello = self._receive_frame(connection)
+ if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
+ self._send(connection, {"ok": False, "error": "authentication-failed"})
+ continue
+ client_nonce = hello["nonce"]
+ server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
+ server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
+ self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
+ authenticated_frame = self._receive_frame(connection)
+ request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
+ expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
+ if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
+ self._send(connection, {"ok": False, "error": "authentication-failed"})
+ continue
+ request = json.loads(request_bytes)
+ if not isinstance(request, dict) or request.get("session_id") != self.session_id:
+ self._send(connection, {"ok": False, "error": "authentication-failed"})
+ continue
+ authenticated = True
+ connection.settimeout(SOCKET_TIMEOUT_SECONDS)
+ if self.poisoned and request.get("kind") not in ("status", "ack"):
+ self._send_authenticated(connection, {
+ "ok": False,
+ "error": "session-poisoned-after-ui-timeout",
+ "execution_state": "not_started",
+ }, client_nonce, server_nonce)
+ continue
+ if (
+ request.get("kind") == "command"
+ and not self.dispatcher.can_accept_command()
+ ):
+ self._send_authenticated(connection, {
+ "ok": False,
+ "error": "completion-receipt-capacity-exceeded",
+ "execution_state": "not_started",
+ }, client_nonce, server_nonce)
+ continue
+ request["session_id"] = self.session_id
+ event = threading.Event()
+ cancelled = threading.Event()
+ reply: dict[str, Any] = {}
+ request["_event"] = event
+ request["_cancelled"] = cancelled
+ self.dispatcher.dispatch.emit(request, reply)
+ if not event.wait(SOCKET_TIMEOUT_SECONDS):
+ cancelled.set()
+ self.poisoned = True
+ self._send_authenticated(connection, {
+ "ok": False,
+ "error": "ui-thread-request-timeout",
+ "execution_state": "unknown",
+ "message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
+ }, client_nonce, server_nonce)
+ else:
+ if request.get("kind") == "status" and reply.get("ok") is True:
+ reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
+ self._send_authenticated(connection, reply, client_nonce, server_nonce)
+ except socket.timeout:
+ if authenticated:
+ self._send_authenticated(connection, {
+ "ok": False,
+ "error": "request-timeout",
+ "execution_state": "unknown",
+ "message": "The command may have produced partial or persistent effects. Do not retry automatically.",
+ }, client_nonce, server_nonce)
+ except Exception as error:
+ self._send(connection, {"ok": False, "error": str(error)})
+ self.server.close()
+ try:
+ self.descriptor.unlink(missing_ok=True)
+ except OSError:
+ pass
+
+ @staticmethod
+ def _receive_frame(connection: socket.socket) -> dict[str, Any]:
+ data = bytearray()
+ deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
+ while b"\n" not in data:
+ if len(data) >= MAX_FRAME_BYTES:
+ raise ValueError("frame-limit-exceeded")
+ remaining = deadline - time.monotonic()
+ if remaining <= 0:
+ raise socket.timeout("authentication frame timed out")
+ connection.settimeout(remaining)
+ part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
+ if not part:
+ raise ValueError("incomplete authentication frame")
+ data.extend(part)
+ line, remainder = bytes(data).split(b"\n", 1)
+ if len(line) > MAX_FRAME_BYTES or remainder:
+ raise ValueError("invalid framed request")
+ value = json.loads(line)
+ return value if isinstance(value, dict) else {}
+
+ def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
+ encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
+ if encoded is None:
+ value = self._truncated_response(value)
+ encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
+ if encoded is None:
+ # This fixed-size fallback is checked through the same signed framing path.
+ value = {
+ "ok": False,
+ "error": "response-limit-exceeded",
+ "execution_state": value.get("execution_state", "unknown"),
+ "output": None,
+ "output_truncated": True,
+ "message": "The result exceeded the bridge frame limit; command completion state is preserved.",
+ }
+ encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
+ if encoded is None:
+ return
+ try:
+ connection.sendall(encoded)
+ except OSError:
+ pass
+
+ def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
+ if not _within_response_limit(value):
+ return None
+ prefix = b'{"response":"'
+ suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
+ frame_size = len(prefix) + len(suffix)
+ response_parts: list[bytes] = []
+ escaped_parts: list[bytes] = []
+ for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
+ raw = fragment.encode("utf-8")
+ escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
+ frame_size += len(escaped)
+ if frame_size > MAX_FRAME_BYTES:
+ return None
+ response_parts.append(raw)
+ escaped_parts.append(escaped)
+ proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
+ proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
+ for part in response_parts:
+ proof.update(part)
+ return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
+
+ @staticmethod
+ def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
+ response: dict[str, Any] = {
+ "ok": value.get("ok") is True,
+ "output": None,
+ "execution_state": value.get("execution_state", "unknown"),
+ "output_truncated": True,
+ "message": "The command result exceeded the bridge frame limit; completion state is preserved and output was omitted.",
+ }
+ for key in ("current_file", "document_generation", "cutter_version", "identity_status"):
+ if key in value:
+ response[key] = value[key]
+ if value.get("ok") is not True and isinstance(value.get("error"), str):
+ response["error"] = value["error"]
+ return response
+
+ @staticmethod
+ def _send(connection: socket.socket, value: dict[str, Any]) -> None:
+ if not _within_response_limit(value):
+ value = {
+ "ok": False,
+ "error": "response-limit-exceeded",
+ "execution_state": value.get("execution_state", "unknown"),
+ "output_truncated": True,
+ "message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
+ }
+ parts: list[bytes] = []
+ encoded_size = 0
+ for part in json.JSONEncoder(ensure_ascii=False).iterencode(value):
+ encoded_part = part.encode("utf-8")
+ encoded_size += len(encoded_part)
+ if encoded_size > MAX_FRAME_BYTES:
+ fallback = {
+ "ok": False,
+ "error": "response-limit-exceeded",
+ "execution_state": value.get("execution_state", "unknown"),
+ "output_truncated": True,
+ "message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
+ }
+ parts = [json.dumps(fallback, separators=(",", ":")).encode("utf-8")]
+ break
+ parts.append(encoded_part)
+ encoded = b"".join(parts)
+ try:
+ connection.sendall(encoded + b"\n")
+ except OSError:
+ # A disconnected caller must not terminate the bridge thread or
+ # prevent later commands from being served.
+ pass
+
+ def stop(self) -> None:
+ self.stopping = True
+ try:
+ self.server.close()
+ except OSError:
+ pass
+
+
+class ReaCutterPlugin(cutter.CutterPlugin):
+ name = "REA Local Bridge"
+ description = "Local REA analysis bridge for the active Cutter session"
+ version = "1.0"
+ author = "REA"
+
+ def __init__(self) -> None:
+ super().__init__()
+ self._dispatcher: _RequestDispatcher | None = None
+ self._bridge: _BridgeServer | None = None
+
+ def setupPlugin(self) -> None:
+ return
+
+ def setupInterface(self, main: Any) -> None:
+ self._dispatcher = _RequestDispatcher()
+ self._bridge = _BridgeServer(self._dispatcher)
+ self._bridge.start()
+
+ def terminate(self) -> None:
+ if self._bridge is not None:
+ self._bridge.stop()
+ self._bridge.join(timeout=2)
+ self._bridge = None
+ self._dispatcher = None
+
+
+def create_cutter_plugin() -> ReaCutterPlugin:
+ return ReaCutterPlugin()
+
+
+def _within_response_limit(value: Any) -> bool:
+ pending = [value]
+ seen: set[int] = set()
+ size = 0
+ while pending:
+ current = pending.pop()
+ if isinstance(current, str):
+ size += 2
+ if current.isascii():
+ size += len(current)
+ size += current.count('"') + current.count("\\")
+ size += sum(5 for character in current if ord(character) < 0x20)
+ else:
+ size += 4 * len(current)
+ size += current.count('"') + current.count("\\")
+ size += sum(5 for character in current if ord(character) < 0x20)
+ elif isinstance(current, dict):
+ identity = id(current)
+ if identity in seen:
+ return False
+ seen.add(identity)
+ size += 2 + 4 * len(current)
+ if size > MAX_FRAME_BYTES:
+ return False
+ pending.extend(current.keys())
+ pending.extend(current.values())
+ elif isinstance(current, (list, tuple)):
+ identity = id(current)
+ if identity in seen:
+ return False
+ seen.add(identity)
+ size += 2 + 2 * len(current)
+ if size > MAX_FRAME_BYTES:
+ return False
+ pending.extend(current)
+ else:
+ size += 16
+ if size > MAX_FRAME_BYTES:
+ return False
+ return True
diff --git a/bridge/ghidra/ReaGhidraBridge.java b/bridge/ghidra/ReaGhidraBridge.java
index 84e809ccb6..df639a6732 100644
--- a/bridge/ghidra/ReaGhidraBridge.java
+++ b/bridge/ghidra/ReaGhidraBridge.java
@@ -5,9 +5,11 @@ import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
+import java.io.InputStream;
import java.math.BigInteger;
import java.net.InetAddress;
import java.net.InetSocketAddress;
+import java.net.SocketTimeoutException;
import java.net.StandardProtocolFamily;
import java.net.UnixDomainSocketAddress;
import java.nio.channels.Channels;
@@ -81,6 +83,7 @@ import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.FunctionIterator;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.listing.InstructionIterator;
+import ghidra.program.model.listing.Listing;
import ghidra.program.model.listing.Variable;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
@@ -297,7 +300,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
Files.move(pendingPath, endpointPath);
}
}
- acceptClient(server, descriptor);
+ acceptAuthenticatedLoopbackClient(server, descriptor);
}
finally {
Files.deleteIfExists(pendingPath);
@@ -309,6 +312,50 @@ public final class ReaGhidraBridge extends HeadlessScript {
return System.getProperty("os.name", "").startsWith("Windows");
}
+ private static String readLoopbackHandshake(
+ SocketChannel client, int maxBytes, long timeoutMillis) throws IOException {
+ long deadline = System.nanoTime() + timeoutMillis * 1_000_000L;
+ InputStream input = client.socket().getInputStream();
+ ByteArrayOutputStream bytes = new ByteArrayOutputStream();
+ while (true) {
+ long remaining = deadline - System.nanoTime();
+ if (remaining <= 0) throw new SocketTimeoutException("Bridge authentication timed out");
+ client.socket().setSoTimeout((int) Math.max(1,
+ Math.min(Integer.MAX_VALUE, (remaining + 999_999L) / 1_000_000L)));
+ int value = input.read();
+ if (value == -1) return null;
+ if (value == '\n') return bytes.toString(StandardCharsets.UTF_8);
+ if (bytes.size() == maxBytes) throw new IOException("Bridge authentication frame is too large");
+ bytes.write(value);
+ }
+ }
+
+ private void acceptAuthenticatedLoopbackClient(
+ ServerSocketChannel server, SessionDescriptor descriptor) throws Exception {
+ while (true) {
+ try (SocketChannel client = server.accept()) {
+ Request first;
+ try {
+ String line = readLoopbackHandshake(client, 64 * 1024, 1_000);
+ if (line == null) continue;
+ first = parseRequest(line, descriptor.token);
+ }
+ catch (IOException | RuntimeException rejected) {
+ // A rejected unauthenticated peer cannot consume the bridge session.
+ continue;
+ }
+ client.socket().setSoTimeout(0);
+ try (BufferedReader reader = new BufferedReader(
+ Channels.newReader(client, StandardCharsets.UTF_8));
+ BufferedWriter writer = new BufferedWriter(
+ Channels.newWriter(client, StandardCharsets.UTF_8))) {
+ serveClient(descriptor, reader, writer, first);
+ }
+ return;
+ }
+ }
+ }
+
private void acceptClient(
ServerSocketChannel server,
SessionDescriptor descriptor) throws Exception {
@@ -319,26 +366,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
BufferedWriter writer = new BufferedWriter(
Channels.newWriter(client, StandardCharsets.UTF_8)
)) {
- serveClient(descriptor, reader, writer);
+ serveClient(descriptor, reader, writer, null);
}
}
private void serveClient(
SessionDescriptor descriptor,
BufferedReader reader,
- BufferedWriter writer) throws Exception {
+ BufferedWriter writer,
+ Request first) throws Exception {
while (true) {
- String line = reader.readLine();
- if (line == null) {
- return;
- }
- Request request;
- try {
- request = parseRequest(line, descriptor.token);
- }
- catch (RuntimeException exception) {
- writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
- return;
+ Request request = first;
+ first = null;
+ if (request == null) {
+ String line = reader.readLine();
+ if (line == null) return;
+ try {
+ request = parseRequest(line, descriptor.token);
+ }
+ catch (RuntimeException exception) {
+ writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
+ return;
+ }
}
try {
if (request.method.equals("shutdown")) {
@@ -405,7 +454,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
case "xrefs" -> xrefs(request.params);
case "analyze_function" -> {
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
- yield functionObservation(observation.dossier(), observation.limitations());
+ JsonObject result = functionObservation(observation.dossier(), observation.limitations());
+ result.add("relationship_coverage", observation.relationshipCoverage());
+ yield result;
}
case "set_address_name", "set_addresses_names" -> {
if (!descriptor.transport.equals("unix-socket"))
@@ -884,6 +935,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject result = functionObservation(
value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value),
noReturnCallLimitations(
+ function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
);
@@ -975,7 +1027,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
return functionObservation(
GSON.toJsonTree(renderAssembly(scan.instructions)),
- noReturnCallLimitations(scan.instructions)
+ noReturnCallLimitations(function, scan.instructions)
);
}
@@ -994,7 +1046,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add(
"The fast path does not invoke the decompiler or scan whole-program names and strings."
);
- List boundaryLimitations = noReturnCallLimitations(scan.instructions);
+ List boundaryLimitations = noReturnCallLimitations(function, scan.instructions);
for (String limitation : boundaryLimitations) limitations.add(limitation);
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
@@ -1016,12 +1068,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Function item : ordered) {
result.add(canonicalAddress(item.getEntryPoint()));
}
- return functionObservation(
- result,
- noReturnCallLimitations(
+ if (callers) {
+ JsonObject observation = functionObservation(result, noReturnCallLimitations(function,
currentProgram.getListing().getInstructions(function.getBody(), true)
- )
+ ));
+ observation.add("relationship_coverage", callRelationshipCoverage(null,
+ unrepresentedIncomingCalls(function, observed)));
+ monitor.checkCancelled();
+ return observation;
+ }
+ CallBoundaryObservation calls = callBoundaryObservation(
+ function, currentProgram.getListing().getInstructions(function.getBody(), true), observed, false, null
);
+ return callRelationshipObservation(result, calls);
}
private JsonObject procedureInfo(JsonObject params) throws Exception {
@@ -1041,6 +1100,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
return functionObservation(
result,
noReturnCallLimitations(
+ function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
);
@@ -1071,18 +1131,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.addProperty("direction", direction);
result.add("references", edges);
result.addProperty("reference_kinds_available", true);
- JsonArray unresolvedCalls = new JsonArray();
- if (direction.equals("outgoing")) for (Instruction instruction : scanInstructions(function, Integer.MAX_VALUE).instructions) {
- if (!instruction.getFlowType().isCall()) continue;
- boolean resolved = false;
- for (Reference reference : instruction.getReferencesFrom()) if (reference.getReferenceType().isCall()) resolved = true;
- if (!resolved) {
- JsonObject site = new JsonObject();
- site.addProperty("address", canonicalAddress(instruction.getAddress()));
- site.addProperty("reason", "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
- unresolvedCalls.add(site);
- }
- }
+ JsonArray unresolvedCalls = direction.equals("outgoing")
+ ? callBoundaryObservation(
+ function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
+ ).unresolvedCalls() : new JsonArray();
result.add("unresolved_calls", unresolvedCalls);
return result;
}
@@ -1490,19 +1542,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
DecompileResults decompilation = decompile(function);
String pseudocode = decompiledText(decompilation);
+ JsonObject availability = new JsonObject();
+ availability.add("pseudocode", facetAvailability(
+ pseudocode != null,
+ function.isExternal() ? "Ghidra does not decompile external functions."
+ : function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
+ : "Ghidra completed decompilation without returning pseudocode."
+ ));
if (pseudocode == null) {
pseudocode = "";
}
- List incomingReferences = collectReferences(function, "incoming")
- .stream()
+ List bodyIncoming = collectReferences(function, "incoming");
+ List incomingReferences = bodyIncoming.stream()
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
.toList();
List outgoingReferences = collectReferences(function, "outgoing");
JsonArray incoming = referenceEdges(incomingReferences);
JsonArray outgoing = referenceEdges(outgoingReferences);
- JsonArray comments = comments(scan.instructions);
- JsonArray callers = procedureIdentities(function.getCallingFunctions(monitor));
- JsonArray callees = procedureIdentities(function.getCalledFunctions(monitor));
+ JsonArray comments = comments(function);
+ Set callerFunctions = function.getCallingFunctions(monitor);
+ JsonArray callers = procedureIdentities(callerFunctions);
+ Set calleeFunctions = function.getCalledFunctions(monitor);
+ JsonArray callees = procedureIdentities(calleeFunctions);
JsonArray referencedStrings = referencedStrings(outgoingReferences);
JsonArray referencedNames = referencedNames(outgoingReferences);
JsonArray blocks = basicBlockValues(function);
@@ -1536,6 +1597,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
nativeApiBoundary(function, decompilation)
);
result.add("native_value_flow", nativeValueFlow(decompilation));
+ // Successful full queries distinguish collected empty sets from missing facts.
+ // Availability does not claim that Ghidra resolved every runtime relationship.
+ for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
+ availability.add(facet, facetAvailability(true, null));
+ }
+ availability.add("assembly", facetAvailability(
+ !scan.truncated(), "The function instruction scan was truncated."
+ ));
+ result.add("facet_availability", availability);
JsonArray limitations = new JsonArray();
limitations.add(
"Unresolved computed or indirect flows without target addresses are not represented as reference edges."
@@ -1551,7 +1621,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
);
if (swift && aarch64) {
limitations.add(
- "Swift String literals are decoded from constant register pairs at calls, returns and paired stores; literals stored one word at a time, chosen by conditional selects, passed through unresolved jumps, or built at runtime are not reported."
+ "Swift String literals are decoded from constant register pairs at calls, returns and paired stores. Literals stored one word at a time, chosen by conditional selects, or built at runtime are not reported; candidate pairs at unresolved jumps are retained as unknown omissions."
);
if (swiftLiterals.omissions().size() > 0) {
limitations.add(
@@ -1563,8 +1633,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
);
}
- List boundaryLimitations = noReturnCallLimitations(scan.instructions);
- for (String limitation : boundaryLimitations) {
+ CallBoundaryObservation calls = callBoundaryObservation(
+ function, scan.instructions, calleeFunctions, true, outgoingReferences);
+ result.add("unresolved_calls", calls.unresolvedCalls());
+ for (String limitation : calls.limitations()) {
limitations.add(limitation);
}
if (decompilation != null && decompilation.getErrorMessage() != null &&
@@ -1572,7 +1644,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
}
result.add("limitations", limitations);
- return new FunctionAnalysisObservation(result, boundaryLimitations);
+ monitor.checkCancelled();
+ int missingCallers = 0;
+ for (Reference reference : bodyIncoming) {
+ monitor.checkCancelled();
+ if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
+ }
+ return new FunctionAnalysisObservation(result, calls.limitations(),
+ callRelationshipCoverage(calls, missingCallers));
}
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
@@ -3276,54 +3355,276 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
- private List noReturnCallLimitations(Iterable instructions)
+ private static JsonObject facetAvailability(boolean available, String reason) {
+ JsonObject result = new JsonObject();
+ result.addProperty("available", available);
+ if (!available) result.addProperty("reason", reason);
+ return result;
+ }
+
+ private JsonObject callRelationshipObservation(JsonElement value, CallBoundaryObservation calls)
+ throws Exception {
+ monitor.checkCancelled();
+ JsonObject result = functionObservation(value, calls.limitations());
+ result.add("relationship_coverage", callRelationshipCoverage(calls, null));
+ return result;
+ }
+
+ private static JsonObject callRelationshipCoverage(
+ CallBoundaryObservation calls, Integer missingCallers) {
+ JsonObject coverage = completeResolvedCallCoverage();
+ Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
+ JsonArray directions = new JsonArray();
+ if (missingCallers != null && missingCallers > 0) directions.add("callers");
+ String callerLimitation = missingCallers != null && missingCallers > 0
+ ? "Ghidra's caller inventory omits sources of " + missingCallers +
+ " call reference(s) into this function's body." : null;
+ if ((calls != null && omitted == null) || (calls == null && missingCallers == null)) {
+ coverage.addProperty("status", "unknown");
+ String reason = calls == null ? "No call inventory was examined."
+ : "The callee inventory was not examined alongside decoded call targets.";
+ coverage.addProperty("reason", reason + (callerLimitation == null ? "" : " " + callerLimitation));
+ if (calls == null && missingCallers == null) directions.add("callers");
+ directions.add("callees");
+ coverage.add("limited_directions", directions);
+ return coverage;
+ }
+ JsonArray limitations = new JsonArray();
+ if (callerLimitation != null) limitations.add(callerLimitation);
+ if (omitted != null && omitted > 0) {
+ limitations.add("Ghidra's callee inventory omits targets of " + omitted +
+ " call reference(s) or decoded call(s) without destinations.");
+ directions.add("callees");
+ }
+ if (!limitations.isEmpty()) {
+ coverage.addProperty("status", "partial");
+ coverage.add("limitations", limitations);
+ coverage.add("limited_directions", directions);
+ }
+ return coverage;
+ }
+
+ private static JsonObject completeResolvedCallCoverage() {
+ JsonObject coverage = new JsonObject();
+ coverage.addProperty("status", "complete");
+ JsonArray diagnostics = new JsonArray();
+ diagnostics.add(
+ "Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals."
+ );
+ coverage.add("diagnostics", diagnostics);
+ return coverage;
+ }
+
+ private boolean unrepresentedCaller(Reference reference, Set callers) {
+ if (!reference.getReferenceType().isCall()) return false;
+ Function caller = containingFunction(reference.getFromAddress());
+ return caller == null || !callers.contains(caller);
+ }
+
+ private int unrepresentedIncomingCalls(Function function, Set callers)
+ throws Exception {
+ int[] omitted = { 0 };
+ visitReferences(function, "incoming", reference -> {
+ if (unrepresentedCaller(reference, callers)) omitted[0]++;
+ });
+ return omitted[0];
+ }
+
+ private List noReturnCallLimitations(
+ Function function, Iterable instructions)
throws Exception {
List limitations = new ArrayList<>();
+ Instruction beforePrevious = null;
+ Instruction previous = null;
for (Instruction instruction : instructions) {
monitor.checkCancelled();
- if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
- continue;
+ String limitation = terminalCallLimitation(instruction);
+ if (limitation != null) limitations.add(limitation);
+ if (beforePrevious != null && previous != null) {
+ String seh = borlandSehHandlerLimitation(
+ function, beforePrevious, previous, instruction);
+ if (seh != null) limitations.add(seh);
}
- Address[] destinations = instruction.getFlows();
- StringBuilder limitation = new StringBuilder();
- limitation.append("Ghidra Listing reports a terminal call at ")
- .append(canonicalAddress(instruction.getAddress()));
- if (destinations.length == 0) {
- limitation.append(" with no resolved callee; its no-return basis is unknown.");
- }
- else {
- limitation.append(" to ");
- for (int index = 0; index < destinations.length; index++) {
- if (index > 0) limitation.append(", ");
- Address destination = destinations[index];
- limitation.append(canonicalAddress(destination));
- Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
- if (target == null) {
- limitation.append(" (callee identity unknown)");
- }
- else {
- limitation.append(" ").append(procedureName(target));
- limitation.append(" (external=").append(target.isExternal());
- limitation.append(", thunk=").append(target.isThunk());
- limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
- }
- }
- limitation.append("; fallthrough is excluded by Ghidra's flow model.");
- Function target = destinations.length == 1
- ? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
- : null;
- if (target != null && target.hasNoReturn()) {
- limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
- }
- else {
- limitation.append(" The reason for the terminal flow is not established by this observation.");
- }
- }
- limitations.add(limitation.toString());
+ beforePrevious = previous;
+ previous = instruction;
}
return limitations;
}
+ private CallBoundaryObservation callBoundaryObservation(
+ Function function, Iterable instructions, Set returnedCallees,
+ boolean includeUnresolvedSites, List observedReferences)
+ throws Exception {
+ List limitations = new ArrayList<>();
+ CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
+ Instruction beforePrevious = null;
+ Instruction previous = null;
+ if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
+ else for (Reference reference : observedReferences) {
+ monitor.checkCancelled();
+ calls.observe(reference);
+ }
+ for (Instruction instruction : instructions) {
+ monitor.checkCancelled();
+ String limitation = terminalCallLimitation(instruction);
+ if (limitation != null) limitations.add(limitation);
+ if (beforePrevious != null && previous != null) {
+ String seh = borlandSehHandlerLimitation(
+ function, beforePrevious, previous, instruction);
+ if (seh != null) limitations.add(seh);
+ }
+ beforePrevious = previous;
+ previous = instruction;
+ if (!instruction.getFlowType().isCall()) continue;
+ boolean destination = false;
+ for (Reference reference : instruction.getReferencesFrom()) {
+ if (!reference.getReferenceType().isCall()) continue;
+ destination = true;
+ break;
+ }
+ if (!destination) {
+ calls.unrepresented++;
+ calls.unresolved(instruction.getAddress(),
+ "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
+ }
+ }
+ return new CallBoundaryObservation(List.copyOf(limitations),
+ returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
+ }
+
+ private String borlandSehHandlerLimitation(
+ Function function, Instruction handlerPush, Instruction framePush,
+ Instruction frameLink) {
+ if (!immediatelyPrecedes(handlerPush, framePush) ||
+ !immediatelyPrecedes(framePush, frameLink) ||
+ !handlerPush.getMnemonicString().equalsIgnoreCase("PUSH") ||
+ handlerPush.getNumOperands() != 1 ||
+ !framePush.getMnemonicString().equalsIgnoreCase("PUSH") ||
+ framePush.getNumOperands() != 1 ||
+ !frameLink.getMnemonicString().equalsIgnoreCase("MOV") ||
+ frameLink.getNumOperands() != 2 ||
+ !sameFsMemoryOperand(framePush, frameLink) ||
+ !hasRegisterOperand(frameLink, 1, "ESP"))
+ return null;
+ Object[] handlerOperand = handlerPush.getOpObjects(0);
+ if (handlerOperand.length != 1 || !(handlerOperand[0] instanceof Scalar scalar) ||
+ scalar.bitLength() != 32)
+ return null;
+ Address handler = currentProgram.getAddressFactory().getDefaultAddressSpace()
+ .getAddress(scalar.getUnsignedValue());
+ if (function.getBody().contains(handler)) return null;
+ return "Ghidra Listing shows a Borland/Delphi-style SEH frame registration at " +
+ canonicalAddress(frameLink.getAddress()) + " (push " + canonicalAddress(handler) +
+ " followed by fs-segment stack chaining); handler " + canonicalAddress(handler) +
+ " lies outside this procedure's body, so callees, strings, and cross-references may omit the handler region.";
+ }
+
+ private static boolean immediatelyPrecedes(Instruction first, Instruction second) {
+ Address fallThrough = first.getFallThrough();
+ return fallThrough != null && fallThrough.equals(second.getAddress());
+ }
+
+ private static boolean sameFsMemoryOperand(Instruction first, Instruction second) {
+ String firstOperand = normalizedFsMemoryOperand(first);
+ String secondOperand = normalizedFsMemoryOperand(second);
+ return firstOperand != null && firstOperand.equalsIgnoreCase(secondOperand);
+ }
+
+ private static String normalizedFsMemoryOperand(Instruction instruction) {
+ String representation = instruction.getDefaultOperandRepresentation(0);
+ representation = representation.replaceAll("(?i)\\b(?:byte|word|dword|qword)\\s+ptr\\s+", "")
+ .replaceAll("\\s+", "");
+ return representation.regionMatches(true, 0, "fs:", 0, 3) &&
+ representation.indexOf('[') >= 0 ? representation : null;
+ }
+
+ private static boolean hasRegisterOperand(
+ Instruction instruction, int operand, String name) {
+ for (Object object : instruction.getOpObjects(operand)) {
+ if (object instanceof Register register && register.getName().equalsIgnoreCase(name))
+ return true;
+ }
+ return false;
+ }
+
+ private final class CallBoundaryReferences {
+ private final Set returnedCallees;
+ private final TreeMap unresolved;
+ private int unrepresented;
+
+ CallBoundaryReferences(Set returnedCallees, boolean includeUnresolvedSites) {
+ this.returnedCallees = returnedCallees;
+ unresolved = includeUnresolvedSites ? new TreeMap<>() : null;
+ }
+
+ void observe(Reference reference) {
+ if (!reference.getReferenceType().isCall()) return;
+ // Reference identity follows the same containing-function owner as referenceEdge.
+ Function target = containingFunction(reference.getToAddress());
+ if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
+ unrepresented++;
+ if (target == null) unresolved(reference.getFromAddress(),
+ "A call destination has no known containing Ghidra procedure");
+ }
+
+ void unresolved(Address address, String reason) {
+ if (unresolved != null) unresolved.put(address, reason);
+ }
+
+ JsonArray unresolvedCalls() {
+ if (unresolved == null) return null;
+ JsonArray result = new JsonArray();
+ for (var item : unresolved.entrySet()) {
+ JsonObject site = new JsonObject();
+ site.addProperty("address", canonicalAddress(item.getKey()));
+ site.addProperty("reason", item.getValue());
+ result.add(site);
+ }
+ return result;
+ }
+ }
+
+ private String terminalCallLimitation(Instruction instruction) {
+ if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal())
+ return null;
+ Address[] destinations = instruction.getFlows();
+ StringBuilder limitation = new StringBuilder();
+ limitation.append("Ghidra Listing reports a terminal call at ")
+ .append(canonicalAddress(instruction.getAddress()));
+ if (destinations.length == 0) {
+ limitation.append(" with no resolved callee; its no-return basis is unknown.");
+ }
+ else {
+ limitation.append(" to ");
+ for (int index = 0; index < destinations.length; index++) {
+ if (index > 0) limitation.append(", ");
+ Address destination = destinations[index];
+ limitation.append(canonicalAddress(destination));
+ Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
+ if (target == null) {
+ limitation.append(" (callee identity unknown)");
+ }
+ else {
+ limitation.append(" ").append(procedureName(target));
+ limitation.append(" (external=").append(target.isExternal());
+ limitation.append(", thunk=").append(target.isThunk());
+ limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
+ }
+ }
+ limitation.append("; fallthrough is excluded by Ghidra's flow model.");
+ Function target = destinations.length == 1
+ ? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
+ : null;
+ if (target != null && target.hasNoReturn()) {
+ limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
+ }
+ else {
+ limitation.append(" The reason for the terminal flow is not established by this observation.");
+ }
+ }
+ return limitation.toString();
+ }
+
private static String renderAssembly(List instructions) {
return String.join("\n", renderAssemblyLines(instructions));
}
@@ -3347,15 +3648,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List collectReferences(
Function function,
String direction) throws Exception {
+ TreeMap observed = new TreeMap<>();
+ visitReferences(function, direction, reference -> addReference(observed, reference));
+ List result = new ArrayList<>(observed.values());
+ result.sort(REFERENCE_ORDER);
+ return result;
+ }
+
+ @FunctionalInterface
+ private interface ReferenceVisitor {
+ void accept(Reference reference) throws Exception;
+ }
+
+ private void visitReferences(Function function, String direction, ReferenceVisitor visitor)
+ throws Exception {
// References can target instruction interiors or originate in embedded
// data. Use actual body ownership, including an external's exact entry.
- AddressSet scope = new AddressSet(function.getBody());
- scope.add(function.getEntryPoint());
+ AddressSet scope = functionObservationScope(function);
boolean outgoing = direction.equals("outgoing");
AddressIterator addresses = outgoing
? currentProgram.getReferenceManager().getReferenceSourceIterator(scope, true)
: currentProgram.getReferenceManager().getReferenceDestinationIterator(scope, true);
- TreeMap observed = new TreeMap<>();
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
@@ -3363,7 +3676,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Reference reference : currentProgram.getReferenceManager()
.getReferencesFrom(address)) {
monitor.checkCancelled();
- addReference(observed, reference);
+ if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
else {
@@ -3371,19 +3684,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
.getReferencesTo(address);
while (iterator.hasNext()) {
monitor.checkCancelled();
- addReference(observed, iterator.next());
+ Reference reference = iterator.next();
+ if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
}
- List result = new ArrayList<>(observed.values());
- result.sort(REFERENCE_ORDER);
- return result;
}
private static void addReference(TreeMap observed, Reference reference) {
- if (reference.isEntryPointReference()) {
- return;
- }
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
canonicalAddress(reference.getToAddress()) + "\u0000" +
reference.getReferenceType().getName() + "\u0000" +
@@ -3392,6 +3700,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
observed.putIfAbsent(key, reference);
}
+ private static AddressSet functionObservationScope(Function function) {
+ AddressSet scope = new AddressSet(function.getBody());
+ scope.add(function.getEntryPoint());
+ return scope;
+ }
+
private JsonArray referenceEdges(List references) {
JsonArray result = new JsonArray();
for (Reference reference : references) {
@@ -3545,23 +3859,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
- private static JsonArray comments(List instructions) {
+ private JsonArray comments(Function function) throws Exception {
+ Listing listing = currentProgram.getListing();
+ AddressSet scope = functionObservationScope(function);
+ AddressIterator addresses = listing.getCommentAddressIterator(scope, true);
TreeMap observed = new TreeMap<>();
- for (Instruction instruction : instructions) {
+ while (addresses.hasNext()) {
+ monitor.checkCancelled();
+ Address address = addresses.next();
for (CommentType type : CommentType.values()) {
- String text = instruction.getComment(type);
+ String text = listing.getComment(type, address);
if (text == null || text.isEmpty()) {
continue;
}
String kind = type == CommentType.EOL ? "inline" : "comment";
JsonObject item = new JsonObject();
- item.addProperty("address", canonicalAddress(instruction.getAddress()));
+ item.addProperty("address", canonicalAddress(address));
item.addProperty("kind", kind);
item.addProperty("text", text);
- String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
+ String key = canonicalAddress(address) + "\u0000" +
kind + "\u0000" + type.name() + "\u0000" + text;
observed.put(key, item);
- }
+ }
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
@@ -3632,13 +3951,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
* The canonical empty string is a small literal with a zero count.
*/
private SwiftStringLiteralDecode swiftStringLiterals(Function function) throws Exception {
+ return swiftStringLiterals(function, new SwiftLiteralBudget());
+ }
+
+ // An explicit budget makes capacity omissions deterministic in producer tests.
+ private SwiftStringLiteralDecode swiftStringLiterals(
+ Function function,
+ SwiftLiteralBudget budget
+ ) throws Exception {
Register[] registers = new Register[31];
for (int index = 0; index < registers.length; index++) {
registers[index] = currentProgram.getRegister("x" + index);
}
TreeMap observed = new TreeMap<>();
TreeMap omitted = new TreeMap<>();
- SwiftLiteralBudget budget = new SwiftLiteralBudget();
Map selectedValues = new HashMap<>();
SymbolicPropogator propagator = new SymbolicPropogator(currentProgram, false);
propagator.flowConstants(function.getEntryPoint(), function.getBody(), new ContextEvaluatorAdapter() {
@@ -3646,6 +3972,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
public boolean evaluateContextBefore(VarnodeContext context, Instruction instruction) {
// Returning true ends only the current flow, so every queued flow checks the shared stop.
if (budget.stopped()) return true;
+ FlowType flow = instruction.getFlowType();
+ Address[] flowTargets = instruction.getFlows();
+ boolean unresolvedJump = flow.isJump() && !flow.isConditional() && flowTargets.length == 0;
for (Register[] pair : swiftStringPairs(function, instruction, registers)) {
Long word = registerConstant(context, pair[0]);
Long object = registerConstant(context, pair[1]);
@@ -3695,12 +4024,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject item = overCapacity
? swiftStringOmission(layout, word, object)
: swiftStringItem(value, layout, word, object);
+ if (unresolvedJump && !overCapacity)
+ item = swiftStringOmission(layout, word, object, "unresolved_jump");
item.addProperty("source_address", canonicalAddress(instruction.getAddress()));
JsonArray pairRegisters = new JsonArray();
pairRegisters.add(pair[0].getName());
pairRegisters.add(pair[1].getName());
item.add("registers", pairRegisters);
- (overCapacity ? omitted : observed).put(key, item);
+ (overCapacity || unresolvedJump ? omitted : observed).put(key, item);
if (overCapacity) {
budget.stop();
return true;
@@ -3736,11 +4067,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (operation.getOpcode() == PcodeOp.RETURN) returns = true;
}
// A jump is a proven tail call only when it has resolved targets and all leave the
- // body. A targetless computed jump can still dispatch inside the function.
+ // body. For unresolved targetless jumps, candidate literals remain unknown omissions.
Address[] targets = instruction.getFlows();
boolean tailCall = flow.isJump() && !flow.isConditional() && targets.length > 0 &&
Arrays.stream(targets).allMatch(target -> !function.getBody().contains(target));
- if (!flow.isCall() && !returns && !tailCall) return pairs;
+ boolean unresolvedJump = flow.isJump() && !flow.isConditional() && targets.length == 0;
+ if (!flow.isCall() && !returns && !tailCall && !unresolvedJump) return pairs;
// A String occupies two consecutive registers among x0-x7, or x0 and x1 when returned.
int lastFirstRegister = returns ? 0 : 6;
for (int index = 0; index <= lastFirstRegister; index++) {
@@ -3883,6 +4215,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
remaining = Math.max(0, (runtime.maxMemory() - runtime.totalMemory() + runtime.freeMemory()) / 2);
}
+ SwiftLiteralBudget(long remaining) {
+ if (remaining < 0) throw new IllegalArgumentException("Swift literal budget must be nonnegative");
+ this.remaining = remaining;
+ }
+
boolean takeState() {
if (remaining < BYTES_PER_RECORD) return false;
remaining -= BYTES_PER_RECORD;
@@ -3921,6 +4258,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
private static JsonObject swiftStringOmission(SwiftLiteralLayout layout, long word, long object) {
+ return swiftStringOmission(layout, word, object, "capacity");
+ }
+
+ private static JsonObject swiftStringOmission(
+ SwiftLiteralLayout layout,
+ long word,
+ long object,
+ String reason
+ ) {
JsonObject item = new JsonObject();
item.addProperty("representation", layout.representation());
if (layout.address() == null) {
@@ -3929,7 +4275,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
item.addProperty("address", canonicalAddress(layout.address()));
}
item.addProperty("byte_length", layout.byteLength());
- item.addProperty("reason", "capacity");
+ item.addProperty("reason", reason);
item.add("words", swiftStringWords(word, object));
return item;
}
@@ -4497,7 +4843,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
private record InventoryItem(Address address, String value, JsonObject facts) {}
private record AddressNameRequest(String inputAddress, Address address, String name) {}
private record FunctionEntry(Function function, InventoryItem item) {}
- private record FunctionAnalysisObservation(JsonObject dossier, List limitations) {}
+ private record FunctionAnalysisObservation(JsonObject dossier, List limitations, JsonObject relationshipCoverage) {}
+ private record CallBoundaryObservation(List limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
diff --git a/bridge/hopper_bridge.py b/bridge/hopper_bridge.py
index 92a473c061..0425cc0d91 100644
--- a/bridge/hopper_bridge.py
+++ b/bridge/hopper_bridge.py
@@ -644,7 +644,7 @@ def _analyze_function(document, params):
"end": _hex(_basic_block_end(procedure, block)),
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
})
- pseudo = _pseudocode(document, procedure) or ""
+ pseudo = _pseudocode(document, procedure)
assembly_lines = _assembly(procedure).splitlines()
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
@@ -706,7 +706,7 @@ def _analyze_function(document, params):
"signature": procedure.signatureString(),
"locals": _procedure_locals(procedure),
},
- "pseudocode": pseudo,
+ "pseudocode": pseudo or "",
"assembly": assembly_lines,
"comments": comments,
"callers": callers, "callees": callees,
@@ -718,6 +718,12 @@ def _analyze_function(document, params):
"basic_blocks": blocks,
"native_api": None,
"native_value_flow": None,
+ "facet_availability": {
+ "pseudocode": {"available": True} if pseudo is not None else _unavailable("Hopper did not supply pseudocode."),
+ **{facet: {"available": True} for facet in (
+ "assembly", "comments", "callers", "callees", "references", "strings_names", "cfg"
+ )},
+ },
"limitations": [
"Native CallReference classifications are observed where available; detailed reference flags remain unknown.",
"Hopper's public Python API does not expose equivalent external or thunk classification in this dossier.",
@@ -730,7 +736,12 @@ def _analyze_function(document, params):
def _dispatch(method, params):
"""Dispatch only the closed operation set implemented by REA's public tools."""
if method == "health":
- return {"name": "REA Hopper bridge", "version": "1.0.0", "run_id": REA_RUN_ID}
+ return {
+ "name": "REA Hopper bridge",
+ "version": "1.0.0",
+ "run_id": REA_RUN_ID,
+ "process_id": os.getpid(),
+ }
if method in ("shutdown", "shutdown_document"):
document = _session_document()
if document is None:
diff --git a/bridge/mitmproxy/capture.py b/bridge/mitmproxy/capture.py
index dcd90e745e..4123060120 100644
--- a/bridge/mitmproxy/capture.py
+++ b/bridge/mitmproxy/capture.py
@@ -41,6 +41,87 @@ class BoundedReader:
return self.handle.read(count)
+SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
+SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
+NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
+URL_IGNORABLE_CONTROLS = "\t\n\r"
+URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
+
+
+def authority_start(text, base_scheme=None):
+ """Return where an authority begins and whether it is special, or None.
+
+ Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
+ special schemes (and scheme-relative references) skip every leading "/"
+ or "\\" before the authority, so `https://\\user:pass@host` still has
+ userinfo. File URLs carry no userinfo and keep their "//" boundary.
+ """
+ scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
+ name = scheme.group(1).lower() if scheme is not None else None
+ if name is not None and name in SPECIAL_SCHEMES and name != "file":
+ # A same-scheme Location without two slash-equivalent separators is
+ # relative to the recorded request, so its @ belongs to the path.
+ following = text[scheme.end():scheme.end() + 2]
+ if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
+ return None
+ start = scheme.end()
+ else:
+ if scheme is not None:
+ if text[scheme.end():scheme.end() + 2] != "//":
+ return None
+ return scheme.end() + 2, name in SPECIAL_SCHEMES
+ # A non-special base treats backslashes as authority data and only
+ # recognizes two forward slashes as a network reference.
+ if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
+ prefix = re.match(r"^[\x00-\x20]*//", text)
+ return None if prefix is None else (prefix.end(), False)
+ prefix = re.match(NETWORK_PREFIX, text)
+ if prefix is None:
+ return None
+ start = prefix.end()
+ if base_scheme == "file":
+ return start, True
+ while start < len(text) and text[start] in "/\\":
+ start += 1
+ return start, True
+
+
+def without_userinfo(value, base_scheme=None):
+ """Remove an authority's userinfo using the WHATWG authority boundary.
+
+ Special schemes and scheme-relative references also end the authority
+ at a backslash, so later path text containing "@" is not userinfo.
+ """
+ binary = isinstance(value, bytes)
+ text = value.decode("latin-1") if binary else value
+ # WHATWG removes these controls before interpreting scheme/separators.
+ # Keep that parse view separate from the exact source spelling and bytes.
+ parsed_text = text.translate(URL_CONTROL_TRANSLATION)
+ located = authority_start(parsed_text, base_scheme)
+ if located is None:
+ return value
+ start, special = located
+ terminators = "/?#\\" if special else "/?#"
+ end = start
+ while end < len(parsed_text) and parsed_text[end] not in terminators:
+ end += 1
+ at = parsed_text.rfind("@", start, end)
+ if at < start:
+ return value
+ parsed_offset = 0
+ raw_start = 0
+ for raw_offset, char in enumerate(text):
+ if char in URL_IGNORABLE_CONTROLS:
+ continue
+ if parsed_offset == start:
+ raw_start = raw_offset
+ if parsed_offset == at:
+ safe = text[:raw_start] + text[raw_offset + 1 :]
+ return safe.encode("latin-1") if binary else safe
+ parsed_offset += 1
+ raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
+
+
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
@@ -58,16 +139,23 @@ def project_record(state, secrets):
return None
return text
- def transport_value(value, path, transport_url):
+ def transport_value(value, path, transport_url, base_scheme):
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
return value, False
binary = isinstance(value, bytes)
- pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
- replacement = rb"\1" if binary else r"\1"
separator = b"@" if binary else "@"
- safe_url = re.sub(pattern, replacement, value, flags=re.I)
- if path.endswith("/authority") and separator in safe_url:
- safe_url = safe_url.rsplit(separator, 1)[1]
+ if transport_url:
+ safe_url = without_userinfo(value, base_scheme)
+ elif path.endswith("/authority"):
+ safe_url = value.rsplit(separator, 1)[-1]
+ else:
+ # Native request.path is origin-form unless it explicitly carries
+ # a scheme (absolute-form). Leading separators are path evidence,
+ # not a header URL reference's network authority.
+ text = value.decode("latin-1") if binary else value
+ safe_url = without_userinfo(value) if re.match(
+ SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
+ ) else value
changed = safe_url != value
if changed:
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -80,7 +168,7 @@ def project_record(state, secrets):
del numbers[before[1]:]
del redactions[before[2]:]
- def visit(value, path, depth=0, credential=False, transport_url=False):
+ def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
@@ -90,7 +178,7 @@ def project_record(state, secrets):
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
return None
if isinstance(value, bytes):
- safe_bytes, credential_url = transport_value(value, path, transport_url)
+ safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
@@ -107,7 +195,7 @@ def project_record(state, secrets):
return None
return safe_text
if isinstance(value, str):
- safe_text, _ = transport_value(value, path, transport_url)
+ safe_text, _ = transport_value(value, path, transport_url, base_scheme)
return redact(safe_text, path, value)
if isinstance(value, bool) or value is None:
return value
@@ -125,10 +213,16 @@ def project_record(state, secrets):
for index, field in enumerate(value):
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
- result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
+ result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
return result
- return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
+ return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
if isinstance(value, dict):
+ if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
+ request = value.get("request")
+ scheme = request.get("scheme") if isinstance(request, dict) else None
+ if isinstance(scheme, bytes):
+ scheme = scheme.decode("latin-1")
+ base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
result = {}
for key, item in value.items():
if not isinstance(key, str):
@@ -139,7 +233,7 @@ def project_record(state, secrets):
validate_omitted(item, pointer(path, key), depth + 1)
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
continue
- result[key] = visit(item, pointer(path, key), depth + 1)
+ result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
return result
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
diff --git a/bridge/mitmproxy/test_capture.py b/bridge/mitmproxy/test_capture.py
new file mode 100644
index 0000000000..f0a1106b1b
--- /dev/null
+++ b/bridge/mitmproxy/test_capture.py
@@ -0,0 +1,174 @@
+"""Source-owned tests for the mitmproxy adapter's record projection."""
+
+import importlib.util
+import os
+import sys
+import types
+import unittest
+from unittest import mock
+
+
+def load_capture():
+ """Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
+ fake_mitmproxy = types.ModuleType("mitmproxy")
+ fake_mitmproxy.ctx = types.SimpleNamespace()
+ fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
+ fake_io = types.ModuleType("mitmproxy.io")
+ fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
+ stand_ins = {
+ "mitmproxy": fake_mitmproxy,
+ "mitmproxy.io": fake_io,
+ "mitmproxy.io.tnetstring": fake_io.tnetstring,
+ }
+ spec = importlib.util.spec_from_file_location(
+ "rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
+ )
+ module = importlib.util.module_from_spec(spec)
+ with mock.patch.dict(sys.modules, stand_ins):
+ spec.loader.exec_module(module)
+ return module
+
+
+capture = load_capture()
+
+
+def location(value, scheme=None):
+ state = {"response": {"headers": [[b"Location", value]]}}
+ if scheme is not None:
+ state["request"] = {"scheme": scheme}
+ record = capture.project_record(state, [])
+ sidecar = next(
+ field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
+ )
+ return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
+
+
+class TransportUserinfoTests(unittest.TestCase):
+ def test_ignorable_controls_cannot_hide_authority_credentials(self):
+ for control in [b"\t", b"\n", b"\r"]:
+ for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
+ with self.subTest(value=value):
+ text, sidecar, redactions = location(value, b"https")
+ self.assertNotIn("user", text)
+ self.assertNotIn("pass", text)
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(len(redactions), 1)
+
+ def test_mixed_network_separators_remove_credentials(self):
+ for prefix in [b"\\\\", b"/\\", b"\\/"]:
+ with self.subTest(prefix=prefix):
+ text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
+ self.assertEqual(text, prefix.decode("ascii") + "host/path")
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(len(redactions), 1)
+
+ def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
+ text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
+ self.assertEqual(text, "//host/x")
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(len(redactions), 1)
+ text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
+ self.assertEqual(text, "/\\invoice@customer")
+ self.assertEqual(sidecar["state"], "retained")
+ self.assertEqual(redactions, [])
+
+ def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
+ for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
+ with self.subTest(value=value):
+ record = capture.project_record({"request": {"path": value}}, [])
+ self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
+ self.assertEqual(record["binary_fields"][0]["state"], "retained")
+ self.assertEqual(record["redactions"], [])
+ record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
+ self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
+ self.assertEqual(record["binary_fields"][0]["state"], "redacted")
+
+ def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
+ for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
+ with self.subTest(value=value):
+ text, sidecar, redactions = location(value, b"https")
+ self.assertEqual(text, value.decode("ascii"))
+ self.assertEqual(sidecar["state"], "retained")
+ self.assertEqual(redactions, [])
+
+ def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
+ for value, scheme, expected in [
+ (b"https://user:pass@host/path", b"https", "https://host/path"),
+ (b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
+ (b"https:user:pass@host/path", b"http", "https:host/path"),
+ ]:
+ with self.subTest(value=value, scheme=scheme):
+ text, sidecar, redactions = location(value, scheme)
+ self.assertEqual(text, expected)
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(len(redactions), 1)
+
+ def test_backup_location_uses_its_own_request_scheme(self):
+ record = capture.project_record({
+ "request": {"scheme": b"http"},
+ "backup": {
+ "request": {"scheme": b"https"},
+ "response": {"headers": [[b"Location", b"https:invoice@customer"]]},
+ },
+ }, [])
+ self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
+ self.assertEqual(record["redactions"], [])
+
+ def test_backslash_ends_a_special_scheme_authority(self):
+ text, sidecar, redactions = location(b"https://example.test\\part@tail")
+ self.assertEqual(text, "https://example.test\\part@tail")
+ self.assertEqual(sidecar["state"], "retained")
+ self.assertIsNotNone(sidecar["content_base64"])
+ self.assertEqual(redactions, [])
+
+ def test_userinfo_before_a_backslash_is_still_removed(self):
+ text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
+ self.assertEqual(text, "https://example.test\\path")
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(
+ redactions,
+ [{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
+ )
+
+ def test_ordinary_userinfo_is_removed(self):
+ text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
+ self.assertEqual(text, "https://example.test/a@b")
+
+ def test_empty_userinfo_marker_is_removed(self):
+ self.assertEqual(
+ capture.without_userinfo("https://@host"), "https://host"
+ )
+
+ def test_special_schemes_skip_leading_slashes_before_userinfo(self):
+ for value, expected in [
+ (b"https://\\user:pass@host/path", b"https://\\host/path"),
+ (b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
+ (b"https:user:pass@host", b"https:host"),
+ (b"//\\user:pass@host", b"//\\host"),
+ ]:
+ with self.subTest(value=value):
+ self.assertEqual(capture.without_userinfo(value), expected)
+ text, sidecar, redactions = location(b"https://\\user:pass@host/path")
+ self.assertEqual(text, "https://\\host/path")
+ self.assertEqual(sidecar["state"], "redacted")
+ self.assertEqual(len(redactions), 1)
+
+ def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
+ for value in [b"///u@h", b"////u@h"]:
+ with self.subTest(value=value):
+ text, sidecar, redactions = location(value, b"file")
+ self.assertEqual(text, value.decode("ascii"))
+ self.assertEqual(sidecar["state"], "retained")
+ self.assertIsNotNone(sidecar["content_base64"])
+ self.assertEqual(redactions, [])
+
+ def test_file_urls_keep_path_text(self):
+ self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
+
+ def test_non_special_schemes_keep_backslashes_in_the_authority(self):
+ self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
+ self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts
index b4cf2e4ade..440338f987 100644
--- a/docs/.vitepress/config.ts
+++ b/docs/.vitepress/config.ts
@@ -44,6 +44,7 @@ export default defineConfig({
link: "/web-network-captures",
},
{ text: "Electron observation", link: "/electron-observation" },
+ { text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{ text: "Android applications", link: "/android-analysis" },
@@ -63,6 +64,7 @@ export default defineConfig({
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
+ { text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
diff --git a/docs/adb-device-analysis.md b/docs/adb-device-analysis.md
index 12fd77cd25..8c94df25f7 100644
--- a/docs/adb-device-analysis.md
+++ b/docs/adb-device-analysis.md
@@ -73,32 +73,32 @@ existing analysis families directly, for example
## Operations
-| Tool | Kind | What it does |
-| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
-| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
-| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
-| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
-| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
-| `list_adb_services` | read | `service list` binder services and interfaces |
-| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
-| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
-| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
-| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
-| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
-| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
-| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
-| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
-| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
-| `pull_adb_file` | read | One absolute device path pulled and digested |
-| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
-| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
-| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); refuses existing targets without explicit overwrite; verifies via the device's own sha256sum |
-| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
-| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
-| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
-| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
-| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
+| Tool | Kind | What it does |
+| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
+| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
+| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
+| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
+| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
+| `list_adb_services` | read | `service list` binder services and interfaces |
+| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
+| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
+| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
+| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
+| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
+| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
+| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
+| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
+| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
+| `pull_adb_file` | read | One absolute device path pulled and digested |
+| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
+| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
+| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
+| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
+| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
+| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
+| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
+| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
## Interpret the result
@@ -118,6 +118,16 @@ existing analysis families directly, for example
while POSIX hosts retain valid POSIX filenames such as names with spaces or
colons. A package pull checks reported names against the destination
filesystem before pulling, and refuses names that would share a local path.
+- **Package path refusals are diagnosable.** Refusals from `pull_adb_package`
+ return `unreadable_output` with `retryable: false`, keep the `pm path` output
+ in `details.captured_output`, and name the refusal in `details.reason`.
+ `unsafe_device_path` lists refused entries in
+ `details.diagnostics.rejected_paths` as `{device_path, reason}`;
+ `duplicate_local_destination` reports the shared `file_name` and both
+ `device_paths`; `local_destination_unavailable` reports the `device_path`,
+ `file_name`, and filesystem `error_code`. Diagnostics retain raw device
+ strings, while the caller-facing message escapes invisible and bidirectional
+ characters so they cannot hide text or reorder the path.
- **Splits are part of the answer.** `pm path` enumerates the base APK and
every installed split; pulls cover the whole set. System packages whose
APKs keep other names report the `unknown` role rather than guessing.
@@ -127,8 +137,10 @@ existing analysis families directly, for example
partial coverage instead of dropping rows.
- **Mutations are explicit.** `push_adb_file`, `install_adb_package`, and
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
- pushes refuse existing device files unless `overwrite` is set and verify
- the transfer with the device's own `sha256sum`; installs preserve the
+ pushes check for existing device files before transfer and refuse them
+ unless `overwrite` is set. The check is not atomic, so a different process
+ can create the path before `adb push` writes it. When available, the device's
+ `sha256sum` verifies the transfer; installs preserve the
device's `Failure [...]` reason verbatim.
- **Producer identity stays attached.** Every result embeds the producing
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
diff --git a/docs/android-analysis.md b/docs/android-analysis.md
index 50087f4648..1bad043531 100644
--- a/docs/android-analysis.md
+++ b/docs/android-analysis.md
@@ -9,6 +9,10 @@ This family is present on repository main and in npm 4.1.0. Check the
[released package boundary](installation.md#released-package-and-main) and the
connected server's tool list before selecting it.
+On Linux, JADX launches an REA-owned process and requires a procps-compatible
+`ps` on REA's `PATH`; see the shared
+[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
+
## Optional package delivery
The MCP `inspect_android_package` request keeps complete output when `detail` is
@@ -33,10 +37,40 @@ unverified.
## Supply tools
Use a **full JDK 17 or newer** (including its compiler module) and
-**jadx-headless-mcp 0.7.1**. REA runs its packaged Java metadata bridge in
-source-file mode against the unmodified engine JAR. REA does not install Java,
-an Android SDK, JADX, a device service or an emulator. Set the path in the shell
-or MCP server environment:
+[**jadx-headless-mcp 0.7.1**](https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar).
+Verify the downloaded JAR's SHA-256 against
+[REA's pinned release metadata](https://github.com/morluto/rea/blob/main/src/android/JadxRelease.ts)
+before using it. The following download and verification commands work for npm
+package consumers as well as repository checkouts; the repository-only
+`npm run fixtures:android` helper is not included in the npm package.
+
+On macOS or Linux, choose a destination and verify the pinned digest:
+
+```sh
+mkdir -p "$HOME/.local/share/rea/android" &&
+cd "$HOME/.local/share/rea/android" &&
+curl -fL -o jadx-headless-mcp-0.7.1-all.jar https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar &&
+if command -v sha256sum >/dev/null; then
+ echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | sha256sum -c -
+else
+ echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | shasum -a 256 -c -
+fi &&
+export REA_JADX_MCP_JAR="$PWD/jadx-headless-mcp-0.7.1-all.jar"
+```
+
+On Windows, run the equivalent in PowerShell:
+
+```powershell
+$jar = Join-Path $HOME '.local\share\rea\android\jadx-headless-mcp-0.7.1-all.jar'
+New-Item -ItemType Directory -Force (Split-Path $jar) | Out-Null
+Invoke-WebRequest -Uri 'https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar' -OutFile $jar
+if ((Get-FileHash -Algorithm SHA256 $jar).Hash.ToLowerInvariant() -ne '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75') { throw 'JADX JAR SHA-256 does not match REA pinned metadata' }
+$env:REA_JADX_MCP_JAR = $jar
+```
+
+REA runs its packaged Java metadata bridge in source-file mode against the
+unmodified engine JAR. REA does not install Java, an Android SDK, JADX, a device
+service or an emulator. Set the path in the shell or MCP server environment:
```sh
export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
diff --git a/docs/architecture.mermaid b/docs/architecture.mermaid
index 753c57024e..90dd41ab17 100644
--- a/docs/architecture.mermaid
+++ b/docs/architecture.mermaid
@@ -38,6 +38,10 @@ graph TD
JS_CODEC["src/javascript/sourceMaps/ pinned upstream map codec owned heap-limited Node process, complete point evidence"]
INSPECTOR["src/inspector/ passive V8 Inspector discovery and capture script and target locations"]
DOTNET["src/dotnet/ Managed static metadata provider PE/CLI metadata inventory, members, IL"]
+ OBJDUMP["src/objdump/ GNU objdump command profile"]
+ RIZIN["src/rizin/ Static command profile and persistent debugger sessions"]
+ GDB["src/gdb/ Persistent GDB/MI sessions owned process lifecycle and target state"]
+
FIRMWARE["src/firmware/ Binwalk / Unblob adapters owned processes, report normalization, private extraction"]
ANDROID["src/android/ Static APK adapter owned SDK stdio, pinned headless JADX, normalization"]
REFERENCE["src/reference/ Filesystem reference-source reader bounded traversal, validation, errors"]
@@ -50,7 +54,7 @@ graph TD
end
subgraph "Contracts Layer"
- CONTRACTS["src/contracts/ Caller-visible schemas for analysis, managed, browser, Electron, application, and session tools"]
+ CONTRACTS["src/contracts/ Caller-visible schemas for analysis, reverse engineering, managed, browser, Electron, application, and session tools"]
ERRORS["src/contracts/errorSchemas.ts Shared analysis error schemas CLI diagnostics, generated error catalog"]
end
@@ -122,6 +126,14 @@ graph TD
INSPECTOR --> DOMAIN
APP --> DOTNET
DOTNET --> DOMAIN
+ COMPOSITION --> OBJDUMP
+ OBJDUMP --> DOMAIN
+ COMPOSITION --> RIZIN
+ RIZIN --> DOMAIN
+ CLI --> GDB
+ SERVER --> GDB
+ GDB --> PROCESS
+ GDB --> DOMAIN
HOPPER --> HOPPER_BRIDGE
GHIDRA --> GHIDRA_BRIDGE
HOPPER --> PROCESS
diff --git a/docs/browser-observation.md b/docs/browser-observation.md
index 879d01520a..9479f4b3c6 100644
--- a/docs/browser-observation.md
+++ b/docs/browser-observation.md
@@ -18,7 +18,7 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
-- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata.
+- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
@@ -190,6 +190,36 @@ rea discover-webmcp-tools http://127.0.0.1:9222 TARGET_ID \
--json
```
+### WebMCP discovery prerequisites
+
+`discover_webmcp_tools` reports only what the attached browser's experimental
+`WebMCP` CDP domain exposes. When a browser rejects the unknown domain, the
+result reports `status: "unavailable"`. Whether a page can register tools at
+all depends on several conditions that are not interchangeable:
+
+- A site's own origin-trial token can enable the API for that origin
+ **provided the trial is still active, the browser version and platform
+ support it, and user settings permit it** — token presence alone is not a
+ guarantee. The trial is announced for Chrome 149 and later.
+- The `WebMCPTesting` feature — `--enable-features=WebMCPTesting` at launch, or
+ `chrome://flags/#enable-webmcp-testing` in a normal profile — enables the
+ API page-side for that browser instance, independent of per-site tokens.
+ In one pilot run, a site whose registration code was present but whose
+ served token was scoped for third-party use registered 0 tools under a
+ default launch and 4 under the flag; that the token did not enroll the
+ first-party document is consistent with this but is an inference, not a
+ measured Chrome behavior.
+
+In Chrome 154 specifically (one tested version), `WebMCP.enable` answered
+under a default launch — the debug domain was not flag-gated in that run.
+Other versions may differ, so an unattributed zero-tool result should be
+read as registration state only after domain availability is confirmed.
+
+Discovery is attach-only: the command needs a reachable loopback CDP endpoint
+and a target ID from `list_browser_targets`, and it never navigates, reloads,
+or evaluates the page. Observe the page in the state you want to measure;
+registered tools reflect the document as loaded.
+
Accessibility names/descriptions, console primitive text, JSON body shapes, WebSocket shapes, script content, and storage key names are selected independently:
```bash
diff --git a/docs/browser-scenario-contract.md b/docs/browser-scenario-contract.md
index 403865be8c..a581129b70 100644
--- a/docs/browser-scenario-contract.md
+++ b/docs/browser-scenario-contract.md
@@ -15,6 +15,12 @@ there is no separate origin allowlist or request-interception layer. Environment
settings (including service workers blocked by default), empty storage, and a
final sanitized URL capture are supplied by default. Set
`environment.service_workers` to `allow` when the application requires them.
+For attached browsers, `block` rejects captures when the selected page or its
+frames are controlled by a service worker or already have a registration,
+including after navigation. Existing caller registrations are never removed.
+Navigation can run destination code before a newly discovered worker is rejected;
+that rejection does not roll back browser effects. REA removes its owned
+future-document scripts, restores live frame methods, then disconnects.
Add secret declarations, storage seeds, and additional artifact or event
capture only when needed. Action, secret, and storage counts are not capped.
Duration, action, and navigation
@@ -249,8 +255,12 @@ Each observed request object receives a capture-scoped `transaction_id`.
Requests sharing a URL remain distinct. Redirects link to the preceding
transaction only when it was observed; a null predecessor does not establish
that no earlier hop existed. Request events require both transaction fields;
-an absent ID is malformed capture data, while a null redirect predecessor is
-an explicit unknown.
+an absent ID is malformed capture data. `redirected_from_transaction_state`
+records `root` when Playwright observed no predecessor and `unknown` when a
+predecessor exists but was outside the capture. A linked predecessor is carried
+by `redirected_from_transaction_id` with no state field. Older captures without
+the state field retain the conservative meaning that a null predecessor is
+unknown.
`scenario.network_content` records the resolved content selection. Comparing
captures with different selections reports network evidence as unknown with a
@@ -284,8 +294,10 @@ browser cleanup, with a provider-owned five-second timeout per read and
cancellation support. Failed or unexposed reads retain per-content states and make selected
event coverage incomplete. Body events can arrive during later steps or final
drain; event bounds describe receipt during the step, while the source event
-reference establishes the transaction phase. Comparison attributes content to
-its source event's step and excludes the sequence reference from normalization.
+reference establishes the transaction phase. Comparison attributes content, like
+every other event of its transaction, to the step that sent the request (the
+receipt step when that request was not observed) and excludes the sequence
+reference from normalization.
Playwright scenario capture does not supply initiator stacks; passive CDP
observation retains its own initiator evidence.
@@ -300,7 +312,22 @@ an unchanged claim; observed differences may still prove a changed result.
The comparison records its full normalization commitment and SHA-256 digest.
Built-in rules exclude step elapsed time and event sequence/index fields,
compare screenshots by content digest, and compare DOM/accessibility captures
-by normalized text. Optional caller rules are bounded exact-literal
+by normalized text. Network events are grouped by transaction without their
+capture-local transaction IDs, attributed to the step that sent the request,
+and compared as an unordered collection after caller rules apply, because
+concurrent responses arrive in any order and can land in a later step; a
+redirect retains the complete predecessor ancestry, including each hop's own
+request, response outcome, and selected network content. Each transaction's
+capture-local IDs, sequence/index fields, and content source-sequence anchor
+are removed. Caller rules normalize retained strings before hashing. The
+`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
+as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
+list of its normalized event SHA-256 digests, and `redirected_from` is its
+predecessor fingerprint or null for a root. Selected transaction fingerprints
+are sorted for the step's network comparison. Temporary numeric graph indices
+are lookup references only and do not enter the final digest. Missing or cyclic
+predecessor evidence yields unknown comparability instead of an unchanged claim.
+Other events keep receipt order. Optional caller rules are bounded exact-literal
replacements over named artifact kinds. Rules and artifact-kind lists are
sorted canonically before application, making the same policy reproducible
regardless of input order. Rules operate only on already-redacted durable
@@ -318,3 +345,7 @@ selection. Captures with different selections report event evidence as unknown
with a recovery instruction, even when no selected family emitted an event.
Captures written before this field existed omit it, so their event evidence is
also unknown.
+
+Network normalization also commits the removal of capture-local transaction IDs,
+attribution to the requesting step, and unordered transaction comparison. These
+built-in rule identities participate in the normalization digest.
diff --git a/docs/cli.md b/docs/cli.md
index c801bf006d..76dde4f4fb 100644
--- a/docs/cli.md
+++ b/docs/cli.md
@@ -263,7 +263,12 @@ tree matches that commit remains unknown. It honors the source tree's `.gitignor
and applies default exclusions for common generated and dependency paths such
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
matched pattern and whether it came from project, default, caller, or sensitive
-path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
+path policy. Links into excluded paths retain their internal target metadata;
+the exclusion explains why the target has no inventory entry, and the inventory
+remains partial. Exclusions record `entry_kind` when the importer observed it;
+older saved exclusions without that field have unknown kind. An excluded
+ancestor proves an internal descendant only when its kind is `directory`.
+Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
patterns when you want to mark selected paths as sensitive; those patterns are
reported separately as configured-secret exclusions.
@@ -287,6 +292,7 @@ Choose the guide for the target and intended interaction:
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
+- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
diff --git a/docs/client-configuration.md b/docs/client-configuration.md
index 409bc7f196..b661983c9c 100644
--- a/docs/client-configuration.md
+++ b/docs/client-configuration.md
@@ -129,6 +129,12 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
The sizes above come from REA's own `tools/list`. A live Claude Code session
against DeepSeek's API is unverified.
+For a client whose stdio reader rejects the complete catalog frame, add
+`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
+client. This omits only advertised output schemas; REA still validates complete
+structured results against its canonical contracts. See [MCP output schema
+profiles](mcp-contracts.md#output-schemas).
+
## Kimi Code
Setup registers `rea` in `~/.kimi-code/mcp.json`, or
@@ -265,6 +271,21 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
+## WorkBuddy
+
+For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
+global `~/.workbuddy/mcp.json` (a standard `mcpServers` JSON file at the user
+home root on every platform). Entries carry `command`, `args`, and `env`;
+stdio is the default transport. WorkBuddy discovers personal skills from
+`~/.workbuddy/skills` (`%userprofile%\.workbuddy\skills` on Windows), so
+setup installs its skill copy there.
+
+The global MCP path is a plain JSON file; setup preserves every other server
+and top-level key when it upserts `rea`. WorkBuddy also reads a project-level
+`/.workbuddy/mcp.json`, which setup does not touch: it registers the
+user-level file only. Loading this registration and real REA tool calls
+through WorkBuddy are verified.
+
## ZCode
For [ZCode](https://zcode.z.ai/en/docs/mcp-services), setup registers `rea` in the nested `mcp.servers` table of the
@@ -279,7 +300,11 @@ the first native server into an empty table, it copies the fallback's
`mcpServers` entries into the native table so they keep loading, and leaves
the fallback file untouched. Existing REA environment overrides are preserved
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
-being silently suppressed; repair or remove it and rerun. ZCode disables a
+being silently suppressed; repair or remove it and rerun. Uninstall removes an
+REA-owned `rea` entry from both the native table and the fallback, backing up
+each changed file, so an emptied native table cannot load a leftover fallback
+registration; other fallback servers, settings and a foreign `rea` entry are
+kept. ZCode disables a
server with `enable: false` on its entry; doctor reports such a registration
as stale, and rerunning setup removes the flag to repair it. Native configuration
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
diff --git a/docs/client-verification.md b/docs/client-verification.md
index 38d0bf3996..575cc61d24 100644
--- a/docs/client-verification.md
+++ b/docs/client-verification.md
@@ -377,6 +377,11 @@ Such a stall leaves the initial page unloaded with empty or placeholder CDP URLs
it does not establish permission to capture that page. See
[the keyring diagnosis](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686).
+On Windows, browser verifier lanes that launch an owned fixture require a
+standard, non-elevated caller token. They inspect the verifier caller before
+allocating fixture resources and refuse elevated or unobservable tokens; they do
+not change caller authority or claim to inspect the browser process token.
+
`verify:browser` also captures a source-owned noise canvas as a real PNG above
8 MiB through the CLI and stdio MCP, with complete byte/digest parity and real PNG
decoding. Its SDK client explicitly permits the larger inline JSON response;
diff --git a/docs/electron-fuses.md b/docs/electron-fuses.md
new file mode 100644
index 0000000000..206ea24bb1
--- /dev/null
+++ b/docs/electron-fuses.md
@@ -0,0 +1,82 @@
+# Electron fuse state
+
+`inspect_electron_fuses` / `inspect-electron-fuses` answers whether a packaged
+Electron app will honor `--inspect` and `--inspect-brk`, `ELECTRON_RUN_AS_NODE`,
+embedded ASAR integrity validation, and loading application code only from
+`app.asar`. It reads the fuse wire stored in the Electron binary. It does not
+launch the app, modify fuse bytes, or use the network.
+
+```bash
+rea inspect-electron-fuses /Applications/Example.app --json
+```
+
+The CLI resolves a relative path against the caller's working directory. MCP
+requires an absolute filesystem path and preserves that exact selection:
+
+```json
+{
+ "name": "inspect_electron_fuses",
+ "arguments": { "path": "/Applications/Example.app" }
+}
+```
+
+Check this before `observe_javascript_runtime` with `electron-main`. When
+`EnableNodeCliInspectArguments` is disabled or removed, Electron ignores
+Inspector flags, and a later target listing can simply show no main-process
+target.
+
+## What is read
+
+On macOS, pass the `.app` bundle or the Electron Framework binary. Bundle and
+framework selection first tries the top-level `Electron Framework` binary,
+then `Versions/Current/Electron Framework`, then
+`Versions/A/Electron Framework`. Symlinks are followed only while they stay
+inside that framework.
+On Windows and Linux, pass the main Electron executable. The wire lives in
+that executable rather than in `app.asar`.
+
+The inline Evidence result contains:
+
+- the selected path, the resolved regular file, its SHA-256, size, and container
+ format (`mach-o`, `mach-o-universal`, `pe`, or `elf`);
+- one slice entry for a thin Mach-O, PE, or ELF file, and one entry per slice of
+ a universal Mach-O, with file-absolute `sentinel_offset` values;
+- `wire_version`, `declared_length`, and ordered fuses `{ index, name, state, raw_byte }`;
+- `sentinel_count`, including zero and more than one;
+- `runtime_implications`, each marked `relationship: "derivation"` and tied to
+ the fuse index it came from.
+
+Version 1 names, in index order, are `RunAsNode`, `EnableCookieEncryption`,
+`EnableNodeOptionsEnvironmentVariable`, `EnableNodeCliInspectArguments`,
+`EnableEmbeddedAsarIntegrityValidation`, `OnlyLoadAppFromAsar`,
+`LoadBrowserProcessSpecificV8Snapshot`, `GrantFileProtocolExtraPrivileges`,
+and `WasmTrapHandlers`. An index past that table, or any index on another wire
+version, keeps `name: null` and the raw byte.
+
+Wire bytes are `0x30` disabled, `0x31` enabled, and `0x72` removed. Any other
+byte is `unknown`. A removed fuse is not enabled: Inspector flags and
+`ELECTRON_RUN_AS_NODE` are not honored, and application code is not restricted
+to `app.asar`. On macOS and Windows, a removed or disabled
+`EnableEmbeddedAsarIntegrityValidation` byte means embedded ASAR integrity is
+not enforced, and an enabled byte means it is enforced. Electron compiles that
+validation only on macOS and Windows. On a Linux ELF executable the observed
+byte is preserved and the enforcement outcome is `unknown`.
+
+## Unknown and multiple sentinels
+
+No sentinel means the fuse state is unknown. It does not mean the fuses are
+disabled. A wire that continues past the slice is reported as truncated, with
+only the fuse bytes that were present. Two or more sentinels in one slice are
+all reported. REA does not pick one.
+
+Fuse bytes do not prove that the application enables Inspector some other way,
+for example through `app.commandLine` or a remote-debugging switch in
+application code.
+
+The selected file must be a regular file, or a bundle that resolves to one.
+Symlinks are followed only inside `Electron Framework.framework`. The reader
+admits at most 1 GiB, searches with 256 KiB windows, and keeps at most 32
+sentinels per slice. A real nine-fuse, one-slice result is about 1,200 CLI
+tokens; a two-slice result is about 1,300 tokens. Exact size varies with path
+lengths and limitations. This inspection creates no temporary files and does
+not modify the selected bytes.
diff --git a/docs/electron-observation.md b/docs/electron-observation.md
index eb98a41f35..e702978d94 100644
--- a/docs/electron-observation.md
+++ b/docs/electron-observation.md
@@ -7,6 +7,10 @@ This passive runtime surface is distinct from the target-free static
workflow. Static analysis reads a local ASAR or extracted directory without
executing application code. Passive observation inspects an existing Electron
page. The results keep static inferences and runtime observations distinct.
+Packaged fuse state is a separate static read:
+[`inspect_electron_fuses`](electron-fuses.md) reports whether this binary will
+honor Inspector flags, `ELECTRON_RUN_AS_NODE`, embedded ASAR integrity, and
+`app.asar`-only loading.
Use the separate
[`reconcile_javascript_runtime`](javascript-runtime-reconciliation.md) workflow
when both Evidence sets already exist.
diff --git a/docs/firmware-analysis.md b/docs/firmware-analysis.md
index 1f4ff14d60..152819177f 100644
--- a/docs/firmware-analysis.md
+++ b/docs/firmware-analysis.md
@@ -9,6 +9,10 @@ No device, root privilege, mount, emulation or execution of extracted programs
is needed. Arbitrary firmware bytes do not pass native executable admission;
select a returned ELF separately for supported Ghidra analysis.
+On Linux, Binwalk and Unblob run as REA-owned processes and require a
+procps-compatible `ps` on REA's `PATH`; see the shared
+[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
+
## Caller-supplied tools
Provide existing tools using absolute executable paths, scoped to your shell:
@@ -103,6 +107,23 @@ build's source revision, so that field remains unknown.
engine finished its configured extraction without those reported conditions;
default skip rules and format coverage still apply. It does not establish
complete firmware reconstruction.
+- `extraction_gaps` makes those conditions recoverable: a row identifies an
+ unextracted region or a failed task/stage. A chunk retains its identity and
+ byte range, plus its original-file range when it belongs to the selected input.
+ A multi-file stage retains its reported ID and handler with unknown byte ranges;
+ a standalone task failure keeps unavailable stage identity null. Rows retain exact dependency names from
+ the producer's missing-dependency reports; an accepted report with no names
+ remains a failure with unknown dependencies; no executable identity or specific installation step is inferred. Each row's `continuation_id`
+ references one instruction in `extraction_continuations`, so many regions
+ share recovery prose without losing any region identity. Instructions use
+ already configured capabilities where one exists. Every extraction rerun
+ requires a fresh absent `output_directory`, preserving the prior evidence.
+ Nested unknown chunks use `inspect_firmware_regions` on the actual published
+ child rather than repeating the same extraction with unchanged handlers.
+ Inspection requires separately configured Binwalk (`REA_BINWALK_COMMAND`);
+ without it, the inspection step is unavailable and the region remains unexamined. A listed gap does not
+ mean the target is corrupt or that a retained region is empty; REA installs
+ nothing.
- The destination must be absent, absolute, with an existing parent. Publication
uses exclusive, symlink-resistant writes and durable digest readback. Existing
destinations are never overwritten. This is not an atomic directory rename:
@@ -167,6 +188,12 @@ records gzip offset/length and extracted file hashes. Real verification checks
CLI/MCP, full-file and selected-range extraction, unknown chunks, depth limits,
content parity and unchanged source bytes. No binary fixture is committed.
+The optional missing-extractor lane asserts dependency names and the owning
+handler in `extraction_gaps`. Those assertions must be run against the pinned
+Unblob release to establish real-provider coverage; synthetic producer fixtures
+alone do not verify Unblob's report nesting or stage identities. The gap ownership
+claims remain unverified against real Unblob until that lane runs.
+
Additional lanes are explicit, so the base lane stays small:
```sh
diff --git a/docs/ghidra-mips.md b/docs/ghidra-mips.md
index f010b4f4ca..397741a6ad 100644
--- a/docs/ghidra-mips.md
+++ b/docs/ghidra-mips.md
@@ -52,13 +52,15 @@ section names or the record being inside the initial 4 KiB header probe.
Program and section declarations must agree; duplicate, conflicting, truncated
or out-of-file records are rejected. Stripped section tables and section-only
records are supported. Extended table numbering is explicitly unsupported.
-For carved ELFs whose section table extends beyond the captured file, REA
-examines every complete captured section header and records the unavailable
-headers as a limitation. A valid program-header ABI record can still establish
-the supported declaration. Available section declarations must agree with it;
-their duplicate, conflicting or unreadable records remain errors. A missing
-program-header record does not make ABI declarations in uncaptured sections
-absent or supply a default ABI.
+GNU `SHT_GNU_ATTRIBUTES` sections are refused by numeric section type until REA
+parses and reconciles their declarations; section names and payload sizes do not
+control detection or allocation. For carved ELFs whose section table extends
+beyond the captured file, REA examines every complete captured section header
+and records the unavailable headers as a limitation. The provider refuses an
+incomplete ABI declaration inspection even when a program-header ABI record is
+valid, because an unavailable section header could carry an unsupported or
+conflicting declaration. A missing program-header record does not make ABI
+declarations in uncaptured sections absent or supply a default ABI.
The field layout follows
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
diff --git a/docs/ghidra-ppc.md b/docs/ghidra-ppc.md
new file mode 100644
index 0000000000..0341583785
--- /dev/null
+++ b/docs/ghidra-ppc.md
@@ -0,0 +1,90 @@
+# PowerPC ELF analysis through Ghidra
+
+This lane extends native ELF analysis to 32-bit big-endian PowerPC; it does
+not execute PowerPC code. It is the first milestone of
+[#1741](https://github.com/morluto/rea/issues/1741). GameCube/Wii DOL
+executables, PPC64 and other PowerPC variants are separate follow-ups.
+
+## Admission boundary
+
+ELF `EM_PPC` (20) is retained as the provider-neutral `powerpc` family. The
+target also retains class, byte order, `e_type` and raw `e_flags`. Recognition
+does not mean every provider or PowerPC variant is supported. `EM_PPC64` (21)
+is not recognized as this family.
+
+The Ghidra lane admits ELF32 `ET_EXEC`, big endian. ELF64, little-endian,
+relocatable and shared objects are refused with a specific unsupported
+constraint. `EF_PPC_EMB` declares an embedded variant and is refused; the
+remaining `e_flags` bits only declare relocation conventions and do not change
+instruction decoding.
+
+## Bound language
+
+Ghidra's ELF loader selects `PowerPC:BE:32:e500:default` for a plain `EM_PPC`
+header with `e_flags` 0. e500 is the embedded Book-E variant whose SPE unit
+replaces the classic floating-point unit, so it misinterprets ordinary
+750-family code. Nothing in the header distinguishes the two.
+
+The lane therefore imports with `PowerPC:BE:32:default` and compiler spec
+`default`, commits that choice in the analysis profile as
+`language_selection: "provider-pinned-v1"`, and checks the language Ghidra
+reports after import. A different language set through
+`REA_GHIDRA_LANGUAGE_ID` (and optionally `REA_GHIDRA_COMPILER_SPEC_ID`)
+replaces the lane default and is admitted against the installed language
+catalog as usual; for example `PowerPC:BE:32:e500` for genuine e500 code.
+
+PowerPC header facts are committed in the analysis profile so interpretation
+changes invalidate profile-bound snapshots. The support-lane identity is
+versioned independently of file identity. Evidence, lifecycle outputs and saved
+snapshots preserve the `powerpc` family. Hopper and IDA adapters do not
+implicitly gain PowerPC support. Windows Ghidra P0 remains PE x86/x86-64 only.
+
+## Use
+
+With a caller-installed Ghidra and compatible JDK on a supported Linux/macOS host:
+
+```bash
+rea analyze /absolute/path/to/ppc.elf --provider ghidra --json
+rea function /absolute/path/to/ppc.elf function_name --provider ghidra --json
+```
+
+MCP uses the existing tools, including `open_binary` with an absolute path and
+`provider_id: "ghidra"`. No PowerPC-specific tool catalog or emulator is
+introduced.
+
+## Verification
+
+The source-owned freestanding fixture is `tests/conformance/c/ppc.c`. The
+optional cross-target lane requires a caller-supplied Clang with the PowerPC
+target, LLD, GNU `readelf`, Ghidra and its compatible JDK; ordinary host-native
+checks do not acquire these tools. `REA_PPC_CLANG` selects Clang and
+`REA_PPC_READELF` selects GNU readelf (for example
+`/opt/homebrew/opt/binutils/bin/readelf` on macOS). These are test-time
+prerequisites, not REA runtime providers.
+
+```bash
+GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra npm run verify:ghidra:ppc
+```
+
+The lane compiles the fixture for a 750-family core (`-mcpu=750`) and never
+executes it. GNU readelf independently supplies header facts and symbol
+addresses, compared with the production REA target resolver before provider
+startup. Its version, bounded command output and selected tool path are
+retained in the verification report. The parser deliberately recognizes only
+this fixture's GNU output, rejecting missing, ambiguous or unexpected fields
+rather than substituting defaults.
+
+Existing public MCP operations check Ghidra's **observed** language and
+compiler specification and loaded-file hashes; an e500 import fails the lane.
+The source-owned `ppc-probe.S` supplies fixed `addi` and conditional-branch
+encodings: verification checks bytes, length, immediate and direct destination,
+independent of decompiler variable names. Global and string bytes are checked at
+independently reported symbol addresses.
+
+CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
+identity, snapshot round-trip and final process cleanup remain covered.
+Pseudocode presence is a liveness check, not proof of semantic equivalence.
+Instruction decoding and reader agreement do not establish runtime or complete
+floating-point semantics. Parser/profile tests are narrower checks, not
+substitutes for this real-provider run. Report the actual lanes run and leave
+unexecuted verification pending.
diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md
index fd23dcd073..819d55ee56 100644
--- a/docs/ghidra-provider.md
+++ b/docs/ghidra-provider.md
@@ -7,6 +7,8 @@ Executable bytes remain unchanged.
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
configuration, startup deadlines, resource controls, and cleanup recovery.
+[Linux Ghidra launches](installation.md#linux-owned-process-prerequisite) also
+require a procps-compatible `ps` on REA's `PATH` for process ownership checks.
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
snapshot parser and a separate optional annotation extension. Check the
@@ -154,12 +156,15 @@ An inferred no-return call can terminate that body before subsequent bytes.
Pseudocode, assembly, procedure info, instruction windows, callee and caller
reads, and dossiers report observed terminal call sites in Evidence limitations,
including resolved callee names, addresses, and FunctionManager flags when
-available. Instruction windows
-and dossiers also retain these warnings in their result limitations. An
-unresolved callee or unestablished terminal-flow basis remains explicit. These
-observations qualify the analysis; they do not prove that a no-return flag is
-incorrect or recover excluded code. Analysis profiles version this observation
-contract so older snapshots cannot replay unqualified results.
+available. On x86-32, the same views also report the exact Borland/Delphi-style
+FS exception-registration pattern when its handler address falls outside the
+function body. This is a static registration observation; it does not establish
+handler execution or change the body. Instruction windows and dossiers also
+retain these warnings in their result limitations. An unresolved callee or
+unestablished terminal-flow basis remains explicit. These observations qualify
+the analysis; they do not prove that a no-return flag is incorrect or recover
+excluded code. Analysis profiles version this observation contract so older
+snapshots cannot replay unqualified results.
On non-Windows hosts, the ephemeral import repair clears no-return flags on the
known returning external ABI names `malloc`, `__tls_get_addr`, `memcpy`, `memset`,
@@ -189,7 +194,7 @@ host/provider acceptance run with only the host compiler.
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
-These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
+These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
and project, endpoint, process, and runtime cleanup. The independent native
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
diff --git a/docs/go-binaries.md b/docs/go-binaries.md
index 155780984b..134a8c74d4 100644
--- a/docs/go-binaries.md
+++ b/docs/go-binaries.md
@@ -58,6 +58,12 @@ original stable file snapshot, including pointer-backed strings. It does not
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
container as one selected architecture.
+For ELF images without a named build-info section, discovery checks each
+nonempty writable, non-executable load segment in program-header order. This
+includes sectionless PIE images with metadata in a later data segment. Named
+sections retain priority, and alignment, file mapping and record validation
+remain required.
+
Historical moduleless Go binaries can store their empty module-string header in
declared zero-initialized virtual memory rather than file-backed bytes. The reader
recognizes this only when the complete optional module header lies in that memory
@@ -78,9 +84,13 @@ host permission denial, changed artifacts and cancellation retain distinct
errors. The selected file is never modified; this inspection creates no
temporary files, starts no subprocesses and accesses no network endpoint.
-Resource guards bound the selected file to 256 MiB, aggregate embedded strings
+Resource guards bound the selected file and cumulative header searches to 256 MiB
+each, aggregate embedded strings
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
-report to 8 MiB. Oversized results fail without partial success. MCP also applies
+report to 8 MiB. Identical search spans with the same virtual alignment are
+searched once; partially overlapping spans consume the cumulative search budget
+and fail with a CPU resource constraint when it is exhausted. Oversized results
+fail without partial success. MCP also applies
its [response transport budget](mcp-contracts.md), which counts both text and
structured content; a report within the metadata limit can still require export
from retained Evidence. These limits are not operating-system memory or CPU
diff --git a/docs/ida-provider.md b/docs/ida-provider.md
index 012f75bfb5..4e44605168 100644
--- a/docs/ida-provider.md
+++ b/docs/ida-provider.md
@@ -6,7 +6,9 @@ Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP
## Supported upstream versions
-Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. Upstream's `main` branch has since moved to 2.0.0, which renamed the GUI tools and dropped `idb_close`, so it serves neither profile. Installing from that branch fails at startup with the missing tool names in the error diagnostics.
+Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. The attached profile requires the legacy GUI tool names; upstream 2.0.0 renamed those tools, and REA has not established an equivalent attached-mode lifecycle and ownership contract for that catalog. It therefore fails closed with the missing tool names in the error diagnostics.
+
+The headless profile is defined by the database-supervisor API, not by an upstream version string. The upstream source snapshot [`c133c385`](https://github.com/mrexodia/ida-pro-mcp/tree/c133c3853faa111a9b00ee615c013b720d0c4acd) includes `idb_close` (introduced by [upstream PR #495](https://github.com/mrexodia/ida-pro-mcp/pull/495)) alongside `idb_open` and `idb_list`. Earlier builds may report the same `2.0.0` version while lacking that tool, so REA checks the live catalog rather than treating a version number or branch name as a compatibility guarantee.
| Mode | Upstream profile | Known working |
| -------- | ------------------- | ------------------------------------------------------ |
@@ -137,11 +139,11 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client .recovery` copy of the current bytes.
Both snapshots remain available; a failed rollback reports the current recovery
copy's path and the restore failure.
Setup also retains valid environment overrides in REA's existing server entry,
-such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
+such as `REA_MCP_INPUT_SCHEMA_PROFILE`, `REA_MCP_OUTPUT_SCHEMA_PROFILE`, and
+`REA_MCP_MAX_RESPONSE_BYTES`.
Detected provider settings replace only matching environment keys.
Setup and doctor report a malformed REA environment map or invalid runtime
setting before setup writes files. Repair the named setting in the client
configuration; setup does not silently discard the rest of the map.
+This includes malformed loader/exclusion JSON arrays and a Ghidra compiler
+specification without its required language identifier. Valid overrides keep
+their original strings, including empty Hopper arguments. DeepSeek Harness
+`!!js` environment expressions remain unevaluated and preserved; registration
+alignment does not verify the runtime values those expressions will produce.
+For layered OpenCode files, each file's values are checked before merging, while
+setting dependencies are checked against the effective merged environment.
+Setup preserves each layer's overrides instead of copying inherited settings
+into the file it updates.
After setup, REA reports which agents, analysis tools, and workflow files passed
its final checks. Restart any agent named in the completion message, then begin
@@ -129,6 +139,150 @@ Setup pins package-runner MCP registrations to the exact installed REA version,
installs the matching skill and on-demand references in the same plan, and adds
`startup_timeout_sec = 30` for Codex and Grok Build.
+## Limit REA to one project or agent profile
+
+Setup registers REA for the whole client. For Claude Code it writes
+`mcpServers.rea` in `~/.claude.json` and installs the bundled skill under
+`~/.claude/skills`, so every project that Claude Code opens can call REA. To
+offer REA's tools only where they're wanted, use one of the routes below.
+
+Both routes control which sessions discover REA's MCP server and skill. They
+leave shell commands, filesystem access and network access available. An agent
+under the same OS account can run `rea` from `PATH` or `npx` and read other
+profiles and REA output. For clean-room separation, use an implementation
+environment with filesystem and network restrictions that prevent reading or
+fetching REA, the target and analysis output, including through shared
+installations and package registries. A separate account or container must
+enforce those restrictions.
+
+For Claude Code, check existing default-profile state before choosing either
+route. Setup is additive: a previous user-level REA registration and personal
+skill remain active in other projects. Back up the default configuration and
+REA skill first, then, with `CLAUDE_CONFIG_DIR` unset, remove the REA user
+registration with `claude mcp remove --scope user rea` and remove the known
+REA-installed `~/.claude/skills/reverse-engineer-anything` directory. Preserve
+unrelated registrations and skills. `rea uninstall` applies to other clients
+as well, so it is broader than this Claude-only migration.
+
+**A separate Claude Code profile.** Claude Code keeps settings, skills and session
+history under `CLAUDE_CONFIG_DIR`. Setup follows the same
+variable: it writes the registration to `$CLAUDE_CONFIG_DIR/.claude.json` and
+the skill to `$CLAUDE_CONFIG_DIR/skills`.
+
+```bash
+CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents setup --client claude_code
+CLAUDE_CONFIG_DIR=~/.claude-rea claude
+```
+
+A new profile may need authentication. Console sign-ins without an API key are
+stored outside `CLAUDE_CONFIG_DIR`; see
+[Claude Code authentication](https://code.claude.com/docs/en/authentication#log-in-with-multiple-accounts).
+Sessions started without the variable use the default profile's registration,
+skill and history.
+
+Inline assignments apply only to that command. Keep the same prefix for every
+profile check, setup rerun, update, or removal; bare commands use the current
+environment and can otherwise inspect or recreate default-profile state:
+
+```bash
+CLAUDE_CONFIG_DIR=~/.claude-rea claude mcp list
+CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents doctor --client claude_code --skill --json
+CLAUDE_CONFIG_DIR=~/.claude-rea npx -y rea-agents@latest setup --client claude_code --dry-run
+```
+
+After reviewing an update plan, rerun that same prefixed setup command without
+`--dry-run` and approve its exact changes. Profile removal likewise needs the
+prefix on `claude mcp remove --scope user rea` and any `rea uninstall` command;
+keep the broader client scope of uninstall in mind. Restart the selected
+profile after changing its pinned registration.
+
+`claude mcp remove` removes only the registration. To remove REA from this
+profile without running the broader uninstall flow, first back up the profile's
+REA-managed skill directory, then remove
+`$CLAUDE_CONFIG_DIR/skills/reverse-engineer-anything` as well. Keep other skills
+in the profile intact.
+
+The commands above use POSIX shell assignment syntax. In native Windows
+PowerShell, select the profile for the whole setup/launch/check session and
+restore the previous selection afterward:
+
+```powershell
+$previousClaudeConfigDir = $env:CLAUDE_CONFIG_DIR
+try {
+ $env:CLAUDE_CONFIG_DIR = Join-Path $HOME ".claude-rea"
+ npx rea-agents setup --client claude_code
+ claude mcp list
+ npx rea-agents doctor --client claude_code --skill --json
+ npx -y rea-agents@latest setup --client claude_code --dry-run
+ # After reviewing the plan, repeat setup without --dry-run and approve it.
+ claude
+ # Profile removal, when intended, also belongs inside this selected scope:
+ # claude mcp remove --scope user rea
+} finally {
+ if ($null -eq $previousClaudeConfigDir) {
+ Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue
+ } else {
+ $env:CLAUDE_CONFIG_DIR = $previousClaudeConfigDir
+ }
+}
+```
+
+Use PowerShell for these Windows examples rather than pasting POSIX assignments
+into cmd.exe. Keep profile updates, readiness checks and removal inside the
+same selected environment; the `finally` block restores the previous profile
+or unsets the variable when it was absent.
+
+**One project.** Install without configuring Claude Code: use the curl
+installer's `--no-setup`, or run setup only for other clients. Then register
+REA from the project directory at Claude Code's local scope.
+
+This route uses the default Claude profile. Keep `CLAUDE_CONFIG_DIR` unset for
+the entire local lifecycle: registration, backup, listing, version replacement
+and removal. In POSIX shells run `unset CLAUDE_CONFIG_DIR`; in PowerShell run
+`Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue` before these
+commands. Record an existing selection first if you intend to restore it after
+finishing. The configuration path below is the default profile's
+`~/.claude.json`; a different active profile uses its own config root instead.
+
+Register with the variable unset:
+
+
+
+```bash
+claude mcp add --scope local rea -- npx -y rea-agents@6.4.0 mcp
+```
+
+On native Windows, use the same project directory and wrap the package-runner
+shim so the stdio client can launch it:
+
+```powershell
+claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.4.0 mcp
+```
+
+
+
+Local scope stores the entry under that directory's path in `~/.claude.json`,
+so other projects don't load it. This route doesn't install the bundled skill.
+Avoid `--scope project`, which writes `.mcp.json` into the repository where it
+can be committed or copied. `rea doctor` and `rea uninstall` read only the
+user-level entry, so they don't report or remove a local-scope registration;
+remove it with `claude mcp remove --scope local rea` from the same directory.
+`rea update` doesn't refresh it either, so it stays on the version it names.
+To move it to a newer release, first back up the project entry in
+`~/.claude.json` and record its environment and other metadata. In particular,
+preserve caller-selected overrides such as `REA_MCP_INPUT_SCHEMA_PROFILE` and
+`REA_MCP_MAX_RESPONSE_BYTES`. Run `claude mcp remove --scope local rea` from the
+same directory, then recreate it with the platform-appropriate command above
+and the new version. Reapply every recorded environment value with Claude
+Code's `mcp add --env NAME=value` options and restore any other retained entry
+metadata before restarting the project session. Removal and re-addition do not
+preserve these values automatically. Running setup for Claude Code instead
+would add a user-level entry for every project.
+
+Check the separate profile with the prefixed `claude mcp list` above and compare
+with the default profile with `CLAUDE_CONFIG_DIR` unset. For local scope, run
+`claude mcp list` from the selected project and again from another project.
+
## Check readiness for your task
Run `rea doctor` when you need diagnosis. It reads host prerequisites and agent
@@ -136,6 +290,15 @@ configuration without changing them. Without options, it audits every detected
registration, the installed skill and optional analysis engines; its overall
`healthy` value can be false while your chosen workflow works.
+Doctor's `providerInspections` are host-level engine and registration checks,
+not a readiness row for every advertised MCP adapter surface. `tools/list`
+remains the complete stable catalog. Call `binary_session` with `{}` before
+choosing an operation; it works before a target is opened, and
+`result.tool_availability` reports per-operation availability, reasons and
+client requirements for the current session state. It does not guarantee that
+a particular caller-supplied input will be valid. See
+[MCP discovery](mcp-contracts.md#identity-and-discovery).
+
Select the part you want to check:
| Task | Readiness check |
@@ -157,6 +320,21 @@ on `open_binary`. A `capability_unavailable` failure carries
`details.candidate_ids`, while `provider_unavailable` asks you to repair the
selected engine. See [provider selection](cli.md#choose-a-provider).
+### Linux owned-process prerequisite
+
+On Linux, REA's shared process supervisor requires a procps-compatible `ps`
+on REA's `PATH` for ownership checks before launching a child. Ghidra, JADX,
+firmware tools and static JavaScript/Electron analysis use these checks.
+Install your distribution's `procps` or `procps-ng` package, or put another
+procps-compatible `ps` on `PATH` before
+running those workflows. A missing or incompatible command causes a refusal
+before the supervised child starts.
+
+The public error code varies by caller: `capability_unavailable`,
+`provider_unavailable` or `execution_failure`. Some provider adapters report
+only a generic startup error. Check this prerequisite before following
+provider-specific recovery advice.
+
## Diagnose, update, and remove
`rea doctor --json` is strictly read-only. `rea update` updates only the npm or Bun global installation that owns the running CLI. Source checkouts and package-runner copies must be updated through the mechanism that owns them; a fresh package-runner invocation can use `npx rea-agents@latest` or `bunx rea-agents@latest`. `rea uninstall` removes only REA-owned agent registrations and skill files; `--purge-data` additionally removes REA cache and state paths.
@@ -288,11 +466,12 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| Qoder | `qoder` |
| Factory Droid | `droid` |
| Trae | `trae` |
+| WorkBuddy | `workbuddy` |
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
-`dsh` and `kimi` require a release containing them. The registry check above does
-not cover them.
+`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
+check above does not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
@@ -334,6 +513,20 @@ document for the same target. REA does not currently identify, focus, or reuse
that existing GUI document, and the supported launcher exposes no attach or
reuse action for it.
+REA currently declines saved Hopper database (`.hop`) targets on macOS because
+the native `open database` launch route does not run REA's bridge script. To
+analyze that program with REA, select its original executable as the target and
+open it through Hopper; this starts the supported bridge route. This does not
+recover database-only annotations or analysis changes. Saved-database analysis
+on Linux continues to use its supported demo-launch route.
+
+Hopper lease directories must be readable and owned by the current user.
+REA validates and secures an opened directory before reserving a lease. If
+opening that directory fails with a permission error, the diagnostic identifies
+the path and recovery steps while leaving its permissions unchanged. Check
+ownership and parent-directory access, then restore owner-only read, write, and
+search permissions (`0700`) on a directory you own before retrying.
+
On supported Linux distributions, approved setup verifies Hopper's official
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
REA runs the supported demo build on a private Xvfb display and selects Hopper's
@@ -346,6 +539,13 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
`sudo`. `rea doctor --provider hopper --json` reports the selected
strategy and both host and effective mount facts.
+When the package-manager step fails, setup reports the failing command and its
+captured output, so a package conflict is distinguishable from an authorization
+failure. On CachyOS, an unversioned compatibility package such as
+`zlib-ng-compat` may conflict with Hopper's versioned `zlib` dependency. Resolve
+the package conflict yourself or install Hopper's official package manually,
+then rerun `rea setup`.
+
The supported Linux demo forwards additional launches into its existing
application, even across private displays. REA therefore reserves one Linux
Hopper application per user across cooperating REA processes. A competing
@@ -370,8 +570,9 @@ sessions before removing that recovery directory and retrying.
On macOS, REA uses
`/Applications/Hopper Disassembler.app/Contents/MacOS/hopper` by default.
On Linux, it prefers executable `/opt/hopper/bin/Hopper`, then executable
-`~/.local/share/rea/hopper/bin/Hopper`; if neither exists, the former remains
-the diagnostic fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
+`~/.local/share/rea/hopper/bin/Hopper`, followed by `/opt/hopper-v4/bin/hopper`
+and `/usr/bin/hopper`; if none exists, the former remains the diagnostic
+fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
```bash
export HOPPER_LAUNCHER_PATH=/absolute/path/to/Hopper
@@ -493,7 +694,13 @@ copying license secrets into logs, or killing unrelated Hopper processes.
REA connects to an existing Ghidra installation on Linux x64/arm64, macOS x64/arm64,
or experimental Windows x64 P0.
It accepts Ghidra 12.1.x and the 64-bit full JDK declared by that installation's
-`application.java.min` and `application.java.max`. Current 12.1 releases require
+`application.java.min` and `application.java.max`. The JDK's reported platform must also
+map to the same native decompiler directory as the host: a JDK running under
+emulation (for example x86_64 under Rosetta on Apple silicon) or reporting a
+non-host `os.name` fails the doctor
+even when it is an otherwise accepted 64-bit full JDK. Select a matching JDK
+through `JAVA_HOME`; adding another decompiler in the JVM-selected directory
+does not satisfy this host-validated admission rule. Current 12.1 releases require
JDK 21 or newer and set no maximum. The bridge is verified with Ghidra 12.1.4
and JDK 21. Each installation must include the native decompiler for the host
architecture in `Ghidra/Features/Decompiler/os//` or the corresponding
@@ -505,7 +712,7 @@ reported as incompatible instead of being admitted from its directory name, and
a header that cannot be read or recognized is reported as unknown. REA does not
build or install native tools, or change Gatekeeper quarantine settings.
-The adapter exposes 25 read-only operations: thirteen inventory/name/search
+The adapter exposes 26 read-only operations: fourteen inventory/name/search
operations and twelve function-analysis operations. These cover metadata,
decompilation, assembly, resolved calls, typed references, xrefs, function
dossiers, instructions, recovered data types, measured load mappings, loaded
@@ -553,9 +760,22 @@ installation remains unavailable on Windows. Setup never installs Ghidra,
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
+When `GHIDRA_INSTALL_DIR` is unset, doctor and the Ghidra provider also search
+`PATH` for `ghidra` or `ghidraRun`, resolve the selected launcher through its
+real path, and accept it only when that location belongs to a Ghidra tree with
+the application metadata and headless launcher. An explicitly supplied
+`GHIDRA_INSTALL_DIR` always takes precedence, including when it is invalid;
+REA reports that error instead of silently selecting another installation.
+This discovery is read-only and does not execute the Ghidra launcher.
+
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
+It also checks that the reported JVM `os.arch` and `os.name` map to the same native decompiler
+directory validated for the host; unrecognized reported architectures or a
+non-host operating system are refused. LaunchSupport overrides of either
+property, including through helper-environment JVM option variables, are
+rejected at launch.
When Java is found through `PATH`, setup records its observed JDK home so GUI
MCP clients do not depend on an incidental shell path. Setup shows every exact
environment entry in its plan, writes only after approval, and never downloads,
@@ -788,7 +1008,7 @@ On a controlled Windows x64 runner, use
`npm run verify:ghidra:windows`. The verifier generates a deterministic native
PE fixture from source bytes and requires the Windows native authority before
opening the provider. The controls are implemented on main; see the
-[release boundary](#released-package-and-main). The lane verifies all 25 admitted
+[release boundary](#released-package-and-main). The lane verifies all 26 admitted
read-only operations, digest identity, and runtime cleanup. The separate
`verify:ghidra:windows:package` lane checks the packed CLI/MCP boundary with an
ordinary user. It requires the built Windows native bundle and an existing
diff --git a/docs/javascript-artifact-reconstruction.md b/docs/javascript-artifact-reconstruction.md
index 090902ff89..b166bf65eb 100644
--- a/docs/javascript-artifact-reconstruction.md
+++ b/docs/javascript-artifact-reconstruction.md
@@ -282,9 +282,14 @@ suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
and links whose callee was renamed by a minifier are not recognized.
Each recovered bundle module retains the exact factory-source digest. A complete
-bounded AST also receives a `babel-ast-v1` structural fingerprint that ignores
+bounded AST also receives a `babel-ast-v2` structural fingerprint that ignores
ordinary identifier names while retaining syntax, literals, operators, object
-keys, and member properties. If fingerprint construction reaches its structural
+keys, and member properties. Computed member identifiers retain their access
+syntax without treating a local variable's spelling as a fixed property; template
+literal contents remain part of the fingerprint. Version comparison keeps the
+algorithm in its matching key, so retained `babel-ast-v1` fingerprints cannot match
+these new fingerprints. Re-analyze both snapshots to compare their new structure.
+If fingerprint construction reaches its structural
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
prefix is never presented as a complete stable fingerprint.
@@ -619,3 +624,11 @@ Unresolved HTML references retain their declaration, source range, and resolutio
reason in the renderer observations.
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
line endings, preserving UTF-16 columns.
+Inline JavaScript in local HTML, meaning classic or module ``,
+ ``,
).join("");
page = `${annotations}`;
const fixtureUrl = `${fixtureOrigin}/capture-metadata-budget`;
diff --git a/scripts/lib/browser-module-e2e.mjs b/scripts/lib/browser-module-e2e.mjs
index 2bb656adb5..5c6a154f31 100644
--- a/scripts/lib/browser-module-e2e.mjs
+++ b/scripts/lib/browser-module-e2e.mjs
@@ -14,36 +14,42 @@ import { webModuleTraceResultSchema } from "../../dist/domain/webModuleTrace.js"
import { NativeModuleResolver } from "../../dist/browser/modules/NativeModuleResolver.js";
import { startBrowserModuleSite } from "../fixtures/browser-module-site.mjs";
import { mcpTextValue } from "./mcp-verifier-results.mjs";
+import { closeRuntimeFixtureResources } from "./browser-runtime-fixture-lifecycle.mjs";
/** Check native URL resolution against an independently loaded module graph through CLI/MCP. */
export async function verifyBrowserModules(
executable,
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
) {
- const site = await startBrowserModuleSite();
- const root = await mkdtemp(join(tmpdir(), "rea-browser-modules-"));
- const env = {
- ...process.env,
- REA_LOG_LEVEL: "silent",
- REA_BROWSER_EXECUTABLE: executable,
- HOPPER_LAUNCHER_PATH: "/rea-unconfigured-provider/hopper",
- };
- const cli = async (...args) => {
- const { stdout } = await promisify(execFile)(
- process.execPath,
- [entrypoint, ...args, "--json"],
- { env, timeout: 45000, maxBuffer: 16 * 1024 * 1024 },
- );
- return parseEvidence(JSON.parse(stdout));
- };
- const transport = new StdioClientTransport({
- command: process.execPath,
- args: [entrypoint, "mcp"],
- env,
- stderr: "pipe",
- });
- const client = new Client({ name: "module-real-e2e", version: "1" });
+ let site;
+ let root;
+ let transport;
+ let client;
+ let primaryError;
try {
+ site = await startBrowserModuleSite();
+ root = await mkdtemp(join(tmpdir(), "rea-browser-modules-"));
+ const env = {
+ ...process.env,
+ REA_LOG_LEVEL: "silent",
+ REA_BROWSER_EXECUTABLE: executable,
+ HOPPER_LAUNCHER_PATH: "/rea-unconfigured-provider/hopper",
+ };
+ const cli = async (...args) => {
+ const { stdout } = await promisify(execFile)(
+ process.execPath,
+ [entrypoint, ...args, "--json"],
+ { env, timeout: 45000, maxBuffer: 16 * 1024 * 1024 },
+ );
+ return parseEvidence(JSON.parse(stdout));
+ };
+ transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [entrypoint, "mcp"],
+ env,
+ stderr: "pipe",
+ });
+ client = new Client({ name: "module-real-e2e", version: "1" });
const oracle = await loadingOracle(executable, site);
const scenario = {
browser: { mode: "launch", executable_path: executable },
@@ -298,11 +304,17 @@ export async function verifyBrowserModules(
map_identity_verified: true,
implicit_refetches: 0,
};
+ } catch (cause) {
+ primaryError = cause;
+ throw cause;
} finally {
- await client.close();
- await transport.close();
- await site.close();
- await rm(root, { recursive: true, force: true });
+ const cleanup = [];
+ if (client !== undefined) cleanup.push(() => client.close());
+ if (transport !== undefined) cleanup.push(() => transport.close());
+ if (site !== undefined) cleanup.push(() => site.close());
+ if (root !== undefined)
+ cleanup.push(() => rm(root, { recursive: true, force: true }));
+ await closeRuntimeFixtureResources(cleanup, primaryError);
}
}
diff --git a/scripts/lib/browser-nested-source-map-e2e.mjs b/scripts/lib/browser-nested-source-map-e2e.mjs
index ff2af88632..d7bac5ee22 100644
--- a/scripts/lib/browser-nested-source-map-e2e.mjs
+++ b/scripts/lib/browser-nested-source-map-e2e.mjs
@@ -5,6 +5,7 @@ import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { TraceMap, eachMapping } from "@jridgewell/trace-mapping";
import { CdpBrowserProvider } from "../../dist/browser/CdpBrowserProvider.js";
+import { listBrowserTargetsInputSchema } from "../../dist/domain/browserObservation.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { parseEvidence } from "../../dist/domain/evidence.js";
import { traceSourceMap } from "../../dist/javascript/sourceMaps/TraceSourceMap.js";
@@ -89,10 +90,12 @@ async function verifyCase(executable, entrypoint, scenario) {
fixture = await startRuntimeFixtureBrowser(executable, site.origin);
const endpoint = `http://127.0.0.1:${fixture.port}`;
const provider = new CdpBrowserProvider();
- const targets = await provider.listTargets({
- cdp_endpoint: endpoint,
- allowed_origins: [site.origin],
- });
+ const targets = await provider.listTargets(
+ listBrowserTargetsInputSchema.parse({
+ cdp_endpoint: endpoint,
+ allowed_origins: [site.origin],
+ }),
+ );
if (!targets.ok) throw targets.error;
const target = targets.value.targets[0];
assert.ok(target, "Owned fixture target missing");
diff --git a/scripts/lib/browser-network-e2e.mjs b/scripts/lib/browser-network-e2e.mjs
index 4cf369f63a..33aad95c77 100644
--- a/scripts/lib/browser-network-e2e.mjs
+++ b/scripts/lib/browser-network-e2e.mjs
@@ -14,6 +14,7 @@ import {
} from "../fixtures/browser-network-site.mjs";
import { assertBrowserNetworkEvidence } from "./browser-network-assertions.mjs";
import { scenarioProfiles } from "./browser-scenario-verifier.mjs";
+import { closeRuntimeFixtureResources } from "./browser-runtime-fixture-lifecycle.mjs";
/** Exercise selected network content through real CLI and stdio MCP, one browser at a time. */
export async function verifyBrowserNetworkEvidence(
@@ -21,9 +22,14 @@ export async function verifyBrowserNetworkEvidence(
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
attachment,
) {
- const site = await startBrowserNetworkSite();
- const profilesBefore = await scenarioProfiles();
+ let site;
+ let profilesBefore;
+ let transport;
+ let client;
+ let primaryError;
try {
+ site = await startBrowserNetworkSite();
+ profilesBefore = await scenarioProfiles();
const scenario = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: executable },
start_url: { url: site.origin },
@@ -97,58 +103,50 @@ export async function verifyBrowserNetworkEvidence(
() => assertBrowserNetworkEvidence(swapped, site.origin),
/associated with a different same-URL request/,
);
- const transport = new StdioClientTransport({
+ transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env,
stderr: "pipe",
});
- const client = new Client({ name: "network-real-e2e", version: "1" });
+ client = new Client({ name: "network-real-e2e", version: "1" });
let mcpProof;
- try {
- await client.connect(transport);
- const result = await client.callTool(
- { name: "capture_browser_scenario", arguments: scenario },
+ await client.connect(transport);
+ const result = await client.callTool(
+ { name: "capture_browser_scenario", arguments: scenario },
+ { timeout: 60_000 },
+ );
+ assert.notEqual(result.isError, true, JSON.stringify(result));
+ const evidence = parseEvidence(result.structuredContent);
+ mcpProof = assertBrowserNetworkEvidence(
+ browserScenarioCaptureSchema.parse(evidence.normalized_result),
+ site.origin,
+ );
+ if (attachment !== undefined) {
+ const connectedScenario = browserScenarioSchema.parse({
+ ...scenario,
+ browser: { mode: "connect", ...attachment },
+ });
+ const connected = await client.callTool(
+ { name: "capture_browser_scenario", arguments: connectedScenario },
{ timeout: 60_000 },
);
- assert.notEqual(result.isError, true, JSON.stringify(result));
- const evidence = parseEvidence(result.structuredContent);
- mcpProof = assertBrowserNetworkEvidence(
- browserScenarioCaptureSchema.parse(evidence.normalized_result),
- site.origin,
- );
- if (attachment !== undefined) {
- const connectedScenario = browserScenarioSchema.parse({
- ...scenario,
- browser: { mode: "connect", ...attachment },
- });
- const connected = await client.callTool(
- { name: "capture_browser_scenario", arguments: connectedScenario },
- { timeout: 60_000 },
- );
- assert.notEqual(connected.isError, true, JSON.stringify(connected));
- const connectedEvidence = parseEvidence(connected.structuredContent);
- assert.deepEqual(
- assertBrowserNetworkEvidence(
- browserScenarioCaptureSchema.parse(
- connectedEvidence.normalized_result,
- ),
- site.origin,
- "disconnected-external",
+ assert.notEqual(connected.isError, true, JSON.stringify(connected));
+ const connectedEvidence = parseEvidence(connected.structuredContent);
+ assert.deepEqual(
+ assertBrowserNetworkEvidence(
+ browserScenarioCaptureSchema.parse(
+ connectedEvidence.normalized_result,
),
- cliProof,
- );
- assert.ok(
- (await fetch(`${attachment.cdp_endpoint}/json/version`)).ok,
- "Network capture closed the external browser",
- );
- }
- } finally {
- try {
- await client.close();
- } finally {
- await transport.close();
- }
+ site.origin,
+ "disconnected-external",
+ ),
+ cliProof,
+ );
+ assert.ok(
+ (await fetch(`${attachment.cdp_endpoint}/json/version`)).ok,
+ "Network capture closed the external browser",
+ );
}
assert.deepEqual(mcpProof, cliProof);
for (const path of [
@@ -180,7 +178,14 @@ export async function verifyBrowserNetworkEvidence(
profile_cleanup: true,
connected_network_verified: attachment !== undefined,
};
+ } catch (cause) {
+ primaryError = cause;
+ throw cause;
} finally {
- await site.close();
+ const cleanup = [];
+ if (client !== undefined) cleanup.push(() => client.close());
+ if (transport !== undefined) cleanup.push(() => transport.close());
+ if (site !== undefined) cleanup.push(() => site.close());
+ await closeRuntimeFixtureResources(cleanup, primaryError);
}
}
diff --git a/scripts/lib/browser-runtime-e2e.mjs b/scripts/lib/browser-runtime-e2e.mjs
index c4cb4f09f6..78d9d46158 100644
--- a/scripts/lib/browser-runtime-e2e.mjs
+++ b/scripts/lib/browser-runtime-e2e.mjs
@@ -8,11 +8,13 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { CdpBrowserProvider } from "../../dist/browser/CdpBrowserProvider.js";
+import { listBrowserTargetsInputSchema } from "../../dist/domain/browserObservation.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { parseEvidence } from "../../dist/domain/evidence.js";
import { webExecutionSchema } from "../../dist/domain/webExecution.js";
import { webEventListenersSchema } from "../../dist/domain/webEventListeners.js";
import { startBrowserRuntimeSite } from "../fixtures/browser-runtime-site.mjs";
+import { requireOrdinaryWindowsVerifierCaller } from "./windows-ordinary-user-verifier.mjs";
import { mcpTextValue } from "./mcp-verifier-results.mjs";
import {
startRuntimeFixtureBrowser,
@@ -36,6 +38,7 @@ export async function verifyBrowserRuntime(
executable,
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
) {
+ await requireOrdinaryWindowsVerifierCaller();
await access(executable);
const site = await startBrowserRuntimeSite();
let fixture;
@@ -45,10 +48,12 @@ export async function verifyBrowserRuntime(
try {
fixture = await startRuntimeFixtureBrowser(executable, site.origin);
const endpoint = `http://127.0.0.1:${fixture.port}`;
- const targets = await new CdpBrowserProvider().listTargets({
- cdp_endpoint: endpoint,
- allowed_origins: [site.origin],
- });
+ const targets = await new CdpBrowserProvider().listTargets(
+ listBrowserTargetsInputSchema.parse({
+ cdp_endpoint: endpoint,
+ allowed_origins: [site.origin],
+ }),
+ );
if (!targets.ok) throw targets.error;
const target = targets.value.targets[0];
assert.ok(target, "owned fixture page target missing");
diff --git a/scripts/lib/browser-runtime-fixture-lifecycle.mjs b/scripts/lib/browser-runtime-fixture-lifecycle.mjs
index 88a76e5733..78709646d4 100644
--- a/scripts/lib/browser-runtime-fixture-lifecycle.mjs
+++ b/scripts/lib/browser-runtime-fixture-lifecycle.mjs
@@ -12,9 +12,11 @@ import {
BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
waitForBrowserDevtoolsPort,
} from "../../dist/browser/BrowserProcessStartup.js";
+import { requireOrdinaryWindowsVerifierCaller } from "./windows-ordinary-user-verifier.mjs";
/** Launch the synthetic fixture browser and await CDP in its own verified process group. */
export async function startRuntimeFixtureBrowser(executable, origin) {
+ await requireOrdinaryWindowsVerifierCaller();
const profile = await mkdtemp(join(tmpdir(), "rea-real-web-runtime-"));
let owned;
try {
diff --git a/scripts/lib/browser-scenario-service-worker-setup-e2e.mjs b/scripts/lib/browser-scenario-service-worker-setup-e2e.mjs
new file mode 100644
index 0000000000..ace6fc5675
--- /dev/null
+++ b/scripts/lib/browser-scenario-service-worker-setup-e2e.mjs
@@ -0,0 +1,260 @@
+import assert from "node:assert/strict";
+import { once } from "node:events";
+import { createServer, request } from "node:http";
+import { connect } from "node:net";
+
+import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
+import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
+import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
+
+const beforeDeadline = async (promise, label, timeoutMs = 2_000) => {
+ let timer;
+ try {
+ return await Promise.race([
+ promise,
+ new Promise((_, reject) => {
+ timer = setTimeout(
+ () => reject(new Error(`${label} did not settle`)),
+ timeoutMs,
+ );
+ }),
+ ]);
+ } finally {
+ clearTimeout(timer);
+ }
+};
+
+// Real loopback proxy: Playwright's first transport is forwarded to Chrome,
+// then only the policy owner's discovery/handshake is deliberately withheld.
+const startProxy = async (endpoint, mode, ownerOnly = false) => {
+ const upstream = new URL(endpoint);
+ const sockets = new Set();
+ let versions = 0;
+ let upgrades = 0;
+ let enabled = true;
+ let entered;
+ const stalled = new Promise((resolve) => {
+ entered = resolve;
+ });
+ let closed;
+ let stalledClose;
+ const stall = (socket) => {
+ socket.resume();
+ socket.once("end", () => socket.end());
+ stalledClose = new Promise((resolve) => {
+ closed = resolve;
+ });
+ socket.once("close", () => closed());
+ entered();
+ };
+ const server = createServer((incoming, response) => {
+ const versionRequest = incoming.url.replace(/\/$/, "") === "/json/version";
+ if (versionRequest) versions += 1;
+ if (
+ enabled &&
+ mode === "http-discovery" &&
+ versions === 2 &&
+ versionRequest
+ ) {
+ stall(incoming.socket);
+ return;
+ }
+ const outgoing = request(new URL(incoming.url, upstream), (received) => {
+ const chunks = [];
+ received.on("data", (chunk) => chunks.push(chunk));
+ received.on("end", () => {
+ const content = Buffer.concat(chunks).toString("utf8");
+ response.writeHead(received.statusCode, {
+ "content-type": "application/json",
+ });
+ response.end(
+ content.replaceAll(
+ `127.0.0.1:${upstream.port}`,
+ `127.0.0.1:${server.address().port}`,
+ ),
+ );
+ });
+ });
+ outgoing.on("error", () => response.destroy());
+ response.on("close", () => outgoing.destroy());
+ outgoing.end();
+ });
+ server.on("connection", (socket) => {
+ sockets.add(socket);
+ socket.on("close", () => sockets.delete(socket));
+ });
+ server.on("upgrade", (incoming, socket, head) => {
+ upgrades += 1;
+ if (
+ enabled &&
+ mode !== "http-discovery" &&
+ upgrades === (ownerOnly ? 1 : 2)
+ ) {
+ stall(socket);
+ return;
+ }
+ const outgoing = connect(Number(upstream.port), "127.0.0.1", () => {
+ const headers = incoming.rawHeaders.flatMap((value, index, all) =>
+ index % 2 === 0 ? [`${value}: ${all[index + 1]}`] : [],
+ );
+ outgoing.write(
+ `${incoming.method} ${incoming.url} HTTP/${incoming.httpVersion}\r\n${headers.join("\r\n")}\r\n\r\n`,
+ );
+ if (head.length > 0) outgoing.write(head);
+ outgoing.pipe(socket);
+ socket.pipe(outgoing);
+ });
+ outgoing.on("error", () => socket.destroy());
+ socket.on("error", () => outgoing.destroy());
+ socket.on("close", () => outgoing.destroy());
+ });
+ server.listen(0, "127.0.0.1");
+ await once(server, "listening");
+ const origin = `http://127.0.0.1:${server.address().port}`;
+ let proxyEndpoint = origin;
+ if (mode === "browser-websocket") {
+ const version = await (await fetch(`${endpoint}/json/version`)).json();
+ proxyEndpoint = version.webSocketDebuggerUrl.replace(
+ `127.0.0.1:${upstream.port}`,
+ `127.0.0.1:${server.address().port}`,
+ );
+ }
+ return {
+ endpoint: proxyEndpoint,
+ stalled,
+ waitClosed: () => stalledClose,
+ resume: () => {
+ enabled = false;
+ },
+ close: async () => {
+ const done = once(server, "close");
+ server.close();
+ for (const socket of sockets) socket.destroy();
+ await done;
+ },
+ };
+};
+
+export const verifyScenarioPolicySetup = async (
+ endpoint,
+ targetId,
+ origin,
+ page,
+) => {
+ const result = {};
+ for (const mode of ["http-websocket", "http-discovery", "deadline"]) {
+ const proxy = await startProxy(endpoint, mode);
+ const provider = createBrowserScenarioProvider(process.env);
+ const scenario = browserScenarioSchema.parse({
+ browser: {
+ mode: "connect",
+ cdp_endpoint: proxy.endpoint,
+ target_id: targetId,
+ },
+ start_url: { url: origin },
+ environment: { service_workers: "block" },
+ actions: [
+ { step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
+ ],
+ capture: { events: [], after_each_step: ["dom"], at_end: ["dom"] },
+ });
+ const cancellation = new AbortController();
+ let failure;
+ try {
+ const pending = provider.captureScenario(scenario, {
+ signal: cancellation.signal,
+ });
+ await beforeDeadline(proxy.stalled, `${mode} setup`);
+ if (mode !== "deadline") cancellation.abort();
+ const cancelled = await beforeDeadline(
+ pending,
+ `${mode} termination`,
+ mode === "deadline" ? 12_000 : 2_000,
+ );
+ assert.equal(cancelled.ok, false);
+ assert.equal(cancelled.error.cleanupIncomplete, false);
+ if (mode === "deadline") assert.equal(cancelled.error.reason, "timeout");
+ await beforeDeadline(proxy.waitClosed(), `${mode} socket release`);
+ assert.equal(
+ await page.evaluate(() =>
+ navigator.serviceWorker.register.toString().includes("[native code]"),
+ ),
+ true,
+ );
+ proxy.resume();
+ const next = await beforeDeadline(
+ provider.captureScenario(scenario),
+ `${mode} target reacquisition`,
+ );
+ if (!next.ok) throw next.error;
+ assert.equal(next.value.browser.cleanup, "disconnected-external");
+ result[mode] = {
+ ...(mode === "deadline" ? { timed_out: true } : { cancelled: true }),
+ socket_closed: true,
+ target_reused: true,
+ };
+ } catch (cause) {
+ failure = cause;
+ throw cause;
+ } finally {
+ cancellation.abort();
+ try {
+ await proxy.close();
+ } finally {
+ try {
+ await provider.close();
+ } catch (cleanupFailure) {
+ if (failure !== undefined)
+ throw new AggregateError([failure, cleanupFailure]);
+ throw cleanupFailure;
+ }
+ }
+ }
+ }
+ const proxy = await startProxy(endpoint, "browser-websocket", true);
+ const policy = new PlaywrightScenarioServiceWorkers(
+ page,
+ proxy.endpoint,
+ targetId,
+ );
+ try {
+ const rejected = assert.rejects(
+ policy.initialize(),
+ (error) => error._tag === "AnalysisCancelledError",
+ );
+ await beforeDeadline(proxy.stalled, "browser websocket approval handshake");
+ await beforeDeadline(policy.close(), "browser websocket cleanup");
+ await rejected;
+ await beforeDeadline(
+ proxy.waitClosed(),
+ "browser websocket socket release",
+ );
+ proxy.resume();
+ const next = new PlaywrightScenarioServiceWorkers(
+ page,
+ proxy.endpoint,
+ targetId,
+ );
+ try {
+ await beforeDeadline(
+ next.initialize(),
+ "browser websocket reacquisition",
+ );
+ } finally {
+ await next.close();
+ }
+ result["browser-websocket"] = {
+ cancelled: true,
+ socket_closed: true,
+ target_reused: true,
+ direct_owner: true,
+ };
+ } finally {
+ try {
+ await policy.close();
+ } finally {
+ await proxy.close();
+ }
+ }
+ return result;
+};
diff --git a/scripts/lib/browser-scenario-service-workers-e2e.mjs b/scripts/lib/browser-scenario-service-workers-e2e.mjs
new file mode 100644
index 0000000000..3af2f4f8d6
--- /dev/null
+++ b/scripts/lib/browser-scenario-service-workers-e2e.mjs
@@ -0,0 +1,605 @@
+import assert from "node:assert/strict";
+import { once } from "node:events";
+import { createServer } from "node:http";
+import { setTimeout as delay } from "node:timers/promises";
+
+import { chromium } from "playwright-core";
+
+import { PlaywrightScenarioSession } from "../../dist/browser/PlaywrightScenarioSession.js";
+import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
+import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
+import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
+import { runScenarioCli } from "./browser-scenario-verifier.mjs";
+import { verifyScenarioPolicySetup } from "./browser-scenario-service-worker-setup-e2e.mjs";
+
+const startSite = async () => {
+ let otherOrigin;
+ const server = createServer((request, response) => {
+ const path = new URL(request.url, "http://localhost").pathname;
+ if (path === "/sw.js") {
+ response.setHeader("content-type", "application/javascript");
+ response.end(`self.addEventListener("install", () => self.skipWaiting());
+ self.addEventListener("activate", event => event.waitUntil(clients.claim()));
+ self.addEventListener("fetch", event => {
+ if (new URL(event.request.url).pathname === "/probe")
+ event.respondWith(new Response("worker"));
+ });`);
+ } else if (path === "/probe") {
+ response.end("network");
+ } else if (path === "/failed-navigation") {
+ response.destroy();
+ } else if (path === "/stalled-image.png") {
+ response.writeHead(200, { "content-type": "image/png" });
+ response.flushHeaders();
+ } else if (path === "/pending-image" || path === "/pending-child") {
+ response.setHeader("content-type", "text/html");
+ response.end(
+ '',
+ );
+ } else if (path === "/pending-frame") {
+ response.setHeader("content-type", "text/html");
+ response.end(
+ '',
+ );
+ } else {
+ response.setHeader("content-type", "text/html");
+ response.end(`